237 Mn/s for Stockfish on an 2xEPYC 7742

[Ovyron]

He said that he doubt that any human in the world has any chance of getting a 50% perf against SF at 50Knps. That's not an angry statement, that's the obvious truth.

Thanks Alayan for the support. For the record I stop doing things that anger me, so I never post on Talkchess angrily, it's often people with low patience that perceive me as angry (there's a distorted copy of me in their head, and that me is angry), but I actually enjoy these discussions or I wouldn't come here

[Rom77]

All my tests show that SF at 50K/sec - might be somewhere at the GM level - whether it is a Super GM or a more ordinary GM, who knows. I would be interested in hearing from Larry Kaufman on this. The Revelation is a device that reportedly plays at GM level and it see around 100K/nps ( roughly - I do not own one) . Perhaps only a GM knows for sure.

With Threadripper 3970X, you can run the Stockfish test against 2005 programs directly. For example, in cutechess-cli, with a handicap in time. Stockfish vs. Fruit 2.1, in 1 game/thread mode. Command line example:

cutechess-cli.exe ^
-tournament gauntlet -rounds 128 -concurrency 64 ^
-openings file=Book/2moves_v2_small.pgn format=pgn order=sequential start=1 ^
-draw movenumber=300 movecount=5 score=3 -resign movecount=5 score=800 -recover -repeat -debug -pgnout Result/Stock_vs_Fruit_2.1.pgn ^
-engine name="Stockfish 11" cmd=Engine/stockfish_11_x64_modern.exe option.Threads=1 option.Hash=8 option."Move Overhead"=1 option."Minimum Thinking Time"=0 option."Slow Mover"=65 option.Contempt=0 tc=0/0:08+0.02 ^
-engine name="Fruit 2.1" cmd=Engine/fruit_21.exe option.Hash=512 tc=0/136:32+20.48 ^
-each proto=uci restart=on timemargin=50 >>log_Stock_vs_Fruit.txt

pause

Time 1:1024
____________________________
Sorry for using google translate

[Vinvin]

Surpassing Deep Blue's 200 Million nps.

But 22 years later...

But now, Stockfish at 1 Mn/s is stronger than Deep Blue at 200 Mn/s.

Stockfish at 1 kn/s is stronger than Deep Blue.

Is it? I’m not sure that’s right

Stockfish should be GM level even at 100 n/s classical time control.

Probably 0.5 knps at classical time control, if my old inferrence works. Super GM level at some 5 knps.

I often say that Stockfish reach 2600-2700 Elo with 1 second by move on an average home machine (around 2 kn/s) or a fast phone against human on a long game (40 moves in 2h). This view set SF speed around 10 kn/s.

I do not believe this is correct. At 5K nodes per second, you get to search 5K nodes and you are done. Not much of a search. I have the ability in Crafty to search a specific number of nodes (approximately). For a simple test, I took the newer was positions (201 total) and ran them against standard Crafty at 1 second per move on fairly good hardware (60M nodes per second). It correctly solved all 201 positions. I then re-ran the same test, except telling crafty to stop after searching 5K nodes per second (5K nodes total here) it only solved 114 out of the 201 positions. That hardly sounds like a GM to me. If you stretch it to 60 seconds, normal Crafty still gets all 201. Crafty searching 300K nodes (60 seconds x 5K nodes per second) gets 171 right.

That 5K nodes per second is a REAL restriction. Many were doing 5K nodes per second in the 70's and 80's. And far beyond. Without a GM being produced.

I am certain that the 2700 Elo at 2K nodes per second is a wild exaggeration of reality. Maybe 500K nodes per second, possible. Certainly not 2K.

I'm clarifying my assertion (because there was a typo and it was not clear) :

Stockfish with 1 second of search on an average computer (2 Mn/sec) is as strong as a GM (2500 to 2600 Elo) playing a serious game (average 3 minutes per move).
Averaging : 2 millions nodes for one move in 3 minutes means an average speed around 10 Kn/sec.

Bob, don't forget Stockfish is 450 Elo above Crafty.

In the same thread other figures was given :

- Stockfish at 1 kn/s is stronger than Deep Blue.

- Stockfish should be GM level even at 100 n/s classical time control.

- Probably 0.5 knps at classical time control, if my old inferrence works. Super GM level at some 5 knps.

[bob]

How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.

1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.

As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.

I would be willing to play stockfish at 100 nodes per second myself, and I have not actively played chess in 30 years (for tournaments anyway).

18000 nodes total is NOT going to play like a GM, in any shape for form.

[Uri Blass]

How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.

1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.

As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.

I believe 360K nodes per move is enough for stockfish to beat GM's at 180 seconds per move.

Stockfish is basically optimized for bullet games of 60+0.6 with one core.
It get more than 360K nodes per move in these conditions but still part of the testing is done with 10+0.1 time control that is not a lot faster than 360K nodes per move.

[Raphexon]

How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.

1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.

As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.

I would be willing to play stockfish at 100 nodes per second myself, and I have not actively played chess in 30 years (for tournaments anyway).

18000 nodes total is NOT going to play like a GM, in any shape for form.

Here's SF with nodes limited trashing past engines.

http://talkchess.com/forum3/viewtopic.php?f=6&t=72485

Still think the test was a bit flawed because he didn't even use a small increment but it just shows how hard SF beats down older engines even with massive time odds.

And if you use Cutechess or Arena you can easily limit SF to 18k nodes per move while you time your moves on your own.
SF8 is stronger than current SFDev at such low TC though.

You'll be surprised just how strong SF is, even at low node counts.

[Vinvin]

How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.

1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.

As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.

I would be willing to play stockfish at 100 nodes per second myself, and I have not actively played chess in 30 years (for tournaments anyway).

18000 nodes total is NOT going to play like a GM, in any shape for form.

Here's SF with nodes limited trashing past engines.

http://talkchess.com/forum3/viewtopic.php?f=6&t=72485
...

Yes, good memory !
Summary after 363 games played :

Stockfish (20 kn/s) vs Fritz in Bahrain(3600 kn/s)
1 Stockfish 271119 64 BMI2 : 2969 74 72 90 71.7 % 2807 14.4 %

Stockfish (10 kn /s) vs Junior 6 (5000 kn/s)
1 Stockfish 271119 64 BMI2 : 2916 61 59 150 77.3 % 2702 14.7 %

third test is Stockfish (20 kn/s) vs Shredder 9
1 Stockfish 101219 64 BMI2 : 3108 66 63 123 80.9 % 2858 20.3 %

Conclusion : 3 tests againts Fritz, Junior, Shredder the average elo of Stockfish (20kn/s) is 3035.

Ratings of opponents :
Fritz in Bahrain vs World Chess Champion Kramnik (2807 elo) match has ended with a 4-4 score at 2002 : https://www.chessprogramming.org/Kramni ... Fritz_2002
Deep Junior 6 perf 2702 at Dortmund 2000 : https://theweekinchess.com/html/twic297.html#2
Shredder 9 at 2005 Mercosur Cup and at 2005 Vicente Lopez, perf = 2783

[bob]

How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.

1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.

As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.

I believe 360K nodes per move is enough for stockfish to beat GM's at 180 seconds per move.

Stockfish is basically optimized for bullet games of 60+0.6 with one core.
It get more than 360K nodes per move in these conditions but still part of the testing is done with 10+0.1 time control that is not a lot faster than 360K nodes per move.

You should test this yourself. You might be surprised. It will be tactically very weak with such shallow searches.

Another type of test would be stockfish on an old pentium pro 200. Might hit 50K nodes per second. 6 secs per move for stockfish, 180 seconds per move for human? I know how computers performed back then. They were becoming pretty problematic against GM players at very fast games. But give the GMs 3 minutes to defeat a 300K node (total search tree) program would not offer him much of a challenge. Note that the program could NOT "ponder" which would skew the total nodes searched unacceptably. Would be interesting to see how deeply stockfish could search given a computational budget of 300K nodes total.

For your 10 + .1 time control, assume first time control = 40 moves, so that is 14 seconds used over 40 moves, or a third of a second per move. Crafty, on my MacBook hits around 10M nodes per second. That would equate to about 3M nodes per move, if pondering is not allowed. That is still 10x the 300K. I could always be wrong here, but my experience says 300K total nodes per search is not enough to play at GM level when a GM has three full minutes to use per move. Larry K might dive in here with his thoughts. Just doesn't seem like "enough tree" to me based on what we saw back when those sized trees were quite common.

[M ANSARI]

You'll be surprised just how strong SF is, even at low node counts.

This is especially true with the latest version of SF. I think SF has gained incredible strength at low nodes thanks to tuning against Lc0!

[Alayan]

You should test this yourself. You might be surprised. It will be tactically very weak with such shallow searches.

Another type of test would be stockfish on an old pentium pro 200. Might hit 50K nodes per second. 6 secs per move for stockfish, 180 seconds per move for human? I know how computers performed back then. They were becoming pretty problematic against GM players at very fast games. But give the GMs 3 minutes to defeat a 300K node (total search tree) program would not offer him much of a challenge. Note that the program could NOT "ponder" which would skew the total nodes searched unacceptably. Would be interesting to see how deeply stockfish could search given a computational budget of 300K nodes total.
....
Just doesn't seem like "enough tree" to me based on what we saw back when those sized trees were quite common.

You make the incredibly... weird ... assumption that Stockfish would be just as weak as engines of the 90s with a node count in the hundreds of thousands. Considering that engines have gained over 1000+ elo since then and that proper statistical testing of patches and efficient tuning were basically unknown at the time, you should know better.

Sure, if you lower the node count enough, like maclane likes to do to 1K nodes, Stockfish becomes unremarkable as its advanced heuristics and refined values don't get a chance to matter. But 100k nodes, while not enough to fully allow SF to unleash its potential, is already more than enough to utterly crush any old engine with millions of nodes per move.

This is especially true with the latest version of SF. I think SF has gained incredible strength at low nodes thanks to tuning against Lc0!

Stockfish has not been tuned in the least against Lc0.

The gains at low node counts are also much lower than at higher node counts (probably even regressive at some very low node counts).