You'll be surprised just how strong SF is, even at low node counts.
This is especially true with the latest version of SF. I think SF has gained incredible strength at low nodes thanks to tuning against Lc0!
I've tested numerous SF versions on 1+0.01s on my i5-4460, i5-3470 and an i3-6100U and current SFDev definitely isn't stronger than SF7, 8, 9 and 10 at ultra low TC.
Current development version has received numerous patches that are either elo neutral or elo negative on very low TC but elo positive on longer TC.
That's why there's probably a regression.
These are the 20000 test games I did on my i3-6100U laptop:
TC= 1sec + 0.01sec.
Settings are all default.
I think comparing move generator to search speed is much more ideal.
For exampke my engine generates 20 mnps and search 4 mnps. So the slowdown here is 5x. That is due to the fact Sapeli has so simple evaluation + search. It doesn't slow down.
However Stockfish:
They have removed NPS but to me it seems to take about 1 second. So 119 mnps. Compared to Sapeli's 20 mnps! That 6x faster.
bob wrote: ↑Thu Jan 16, 2020 7:21 pm
How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.
1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.
As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.
I believe 360K nodes per move is enough for stockfish to beat GM's at 180 seconds per move.
Stockfish is basically optimized for bullet games of 60+0.6 with one core.
It get more than 360K nodes per move in these conditions but still part of the testing is done with 10+0.1 time control that is not a lot faster than 360K nodes per move.
You should test this yourself. You might be surprised. It will be tactically very weak with such shallow searches.
Another type of test would be stockfish on an old pentium pro 200. Might hit 50K nodes per second. 6 secs per move for stockfish, 180 seconds per move for human? I know how computers performed back then. They were becoming pretty problematic against GM players at very fast games. But give the GMs 3 minutes to defeat a 300K node (total search tree) program would not offer him much of a challenge. Note that the program could NOT "ponder" which would skew the total nodes searched unacceptably. Would be interesting to see how deeply stockfish could search given a computational budget of 300K nodes total.
For your 10 + .1 time control, assume first time control = 40 moves, so that is 14 seconds used over 40 moves, or a third of a second per move. Crafty, on my MacBook hits around 10M nodes per second. That would equate to about 3M nodes per move, if pondering is not allowed. That is still 10x the 300K. I could always be wrong here, but my experience says 300K total nodes per search is not enough to play at GM level when a GM has three full minutes to use per move. Larry K might dive in here with his thoughts. Just doesn't seem like "enough tree" to me based on what we saw back when those sized trees were quite common.
10M nodes per second is not correct for stockfish when it does not use more than one thread.
I can see the number of nodes per second in the page of the testing
I see 1.41M nodes per second(it is not constant and can be slightly more than it or slightly less than it).
with 1/3 second per move you get initially 470K nodes per move that is more than 360K nodes per move but not by a big difference and not in the first 40 moves you have to use only 1/10 second per move not to lose on time so you get less than 360K nodes per move.
It lost 40-50 elo at 2s+0.02 ; lost ~4 elo at 10s+0.1 @ 1.6mnps ; gained ~1 elo at 20s+0.2 ; gained ~5 elo at 60s+0.6 and around ~12 elo at 180s+1.8.
I think that it is better if stockfish comes with some versions that are optimized for different time control including 2s+0.02 so maclane does not complain that stockfish play weak at low number of nodes per move because he will be able to use also a version that does not use the counterproductive changes at 2+0.02
bob wrote: ↑Thu Jan 16, 2020 7:21 pm
How far it is above Crafty really doesn't affect my comments. 2K nodes per second is not going to beat a GM. Might not beat a master, since most of the speculative pruning stuff really needs exceptional search depth to bypass the holes it causes.
1 second per move vs 180 seconds per move? Maybe. But certainly not 360K nodes total vs 180 seconds for a GM.
As far as the other numbers you quoted, I don't think any of 'em are even close to reality. Stockfish at 1K nodes per second is going to get roasted by DB. Completely roasted. That is only a factor of 100,000 in terms of nodes per second. Would be pretty easy to test. Take the online Cray Blitz source, and let it run for 180 * 200M nodes total and limit stockfish to 360K total. Cray Blitz was fairly similar to DB and others of that time frame, primarily using recursive null-move but nothing else regarding forward pruning, except maybe in the last few plies before q-search. Would be an interesting thing to try, but it would certainly take forever for a single game. I'm pretty sure nobody would want to scale that down to 2M nodes per second vs 10 nodes per second to keep the time ratio the same. I can certainly predict how badly that will end.
I would be willing to play stockfish at 100 nodes per second myself, and I have not actively played chess in 30 years (for tournaments anyway).
18000 nodes total is NOT going to play like a GM, in any shape for form.
With SF_dev at 5 kn/s I have no doubts SF_dev is no less than 2700 FIDE Elo level at tournament time control. Bob, software has advanced much. In some maybe 600-700 computer Elo-ish points since 1990s. Meaning some 400 FIDE Elo-ish points against humans.
There are many ways to reach this conclusion. One of the many is that Rebel 9 in 1997 was already some 2600 FIDE Elo level TPR on P200 in a good number of games against humans, having some 60 kn/s IIRC.
There was an analysis in the programming forum a while back that showed software advance in strength is exponential, just like the hardware advance.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.