nodes equivalence of chess engines

[Uri Blass]

I am interested in the function f(n,x,y) that means:
n nodes of engine x are equivalent to y nodes of engine y(a similiar rating).

I believe that usually not only f(n)/n is getting bigger for significantly stronger engines but also that log(f(n))/log(n) is getting bigger for smart engines relative to stupid engines.

if the advantage of the better engine is only speed than f(n)=an for a>1.
I believe the advantage of some smart search trick like better order of moves in the best case can be f(n)=O(n^a) for a>1
If the advantage is some smart evaluation than I believe we can have f(n)>O(n^a) for every reasonable n that we can test.

[towforce]

I am interested in the function f(n,x,y) that means:
n nodes of engine x are equivalent to y nodes of engine y(a similiar rating).

I believe that usually not only f(n)/n is getting bigger for significantly stronger engines but also that log(f(n))/log(n) is getting bigger for smart engines relative to stupid engines.

if the advantage of the better engine is only speed than f(n)=an for a>1.
I believe the advantage of some smart search trick like better order of moves in the best case can be f(n)=O(n^a) for a>1
If the advantage is some smart evaluation than I believe we can have f(n)>O(n^a) for every reasonable n that we can test.

The Good News

I can offer you a really simple solution: divide elo by number of nodes. This will give you "elo per node"

Two simple examples

1. elo 2000, nodes 1000: 2000 / 1000 = 2 elo per node

2. elo 2000, nodes 10000: 2000 / 10000 = 0.2 elo per node


The Bad News

Engines own reporting on number of nodes is neither reliable nor consistent: engine A will report differently to engine B for the same number of nodes looked at.

Then you have engines that will pre-calculate an eval, and not recalculate the eval for nodes generated from this position: they will merely amend it. This enables them to view a much larger number of nodes (because much less work is being done in the eval), at the cost of reduced accuracy. There is evidence that Fritz was doing this 25 years ago (I am willing to share the evidence if anyone is interested. The evidence is good IMO, but falls short of proof). Having said all that:

1. It got a lot more nodes looked at in the time available
2. It seems to have been good for elo ratings
3. The net result would be higher elo overall, but a lower elo per node

[Uri Blass]

I am interested in the function f(n,x,y) that means:
n nodes of engine x are equivalent to y nodes of engine y(a similiar rating).

I believe that usually not only f(n)/n is getting bigger for significantly stronger engines but also that log(f(n))/log(n) is getting bigger for smart engines relative to stupid engines.

if the advantage of the better engine is only speed than f(n)=an for a>1.
I believe the advantage of some smart search trick like better order of moves in the best case can be f(n)=O(n^a) for a>1
If the advantage is some smart evaluation than I believe we can have f(n)>O(n^a) for every reasonable n that we can test.

The Good News

I can offer you a really simple solution: divide elo by number of nodes. This will give you "elo per node"

Two simple examples

1. elo 2000, nodes 1000: 2000 / 1000 = 2 elo per node

2. elo 2000, nodes 10000: 2000 / 10000 = 0.2 elo per node


The Bad News

Engines own reporting on number of nodes is neither reliable nor consistent: engine A will report differently to engine B for the same number of nodes looked at.

Then you have engines that will pre-calculate an eval, and not recalculate the eval for nodes generated from this position: they will merely amend it. This enables them to view a much larger number of nodes (because much less work is being done in the eval), at the cost of reduced accuracy. There is evidence that Fritz was doing this 25 years ago (I am willing to share the evidence if anyone is interested. The evidence is good IMO, but falls short of proof). Having said all that:

1. It got a lot more nodes looked at in the time available
2. It seems to have been good for elo ratings
3. The net result would be higher elo overall, but a lower elo per node

The solution you propose does not solve the problem because it does not tell me node equivalence of different engines.
elo has a limit so elo divided by nodes is going to 0 when nodes increase.

I will give examples from my elo list:

Movei and Stockfish.
Movei00_8_438 16384 nodes : 1779.9
Stockfish 16.1 512 nodes : 1709.4
Movei00_8_438 8192 nodes : 1642.6
It seems 512 nodes of stockfish is equivalent to something between 8192 and 16384 nodes of movei(did not make enough games to know but I can guess
It seems f(512)=11500
Movei00_8_438 262144 nodes : 2123.8
Stockfish 16.1 1024nodes : 2042.8
Movei00_8_438 131072 nodes : 2034.1
I will make an estimate f(1024)=135000

Stockfish 16.1 2048nodes : 2295.3
Movei00_8_438 1048576 nodes : 2270.0

Did not test more nodes with movei to give it a rating but I will make an estimate
f(2048)=1350000

It seems multiplying the nodes of movei by 2 is equivalent to multiplying the nodes of movei by more than 10.

Let look at a different engine stash that is the strongest engine that I tested at more than 1M nodes per move

Stash v35.0 1048576 nodes : 2821.8
Stockfish 16.1 8192nodes : 2778.8
Stash v35.0 524288 nodes : 2701.8
I guess we have something like g(8192)=900000

Stash v35.0 262144 nodes : 2637.9
Stockfish 16.1 4096nodes : 2562.3
Stash v35.0 131072 nodes : 2543.5
I guess we have somethin like g(4096)=150000

Stash v35.0 65536 nodes : 2354.3
Stockfish 16.1 2048nodes : 2295.3
Stash v35.0 32768 nodes : 2229.5

I guess we have something like g(2048)=50000

Stash v35.0 16384 nodes : 2067.8
Stockfish 16.1 1024nodes : 2042.8
Stash v35.0 8192 nodes : 1794.3
I guess g(1024)=15000
Stash v35.0 8192 nodes : 1794.3
Stockfish 16.1 512 nodes : 1709.4
Stash v35.0 4096 nodes : 1546.7

I guess something like g(512)=7000

My rating is not accurate and I did not calculate estimate but simply throw from my head something that seem reasonable.
Of course a real research should include a more games and testing with more nodes setting.

[Uri Blass]

For what Fritz was doing or other old engines or my own engine from more than 10 years ago they did something really stupid relative to stockfish because probably if you start with the same playing strength at fixed nodes and multiply the number of nodes by 2 you probably need to multiply the number of nodes of Fritz or Hiarcs by 10.

I do not believe in fast and slow searchers.
There are only smart engines like stockfish and relatively stupid engines like Fritz or Hiarcs or my own engine.

I even do not call them smart searchers or stupid searcher because I guess a big part of the advantage by multiplying the nodes by 2 is not the search but the evaluation function.

[smatovic]

Idk for sure, but maybe you are looking for the EBF of the specific engine, if you know EBF you can compare nodes searched?

https://www.chessprogramming.org/Branch ... ing_Factor
https://www.chessprogramming.org/Branching_Factor

--
Srdja

[Uri Blass]

Idk for sure, but maybe you are looking for the EBF of the specific engine, if you know EBF you can compare nodes searched?

https://www.chessprogramming.org/Branch ... ing_Factor
https://www.chessprogramming.org/Branching_Factor

--
Srdja

EBF is meaningless because one ply of different engines with different pruning and extensions mean different things and I am interested not in chess depth but in playing strength that you get for example from doubling the time that is similiar to doubling the number of nodes for most engines and I prefer doubling the number of nodes because I can get deterministic engines by that way.

Edit:playing strength is not elo because I mean playing strength that you get when you start from the same level with different engines.
2700 rating of stockfish16.1 earn clearly more from doubling the time relative 2700 rating of stockfish11.