KomodoDragon 3.0 x64 4CPU (MCTS)

[Werner]

1 Stockfish 15.0NNUE x64 1CPU +131 +44/=48/-8 68.00% 68.0/100 3611
2 KDragon 3.0 x64 4CPU (MCTS) -131 +8/=48/-44 32.00% 32.0/100 3480 unbalanced openings uho

1 Stockfish 15.0NNUE x64 1CPU +28 +8/=92/-0 54.00% 54.0/100 3611
2 KDragon 3.0 x64 4CPU (MCTS) -28 +0/=92/-8 46.00% 46.0/100 3583 balanced openings +100 ??

KDragon 3.0 x64 4CPU (MCTS) - LCZero 0.29RC0 782606 52.5 - 47.5 +6/=93/-1 52.50%
KDragon 3.0 x64 4CPU (MCTS) - Stockfish 15.0NNUE x64 1CPU 78.0 - 122.0 +8/=140/-52 39.00%
KDragon 3.0 x64 4CPU (MCTS) - LCZero 0.28.0 CUDNN (610141) 48.0 - 52.0 +2/=92/-6 48.00%
KDragon 3.0 x64 4CPU (MCTS) - FatFritz 2.1NNUE x64 1CPU 49.5 - 50.5 +2/=95/-3 49.50%

results soon in our list

[lkaufman]

1 Stockfish 15.0NNUE x64 1CPU +131 +44/=48/-8 68.00% 68.0/100 3611
2 KDragon 3.0 x64 4CPU (MCTS) -131 +8/=48/-44 32.00% 32.0/100 3480 unbalanced openings uho

1 Stockfish 15.0NNUE x64 1CPU +28 +8/=92/-0 54.00% 54.0/100 3611
2 KDragon 3.0 x64 4CPU (MCTS) -28 +0/=92/-8 46.00% 46.0/100 3583 balanced openings +100 ??

KDragon 3.0 x64 4CPU (MCTS) - LCZero 0.29RC0 782606 52.5 - 47.5 +6/=93/-1 52.50%
KDragon 3.0 x64 4CPU (MCTS) - Stockfish 15.0NNUE x64 1CPU 78.0 - 122.0 +8/=140/-52 39.00%
KDragon 3.0 x64 4CPU (MCTS) - LCZero 0.28.0 CUDNN (610141) 48.0 - 52.0 +2/=92/-6 48.00%
KDragon 3.0 x64 4CPU (MCTS) - FatFritz 2.1NNUE x64 1CPU 49.5 - 50.5 +2/=95/-3 49.50%

results soon in our list

Do you include both balanced and unbalanced openings in your rating list, or just the balanced ones? It is normal for unbalanced openings to produce triple the rating difference between engines compared to balanced openings; the more than quadruple elo gap is a bit higher than that, but within sample error of it. Combining balanced and unbalanced openings in the same rating list would make the list highly subjective; the tester could inflate or deflate an engine's rating significantly by the choice of book. I think that the only solution may be at some point to just start new lists with the unbalanced books; perhaps we're not quite at that point yet, but soon it will be unavoidable I think. With balanced books at Rapid TC and 4 cpus it may be difficult to gain even another 50 elo, but with the unbalanced openings the sky is the limit, even another thousand elo may be possible.

[Werner]

This was only a test with UHO_2022_8mvs_+110_+119 as you wrote: " about double that with unbalanced openings" and at least less draws worked.
Generally I think, an engine should be tested with all sorts of openings: balanced, unbalanced, short, long, gambit, closed, open, all ECO.....

[lkaufman]

This was only a test with UHO_2022_8mvs_+110_+119 as you wrote: " about double that with unbalanced openings" and at least less draws worked.
Generally I think, an engine should be tested with all sorts of openings: balanced, unbalanced, short, long, gambit, closed, open, all ECO.....

In principle I agree with you, but mixing balanced and unbalanced openings only works if all testers have to follow the same rules, which is not the case with the testing groups. If each tester can choose balanced or unbalanced, the resultant ratings will depend heavily on tester biases.