NICE predictions for Ethereal 12.25, SlowChess 2.2, Igel 2.50 and Wasp 4.00

[Rebel]

NICE predicts +60 elo for Wasp 4.0

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
15  Wasp 4.00        299925  11:45:50.1  20727  40000  2999  400000  74.98%   1000   128    1     0
23  Wasp 3.75        293908  11:44:35.3  20203  40000  2939  400000  73.48%   1000   128    1     0

NICE predicts +34 elo for Ethereal 12.25

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                              Max            Time   Hash          
   Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
5  Ethereal 12.25   312493  11:30:24.1  21959  40000  3124  400000  78.12%   1000   128    1     0
8  Ethereal 12      309088  11:30:20.9  21840  40000  3090  400000  77.27%   1000   128    1     0

NICE predicts +35 elo for SlowChess 2.2

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
 4  SlowChess 2.2    313139  10:30:56.4  21860  40000  3131  400000  78.28%   1000   128    1     0
 7  SlowChess 2.1    309590  10:31:56.2  21595  40000  3096  400000  77.40%   1000   128    1     0

NICE predicts +14 elo for Igel 2.5

Code: Select all

    EPD  : epd\lc1.epd
    Time : 1000ms
                                                               Max            Time   Hash          
    Engine           Points  Used Time   Found   Pos    Elo   Score   Score    ms     Mb  Cpu  Errors
25  igel 2.50        294457  11:29:26.8  20369  40000  2944  400000  73.61%   1000   128    1     0
28  igel 2.40        293056  11:28:38.5  20261  40000  2930  400000  73.26%   1000   128    1    41

Provisional SRL - http://rebel13.nl/provisional.html

The hunt season for Houdini (aka Komofish) is open.

Congrats in advance

[jonkr]

It's interesting that the elo increase prediction is about half what I was seeing in test matches. But considering the already high position of SlowChess in the list looks more like a case of older versions scoring higher than match-play performance. My guess is maybe this type of testing is more sensitive to eval changes than search changes?

[Rebel]

It's interesting that the elo increase prediction is about half what I was seeing in test matches. But considering the already high position of SlowChess in the list looks more like a case of older versions scoring higher than match-play performance. My guess is maybe this type of testing is more sensitive to eval changes than search changes?

I am trying to find my way in this unexplored area but looking at the 250ms | 500ms | 1000ms | 4000ms overview I am inclined to believe search matters.

http://rebel13.nl/download/srl-all.html

Notice how well the elo ratings nicely increase.

A way to look at this type of testing, 100,000 positions at 250ms takes about 25 minutes (I am using 20 cores), 100,000 positions reflect 100,000 / 50 = 2000 games. 50 stands for the average of moves that matter in games and probably 50 is too high already. 200,000 positions within one hour representing 4000 games.

[voffka]

In my self tests, Igel 2.5.0 is around 70 elo stronger in 10+0.1 and around 40+ elo stronger at 60+0.6, so I am curios how this will hold in rating lists as well as with NICE framework predictions.

[jonkr]

Looking more I do notice the elo range is compressed a bit from what I get in blitz guantlets so that's part of it and to be expected, but it does seem possible the evaluation changes could matter more. The upgrade from SlowChess 1.9 to 2.0 was very much evaluation compared to the later updates and that is the highest/closest match in the NICE list to my estimates.

[Alayan]

Ethereal 12.25 is mostly search improvements over 12.00, but the jump in NICE 1.0s is bigger than our estimated elo gain.