NICE, 5 years after

[Rebel]

NICE 1.3 is the successor of the 5 year old NICE 1.2 as documented on the old REBEL website.

With NICE you can easily and by fast computing create reasonable valid rating lists based on a large volume of EPD positions which are analyzed by Stockfish 17 at long time control with MultiPV=4 and the 4 best moves of each position are calculated a bonus, for the best move 10 points and for the remaining 3 moves 0-9 points based on the score difference with the best move.

....

link

[flok]

linux binary is missing

[Jouni]

78.830 positions. This is a serious testsuite!

[peter]

With NICE you can easily and by fast computing create reasonable valid rating lists based on a large volume of EPD positions which are analyzed by Stockfish 17 at long time control with MultiPV=4 and the 4 best moves of each position are calculated a bonus, for the best move 10 points and for the remaining 3 moves 0-9 points based on the score difference with the best move.

Thanks, Ed.
Just downloaded and started looking into the .epd- files.
Here's nr. 2 of SF17-1.epd:

2r2bk1/7p/3pR3/2pP2p1/8/pP4P1/P7/2K1B3 w - - c0 "Bd2=10, g4=9, Kd2=8, Kc2=8";

And here some SF dev.- output with 30 threads (16x3.5GHz CPU) and MultiPV=4 to this one position:

[d]2r2bk1/7p/3pR3/2pP2p1/8/pP4P1/P7/2K1B3 w - -

Code: Select all

Engine: Stockfish 241222 (8192 MB)
von the Stockfish developers (see AUTHORS f

64     3:24   0.00     1.g4 Tc7 2.Te4 Tf7 3.Ta4 h5 4.gxh5 Kh7 
                       5.Txa3 Kh6 6.Ta8 Kxh5 7.Kc2 Kg4 8.a4 Kf3 
                       9.Kd3 Lg7 10.Td8 Le5 11.a5 g4 12.b4 cxb4 
                       13.Lxb4 g3 14.Lxd6 (1.612.748.049) 7883 
                       TB:6.416.868 

64     3:24   0.00     1.Kc2 Tc7 2.Kd3 h5 3.Lc3 h4 4.gxh4 gxh4 
                       5.Tg6+ Lg7 6.Txd6 h3 7.Td8+ Kh7 8.d6 Tc6 
                       9.Lxg7 Kxg7 10.Td7+ Kg6 11.Td8 (1.612.748.049) 7883 
                       TB:6.416.868 

64     3:24   0.00     1.Kd2 Tc7 2.Kd3 h5 3.Lc3 h4 4.gxh4 gxh4 
                       5.Tg6+ Lg7 6.Txd6 h3 7.Td8+ Kh7 8.d6 Tc6 
                       9.Lxg7 Kxg7 10.Td7+ Kg6 11.Td8 (1.612.748.049) 7883 
                       TB:6.416.868 

64     3:24   0.00     1.Ld2 g4 2.Kc2 h5 3.Kd3 Kf7 4.Lg5 Tb8 
                       5.Tf6+ Kg7 6.Te6 Kf7 (1.612.748.049) 7883 
                       TB:6.416.868

4 best ranked moves all get 0.00 eval, so how are the rewarding- points to be justified?

Next example, pos. nr.4:

r1bq1rk1/pp2bppp/4pn2/1N6/1PP5/P1Rp2P1/1B1P1PBP/3QN1K1 b - - c0 "Bd7=10, a6=6, Rb8=5, Nd7=2";

[d]r1bq1rk1/pp2bppp/4pn2/1N6/1PP5/P1Rp2P1/1B1P1PBP/3QN1K1 b - -

Code: Select all

Engine: Stockfish 241222 (8192 MB)
von the Stockfish developers (see AUTHORS f

40     3:18   +0.53    1...Tb8 2.Txd3 Ld7 3.Le5 Tc8 4.Lxf6 Lxf6 
                       5.Sd6 Dc7 6.Sxb7 h6 7.Db3 Tb8 8.Sa5 Tfd8 
                       9.Le4 Le7 10.Dc3 f5 11.Lg2 Lf6 
                       12.Dc2 Kh8 13.Te3 e5 14.Te2 a6 (3.268.489.795) 16477 

40     3:18   +0.56    1...Ld7 2.Txd3 Tc8 3.Lxf6 Lxf6 4.Sd6 Dc7 
                       5.Sxb7 h6 6.Db3 Tb8 7.Sa5 Tfd8 8.Le4 Le7 
                       9.Dc3 f5 10.Lg2 Lf6 11.Dc2 Kh8 
                       12.Te3 e5 13.Te2 h5 14.d4 exd4 (3.268.489.795) 16477 

40     3:18   +0.72    1...a6 2.Txd3 Ld7 3.Sd6 Lxd6 4.Txd6 Se8 
                       5.Td4 Dc7 6.Sd3 Lc6 7.Lxc6 bxc6 
                       8.Lc3 Sd6 9.Dg4 Sf5 10.Tf4 f6 11.Sc5 Tae8 
                       12.Sxa6 Dc8 13.Sc5 e5 14.Txf5 g6 (3.268.489.795) 16477 

40     3:18   +0.82    1...Sd7 2.Txd3 a6 3.Sd6 Lxd6 4.Txd6 a5 
                       5.c5 axb4 6.axb4 Ta6 7.Td3 De7 8.Db3 Sf6 
                       9.b5 Ta5 10.Db4 Ta2 11.Le5 Te8 
                       12.Dc4 Ta5 13.Lc3 Ta8 14.Te3 Dd8 (3.268.489.795) 16477 

Move order isn't the one in .epd given and rewarded.

Nr. 5:
4R3/1pr3bk/p3Q1p1/7p/6rP/6P1/1P6/1K6 w - - c0 "Qe1=10, Qg8+=8, Re7=8, Qd6=8";

[d]4R3/1pr3bk/p3Q1p1/7p/6rP/6P1/1P6/1K6 w - -

Code: Select all

Engine: Stockfish 241222 (8192 MB)
von the Stockfish developers (see AUTHORS f

77     4:38   0.00     1.Dg8+ Kh6 2.Db3 b5 3.Te6 a5 4.De3+ Kh7 
                       5.Dd3 a4 6.Ta6 Te7 7.Ka2 b4 8.Txa4 Tge4 
                       9.Ta5 b3+ 10.Dxb3 Te2 11.Ta3 Lxb2 
                       12.Dxb2 Txb2+ 13.Kxb2 Te2+ 14.Kc1 (3.716.957.431) 13364 
                       TB:23.792.431 

77     4:38   0.00     1.De1 b5 2.Te4 Txe4 3.Dxe4 Tc4 4.De1 Kh6 
                       5.b3 Tc3 6.g4 Txb3+ 7.Ka2 Tb2+ 8.Ka3 hxg4 
                       9.De4 b4+ 10.Ka4 Ta2+ 11.Kxb4 Tb2+ 
                       12.Ka3 Tb5 13.Dxg4 Ta5+ 14.Kb3 (3.716.957.431) 13364 
                       TB:23.792.431 

77     4:38   0.00     1.Dd6 Tf7 2.Te7 Txe7 3.Dxe7 a5 4.b3 Txg3 
                       5.Dxb7 Th3 6.Dd5 Txh4 7.Dxa5 Td4 
                       8.Kc2 h4 9.Dg5 Tb4 10.De3 Tg4 11.Kd1 Tg3 
                       12.Df4 Txb3 13.Ke2 Tb2+ 14.Kf3 (3.716.957.431) 13364 
                       TB:23.792.431 

76     4:38   0.00     1.Te7 Txe7 2.Dxe7 a5 3.b3 Txg3 
                       4.Dxb7 Th3 5.Dd5 Txh4 6.Dxa5 Td4 
                       7.Kc2 h4 8.Dg5 Tb4 9.De3 Tg4 10.Kd1 Tg3 
                       11.Df4 Txb3 12.Ke2 Tb2+ 13.Kf3 h3 
                       14.Dh4+ (3.716.957.431) 13364 
                       TB:23.792.431 

Move order isn't the one given and rewarded in .epd and all output- moves show same 0.00- eval again.
Stopped here so far, regards

[Rebel]

With NICE you can easily and by fast computing create reasonable valid rating lists based on a large volume of EPD positions which are analyzed by Stockfish 17 at long time control with MultiPV=4 and the 4 best moves of each position are calculated a bonus, for the best move 10 points and for the remaining 3 moves 0-9 points based on the score difference with the best move.

Thanks, Ed.

You are most welcome Peter.

Just downloaded and started looking into the .epd- files.
Here's nr. 2 of SF17-1.epd:

2r2bk1/7p/3pR3/2pP2p1/8/pP4P1/P7/2K1B3 w - - c0 "Bd2=10, g4=9, Kd2=8, Kc2=8";

And here some SF dev.- output with 30 threads (16x3.5GHz CPU) and MultiPV=4 to this one position:

[d]2r2bk1/7p/3pR3/2pP2p1/8/pP4P1/P7/2K1B3 w - -

Code: Select all

Engine: Stockfish 241222 (8192 MB)
von the Stockfish developers (see AUTHORS f

64     3:24   0.00     1.g4 Tc7 2.Te4 Tf7 3.Ta4 h5 4.gxh5 Kh7 
                       5.Txa3 Kh6 6.Ta8 Kxh5 7.Kc2 Kg4 8.a4 Kf3 
                       9.Kd3 Lg7 10.Td8 Le5 11.a5 g4 12.b4 cxb4 
                       13.Lxb4 g3 14.Lxd6 (1.612.748.049) 7883 
                       TB:6.416.868 

64     3:24   0.00     1.Kc2 Tc7 2.Kd3 h5 3.Lc3 h4 4.gxh4 gxh4 
                       5.Tg6+ Lg7 6.Txd6 h3 7.Td8+ Kh7 8.d6 Tc6 
                       9.Lxg7 Kxg7 10.Td7+ Kg6 11.Td8 (1.612.748.049) 7883 
                       TB:6.416.868 

First of all these positions are just game fragments and do not contain hard nuts to crack for top engines, the whole idea of NICE is to list engines by strength in a reasonable valid way and most important very fast. Hence analyzing position at 3 minutes (as you do above) makes no sense, to analyze the 79.xxx positions at 3 minutes would means only 20 per hour, meaning 79.000 / 20 = 3950 hours = 164 days, almost half a year. Instead the positions were analyzed at 10 seconds per move and using the tool it is advised to use time controls below 10 seconds per move, 100ms, 250ms, 500ms, 1 second, 2.5 second max. Sorry, it's not about solving hard positions this time.

[peter]

Sorry, it's not about solving hard positions this time.

I know the principles of solving positions without game changing forced best moves to be found but several near to each other candidate moves to be discriminated to each other quite will too. I use MEA with STC and VSTC a lot, but even more the solutions and the rewarding points should be of reproducible move order and differences between the moves to be chosen by the engine (or the human player as well). Suites like the one of the link to the MEA- download of 2112.epd I gave here

viewtopic.php?p=974741#p974741

are to me examples of such kind. If a certain SF- version with 10" per position and MultiPV=4 (costing time to depth and time to solution often for unforced positions even more than sinlge primary does) gives quite a different move order and eval but even the same SF version on bigger hardware- time does, the results you get by comparing engines of bigger differences to each other aren't anything else but a SIMEX- test to the certain single SF version you used to evaluate the positions of the suite with, and even the similarity is to be judged for the same small (but again not much smaller at all neither) hardware- TC only (and to see it strictly the same numbers of multiPV- lines also), regards

[chessica]

Sorry, it's not about solving hard positions this time.

I know the principles of solving positions without game changing

But the question is whether it is enough...

According to the Piffpaffpuff method you are of course right, but what is the difference between
the other variants in the game. King against king is also a draw, also wrong bishop in the pawn ending... So there are different and it should accordingly be reflected in different evaluation points.