The Speedy Rating List

[Rebel]

Given that a lot of engines have exactly 41 errors it would be interesting to know which positions (or even 1 of them) these occur in.

4Rk2/2R5/2p3p1/3p4/3P3r/P7/1P3P1p/7K b - - bm Kxe8; ce -32000; acd 0;
k5rb/3r4/1ppP4/p1p1p1p1/P1P1Pp1p/1P1R4/3R1PPN/3q1K2 w - - bm Rxd1; ce -32000; acd 0;
8/3P2k1/5p2/6pp/4p3/1b2P2P/6r1/2BK4 w - - bm Ke1; ce -32000; acd 0;
3r2k1/6p1/4p3/1p1nB1Qp/1PpP3P/R1P2P2/1r4qK/8 w - - bm Qxg2; ce -32000; acd 0;

'What they have in common, it's called the chopper move.

[Ras]

'What they have in common, it's called the chopper move.

How can an engine have trouble selecting the one and only legal move in a position?

[Terje]

'What they have in common, it's called the chopper move.

How can an engine have trouble selecting the one and only legal move in a position?

I'm guessing they select it and play it without giving an info string. Also not spending their time thinking, wasting that turns time which they could use to fill TT. Playing only-moves instantly is an error in 'go infinite' and a bad choice in 'go movetime'.

Confirmed to be the case in rofChade (works correctly in 'go infinite').

[Rebel]

Added 13 engines to the 4000ms list

Winter 0.8
ProDeo 2.2
Benjamin
CT800 1.40
Minic 2.32
Fire 7.1
Weiss 0.10
Andscacs 0.95
Arasan 22
Defenchess 2.2
Fruit 2.1
Shredder 13
Amoeba 3.1

And a new leader on the scaling list.

http://rebel13.nl/download/speedy-rating-list.html

[lkaufman]

Added 13 engines to the 4000ms list

Winter 0.8
ProDeo 2.2
Benjamin
CT800 1.40
Minic 2.32
Fire 7.1
Weiss 0.10
Andscacs 0.95
Arasan 22
Defenchess 2.2
Fruit 2.1
Shredder 13
Amoeba 3.1

And a new leader on the scaling list.

http://rebel13.nl/download/speedy-rating-list.html

Comparing to the CEGT blitz list (I picked CEGT because it uses ORDO, which agrees with normal elo calculations for match results), the difference between Stockfish 11 and Fruite 2.1 (I picked it as it's near the bottom of your list, easy to find on cegt, and a very well known reference point) is 990 elo on CEGT, 532 elo on your list. So doubling your ratings and subtracting a constant would not be way off, but multiplying by 1.86 with a suitable subtraction would be ideal. Of course a proper analysis of all engines on both lists will give a somewhat different ratio, but probably not too far from this 1.86 value.

[Rebel]

Yes, I will give it attention for the next update.

[Rebel]

@Larry

On a second thought -

Code: Select all

CEGT 40/4       SF11 3486
CEGT 40/20      SF11 3450
CEGT 40/120     SF11 3409

CCRL 40/2       SF11 3534
CCRL 40/15      SF11 3433

FGRL 60s+0.6    SF11 3497
FGRL 10m+6.0    SF11 3426
FGRL 60m+15.0   SF11 3391

On all 3 rating lists the longer time control the more the elo of an engine drops, with SRL it's exactly the other way around.

Instinctively more natural, one ply deeper it is not, I understand

I think I will keep my elo formula.

BTW, the NICE tool is developed for engine tuning, that it also can produce a reasonable reliable rating list was an unexpected bonus.

[cucumber]

@Larry

On a second thought -

Code: Select all

CEGT 40/4       SF11 3486
CEGT 40/20      SF11 3450
CEGT 40/120     SF11 3409

CCRL 40/2       SF11 3534
CCRL 40/15      SF11 3433

FGRL 60s+0.6    SF11 3497
FGRL 10m+6.0    SF11 3426
FGRL 60m+15.0   SF11 3391

On all 3 rating lists the longer time control the more the elo of an engine drops, with SRL it's exactly the other way around.

Instinctively more natural, one ply deeper it is not, I understand

I think I will keep my elo formula.

BTW, the NICE tool is developed for engine tuning, that it also can produce a reasonable reliable rating list was an unexpected bonus.

I had no idea that you made the tools behind this public or that they were meant for tuning!

I’ve spent all week working on engine tuning using EPDs! I even made a thread asking for help getting quiescent positions. I wish I knew about your tool and datasets beforehand. NICE is a really cool project and really well executed. Thanks for making it public.

[Patrice Duhamel]

I'm curious to see Cheese score, it's possible to add it in the list ? or to download LC1.epd ?

[Rebel]

I'm curious to see Cheese score, it's possible to add it in the list ? or to download LC1.epd ?

If Cheese is still developed I will include it.

The lc1.epd is part of NICE.

http://rebel13.nl/misc/nice.html