ECO Classification of EPD records

[Dann Corbit]

Is there any utility that will give me the correct ECO classification for a collection of EPD records?

[noctiferus]

If I understand correctly, it should be a byproduct of Swami question on sts-x importances.
At the moment, the regression method has been applied to about thirty engines, with prediicted cegt elo error +- 50, more or less, but there is some instability. I'm waiting for other engines results, and a more well founded regression analysis .

Of course, you should run your own regression on your epd set.

[michiguel]

Is there any utility that will give me the correct ECO classification for a collection of EPD records?

I wonder what happens if you transform it to pgn, load it with scid, and give the opening classification. I have not tried.

Miguel

[noctiferus]

Sorry, I'm too focused on evaluating ELO performance by STS.
Beg your pardon for this misreading ... (I grabbed a cow by the balls ... )

[swami]

If I understand correctly, it should be a byproduct of Swami question on sts-x importances.
At the moment, the regression method has been applied to about thirty engines, with prediicted cegt elo error +- 50, more or less, but there is some instability. I'm waiting for other engines results, and a more well founded regression analysis .

Of course, you should run your own regression on your epd set.

Hi Enrico,

I have searched the ratings list and added these engines in the list of to be tested. I'd use 8 STS suites and append the results to the excel file. In future, I'll test them again for the partial credit scores once when the new version STSStat is released which I hope would come with mathematical calculations of Total/Average scores from all the test suites.

Filtered out engines that have established ratings in CEGT/CCRL:

Code: Select all

Frenzee
ETChess
Chiron 0.8.7
Greko
Arion 1.7
Little Thought 1.04
Gosu 0.16
Leila 0.53h
Homer 2.01
Crazy Bishop 0052
Queen 3.09
Chezzz 1.0.3
Bruja 1.9
Phalanx XXII
Amateur 2.8.2
Popochin 3.2
Waxman 2009
KnightX 1.92
EXchess 5.01
Thors Hammer 2.28
Neurosis 2.5
Averno 0.81
Comet B68
Feuerstein 0.4.6.1
Dragon 4.6
SOS
Chronos
Bright
Thinker 
ChessMaster
Twisted Logic
Spike
StockFish
Fruit
Toga

Test datas of these engines would give better predictions of elo contributions of all the test suites.

Will run the test tonight, hopefully will be ready by monday.

[noctiferus]

Hi, Swami

Fine. May I suggest to specify which versions are you testing, in order to, if the case, countercheck the correspondence to ratings?

BTW - Which is the test time you are using: CEGT 40\4 for all? and for CCRL?
THX.
E.

[swami]

Hi, Swami

Fine. May I suggest to specify which versions are you testing, in order to, if the case, countercheck the correspondence to ratings?

BTW - Which is the test time you are using: CEGT 40\4 for all? and for CCRL?
THX.
E.

Sure I'll specify in excel file which engine versions have been tested.

I used CEGT and CCRL 40 moves in 4 minutes list.

[noctiferus]

I asked it because at least in one case (leaving apart Bison), my results on Cerebro were definitely anomalous, performing on your test 100 elos more than CEGT. May be bad time management in games, or poor model, due to a short Elo range.

Anyway, when we'll see it , looking at a more spreaded elo range testsets, we should get a more stable model, I hope, and outlier detection could also help authors. May be I'm too optimistic..

[jwes]

If I understand correctly, it should be a byproduct of Swami question on sts-x importances.
At the moment, the regression method has been applied to about thirty engines, with prediicted cegt elo error +- 50, more or less, but there is some instability. I'm waiting for other engines results, and a more well founded regression analysis .

Of course, you should run your own regression on your epd set.

Hi Enrico,

I have searched the ratings list and added these engines in the list of to be tested. I'd use 8 STS suites and append the results to the excel file. In future, I'll test them again for the partial credit scores once when the new version STSStat is released which I hope would come with mathematical calculations of Total/Average scores from all the test suites.

Filtered out engines that have established ratings in CEGT/CCRL:

Code: Select all

Frenzee
ETChess
Chiron 0.8.7
Greko
Arion 1.7
Little Thought 1.04
Gosu 0.16
Leila 0.53h
Homer 2.01
Crazy Bishop 0052
Queen 3.09
Chezzz 1.0.3
Bruja 1.9
Phalanx XXII
Amateur 2.8.2
Popochin 3.2
Waxman 2009
KnightX 1.92
EXchess 5.01
Thors Hammer 2.28
Neurosis 2.5
Averno 0.81
Comet B68
Feuerstein 0.4.6.1
Dragon 4.6
SOS
Chronos
Bright
Thinker 
ChessMaster
Twisted Logic
Spike
StockFish
Fruit
Toga

Test datas of these engines would give better predictions of elo contributions of all the test suites.

Will run the test tonight, hopefully will be ready by monday.

I have been doing something similar, and have trouble with epd analysis in Arena with these engines

Code: Select all

Amateur 2.8.2
Chronos
Crazy Bishop 0052
Gosu 0.16

What time control are you running at?

[swami]

If I understand correctly, it should be a byproduct of Swami question on sts-x importances.
At the moment, the regression method has been applied to about thirty engines, with prediicted cegt elo error +- 50, more or less, but there is some instability. I'm waiting for other engines results, and a more well founded regression analysis .

Of course, you should run your own regression on your epd set.

Hi Enrico,

I have searched the ratings list and added these engines in the list of to be tested. I'd use 8 STS suites and append the results to the excel file. In future, I'll test them again for the partial credit scores once when the new version STSStat is released which I hope would come with mathematical calculations of Total/Average scores from all the test suites.

Filtered out engines that have established ratings in CEGT/CCRL:

Code: Select all

Frenzee
ETChess
Chiron 0.8.7
Greko
Arion 1.7
Little Thought 1.04
Gosu 0.16
Leila 0.53h
Homer 2.01
Crazy Bishop 0052
Queen 3.09
Chezzz 1.0.3
Bruja 1.9
Phalanx XXII
Amateur 2.8.2
Popochin 3.2
Waxman 2009
KnightX 1.92
EXchess 5.01
Thors Hammer 2.28
Neurosis 2.5
Averno 0.81
Comet B68
Feuerstein 0.4.6.1
Dragon 4.6
SOS
Chronos
Bright
Thinker 
ChessMaster
Twisted Logic
Spike
StockFish
Fruit
Toga

Test datas of these engines would give better predictions of elo contributions of all the test suites.

Will run the test tonight, hopefully will be ready by monday.

I have been doing something similar, and have trouble with epd analysis in Arena with these engines

Code: Select all

Amateur 2.8.2
Chronos
Crazy Bishop 0052
Gosu 0.16

What time control are you running at?

Yes, I just installed them and only some of them actually support analysis mode in Arena. A few of them don't.

These engines do work in Arena EPD Testing:

Code: Select all

Arion
Averno
Chezz
Comet
Exchess
Feuerstein
Homer
KnightX
Leila
Little Thought
TCB
Thorshammer
SOS
Bright
Thinker 
ChessMaster
Twisted Logic
Spike
StockFish
Fruit
Toga