Engine Test Suites: ECO Approach

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, chrisw, Rebel

Frank Quisinsky
Posts: 6837
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Engine Test Suites: ECO Approach

Post by Frank Quisinsky »

Hi,

an other way is to use the PGN Random Arena option. Tested with more as 40.000 eng-eng games. The database is available on Arena webpage.

Created with GM games

Arena using one of the games per random for an eng-eng match. You have different options.

Example:
Use PGN-Random database up to move 8, 10, ... you can set the move number.

I worked a long time on this database and killed a lot of positions where engines had problems (to fast remis or win / lost after few moves). The database is optimize by hand.

I think, this one is the best method for creating a rating list. After 20.000 ATL-4 games I found 8 games only, Remis up to move 16. A very good result I think :-)

Easy to use:
Arena writes in PGN files the game number of the database and the player names as comment in PGN file. For analyses you can see directly, from which games are the first moves. Helps a lot if you later analyse games.

BTW:
If you compare a rating list created with positions, like Nunn or PGN-Random other databases and compare it with the books, like Harry Schnapp or Sedat Canbaz you will find no differences.

ELO will be +-5 the same.

Interesting is also the Strong book from ChessBase (included on the ChessBase PowerBook DVD from ChessBase).

I think, its more interesting to play with tge book of Sedat, ChessBase or PGN-Random option because you will never see each time the same opening positions.

In my opinion the best opening books from ChessBase are:

1. Shredder 12

2. Nimzo 8, perhaps the best one.

3. Perfect 2009 (Sedat Canbaz)
4. Strong 2010 (Fritz PowerBook).

For Shredder 12 GUI:

1. Shredder 12 opening book
2. Perfect 12.2 8 moves (Sedat Canbaz)

For Arena GUI

1. PGN-Random option
Unfortunately, other GUIs don't have such a strong eng-eng option.

Perhaps the comments will help a little bit.

Best
Frank

Of course:
You can also create your own PGN-Random database.
Togga

Re: Engine Test Suites: ECO Approach

Post by Togga »

why dont you go to eco E99 ?
but only to E97 ?