Battle of the Forgotten

[Carlos777]

Hi guys,

This is a tournament that started with few engines under Arena just to find a suitable opponent for me to play with. Then, I got more interested in testing low ELO engines. It has been fun so far, looking for information about these engines that few people seem to be interested in, trying to find some that are not longer developed, making them work and looking them play. So, I decided to run a RRx2 tournament with a initial field of 40 engines.

The conditions to participate are:

1. Engines should have an approx. ELO of 1700 or lower.
2. The engine must support incremental time management (1'+1".)
3. Engines should not be buggy (they should be able to play under the rules of standard chess, meaning not making illegal moves and/or crashing.)
4. Engines should not lose frequently on time.
5. Engines should accept to play with ponder off
6. Clones are not allowed (ex. KasparovX, Replicant, etc.)
7. Engines must work under Arena 2.01

After the initial cycle is finished, 15 engines will be replaced by other new 15 and a new cycle will start. Games between remaining engines will not be replayed.
I will remove these 15 engines based on this:

1. Buggy engines (crashes, illegal moves and losses on time.)
2. If there are 2 or more engines by the same author, only the one with less points will remain.
3. The rest of them will be the ones that got more points.

For example: Imagine there were 2 buggy engines, 1 eliminated by having more points than other (being both of the same author,) and the remaining 13 would be the 13 first places.

The conditions of the tournament:

- Pentium D 2.8GHz
- Windows XP 32 bits
- Approx. 64-128 Mb hash
- Ponder OFF
- Time control: 1'+1"
- Own book if available, if not HS-20moves book limited to 10 moves.
- Engines can use both cores
- Round Robin x2

This is a crosstable of the current cycle:

Code: Select all

    Engine             Score   Ja KM Si Si Ro Cu Yo My Da Ro Oz RB Tr Ed Nu Re Sj Na Al Ch Ad Mi Xa Br Cl Ge Ch AL Ar Fi Yo Ge Da Fr Jo Sa Ch Ne Ac Po    S-B
01: Jabba 1.0          46.0/53 ·· 10 0= 11 00 11 11 11 11 11 11 11 11 11 11    1  =1 1  =  1     11 0  1  1  1  1  =     11 11 1  1  1  11 1     1      826.25
02: KMTChess 1.1       42.5/50 01 ·· 0= 11 11 11 00 11 10 11 11 10 11 11 11    =  11 1     1  1  11 1           =  1     11 11    1  1  11 1     1  1   765.00
03: Simplex 0.0        42.0/49 1= 1= ·· 01 == 11 01 01 =1 11 11 11 11 11 11 =  0  11          1  11          1     1     11 11    1  1  11 1  1  1  1   764.50
04: Sissa 1.00         41.5/53 00 00 10 ·· =1 11 1= =0 11 11 10 11 11 11 11 =  1  11 1  0     0  11 1  1  1  =  1  1     11 11    1  1  11 1        1   686.50
05: Roce 0.0390        38.5/46 11 00 == =0 ·· 10 1= 10 11 11 11 11 1= 11 11    1  11          1  11             1        11 11       1  11 1  1  1  1   686.50
06: Cuckoo Chess 1.00  36.5/50 00 00 00 00 01 ·· 11 =1 11 10 01 11 11 11 11    =  11          1  11       1     1  =  1  11 11 0  1  1  11 1     1  1   547.75
07: Yoda 1.30          35.0/49 00 11 10 0= 0= 00 ·· 11 =0 =1 =1 11 11 11 10 =  0  11 1           11 1  0  1           1  11 11    1  1  11    1  1      584.75
08: Myrddin 0.81a2     34.0/50 00 00 10 =1 01 =0 00 ·· 11 11 =1 =0 11 11 11    0  11 0        0  11       0  1  1  1     11 11 1  1  1  11       1  1   536.00
09: Dabbaba 2.46       31.5/53 00 01 =0 00 00 00 =1 00 ·· 1= 11 10 11 11 01    0  11    1  0  =  11 1        0  1  0  1  11 11 0  =  1  11 1  1     1   435.50
10: Rocinante 1.01     24.0/50 00 00 00 00 00 01 =0 00 0= ·· == 00 11 0= 11 0     11 1     1     11 0  0  =     0  0  1  11 11       1  11    1  1      286.50
11: Ozwald 0.43        23.0/43 00 00 00 01 00 10 =0 =0 00 == ·· 01 1= 11 11 0     =1 0           11       1              11 11       1  11       1      335.75
12: RBrChess 2.16      22.0/45 00 01 00 00 00 00 00 =1 01 11 10 ·· 00 00 11       11       0  0  11 =     0              11 11       1  11    1  1      316.50
13: Trex 1.9           17.5/49 00 00 00 00 0= 00 00 00 00 00 0= 11 ·· 01 00 0  =  01    0  1     01    0  1           1  11 01       0  11    1  1  1   179.25
14: Eden 0.013         17.0/50 00 00 00 00 00 00 00 00 00 1= 00 11 10 ·· 01 0     01 0  0  0  0  11 0     0        0  0  1= 11          11 1  1  1      180.75
15: Numpty 0.5pr       14.0/43 00 00 00 00 00 00 01 00 10 00 00 00 11 10 ··    0  =0             10             0  0     11 11       =  11       1      175.00
16: Reger 0.09         13.0/16       =  =        =        1  1     1  1     ··             1  0  1           =           1  1  1        1           1   190.00
17: Sjakk 1.1.3        12.0/18 0  =  1  0  0  =  1  1  1           =     1     ··       1        1  =  0        1  1                          1         246.75
18: Nanook 0.17        12.0/44 =0 00 00 00 00 00 00 00 00 00 =0 00 10 10 =1       ·· 0     0  0  11                   1  == ==       0  1=       1      138.25
19: Alibaba 1.17       11.5/18 0  0     0        0  1     0  1        1           1  ··    0              1  =        1  1  1  1        1  1            132.75
20: Chesley r317 ja    11.0/14 =        1              0           1  1        0        ··          1  =        1  1     1  1     1     1               152.25
21: Adamant 1.7        11.0/17 0  0                    1  0     1  0  1     0     1  1     ··             1  0        1  1  1  1           1            132.00
22: Micro-Max 4.8w     9.5/14     0  0  1  0  0     1  =        1     1     1     1           ·· 1                    1                 1               176.75
23: Xadreco 5.7        9.5/48  00 00 00 00 00 00 00 00 00 00 00 00 10 00 01 0  0  00          0  ··          0  0     0  1= 11    0  0  11       1  1    66.25
24: Brembo 0.62        9.0/16  1  0     0        0     0  1     =     1        =        0           ·· 1        1  =        1     =     1               138.50
25: Cleo 1             8.5/13  0        0        1        1        1           1        =           0  ··       1  0        1     1                 1   114.50
26: Gedeone 1620       8.5/19  0        0     0  0  1     =  0  1  0  1              0     0              ·· 0        =  1  1  =           1     1      108.50
27: Chessrikus 1.466   8.5/14  0     0  =           0  1                    =        =     1     1        1  ··       0     1  1           1            108.00
28: ALChess1.5b        8.0/19  0  =     0  0  0     0  0  1              1     0        0        1  0  0        ·· 1     1  1     =  1                   98.50
29: ArabianKnight 0.88 7.5/16  =  0  0  0     =     0  1  1           1  1     0        0           =  1        0  ··             1                     146.25
30: Fimbulwinter 5.02  7.5/19                 0  0     0  0        0  1           0  0     0  0  1        =  1        ·· 1  1  0        1  0     1       58.75
31: Youk 1.05          7.5/49  00 00 00 00 00 00 00 00 00 00 00 00 00 0= 00 0     == 0  0  0     0=       0     0     0  ·· 10    0  1  =1    1  1       46.00
32: Geko 0.43          7.0/53  00 00 00 00 00 00 00 00 00 00 00 00 10 00 00 0     == 0  0  0     00 0  0  0  0  0     0  01 ··    0     == 0  1  1  1    43.50
33: Damas9             5.5/12  0              1     0  1                    0        0     0              =  0        1        ··          1  1          83.25
34: Frittle 0.5        5.5/17  0  0  0  0     0  0  0  =                                0        1  =  0        =  0     1  1     ·· 1                   53.25
35: Joana_DC           5.0/21  0  0  0  0  0  0  0  0  0  0  0  0  1     =        1              1              0        0        0  ·· =        1       48.75
36: Sachy 0.2.01       4.5/48  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0     0= 0  0     0  00 0                 0  =0 ==       =  ··    =  =  1    20.25
37: Cheoss 0.649       2.0/15  0  0  0  0  0  0        0              0              0     0              0  0        1     1  0           ··            14.50
38: Neg 0.3d           1.5/14        0     0     0     0  0     0  0  0        0                                         0  0  0        =     ··    1     2.25
39: Ace v0.1           0.5/21  0  0  0     0  0  0  0     0  0  0  0  0  0        0              0        0           0  0  0        0  =        ··       2.25
40: Pos-1.20           0.0/14     0  0  0  0  0     0  0           0        0                    0     0                    0           0     0     ··    0.00

651 of 1560 games played

Buggy engines still in the tournament (I will take a decision about them later)

- Joana DC by Juan Pablo Mendoza (does not accept opponent's short castle and loses by time.)
- Cheoss 0.649 plays illegal moves sometimes, but when not it has a decent style of play.
- Yoda 1.30 does not accept opponent's underpromotion to other piece different of a queen.
- Youk loses on times seldomly I am not sure if the reason is for making illegal moves.

As you can notice, I started testing old versions of Dabbaba, KMTChess and Myrdinn, they are too strong for this tournament, so there is no point in replace them with latest versions. Anyway, latest versions (at the time of the tournament begins) will be used preferably.

So far, these engines are disqualified of the tournament:

Not working under Arena: (if someone has an idea on how to make them work, please reply)

ChessV 0.94

Lose on time:

Alice 0.3.5
Blitzkibitz rev4
Hippocampe 0.4.2
Mainsworthy 64.01.7
NSVChess 0.14
Suff 0.3a

Illegal moves:

Dendron 0.42ja (castles under check)

Always ponder:

Apilchess 1.05-r1b ja
Braincrack alpha JA
CPW 050908ja
Plisk 0.12

Crash:

Chess0 v0.3.2

Do not support x+y time management:

Chad's Chess 0.15 (this is a pitty, because chad is a nice engine to play with, maybe I will consider to run a gauntlet at 40/1 at the end of the cycles.)

Sites visited to download the engines, check authors, versions, etc.:

Jim Ablett's site http://homepages.tesco.net/henry.ablett/jims.html
Dan Corbit's site http://cap.connx.com/chess-engines/
Leo Dijskman's Last Update page http://wbec-ridderkerk.nl/html/lastupdates.htm
Guenther Simon's RWBC Anthology page http://www.rwbc-chess.de/wb_chron.htm
Arena webpage http://www.playwitharena.com
Computer chess wiki list (not available atm, need to be signed in)
Olivier Deville's site http://www.open-aurec.com/chesswar/

Special thanks to Tony Thomas for this useful information:
http://talkchess.com/forum/viewtopic.php?t=21046

Using this info I avoided testing some buggy engines.


In the waiting list for the next cycle:

Brutus RND
Cassandre 0.24
Chenard 1.136ja
Cpp1
Deep Duke 1.00
DreamChess 0.2.0
Gargamella 0.5.0
GFC 2.0
Giuchess 1.0b2DC
GrayMatter rev1569
Hanzo the Razor 1.1.1.1
Hoichess 1.03
Hoplite 2.1.1
Iach 0.56z
Ifrit j2.7
IQ23 0.03
Jars 1.75a
Jchess 1.0
Jester 0.83 dc
Kace 0.82ja (reports of being buggy)
Kanguruh 1.93
LarsenVB 0.05
Marginal 0.1
Marquis 1.6
MFchess 1.3ja
Microchess 1976
Minimardi 1.3
Minimax
Mizar 3.0
Mskcp 1.44ja (does this engine play standard chess?)
Needle 0.53.1
Neurone 0.04
O'Chess rv2
Philemon 0.5
Pierre 1.7ja
Piranha 0.5
Pooky 2.7
Predateur 0.15
Protej 0.56a
Pyotr 0.6ja
Qutechess 1.01t rev3 ja
RedQueen 0.2ja
Roque 1.1
Schola 1.1
SdBc 0.41.40
Sharper 0.17
Simple 0048
Sinapse 1.0
Skaki 1.23 ja
Slibo 0.49 ja
StAndersen 1.31c ja
Talvmenni (reports of being buggy)
Tarrasch 0.906
Tiffanys 0.5ja
Tikov 0.5
Toledo Nanochess
TSCP 1.81
Vanillachess 2.6g
Warrior 1.03
etc.

Please, report if some of these engines are buggy and feel free to recommend others.

Regards,
Carlos

[swami]

Thank you.

Interesting information about engines in 1800-2000 range of the ratings scale.

[Carlos777]

Thank you.

Interesting information about engines in 1800-2000 range of the ratings scale.

Hi Swami,

Thanks. I just downloaded your spreadsheet with the results of the engines in your STS tests. Very interesting info. Can you make another one with all engines tested so far?

Best,
Carlos

Edit:
- Surprising results for Booot 4.15 (not in a good way )
- The version of Bison is 9.8 or 9.11 ?

[Graham Banks]

Nice concept Carlos.
http://www.kirill-kryukov.com/chess/dis ... =forgotten
Mine had a different concept.

[swami]

Thank you.

Interesting information about engines in 1800-2000 range of the ratings scale.

Hi Swami,

Thanks. I just downloaded your spreadsheet with the results of the engines in your STS tests. Very interesting info. Can you make another one with all engines tested so far?

Best,
Carlos

Yes, I'm working on it. I hope to get around to finally testing engines in lower order, and add them to the list.

[Carlos777]

Nice concept Carlos.
http://www.kirill-kryukov.com/chess/dis ... =forgotten
Mine had a different concept.

Hi Graham,

I swear I did not see your tournament until now.
Sorry to name my tourney with the same as yours.

Yes, a different concept. I think it would be interesting to some people, at least it is a lot of fun for me.

Best,
Carlos

[Graham Banks]

Nice concept Carlos.
http://www.kirill-kryukov.com/chess/dis ... =forgotten
Mine had a different concept.

Hi Graham,

I swear I did not see your tournament until now.
Sorry to name my tourney with the same as yours.

Yes, a different concept. I think it would be interesting to some people, at least it is a lot of fun for me.

Best,
Carlos

Just shows that great minds think alike.

[swami]

Thank you.

Interesting information about engines in 1800-2000 range of the ratings scale.

Hi Swami,

Thanks. I just downloaded your spreadsheet with the results of the engines in your STS tests. Very interesting info. Can you make another one with all engines tested so far?

Best,
Carlos

Edit:
- Surprising results for Booot 4.15 (not in a good way )
- The version of Bison is 9.8 or 9.11 ?

I just checked the list, Bison is at 9.6a. Whoops didn't update. Perhaps I will run the suite again with Bison 9.8 and see what the scores turn out to be.

Yes, Booot is one of the very rare exceptions that doesn't correspond well when subjected to test suites testing. Just like Zappa and Smarthink 0.17a with poor time management in bullet.

[Carlos777]

Update:

Code: Select all

    Engine             Score   Ja Si KM Ro Si Cu Yo My Da Oz Ro RB Ch Tr Re Mi Sj Ad Ed Nu Al Br Cl AL Na Ar Ch Xa Ge Fr Fi Da Yo Ge Jo Sa Ne Ac Po Cp    S-B
01: Jabba 1.0          50.5/58 ·· 0= 10 00 11 11 11 11 11 11 11 11 =  11 1  =  1  1  11 11 1  0  1  1  =1 =  1  11 1  1  1  1  11 11 1  11 1  1  1  1   918.00
02: Simplex 0.0        44.5/52 1= ·· 1= == 01 11 01 01 =1 11 11 11    11 =  1  0  1  11 11 =           11 1  1  11 1  1     1  11 11 1  11 1  1  1      853.75
03: KMTChess 1.1       43.5/51 01 0= ·· 11 11 11 00 11 10 11 11 10    11    1  =  1  11 11 1  1     =  11 1     11    1     1  11 11 1  11    1  1  1   834.50
04: Roce 0.0390        43.5/51 11 == 00 ·· =0 10 1= 10 11 11 11 11    1=    1  1  1  11 11 1        1  11    1  11 1     1  1  11 11 1  11 1  1  1      794.25
05: Sissa 1.00         42.5/54 00 10 00 =1 ·· 11 1= =0 11 10 11 11 0  11 =  0  1     11 11 1  1  1  1  11 1  =  11 1  1     1  11 11 1  11       1  1   749.25
06: Cuckoo Chess 1.0   37.5/52 00 00 00 01 00 ·· 11 =1 11 01 10 11    11    1  =  0  11 11          1  11 =  1  11 1  1  1  0  11 11 1  11    1  1  1   594.50
07: Yoda 1.30          36.0/51 00 10 11 0= 0= 00 ·· 11 =0 =1 =1 11 0  11 =     0     11 10 1  1  0     11 1     11 1  1  1     11 11 1  11 1  1         630.75
08: Myrddin 0.81a2     34.0/50 00 10 00 01 =1 =0 00 ·· 11 =1 11 =0    11    0  0     11 11 0        1  11 1  1  11 0  1     1  11 11 1  11    1  1      572.00
09: Dabbaba 2.46       30.5/52 00 =0 01 00 00 00 =1 00 ·· 11 1= 10 1  11    =  0  0  11 01    1     1  11 0  0  11    =  1  0  11 11 1  11 1     1      471.50
10: Ozwald 0.43        25.5/48 00 00 00 00 01 10 =0 =0 00 ·· == 01    1= 0        0  11 11 0           =1    0  11 1     1  =  11 11 1  11 1  1         360.75
11: Rocinante 1.01     24.0/50 00 00 00 00 00 01 =0 00 0= == ·· 00    11 0        1  0= 11 1  0  0  0  11 0     11 =     1     11 11 1  11 1  1         308.25
12: RBrChess 2.16      22.5/51 00 00 01 00 00 00 00 =1 01 10 11 ·· 0  00    0  0  0  00 11    =  =  0  11 0     11 0  0        11 11 1  11 1  1         336.50
13: Chesley r317 ja    18.0/21 =           1     1     0        1  ·· 1     1  0     1  1     1  =  1     1           1        1  1  1  1        1  1   272.75
14: Trex 1.9           18.0/52 00 00 00 0= 00 00 00 00 00 0= 00 11 0  ·· 0     =  1  01 00    0  0  0  01 =     01 1     1     11 01 0  11 1  1  1      201.00
15: Reger 0.09         17.5/23 0  =        =     =        1  1        1  ·· 0     1  1     =           1     =  1  0     1  1  1  1     1  1  1  1      235.50
16: Micro-Max 4.8w     17.0/24 =  0  0  0  1  0     1  =        1  0     1  ·· 1     1        1  1  =  1  1     1     =  1           1  1        1      282.00
17: Sjakk 1.1.3        16.0/23 0  1  =  0  0  =  1  1  1        1  1  =     0  ··       1     =  0  1     1     1     1              1     1     1      310.50
18: Adamant 1.7        16.0/24 0  0  0  0     1        1  1  0  1     0  0        ·· 1     1           1     0  1  1     1  1  1  1        1  1     1   213.50
19: Eden 0.013         16.0/49 00 00 00 00 00 00 00 00 00 00 1= 11 0  10 0  0     0  ·· 01 0  0        01 0     11 0     0     1= 11    11 1  1         185.25
20: Numpty 0.5pr       15.5/48 00 00 00 00 00 00 01 00 10 00 00 00 0  11       0     10 ··    0  0  0  =0 0     10    =        11 11 =  11    1  1      181.50
21: Alibaba 1.17       14.5/23 0  =  0  0  0     0  1     1  0           =        0  1     ··          1     =  1  1     1  1  1  1     1  1  1         183.75
22: Brembo 0.62        14.0/22 1     0     0     0     0     1  =  0  1     0  =     1  1     ·· 1  1     =           =     1     1  1  1        1      206.00
23: Cleo 1             14.0/20 0           0     1           1  =  =  1     0  1        1     0  ·· 1     0           1        1  1  1  1        1  1   176.75
24: ALChess1.5b        12.5/24 0     =  0  0  0     0  0     1  1  0  1     =  0        1     0  0  ··    1     1     =        1  1  1  1        1      162.50
25: Nanook 0.17        12.5/48 =0 00 00 00 00 00 00 00 00 =0 00 00    10 0  0     0  10 =1 0           ··    0  11 =     1  0  == == 0  1=    1         151.50
26: ArabianKnight 0.88 12.0/23 =  0  0     0  =  0  0  1     1  1  0  =     0  0     1  1     =  1  0     ··          1              1  1        1      202.50
27: Chessrikus 1.466   11.5/19 0  0     0  =  0     0  1  1              =        1        =           1     ·· 1  1     0  1     1        1  1         160.25
28: Xadreco 5.7        10.5/52 00 00 00 00 00 00 00 00 00 00 00 00    10 0  0  0  0  00 01 0        0  00    0  ··    0  0  0  1= 11 0  11    1  1  1    68.25
29: Gedeone 1620       10.0/23 0  0     0  0  0  0  1     0  =  1     0  1        0  1     0           =     0     ··    =  =  1  1        1  1         134.00
30: Frittle 0.5        9.5/23  0  0  0     0  0  0  0  =        1  0        =  0        =     =  0  =     0     1     ··       1  1  1  1        1      102.75
31: Fimbulwinter 5.02  8.5/23  0        0     0  0     0  0  0        0  0  0     0  1     0           0     1  1  =     ·· 0  1  1     1  1  1          65.00
32: Damas9             8.0/20  0  0  0  0  0  1     0  1  =              0        0        0  0        1     0  1  =     1  ··             1  1         120.25
33: Youk 1.05          7.5/50  00 00 00 00 00 00 00 00 00 00 00 00 0  00 0        0  0= 00 0     0  0  ==       0= 0  0  0     ·· 10 1  =1 1  1          48.50
34: Geko 0.43          7.0/52  00 00 00 00 00 00 00 00 00 00 00 00 0  10 0        0  00 00 0  0  0  0  ==    0  00 0  0  0     01 ··    == 1  1  1       45.50
35: Joana_DC no book   6.0/28  0  0  0  0  0  0  0  0  0  0  0  0  0  1     0  0        =     0  0  0  1  0     1     0        0     ·· =     1  1       51.50
36: Sachy 0.2.01       4.5/52  00 00 00 00 00 00 00 00 00 00 00 00 0  00 0  0        00 00 0  0  0  0  0= 0     00    0  0     =0 == =  ·· =  =  1       21.50
37: Neg 0.3d           2.5/23  0  0     0        0     0  0  0  0     0  0     0  0  0     0                 0     0     0  0  0  0     =  ·· 1  1        2.75
38: Ace v0.1           0.5/27  0  0  0  0     0  0  0     0  0  0     0  0        0  0  0  0           0     0  0  0     0  0  0  0  0  =  0  ··          2.25
39: Pos-1.20           0.0/24  0  0  0  0  0  0     0  0           0  0  0  0  0        0     0  0  0     0     0     0           0  0  0  0     ··       0.00
39: Cpp1               0.0/8   0     0     0  0                    0              0              0              0                                   ··    0.00

734 of 1560 games played

Cheoss 0.649 was removed due losses on time (5 out of 14 games = 35.7%,) some times early in the game (apparently illegal moves. BTW, anybody know how to save the debug files automatically in Arena?)

Cpp1 replaced Cheoss.

Buggy engines still in the tournament (I will take a decision about them later)

- Joana DC by Juan Pablo Mendoza (when playing with white pieces does not accept opponent's short castle and loses by time. 4 losses on time out of 25 games.)
- Yoda 1.30 does not accept opponent's underpromotion to other piece different of a queen.

So far, these engines are disqualified of the tournament:

Not working under Arena: (if someone has an idea on how to make them work, please reply)

ChessV 0.94

Lose on time:

Alice 0.3.5
Blitzkibitz rev4
Cheoss 0.649
Hippocampe 0.4.2
Mainsworthy 64.01.7
NSVChess 0.14
Suff 0.3a

Illegal moves:

Dendron 0.42ja (castles under check)

Always ponder:

Apilchess 1.05-r1b ja
Braincrack alpha JA
CPW 050908ja
Plisk 0.12

Crash:

Chess0 v0.3.2

Do not support x+y time management:

Chad's Chess 0.15

[Richard Allbert]

Jabba better than expected.

Thanks a lot for the result, I'll post it on my blog

Regards

Richard