Time-Odds tournament

[smrf]

Sorry, my site is currently taken off. Nevertheless a download of the donationware version always is possible as well as of the donators' bonus engine (which needs a personal key set).

a) http://www.10x8.net/down/SmNewSetup.exe
b) http://www.10x8.net/down/SmirfEngine.dll

[Ovyron]

Thank you for the links.

[beachknight]

With the help of Smirfoglot, I have carried out
few tests with BC versions of Smirf, 168h and
171c, resp.

That was not possible with the MS versions.
I suppose that testing them, ie 171e, 171g
and new 172 series, require other keys, read
personal key sets.

TIA,

[smrf]

Well, the BC engine, which is included in the donationware SMIRF GUI package, is thought as an open chance to learn all about the SMIRF FullChess project. It is only limited in its cache size. A public key set is included for that free donationware version. In contrast the donators' bonus engine is always more actual, mature and nearly 100% faster, and has no cache size limits. It is reserved for those people, who have supported the SMIRF FullChess project by money or additional work.

Because of the fact, that most people in our days are not willing to perform tests manually, SMIRF has not been well investigated. But since HGM has written his SMIRF-o-Glot Winboard adapter (which is working for the donationware engine and the bonus version, if keys are on hand), the strength of my first written playing chess program could be found out by automated tests as usual today.

HG Mullers finishing automated 10x8 chess engine test event still used a SMIRF version, which has not been fully aware to time frames of automated matches. Engine MS-172b should do better now and moreover has some old bugs erased. Nevertheless already the previous bonus engine of SMIRF could place itself near the top of 10x8 engines.

The donationware package still is far behind of the new top SMIRF's abilities.

[Uri Blass]

The table so far
If we ignore the games of fairymax/24 is the following:

1. Joker80 86% 38.0 / 44
6. Joker80 / 3 75% 33.0 / 44
9. Joker80 / 9 58% 25.5 / 44
11. Joker80 / 24 52% 23.0 / 44
18. Joker80 / 54 34% 15.0 / 44

2. Smirf 82% 36.0 / 44
8. Smirf / 3 65% 28.5 / 44
12. Smirf / 9 45% 20.0 / 44

3. TSCP Gothic 80% 35.0 / 44
7. TSCP Gothic / 3 67% 29.5 / 44
14. TSCP Gothic / 9 45% 20.0 / 44
17. TSCP Gothic / 24 34% 15.0 / 44

4. TJchess Bird 78% 34.5 / 44
5. TJchess / 3 76% 33.5 / 44
10. TJchess / 9 53% 23.5 / 44
16. TJchess / 24 37% 16.5 / 44
21. TJchess / 54 17% 7.5 / 44

13. Faiiry-Max / 3 45% 20.0 / 44
15. Fairy-Max 4.8 t 40% 17.5 / 44
20. Fairy-Max / 9 23% 10.0 / 44

19. ArcBishop80 1.00 26% 11.5 / 44
22. BigLion80 2.23 16% 7.0 / 44
23. Chancellor 1.00d 13% 6.0 / 44

I suspect based on the results that Fairy-Max and maybe Tjchess have bugs that reduce their performance at long time control.

Uri

[hgm]

Perhaps some explanation:

The basic tournament with 23 engines was finished, but failed to accomplish one of its goals, namely to do more symmetric testing on ArcBishop80, BigLion80 and Chancellor (i.e. not only against stronger opponents). So I decided to add a few more strongly handicapped versions of the other engines, which I now play as gauntlets that will be added to the same PGN file, and thus appear in the cross table.

I am currently playing Fairy-Max/24; later I will add also TSCP-G/54 and Fairy-Max/54.

The other goal was to see how the rating of the various engines suffered from time odds. I will analize this after the games from these new engines are completed as well.

I know that the Smirf version I am playing here is not the best / newest one, which is why I did not add too many dumbed-down versions of it.

[Uri Blass]

Perhaps some explanation:

The basic tournament with 23 engines was finished, but failed to accomplish one of its goals, namely to do more symmetric testing on ArcBishop80, BigLion80 and Chancellor (i.e. not only against stronger opponents). So I decided to add a few more strongly handicapped versions of the other engines, which I now play as gauntlets that will be added to the same PGN file, and thus appear in the cross table.

I am currently playing Fairy-Max/24; later I will add also TSCP-G/54 and Fairy-Max/54.

The other goal was to see how the rating of the various engines suffered from time odds. I will analize this after the games from these new engines are completed as well.

I know that the Smirf version I am playing here is not the best / newest one, which is why I did not add too many dumbed-down versions of it.

Very strange performance of fairymax.
The results suggest that playing strength of fairy-max is not monotonic function of time.

13. Faiiry-Max / 3 45% 20.5 / 46 (1065.0, 343.3)
15. Fairy-Max 4.8 t 40% 18.5 / 46 (1069.0, 293.8)
19. Fairy-Max /24 30% 14.0 / 46 (1078.0, 198.5)
21. Fairy-Max / 9 23% 10.5 / 46 (1085.0, 188.5)

Edit:It is also possible that there is some problem with the experiment
and one program was slowed down during part of the games.

It is also possible that it is because of luck but I think that the results suggest to check if there was some problem.

Uri

[Uri Blass]

Perhaps some explanation:

The basic tournament with 23 engines was finished, but failed to accomplish one of its goals, namely to do more symmetric testing on ArcBishop80, BigLion80 and Chancellor (i.e. not only against stronger opponents). So I decided to add a few more strongly handicapped versions of the other engines, which I now play as gauntlets that will be added to the same PGN file, and thus appear in the cross table.

I am currently playing Fairy-Max/24; later I will add also TSCP-G/54 and Fairy-Max/54.

The other goal was to see how the rating of the various engines suffered from time odds. I will analize this after the games from these new engines are completed as well.

I know that the Smirf version I am playing here is not the best / newest one, which is why I did not add too many dumbed-down versions of it.

Very strange performance of fairymax.
The results suggest that playing strength of fairy-max is not monotonic function of time.

13. Faiiry-Max / 3 45% 20.5 / 46 (1065.0, 343.3)
15. Fairy-Max 4.8 t 40% 18.5 / 46 (1069.0, 293.8)
19. Fairy-Max /24 30% 14.0 / 46 (1078.0, 198.5)
21. Fairy-Max / 9 23% 10.5 / 46 (1085.0, 188.5)

Edit:It is also possible that there is some problem with the experiment
and one program was slowed down during part of the games.

It is also possible that it is because of luck but I think that the results suggest to check if there was some problem.

Uri

I wonder what happened to the table because I now see less games in the table and less participants

BigLion is now leading the tournament

1. BigLion80 2.23 100% 4.0 / 4 ( 0.0, 0.0)
2. TSCP Gothic / 24 100% 4.0 / 4 ( 0.0, 0.0)
3. TJchess / 24 100% 4.0 / 4 ( 0.0, 0.0)
4. Faiiry-Max / 3 87% 3.5 / 4 ( 2.0, 1.8)
5. TJchess / 54 75% 3.0 / 4 ( 4.0, 3.0)
6. Fairy-Max / 9 62% 2.5 / 4 ( 6.0, 3.8)
7. Fairy-Max 4.8 t 50% 2.0 / 4 ( 8.0, 4.0)
8. TJchess / 9 50% 2.0 / 4 ( 8.0, 4.0)
9. Joker80 / 3 50% 1.0 / 2 ( 2.0, 1.0)
10. TJchess / 3 50% 1.0 / 2 ( 2.0, 1.0)
11. TSCP Gothic / 3 37% 1.5 / 4 ( 10.0, 3.8)
12. Fairy-Max /24 25% 1.0 / 4 ( 12.0, 3.0)
13. TSCP Gothic / 9 12% 0.5 / 4 ( 14.0, 1.8)
14. Fairy-Max / 54 0% 0.0 / 4 ( 16.0, 0.0)
15. Chancellor 1.00d 0% 0.0 / 4 ( 16.0, 0.0)
16. TSCP Gothic / 54 0% 0.0 / 4 ( 16.0, 0.0)

[hgm]

Very strange performance of fairymax.
The results suggest that playing strength of fairy-max is not monotonic function of time.

13. Faiiry-Max / 3 45% 20.5 / 46 (1065.0, 343.3)
15. Fairy-Max 4.8 t 40% 18.5 / 46 (1069.0, 293.8)
19. Fairy-Max /24 30% 14.0 / 46 (1078.0, 198.5)
21. Fairy-Max / 9 23% 10.5 / 46 (1085.0, 188.5)

Edit:It is also possible that there is some problem with the experiment
and one program was slowed down during part of the games.

It is also possible that it is because of luck but I think that the results suggest to check if there was some problem.

Indeed the behavior of Fairy-Max is very suspicious. I know that Fairy-Max is buggy: it frequently hangs, and even was responsible for spoiling earlier tournaments by soaking up CPU time by many hanging processes. But now I use a kill list that kills any Fairy-Max processes after every game. So I believe the current tournament to be free of such corruption.

Unfortunately I have never been able to find a position where Fairy-Max reproducibly crashes. If I take positions where it was hanging on, and rerun them, it always finds a move without problems. It is supposed to be nearly the same as micro-Max, and micro-Max never hangs. Although sometimes it thinks extremely long, like 5 min on one move in a 2-min game. But in thos cases a normal search is going on, and eventually it always comes with a move. But the memory foortprint of a hanging Fairy-Max process goes to nearly zero, so it cannot be doing normal search (as it would then access most of the hash table).

Anyway, an explanation of the non-montonic behavior might be that there is a fixed probability per unit of time for Fairy-Max crashing. That would mean that longer time controls are counter-productive, as at some point the probability for a crash (and forfeiting on time) would approach 100%. Only games played fast enough that there is a significant probability to finish them without a crash could ever be won.

I have not spent a great deal of time on Fairy-Max, (it was never intended as an engine for serious competition, I made it only for determining piece values of exotic pieces, in very fast games), but I really should put it under the microscope to get rid of this very annoying bug. The frequency with which it happens, and the irreproducibility, suggests that hash collissions are somehow involved.

But it could be that

[hgm]

I wonder what happened to the table because I now see less games in the table and less participants

BigLion is now leading the tournament

1. BigLion80 2.23 100% 4.0 / 4 ( 0.0, 0.0)
2. TSCP Gothic / 24 100% 4.0 / 4 ( 0.0, 0.0)
3. TJchess / 24 100% 4.0 / 4 ( 0.0, 0.0)
4. Faiiry-Max / 3 87% 3.5 / 4 ( 2.0, 1.8)
5. TJchess / 54 75% 3.0 / 4 ( 4.0, 3.0)
6. Fairy-Max / 9 62% 2.5 / 4 ( 6.0, 3.8)
7. Fairy-Max 4.8 t 50% 2.0 / 4 ( 8.0, 4.0)
8. TJchess / 9 50% 2.0 / 4 ( 8.0, 4.0)
9. Joker80 / 3 50% 1.0 / 2 ( 2.0, 1.0)
10. TJchess / 3 50% 1.0 / 2 ( 2.0, 1.0)
11. TSCP Gothic / 3 37% 1.5 / 4 ( 10.0, 3.8)
12. Fairy-Max /24 25% 1.0 / 4 ( 12.0, 3.0)
13. TSCP Gothic / 9 12% 0.5 / 4 ( 14.0, 1.8)
14. Fairy-Max / 54 0% 0.0 / 4 ( 16.0, 0.0)
15. Chancellor 1.00d 0% 0.0 / 4 ( 16.0, 0.0)
16. TSCP Gothic / 54 0% 0.0 / 4 ( 16.0, 0.0)

I started a new tournament (TOT2) in a hurry, as the other ne was finished, and I had to leave for Zurich in a hurry. So I had no time yet to put the cross table and zipped PGN games on my regular website. Btw, the cross table would still be there, it is just that I changed the link in the viewer page to point to the new one (TOT2x.txt) in stead of the old one (TOT1x.pgn). You can always paste the link in your browser to request the page. (Right-click on the link and ask for properties to see where the pgn and cross-table files are stored on my computer.)

The newly-started tournament unfortunately got stuck, because one of the engines was still rigged for playing Bird-Chess (and the Bird.pgn only contained one position, while to play 4 games per pairing 2 were needed). I could find no way to recover from this: PSWBTM does not allow me to change the engine options once the match has started. So I had to abort the tournament, and start a new one.

The idea now is to get a bit better statistics for the games between the lowest ranked engines of TOT1, by increasing their number of games from 2 to 6 through adding 4 new games from 2 new opening positions. For now I did not include the engines handicapped by a factor 3, perhaps I will add these later as gauntlets.