Quick steps on the dancefloor (2): Discocheck 3.6

[Mike S.]

I have repeated the test reported under

http://www.talkchess.com/forum/viewtopic.php?t=44005

with version 3.6. Conditions and opponents were identical.

DiscoCheck 3.6 scored much better!

38.5/100, 3.51 had scored 29.0/100. Congratulations - Texel 1.01 has been beaten 11-9.

Code: Select all

Discocheck 3.6   - Hannibal 1.2           5.0 - 15.0    +2/=6/-12    25.00%
Discocheck 3.6   - MinkoChess 1.3         7.0 - 13.0    +5/=4/-11    35.00%
Discocheck 3.6   - Quazar 0.4             9.0 - 11.0    +7/=4/-9     45.00%
Discocheck 3.6   - Texel 1.01             11.0 - 9.0    +8/=6/-6     55.00%
Discocheck 3.6   - Shredder Classic 4     6.5 - 13.5    +4/=5/-11    32.50%

Discocheck 3.6 total: 38.5/100

(In 5/100 games, losses on time happened which were adjudicated as draws, according to the positions.)

Games (CBV): http://remixshare.com/download/gglqu

[lucasart]

I have repeated the test reported under

http://www.talkchess.com/forum/viewtopic.php?t=44005

with version 3.6. Conditions and opponents were identical.

DiscoCheck 3.6 scored much better!

38.5/100, 3.51 had scored 29.0/100. Congratulations - Texel 1.01 has been beaten 11-9.

Code: Select all

Discocheck 3.6   - Hannibal 1.2           5.0 - 15.0    +2/=6/-12    25.00%
Discocheck 3.6   - MinkoChess 1.3         7.0 - 13.0    +5/=4/-11    35.00%
Discocheck 3.6   - Quazar 0.4             9.0 - 11.0    +7/=4/-9     45.00%
Discocheck 3.6   - Texel 1.01             11.0 - 9.0    +8/=6/-6     55.00%
Discocheck 3.6   - Shredder Classic 4     6.5 - 13.5    +4/=5/-11    32.50%

Discocheck 3.6 total: 38.5/100

(In 5/100 games, losses on time happened which were adjudicated as draws, according to the positions.)

Games (CBV): http://remixshare.com/download/gglqu

Thanks for your interest in Discocheck. But there was probably a bit of luck there: Texel is certainly better than Discocheck, by a significant margin. You'll find that Discocheck scores better and better as the time control decreases.
At 8"+0.2" time control (which is stupidly fast), DC 3.6 scored +22 elo compared to DC 3.5.1, against a few different engines in the same conditions, after 2000 games.
My guess is that, at normal time control, Texel is still 150 elo stronger than DC 3.6. DC gets closer at super fast tc though.

[lucasart]

I have repeated the test reported under

http://www.talkchess.com/forum/viewtopic.php?t=44005

with version 3.6. Conditions and opponents were identical.

DiscoCheck 3.6 scored much better!

38.5/100, 3.51 had scored 29.0/100. Congratulations - Texel 1.01 has been beaten 11-9.

Code: Select all

Discocheck 3.6   - Hannibal 1.2           5.0 - 15.0    +2/=6/-12    25.00%
Discocheck 3.6   - MinkoChess 1.3         7.0 - 13.0    +5/=4/-11    35.00%
Discocheck 3.6   - Quazar 0.4             9.0 - 11.0    +7/=4/-9     45.00%
Discocheck 3.6   - Texel 1.01             11.0 - 9.0    +8/=6/-6     55.00%
Discocheck 3.6   - Shredder Classic 4     6.5 - 13.5    +4/=5/-11    32.50%

Discocheck 3.6 total: 38.5/100

(In 5/100 games, losses on time happened which were adjudicated as draws, according to the positions.)

Games (CBV): http://remixshare.com/download/gglqu

Thanks for your interest in Discocheck. But there was probably a bit of luck there: Texel is certainly better than Discocheck, by a significant margin. You'll find that Discocheck scores better and better as the time control decreases.
At 8"+0.2" time control (which is stupidly fast), DC 3.6 scored +22 elo compared to DC 3.5.1, against a few different engines in the same conditions, after 2000 games.
My guess is that, at normal time control, Texel is still 150 elo stronger than DC 3.6. DC gets closer at super fast tc though.

I was curious to measure against Texel, so I played 200 games in 1'+1" and got the following result

Score of DiscoCheck 3.6 Boney M vs Texel 1.01 JA 64-bit: 48 - 96 - 56 [0.38] 200
ELO difference: -85

So even at this relatively fast time control, Texel won by a good margin. Here's the shortest win by Texel:

Code: Select all

[White "Texel 1.01 JA 64-bit"]
[Black "DiscoCheck 3.6 Boney M"]
[Result "1-0"]
[PlyCount "77"]
[TimeControl "60+1"]

1. e4 {book} c5 {book} 2. Nf3 {book} d6 {book} 3. d4 {book} cxd4 {book}
4. Nxd4 {book} Nf6 {book} 5. Nc3 {book} a6 {book} 6. Bg5 {book} e6 {book}
7. f4 {book} Qb6 {book} 8. Nb3 {book} Nbd7 {book} 9. Qd2 {+0.03/13 2.8s}
Be7 {2.7s} 10. O-O-O {-0.03/14 2.7s} O-O {2.7s} 11. Kb1 {0.00/14 2.6s}
Qc7 {2.6s} 12. Be2 {-0.07/14 2.8s} b6 {2.6s} 13. f5 {+0.12/13 2.6s} exf5 {2.6s}
14. exf5 {+0.23/14 2.5s} Bb7 {2.5s} 15. Qf4 {+0.23/14 2.4s} Bxg2 {2.5s}
16. Rhg1 {+0.85/14 2.4s} Bb7 {2.4s} 17. Bh6 {+2.06/15 2.4s} g6 {3.4s}
18. Bxf8 {+2.13/14 2.3s} Rxf8 {2.3s} 19. Nd4 {+2.16/14 2.3s} Ne5 {2.3s}
20. fxg6 {+2.82/15 2.2s} hxg6 {2.3s} 21. h4 {+2.81/14 2.2s} Nd5 {2.2s}
22. Nxd5 {+2.79/15 2.2s} Bxd5 {2.2s} 23. h5 {+3.05/13 2.2s} b5 {2.2s}
24. b3 {+3.31/13 2.1s} Ba8 {2.2s} 25. Bd3 {+5.00/13 2.1s} Qc8 {2.7s}
26. hxg6 {+9.51/13 2.0s} Qh3 {2.1s} 27. gxf7+ {+21.66/14 2.2s} Kh8
28. Nf5 {+23.57/15 2.0s} Bg2 {3.1s} 29. Nxe7 {+24.17/13 2.0s} Kg7 {2.4s}
30. Nf5+ {+24.77/10 1.9s} Kxf7 {2.0s} 31. Nxd6+ {+28.15/12 1.9s} Ke6 {2.9s}
32. Qxf8 {+29.75/11 1.9s} Nxd3 {2.8s} 33. Rxd3 {+M13/9 7.4s} Qxd3 {2.7s}
34. cxd3 {+M11/9 2.4s} Kd5 {1.8s} 35. Rxg2 {+M9/9 1.7s} Kc5 {1.8s}
36. Nb7+ {+M7/8 1.7s} Kb6 37. Qc5+ {+M5/6 0.032s} Kxb7 38. Rg7+ {+M3/4 0.001s}
Kb8 39. Qf8# {+M1/2 0.001s, White mates} 1-0

[Mike S.]

I had the same idea and let run a match, overnight. The result is very similar. Under the following conditions:

2m+1s, Atom N455/1.66 GHz
64 MB hash, ponder off
S.Canbaz' Top-50 openings
DF13-GUI, Windows 7 32-bit

it seems that the difference DiscoCheck 3.6 - Texel 1.01 is very close to 100 Elo only, or even less:

Code: Select all

Disco vs. Texel, Blitz 2m+1s
                                
1   Texel 1.01 JA 32-bit   +96  +50/=27/-23 63.50%   63.5/100
2   DiscoCheck 3.6         -96  +23/=27/-50 36.50%   36.5/100

Some notes:

- The engines had some Nalimov support, IOW if a 4-piece ending, or a 5-piece ending with R-R or Q-Q was reached, the interface finished the game from the tables. I did not investigate if one engine may have benefit more from that, than the other.

- In one game each, the engines stopped working for reasons unclear, thus losing on time. The match continued normally. No time losses otherwise.

- Resign was off, so the shortest win was 41 moves long. Although draw was set to "early", there were three long draws with 103...144 moves (and two long wins).

[Event "Disco vs. Texel, Blitz 2m+1s"]
[Site "NC-110"]
[Date "2012.06.29"]
[Round "21"]
[White "Texel 1.01 JA 32-bit"]
[Black "DiscoCheck 3.6"]
[Result "0-1"]
[ECO "B80"]
[Annotator "0.45;0.49"]
[PlyCount "82"]
[TimeControl "120+1"]

{Intel(R) Atom(TM) CPU N455 @ 1.66GHz 1662 MHz W=11.1 plies; 198kN/s B=19.
0 plies; 106kN/s} 1. e4 c5 2. Nf3 d6 3. d4 cxd4 4. Nxd4 Nf6 5. Nc3 a6 6. Be3 e6
7. f3 b5 {Both last book move} 8. Bd3 {0.45/13 5} Be7 {0.49/13 4 (b4)} 9. O-O {
0.38/13 4} e5 {0.55/13 4 (0-0)} 10. Nde2 {0.47/12 3} O-O {0.56/13 4} 11. a4 {
0.48/12 3 (Nd5)} b4 {0.40/15 4} 12. Nd5 {0.27/12 3} Nxd5 {0.39/15 4} 13. exd5 {
0.22/13 3} Nd7 {0.38/14 4} 14. Qe1 {0.22/12 3 (Qd2)} Qa5 {0.27/12 4} 15. c4 {
0.27/11 3 (c3)} f5 {0.15/11 4 (Nc5)} 16. g3 {0.18/12 3 (Bc2)} Nc5 {0.05/13 4}
17. Bc2 {0.18/12 3} Bd7 {0.04/12 4} 18. b3 {0.20/12 3 (Kg2)} Rae8 {0.09/12 3
(Bf6)} 19. Bd2 {0.15/11 4 (Rd1)} Qb6 {-0.01/12 3 (Rb8)} 20. a5 {0.56/12 3} Qa7
{0.08/14 3} 21. Kh1 {0.51/13 3 (Kg2)} e4 {-0.11/12 3 (Bf6)} 22. Ra2 {0.46/11 3
(Be3)} Bf6 {-0.30/11 3} 23. fxe4 {0.40/12 3 (Be3)} fxe4 {-0.79/12 3} 24. Nf4 {
0.04/12 4} Bg4 {-0.59/12 3 (Qb8)} 25. Bxb4 {0.17/11 3} Bf3+ {-0.41/12 3} 26.
Ng2 {0.28/12 3 (Rxf3)} Bd4 {-1.08/11 3 (Bg5)} 27. Qb1 {-0.39/12 9 (Qd2)} Qd7 {
-1.34/11 3 (Qf7)} 28. Bxc5 {-0.70/11 2} dxc5 {-1.29/12 3} 29. b4 {-1.49/12 4
(Qe1)} Qh3 {-3.24/12 3 (Qg4)} 30. Bd3 {-4.93/13 5 (Bb3)} exd3 {-4.74/12 3} 31.
Qxd3 {-8.71/13 2} Be4 {-7.70/13 3 (Bxg2+)} 32. Qd1 {-17.57/15 5} Rxf1+ {-10.75/
13 4} 33. Qxf1 {-9.19/2 0} Re5 {-14.91/14 4 (Bxg2+)} 34. d6 {-#6/11 8 (Rf2)}
Rh5 {-#15/8 3} 35. Qg1 {-#5/10 2} Bxg1 {-#13/8 3 (Qxg3)} 36. Kxg1 {-#6/9 2}
Qxh2+ {-#11/9 3} 37. Kf1 {-#5/8 2} Qh1+ {-#9/10 2} 38. Kf2 {-#4/8 2 (Ke2)}
Qxg2+ {-#7/20 2} 39. Ke3 {-#3/7 0} Qf3+ {-#5/86 5} 40. Kd2 {-#2/2 0} Rh2+ {-#3/
127 0} 41. Kc1 {-#1/3 0} Qh1# {0.01/127 0} 0-1

[Event "Disco vs. Texel, Blitz 2m+1s"]
[Site "NC-110"]
[Date "2012.06.30"]
[Round "72"]
[White "DiscoCheck 3.6"]
[Black "Texel 1.01 JA 32-bit"]
[Result "0-1"]
[ECO "D44"]
[Annotator "0.62;0.47"]
[PlyCount "84"]
[TimeControl "120+1"]

{Intel(R) Atom(TM) CPU N455 @ 1.66GHz 1662 MHz W=16.1 plies; 105kN/s B=11.
0 plies; 228kN/s} 1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 e6 5. Bg5 dxc4 6. e4 b5
7. e5 h6 8. Bh4 g5 9. Nxg5 hxg5 10. Bxg5 {Both last book move} Nbd7 {0.47/12 4}
11. Qf3 {0.62/12 4 (exf6)} Bb7 {0.59/11 3 (Bh6)} 12. exf6 {0.61/11 4} Qa5 {0.
53/11 3} 13. h4 {0.52/11 4 (h3)} O-O-O {0.13/11 3} 14. Qg3 {-0.07/11 4 (Qe3)}
c5 {-0.37/11 3} 15. Be3 {-0.57/11 4 (Rd1)} cxd4 {-0.58/11 3} 16. Bxd4 {-1.02/
13 4} Nc5 {-1.03/11 3 (e5)} 17. O-O-O {-0.82/11 4 (Bxc5)} b4 {-2.26/10 3} 18.
Bxc4 {-0.98/11 4} Rg8 {-1.67/10 3 (Bd6)} 19. Qe3 {-1.00/10 4} bxc3 {-1.41/10 3
(Bxg2)} 20. Bxc3 {-0.36/11 4} Rxd1+ {-1.03/11 4 (Qa4)} 21. Rxd1 {-0.26/13 4}
Qa4 {-1.26/11 3} 22. Rd4 {-0.23/13 3} Qc6 {-0.81/12 3} 23. Qd2 {-0.28/12 3} Nd7
{-0.78/11 4} 24. Bb3 {-0.46/12 3} Nb6 {-1.61/12 5 (Ba6)} 25. Bd1 {-0.83/10 3
(Rd8+)} Bc5 {-2.03/12 3 (Qc7)} 26. Bf3 {-1.95/12 3} Qc7 {-2.13/13 3} 27. Rg4 {
-2.19/13 3} Rxg4 {-2.06/13 3} 28. Bxg4 {-2.24/13 3} Bxg2 {-2.10/12 3} 29. f3 {
-2.14/13 3} Qh2 {-2.20/11 3} 30. Qg5 {-0.97/13 5} Qg1+ {-1.49/11 7} 31. Kc2 {
-0.91/13 3} Qf2+ {-2.25/11 2} 32. Kb3 {-0.82/13 5} Bf1 {-3.77/11 2 (Kb7)} 33.
Bd2 {-3.29/10 3 (Qd2)} Kb7 {-4.85/11 3} 34. f4 {-4.09/10 4 (Kc2)} Kc6 {-9.05/
11 3 (Bd3)} 35. f5 {-6.75/10 3} Nc4 {-10.16/12 2} 36. Bc3 {-7.02/10 4} Ne3 {
-17.01/12 2} 37. Qxe3 {-14.67/11 3} Qxe3 {-18.01/10 2} 38. a3 {-16.27/10 3
(fxe6)} Qe4 {-17.09/9 2} 39. Bd4 {-23.09/9 4 (h5)} Bxd4 {-#6/7 3 (Qxg4)} 40.
Ka2 {-#8/9 2 (fxe6)} Bc4+ {-#3/6 0 (Qxg4)} 41. Ka1 {-#4/127 0} Qe1+ {-#2/4 0}
42. Bd1 {-#2/1 0} Qxd1# {-#1/2 0} 0-1

[lucasart]

- The engines had some Nalimov support, IOW if a 4-piece ending, or a 5-piece ending with R-R or Q-Q was reached, the interface finished the game from the tables. I did not investigate if one engine may have benefit more from that, than the other.

Even 5 men tablebase add virtually zero elo, contrary to what people tend to think. However, they make some endgames look cleaner and more precise. But in most cases the result of the game is the same with or without. Cases where EGTB actually help are very rare, and to compensate for that benefit, they slow down the search and take a lot of RAM. Anyway, DiscoCheck does not use tablebases (except for a built in bitbase generated at runtime, for the KPK endgame). From what I see in the command line, Texel does not use EGTB either. So the impact of this is really zero (unless the GUI plays the endgame instead of the engines, when a 5 men or less is reached).

- In one game each, the engines stopped working for reasons unclear, thus losing on time. The match continued normally. No time losses otherwise.

I've played tens of thousands of games with DiscoCheck without a single crash, and I've never seen Texel crash either. I'm using cutechess-cli on linux: it's a nerdy command line tool, but it's very fast and a lot more reliable than any GUI. Also, it allows me to play games in parralel: since both engines are using one thread only, and I have 2 CPU, I can safely play 2 games in // all the time, and divide the time of testing by 2 (though if you use a laptop, I hope for your sake that you have a good fan).
As for time losses, cutechess-cli allows me to set a tolerence. I typically use sth like 100ms, accounting for delay in the GUI, from the system load (other apps running) or internal I/O and processing of commands, and also the time spent between two polls (DC looks at the clock only every 512 nodes searched). Above such a small margin, I consider that the engine is at fault, and it deserves to loose