Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Vinvin
Posts: 4289
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Vinvin » Wed Aug 14, 2019 10:55 am

Uri Blass wrote:
Wed Aug 14, 2019 4:42 am
I think that the point was to play against the top programs of their time.

Rebel6 was a top program in 1994
Rebel12 was not a top program when it appeared.
Yes, good point !
In 1994-1995, Rebel 6 was on the top of the SSDF list, just after Chess Genius, the number 1.
1994/7 : https://groups.google.com/forum/#!msg/r ... UJwgSmzRoJ
1995/6 : https://groups.google.com/forum/#!topic ... GRnYtzTth4

Thanks !

Vinvin
Posts: 4289
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Vinvin » Wed Aug 14, 2019 11:39 am

mwyoung wrote:
Sat Mar 29, 2014 4:21 pm
sainzlei wrote:Rebel 6 will be lost 100 times in a row .
This look like not a good idea math between both engine .

:shock:
It may lose 100 - 0 even with a x3 time advantage. The match is not about the math. Anyone can look at rating numbers. I had a request of how do engines of today play better, from the ones in the past. One fun way to show this is to let them play.

So we are going to step threw time in a series of matches with the older engine always having a x3 time advantage, and both engines playing one core for consistency.

Rebel 6 (1994)
Fritz 6 (2000)
Fruit 2 (2005-7)
Rybka 3 (2008)
Houdini 1.5a (2010-11)
The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit

User avatar
Guenther
Posts: 2935
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Guenther » Wed Aug 14, 2019 12:04 pm

Vinvin wrote:
Wed Aug 14, 2019 11:39 am
mwyoung wrote:
Sat Mar 29, 2014 4:21 pm
sainzlei wrote:Rebel 6 will be lost 100 times in a row .
This look like not a good idea math between both engine .

:shock:
It may lose 100 - 0 even with a x3 time advantage. The match is not about the math. Anyone can look at rating numbers. I had a request of how do engines of today play better, from the ones in the past. One fun way to show this is to let them play.

So we are going to step threw time in a series of matches with the older engine always having a x3 time advantage, and both engines playing one core for consistency.

Rebel 6 (1994)
Fritz 6 (2000)
Fruit 2 (2005-7)
Rybka 3 (2008)
Houdini 1.5a (2010-11)
The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit
Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.
Current foe list count : [92 - still rising]
http://rwbc-chess.de/chronology.htm

Vinvin
Posts: 4289
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Vinvin » Wed Aug 14, 2019 12:09 pm

Guenther wrote:
Wed Aug 14, 2019 12:04 pm
Vinvin wrote:
Wed Aug 14, 2019 11:39 am
mwyoung wrote:
Sat Mar 29, 2014 4:21 pm
sainzlei wrote:Rebel 6 will be lost 100 times in a row .
This look like not a good idea math between both engine .

:shock:
It may lose 100 - 0 even with a x3 time advantage. The match is not about the math. Anyone can look at rating numbers. I had a request of how do engines of today play better, from the ones in the past. One fun way to show this is to let them play.

So we are going to step threw time in a series of matches with the older engine always having a x3 time advantage, and both engines playing one core for consistency.

Rebel 6 (1994)
Fritz 6 (2000)
Fruit 2 (2005-7)
Rybka 3 (2008)
Houdini 1.5a (2010-11)
The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit
Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.
I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ? :-)

User avatar
Guenther
Posts: 2935
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Guenther » Wed Aug 14, 2019 1:39 pm

Vinvin wrote:
The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit
Guenther wrote:
Wed Aug 14, 2019 12:04 pm
Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.
Vinvin wrote: I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ? :-)
Are we talking about the same games, the CCRL games Houdini 1.5a-64 vs. SF 10-64?
From my experience any perf for more than 40/50 games outside ~170 err level might have a problem
and here it says the diff is 266.
Below is game one. 359 seconds for depth 22 looks already somehow strange in move 14.

[Event "CCRL 40/40"]
[Site "CCRL"]
[Date "2019.05.06"]
[Round "666.1.622"]
[White "Houdini 1.5a 64-bit"]
[Black "Stockfish 10 64-bit"]
[Result "0-1"]
[WhiteElo "3110"]
[BlackElo "3387"]
[ECO "E00"]
[Opening "Catalan opening"]
[PlyCount "197"]

1. d4 {book} Nf6 {book}
2. c4 {book} e6 {book}
3. g3 {book} c5 {book}
4. Nf3 {book} cxd4 {book}
5. Nxd4 {book} d5 {book}
6. Bg2 {book} e5 {book}
7. Nf3 {book} d4 {book}
8. O-O {book} Nc6 {book}
9. e3 {book} Be7 {book}
10. exd4 {book} exd4 {book}
11. Re1 {+0.00/19 87} O-O {+0.09/25 58}
12. Bf4 {+0.04/19 74} Nh5 {+0.07/29 51}
13. Be5 {+0.06/20 89} Bb4 {+0.00/27 27}
14. Nbd2 {+0.15/22 359} f6 {+0.69/29 21}
15. Bxd4 {+0.23/21 94} Nxd4 {-0.13/35 79}
16. Nxd4 {+0.24/20 0} Qxd4 {+0.10/33 21}
17. Qxh5 {+0.25/22 38} Qxd2 {+0.00/35 69}
18. a3 {+0.25/21 0} Qxe1+ {+0.00/32 40}
19. Rxe1 {+0.27/20 5} Bxe1 {+0.00/34 28}
20. Be4 {+0.21/24 88} h6 {+0.00/35 26}
21. Qd1 {+0.41/21 40} Bxf2+ {+0.00/36 40}
22. Kxf2 {+0.42/19 0} Be6 {+0.00/37 68}
23. Bxb7 {+0.24/20 43} Rab8 {+0.00/34 32}
24. Qe2 {+0.23/23 47} Bf7 {+0.00/35 32}
25. Be4 {+0.36/23 85} Rfd8 {+0.00/35 31}
26. b4 {+0.19/22 54} Rd4 {+0.00/35 38}
27. Bd5 {+0.12/23 141} Bxd5 {+0.00/34 42}
28. cxd5 {+0.13/22 0} Rd8 {+0.00/34 39}
29. Qa6 {-0.03/24 126} Rd2+ {+0.00/37 101}
30. Kf3 {-0.03/24 41} R8xd5 {+0.00/34 32}
31. Qxa7 {-0.03/24 56} Kh7 {+0.00/34 67}
32. Qa6 {-0.03/22 37} h5 {+0.00/32 48}
33. a4 {-0.03/23 54} Rxh2 {+0.00/32 43}
34. Ke4 {+0.03/20 52} Rg5 {+0.00/34 149}
35. Kf3 {-0.02/21 37} h4 {+0.00/35 44}
36. gxh4 {-0.07/23 49} Rh3+ {+0.00/36 70}
37. Kf2 {-0.07/25 54} Rg4 {+0.00/37 53}
38. b5 {-0.07/22 52} Rb3 {+0.00/38 71}
39. Qa8 {-0.02/22 78} Rc4 {+0.14/35 123}
40. Ke2 {-0.07/23 26} Rxh4 {+0.03/40 188}
41. Kf2 {-0.07/24 61} Rh2+ {+1.30/33 72}
42. Kg1 {-0.07/23 6} Rd2 {+0.21/36 89}
43. Qh1+ {-0.07/25 40} Kg6 {+0.00/40 76}
44. Qe4+ {-0.07/23 0} Kg5 {+0.00/41 36}
45. Qe1 {-0.07/25 51} Rdb2 {+0.83/31 33}
46. Qc1+ {-0.07/24 41} Kg6 {+0.87/32 34}
47. Qf4 {-0.07/25 43} Rb4 {+0.77/34 87}
48. Qg3+ {-0.07/23 0} Kh5 {+0.80/36 49}
49. Qh3+ {-0.07/25 57} Kg5 {+0.72/35 65}
50. Qd7 {-0.07/25 74} Kh4 {+0.50/34 62}
51. a5 {-0.18/23 93} Rg4+ {+0.43/34 89}
52. Kf1 {-0.18/22 0} Rc4 {+0.73/35 112}
53. Qd1 {-0.19/21 0} Rxb5 {+0.77/32 23}
54. Qe1+ {-0.07/23 58} Kg5 {+0.84/31 31}
55. Qd2+ {-0.07/24 86} Kg6 {+0.84/35 41}
56. a6 {-0.07/23 57} Ra4 {+0.90/30 24}
57. Qd3+ {-0.13/24 46} Rf5+ {+1.21/32 26}
58. Ke2 {-0.14/26 75} Ra5 {+1.25/33 48}
59. Qc4 {-0.21/25 81} Rfe5+ {+1.23/34 90}
60. Kf2 {-0.30/25 195} Rec5 {+1.25/36 47}
61. Qg4+ {-0.25/24 41} Kh7 {+1.25/35 16}
62. Qe4+ {-0.28/24 45} Rf5+ {+1.30/33 23}
63. Kg1 {-0.29/24 31} g6 {+1.94/32 21}
64. Qh4+ {-0.28/24 61} Kg7 {+2.27/31 24}
65. Qc4 {-0.28/25 31} Rfc5 {+3.14/31 22}
66. Qb3 {-0.35/25 150} Ra1+ {+3.65/34 22}
67. Kf2 {-1.07/25 54} Rc7 {+4.12/36 37}
68. Qb6 {-1.07/24 1} Re7 {+4.37/35 31}
69. Kf3 {-1.07/26 33} Ra3+ {+4.97/33 46}
70. Kg2 {-1.16/27 70} Ra2+ {+5.10/35 41}
71. Kg3 {-1.15/25 90} Ra4 {+3.81/38 117}
72. Kg2 {-1.07/25 30} Rae4 {+5.16/35 39}
73. Qb8 {-1.15/26 28} Ra4 {+5.20/38 29}
74. Qb6 {-0.07/25 0} Ra2+ {+5.22/37 22}
75. Kg3 {-0.07/34 27} g5 {+5.43/39 91}
76. Kf3 {-0.97/24 22} Ra3+ {+5.77/32 23}
77. Kf2 {-0.97/27 37} Rf7 {+6.07/36 89}
78. Ke2 {-0.97/25 33} Rd7 {+6.27/36 99}
79. Kf2 {-0.97/25 38} Rd2+ {+6.42/33 26}
80. Ke1 {-0.65/20 3} Rda2 {+6.76/36 106}
81. Qb7+ {-1.73/24 289} Kg6 {+6.76/36 38}
82. Qe4+ {-1.76/24 31} Kh5 {+6.76/40 45}
83. a7 {-1.77/24 64} f5 {+10.88/40 207}
84. Qxf5 {-1.83/23 33} Re3+ {+11.08/33 25}
85. Kd1 {-1.85/24 33} Rxa7 {+11.08/40 39}
86. Kd2 {-1.89/25 52} Rae7 {+11.08/35 41}
87. Qd5 {-1.95/23 36} Re2+ {+11.08/31 45}
88. Kc3 {-3.09/25 252} R7e3+ {+11.18/29 79}
89. Kc4 {-3.18/24 95} Re4+ {+11.27/32 68}
90. Kc3 {-3.33/25 201} Kh4 {+12.09/32 106}
91. Qd8 {-3.33/21 168} Re5 {+12.18/27 20}
92. Kd3 {-3.41/21 29} Kg3 {+12.56/31 25}
93. Qb6 {-3.49/21 23} Kg2 {+73.15/30 52}
94. Qg6 {-3.73/21 19} R2e4 {+152.99/29 34}
95. Qh6 {-4.71/22 138} g4 {+153.06/30 25}
96. Qf6 {-4.88/20 18} g3 {+153.07/33 21}
97.Qd6 {-5.96/20 110} Kg1 {+153.12/37 49}
98. Kc2 {-6.40/17 14} g2 {+153.15/29 30}
99. Kb3 {-9.60/17 80}
0-1
Current foe list count : [92 - still rising]
http://rwbc-chess.de/chronology.htm

Vinvin
Posts: 4289
Joined: Thu Mar 09, 2006 8:40 am
Full name: Vincent Lejeune

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Vinvin » Wed Aug 14, 2019 6:06 pm

Guenther wrote:
Wed Aug 14, 2019 1:39 pm
Vinvin wrote:
The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit
Guenther wrote:
Wed Aug 14, 2019 12:04 pm
Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.
Vinvin wrote: I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ? :-)
Are we talking about the same games, the CCRL games Houdini 1.5a-64 vs. SF 10-64?
From my experience any perf for more than 40/50 games outside ~170 err level might have a problem
and here it says the diff is 266.
Below is game one. 359 seconds for depth 22 looks already somehow strange in move 14.
...
The time control is 40 moves in 40 minutes, so it's not crazy to use 6 minutes for a move in the opening.

I just ran a tournament SF_190726 vs H1.5 and I got a similar results : only 9 draws for H1.5 in 128 games (96.5%-3.5%) :

Code: Select all

Score of Houdini_15_x64 vs stockfish_19072603_x64_modern: 0 - 119 - 9 [0.035]
ELO difference: -575.38 +/- 132.50

128 of 128 games finished.
Interface : Cute Chess
TC : 3m+2s
No EGTB
1 core on i7@4GHz
Opening book : NoomenShortLinesTestsuite.pgn

128 games in the zip file.
Attachments
H1.5 vs SF190726.zip
(133.68 KiB) Downloaded 5 times

User avatar
Guenther
Posts: 2935
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Retrospect Match: Rebel 6 (1994) - Stockfish (2014)

Post by Guenther » Wed Aug 14, 2019 6:36 pm

Vinvin wrote:
Wed Aug 14, 2019 6:06 pm
Guenther wrote:
Wed Aug 14, 2019 1:39 pm
Vinvin wrote:
The time when Houdini 1.5a is beaten more than 95% is probably already reached also :
A recent test on CCRL 40/40 : Houdini 1.5a 64-bit - Stockfish 10 64-bit : 1.5 − 50.5 (+0−49=3)

Link to the page : http://ccrl.chessdom.com/ccrl/4040/cgi/ ... _5a_64-bit
Guenther wrote:
Wed Aug 14, 2019 12:04 pm
Note that the result is far outside the expected performance.
Actually it should trigger some cross-check of the games and game environment.
Vinvin wrote: I followed some games during the live broadcast and everything seems OK to me.
May be @mwyoung can replay this experience ? :-)
Are we talking about the same games, the CCRL games Houdini 1.5a-64 vs. SF 10-64?
From my experience any perf for more than 40/50 games outside ~170 err level might have a problem
and here it says the diff is 266.
Below is game one. 359 seconds for depth 22 looks already somehow strange in move 14.
...
The time control is 40 moves in 40 minutes, so it's not crazy to use 6 minutes for a move in the opening.

But not with only reaching depth 22. Note that the real TC is 32/40 BTW. (I can deduct this with my stats tools)
I have already 2 draws after two games at 32/40 on my hardware which seems to be similar speed, as the used hardware...

The chance for a perf diff of 266 over 52 games is extremely unlikely...

You should be aware also that I have already found several times anomalies in CCRL matches and all those games had to be repeated
or cancelled. (all of those had err levels >150)

Stats for the first two games for Houdini:
(complete breakdown of depths in game 2 for Houdini in middlegame and exitus - each second or third game has that pattern...)

Code: Select all

game 1
+0.00/19 87;w;0-1
+0.04/19 74;w;0-1
+0.06/20 89;w;0-1
+0.15/22 359;w;0-1
+0.23/21 94;w;0-1
+0.24/20 0;w;0-1
+0.25/22 38;w;0-1
+0.25/21 0;w;0-1
+0.27/20 5;w;0-1
+0.21/24 88;w;0-1
+0.41/21 40;w;0-1
+0.42/19 0;w;0-1
+0.24/20 43;w;0-1
+0.23/23 47;w;0-1
+0.36/23 85;w;0-1
+0.19/22 54;w;0-1
+0.12/23 141;w;0-1
+0.13/22 0;w;0-1
-0.03/24 126;w;0-1
-0.03/24 41;w;0-1
-0.03/24 56;w;0-1
-0.03/22 37;w;0-1
-0.03/23 54;w;0-1
+0.03/20 52;w;0-1
-0.02/21 37;w;0-1
-0.07/23 49;w;0-1
-0.07/25 54;w;0-1
-0.07/22 52;w;0-1
-0.02/22 78;w;0-1
-0.07/23 26;w;0-1
-0.07/24 61;w;0-1
-0.07/23 6;w;0-1
-0.07/25 40;w;0-1
-0.07/23 0;w;0-1
-0.07/25 51;w;0-1
-0.07/24 41;w;0-1
-0.07/25 43;w;0-1
-0.07/23 0;w;0-1
-0.07/25 57;w;0-1
-0.07/25 74;w;0-1
-0.18/23 93;w;0-1
-0.18/22 0;w;0-1
-0.19/21 0;w;0-1
-0.07/23 58;w;0-1
-0.07/24 86;w;0-1
-0.07/23 57;w;0-1
-0.13/24 46;w;0-1
-0.14/26 75;w;0-1
-0.21/25 81;w;0-1
-0.30/25 195;w;0-1
-0.25/24 41;w;0-1
-0.28/24 45;w;0-1
-0.29/24 31;w;0-1
-0.28/24 61;w;0-1
-0.28/25 31;w;0-1
-0.35/25 150;w;0-1
-1.07/25 54;w;0-1
-1.07/24 1;w;0-1
-1.07/26 33;w;0-1
-1.16/27 70;w;0-1
-1.15/25 90;w;0-1
-1.07/25 30;w;0-1
-1.15/26 28;w;0-1
-0.07/25 0;w;0-1
-0.07/34 27;w;0-1
-0.97/24 22;w;0-1
-0.97/27 37;w;0-1
-0.97/25 33;w;0-1
-0.97/25 38;w;0-1
-0.65/20 3;w;0-1
-1.73/24 289;w;0-1
-1.76/24 31;w;0-1
-1.77/24 64;w;0-1
-1.83/23 33;w;0-1
-1.85/24 33;w;0-1
-1.89/25 52;w;0-1
-1.95/23 36;w;0-1
-3.09/25 252;w;0-1
-3.18/24 95;w;0-1
-3.33/25 201;w;0-1
-3.33/21 168;w;0-1
-3.41/21 29;w;0-1
-3.49/21 23;w;0-1
-3.73/21 19;w;0-1
-4.71/22 138;w;0-1
-4.88/20 18;w;0-1
-5.96/20 110;w;0-1
-6.40/17 14;w;0-1
-9.60/17 80;w;0-1
game 2
+0.02/20 141;b;1-0
-0.02/20 97;b;1-0
-0.06/19 97;b;1-0
-0.02/20 80;b;1-0
-0.07/20 58;b;1-0
-0.04/20 55;b;1-0
-0.16/18 48;b;1-0
-0.18/20 63;b;1-0
-0.17/18 0;b;1-0
-0.22/18 48;b;1-0
-0.22/19 62;b;1-0
-0.33/19 105;b;1-0
-0.27/20 240;b;1-0
-0.25/18 49;b;1-0
-0.44/19 46;b;1-0
-0.36/18 50;b;1-0
-0.44/19 156;b;1-0
-0.44/17 0;b;1-0
-0.69/19 140;b;1-0
-1.31/18 103;b;1-0
-1.41/17 55;b;1-0
-1.37/15 38;b;1-0
-1.39/16 21;b;1-0
-3.26/17 41;b;1-0
-3.58/17 18;b;1-0
-3.80/17 31;b;1-0
-4.30/16 12;b;1-0
-5.73/16 20;b;1-0
-6.97/16 23;b;1-0
-9.93/15 8;b;1-0
-11.48/17 99;b;1-0
Attachments
Houdini_1.5a_64-bit_comment.txt.7z
(8.56 KiB) Downloaded 4 times
extract.7z
(30.83 KiB) Downloaded 4 times
Current foe list count : [92 - still rising]
http://rwbc-chess.de/chronology.htm

Post Reply