EGTB value

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

wgarvin
Posts: 838
Joined: Thu Jul 05, 2007 5:03 pm
Location: British Columbia, Canada

Re: EGTB value - final test results

Post by wgarvin »

bob wrote:You only want to probe after a capture, nowhere else. As soon as you reach an EGTB position, you don't want to search further since you have perfect information. And if you are not currently in an EGTB position, the only way to get there is to capture something and take the position down to 3-4-5-6 pieces total, depending on which EGTBs you have.
Also after a promotion, which I guess chess programmers are used to thinking of as a "capture". :wink:

And if you had "partial" databases of some sort (e.g. lossy compressed bitbases that might return "I don't know" for a small percentage of positions) then you'd still need to probe after every move as long as you are currently "in" a database ending. Of course the only way to enter a database ending is via a capture or promotion, so you'd just store which database you're in as part of the position, and update it after each capture or promotion.
Jouni
Posts: 3283
Joined: Wed Mar 08, 2006 8:15 pm

Re: EGTB value - final test results

Post by Jouni »

I ran 300 games test with R3 TB vs Rybka 3 no TB. 5 piece TBs and 60/1' level, no GUI TB . Result:

R3 TB - R3 no TB 155 - 145

Funnily this is same +12 as some posted to Rybka forum! Maybe Rybka is getting more benefit than Crafty, because it's missing same basic knowledge as wrong bishop? But after browsing all games I don't find one game were tablebases have given point or half :? The biggest advantage
from bases is to get out of these games:

[Event "TB test"]
[Site "?"]
[Date "2010.10.06"]
[Round "5"]
[White "Rybka 3 B"]
[Black "Rybka 3"]
[Result "1/2-1/2"]
[ECO "D46"]
[Annotator "0.70;0.40"]
[PlyCount "270"]
[TimeControl "60/60:0/0:0/0"]

{Pentium(R) Dual-Core CPU E5300 @ 2.60GHz 0 MHz W=18.9 ply; 164kN/s;
Fritz10.ctg B=39.2 ply; 96kN/s; 65 259 TBAs; Fritz10.ctg} 1. d4 {B/0 0} Nf6 {
B/0 0} 2. c4 {B/0 0} e6 {B/0 0} 3. Nc3 {B/0 0} d5 {B/0 0} 4. Nf3 {B/0 0} c6 {
B/0 0} 5. e3 {B/0 0} Nbd7 {B/0 0} 6. Bd3 {B/0 0} Bd6 {B/0 0} 7. e4 {B/0 0} dxe4
{B/0 0} 8. Nxe4 {B/0 0} Nxe4 {B/0 0} 9. Bxe4 {B/0 0} e5 {B/0 0} 10. O-O {B/0 0}
exd4 {Both last book move B/0 0} 11. Qxd4 {0.70/11 2} Qf6 {0.40/12 2} 12. Rd1 {
0.60/11 2} Bc7 {(Qxd4) 0.19/12 15} 13. Qe3 {(Bc2) 0.48/10 2} O-O {0.16/11 1}
14. Re1 {0.48/10 2} Rd8 {(h6) 0.21/12 3} 15. Bc2 {0.43/12 3} Nf8 {0.21/11 1}
16. Qb3 {0.47/12 2} Ne6 {0.26/11 1} 17. Bd2 {0.44/11 1} b6 {0.22/11 2} 18. Qc3
{0.41/11 2} Qxc3 {0.17/11 1} 19. Bxc3 {0.39/12 1} h6 {0.18/12 2} 20. Be4 {
(h3) 0.37/11 2} Bb7 {0.21/10 1} 21. Bf5 {0.36/11 1} Nc5 {0.23/10 1} 22. g3 {
0.40/10 1} Kf8 {(Bd6) 0.21/10 1} 23. Bc2 {(b3) 0.40/11 1} Ne6 {0.28/10 1} 24.
Nh4 {0.39/11 1} Nd4 {(Rd7) 0.20/10 1} 25. Be4 {0.40/11 1} Rab8 {(Be5) 0.24/11 1
} 26. Kg2 {0.59/11 1} b5 {(Rd7) 0.31/11 1} 27. Nf3 {(Rad1) 0.52/10 1} Ne6 {
(Nxf3) 0.33/9 1} 28. cxb5 {(Rad1) 0.34/9 1} cxb5 {0.16/10 0} 29. Rac1 {
(Bxb7) 0.34/11 1} Bxe4 {0.13/10 1} 30. Rxe4 {0.36/12 2} Rbc8 {(a5) 0.15/11 3}
31. Nd4 {0.31/12 2} Nxd4 {0.15/12 0} 32. Bxd4 {(Rxd4) 0.32/13 1} Bd6 {0.17/10 0
} 33. Rd1 {(Rxc8) 0.31/12 1} a6 {0.13/11 1} 34. Bb6 {0.33/12 1} Rd7 {0.15/11 0}
35. Re2 {0.36/11 0} Rc6 {(f6) 0.14/11 1} 36. Ba5 {(Be3) 0.34/12 1} Re7 {
0.20/12 1} 37. Red2 {0.33/11 1} Re6 {0.20/12 1} 38. f4 {0.27/11 2} f5 {
0.09/11 1} 39. Kf3 {(h4) 0.27/11 1} Ke7 {0.13/10 0} 40. h4 {0.29/11 1} g6 {
0.10/12 0} 41. Bc3 {0.31/11 0} Kf7 {0.08/12 1} 42. Rd5 {(Rd3) 0.31/12 2} Bc5 {
0.18/11 1} 43. R1d3 {(Be5) 0.32/12 0} Red6 {0.21/11 1} 44. h5 {0.28/13 1} Rxd5
{0.21/12 0} 45. Rxd5 {0.28/14 0} Ke6 {(Be7) 0.21/12 1} 46. Re5+ {0.28/12 0} Kf7
{0.21/11 0} 47. hxg6+ {0.28/13 1} Kxg6 {0.21/11 0} 48. g4 {(Rd5) 0.37/13 1}
fxg4+ {0.18/11 0} 49. Kxg4 {0.37/12 0} b4 {0.23/12 1} 50. Be1 {(Bd2) 0.42/13 1}
Bd4 {(Kf7) 0.17/12 0} 51. f5+ {0.36/13 1} Kf7 {0.35/12 1} 52. Re4 {0.29/13 1}
Bxb2 {0.22/12 0} 53. Rxb4 {0.21/13 1} Bf6 {0.20/12 0} 54. Ra4 {(Rb7+) 0.22/13 2
} Be5 {0.21/12 1} 55. Bf2 {0.22/12 1} Bc7 {(Bf6) 0.19/11 1} 56. Rb4 {
(Re4) 0.37/12 1} Rc2 {0.33/11 1} 57. Bh4 {0.34/13 1} Be5 {0.21/12 1} 58. a4 {
0.22/12 2} a5 {0.20/12 1} 59. Re4 {(Rb7+) 0.17/11 1} Bd6 {0.15/13 1} 60. Bd8 {
(Rd4) 0.17/10 0} Rg2+ {(Bb4) 0.15/11 0} 61. Kh5 {0.12/15 2} Rh2+ {0.11/14 2}
62. Rh4 {0.16/16 2} Rd2 {0.09/14 2} 63. Bxa5 {0.15/16 2} Rd5 {0.07/15 1} 64.
Be1 {0.15/15 1} Rxf5+ {0.09/14 2} 65. Kg4 {0.15/15 2} Re5 {(Rf4+) 0.06/15 6}
66. Rxh6 {0.21/14 1} Re4+ {(Be7) 0.00/15 1} 67. Kf3 {0.30/14 1} Rxe1 {
(Rf4+) 0.00/16 1} 68. Rxd6 {0.62/14 1} Rf1+ {(Ke7) 0.00/59 0} 69. Ke3 {
(Ke4) 0.72/17 2} Rh1 {(Ke7) 0.00/59 0} 70. Ra6 {(a5) 0.79/15 1} Ra1 {
(Ke7) 0.00/59 0} 71. Kd4 {(a5) 1.29/18 3} Ra3 {(Rc1) 0.00/59 0} 72. a5 {
1.58/15 1} Ra4+ {(Ke7) 0.00/59 0} 73. Kd5 {(Ke5) 2.09/15 2} Ke7 {0.00/59 0} 74.
Ra7+ {2.09/18 1} Kd8 {0.00/59 0} 75. Kc6 {1.58/19 3} Rc4+ {0.00/59 0} 76. Kb6 {
(Kb5) 1.58/19 2} Rb4+ {0.00/59 0} 77. Ka6 {(Kc5) 1.58/21 2} Kc8 {0.00/59 0} 78.
Rh7 {(Re7) 1.58/21 1} Kb8 {0.00/59 0} 79. Rh8+ {1.58/22 1} Kc7 {0.00/59 0} 80.
Ka7 {1.58/22 1} Rb1 {(Kc6) 0.00/59 0} 81. a6 {1.73/18 1} Ra1 {(Rd1) 0.00/59 0}
82. Rh2 {(Rh6) 1.73/18 1} Ra4 {(Rc1) 0.00/59 0} 83. Rc2+ {2.95/16 1} Kd7 {
0.00/59 0} 84. Kb6 {2.95/18 1} Rb4+ {0.00/59 0} 85. Ka5 {2.95/19 0} Rb3 {
(Rb1) 0.00/59 0} 86. a7 {2.95/18 1} Ra3+ {0.00/59 0} 87. Kb6 {2.95/20 1} Rb3+ {
0.00/59 0} 88. Kc5 {2.95/21 0} Ra3 {0.00/59 0} 89. Rf2 {2.95/21 1} Kc8 {
(Kc7) 0.00/59 0} 90. Rf7 {2.95/20 0} Ra2 {(Ra6) 0.00/59 0} 91. Kb5 {
(Re7) 2.95/21 1} Ra1 {(Rb2+) 0.00/59 0} 92. Kb6 {2.95/22 1} Rb1+ {0.00/59 0}
93. Ka5 {(Kc6) 2.95/23 0} Ra1+ {0.00/59 0} 94. Kb5 {2.95/25 1} Rb1+ {
(Ra3) 0.00/59 0} 95. Kc6 {2.95/24 0} Rc1+ {0.00/59 0} 96. Kd5 {2.95/24 1} Rd1+
{(Ra1) 0.00/59 0} 97. Kc4 {(Kc5) 2.95/23 1} Ra1 {0.00/59 0} 98. Rg7 {
(Kb5) 2.95/24 1} Ra6 {(Ra2) 0.00/59 0} 99. Kc5 {(Kb5) 2.95/23 0} Ra3 {0.00/59 0
} 100. Kb6 {2.95/23 0} Rb3+ {0.00/59 0} 101. Kc6 {2.95/25 1} Rc3+ {0.00/59 0}
102. Kd6 {2.95/25 1} Rd3+ {0.00/59 0} 103. Ke5 {2.95/25 1} Ra3 {0.00/59 0} 104.
Rf7 {(Re7) 2.95/24 0} Ra4 {(Ra5+) 0.00/59 0} 105. Kd6 {2.95/24 1} Ra6+ {
0.00/59 0} 106. Kd5 {2.95/26 1} Ra1 {(Ra2) 0.00/59 0} 107. Rh7 {2.95/25 1} Ra3
{(Ra5+) 0.00/59 0} 108. Kc6 {(Kc4) 2.95/24 1} Ra6+ {(Rc3+) 0.00/59 0} 109. Kc5
{(Kb5) 2.95/25 0} Ra2 {(Ra3) 0.00/59 0} 110. Rf7 {(Re7) 2.95/24 1} Rc2+ {
0.00/59 0} 111. Kd6 {(Kb4) 2.95/24 1} Rd2+ {0.00/59 0} 112. Kc6 {
(Ke5) 2.95/26 1} Rc2+ {0.00/59 0} 113. Kd5 {(Kb6) 2.95/27 1} Ra2 {
(Rd2+) 0.00/59 0} 114. Rg7 {(Re7) 2.95/26 2} Ra1 {(Rd2+) 0.00/59 0} 115. Ke6 {
(Kd4) 2.95/25 1} Ra6+ {(Ra3) 0.00/59 0} 116. Kf5 {2.95/26 1} Ra5+ {
(Ra4) 0.00/59 0} 117. Ke4 {2.95/25 1} Ra6 {(Ra4+) 0.00/59 0} 118. Rh7 {
2.95/25 2} Ra1 {(Ra3) 0.00/59 0} 119. Kd4 {(Ke5) 2.95/25 2} Rd1+ {
(Ra2) 0.00/59 0} 120. Kc5 {2.95/25 1} Rc1+ {(Ra1) 0.00/59 0} 121. Kb6 {
(Kd5) 2.95/27 2} Rb1+ {0.00/59 0} 122. Ka6 {2.95/28 1} Ra1+ {0.00/59 0} 123.
Kb5 {2.95/29 1} Ra2 {0.00/59 0} 124. Kc4 {(Kb6) 2.95/27 1} Ra1 {0.00/59 0} 125.
Rf7 {(Kd4) 2.95/27 1} Rc1+ {(Ra5) 0.00/59 0} 126. Kb3 {(Kb4) 2.95/26 1} Ra1 {
0.00/59 0} 127. Kb4 {(Rh7) 2.95/26 1} Ra2 {0.00/59 0} 128. Rh7 {2.95/27 2} Ra1
{0.00/59 0} 129. Rg7 {(Kb3) 2.95/27 1} Rb1+ {0.00/59 0} 130. Ka3 {
(Ka5) 2.95/27 1} Ra1+ {0.00/59 0} 131. Kb3 {(Kb4) 2.95/28 2} Ra5 {
(Ra6) 0.00/59 0} 132. Kb4 {(Rh7) 2.95/27 2} Ra2 {(Ra6) 0.00/59 0} 133. Kb5 {
2.95/26 2} Ra1 {(Rb2+) 0.00/59 0} 134. Kc6 {(Re7) 0.00/23 14} Rc1+ {0.00/59 0}
135. Kd6 {(Kb6) 0.00/32 4} Rd1+ {Draw accepted 0.00/59 0} 1/2-1/2

Jouni
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: EGTB value - final test results

Post by bob »

wgarvin wrote:
bob wrote:You only want to probe after a capture, nowhere else. As soon as you reach an EGTB position, you don't want to search further since you have perfect information. And if you are not currently in an EGTB position, the only way to get there is to capture something and take the position down to 3-4-5-6 pieces total, depending on which EGTBs you have.
Also after a promotion, which I guess chess programmers are used to thinking of as a "capture". :wink:

And if you had "partial" databases of some sort (e.g. lossy compressed bitbases that might return "I don't know" for a small percentage of positions) then you'd still need to probe after every move as long as you are currently "in" a database ending. Of course the only way to enter a database ending is via a capture or promotion, so you'd just store which database you're in as part of the position, and update it after each capture or promotion.
For normal EGTB positions, you don't need to search after a promotion, assuming say that in a KRPKR position the pawn can promote to a queen. The KRPKR will already have the correct distance to mate including the promotion, any resulting captures, etc. If you don't have KRPKR but do have KRQKR, KRRKR, etc, then you would need to probe after a promotion. I would hope nobody would omit KRPKR and have the others.
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: EGTB value - final test results

Post by bob »

Jouni wrote:I ran 300 games test with R3 TB vs Rybka 3 no TB. 5 piece TBs and 60/1' level, no GUI TB . Result:

R3 TB - R3 no TB 155 - 145

Funnily this is same +12 as some posted to Rybka forum! Maybe Rybka is getting more benefit than Crafty, because it's missing same basic knowledge as wrong bishop? But after browsing all games I don't find one game were tablebases have given point or half :? The biggest advantage
from bases is to get out of these games:

[Event "TB test"]
[Site "?"]
[Date "2010.10.06"]
[Round "5"]
[White "Rybka 3 B"]
[Black "Rybka 3"]
[Result "1/2-1/2"]
[ECO "D46"]
[Annotator "0.70;0.40"]
[PlyCount "270"]
[TimeControl "60/60:0/0:0/0"]

{Pentium(R) Dual-Core CPU E5300 @ 2.60GHz 0 MHz W=18.9 ply; 164kN/s;
Fritz10.ctg B=39.2 ply; 96kN/s; 65 259 TBAs; Fritz10.ctg} 1. d4 {B/0 0} Nf6 {
B/0 0} 2. c4 {B/0 0} e6 {B/0 0} 3. Nc3 {B/0 0} d5 {B/0 0} 4. Nf3 {B/0 0} c6 {
B/0 0} 5. e3 {B/0 0} Nbd7 {B/0 0} 6. Bd3 {B/0 0} Bd6 {B/0 0} 7. e4 {B/0 0} dxe4
{B/0 0} 8. Nxe4 {B/0 0} Nxe4 {B/0 0} 9. Bxe4 {B/0 0} e5 {B/0 0} 10. O-O {B/0 0}
exd4 {Both last book move B/0 0} 11. Qxd4 {0.70/11 2} Qf6 {0.40/12 2} 12. Rd1 {
0.60/11 2} Bc7 {(Qxd4) 0.19/12 15} 13. Qe3 {(Bc2) 0.48/10 2} O-O {0.16/11 1}
14. Re1 {0.48/10 2} Rd8 {(h6) 0.21/12 3} 15. Bc2 {0.43/12 3} Nf8 {0.21/11 1}
16. Qb3 {0.47/12 2} Ne6 {0.26/11 1} 17. Bd2 {0.44/11 1} b6 {0.22/11 2} 18. Qc3
{0.41/11 2} Qxc3 {0.17/11 1} 19. Bxc3 {0.39/12 1} h6 {0.18/12 2} 20. Be4 {
(h3) 0.37/11 2} Bb7 {0.21/10 1} 21. Bf5 {0.36/11 1} Nc5 {0.23/10 1} 22. g3 {
0.40/10 1} Kf8 {(Bd6) 0.21/10 1} 23. Bc2 {(b3) 0.40/11 1} Ne6 {0.28/10 1} 24.
Nh4 {0.39/11 1} Nd4 {(Rd7) 0.20/10 1} 25. Be4 {0.40/11 1} Rab8 {(Be5) 0.24/11 1
} 26. Kg2 {0.59/11 1} b5 {(Rd7) 0.31/11 1} 27. Nf3 {(Rad1) 0.52/10 1} Ne6 {
(Nxf3) 0.33/9 1} 28. cxb5 {(Rad1) 0.34/9 1} cxb5 {0.16/10 0} 29. Rac1 {
(Bxb7) 0.34/11 1} Bxe4 {0.13/10 1} 30. Rxe4 {0.36/12 2} Rbc8 {(a5) 0.15/11 3}
31. Nd4 {0.31/12 2} Nxd4 {0.15/12 0} 32. Bxd4 {(Rxd4) 0.32/13 1} Bd6 {0.17/10 0
} 33. Rd1 {(Rxc8) 0.31/12 1} a6 {0.13/11 1} 34. Bb6 {0.33/12 1} Rd7 {0.15/11 0}
35. Re2 {0.36/11 0} Rc6 {(f6) 0.14/11 1} 36. Ba5 {(Be3) 0.34/12 1} Re7 {
0.20/12 1} 37. Red2 {0.33/11 1} Re6 {0.20/12 1} 38. f4 {0.27/11 2} f5 {
0.09/11 1} 39. Kf3 {(h4) 0.27/11 1} Ke7 {0.13/10 0} 40. h4 {0.29/11 1} g6 {
0.10/12 0} 41. Bc3 {0.31/11 0} Kf7 {0.08/12 1} 42. Rd5 {(Rd3) 0.31/12 2} Bc5 {
0.18/11 1} 43. R1d3 {(Be5) 0.32/12 0} Red6 {0.21/11 1} 44. h5 {0.28/13 1} Rxd5
{0.21/12 0} 45. Rxd5 {0.28/14 0} Ke6 {(Be7) 0.21/12 1} 46. Re5+ {0.28/12 0} Kf7
{0.21/11 0} 47. hxg6+ {0.28/13 1} Kxg6 {0.21/11 0} 48. g4 {(Rd5) 0.37/13 1}
fxg4+ {0.18/11 0} 49. Kxg4 {0.37/12 0} b4 {0.23/12 1} 50. Be1 {(Bd2) 0.42/13 1}
Bd4 {(Kf7) 0.17/12 0} 51. f5+ {0.36/13 1} Kf7 {0.35/12 1} 52. Re4 {0.29/13 1}
Bxb2 {0.22/12 0} 53. Rxb4 {0.21/13 1} Bf6 {0.20/12 0} 54. Ra4 {(Rb7+) 0.22/13 2
} Be5 {0.21/12 1} 55. Bf2 {0.22/12 1} Bc7 {(Bf6) 0.19/11 1} 56. Rb4 {
(Re4) 0.37/12 1} Rc2 {0.33/11 1} 57. Bh4 {0.34/13 1} Be5 {0.21/12 1} 58. a4 {
0.22/12 2} a5 {0.20/12 1} 59. Re4 {(Rb7+) 0.17/11 1} Bd6 {0.15/13 1} 60. Bd8 {
(Rd4) 0.17/10 0} Rg2+ {(Bb4) 0.15/11 0} 61. Kh5 {0.12/15 2} Rh2+ {0.11/14 2}
62. Rh4 {0.16/16 2} Rd2 {0.09/14 2} 63. Bxa5 {0.15/16 2} Rd5 {0.07/15 1} 64.
Be1 {0.15/15 1} Rxf5+ {0.09/14 2} 65. Kg4 {0.15/15 2} Re5 {(Rf4+) 0.06/15 6}
66. Rxh6 {0.21/14 1} Re4+ {(Be7) 0.00/15 1} 67. Kf3 {0.30/14 1} Rxe1 {
(Rf4+) 0.00/16 1} 68. Rxd6 {0.62/14 1} Rf1+ {(Ke7) 0.00/59 0} 69. Ke3 {
(Ke4) 0.72/17 2} Rh1 {(Ke7) 0.00/59 0} 70. Ra6 {(a5) 0.79/15 1} Ra1 {
(Ke7) 0.00/59 0} 71. Kd4 {(a5) 1.29/18 3} Ra3 {(Rc1) 0.00/59 0} 72. a5 {
1.58/15 1} Ra4+ {(Ke7) 0.00/59 0} 73. Kd5 {(Ke5) 2.09/15 2} Ke7 {0.00/59 0} 74.
Ra7+ {2.09/18 1} Kd8 {0.00/59 0} 75. Kc6 {1.58/19 3} Rc4+ {0.00/59 0} 76. Kb6 {
(Kb5) 1.58/19 2} Rb4+ {0.00/59 0} 77. Ka6 {(Kc5) 1.58/21 2} Kc8 {0.00/59 0} 78.
Rh7 {(Re7) 1.58/21 1} Kb8 {0.00/59 0} 79. Rh8+ {1.58/22 1} Kc7 {0.00/59 0} 80.
Ka7 {1.58/22 1} Rb1 {(Kc6) 0.00/59 0} 81. a6 {1.73/18 1} Ra1 {(Rd1) 0.00/59 0}
82. Rh2 {(Rh6) 1.73/18 1} Ra4 {(Rc1) 0.00/59 0} 83. Rc2+ {2.95/16 1} Kd7 {
0.00/59 0} 84. Kb6 {2.95/18 1} Rb4+ {0.00/59 0} 85. Ka5 {2.95/19 0} Rb3 {
(Rb1) 0.00/59 0} 86. a7 {2.95/18 1} Ra3+ {0.00/59 0} 87. Kb6 {2.95/20 1} Rb3+ {
0.00/59 0} 88. Kc5 {2.95/21 0} Ra3 {0.00/59 0} 89. Rf2 {2.95/21 1} Kc8 {
(Kc7) 0.00/59 0} 90. Rf7 {2.95/20 0} Ra2 {(Ra6) 0.00/59 0} 91. Kb5 {
(Re7) 2.95/21 1} Ra1 {(Rb2+) 0.00/59 0} 92. Kb6 {2.95/22 1} Rb1+ {0.00/59 0}
93. Ka5 {(Kc6) 2.95/23 0} Ra1+ {0.00/59 0} 94. Kb5 {2.95/25 1} Rb1+ {
(Ra3) 0.00/59 0} 95. Kc6 {2.95/24 0} Rc1+ {0.00/59 0} 96. Kd5 {2.95/24 1} Rd1+
{(Ra1) 0.00/59 0} 97. Kc4 {(Kc5) 2.95/23 1} Ra1 {0.00/59 0} 98. Rg7 {
(Kb5) 2.95/24 1} Ra6 {(Ra2) 0.00/59 0} 99. Kc5 {(Kb5) 2.95/23 0} Ra3 {0.00/59 0
} 100. Kb6 {2.95/23 0} Rb3+ {0.00/59 0} 101. Kc6 {2.95/25 1} Rc3+ {0.00/59 0}
102. Kd6 {2.95/25 1} Rd3+ {0.00/59 0} 103. Ke5 {2.95/25 1} Ra3 {0.00/59 0} 104.
Rf7 {(Re7) 2.95/24 0} Ra4 {(Ra5+) 0.00/59 0} 105. Kd6 {2.95/24 1} Ra6+ {
0.00/59 0} 106. Kd5 {2.95/26 1} Ra1 {(Ra2) 0.00/59 0} 107. Rh7 {2.95/25 1} Ra3
{(Ra5+) 0.00/59 0} 108. Kc6 {(Kc4) 2.95/24 1} Ra6+ {(Rc3+) 0.00/59 0} 109. Kc5
{(Kb5) 2.95/25 0} Ra2 {(Ra3) 0.00/59 0} 110. Rf7 {(Re7) 2.95/24 1} Rc2+ {
0.00/59 0} 111. Kd6 {(Kb4) 2.95/24 1} Rd2+ {0.00/59 0} 112. Kc6 {
(Ke5) 2.95/26 1} Rc2+ {0.00/59 0} 113. Kd5 {(Kb6) 2.95/27 1} Ra2 {
(Rd2+) 0.00/59 0} 114. Rg7 {(Re7) 2.95/26 2} Ra1 {(Rd2+) 0.00/59 0} 115. Ke6 {
(Kd4) 2.95/25 1} Ra6+ {(Ra3) 0.00/59 0} 116. Kf5 {2.95/26 1} Ra5+ {
(Ra4) 0.00/59 0} 117. Ke4 {2.95/25 1} Ra6 {(Ra4+) 0.00/59 0} 118. Rh7 {
2.95/25 2} Ra1 {(Ra3) 0.00/59 0} 119. Kd4 {(Ke5) 2.95/25 2} Rd1+ {
(Ra2) 0.00/59 0} 120. Kc5 {2.95/25 1} Rc1+ {(Ra1) 0.00/59 0} 121. Kb6 {
(Kd5) 2.95/27 2} Rb1+ {0.00/59 0} 122. Ka6 {2.95/28 1} Ra1+ {0.00/59 0} 123.
Kb5 {2.95/29 1} Ra2 {0.00/59 0} 124. Kc4 {(Kb6) 2.95/27 1} Ra1 {0.00/59 0} 125.
Rf7 {(Kd4) 2.95/27 1} Rc1+ {(Ra5) 0.00/59 0} 126. Kb3 {(Kb4) 2.95/26 1} Ra1 {
0.00/59 0} 127. Kb4 {(Rh7) 2.95/26 1} Ra2 {0.00/59 0} 128. Rh7 {2.95/27 2} Ra1
{0.00/59 0} 129. Rg7 {(Kb3) 2.95/27 1} Rb1+ {0.00/59 0} 130. Ka3 {
(Ka5) 2.95/27 1} Ra1+ {0.00/59 0} 131. Kb3 {(Kb4) 2.95/28 2} Ra5 {
(Ra6) 0.00/59 0} 132. Kb4 {(Rh7) 2.95/27 2} Ra2 {(Ra6) 0.00/59 0} 133. Kb5 {
2.95/26 2} Ra1 {(Rb2+) 0.00/59 0} 134. Kc6 {(Re7) 0.00/23 14} Rc1+ {0.00/59 0}
135. Kd6 {(Kb6) 0.00/32 4} Rd1+ {Draw accepted 0.00/59 0} 1/2-1/2

Jouni
I think that result is heavily weighted by pure variance. The error bar is significant for 300 games...
ernest
Posts: 2041
Joined: Wed Mar 08, 2006 8:30 pm

Re: EGTB value - final test results

Post by ernest »

Jouni wrote:I ran 300 games test with R3 TB vs Rybka 3 no TB. 5 piece TBs and 60/1' level, no GUI TB
Hi Jouni,
Interesting, but if you have a moment, could you run it again, in order to get one more point in the error bar?

Also remember R3 has an EGTB bug when running multicore (this bug can be eliminated by a manual trick: see the Rybka Forum)
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: EGTB value - final test results

Post by hgm »

A score of 1.67% above equality is not even a single standard deviation in 300 games (which is 2.3%). When you play equal engines against each other, you would already more often than not have an outcome with a larger difference. So this shows absolutely nothing.
Jouni
Posts: 3283
Joined: Wed Mar 08, 2006 8:15 pm

Re: EGTB value - final test results

Post by Jouni »

I played 200 more games and total result was:

R3 TB - R3 no TB 262 - 238 e.g. +17 points

Of course 500 games is nothing, but note:

1) I used dual Rybka with infamous TB bug
2) TB version won ALL 5 100 games match with 53 50.5 51.5 52.5 54.5

Jouni
User avatar
hgm
Posts: 27790
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: EGTB value - final test results

Post by hgm »

The latter does not contain any information. If conditions were the same, only the total contains information on the winning probablility.

12 extra points out of 500 (so 2.4%) is a bit more significant than the previous result, as the Standard Deviation is now 1.8%.

In fact the detail results are a bit suspect. The standard deviation in 100 games should be 4 points. That means that typically in 32% of such runs the result should deviate from the average by more than 4 points, i.e. be below 48.5 or above 56.5. But in fact all 5 runs are within 2 points from the average. This has a very low probability.

Of course it is not really a miracle if some programs would benefit where others would not. I am sure that micro-Max would benefit a lot from EGTB, as it is totally stupid, and thinks that KNK is a lot better than KPPK. EGTB would enlighten it in this respect. And I am equally sure Crafty contains a lot of end-game knowledge in its evaluation. So the conclusion of Bob's tests should not be that EGTB can never do any good. Just that coding the same knowledge by special purpose code outperforms storng it in huge tables.
jwes
Posts: 778
Joined: Sat Jul 01, 2006 7:11 am

Re: EGTB value - final test results

Post by jwes »

hgm wrote:The latter does not contain any information. If conditions were the same, only the total contains information on the winning probablility.
I was wondering about that. It seems that the probability of one of two equal programs winning 5 short matches would be 3%. I would think that by running several short tests, you would decrease false positives with a corresponding increase of false negatives.
Milos
Posts: 4190
Joined: Wed Nov 25, 2009 1:47 am

Re: EGTB value - final test results

Post by Milos »

jwes wrote:I was wondering about that. It seems that the probability of one of two equal programs winning 5 short matches would be 3%. I would think that by running several short tests, you would decrease false positives with a corresponding increase of false negatives.
Nope.
If you run 100 short matches of 5 games, 5 matches of 100 games or 500 games at once the total variance is absolutely the same.