Moderators: hgm, Rebel, chrisw
Code: Select all
[PLY=14] vs [PLY=15] 50-36.6=13.4% = 93 elo D14 = 64:12:24 101 games
D15 = 127:50:03
I would like to try this stealing some cluster time. But I am not sure how to replicate your conditions regarding the different search depths (extensions?) to simulate a timed control test while using fixed depth. My search behaviour is also probably different from Pro Deo's (different reductions/extensions) so there may not be much point to try and replicate the conditions. Just going to use fixed depths will probably show so magnified gains per ply at lower depths but I don't see much point fixing that by applying some extensions..Rebel wrote:I have finished my long lasting experiment on diminishing returns and I must say I am quite optimistic that within 10-15 years the 4000 elo barrier will be crossed.
http://www.top-5000.nl/ply.htm
Even at longer time controls such as the above last match (average 87 minutes for 1 game) ProDeo keeps profiting big time, 93 elo. Since only 100 games were played that number is not so clear.Code: Select all
[PLY=14] vs [PLY=15] 50-36.6=13.4% = 93 elo D14 = 64:12:24 101 games D15 = 127:50:03
Those with better hardware (Bob ?) could do a similar experiment to predict the elo future of computer chess.
Since the interface I was using (Arena) does not support unequal PLY matches I added some code first to run this experiment. The parameter [PLY = x] tells the engine to perform as normal but to move after x plies overruling whatever level the GUI is set. And so I can run unequal PLY matches under every GUI.Daniel Shawul wrote:I would like to try this stealing some cluster time. But I am not sure how to replicate your conditions regarding the different search depths (extensions?) to simulate a timed control test while using fixed depth. My search behaviour is also probably different from Pro Deo's (different reductions/extensions) so there may not be much point to try and replicate the conditions. Just going to use fixed depths will probably show so magnified gains per ply at lower depths but I don't see much point fixing that by applying some extensions..Rebel wrote:I have finished my long lasting experiment on diminishing returns and I must say I am quite optimistic that within 10-15 years the 4000 elo barrier will be crossed.
http://www.top-5000.nl/ply.htm
Even at longer time controls such as the above last match (average 87 minutes for 1 game) ProDeo keeps profiting big time, 93 elo. Since only 100 games were played that number is not so clear.Code: Select all
[PLY=14] vs [PLY=15] 50-36.6=13.4% = 93 elo D14 = 64:12:24 101 games D15 = 127:50:03
Those with better hardware (Bob ?) could do a similar experiment to predict the elo future of computer chess.
Code: Select all
Num. Name games score
0 scorpio6 6271 697.5
1 scorpio7 7460 1911
2 scorpio12 4550 3297.5
3 scorpio15 1524 1395
4 scorpio8 8614 3285
5 scorpio13 3298 2658.5
6 scorpio14 2362 2058.5
7 scorpio9 9259 4637
8 scorpio10 7901 4630.5
9 scorpio11 6531 4314.5
Rank Name Elo + - games score oppo. draws
1 scorpio15 406 23 22 1524 92% -144 15%
2 scorpio14 344 16 16 2362 87% -93 19%
3 scorpio13 278 12 12 3298 81% -41 25%
4 scorpio12 192 10 9 4550 72% -26 28%
5 scorpio11 94 8 8 6531 66% -54 28%
6 scorpio10 -7 7 7 7901 59% -87 28%
7 scorpio9 -122 7 7 9259 50% -129 25%
8 scorpio8 -248 7 7 8614 38% -131 24%
9 scorpio7 -388 8 8 7460 26% -136 21%
10 scorpio6 -550 10 10 6271 11% -121 15%
IMHO, your results are consistent with others. If I apply my logarithmic approach of Y(depth_i) ~ m*ln(depth_i) + n, I get Y(depth_i) ~ 1057*ln(depth_i) - 2443, R² ~ 0.9996:Daniel Shawul wrote:Code: Select all
Rank Name Elo + - games score oppo. draws 1 scorpio15 406 23 22 1524 92% -144 15% 2 scorpio14 344 16 16 2362 87% -93 19% 3 scorpio13 278 12 12 3298 81% -41 25% 4 scorpio12 192 10 9 4550 72% -26 28% 5 scorpio11 94 8 8 6531 66% -54 28% 6 scorpio10 -7 7 7 7901 59% -87 28% 7 scorpio9 -122 7 7 9259 50% -129 25% 8 scorpio8 -248 7 7 8614 38% -131 24% 9 scorpio7 -388 8 8 7460 26% -136 21% 10 scorpio6 -550 10 10 6271 11% -121 15%
Code: Select all
Y(depth) is rounded up to 0.1 Elo:
Depth: Rating: Y(depth) Y(depth) - Rating:
------ ------- -------- ------------------
6 -550 -549.1 0.9
7 -388 -386.2 1.8
8 -248 -245 3
9 -122 -120.5 1.5
10 -7 -9.2 -2.2
11 94 91.6 -2.4
12 192 183.5 -8.5
13 278 268.2 -9.8
14 344 346.5 2.5
15 406 419.4 13.4Code: Select all
Num. Name games score
0 scorpio6 8271 1132
1 scorpio7 10271 3084.5
2 scorpio9 12743 6364.5
3 scorpio11 10027 6117
4 scorpio13 5622 4018
5 scorpio8 12148 5167
6 scorpio10 11458 6376
7 scorpio12 7839 5126.5
8 scorpio15 2134 1809.5
9 scorpio14 3743 2933
Rank Name Elo + - games score oppo. draws
1 scorpio15 406 15 15 2134 85% -17 25%
2 scorpio14 340 11 10 3743 78% 35 29%
3 scorpio13 272 8 8 5622 71% 63 33%
4 scorpio12 188 7 7 7839 65% 45 33%
5 scorpio11 91 6 7 10027 61% -8 32%
6 scorpio10 -8 6 6 11458 56% -61 31%
7 scorpio9 -122 6 6 12743 50% -127 28%
8 scorpio8 -248 6 6 12148 43% -174 26%
9 scorpio7 -381 7 7 10271 30% -183 24%
10 scorpio6 -539 8 8 8271 14% -167 18%
Please take a look to my logarithmic fit in this topic.Daniel Shawul wrote:Hi Jesuz:
It seems to me that the rate of diminishing is not as fast as I expected. It may be because the deeper depths didn't play enough games to tell their elo differences. Average opponent elo is still in a bad shape. The elo delta b/n 13&14 and 14&15 is 68 and 66 respectively. I think it will be very hard to prove that there is a significant diminishing of elo return per ply at larger depths. Even on the lower end of the list 6,7, there is not much of that effect per ply.Elo delta b/n PLY and PLY + 1 from top to bottomCode: Select all
Num. Name games score 0 scorpio6 8271 1132 1 scorpio7 10271 3084.5 2 scorpio9 12743 6364.5 3 scorpio11 10027 6117 4 scorpio13 5622 4018 5 scorpio8 12148 5167 6 scorpio10 11458 6376 7 scorpio12 7839 5126.5 8 scorpio15 2134 1809.5 9 scorpio14 3743 2933 Rank Name Elo + - games score oppo. draws 1 scorpio15 406 15 15 2134 85% -17 25% 2 scorpio14 340 11 10 3743 78% 35 29% 3 scorpio13 272 8 8 5622 71% 63 33% 4 scorpio12 188 7 7 7839 65% 45 33% 5 scorpio11 91 6 7 10027 61% -8 32% 6 scorpio10 -8 6 6 11458 56% -61 31% 7 scorpio9 -122 6 6 12743 50% -127 28% 8 scorpio8 -248 6 6 12148 43% -174 26% 9 scorpio7 -381 7 7 10271 30% -183 24% 10 scorpio6 -539 8 8 8271 14% -167 18%
66
68
84
97
99
114
126
133
158
*
It is only a cheap model but I honestly think that my fit works reasonably well (with enough played games) because I often get values of R² > 0.999. So, in your case, delta between depth and (depth - 1) should be around 1000/depth or 1100/depth (the numerator is the slope of the adjusted line by least squares and it depends on the tested engine) with reasonably high depths. Once again: good luck!Ajedrecista wrote:I choose a line because of some reasons: when depth d tends to infinity, then ln(d) ~ 1 + 1/2 + 1/3 + ... + 1/d and ln(d - 1) ~ 1 + 1/2 + 1/3 + ... + 1/(d - 1); delta_x = ln(d) - ln(d - 1) = ln[d/(d - 1)] ~ 1/d. If Y(x) = mx + n, dY/dx = m; estimate Elo gain = delta_Y = m*delta_x ~ m/d ---> 0 if d ---> infinity (diminishing return exists with this model).
A quadratic function fails with the same previous analysis: Y(x) = ax² + bx + c; dY/dx = 2ax + b; delta_x ~ 1/d (the same as before); estimate Elo gain = delta_Y = (dY/dx)*delta_x = (2ax + b)/d ~ {2a*[d + (d - 1)]/2 + b}/d ~ 2a = constant: diminishing return does not exist with this model (the same with other polynomials of higher degree). In dY/dx, I choose the average mean x ~ [d + (d - 1)]/2 because it makes sense to me. I am not taking into account error bars.
Code: Select all
Num. Name games score
0 scorpio6 10271 1567.5
1 scorpio7 13366 4279.5
2 scorpio9 16814 8437
3 scorpio11 14023 8185.5
4 scorpio13 9754 6111
5 scorpio8 16286 7323
6 scorpio10 15488 8442.5
7 scorpio12 11826 7154.5
8 scorpio14 6813 4703
9 scorpio15 4227 3230.5
Rank Name Elo + - games score oppo. draws
1 scorpio15 409 9 9 4227 76% 141 36%
2 scorpio14 338 8 7 6813 69% 149 38%
3 scorpio13 268 6 6 9754 63% 146 39%
4 scorpio12 184 6 6 11826 60% 87 37%
5 scorpio11 91 5 5 14023 58% 16 34%
6 scorpio10 -9 6 5 15488 55% -52 32%
7 scorpio9 -123 6 5 16814 50% -129 29%
8 scorpio8 -245 6 5 16286 45% -195 27%
9 scorpio7 -380 6 6 13366 32% -209 25%
10 scorpio6 -534 7 7 10271 15% -195 20%
Code: Select all
71
70
84
93
100
114
122
135
154
*
I am glad to see that the logarithmic fit works fine! Well, regarding my model, this is a drawback... but more than 20000 Elo between depth 1 and depth d (in this case: Y(d) - Y(1) ~ 1037*ln(d/1) = 1037*ln(d)) would come with d > 2.37e+8, and I doubt that those depths can be reached. Please take a look to the first point of this web:Daniel Shawul wrote:Hi Jesuz:
I understand your logarithmic fit but note that my data does not show significant diminishing returns so far. Maybe I need to test at bigger depths to show that but depth-15 is already too much. After a day and half run, here is new data with logarithmic interpolationCode: Select all
Num. Name games score 0 scorpio6 10271 1567.5 1 scorpio7 13366 4279.5 2 scorpio9 16814 8437 3 scorpio11 14023 8185.5 4 scorpio13 9754 6111 5 scorpio8 16286 7323 6 scorpio10 15488 8442.5 7 scorpio12 11826 7154.5 8 scorpio14 6813 4703 9 scorpio15 4227 3230.5 Rank Name Elo + - games score oppo. draws 1 scorpio15 409 9 9 4227 76% 141 36% 2 scorpio14 338 8 7 6813 69% 149 38% 3 scorpio13 268 6 6 9754 63% 146 39% 4 scorpio12 184 6 6 11826 60% 87 37% 5 scorpio11 91 5 5 14023 58% 16 34% 6 scorpio10 -9 6 5 15488 55% -52 32% 7 scorpio9 -123 6 5 16814 50% -129 29% 8 scorpio8 -245 6 5 16286 45% -195 27% 9 scorpio7 -380 6 6 13366 32% -209 25% 10 scorpio6 -534 7 7 10271 15% -195 20%
As you can see the logarithmic fit of 1037ln(x) - 2397 (similar to yours) is very good and it doesn't show flattening out so quickly. Plot it and you will see you can reach astronomical elos > 20000 elo. That is why I think either more tests are required or there isn't significant dimnishing except at lower depths ...
Elo delta's b/n ply and ply-1 from top to bottom.DanielCode: Select all
71 70 84 93 100 114 122 135 154 *
It means 11,897 or 11,898 plies; with the logarithmic fit and depending on each engine, the difference between depth 1 and this extreme depth would be around 9000, 10000, 12000... Elo, which is too much. But the game advances and the maximum possible depth also diminishes, so maybe differences are less than this upper bound. Differences between one ply and forty plies would be around 4000 Elo (depending on the engine; maybe a little more or a little less) with my cheap approach... I do not know if it is plausible or not, but my model suggests slow diminishing returns between consecutive depths.The longest Chess game theoretically possible is 5,949 moves.
But we know that Perft(8) = 84,998,978,956 and it can be easily checked by JetChess 1.0.0.0 in my slow PC in just two minutes. This wrong number is an historical mistake that is explained at the bottom of this fantastic web site by François Labelle. Regarding Perft(20), please read this topic. It is also explained in Labelle's web.The number of possible ways of playing the first four moves per side in a game of Chess is 318,979,564,000.
According to the America’s Foundation for Chess, there are 169, 518, 829, 100, 544, 000, 000, 000, 000, 000 ways to play the first 10 moves of a game of Chess.
The correct number is Positions(6) = 9,417,681 (easily checked with JetChess in nine seconds in my PC).After each side has played three moves, the pieces could form any one of over nine million possible positions on the board.
The number of unique positions after four plies are 72,078 instead of 72,084; after eight plies: 988,187,354 (number of distinct chess positions after n plies including differences due to availability and possibility of castling and en passant captures). Maybe there are more mistakes in that web. I write this message to reinforce the point of taking the maximum number of possible moves with lots of care.There are 400 different possible positions after one move each. There are 72,084 different possible positions after two moves each. There are over 9 million different possible positions after three moves each. There are over 288 billion different possible positions after four moves each. The number of distinct 40-move games is far greater than the number of electrons in the observable universe.