No need to generate whole 8-man tables. Few 5-10 might be doable right, or is Bojun Guo's hardware limited by RAM in this case?
I remember EG magazine has mentioned before that few 8-man tablebases in other format than syzygy has been generated earlier.
7 Man Syzygy and SSD
Moderators: hgm, Rebel, chrisw
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
-
- Posts: 12038
- Joined: Mon Jul 07, 2008 10:50 pm
Re: 7 Man Syzygy and SSD
I am sure every one will second that.Dann Corbit wrote: ↑Wed Jul 31, 2019 8:59 pm
We should be incredibly grateful for the efforts of Bojun Guo (aka noobpwnftw) and Ronald de Man for the stupendous effort of building the 7 man syzygy files and for making the code public and freely available for use.
It was very altruistic on the part of both parties.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7 Man Syzygy and SSD
But this is the wrong measure. In most games an engine will reach 5-men positions in the search and when they do, they can replace searching a whole subtree with a single probe.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7 Man Syzygy and SSD
I also seem to remember than a few have been generated. They must have been pawnless ones with a lot of duplicate pieces (like KQQQvKQQQ). After removing symmetries, these tables have about as many positions as some 7-men TBs. (My generator does not remove such symmetries, though.)Nordlandia wrote: ↑Thu Aug 01, 2019 7:59 am I remember EG magazine has mentioned before that few 8-man tablebases in other format than syzygy has been generated earlier.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: 7 Man Syzygy and SSD
Is it feasible to generate few 8-man tables in pawnless state or do it involve sophisticated modification of the generator.syzygy wrote: ↑Sat Aug 03, 2019 3:21 pmI also seem to remember than a few have been generated. They must have been pawnless ones with a lot of duplicate pieces (like KQQQvKQQQ). After removing symmetries, these tables have about as many positions as some 7-men TBs. (My generator does not remove such symmetries, though.)Nordlandia wrote: ↑Thu Aug 01, 2019 7:59 am I remember EG magazine has mentioned before that few 8-man tablebases in other format than syzygy has been generated earlier.
-
- Posts: 919
- Joined: Sat May 31, 2014 8:28 am
Re: 7 Man Syzygy and SSD
You seem to be saying that there is low/no correlation correlation between the number of TB hits that occur in the search and what shows up on the board. I'm not buying it. I'm assuming that what appears on the board has a VERY high correlation with what appeared in the search. But, if you have evidence to the contrary I'd be happy to consider it.
Your statement that whole sub-trees can be skipped by a TB hit is clearly true but, I don't have a program that will sum all TB accesses across each TB file for a 7M game database in a “reasonable” period of time. i.e 7,000,000 games * 150 plies per game * 60 second search per position = approximately 2,000 years. Even with a modest database with short games and short searches, 500,000 x 100 x 1 = ~579 days. The limit case would be to have no search at all, which is what I chose.
If you have a better method to obtain the required data please share! Until we have better data, I think the metric I chose is a reasonable one.
Even if it were feasible to generate 8-man pawnless EGTB why would you want to. I checked a few high quality game databases and found less than 1% of games end up in all piece endgames and >95% those are 6 man or less. So something less than 0.04% of games will see an all piece 8-man ending.Nordlandia wrote: ↑Sat Aug 03, 2019 5:39 pmIs it feasible to generate few 8-man tables in pawnless state or do it involve sophisticated modification of the generator.syzygy wrote: ↑Sat Aug 03, 2019 3:21 pmI also seem to remember than a few have been generated. They must have been pawnless ones with a lot of duplicate pieces (like KQQQvKQQQ). After removing symmetries, these tables have about as many positions as some 7-men TBs. (My generator does not remove such symmetries, though.)Nordlandia wrote: ↑Thu Aug 01, 2019 7:59 am I remember EG magazine has mentioned before that few 8-man tablebases in other format than syzygy has been generated earlier.
Pawnful endgames require less memory to generate because they can be broken up into pawn “slices”. (i.e. with a single pawn on the board the TB can be split into 8 separate parts with each part being calculated separately. With two pawns on the board the Tb can be split into 64 separate pieces etc. Note: this doesn't count any reduction due to symmetries.) This can aid in both generation and distribution of the files.
By far the majority of the conversions from one EGTB to another occur by a piece being removed from the game due to capture. A quick look at game databases shows something like 85%+ will convert by captures.
The promotions of a pawn leading to an all-piece endgame which hasn't been calculated is of course possible but it's not the end of the world. In some cases it's not even possible. E.g. Any positions with opposing pawns, one pawn of the opposite color on the same file and in front of the other pawn, results in a position in which neither pawn can promote until a piece or a pawn has been removed from the board thus reducing the position to a 7-man EGTB.
Even in the case were a promotion does happen it's not the end. Many of these positions the promoted piece can be captured in one or more ways. ALL of these captures result in a 7-man TB hit and most of these will resolve the position. Only the rare case of a promotion with no captures or no “good” captures possible will the search have to continue.
This seems to indicate that generating 8-man all-piece endgames can be omitted and the pawnful endgames will still be quite useful.
Regards,
Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.
-
- Posts: 919
- Joined: Sat May 31, 2014 8:28 am
Re: 7 Man Syzygy and SSD
I fixed a few errors and extended the list of RTBW files to fill out an SSD. This assumes that
162GB is used for all syzygy tb 6-man and less. Notice the different order of the first few files
and the removal of some files that were a result of some unusual games in the database I used.
The Sum GB column should be used to fill the drive as nearly as possible. RTBZ files for 7-man
can be stored on a conventional hard drive. The second % column shows the % of all games in
the database that contain a 7-man EGTB position will be found if all files in that row the and
the rows above it are included on the disk.
Regards,
Zenmastur
162GB is used for all syzygy tb 6-man and less. Notice the different order of the first few files
and the removal of some files that were a result of some unusual games in the database I used.
The Sum GB column should be used to fill the drive as nearly as possible. RTBZ files for 7-man
can be stored on a conventional hard drive. The second % column shows the % of all games in
the database that contain a 7-man EGTB position will be found if all files in that row the and
the rows above it are included on the disk.
Code: Select all
Pieces # in DB % SUM % SUM GB GB #/MB
1 4+3 kpppkpp 60,353 0.85% 60,353 9.58% 2.107 2.107 28.639
2 5+2 krpppkr 20,016 0.28% 80,369 12.76% 3.183 1.076 18.606
3 5+2 kppppkp 3,933 0.06% 84,302 13.39% 3.520 0.337 11.672
4 5+2 kqpppkp 1,934 0.03% 86,236 13.69% 3.709 0.189 10.231
5 4+3 krppkrp 165,068 2.33% 251,304 39.90% 26.971 23.261 7.096
6 4+3 kqppkpp 7,116 0.10% 258,420 41.03% 29.080 2.110 3.373
7 5+2 krpppkp 3,337 0.05% 261,757 41.56% 30.173 1.093 3.054
8 5+2 krpppkn 552 0.01% 262,309 41.65% 30.362 0.189 2.914
9 5+2 kbpppkn 1,770 0.03% 264,079 41.93% 31.026 0.664 2.666
10 5+2 krpppkb 676 0.01% 264,755 42.04% 31.283 0.257 2.629
11 4+3 krppkpp 12,457 0.18% 277,212 44.02% 36.338 5.055 2.464
12 5+2 knpppkn 2,322 0.03% 279,534 44.38% 37.298 0.960 2.419
13 4+3 kpppkqp 5,008 0.07% 284,542 45.18% 39.568 2.270 2.206
14 5+2 krrppkr 903 0.01% 285,445 45.32% 40.037 0.469 1.926
15 4+3 kpppkrp 8,882 0.13% 294,327 46.73% 44.682 4.645 1.912
16 4+3 kbppkpp 15,047 0.21% 309,374 49.12% 52.655 7.974 1.887
17 5+2 kbpppkp 2,746 0.04% 312,120 49.56% 54.147 1.492 1.840
18 5+2 krbppkr 3,125 0.04% 315,245 50.05% 56.078 1.930 1.619
19 5+2 kbpppkb 2,812 0.04% 318,057 50.50% 57.875 1.798 1.564
20 4+3 knppkpp 13,315 0.19% 331,372 52.61% 67.565 9.690 1.374
21 5+2 kppppkn 596 0.01% 331,968 52.71% 68.005 0.440 1.355
22 5+2 kqrppkp 744 0.01% 332,712 52.83% 68.561 0.556 1.338
23 5+2 knpppkb 1,883 0.03% 334,595 53.13% 70.008 1.447 1.301
24 4+3 kpppkbp 9,839 0.14% 344,434 54.69% 77.592 7.584 1.297
25 4+3 kpppknp 8,578 0.12% 353,012 56.05% 84.273 6.681 1.284
26 5+2 knpppkp 2,214 0.03% 355,226 56.40% 86.004 1.732 1.278
27 5+2 kqbppkp 745 0.01% 355,971 56.52% 86.610 0.606 1.229
28 5+2 kqpppkr 314 0.00% 356,285 56.57% 86.886 0.276 1.139
29 5+2 krnppkr 2,604 0.04% 358,889 56.98% 89.291 2.405 1.083
30 5+2 kppppkb 587 0.01% 359,476 57.08% 89.923 0.632 0.928
31 4+3 kbppkbp 26,184 0.37% 385,660 61.23% 120.670 30.746 0.852
32 4+3 krppkbp 9,633 0.14% 395,293 62.76% 132.105 11.435 0.842
33 5+2 kqrppkr 569 0.01% 395,862 62.85% 132.814 0.709 0.803
34 5+2 kqnppkp 495 0.01% 396,357 62.93% 133.448 0.634 0.781
35 4+3 knppknp 21,387 0.30% 417,744 66.33% 162.535 29.088 0.735
36 4+3 knppkbp 20,993 0.30% 438,737 69.66% 193.491 30.956 0.678
37 4+3 kbppknp 19,074 0.27% 457,811 72.69% 222.219 28.728 0.664
38 4+3 krppknp 6,275 0.09% 464,086 73.69% 231.852 9.632 0.651
39 5+2 kppppkr 1,090 0.02% 465,176 73.86% 233.614 1.763 0.618
40 4+3 kqppkqp 13,494 0.19% 478,670 76.00% 256.246 22.632 0.596
41 5+2 kqpppkq 2,006 0.03% 480,676 76.32% 259.992 3.746 0.536
42 4+3 kqppkrp 2,364 0.03% 483,040 76.70% 265.499 5.507 0.429
43 4+3 kqrpkrp 2,570 0.04% 485,610 77.10% 271.556 6.058 0.424
44 4+3 kbppkrp 13,357 0.19% 498,967 79.22% 303.163 31.607 0.423
45 4+3 kqrpkpp 558 0.01% 499,525 79.31% 304.485 1.322 0.422
46 5+2 krbppkb 342 0.00% 499,867 79.37% 305.304 0.818 0.418
47 4+3 krppkqr 3,095 0.04% 502,962 79.86% 312.994 7.690 0.402
48 4+3 krbppkn 215 0.00% 503,177 79.89% 313.575 0.581 0.370
49 5+2 kppppkq 426 0.01% 503,603 79.96% 314.774 1.199 0.355
50 5+2 kbnppkn 308 0.00% 503,911 80.01% 315.673 0.899 0.343
51 5+2 kbpppkr 2,243 0.03% 506,154 80.37% 322.570 6.897 0.325
52 5+2 kbnppkb 455 0.01% 506,609 80.44% 324.021 1.451 0.314
53 4+3 krnppkn 186 0.00% 506,795 80.47% 324.647 0.627 0.297
54 4+3 knppkrp 8,395 0.12% 515,190 81.80% 354.559 29.911 0.281
55 4+3 kqppkbp 776 0.01% 515,966 81.92% 357.625 3.066 0.253
56 4+3 krnppkb 175 0.00% 516,141 81.95% 358.321 0.696 0.251
57 4+3 krppkqp 4,188 0.06% 520,329 82.62% 375.011 16.690 0.251
58 4+3 kqbpkpp 699 0.01% 521,028 82.73% 377.801 2.790 0.251
59 4+3 krbpkrp 11,999 0.17% 533,027 84.63% 426.182 48.381 0.248
60 5+2 krbppkp 750 0.01% 533,777 84.75% 429.249 3.067 0.245
61 4+3 kpppkqr 254 0.00% 534,031 84.79% 430.320 1.071 0.237
62 4+3 kqppknp 766 0.01% 534,797 84.91% 433.636 3.316 0.231
63 5+2 knpppkr 1,461 0.02% 536,258 85.15% 440.948 7.312 0.200
64 4+3 krrppkp 215 0.00% 536,473 85.18% 442.051 1.103 0.195
65 4+3 kqnpkpp 629 0.01% 537,102 85.28% 445.395 3.344 0.188
66 4+3 krrpkrp 1,653 0.02% 538,755 85.54% 454.761 9.366 0.176
67 4+3 krnpkrp 10,056 0.14% 548,811 87.14% 512.786 58.025 0.173
68 4+3 krppkrb 4,784 0.07% 553,595 87.90% 541.473 28.687 0.167
69 5+2 krnppkp 559 0.01% 554,154 87.99% 544.981 3.508 0.159
70 4+3 krppkrn 4,111 0.06% 558,265 88.64% 572.816 27.835 0.148
71 4+3 krppkrr 1,279 0.02% 559,544 88.84% 584.584 11.768 0.109
72 4+3 knppkqp 1,158 0.02% 560,702 89.03% 596.022 11.438 0.101
73 4+3 kbppkqp 1,090 0.02% 561,792 89.20% 608.161 12.139 0.090
74 4+3 krbpkpp 951 0.01% 562,743 89.35% 619.313 11.152 0.085
75 5+2 kbnppkr 1,206 0.02% 563,949 89.54% 635.522 16.209 0.074
76 4+3 kpppkqn 213 0.00% 564,162 89.58% 638.727 3.205 0.066
77 4+3 kbnpkpp 1,074 0.02% 565,236 89.75% 655.047 16.320 0.066
78 4+3 kpppkqb 199 0.00% 565,435 89.78% 658.080 3.033 0.066
79 5+2 kbnppkp 475 0.01% 565,910 89.85% 665.343 7.263 0.065
80 4+3 krpppkq 694 0.01% 566,604 89.96% 676.454 11.111 0.062
81 4+3 kqbpkbp 571 0.01% 567,175 90.05% 685.603 9.149 0.062
82 4+3 krrpkrr 1,271 0.02% 568,446 90.26% 706.257 20.655 0.062
83 4+3 kqrpkbp 202 0.00% 568,648 90.29% 709.541 3.283 0.062
84 4+3 krnpkpp 773 0.01% 569,421 90.41% 722.504 12.964 0.060
85 4+3 kpppkrb 333 0.00% 569,754 90.46% 728.664 6.160 0.054
86 4+3 kqnpkbp 501 0.01% 570,255 90.54% 738.215 9.551 0.052
87 4+3 krrpkpp 172 0.00% 570,427 90.57% 741.794 3.579 0.048
88 4+3 kqppkrb 451 0.01% 570,878 90.64% 751.415 9.621 0.047
89 4+3 knnpknp 419 0.01% 571,297 90.71% 760.638 9.223 0.045
90 4+3 krrpkbp 284 0.00% 571,581 90.75% 767.369 6.730 0.042
91 4+3 kqbpkrp 499 0.01% 572,080 90.83% 779.640 12.271 0.041
92 4+3 krppkbn 1,144 0.02% 573,224 91.02% 808.636 28.996 0.039
93 4+3 kbbppkb 235 0.00% 573,459 91.05% 814.627 5.991 0.039
94 4+3 kpppkrn 285 0.00% 573,744 91.10% 821.915 7.288 0.039
95 4+3 krbpkrb 3,105 0.04% 576,849 91.59% 902.196 80.280 0.039
96 4+3 kpppkbn 381 0.01% 577,230 91.65% 912.505 10.309 0.037
97 4+3 krbpkbp 970 0.01% 578,200 91.81% 939.504 26.999 0.036
98 4+3 kqnpknp 365 0.01% 578,565 91.86% 949.713 10.210 0.036
99 4+3 kqppkrn 304 0.00% 578,869 91.91% 958.279 8.566 0.035
100 4+3 krrpkqp 1,226 0.02% 580,095 92.11% 993.723 35.444 0.035
Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7 Man Syzygy and SSD
Just as chess is about reaching a winning pawn endgame, so are pawn endgames about getting a pawn to promote.
That your database doesn't show many promotions is not because the previous sentence is wrong but because games are typically resigned when promotion by the other side becomes inevitable.
What one could do to avoid generating pawnless TBs is assume any promotion is winning. The resulting TB would be very unreliable, but good luck with that.
-
- Posts: 919
- Joined: Sat May 31, 2014 8:28 am
Re: 7 Man Syzygy and SSD
Nice strawman!
But I don't believe it's true. Direct mating attacks, capturing pieces to gain a decisive material advantage, perpetual checks, three-move repetitions, and stalemates are all valid motifs in the endgame.
With an odd number of men (3,5, or 7 men) then it's harder to get a close balance between the two sides. So, many of the odd man EGTB's will naturally lead to more decisive outcomes. That however doesn't diminish the validity of using the EGTB to obtain a draw. With an even number of men (4,6, or 8 men) the positions can be exactly materially balanced. One would expect a much higher percentage of drawn positions in balanced or nearly balanced EGTBs.
So, I don't think it's all about winning. And it certainly isn't all about winning by promoting pawns even if pawns are present on the board.
Again I disagree. More on this later.That your database doesn't show many promotions is not because the previous sentence is wrong but because games are typically resigned when promotion by the other side becomes inevitable.
What makes you think I would have to “assume” anything. E.G.What one could do to avoid generating pawnless TBs is assume any promotion is winning. The resulting TB would be very unreliable, but good luck with that.
[d]2b5/2rk4/3p4/3P4/2BRK3/8/8/8 w KQkq - 0 1
I don't see any assumptions about promotions that need to be made to resolve this endgame or the many other ones like it. In an 8-man EGTB there are positions like this that contain 2, 3, 4, 5, and 6 pawns. It's not possible for any of these to be converted to 8-man pawnless endgame as the pawns are blocked. I'll let you work out the number of permutations of such endgames. It's not a small number. Even if not all pawns are blocked there are a huge number of positions where neither side can make progress!
Anytime there are two or more pawns on the board the is no single move that can convert the endgame to a pawnless 8-man endgame. The more pawns there are the less and less likely it is to ever convert to an 8-man pawnless endgame. There are simply too many other ways to win or draw.
Like I said earlier, 85% (or more) of all 8-man endgames that I looked at covert to a 7-man (or less) endgame by one or more pieces being captured. So, for the vast majority of endgames NO assumptions need be made. They will naturally convert to a 7-man endgame with best play.
Here is an example:
[d]8/8/8/5pR1/2n1k3/6P1/3r1BK1/8 b - - 0 63
Who knows how many positions like this there are. One side can AND should force the conversion to a 7-man EGTB. Absolutely no assumptions need to be made about what happens if a pawn is promoted since the best and direct method to end the game doesn't involve a promotion. I do know there are an absolutely HUGE number of positions like this.
Another one of those positions where no promotion is possible while remaining an 8-man position:
[d]8/1r1k4/R1n5/6K1/1pB5/1P6/8/8 b - - 0 49
A position where a promotion occurs but there is still not a pawnless endgame:
[d]3r4/P7/8/2R5/1K3k2/1B4n1/6p1/8 w - - 0 62
A couple of modified positions where a conversion by promotion to a 8-man pawnless endgame could occur:
[d]3r4/P7/8/2R5/1K3k2/1B4n1/6n1/8 w - - 0 62
[d]3r4/P7/8/2R5/1K3k2/1B4n1/6b1/8 w - - 0 62
Notice that even if the promotion occurs and you now have an 8-man pawnless endgame it doesn't make much of a difference. If the pawnless endgame file doesn't exist and the EGTB routine exits back to the search on the very next ply a capture will be tried and this will convert it to a 7-man pawnless endgame which will immediately resolve the issue without further search.
All of these positions came from “real” human games with the exception of the last two which I derived from a single human game by replacing a pawn with either a night or a bishop so when the promotion occurred it would become a pawnless 8-man EGTB position. It took more time to write this post than it did to find these positions. So the idea that resignation by human players is the reason more pawn promotions aren't seen seem to be unsubstanciated.
As far as I'm concerned the notion that pawnful EGTB become worthless “just” because the pawnless EGTB aren't available is ridiculous. My guess would be that they retain >95% of there full value EVEN if no pawless EGTB are generated.
Of the 8-man EGTB that appear to be worth generating all have at least three pawns. This has a few implications. There is no way to directly convert any of them to a pawnless 8-man EGTB. They can all be broken in to at least 512 pawn slices. This yields smaller files generated and less memory required to do the generations. This indicates that the exact same hardware used to generate the 7-man EGTB would be more than sufficient to generate any of these 8-man EGTB's. Some of the pawn slices will be more important than others, which opens the possibility to forego generating some of the least important slices. So most of these endgames would have very low memory requirements if sufficient pawn slices were used!
Of the top 10 files 2 have 6 pawns each, 2 have 5 pawns each, 5 have 4 pawns each and 1 has 3 pawns.
Code: Select all
# endgame # occ. Sum # occ.
1 krppkrpp 128,688 128,688
2 krpppkrp 55,780 184,468
3 krpppkrp 55,780 240,248
4 kpppkppp 48,227 288,475
5 kbppkbpp 23,218 311,693
6 kbpppkbp 10,955 322,648
7 kppppkpp 10,235 332,883
8 kpppkbpp 8,543 341,426
9 krppkrbp 8,266 349,692
10 kpppknpp 6,829 356,521
11 krppkrnp 6,429 362,950
12 krbppkrp 5,987 368,937
13 kpppkrpp 5,442 374,379
14 krnppkrp 4,899 379,278
15 krpppkpp 4,155 383,433
16 krpppkpp 4,155 387,588
17 kbpppkpp 4,022 391,610
18 knpppkpp 3,588 395,198
19 kpppkqpp 3,472 398,670
20 krppkqrp 2,146 400,816
21 krppppkr 2,056 402,872
22 kqpppkpp 2,025 404,897
23 krrppkrp 1,805 406,702
24 krppkrrp 1,681 408,383
25 krpppkrb 1,597 409,980
26 krpppkrn 1,374 411,354
27 kbppknbp 1,244 412,598
28 kbppkrbp 1,025 413,623
[pgn][Event "?"] [Site "?"] [Date "????.??.??"] [Round "?"] [White "?"] [Black "?"] [Result "1/2-1/2"] [FEN "1B6/4r3/2K2Rp1/1P6/6k1/5p1n/8/8 w - - 21 67"] 67.Rxg6+ Kh4 68.Bg3+ Kh5 69.Rf6 Kg4 70.Bd6 Re2 71.b6 Rc2+ 72.Bc5 Ng5 73.b7 Ne4 74.Rf4+ Kxf4 75.b8=Q+ Kg4 76.Qg8+ Kf5 77.Qh7+ Ke5 78.Qh5+ Kf4 79.Qf7+ Ke5 80.Qc7+ Kf5 81.Qd7+ Kf4 1/2-1/2[/pgn]
Notice no 8-man pawnless endgames is reached, the game converts to a 7-man EGTB before the promotion, there is no resignation prior to the promotion and the point of the promotion is to force a draw by perpetual check or exchange pieces to force insufficient material.
It's early in the morning again so I hope I didn't screw any of the positions up as I modifed the post several times and changed several of the positions.
Regards,
Zenmastur
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.
-
- Posts: 5563
- Joined: Tue Feb 28, 2012 11:56 pm
Re: 7 Man Syzygy and SSD
Fine, so you don't really know much about chess. Then we can end the conversation here.Zenmastur wrote: ↑Tue Aug 06, 2019 9:19 amNice strawman!
But I don't believe it's true. Direct mating attacks, capturing pieces to gain a decisive material advantage, perpetual checks, three-move repetitions, and stalemates are all valid motifs in the endgame.