Sedat Canbaz wrote:Hello Friends,
...Note that I've changed my mind...and for the future,
I don't plan to create anymore engine opening books by GMs games!
Because GMs games winning percentage are not same as Top chess engines !
In this way...I believe the book's openings are easier to optimize !!
.
Sedat,
With all due respect to your book making abilities I think you should reconsider your decision. I think its a mistake not to include human games!
The reason is, computers contribute very little if anything to opening theory. The reason they don't contribute more is they simply regurgitate openings from their books. Until they're out of book they don't calculate and unlike humans they don't do any preparatory analysis before playing in a tournament. So if you use only computer games your books will fall behind current opening theory. You need some source of new play or novelties to keep the book in sync with current praxis. I don't think computer play will provide this in sufficient quantity to stay current.
On a slightly different tack, I've done an analysis of positions that occurred in a 7+-million game data base in an attempt to estimate the possible ELO gains that can be made by positional learning. ( i.e. positions not considered part of an opening) My conclusion was that little to nothing can be gained by this method. I found that positions in the middle game don't repeat often enough for there to be any benefit from this method.
The endgame and openings repeat position often enough to make this worthwhile. EGTB's do a poor job of handling the end game positions. The problem is that they don't have 12, 11, 10, 9, or 8 man table bases. So there is considerable room for improvement that could be filled by positional learning in the endgame. Although this would appear to require a new an innovative approach on how to collect and/or generate and then effectively use incomplete information about a particular endgame.
Openings are a completely different matter. Data gathered from the game databases show some interesting statistic. Before I started gathering data I checked every game for legality. I also removed all duplicate games. This included games between the same players in which the game was in whole a partial repeat of another game by the same players with the same outcome (usually a draw) or in some cases a different outcome. There were also a large number of games that were shortened, had moves altered, had the outcome changed or reversed, had the players names reversed, had incorrect dates, one players name missing etc, I tried to remove all of these games. I also attempted to remove all computer games since they add little or nothing to the openings. It's possible that some duplicates still slipped through by having the players name changed (misspelled etc) in such a fashion that the computer didn't recognize that the game was played by the same players. I also removed any game less than 40 plies long. I considered removing any game less than 60 plies long but the down side was a large amount of useful data would be lost vice the benefit of removing a few more GM draws and games that contained early blunders. Some of the draws are important, and most of the blunder aren't repeated so will have little effect on the statistics. So I decided that a 40 ply limit, while not perfect, was good enough. These efforts when combined, removed a very large number of games from the data sets. i.e. millions of games
Of the 617,791 games in which both players had a verified FIDE rating of 2400 or above, there were 53,112,182 moves played and a total of 37,809,293 unique positions reached.
Of the 37,809,293 positions only 3,516,016 were seen more than once. Only 125,629 were repeated 10 times or more. Seeing a position 10 times in 617,791 games means that
"on average" you would expect to see such a position, roughly speaking, at least twice in 100,000 games.
The number of positions that occurred varied depending on how many plies are considered. Below is a table that shows the depth in plies and the number of positions that were repeated at least 10 times in 617,791 games.
All depths 125,629 100.0% difference 162
60 plies 125,467 99.9% difference 45
50 plies 125,422 99.8% difference 439
40 plies 124,983 99.5% difference 4,841
30 plies 120,142 95.6% difference 31,869
20 plies 88,273 70.3% difference 69,180
10 plies 19,093 15.2% difference 19,093
As can be seen considerably more than half of the positions occur in the first 10 moves (20-plies). Very few repeat positions occur past 40 plies. I actually wanted to do this analysis on much larger data sets, but I was using crafty to extract the data from very large pgn files. I tried doing one with 13+ million games but crafty died about 10 minutes into it. When it did this it would corrupt all the previously written .bin files it had worked on. When I got down to 5,000,000+ games in was still choking but not dying. So if you are working with large book files with crafty its good to exit after making each book so that if the program dies it won't corrupt previous work. With smaller files it would generate the info I wanted but had other problems. It seems that the it can't handle more than 100 sort files. If it creates more than this it won't delete any of them when done and generally the book files is corrupted. It reads large input files ok but can't write a file as large as 2gb. Many of the book files that were much smaller than this limit were also corrupted during creation.
In the end I found that if I kept the number of sort files to 100 or less it seem to work ok as long as the output file was relatively small. The largest game file I was able to do had 4.8+ million games in it. But I'm not going to post any of that data here because it seems to have some anomalies in it. I did one set with 3,181,705 games in it and the output seems to be ok. Here is the data from that set:
All depths 545,645 100.0% difference 2,732
60 Plies 542,913 99.5% difference 88
50 Plies 542,825 99.5% difference 1,589
40 Plies 541,236 99.2% difference 19,830
30 Plies 521,406 95.6% difference 144,150
20 Plies 377,256 69.1% difference 308,323
10 Plies 68,993 12.6% difference 68,993
A few things changed with this set of games that deserve noting. The number of the games increased by a factor of 5.15 and the quality of the play dropped such that the minimum ELO is now only 2,000. One thing that didn't change is the number of times a position had to be repeated to be included. It stayed at 10 even though the number of games grew by more than a factor of five. This is the primary reason for the inflated numbers. Its not the result of allowing lower rated players into the mix as some might assume. If we increase the number of times a position has to appear to be included in the final output to 51 then the frequency of occurrence between the smaller set and the larger set will be approximately equal. This yields these numbers for 3,181,705 games with each included position occurring 51 times:
All depths 101,603 100.0% difference 106
60 Plies 101,497 99.9% difference 0
50 Plies 101,497 99.9% difference 29
40 Plies 101,468 99.9% difference 1,399
30 Plies 100,069 98.5% difference 16,200
20 Plies 83,869 82.5% difference 59,751
10 Plies 24,118 23.7% difference 24,118
The first thing I noted was an approximate reduction of position by ~20% as compared to the first set of data. Apparently the lower rated players are spreading their play across more lines of play than higher rated player. This causes a decrease in the number of positions that reach the minimum threshold to be include in the data, thus the lower number of positions included. The second thing I noted was that the lower rated players are not following the lines as deeply as the higher rated players causing an increase percentage of included positions in the sub-30 ply range.
The raw numbers are pretty small. Even so I don't think these numbers properly represent the problem size. A better metric would be the number of terminal position. I don't have exact numbers for each set but the smaller set seems to have about 6,000 and the larger set seems to have about 26,000 terminal positions. These numbers are small enough that they are subject to extensive computer analysis. The size of the problem is probably over estimated by these numbers since these ARE NOT (emphasis added) a bunch of unrelated positions. Many of these position are very closely related and should be analyzed as a group. I don't believe that "normal computer analysis" is sufficient to evaluate these positions. By "normal computer analysis" I mean taking a position, feeding it into a chess engine, and then waiting X seconds to get an evaluation. This type of evaluation is completely unreliable and next to worthless for these types of positions in my opinion. One reason I don't believe normal analysis is suitable for this task is that it is based only on tactics. I'm sure that some of the positions will be found to be untenable because the computer will find tactics that change the evaluation of the position. But, I also expect this to be the exception and not the rule.
It surprises me that no one has identified the terminal positions or undertaken their systematic evaluation by way of deep line-of-play analysis.
Regards,
Forrest
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.
