Strong engines make more draws

[smirobth]

Logically it is to be expected, but still it's nice to see some data.

This provides handy evidence in support of my "chess might be almost solved" assertion (click here).

To my way of thinking this result doesn't provide any support for your assertion. Rather it seems to support what most people already believe, the game theoretic result of chess with perfect play is a draw.

Robin, the two assertions are mutually inclusive! One cannot assert that stronger chess engines obtaining more draws is evidence of chess being a theoritical draw unless one also accepts that chess computers are getting close to playing "near-perfect" chess as defined here.

As computers get stronger they make mistakes less often. But that does not necessarily translate into them getting 'close to playing "near-perfect" chess', merely stronger chess than they previously played. Closer to perfect is not that same thing at all as close to perfect, and there are clearly still positions that occur relatively frequently in actual games where computers persist in showing very poor understanding and play(fortresses are a prime example). It seems this situation is unlikely to change dramatically any time soon and until it does change claims that computers are near playing perfect chess are very premature, IMO.

If what you say is correct, then you have also attacked your own assertion that stronger chess computers gettting more draws is evidence of chess being a theoretical draw.

I disagree, since as I explained I do think programs are getting "closer" to perfect chess. The fewer mistakes they make, the more drawn games one would expect to see, since the incidence of error free games would increase. For example if the number of error free games goes from 2% to 6% (closer to perfect) that will increase the number of drawn games by a measurable amount in a very large collection of games, without getting remotely close to near-perfect chess in the entire game collection.

But actually, I don't agree that you are correct: with extra depth of search, two things will happen:

1. The number of positions where computers make mistakes will continue to fall away (as suddenly they see far enough ahead to see the mistake)

2. It becomes less easy to get them into positions where they can make these mistakes

While what you say above about increasing search depth is true for some types of errors computer make, for errors related to fortresses some means of fortress detection will have to be devised, or search depths will need to reach 100 plies. So far neither is in evidence and the programs of today seem to have just about as much trouble with fortresses as they historically always have.

[towforce]

if the number of error free games goes from 2% to 6% (closer to perfect) that will increase the number of drawn games by a measurable amount in a very large collection of games, without getting remotely close to near-perfect chess in the entire game collection.

If the average computer game is 100 moves (200 half moves), then moving from 2% error free to 6% error free would represent very significant progress indeed!

If you wish, I will use the binomial distribution to calculate the actual change that represents in the probability of an error in a given half-move.

While what you say above about increasing search depth is true for some types of errors computer make, for errors related to fortresses some means of fortress detection will have to be devised, or search depths will need to reach 100 plies. So far neither is in evidence and the programs of today seem to have just about as much trouble with fortresses as they historically always have.

I'm sorry, but this just simply cannot be correct IMO.

I have no doubt whatsoever that you could set a position up which would require a 100 ply search to resolve - but you wouldn't find it easy to play a computer into such a position (especially if you didn't know what moves it would make in advance). There was a time (approx 10 years ago) when the "It's the economy, stupid" expression among strong humans who were playing computers was "blocked centre" - but we can clearly see that this strategy is no longer yielding the dividends that it was back then.

[smirobth]

While what you say above about increasing search depth is true for some types of errors computer make, for errors related to fortresses some means of fortress detection will have to be devised, or search depths will need to reach 100 plies. So far neither is in evidence and the programs of today seem to have just about as much trouble with fortresses as they historically always have.

I'm sorry, but this just simply cannot be correct IMO.

I have no doubt whatsoever that you could set a position up which would require a 100 ply search to resolve - but you wouldn't find it easy to play a computer into such a position (especially if you didn't know what moves it would make in advance). There was a time (approx 10 years ago) when the "It's the economy, stupid" expression among strong humans who were playing computers was "blocked centre" - but we can clearly see that this strategy is no longer yielding the dividends that it was back then.

While I have no doubt that in an over the board game I would have extraordinary difficulty playing a strong computer into a position where the computer would blunder due to lack of knowledge about fortresses or other drawn endgame structures, my ability to do so was never the subject. I see these positions in postal games a fairly high percentage of the time and I also have little doubt that computers play other computers into such positions far more often than once every 10,000 moves as in your definition of "near perfect". And when a fortress, or even the possibility of a fortress, arises the computers get very confused. Until computers understand fortresses they won't be able to play "near perfect" chess.

Another way to demonstrate how frequently computers still make errors is to search for actual games that reach a 6 man ending. Then have a program that does not have access to 6 man tablebases analyze the ending. In some 6 man endgame positions the computer will do very well. But in other positions they will make frequent errors. If they sometimes are making that many errors in actual game positions with only 6 men, imagine how many errors they sometimes make with 7 or more men on the board. IMO computers are still a long way from "near perfect" as you have defined the term.

[smirobth]

if the number of error free games goes from 2% to 6% (closer to perfect) that will increase the number of drawn games by a measurable amount in a very large collection of games, without getting remotely close to near-perfect chess in the entire game collection.

If the average computer game is 100 moves (200 half moves), then moving from 2% error free to 6% error free would represent very significant progress indeed!

The 2% and 6% figures were just random numbers I pulled out of the air to demonstrate a point ("closer" to perfect can still be a far cry from "close" to perfect). No need to calculate what progress this would represent; you and I both agree that computers have been making significant progress.

[Kirill Kryukov]

Hi Kirill,

thanks god I mustn't do the counting myself, a nice little Excel Script written on the fly does the job. I extracted all the stuff I got into some html-files.
Just a note, in the upper row 50 means difference of +-0-49 Elo, 100 means +- 50-99 Elo, 150 means +- 100-149 Elo etc. etc. just for clarification.

Here are the links: (Attention ! Huge tables)

For CCRL 40/04: http://www.quarkchess.de/quark/CCRL-404 ... istics.htm
For CEGL 40/04: http://www.quarkchess.de/quark/CEGT-404 ... istics.htm
For CEGT 40/20: http://www.quarkchess.de/quark/CEGT-402 ... istics.htm

It isn't easy to interpret the material, any idea whether I should present it differently ? I have tried with a diagram but you can't really read anything out of it... So for now just pure data...

Greets, Thomas

Thanks Thomas! May be some sort of a graph could be easier to read. Or may be something like our "testing profile" table. Still very interesting!

[towforce]

Hi Kirill,

thanks god I mustn't do the counting myself, a nice little Excel Script written on the fly does the job. I extracted all the stuff I got into some html-files.
Just a note, in the upper row 50 means difference of +-0-49 Elo, 100 means +- 50-99 Elo, 150 means +- 100-149 Elo etc. etc. just for clarification.

Here are the links: (Attention ! Huge tables)

For CCRL 40/04: http://www.quarkchess.de/quark/CCRL-404 ... istics.htm
For CEGL 40/04: http://www.quarkchess.de/quark/CEGT-404 ... istics.htm
For CEGT 40/20: http://www.quarkchess.de/quark/CEGT-402 ... istics.htm

It isn't easy to interpret the material, any idea whether I should present it differently ? I have tried with a diagram but you can't really read anything out of it... So for now just pure data...

Greets, Thomas

Thanks Thomas! May be some sort of a graph could be easier to read. Or may be something like our "testing profile" table. Still very interesting!

Suggestion: the data could be imported into a google spreadsheet at http://docs.google.com/ , a graph drawn in the spreadsheet, the spreadsheet published as an online document, and the URL posted here.

I did an online spreadsheet in the post linked below to answer the (admittedly less interesting) question of how many grains of rice would be needed on a chessboard if there were 2 on the first square, then each subsequent square was the square of the previous square:

http://www.talkchess.com/forum/viewtopi ... 0&start=14