Testing the waters for opening move

Discussion of chess software programming and technical issues.

Moderators: hgm, Dann Corbit, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
PK-4

Testing the waters for opening move

Post by PK-4 » Wed Aug 29, 2007 5:41 pm

Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay

nczempin

Re: Testing the waters for opening move

Post by nczempin » Wed Aug 29, 2007 5:44 pm

PK-4 wrote:Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay
Well, what's keeping you from going through the book offline? Let your engine evaluate each exiting node overnight or over the weekend?

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Testing the waters for opening move

Post by bob » Wed Aug 29, 2007 5:51 pm

PK-4 wrote:Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay
Crafty has had this ability for many years. But there is a danger. As a general rule, a search will accept a gambit that no human would want to accept, because the search can not see deeply enough to find the "punch line".

That being said, it works. However, I don't use 1/5th, I have used 1/10th for the longest, as you don't need much of a search to discover that a move is really positionally bad according to the program's evaluation, and you need a search deeper than what is practical given the time constraints, to be able to avoid the very deep tactical traps that exist in a few book lines.

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Testing the waters for opening move

Post by bob » Wed Aug 29, 2007 5:52 pm

nczempin wrote:
PK-4 wrote:Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay
Well, what's keeping you from going through the book offline? Let your engine evaluate each exiting node overnight or over the weekend?
What do you do when you have a book with _millions_ of positions? :) That might pose a problem for overnight analysis...

Uri Blass
Posts: 8973
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Testing the waters for opening move

Post by Uri Blass » Wed Aug 29, 2007 8:48 pm

bob wrote:
nczempin wrote:
PK-4 wrote:Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay
Well, what's keeping you from going through the book offline? Let your engine evaluate each exiting node overnight or over the weekend?
What do you do when you have a book with _millions_ of positions? :) That might pose a problem for overnight analysis...
You buy many computers and divide the task between them.
Seriously here is what I do

1)The practical solution that I choose is to use result of the games in the pgn and I do not accept the book moves when the side to move did not win with white or did not get at least a draw with black.

2)I do not have million of positions in the book but I was too lazy to write a code to analyze all the opening positions.

3)I think that today it may be better to use comp-comp games for the book and not human-human games(I trust more the winner computer to play the right move relative to the human who won the game).

Uri

bob
Posts: 20923
Joined: Mon Feb 27, 2006 6:30 pm
Location: Birmingham, AL

Re: Testing the waters for opening move

Post by bob » Wed Aug 29, 2007 9:52 pm

Uri Blass wrote:
bob wrote:
nczempin wrote:
PK-4 wrote:Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay
Well, what's keeping you from going through the book offline? Let your engine evaluate each exiting node overnight or over the weekend?
What do you do when you have a book with _millions_ of positions? :) That might pose a problem for overnight analysis...
You buy many computers and divide the task between them.
While I know that was made in jest, if you take a book with 32 million positions, and run them on a cluster with 256 processors, you end up with 32M / 256 units of time required, or basically 128K units of time. If you do an overnight analysis for each, you need 128K days. Or a paultry 360 years. :)


Seriously here is what I do

1)The practical solution that I choose is to use result of the games in the pgn and I do not accept the book moves when the side to move did not win with white or did not get at least a draw with black.

The problem is there are plenty of moves that were played by the winning side which were losing moves, had the opponent played correctly. I have run into significant numbers of these. I've usually refused to play a move that was not played by _several_ different players, thinking that the chances of a blunder are much reduced if that is true. But there are still problems.

2)I do not have million of positions in the book but I was too lazy to write a code to analyze all the opening positions.

3)I think that today it may be better to use comp-comp games for the book and not human-human games(I trust more the winner computer to play the right move relative to the human who won the game).

Uri
That's ok, except the computers don't play certain types of openings very well at all, leaving large holes in your opening book that humans can drive right through.

nczempin

Re: Testing the waters for opening move

Post by nczempin » Thu Aug 30, 2007 7:48 am

bob wrote:
nczempin wrote:
PK-4 wrote:Hi all,
I have created an opening book out of gm2600. Earlier my program exited from the opening phase with an inferior position at times, it did not understand how to proceed and did not recover. To beat this I now allow my program to search with the book move for a small time of about 1/5th of regular think time. This usually avoids inferior positions at the end of opening phase, but I resent the loss of time. Creating a strong book is beyond my resources. I wonder how others tackle this problem. I have not seen any other program spend time thinking in the opening.

P.K.Mukhopadhyay
Well, what's keeping you from going through the book offline? Let your engine evaluate each exiting node overnight or over the weekend?
What do you do when you have a book with _millions_ of positions? :) That might pose a problem for overnight analysis...
I don't think this counter-example is very practical.

In that case one should reduce the number of positions, because I sincerely doubt that there is much gain in going from thousands of focused positions to millions of more or less random positions.

How big are the books of the most successful engines? Maybe we should take those as a starting point.

And how are those built? By humans who go through the book. By your logic, that would be a more infeasible task.

Post Reply