Organising a computer chess event...

[bhlangonijr]

Hardly. It first has a "guidance" facility where moves can be declared as "always" play or "never play" or "play some percent of the time". Once that has been exhausted, it has learning information to help select a move. If that fails, it considers how many times the move was played (in the PGN data used to create the book) and the win/loss percentage for those games. It folds in the static evaluation so that it won't play into a position it considers "busted". And it can even do a short search on the set of book moves to choose between them.

- If you have a line in the book that should NOT be played it is not the case of "cleaning up" that book?
- I guess the win/loss statistic created could be very inadequate if you play against weaker engines or play with diferent control-times.
- Don't you think that by searching a book move gets the idea of using a book pointless? (even if you fix a low depth)

If you don't study opening theory, you won't ever (as a human) be any kind of strong chess player. This same principle applies to computer chess programs. They ought to have to choose every move that is played... I can't get a "stand-in" to play opening moves for _me_ when I play a game...

That's true. I'll reconsider this.

Thanks for the explanation.

[bob]

Hardly. It first has a "guidance" facility where moves can be declared as "always" play or "never play" or "play some percent of the time". Once that has been exhausted, it has learning information to help select a move. If that fails, it considers how many times the move was played (in the PGN data used to create the book) and the win/loss percentage for those games. It folds in the static evaluation so that it won't play into a position it considers "busted". And it can even do a short search on the set of book moves to choose between them.

- If you have a line in the book that should NOT be played it is not the case of "cleaning up" that book?

No. I probably want the move in the book, so that if the opponent plays it, I will know what to do. I just don't want to play it myself. This is exactly how humans memorize book lines in fact...

In openings like the Fried Liver or other wild lines, I know what black should not do as well as what he should do...

- I guess the win/loss statistic created could be very inadequate if you play against weaker engines or play with diferent control-times.

You misunderstand. The book is made from a large PGN game collection. I use _that_ win/lose/draw information to help me decide which book move to play in a given position.

- Don't you think that by searching a book move gets the idea of using a book pointless? (even if you fix a low depth)

First, you _really_ need to study this issue as it is pretty obvious that what is going on is not something you have given a lot of thought to. Why would I want to search? I have two book moves that have a 60% winning history. Which one to play? With no other information, I can search just those two moves and choose the one the program likes better, as that is one it is more likely to understand, although even this has issues...

As I told you previously, a chess program's book is not just a list of moves. It is much more, and choosing which move is an art form in itself...

If you don't study opening theory, you won't ever (as a human) be any kind of strong chess player. This same principle applies to computer chess programs. They ought to have to choose every move that is played... I can't get a "stand-in" to play opening moves for _me_ when I play a game...

That's true. I'll reconsider this.

Thanks for the explanation.

[hgm]

No. I probably want the move in the book, so that if the opponent plays it, I will know what to do. I just don't want to play it myself. This is exactly how humans memorize book lines in fact...

This is different in Polyglot book format: that stores positions, rather than lines, so that you don't have to store bad moves from a position, abut could still store the positions these moves lead to, and the refutation you would have to play there.

In principle, all the computation and move selection you do, could have been pre-calculated. In the end the entire procedure just boils down to picking a move with a certain probability.

I guess the confusion is because you want Crafty to work with a 'raw' book (containing unprocessed PGN frequencies only), while many others use 'cooked' books, and consider the selection process you do in Crafty as part of the book building process.

[bhlangonijr]

I guess the confusion is because you want Crafty to work with a 'raw' book (containing unprocessed PGN frequencies only), while many others use 'cooked' books, and consider the selection process you do in Crafty as part of the book building process.

That's my point. I use pgn-extract + polyglot to generate my books (and use polyglot to play book moves - i still don't have book code in my engine).
I got huge pgn databases from the internet (several GM games + computer rating lists games) and merge it all organising the file by ECO.
Then I generate the book as "cooked" as it can gets. I would keep positions that (presume) are "good" for both sides.
In the case there's a bad position i won't keep this so the engine gets "out of the book" and start searching.
If there's a positional issue that has motivated Crafty keeping bad positions in the book I would consider implement that issue
in my positional evaluation.
This way I got my engine aware of this issue in a generalized form rather than depending upon what I have stored in my book.

Dr Bob I understand what you are doing in Crafty and I agree with you about the importance of the opening
(the engine "participate in" choosing what is better to be played ). By the way, I have read your article about book learning.

I understand why you search book moves (But in my opinion it is somewhat contradictory to search a book move when it comes mainly to avoid the initial search and to lead your engine to known good open lines).

I agree with you (Dr Bob) about that issue but I still believe I can put all this stuff in the pre-build process.
This lead me to think that I can have the book code "detached" from my engine code (letting the GUI or polyglot doing this for me).

[diep]

No. I probably want the move in the book, so that if the opponent plays it, I will know what to do. I just don't want to play it myself. This is exactly how humans memorize book lines in fact...

This is different in Polyglot book format: that stores positions, rather than lines, so that you don't have to store bad moves from a position, abut could still store the positions these moves lead to, and the refutation you would have to play there.

In principle, all the computation and move selection you do, could have been pre-calculated. In the end the entire procedure just boils down to picking a move with a certain probability.

I guess the confusion is because you want Crafty to work with a 'raw' book (containing unprocessed PGN frequencies only), while many others use 'cooked' books, and consider the selection process you do in Crafty as part of the book building process.

How does your book respond to 1.d4 d5 2.e3,Bg4

Does it play the move Nf3 here, or does it capture the bishop?

Storing positions+all playable moves is superior to just storing positions.

Todays engines are too strong to assume that returning 'in book' into a line that some 2300 guy like me played, is the best thing to do.

The problem of building books based upon statistics is dead wrong.
A line gets played forever until 1 clever dude figures out its refutation.

Majority of moves then is dead wrong, but after that the line just doesn't get played anymore once they start finding this SINGLE game in database and do not find an answer to that move anymore.

That is especially true for 2600+ players. Just a SINGLE game can refute lines played already succesfully by 30 guys. But their feeling is: "this line is objectively no good despite that 30 guys have won it before". The seek objectivity, as their opponents are more than strong enough to win a won position, regardless how "tough" it is to find the best moves. A guy like Anand is so really good, if you give him a positoin he'll find the best move usually. Especially just out of book they KNOW they have a good position. If you play some sudden move there, they WILL find the objective best move potentially.

There is no hiding for these guys. Additionally they really study a lot of games in their preparation of their booklines. Most GM's just play an opening or 2 regurarly. So they have to prepare really little compared to what Arturo has to prepare for his openingsbook against other engines.

Statistics and already played bookmoves are useful to study, but going blindfolded for transpositions is very dangerous.

Another problem is transposing back to a mainline where you don't want to get lured into.

So the above is possible to avoid with a trick, as Bob, who is going to react without reading the rest of the lines. Yet more common is transpositions to other mainlines.

Good example is the Fritz type books.

1.Nc3,d5 2.e4

Now YOUR book would transpose to 2..c6 here, just like Fritz books, in most cases.

You transpose to Caro-Kann there.

Nothing wrong with caro-kann.

There is nothing wrong with 2..d4 either

Both are great moves.

Yet the small problem is that an engine like Fritz will ALWAYS LOSE the opening caro-kann when white is an engine say: Hiarcs.

Hiarcs would wipe out fritz there, based upon mobility play. White has good mobility, black doesn't.

More important is that after 1.e4 you might want to play e5 or c5.
But you never prepared caro-kann.

Now then this transposition is REALLY a problem.

You can fix this case, but there is another quadrizillion ways to transpose.

Storing position+moves is a lot better therefore.



Thanks,
Vincent

[bob]

No. I probably want the move in the book, so that if the opponent plays it, I will know what to do. I just don't want to play it myself. This is exactly how humans memorize book lines in fact...

This is different in Polyglot book format: that stores positions, rather than lines, so that you don't have to store bad moves from a position, abut could still store the positions these moves lead to, and the refutation you would have to play there.

In principle, all the computation and move selection you do, could have been pre-calculated. In the end the entire procedure just boils down to picking a move with a certain probability.

How so? RE: book learning. I modify the book during each game. How can _that_ be pre-computed???

I guess the confusion is because you want Crafty to work with a 'raw' book (containing unprocessed PGN frequencies only), while many others use 'cooked' books, and consider the selection process you do in Crafty as part of the book building process.

How does your book respond to 1.d4 d5 2.e3,Bg4

Does it play the move Nf3 here, or does it capture the bishop?

Takes the bishop. Same issue with 1. e4 e5 2. Bg5 a6 3. Nf3. Had this fixed in crafty's book selection for 10 years now because each "signature" has a component from parent position, a component from new position, so that this kind of transposition does not happen. Weeds out a few good ones as well, but that's life.

Storing positions+all playable moves is superior to just storing positions.

Why??

Todays engines are too strong to assume that returning 'in book' into a line that some 2300 guy like me played, is the best thing to do.

Why don't you avoid it then? When you read the PGN to create the book, you know the ratings of each opponent. Nothing says you have to trust 2300 players over 2600 players. In fact, nothing says you should trust a 2300 player at all. I know I don't...

The problem of building books based upon statistics is dead wrong.
A line gets played forever until 1 clever dude figures out its refutation.

There are solutions. You can let date of game be a high-priority aid in decision making. You can also import new games into an old book and flag moves accordingly...

Majority of moves then is dead wrong, but after that the line just doesn't get played anymore once they start finding this SINGLE game in database and do not find an answer to that move anymore.

That is especially true for 2600+ players. Just a SINGLE game can refute lines played already succesfully by 30 guys. But their feeling is: "this line is objectively no good despite that 30 guys have won it before". The seek objectivity, as their opponents are more than strong enough to win a won position, regardless how "tough" it is to find the best moves. A guy like Anand is so really good, if you give him a positoin he'll find the best move usually. Especially just out of book they KNOW they have a good position. If you play some sudden move there, they WILL find the objective best move potentially.

Now you see my point. This is a complex decision that belongs inside the engine, since _it_ is playing the game. One can program anything one wants to address these issues... Putting it into the GUI places it outside of normal control, which is not where it belongs...

There is no hiding for these guys. Additionally they really study a lot of games in their preparation of their booklines. Most GM's just play an opening or 2 regurarly. So they have to prepare really little compared to what Arturo has to prepare for his openingsbook against other engines.

Statistics and already played bookmoves are useful to study, but going blindfolded for transpositions is very dangerous.

I agree, and I don't do that...

Another problem is transposing back to a mainline where you don't want to get lured into.

So the above is possible to avoid with a trick, as Bob, who is going to react without reading the rest of the lines. Yet more common is transpositions to other mainlines.

Good example is the Fritz type books.

1.Nc3,d5 2.e4

Now YOUR book would transpose to 2..c6 here, just like Fritz books, in most cases.

You transpose to Caro-Kann there.

Nothing wrong with caro-kann.

There is nothing wrong with 2..d4 either

Both are great moves.

Yet the small problem is that an engine like Fritz will ALWAYS LOSE the opening caro-kann when white is an engine say: Hiarcs.

Hiarcs would wipe out fritz there, based upon mobility play. White has good mobility, black doesn't.

More important is that after 1.e4 you might want to play e5 or c5.
But you never prepared caro-kann.

Now then this transposition is REALLY a problem.

You can fix this case, but there is another quadrizillion ways to transpose.

Storing position+moves is a lot better therefore.



Thanks,
Vincent

It can certainly work your way. It can _also_ work "my" way... Programmer's choice. But I do want it to be the programmer's choice, _not_ the GUI's choice...

[bob]

I guess the confusion is because you want Crafty to work with a 'raw' book (containing unprocessed PGN frequencies only), while many others use 'cooked' books, and consider the selection process you do in Crafty as part of the book building process.

That's my point. I use pgn-extract + polyglot to generate my books (and use polyglot to play book moves - i still don't have book code in my engine).
I got huge pgn databases from the internet (several GM games + computer rating lists games) and merge it all organising the file by ECO.
Then I generate the book as "cooked" as it can gets. I would keep positions that (presume) are "good" for both sides.
In the case there's a bad position i won't keep this so the engine gets "out of the book" and start searching.
If there's a positional issue that has motivated Crafty keeping bad positions in the book I would consider implement that issue
in my positional evaluation.
This way I got my engine aware of this issue in a generalized form rather than depending upon what I have stored in my book.

Dr Bob I understand what you are doing in Crafty and I agree with you about the importance of the opening
(the engine "participate in" choosing what is better to be played ). By the way, I have read your article about book learning.

I understand why you search book moves (But in my opinion it is somewhat contradictory to search a book move when it comes mainly to avoid the initial search and to lead your engine to known good open lines).

Couple of points. 1. The "search" is way shorter than a normal search, and is just used to make sure that the book move does not appear to be a blunder. Blunders are not that uncommon, particularly if you make a deep book. 2. I rarely use this mode in usual cases anyway, so it is sort of a "back-up" strategy when I am a bit skeptical of the book. Our usual tournament book is made from high-quality games so this is less of an issue, although it never goes away completely.

I agree with you (Dr Bob) about that issue but I still believe I can put all this stuff in the pre-build process.
This lead me to think that I can have the book code "detached" from my engine code (letting the GUI or polyglot doing this for me).

[/quote]

Then there is the issue of "learning". The engine is the thing that sees the scores and the resulting positions, and should be the one that decides what, if anything, is to be learned.

There is so much flexibility in how this is designed, it is not "generic" at all and is really a part of the engine, not the GUI.

[sje]

The GUI is not an engine and so should not be involved with move selection.

The one exception is for use in a thematic tournament. These aren't very common nowadays, but at one time there were such events where every player had to play the first N moves from a prescribed opening. Yet even here the same could be accomplished without move selection help from a GUI; only a "restore saved game" need be supported.

[Dann Corbit]

The GUI is not an engine and so should not be involved with move selection.

The one exception is for use in a thematic tournament. These aren't very common nowadays, but at one time there were such events where every player had to play the first N moves from a prescribed opening. Yet even here the same could be accomplished without move selection help from a GUI; only a "restore saved game" need be supported.

There are literally millions of computer games played in large, online contests/experiments where the opening book moves are provided by the GUI.

I don't see anything wrong with it. Besides thematic tournaments, there are also uniform book tournaments. I think that the people who use the chess engines should get to choose whether to use the engine supplied book or not. For instance, I literally NEVER use Rybka's stock book. The reason? Rybka's book is in ChessBase format and I do not use ChessBase to run tournaments or to perform analysis. So (depending on what I am doing) I either use no book or I use my own choice of book.

I will go farther. I think that the GUI should also provide the fundamental tablebase access. Then we could have:

Engine claim: 'tablebase draw'

GUI arbitration: 'tablebase draw accepted' (and KILL the other engine and restart it if it refuses to accept it).

IMO-YMMV

[hgm]

I am considering to add bitbase-based adjudication to WinBoard, but I am not sure that engines should be allowed to claim a draw. Knowing that some position is a draw is a lot easier than actually salvaging the draw. Suppose Chess has been solved by connecting all supercomputers in the univers for a couple of hundred years in parallel, and the opening position is proven a draw. Should any patzer like me now be allowed to claim a draw when fcing Rybka 3? In Checkers this situation already exists.

It would be very easy to me to make my engine test for KBNK, and claim "tablebase win" when it sees that matrial, without actually having the tablebase and with no idea how to effect the checkmate.

So I think it would be better to give this the status of an adjudication, which it really is. The GUI (by proxy of the user) can declare the game finished when it reaches a stage of a given number of men, irrespective of what the engines think about it. I think it would be very bad to design a protocol tht could allow engines to bluff their way to victory.