Mysterious behaviour in book lines explained ?

[bob]

But I'd bet nobody stores moves, as it is beyond redundant when you already have the Zobrist keys...

That's one lost bet
I do. And I have a strong suspicion that SMK does that in Shredder, and Matthias Feist in the .ctg format.

As far as I'm concerned, it is not about blocking positions. It is about blocking positions from certain move orders.

That makes no sense to me. Position P is equal to position Q if both positions have the same possible moves. Why on earth would the moves that led to the two positions somehow make me accept one, but not the other? Because once I get to the position, I have the _same_ possible moves to make, so what would make me want to play move X in position P one time, and not play move X in the same position when reached from another series of moves???

I'm ignoring potential repetitions which should not be an issue in book lines of course..

Example:
Suppose Crafty is black and you decide to play a Petrov in a tournament against a specific opponent. Will it go for a Ruy in the following line:
1.e4 e5 2.Nf3 Nf6 3.Bb5
with 3..Nc6?
Of course, there's nothing wrong with the Ruy. But perhaps you should not be happy about it as you:
1. Decided to avoid the Ruy with the Petrov
2. May have better moves to reach an advantage (like c6 followed by d5, or by a direct Nxe4), throwing the both of you out of book.

I solve this in a different way. Because there is a worse problem.

1. e4 e5 Nf3 Nf6 Bb5 a6 O-O and now are you really going to play Nc6 as it will almost certainly be your only move to transpose into the book? Would you not choose to play axb5 and win immediately? This can be solved easily (I have done it in Crafty already and explained it previously).

So no, I don't worry about that case of transpositions because it is not an issue for me. I will miss a few transpositions I would prefer to not miss, but I don't make insane blunders either just to transpose back into a book line..

This is just one example, but the initial poster of this thread obviously had some lines where he wanted to block a move instead of a position.

Richard.

Again, why? There is no way to allow good transpositions and block bad ones without a huge cost in size, not to mention the ugly hand-tuning that is needed. In my case above, Crafty will be out of book. If Nc6 is the best move, I will once again be back in book and start to play book moves normally. But I won't make a "connector" move to transpose back in unless the search says it is the best move...

[bob]

Robert does this to, but implicitly.

In Crafty, part of the book zobrist key depends on the parent. So blocking a position means blocking it from (through) that particular parent.

Tony

Ah, I looked in the Crafty code just now (20.14 but I doubt it changed much in this area lately) and saw the construction of the 'temp_hash_key' for each move from the root position in book.c. So basically Crafty uses rootposition (48bits)+position-after-bookmove(16 bits), encoded in a single zobrist key. Indeed, that seems equivalent to my position (64 bits)+move(16 bits) to me. Only difference is that I use a longer zobrist key for the position.
Richard.

Actually that is backward. I use the high-order 16 bits from the parent position Zobrist key, and the lower 48 bits from the actual position zobrist key. This has the effect that all positions produced from the same parent position have the upper 16 bits identical. When I sort the keys before writing them out, all positions from a single parent position are sorted together so that I don't have to wildly probe all over the file. And it solves the ugly transposition problems with no extra effort required.

[pijl]

As far as I'm concerned, it is not about blocking positions. It is about blocking positions from certain move orders.

That makes no sense to me. Position P is equal to position Q if both positions have the same possible moves. Why on earth would the moves that led to the two positions somehow make me accept one, but not the other? Because once I get to the position, I have the _same_ possible moves to make, so what would make me want to play move X in position P one time, and not play move X in the same position when reached from another series of moves???

I think we are now talking about the same thing in a different way.
I store a root position + all moves I allow to be played from that position.
How it got there is not an issue. Once it gets there it is allowed to play any move it has in the book.

The method I thought you were talking about was the one I had in the Baron, where only the target positions of bookmoves are stored, without the original position. This method has the capability of finding its own way back into the book. In that case you could also get the ugly moves from the book that we both came up with (and where you have to deal with repetition issues in games like 1.e4 e5 2.Nf3 Nc6 3.Ng1 Nb8??).

But the method Crafty uses, which is described in another post of this thread would be equivalent to storing position+move, but encoded in a different way.
So as far as I'm concerned we do comparable things.
Richard.