mvk wrote:
Ah, I should express my question more clearly: you intend to use these positions for testing. For that you need divergence: line A and line B should not naturally flow into the same continuation after one or a few moves out of 'book'. If you have a 'dense' opening selection, that effect might spoil a lot of your effort (getting duplicate games nonetheless).
I see. Yes, that is something I thought about, but unfortunately I don't see any simple way to find out. This is a problem every book potentially has. As you say, the denser it gets (high branching factor) the more likely it will happen. My branching factor is not so high now, with the more restrictive conditions (45cp threashold and prune quiet moves that do not improve the static eval). Hopefully the eval improving condition limits that kind of thing as moves tend to go "forward".
On second thoughts, perhaps it's too dense. And I don't need 141k positions anyway. So I extraced a random sample of the 5-ply file
Uri Blass wrote:
Why do you think that engines will never play a gambit?
I think that you are wrong about it and engines can sacrifice for positional reasons.
For example many engines play after 1.e4 d5 2.exd5 not Qxd5 but Nf6
Because engines don't "feel" positions, and openings neither.
IMHO, they will (very) rarely play a King's gambit, or some "unclear" gambit as there are more safe lines. So Lucas idea is good because will force the engine to move in positions it would never go otherwise.
I just played a few games between DiscoCheck and TogaII 3.0 (approx equal strength). I abort the games after 10 moves, because I'm only interested in how engines play the opening.
Looking at these games, I would say that Toga's play looks better in pure opening play. This is perhaps not measured properly by rating lists, because they force engines to play long mainlines, and leave them out of book already castled and developped. What a shame!
1... Nf6 2. Nf3 d5 3. e3 Nc6 4. Bb2 e6 5. Be2 Bd6 6. O-O O-O 7. c4 a6 8.
Nbd2 Bd7 9. d4 Be7 10. a3 Ne8 11. b4 1/2-1/2
[/pgn]
PS: You can see for yourself that engines do play gambits of their own! The difference to humans is that they do not play unsound gambits, of course.
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.
lucasart wrote:I've generated a 4 ply opening book using nothing more than my engine. Basically I start from the starting position, do a perft(4) and eliminate all nodes (interior and leaves) where a 12-ply search returns a score outside acceptable bounds +/-70cp: http://open-chess.org/viewtopic.php?f=5&t=2551
This was really a fun experiment, and provided me with a great opening suite for testing my engine. It's as diversified and shallow as it can be, and contains 52967 positions which enough for most engine testing purposes.
But it could be improved and generalized:
* code it in a GUI like XBoard, so it can be engine agnostic.
* make the diversity of the book (branching factor) tunable in a couple of ways: using multi PV feature of the engine (fixed factor) or tweaking the 70cp threshold.
* genrate a proper book in PolyGlot format, rather than an EPD file.
* parallelize the book generation, and do it recursively with a global hash table to eliminate more efficiently transpositions.
Has anyone ever done that ?
Is anyone interested ? HGM ?
It would be really great. The idea would be to create entirely computer based opening theory. Human opening theory is flawed and biaised: very poor diversity and many dubious lines that comp analysis show to be unsound.
PS: I can't attach files on this forum, so I use Open Chess. Ideally please answer on Open Chess instead (but most people are here and Open Chess is a bit dead).
Can you post also a pgn file with the opening lines from which each position was produced?
About this: Human opening theory is flawed and biaised. I prefer not to give my opinion as most of the current thoery are being validated with powerful engines.
Anyways, everyone is free to think what he/she thinks is "right".
lucasart wrote:But it could be improved and generalized:
* code it in a GUI like XBoard, so it can be engine agnostic.
* make the diversity of the book (branching factor) tunable in a couple of ways: using multi PV feature of the engine (fixed factor) or tweaking the 70cp threshold.
* genrate a proper book in PolyGlot format, rather than an EPD file.
* parallelize the book generation, and do it recursively with a global hash table to eliminate more efficiently transpositions.
Has anyone ever done that ?
Is anyone interested ? HGM ?
It would be really great. The idea would be to create entirely computer based opening theory. Human opening theory is flawed and biaised: very poor diversity and many dubious lines that comp analysis show to be unsound.
I have been working a lot on creating computer-based opening theory the past few months. But the main problem I had to combat was that the engine evaluation in the opening was not very reliable, and it often happened that positions that after a minute analysis still scored as -1.4 would end up at +0.5 after 20 min.
For this reason I did switch to using game-play statistics rather than engine eval to decide on the advantage in each visited position. This is what finally led to XBoard's -mcBookMode.
Uri Blass wrote:
Why do you think that engines will never play a gambit?
I think that you are wrong about it and engines can sacrifice for positional reasons.
For example many engines play after 1.e4 d5 2.exd5 not Qxd5 but Nf6
Because engines don't "feel" positions, and openings neither.
IMHO, they will (very) rarely play a King's gambit, or some "unclear" gambit as there are more safe lines. So Lucas idea is good because will force the engine to move in positions it would never go otherwise.
I believe that the king gambit is a bad gambit and chess programs can play correct gambits.
lucasart wrote:But it could be improved and generalized:
* code it in a GUI like XBoard, so it can be engine agnostic.
* make the diversity of the book (branching factor) tunable in a couple of ways: using multi PV feature of the engine (fixed factor) or tweaking the 70cp threshold.
* genrate a proper book in PolyGlot format, rather than an EPD file.
* parallelize the book generation, and do it recursively with a global hash table to eliminate more efficiently transpositions.
Has anyone ever done that ?
Is anyone interested ? HGM ?
It would be really great. The idea would be to create entirely computer based opening theory. Human opening theory is flawed and biaised: very poor diversity and many dubious lines that comp analysis show to be unsound.
I have been working a lot on creating computer-based opening theory the past few months. But the main problem I had to combat was that the engine evaluation in the opening was not very reliable, and it often happened that positions that after a minute analysis still scored as -1.4 would end up at +0.5 after 20 min.
For this reason I did switch to using game-play statistics rather than engine eval to decide on the advantage in each visited position. This is what finally led to XBoard's -mcBookMode.
I think that with strong chess engines it does not happen often that
positions that after a minute analysis still scored as -1.4 would end up at +0.5 after 20 min.
I'd like to do my own version of this (using different window and programs for verification).
I think I have tools for all of it except generating the successors. I'd also like to have the opening file be pgn instead of epd (so that one gets complete game scores from the normal starting position).
Does anybody know of a tool that takes a pgn file and then generates all the possible one-move continuation pgns from that? Or would a programmer be willing to help supply me with such a tool?
