Polyglot extension tool

[Rebel]

First of all download CoreTemp -- https://www.alcpu.com/CoreTemp/ it shows you how many threads you have, probably 8.

Settings
T=8
H=1024
L=28

Depth=28 may take a long time, about 1 minute per position. So you need approx 5000/8 = 625 minutes = 10 hours.

Doable in 1 run.

Still too long?

Split the 5000 epd positions into parts of 1000 or 2500 and combine the results later.

[Jonathan003]

Hi thanks ED, that helps a lot.

I don't know how many positions there are in the bin book, how can I know this? If I export the bin book to pgn with Lucas Chess I get 5000 different opening lines. (5000 games).

How can I split the epd file in 5 parts? Is there some software to do that? Or do I have to do it in a text editor?

[Guenther]

Hi thanks ED, that helps a lot.

How can I split the epd file in 5 parts? Is there some software to do that? Or do I have to do it in a text editor?

epd is just text (ascii), you can split/copy parts of it as you like and rename the parts. (same as fen and pgn)
(if you only have notepad you could use the free notepad++, because it has much more features and opens
bigger files and most important in your case it has of course a line count, as all real editors - so you can split e.g. after
each 1000th line easily)

example:

Code: Select all

r1bqkb1r/ppp2ppp/1nn5/4p3/8/2N2NP1/PP1PPPBP/R1BQK2R w KQkq - bm O-O d3; id "GS12.01";
rn1qk2r/pb1pbppp/1p2pn2/2p5/2P5/2N2NP1/PP1PPPBP/R1BQ1RK1 w kq - bm d4 Re1 b3; id "GS12.02";
rnbq1rk1/ppp1bpp1/4pn1p/3p4/2PP3B/2N2N2/PP2PPPP/R2QKB1R w KQ - bm e3; id "GS12.03";
r1bqk2r/pp1n1ppp/2pbpn2/3p4/2PP4/2N1PN2/PPQ2PPP/R1B1KB1R w KQkq - bm Be2 Bd3; id "GS12.04";
rnbqkb1r/1p3ppp/p3pn2/2p5/2BP4/4PN2/PP3PPP/RNBQ1RK1 w kq - bm Bb3 a4 Qe2; id "GS12.05";
rnbqk2r/pp2ppbp/3p1np1/8/3NP3/2N1B3/PPP2PPP/R2QKB1R w KQkq - bm f3; id "GS12.06";
r1bqk1nr/pp3pbp/2npp1p1/2p5/4PP2/2NP2P1/PPP3BP/R1BQK1NR w KQkq - bm Nf3; id "GS12.07";
rnbqkb1r/1p3ppp/p2ppn2/8/3NP3/2N5/PPP1BPPP/R1BQK2R w KQkq - bm O-O f4 Be3 a4; id "GS12.08";
r1bqkb1r/pp3ppp/2nppn2/6B1/3NP3/2N5/PPP2PPP/R2QKB1R w KQkq - bm Qd2; id "GS12.09";
rn1qkbnr/pp2ppp1/2p3bp/8/3P3P/6N1/PPP2PP1/R1BQKBNR w KQkq - bm Nf3; id "GS12.10";

[Ozymandias]

IMO the weakest point of the Polyglot book building process is that it equates the weights to 2*W+D, without paying any attention to losses. That means a move that was playes 10 times, with 5 wins and 5 losses, would have a smaller weight than a move that is played 100 times, with 10 wins and 90 losses. That seems wrong. Of course moves that would lose 90% of the time would not be played often in reality, if your game selection is not total crap. But still, this weighting method would underestimate promising novelties.

The main problem with Polyglot, is indeed that the weights ignore losses. This has the effect, not only, of underplaying novelties significantly, but also of overplaying those moves that are already the most played ones. The example you give isn't even an extreme one, try this: +100=0-0 would be played slightly less than +1=199-800. But the more played the move, the more it will be favoured by the weights. Inversely, the less games in the promising move, the less it will be played. +10=0-0 would be played significantly less than +0=2000-8000.

The proposed learning byte is a better approach, but ignoring draws also has its drawbacks (pun intended). It's not the same to play a move that has a track record of +10=0-0 than one with a thousand draws instead; the former is most likely a sure win, while the latter will most probably lead to a draw. But going with something like 2*W+D-2*L, would make things worse; you actually need to give draws a penalty. The simple W-D*1/10-L (or 1/1000 to a lesser degree) steers the game away from draws. Novelties would still be hard to spot, but I don't see an easy solution for that, nor do I to the fact that once you take a ply back, good results for a move get averaged (buried) with the stats of the alternatives in the upcoming position.

[hgm]

The philosophy behind the Polyglot weighting is that it is more significant that the move was played at all, than what the result was (which could be entirely due to errors made later in the game). Although there is probably some truth in this, IMO the Polyglot approach overestimates this aspect.

A very bad flaw in any statistics-based weighting is that it ignores the minimax aspect of Chess. When the 'value' of all daughter positions is known, the value of the position itself should follow from it, rather than being independently derived from its own statistics. Without that, you become vulnerable to the following case: a certain line does pretty well, and gets quite popular. Until a novelty is found that refutes it. After enough games to establish that the novelty indeed is a refutation, but not nearly as many as the line was originally played, people will stop playing it, and the statistics of the preceding moves will become frozen in a state that does not yet reflect that the line is refuted.

A sensible book-building algorithm should not just take statistics of individual positions at face value, but propagate it from the leaves of the opening tree to the initial position in a way that weights the statistics in proportion to how much it deserves to be played. Which would converge to minimax in the limit of large number of plays.

E.g. if a position has a number of moves, each played sufficiently often to be virtually sure that all alternative moves are significantly worse than the best, the statistics of these alternative moves should be completely ignored, and the move leading to the position should get the same statistics as the best move from it. How much sub-optimal moves should be weighted depends on how confident we are that they are indeed sub-optimal, and how sub-optimal they are. Note that even for moves that have been played so often that their statistics is very accurate, the weight should not be zero if they are only slightly sub-optimal, as it is good strategy to surprise your opponent through diversity. (Because this effectively lowers his strength due to lack of preparation.) This is basically the same problem as in Monte-Carlo Tree Search, where the conventional solution sucks as well.

[Rebel]

Not really because the counter can't go higher than 125 or lower than -125.

If a position has, let's say, two move options recorded, and both saturate the learn counter (that's what I mean when I talk about exceeding 125), how do you choose a move? I see a problem there.

Oh, that kind of problem, I thought you meant something more serious.

As the page states the right way of doing things is to add a corresponding file that maintains WLD 32 bit counters. I will do that later together with a util that changes the weight based on these counters.

[Rebel]

Hi thanks ED, that helps a lot.

I don't know how many positions there are in the bin book, how can I know this? If I export the bin book to pgn with Lucas Chess I get 5000 different opening lines. (5000 games).

How can I split the epd file in 5 parts? Is there some software to do that? Or do I have to do it in a text editor?

There are several utils that will do that for you:

40H-EPD - https://40h.000webhostapp.com/
SOMU - http://rebel13.nl/download/utilities.html#somu

But if you only have 5000 EPD lines I would use the advice from Guenther.

[Ozymandias]

the right way of doing things is to add a corresponding file that maintains WLD 32 bit counters. I will do that later together with a util that changes the weight based on these counters.

Wouldn't you be doing two calculations, where one would be discarded in favour of another one? Why not do the last one first and only?

[Dann Corbit]

The philosophy behind the Polyglot weighting is that it is more significant that the move was played at all, than what the result was (which could be entirely due to errors made later in the game). Although there is probably some truth in this, IMO the Polyglot approach overestimates this aspect.

A very bad flaw in any statistics-based weighting is that it ignores the minimax aspect of Chess. When the 'value' of all daughter positions is known, the value of the position itself should follow from it, rather than being independently derived from its own statistics. Without that, you become vulnerable to the following case: a certain line does pretty well, and gets quite popular. Until a novelty is found that refutes it. After enough games to establish that the novelty indeed is a refutation, but not nearly as many as the line was originally played, people will stop playing it, and the statistics of the preceding moves will become frozen in a state that does not yet reflect that the line is refuted.

A sensible book-building algorithm should not just take statistics of individual positions at face value, but propagate it from the leaves of the opening tree to the initial position in a way that weights the statistics in proportion to how much it deserves to be played. Which would converge to minimax in the limit of large number of plays.

E.g. if a position has a number of moves, each played sufficiently often to be virtually sure that all alternative moves are significantly worse than the best, the statistics of these alternative moves should be completely ignored, and the move leading to the position should get the same statistics as the best move from it. How much sub-optimal moves should be weighted depends on how confident we are that they are indeed sub-optimal, and how sub-optimal they are. Note that even for moves that have been played so often that their statistics is very accurate, the weight should not be zero if they are only slightly sub-optimal, as it is good strategy to surprise your opponent through diversity. (Because this effectively lowers his strength due to lack of preparation.) This is basically the same problem as in Monte-Carlo Tree Search, where the conventional solution sucks as well.

I think your reply is quite well said, and contains many interesting ideas.

I would add the following notion:
The book building engine should first alpha-beta mini-max the data provided.
This could (and probably should) be done in two different ways. We can mini-max with scores based on computer analysis, and we can mini-max with scores based on wins/losses/draws.
Now, of course, some of the suggested moves will be different.
We should have an engine that explores the disagreements and tries to resolve them. It might analyze both nodes, and child nodes and possibly also schedule a tournament.
The book should be flexible so that it can be improved with new data over time without having to perform a full rebuild, though a full rebuild would be beneficial, it would also be slow and consume a lot of resources.

It might also be possible to choose the method of scoring a node based on total games played and total depth of analysis.
If we can determine an equality for number of games played compared to the depth of analysis, then we can use game count or depth to decide which analysis to prefer.

For instance, suppose that having 500 games played is equivalent in quality to analyzing to a depth of 41 plies.
Then, if I have 500 games played but the depth is only 38 plies I will prefer using win/loss/draw statistics.
Or, if I have a depth of 60 plies but only 500 games played, then I will trust the computer analysis instead.
Something like that.

This (of course) assumes a database of very high quality games with players of known strength and top-tier engines of known strength.

[Rebel]

This (of course) assumes a database of very high quality games with players of known strength and top-tier engines of known strength.

This is in preparation and will be released soon. Kind of FRITZ' let's check feature but made from engines rated 3000+ single threaded. From the start position we get:

Code: Select all

Reference : consult\3000.dat
Positions : 10.013.033

Moves found for rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq -

Move Depth Score Engine
e2e4   26  0.61  Stockfish 10
d2d4   28  0.50  Stockfish 9
e2e4   22  0.19  Komodo 11
d2d4   23  0.43  Komodo 11
d2d4   24  0.27  Stockfish 8
e2e4   21  0.18  Stockfish 7
g1f3   21  0.21  Stockfish 6
e2e4   23  0.30  Stockfish 6
e2e4   20  0.31  Stockfish 5
e2e4   18  0.20  Stockfish 4
e2e4   19  0.28  Stockfish DD
e2e4   20  0.34  Komodo 10
e2e4   16  0.28  Komodo 9
g1f3   21  0.38  Komodo 9
e2e4   18  0.23  Komodo 8
e2e4   18  0.23  Komodo 7
e2e4   17  0.20  Komodo 6
e2e4   16  0.23  Komodo 5
d2d4   18  0.30  Komodo 5
d2d4   17  0.23  Komodo TCEC
e2e4   18  0.46  Houdini 4
d2d4   20  0.34  Houdini 4
d2d4   18  0.28  Houdini 3
e2e4   19  0.21  Houdini 3
d2d4   19  0.24  Houdini 2
e2e4   16  0.18  Fire 7
e2e4   17  0.10  Fire 5
e2e4   23  0.28  Shredder 13
e2e4   17  0.34  Fizbo 1.6
e2e4   18  0.25  Fizbo 1.7
d2d4   20  0.02  Fritz 14
e2e4   18  0.17  Fritz 15
d2d4   25  0.15  Fritz 16
e2e4   21  0.40  Chiron 2
d2d4   23  0.25  Chiron 2
d2d4   21  0.21  Chiron 3
e2e4   23  0.34  Chiron 4
e2e4   16  0.25  Gull 2
g1f3   17  0.18  Gull 2
e2e4   14  0.21  Gull 3
e2e4   19  0.12  Equinox
d2d4   20  0.23  Equinox
e2e4   18  0.14  Critter 1.6
g1f3   19  0.15  Critter 1.6
d2d4   20  0.17  Critter 1.6
b1c3   15  0.14  Rybka 4
e2e4   16  0.15  Rybka 4
d2d4   19  0.14  Hannibal 1.4
g1f3   20  0.17  Hannibal 1.5
d2d4   21  0.15  Hannibal 1.5