Polyglot - merge-book oddity

[jefk]

HGM wrote

'probing code set the weights to percentage of total'

well yes, and then they add up to 100, don't they. Eg. in SCID, with which i've
set most percentages for my gambit book(s) manually (well at at least
for the first few moves. And i hope/presume that the program is choosing
the frequency of such moves based on such percentages, but i could be wrong.
It's for a gambit training program for human chess players:
https://sourceforge.net/projects/chess-gambiteer/
and it contains quite some odd/rare gambits, so it also has not
been based on real games (with a pgn). Using an engine score
for such positions would be useless, btw, because it would filter
out the gambit moves, which have positional compensation which
sometimes even the modern Nnue engines cannot easily understand.
For high level (eg. correspondence chess) i don't use polyglot anyway
but Aquarium...

PS about WDL or engine score in book ? (and/or with minimax) well in
the past this was discussed here, and it was suggested that a linear
combination of statistics (from a game database) and engine score would be
most robust (in a Chess Assistant type of book it can be done but there's
no minimax). There's some simple calculation from centi-pawns to
percentage and vice versa nowadays, so doing this should be easy.
Based on the 3 variables of WDL (or actually only 2 coz they add up to 100)i
suggest a single score
of Sc1= W/(W+L), SC2 engine score (in win pct) combine the two scores
(with one parameter to value either statistics or engine score), and then
minimax on endnodes; for fast minimax i still have some awesome code btw
which i might upload to sourceforge (rather soon, if you wish).

Yet such a method still is neglecting 'sharpness' of the position (or
possibly of the playing style) which I mentioned earlier in this forum
on some occasions.. IMO for different playing styles you need different
books, aka for humans a different repertoire. For a sharpness or
'Unbalanced'-ness parameter I would suggest something simple as eg
U =(100-D)/100 with D the draw percentage. Then, an aggregated score would
be Sc-new = Sc + c* U with an extra (UCI) parameter c , somewhere between
zero and 0.3 or so. And then again the minimax
NB at the end the Sc-new scores should be 'normalized' again to 100
before calculating back to centipawns ofcourse. The whole thing is
a matter of prototyping, and trying to see if its possible anyway
with the current Polyglot coz i understood there's unfortunately not so
much bits to accommodate for such extra parameters...

PS something similar (sharpness) could actually be used in modern engines
and rather then 'contempt', it may actually lead to more attractive/aggressive
playing style, if the user would like to do so (with the parameter).

PS2 meanwhile i haven't tested the very latest polyglot bookmerge
but i'll gladly leave that to others :_)

[Ozymandias]

Weights purely from statistics are flawed anyway. You would have to minimax the opening tree in MCTS fashion to correctly judge the moves.

MCTS may be good for playing a game based on partial statistics (not all moves have been searched to the end), but when you have results for all stored positions, why not doing something better? If you're going for the most played move, polyglot pretty much does that already.

[hgm]

MCTS does not go for the most-played move, but for the move that minimax-wise would give you the best average result.

Isn't that theoretically the best you can do? If it was possible to do better, why wouldn't MCTS use that possibility?

I am not sure what you mean by "having results". The WDL statistics?

[Ozymandias]

In the Wikipedia article about MCTS they say "Rounds of search are repeated as long as the time allotted to a move remains. Then the move with the most simulations made (i.e. the highest denominator) is chosen as the final answer. " Isn't that the most played move?

Having results = playing games to the end, in MCTS many nodes haven't been played to the end by the time a move has to be made.

As for being the best way of doing it theoretically, I don't know about that when dealing with opening books. The information contained has been compiled over years, not seconds, and it should be taken more seriously than going for "promising" lines.

[jefk]

ozymandias wrote

the best way of doing it theoretically

from other games, it was found that combining a positional (engine) score
with actual played game statistics, gives the best result.
This also is my practical experience, with minimax the engine scores
in a book, you still can get a lot of holes, because it depends on sidelines
(and the engine quality) how good the result will be.
With nowadays Nnue engines apparently able to give a WDL score (*),
it shouldn't be to difficult to make a weighted average with
actual WDL from GM games; then calculate the resulting W/(W+L)
and only then do a full minimax, that should give decent (ie. much
more robust) results.

(*) it also should be possible to combine engine scores in such a way,
with certain weights eg. SF/Dragon/Dragon-MCts etc
Rather then upgrading Polyglot i suppose an extra tool, ie a new book editor
(plus interface) for polyglot books could to the trick(s) eg similar to this tool
(but then better again, and for polyglot books); maybe we can ask pkoziol
(Glass wasn't open course, is my impression)
http://www.pkoziol.cal24.pl/glass/download.htm

PS as for my sharpness suggestion, where i proposed U(nbalanced= 100-D.
this has a theoretical problem that when L is increased, U also is increased,
so a better U parameter probably would be : U = W/(W+D); simple isnt it.
Again, Sf Nnue could maybe already use such a parameter if the Nnue
indeed gives results such as with Leela
https://lczero.org/blog/2020/04/wdl-head/
NB setting a higher U would then lead to more dynamic, speculative
play i presume. Something worth a try IMO, both for an external
book editor as mentioned above, and for the SF engine(s).
And you want could get eternal fame if Chessbase later-on
copies the concept and sells it as FF3
jm2c

[Rebel]

Well, I uploaded the already compiled fixed version from HGM here:
https://drive.google.com/drive/folders/ ... E15NjVyY2M

I will remove this upload now, as HGM has done more essential fixes later anyway.
If there is still interest I could compile the newer fixes and upload again.
Just make a request here or via PM.

Is your compile a separate executable or an Polyglot update?

In case of the latter that would be a serious change as it affects the weights in book-1, not everybody wants that.

[Ozymandias]

from other games, it was found that combining a positional (engine) score
with actual played game statistics, gives the best result.

What engines do best is play, a more or less accurate evaluation is a byproduct of how they're tested. If the DB has reliable information, score should be best.

[hgm]

In the Wikipedia article about MCTS they say "Rounds of search are repeated as long as the time allotted to a move remains. Then the move with the most simulations made (i.e. the highest denominator) is chosen as the final answer. " Isn't that the most played move?

Having results = playing games to the end, in MCTS many nodes haven't been played to the end by the time a move has to be made.

As for being the best way of doing it theoretically, I don't know about that when dealing with opening books. The information contained has been compiled over years, not seconds, and it should be taken more seriously than going for "promising" lines.

In MCTS all nodes should have been played to the end at least once through a 'rollout'.

When you conduct the search yourself, you would have a rule that determines which line to follow to a leave where you can do an additional rollout (such as UCT). If you interpret an existing (opening) tree you obviously don't have that freedom, but you can still weight the results with the frequency you would have played them considering their (WDL) scores.

[Guenther]

Well, I uploaded the already compiled fixed version from HGM here:
https://drive.google.com/drive/folders/ ... E15NjVyY2M

I will remove this upload now, as HGM has done more essential fixes later anyway.
If there is still interest I could compile the newer fixes and upload again.
Just make a request here or via PM.

Is your compile a separate executable or an Polyglot update?

In case of the latter that would be a serious change as it affects the weights in book-1, not everybody wants that.

I don't quite understand your question?

It should be obvious from the thread reading that I just compiled HGMs first change (for everyone who was interested) and did a quick test on it.
Also in your reply, one can actually see my quote, that I had deleted it meanwhile and the reason for the deletion is also given...

[Ozymandias]

In MCTS all nodes should have been played to the end at least once through a 'rollout'.

Doesn't the latest one get interrupted by the TC? If the engine waited for them all to finish, wouldn't it lose on time?

Books could be mini-maxed and that would be an improvement over current polyglot 2*W+D, but this still leaves undesirables. For example, in the fool's mate, if you have a PGN with as many 2... Qg5 as 2... Qh4#, you may go for 2. g4. I see no reason to leave Qg5 for the mini-max process to deal with.