hgm wrote: ↑Mon Dec 17, 2018 9:29 am
So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.
If an engine's "built-in opening repertoire sucks", it's only because the authors & users always intended to use a book with it, so there was no reason to code something special for bookless use. Why this obsession with making engines bookless? Taking away something the authors & users assume will be used & then calling it a "weak spot"...
It wouldn't be hard to make an engine's bookless opening play "suck less" in practice.
jp wrote: ↑Sun Dec 16, 2018 11:06 pmIf you define "having an opening repertoire" as "playing deterministically", then yeah.
But that sort of "opening repertoire" may be a very bad one.
So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.
My conclusion from the results is that lc0 does not like BookX.bin and BookX.bin leads to positions that lc0 does not know to play well(and they are not objectively better positions)
so I can call BookX.bin anti-Leela book.
If I understand correctly BookX.bin ends at move 8.
On the other hand if we use stockfish to build a book based on the moves that it prefers(in the first 8 moves) and call it Booky.bin then Booky.bin
is not good against Lc0 and the question is why it is not good.
There can be 2 different reasons.
reason 1:Stockfish does not know how to play the positions out of Booky.bin that it get against Lc0 and the positions are objectively equal
reason 2:The positions that stockfish get out of Booky.bin are bad
It may be interesting to know what is the correct reason or maybe stockfish's bad results with Booky is by combination of reasons 1 and 2
hgm wrote: ↑Sun Dec 16, 2018 9:05 pm
Any engine has an opening repertoire by definition. Stockfish also prefers some opening moves over others, and would considtently play these.
The game of chess has an opening repertoire too. Some openings draw but many lose, plausibly even as early as the 2nd or 1st move. For example, the Alekhine's defense may be a loss for black, and the King's gambit for white -- and there are worse first moves for white than e4.
with perfect2017 i'm getting pretty even results with rolling test30 nets vs SF10. scoring about +25 elo against SF with the book vs without it; SF10 vs SF10+book is about the same. i'm using absolute FPU & haven't checked for opening variety etc, but i'd be curious if test30 fares any better in kai's conditions.
hgm wrote: ↑Mon Dec 17, 2018 9:29 am
So the meaningful thing to say is not that "AlphaZero has an opening repertoire", but "in contrast to AlphaZero, Stockfish' built-in opening repertoire sucks". Whether the line of reasoning "this is a weak spot of Stockfish, so it would not be fair to include it in any strength test" makes any sense... Well, I suppose anyone can judge that for himself.
If an engine's "built-in opening repertoire sucks", it's only because the authors & users always intended to use a book with it, so there was no reason to code something special for bookless use. Why this obsession with making engines bookless? Taking away something the authors & users assume will be used & then calling it a "weak spot"...
It wouldn't be hard to make an engine's bookless opening play "suck less" in practice.
I think that your assumption is wrong at least for stockfish.
The authors of stockfish do not use a big book or a special strong book when they test changes in the code because they assume stockfish will not need to play the opening by itself.
The book that they use is 2moves_v1.pgn that based on my understanding contain 2 random moves by white and black.
Uri Blass wrote: ↑Mon Dec 17, 2018 8:43 pm
I think that your assumption is wrong at least for stockfish.
The authors of stockfish do not use a big book or a special strong book when they test changes in the code because they assume stockfish will not need to play the opening by itself.
The book that they use is 2moves_v1.pgn that based on my understanding contain 2 random moves by white and black.
They are not specifically testing for opening play & there is no code specific for opening play.
If they were trying to develop its opening play, they would not be using 2moves_v1.pgn.
Uri Blass wrote: ↑Mon Dec 17, 2018 8:43 pm
I think that your assumption is wrong at least for stockfish.
The authors of stockfish do not use a big book or a special strong book when they test changes in the code because they assume stockfish will not need to play the opening by itself.
The book that they use is 2moves_v1.pgn that based on my understanding contain 2 random moves by white and black.
They are not specifically testing for opening play & there is no code specific for opening play.
If they were trying to develop its opening play, they would not be using 2moves_v1.pgn.
I do not understand.
2moves_v1.pgn means that stockfish play by itself in the opening in testing so changes that help in the opening stage can be productive.