can stockfish with book beat leela without book?

[Uri Blass]

I read the following opinion in the lc0's forum:

It's not only a question of whether openings should be used, but also in what way. I can't be sure, but given the almost equal result, I suspect that Leela would be able to win if:
a) Openings were used to provide variety, but for less plies, and not as sharp, or
b) No opening lines were forced but stockfish could use an opening book

I do not believe that b is correct but I may be wrong and I wonder if it is possible to test stockfish using BookX.bin against lc0 with no book to see what are the results(other conditions should be the same as the last lc0-stockfish match from TCEC when lc0 won with no books).

I know that in the past BookX.bin helped stockfish against lc0 but I believe lc0 improved from that time and it was also at faster time control than the last TCEC match
when I guess a book can help more at faster time control.

[jp]

If in b) you give SF the biggest book possible, Lc0 won't win a single game.

Lc0 maybe's got stronger since Kai's book tests, but won't the effects still be the same qualitatively?

[Laskos]

I read the following opinion in the lc0's forum:

It's not only a question of whether openings should be used, but also in what way. I can't be sure, but given the almost equal result, I suspect that Leela would be able to win if:
a) Openings were used to provide variety, but for less plies, and not as sharp, or
b) No opening lines were forced but stockfish could use an opening book

I do not believe that b is correct but I may be wrong and I wonder if it is possible to test stockfish using BookX.bin against lc0 with no book to see what are the results(other conditions should be the same as the last lc0-stockfish match from TCEC when lc0 won with no books).

I know that in the past BookX.bin helped stockfish against lc0 but I believe lc0 improved from that time and it was also at faster time control than the last TCEC match
when I guess a book can help more at faster time control.

a) is probably true, b) is debatable. Maybe using latest BraniFish is even better than BookX for SF. As of now, T30 is marginally the strongest at 60s + 1s time control, not sure at 240s + 4s, as T40 of today is very close to T30 at 60s + 1s TC, but T40 seems to scale better. They just dropped LR some 2 days ago, I expect further gains soon, maybe I will perform in 1-3 days a test of T40 latest net against the latest BrainFish at 60s + 1s (100 games maybe). BookX gave some 80 Elo points boost to SF against Lc0 (no book) compared to 3-4 mover regular GM openings (used by both), probably Cerebellum will fare even better, but as you said, the advantage of the book diminishes with time control.

Also, your entire question is dependent on hardware GPU/CPU balance, my balance is somewhat skewed towards GPU. And as stated, the answer depends on time control used too.

[Uri Blass]

If in b) you give SF the biggest book possible, Lc0 won't win a single game.

Lc0 maybe's got stronger since Kai's book tests, but won't the effects still be the same qualitatively?

Biggest book possible does not make sense because it is going to include many bad lines(for example 1.f3 e5 2.g4 for white)
I doubt if there is a combination of a book and an engine that you can be sure that LC0 won't win a single game against the combination.

[Nay Lin Tun]

For TCEC hardware, I still put my bet on Lco that Lco without book is still better than SF + brainfish (or whatever book you can get on earth) in adjusted allocated time for book use.

What I mean is that , if you give book to one side, it should not be time cheat to one side. In spcc testing, I noticed that Brainfish has approx >+ 80/+100 elo above SF in 1 min bullet. In fact brainfish got not only better opening line but also about 30% time cheat which is non-sense.

If you would like to use brainfish book for stockfish, (use whatever depth the book have, may be depth 40- 60) but just start your time control only when both engines are out of book.

[jp]

If in b) you give SF the biggest book possible, Lc0 won't win a single game.

Biggest book possible does not make sense because it is going to include many bad lines(for example 1.f3 e5 2.g4 for white)
I doubt if there is a combination of a book and an engine that you can be sure that LC0 won't win a single game against the combination.

It won't happen, of course, but if a top GM allowed SF to borrow his opening book, it's very unlikely Lc0 would win a game. When A0 played SF8 with a book (not the latest SF, not the best book), it was almost all draws.

[Leo]

What is BookX.bin ?

[Laskos]

I read the following opinion in the lc0's forum:

It's not only a question of whether openings should be used, but also in what way. I can't be sure, but given the almost equal result, I suspect that Leela would be able to win if:
a) Openings were used to provide variety, but for less plies, and not as sharp, or
b) No opening lines were forced but stockfish could use an opening book

I do not believe that b is correct but I may be wrong and I wonder if it is possible to test stockfish using BookX.bin against lc0 with no book to see what are the results(other conditions should be the same as the last lc0-stockfish match from TCEC when lc0 won with no books).

I know that in the past BookX.bin helped stockfish against lc0 but I believe lc0 improved from that time and it was also at faster time control than the last TCEC match
when I guess a book can help more at faster time control.

Yes, at 4min + 4s time control, Lc0 T41367 beats soundly latest SF_dev + BookX.bin in Cutechess-Cli UI. Unfortunately, the variety of games was not great, I am not sure why, polyglot book choices were constricted in Cutechess and Lc0 itself played almost deterministically. The result in 40 games was:

Score of lc0_41367 vs SF_dev_BookX: 7 - 2 - 31 [0.563] 40
Elo difference: 43.66 +/- 50.34
Finished match

LOS=95%

Also, it seems that above 2min + 2s time control on strong RTX GPU, T40 as of today is stronger than the best T30 nets (and scales better to longer time control). T40 seems to have a bright future at TCEC and CCC, and SF won't probably win finals anymore in those conditions, aside from very weird opening positions.

[Jouni]

In CCC 6 Stockfish is dominating clearly against four Leelas including latest T40. Any excuses ?

[Dann Corbit]

A good book is worth about 100 Elo

https://sites.google.com/site/computers ... book-cs-23
SCCT XXIII's Champion: Attacker - Congrats to Adrian Costinescu!

Code: Select all

        # PLAYER                : RATING    POINTS    GAMES   (%)
       1 Attacker Pro 3        : 3562.9     866.5    1460   59.3%
...
       92 No Book               : 3451.3     281.0     640   43.9%

Difference of 111 Elo (give or take a little).

Now, SF recently won TCEC so it is likely a little stronger on equal ground.
Hence with a book, given a long tournament, winning is likely.

But consider all the unstated variables...
LC0 Elo seems to grow faster than SF Elo.
Shall we give LC0 four of these 2080 TI cards?
https://www.segmentblue.com/best-graphi ... tx-2080-ti
Or how about a whole rack of these:
https://www.nextplatform.com/2018/05/10 ... processor/
But maybe we give Stockfish the "128 cpu's x32 threads Cluster System" that gets 4,801,341,606 NPS using 4096 threads described here:
http://www.ipmanchess.yolasite.com/amd- ... -bench.php

Of course the real question we are all asking ourselves is, "What does the future hold for chess engines?"
We can wonder what direction things will take. Will a hybrid strategy be best?
But of course, nobody knows the answer to any of these questions.
Still, it's always fun to discuss how many angels can dance on the head of a pin.
I think it's 13.