End of Era is there: SF is finally beaten!

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
brianr
Posts: 356
Joined: Thu Mar 09, 2006 2:01 pm

Re: End of Era is there: SF is finally beaten!

Post by brianr » Sat Feb 16, 2019 11:03 pm

Graham Banks wrote:
Sat Feb 16, 2019 9:40 pm
http://www.computerchess.org.uk/ccrl/40 ... 089_64-bit

The CPU version is improving rapidly too.
Just looking to confirm that this is Leela on only one CPU (the BLAS backend I would think), not running with a GPU, correct?
Thanks.

User avatar
Graham Banks
Posts: 33018
Joined: Sun Feb 26, 2006 9:52 am
Location: Auckland, NZ

Re: End of Era is there: SF is finally beaten!

Post by Graham Banks » Sat Feb 16, 2019 11:14 pm

brianr wrote:
Sat Feb 16, 2019 11:03 pm
Graham Banks wrote:
Sat Feb 16, 2019 9:40 pm
http://www.computerchess.org.uk/ccrl/40 ... 089_64-bit

The CPU version is improving rapidly too.
Just looking to confirm that this is Leela on only one CPU (the BLAS backend I would think), not running with a GPU, correct?
Thanks.
Correct. Gabor ran the gauntlet.
My email addresses:
gbanksnz at gmail.com
gbanksnz at yahoo.co.nz

chrisw
Posts: 2009
Joined: Tue Apr 03, 2012 2:28 pm

Re: End of Era is there: SF is finally beaten!

Post by chrisw » Sat Feb 16, 2019 11:24 pm

George Tsavdaris wrote:
Sat Feb 16, 2019 8:41 pm
corres wrote:
Sat Feb 16, 2019 4:37 pm
There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 20.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 21.Bg6 move.
From net30100 to net32000 Leela want to move 21.Be4, 21.Bxc4, 21.0-0.
But using net32500 Leela finds 21.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 21.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.
:shock: :shock:
You have ZERO idea about how Neural Net engines work.
Does anybody? Can you make a non-tautological explanation in, say 300 words, “how neural net engines work” that we can cut out and put in a frame on the wall?

Uri Blass
Posts: 8557
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: End of Era is there: SF is finally beaten!

Post by Uri Blass » Sun Feb 17, 2019 3:35 am

syzygy wrote:
Sat Feb 16, 2019 7:27 pm
Leo wrote:
Sat Feb 16, 2019 6:51 pm
corres wrote:
Sat Feb 16, 2019 5:58 pm
corres wrote:
Sat Feb 16, 2019 4:37 pm
...
There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 19.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 20.Bg6 move.
From net30100 to net32000 Leela want to move 20.Be4, 20.Bxc4, 20.0-0.
But using net32500 Leela finds 20.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 20.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.


I am sorry but I miss-wrote the number of moves.
The text with good move numbers is above.
I agree 100 %. It has to calculate anew the opening every single game. LCZ has already done that before the games starts. Of course these have been the rules for the TCEC for years and it is impressive that a new engine can do what it is doing.
I agree 0%. The probability that LCZ between net32000 and net32500 has seen that specific position is much smaller than 0.000001%. Between net32000 and net32500, LCZ may somehow have learned to assess that position correctly, but that is no reason to believe that LCZ has seen exactly that position.

From net32000 and net32500, LCZ has learned new patterns and one or more of those patterns is what has allowed it to find the right move in that position (and in many other positions). This has nothing to do with having a "good book".
I partly agree.
Having a good book can help also in positions that you never saw in the past.
If you have a book with a lot of chess positions and evaluations of them then you can learn from the book to evaluate better also positions that are not in the book.

The idea that I have is to divide the positions to classes(let decide about classes based on material on the board and pawn structure and maybe it is possible to come with a better idea of division of the positions to classes).

If positions in the same class happen enough times in the database then you can calculate a bias for the class based on difference between static evaluation and evaluation after deep search for all positions in the same class and add the bias to the evaluation function when the program get a position in some class to evaluate.

Note that I believe this idea can work only with many positions otherwise a big part of the positions are in classes that do not happen enough times to get conclusions.

MikeB
Posts: 3393
Joined: Thu Mar 09, 2006 5:34 am
Location: Pen Argyl, Pennsylvania

Re: End of Era is there: SF is finally beaten!

Post by MikeB » Sun Feb 17, 2019 5:27 am

chrisw wrote:
Sat Feb 16, 2019 11:24 pm
George Tsavdaris wrote:
Sat Feb 16, 2019 8:41 pm
corres wrote:
Sat Feb 16, 2019 4:37 pm
There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 20.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 21.Bg6 move.
From net30100 to net32000 Leela want to move 21.Be4, 21.Bxc4, 21.0-0.
But using net32500 Leela finds 21.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 21.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.
:shock: :shock:
You have ZERO idea about how Neural Net engines work.
Does anybody? Can you make a non-tautological explanation in, say 300 words, “how neural net engines work” that we can cut out and put in a frame on the wall?
I certainly cannot - but for those who are willing to read more than 300 words , this is straight from the horse's mouth:

https://www.dropbox.com/s/rp2foutpse7l7 ... 5.pdf?dl=1 It is about succinct as it could be considering the subject matter. Pages 2 thru 6 and and Pages 9 thru 15 are the must reads - others pages can be skipped.

jp
Posts: 755
Joined: Mon Apr 23, 2018 5:54 am

Re: End of Era is there: SF is finally beaten!

Post by jp » Sun Feb 17, 2019 5:36 am

MikeB wrote:
Sun Feb 17, 2019 5:27 am
I certainly cannot - but for those who are willing to read more than 300 words , this is straight from the horse's mouth:
It is about succinct as it could be considering the subject matter. Pages 2 thru 6 and and Pages 9 thru 15 are the must reads - others pages can be skipped.
I don't think that's what Chris meant. That's just saying what they did (and saying it very badly -- that's the v.1 before the reviewers forced them to make changes to make it slightly less bad). What they did is different from philosophical ideas of "how" that often make the mistake of anthropomorphizing computer code ("learn", "plan", "Understand", etc.).

User avatar
M ANSARI
Posts: 3408
Joined: Thu Mar 16, 2006 6:10 pm

Re: End of Era is there: SF is finally beaten!

Post by M ANSARI » Sun Feb 17, 2019 6:25 am

Here is an example of where SF and other AB engines are still superior. Lc0 had totally outplayed SF and tenaciously held an advantage and finally reached a totally winning simplified endgame.

[d]8/2b1k3/8/2pP4/2P1p1N1/8/5PK1/8 w - - 0 72


Here Lc0 played f3! forcing the exchange of the pawn and reaching a winning position. SF and Blue fish and Komodo all had mate scores or something around +153 which is almost the same. Lc0 also had an evaluation of around +9 ... so it seemed it was game over! However, Lc0 just couldn't see the win !!! And ended up drawing. To be fair, I tried the position out against SF and I too was not able to win the position, so it certainly is not an easy win as the Bishop can coordinate really well and you need some waiting moves to slowly let your King invade and close out blocking squares. However that should be trivial for an engine, especially if it has 6 piece EGTB's and there are only 7 pieces remaining. One more thing that needs to be looked at with Lc0. Here is the full game


corres
Posts: 1578
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: End of Era is there: SF is finally beaten!

Post by corres » Sun Feb 17, 2019 7:30 am

corres wrote:
Sat Feb 16, 2019 10:03 pm
George Tsavdaris wrote:
Sat Feb 16, 2019 8:41 pm
corres wrote:
Sat Feb 16, 2019 4:37 pm
There is no any "spectacular" and there is no any "plan" on the side of Leela.
In reality the position after 19.-c4 and the good continuation is in the 32930net of Leela.
To prove this I used the weak GTX 1060 to find the 20.Bg6 move.
From net30100 to net32000 Leela want to move 20.Be4, 20.Bxc4, 20.0-0.
But using net32500 Leela finds 20.Bg6 immediately!
So anywhere between net32000 and net32500 Leela has analysed yet this position during self play
and 20.Bg6 is in the NN of Leela.
Obviously Stockfish also would be much more stronger if it could use a good book instead of analysing every each position.
You have ZERO idea about how Neural Net engines work.
Thanks.
But if during self play LC0 never move in this position Bg6, from what she knows Bg6 is a good move?
For the exact answer we should look through the whole self play game between net32000 and net32500.
I am very curious to know the opinion of the main developers of LC0.
Registering the similarity between pattern is a very complex problem even in an NN system too.
If the developers of LC0 solved this problem I congratulate to they.
(continue)
There may be a place in the structure of Leela where the global chess problems can be manipulated: The policy head of Leela.
Policy head help the search to find possible good moves in certain situations. These "certain situations" are
the same as it is in AB engines: to seek for getting bishop pair/open lines to rocks/free pawns/etc., and to avoid double pawn/checks/discovered checks/opposite bishops/trapped figures/etc. In concrete only the developers of Leela know what are in the policy head of Leela. In generally the content of policy head can be changed automatically during the course of learning ("zero approaching") and manually too.
If in the policy head there is information about how Leela handles such a position being after the 19th move of Black and Leela uses policy head during normal play also she runs a sequence of search to decide the possibility of win/draw/loss of 20.Bg6 and if the possibility of win is rather high she will move 20.Bg6.
So in the case of 20.Bg6 there is no in the value head - that is Leela meets never the position existed after
19.-,c4 - Leela also can find the move 20.Bg6.
It is pity but Chris has right there is no any consequent and easy to understand write down about how Leela works. The developers of Leela handle in closed mode a lot of detailed information about Leela.
Although Leela is an opened project - supposedly...

Alexander Schmidt
Posts: 1086
Joined: Thu May 10, 2007 12:49 pm

Re: End of Era is there: SF is finally beaten!

Post by Alexander Schmidt » Sun Feb 17, 2019 7:39 am

corres wrote:
Sat Feb 16, 2019 10:03 pm
George Tsavdaris wrote:
Sat Feb 16, 2019 8:41 pm
You have ZERO idea about how Neural Net engines work.
Thanks.
But if during self play LC0 never move in this position Bg6, from what she knows Bg6 is a good move?
Leela learns patterns in self play games. In new positions she tries to find this patterns. That's why she is running faster with gpu which are optimized for calculating images. Leela don't save one single position in it's NN. She found something in the position which made her believe is winning. We will likely never know what it was.

corres
Posts: 1578
Joined: Wed Nov 18, 2015 10:41 am
Location: hungary

Re: End of Era is there: SF is finally beaten!

Post by corres » Sun Feb 17, 2019 8:33 am

Alexander Schmidt wrote:
Sun Feb 17, 2019 7:39 am
Leela learns patterns in self play games. In new positions she tries to find this patterns. That's why she is running faster with gpu which are optimized for calculating images. Leela don't save one single position in it's NN. She found something in the position which made her believe is winning. We will likely never know what it was.
NN of Leela does not consist of "policy head" only...

Post Reply