Lc0 51010

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
jp
Posts: 840
Joined: Mon Apr 23, 2018 5:54 am

Re: Lc0 51010

Post by jp » Sat Mar 30, 2019 9:23 am

lkaufman wrote:
Sat Mar 30, 2019 3:02 am
Whether chess is played out depends on how you define the question. It seems pretty clear that if you take the strongest Stockfish and the strongest NN engine on TCEC type hardware, and have them play a long match at TCEC time controls, the results will depend heavily on the openings. If you give each side a really good, deep opening book and have them play only the optimum or near optimum openings, nearly every game will end in a draw.
-------
Anyway none of this seems relevant to me to the question of why computers beat humans. No matter how correct our beliefs about chess might be, we cannot compete with something that searches a million times faster than we can.
Yep.

lkaufman
Posts: 3757
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lc0 51010

Post by lkaufman » Sun Mar 31, 2019 6:07 am

The situation has now become even more dramatic. I tested version 51058 (on my RTX 2080), which is self-rated slightly below 1700, against Fire 7.1 on 4 very fast threads, which is rated 2337 on CEGT blitz, 3425 CCRL blitz. The two day old Lc0 won by 12 to 8 (+70 elo), putting it over 3400 CEGT blitz and nearly 3500 CCRL blitz!! So I guess we won't be hearing that the self-play ratings are inflated anymore, although it remains to be seen whether further gains in self-play ratings continue to translate to real gains. One question: does anyone know roughly how many 2080s would be needed to duplicate the training that this 51xxx series has averaged so far? It doesn't mean much to say that it has trained for two days without stating what the average resources used for the training were. I imagine that they were just a tiny percentage of the resources used to train AlphaZero in 9 hours or so.
Komodo rules!

jp
Posts: 840
Joined: Mon Apr 23, 2018 5:54 am

Re: Lc0 51010

Post by jp » Sun Mar 31, 2019 6:37 am

lkaufman wrote:
Sun Mar 31, 2019 6:07 am
One question: does anyone know roughly how many 2080s would be needed to duplicate the training that this 51xxx series has averaged so far? It doesn't mean much to say that it has trained for two days without stating what the average resources used for the training were. I imagine that they were just a tiny percentage of the resources used to train AlphaZero in 9 hours or so.
NN 51010 has had 559441 games. The NN size is smaller (10x128) than the A0 one and 41xxx NNs (20x256), but the training param is visits=10000 (visits=800 for 41xxx).

I'll let others say what 2080 time that works out to.

User avatar
Guenther
Posts: 3111
Joined: Wed Oct 01, 2008 4:33 am
Location: Regensburg, Germany
Full name: Guenther Simon
Contact:

Re: Lc0 51010

Post by Guenther » Sun Mar 31, 2019 7:52 am

lkaufman wrote:
Sun Mar 31, 2019 6:07 am
The situation has now become even more dramatic. I tested version 51058 (on my RTX 2080), which is self-rated slightly below 1700, against Fire 7.1 on 4 very fast threads, which is rated 2337 on CEGT blitz, 3425 CCRL blitz. The two day old Lc0 won by 12 to 8 (+70 elo), putting it over 3400 CEGT blitz and nearly 3500 CCRL blitz!! So I guess we won't be hearing that the self-play ratings are inflated anymore, although it remains to be seen whether further gains in self-play ratings continue to translate to real gains. One question: does anyone know roughly how many 2080s would be needed to duplicate the training that this 51xxx series has averaged so far? It doesn't mean much to say that it has trained for two days without stating what the average resources used for the training were. I imagine that they were just a tiny percentage of the resources used to train AlphaZero in 9 hours or so.
May be tradeoff for speed is better than expected for 10b nets with less information?
There are no tests about best ratio of max size of NNs vs. speed.
Current foe list count : [101]
http://rwbc-chess.de/chronology.htm

User avatar
Laskos
Posts: 9511
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Lc0 51010

Post by Laskos » Sun Mar 31, 2019 8:58 am

jp wrote:
Sun Mar 31, 2019 6:37 am
lkaufman wrote:
Sun Mar 31, 2019 6:07 am
One question: does anyone know roughly how many 2080s would be needed to duplicate the training that this 51xxx series has averaged so far? It doesn't mean much to say that it has trained for two days without stating what the average resources used for the training were. I imagine that they were just a tiny percentage of the resources used to train AlphaZero in 9 hours or so.
NN 51010 has had 559441 games. The NN size is smaller (10x128) than the A0 one and 41xxx NNs (20x256), but the training param is visits=10000 (visits=800 for 41xxx).

I'll let others say what 2080 time that works out to.
I will try to estimate (writing on the phone, I am on a vacation :) ). About 0.2s/move on 2080, meaning games in about 20s. About 200 games per hour, so half a million games need 2500 hours, or about 100 days needed for training the NN 51010 on a single 2080. Or 100x 2080 GPUs needed to train it in one day.
The main unknown is the time to get 10000 visits, I took it as 0.2s on 2080. So this calculation is just for the order of magnitude.

Uri Blass
Posts: 8609
Joined: Wed Mar 08, 2006 11:37 pm
Location: Tel-Aviv Israel

Re: Lc0 51010

Post by Uri Blass » Sun Mar 31, 2019 9:20 am

mwyoung wrote:
Sat Mar 30, 2019 1:29 am
Uri Blass wrote:
Sat Mar 30, 2019 12:50 am
mwyoung wrote:
Fri Mar 29, 2019 10:09 pm
lkaufman wrote:
Fri Mar 29, 2019 9:55 pm
mwyoung wrote:
Fri Mar 29, 2019 9:35 pm
Laskos wrote:
Fri Mar 29, 2019 8:57 pm
nabildanial wrote:
Fri Mar 29, 2019 8:02 pm
Over 3000 Elo at only half a million games? Wow, that's insane.
It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.
I don't think that flawed thinking and wrong assumptions are the reason I (or any strong player) lose to 3000+ rated engines. The reason is that I struggle to search even one node per second! Of course my thinking and assumptions are far from perfect, I just don't think that they are worse than what engines "think" or "assume". Although I'm not sure I could beat Lc0 in a serious match even with it set to use policy network only, i.e. one node per move, so maybe my last sentence only applies to A/B engines.
I think we make many wrong assumptions in chess. Lets start with the biggest one.

Chess is played out....
I do not understand.
What is the assumption that you mean in this post?

Have you ever asked yourself why Lc0 and A0 have stirred so much excitement with Titled players. It is because Lc0 and A0 have show us what we did not expect.

Or Lc0 and A0 would be like any other chess programs. Humans play chess with certain assumptions, and a thinking of the way chess should be played. Chess programs of the past, and now like Lc0 and A0 have show us things we never considered before.

Uri, you have been on CCC as long as I can remember. How many times over the years have you seen people say X chess program is rated X. And this is almost perfect play. I can remember this many times, and today those chess programs stand no chance against the best chess programs of today. And today's engine will stand no change against tomorrows.

Its like when you argued that chess could be solved, and not grasping the meaning of what 10 ^120 means in terms of solving chess.

Remember it was not that many years ago, players like Capablanca, Fischer, Seirawan... and other thought chess was played out. And needed new variants to keep chess alive. Like Capablanca Chess, Fischer random chess, Seirawan chess...

Chess programs have shown that many of our human assumptions about chess are wrong. And are assumptions are many, but all are subsets of Chess is played out...
I agree that the assumption that we are close to perfect chess was clearly wrong in the past.
I simply did not understand that it is what you meant by chess is played out and this was the reason for my question.

The question if we are close to perfect chess today is a different question.
It is clear that at blitz today we are not close to perfect chess.
The only question is about long time control like TCEC time control.
The claim is that TCEC used some opening book to reduce the number of draws and with good opening book we are going to get almost 100% draws.

larry's words about it:
"It seems pretty clear that if you take the strongest Stockfish and the strongest NN engine on TCEC type hardware, and have them play a long match at TCEC time controls, the results will depend heavily on the openings. If you give each side a really good, deep opening book and have them play only the optimum or near optimum openings, nearly every game will end in a draw."

I am not sure about it and I have 2 problems:
1)How do you define if a book is a good deep opening book?
If you do not get the expected draw result in a game you can claim that the book was not good enough or was not deep enough so you need some definition that is not dependent on the result.

2)How do you define if an opening is the optimum or near the optimum opening?
It is possible that the best move is a move that the engines do not like because they do not see deep enough.

claim 2 of larry:
"The only way the stronger engine can win games is to play suboptimal lines with White to exit from book early and hope to win despite the poor opening moves."

Based on what he claims that a line is suboptimal line?
A line that the best engines do not suggest does not have to be a suboptimal line.

There should be a basis to claim that a line is suboptimal(for example if the engine that beat another engine lose against itself in the same line)
I guess larry means to lines that white has equality instead of minimal advantage and in this case I doubt if it is correct to call it a suboptimal line because the line does not lose and can help to win against weaker engines.

jp
Posts: 840
Joined: Mon Apr 23, 2018 5:54 am

Re: Lc0 51010

Post by jp » Sun Mar 31, 2019 9:34 am

Uri Blass wrote:
Sun Mar 31, 2019 9:20 am

larry's words about it:
"It seems pretty clear that if you take the strongest Stockfish and the strongest NN engine on TCEC type hardware, and have them play a long match at TCEC time controls, the results will depend heavily on the openings. If you give each side a really good, deep opening book and have them play only the optimum or near optimum openings, nearly every game will end in a draw."

I am not sure about it and I have 2 problems:
1)How do you define if a book is a good deep opening book?
If you do not get the expected draw result in a game you can claim that the book was not good enough or was not deep enough so you need some definition that is not dependent on the result.

2)How do you define if an opening is the optimum or near the optimum opening?
It is possible that the best move is a move that the engines do not like because they do not see deep enough.
Your two problems are more problems with how to test the claim than with the claim itself.
If the claim was the opposite ("...every game will end in a win") your two problems would still be there.

The result of the games A0 vs crippled SF8 + book gives some support to the claim, though we cannot be certain without, as you say, defining our "good deep opening book" and testing.

User avatar
Ozymandias
Posts: 1137
Joined: Sun Oct 25, 2009 12:30 am

Re: Lc0 51010

Post by Ozymandias » Sun Mar 31, 2019 10:23 am

Uri Blass wrote:
Sun Mar 31, 2019 9:20 am
The claim is that TCEC used some opening book to reduce the number of draws and with good opening book we are going to get almost 100% draws.

larry's words about it:
"It seems pretty clear that if you take the strongest Stockfish and the strongest NN engine on TCEC type hardware, and have them play a long match at TCEC time controls, the results will depend heavily on the openings. If you give each side a really good, deep opening book and have them play only the optimum or near optimum openings, nearly every game will end in a draw."

I am not sure about it and I have 2 problems:
1)How do you define if a book is a good deep opening book?
If you do not get the expected draw result in a game you can claim that the book was not good enough or was not deep enough so you need some definition that is not dependent on the result.

2)How do you define if an opening is the optimum or near the optimum opening?
It is possible that the best move is a move that the engines do not like because they do not see deep enough.

claim 2 of larry:
"The only way the stronger engine can win games is to play suboptimal lines with White to exit from book early and hope to win despite the poor opening moves."

Based on what he claims that a line is suboptimal line?
A line that the best engines do not suggest does not have to be a suboptimal line.

There should be a basis to claim that a line is suboptimal(for example if the engine that beat another engine lose against itself in the same line)
I guess larry means to lines that white has equality instead of minimal advantage and in this case I doubt if it is correct to call it a suboptimal line because the line does not lose and can help to win against weaker engines.
I'm surprised to see this topic still being debated after so long, maybe you missed the draw death of Freestyle chess. The answer to your questions is there.

User avatar
Eduard
Posts: 224
Joined: Fri Oct 26, 2018 10:58 pm
Location: Germany
Full name: Eduard Nemeth
Contact:

Re: Lc0 51010

Post by Eduard » Sun Mar 31, 2019 12:37 pm

lkaufman wrote:
Sun Mar 31, 2019 6:07 am
The situation has now become even more dramatic. I tested version 51058 (on my RTX 2080), which is self-rated slightly below 1700, against Fire 7.1 on 4 very fast threads, which is rated 2337 on CEGT blitz, 3425 CCRL blitz. The two day old Lc0 won by 12 to 8 (+70 elo), putting it over 3400 CEGT blitz and nearly 3500 CCRL blitz!! [----]
Forgive me for these lines, but I'm a computer chess fan and today is Sunday! I have a lot of fun today. Unbelievable how this new NN plays. :roll:

Albert Silver
Posts: 2869
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Lc0 51010

Post by Albert Silver » Sun Mar 31, 2019 3:42 pm

jp wrote:
Sun Mar 31, 2019 6:37 am
NN 51010 has had 559441 games. The NN size is smaller (10x128) than the A0 one and 41xxx NNs (20x256), but the training param is visits=10000 (visits=800 for 41xxx).
That is inaccurate. It is averaging around 800 visits per move too.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

Post Reply