Lc0 51010

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lc0 51010

Post by lkaufman » Fri Mar 29, 2019 7:20 pm

Werewolf wrote:
Fri Mar 29, 2019 6:40 pm
lkaufman wrote:
Fri Mar 29, 2019 4:51 pm
mwyoung wrote:
Fri Mar 29, 2019 7:43 am
Stockfish was not as impressed as Fruit. Lc0 51012 vs Stockfish

Results.jpg
Well I don't expect a 3250 engine to score much against SF on a powerful computer. The point is that if it is 3250 level with a self-play rating under 1500, what should it be with a 3000+ self-play rating in a few weeks?
Sadly not 3250 + 1500. The self play ratings are..."not to scale" as an architect might put it.
Yes, I know that. But then why is the engine self-rated below 1500 playing at 3250 level? It should be playing way below 1500 level, assuming that the rating of zero for random play is fair. I actually think that random play would be rated lower than zero if CCRL or CEGT tested it. Part of the answer is my hardware, but that hardly seems like a full explanation. Somehow it seems that self-play understates rating differences up to some level, then starts to overstate them beyond that level.
Komodo rules!

nabildanial
Posts: 104
Joined: Thu Jun 05, 2014 3:29 am
Location: Malaysia

Re: Lc0 51010

Post by nabildanial » Fri Mar 29, 2019 8:02 pm

Over 3000 Elo at only half a million games? Wow, that's insane.

User avatar
George Tsavdaris
Posts: 1599
Joined: Thu Mar 09, 2006 11:35 am

Re: Lc0 51010

Post by George Tsavdaris » Fri Mar 29, 2019 8:45 pm

lkaufman wrote:
Fri Mar 29, 2019 4:42 pm
Fritz GUI (technically Komodo 12 ChessBase GUI), and yes, new engine. It was obviously not playing like normal Lc0 versions. Evaluations of positions with material lost were drastically different.
WOW. Then these results are very very impressive.
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....

User avatar
Laskos
Posts: 9442
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: Lc0 51010

Post by Laskos » Fri Mar 29, 2019 8:57 pm

nabildanial wrote:
Fri Mar 29, 2019 8:02 pm
Over 3000 Elo at only half a million games? Wow, that's insane.
It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

dkappe
Posts: 260
Joined: Tue Aug 21, 2018 5:52 pm
Full name: Dietrich Kappe

Re: Lc0 51010

Post by dkappe » Fri Mar 29, 2019 9:16 pm

lkaufman wrote:
Fri Mar 29, 2019 4:21 am
I noticed that Lc0 started a new network from scratch, and after just ten iterations it showed a rating around 1500 (now 1466). I have read that these self-play ratings are inflated...
Larry,

The self-play elo used to be determined by playing consecutive nets against each other and calculating using Bayes ELO formula (I think). That approach resulted in rating inflation over time.

Nowadays a new net plays against a plethora of past nets and the ratings are calculated by running all games (between all nets) through ordo (https://github.com/michiguel/Ordo), with the first net anchored at zero elo. This results in much less rating inflation.

jp
Posts: 818
Joined: Mon Apr 23, 2018 5:54 am

Re: Lc0 51010

Post by jp » Fri Mar 29, 2019 9:25 pm

lkaufman wrote:
Fri Mar 29, 2019 4:42 pm
Fritz GUI (technically Komodo 12 ChessBase GUI), and yes, new engine. It was obviously not playing like normal Lc0 versions. Evaluations of positions with material lost were drastically different.
Not normal & different in a good way or a bad way?

mwyoung
Posts: 1641
Joined: Wed May 12, 2010 8:00 pm

Re: Lc0 51010

Post by mwyoung » Fri Mar 29, 2019 9:35 pm

Laskos wrote:
Fri Mar 29, 2019 8:57 pm
nabildanial wrote:
Fri Mar 29, 2019 8:02 pm
Over 3000 Elo at only half a million games? Wow, that's insane.
It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.
Professing themselves to be wise, they became fools,
Take on me. foes 0

lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lc0 51010

Post by lkaufman » Fri Mar 29, 2019 9:49 pm

jp wrote:
Fri Mar 29, 2019 9:25 pm
lkaufman wrote:
Fri Mar 29, 2019 4:42 pm
Fritz GUI (technically Komodo 12 ChessBase GUI), and yes, new engine. It was obviously not playing like normal Lc0 versions. Evaluations of positions with material lost were drastically different.
Not normal & different in a good way or a bad way?
Well, it is good in the sense that it understands that it is better to be down a piece than to be down two pieces or a queen, which the top rated networks (except the 11xxx ones) have a problem with. So it should be much more efficient in winning a clearly won game and put up much better resistance in a clearly losing game. But presumably this is at the price of playing less precisely in positions where the result is not yet clear.
Komodo rules!

lkaufman
Posts: 3724
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Lc0 51010

Post by lkaufman » Fri Mar 29, 2019 9:55 pm

mwyoung wrote:
Fri Mar 29, 2019 9:35 pm
Laskos wrote:
Fri Mar 29, 2019 8:57 pm
nabildanial wrote:
Fri Mar 29, 2019 8:02 pm
Over 3000 Elo at only half a million games? Wow, that's insane.
It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.
I don't think that flawed thinking and wrong assumptions are the reason I (or any strong player) lose to 3000+ rated engines. The reason is that I struggle to search even one node per second! Of course my thinking and assumptions are far from perfect, I just don't think that they are worse than what engines "think" or "assume". Although I'm not sure I could beat Lc0 in a serious match even with it set to use policy network only, i.e. one node per move, so maybe my last sentence only applies to A/B engines.
Komodo rules!

mwyoung
Posts: 1641
Joined: Wed May 12, 2010 8:00 pm

Re: Lc0 51010

Post by mwyoung » Fri Mar 29, 2019 10:09 pm

lkaufman wrote:
Fri Mar 29, 2019 9:55 pm
mwyoung wrote:
Fri Mar 29, 2019 9:35 pm
Laskos wrote:
Fri Mar 29, 2019 8:57 pm
nabildanial wrote:
Fri Mar 29, 2019 8:02 pm
Over 3000 Elo at only half a million games? Wow, that's insane.
It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.
I don't think that flawed thinking and wrong assumptions are the reason I (or any strong player) lose to 3000+ rated engines. The reason is that I struggle to search even one node per second! Of course my thinking and assumptions are far from perfect, I just don't think that they are worse than what engines "think" or "assume". Although I'm not sure I could beat Lc0 in a serious match even with it set to use policy network only, i.e. one node per move, so maybe my last sentence only applies to A/B engines.
I think we make many wrong assumptions in chess. Lets start with the biggest one.

Chess is played out....
Professing themselves to be wise, they became fools,
Take on me. foes 0

Post Reply