Lc0 51010

[lkaufman]

Stockfish was not as impressed as Fruit. Lc0 51012 vs Stockfish

Results.jpg

Well I don't expect a 3250 engine to score much against SF on a powerful computer. The point is that if it is 3250 level with a self-play rating under 1500, what should it be with a 3000+ self-play rating in a few weeks?

Sadly not 3250 + 1500. The self play ratings are..."not to scale" as an architect might put it.

Yes, I know that. But then why is the engine self-rated below 1500 playing at 3250 level? It should be playing way below 1500 level, assuming that the rating of zero for random play is fair. I actually think that random play would be rated lower than zero if CCRL or CEGT tested it. Part of the answer is my hardware, but that hardly seems like a full explanation. Somehow it seems that self-play understates rating differences up to some level, then starts to overstate them beyond that level.

[nabildanial]

Over 3000 Elo at only half a million games? Wow, that's insane.

[George Tsavdaris]

Fritz GUI (technically Komodo 12 ChessBase GUI), and yes, new engine. It was obviously not playing like normal Lc0 versions. Evaluations of positions with material lost were drastically different.

WOW. Then these results are very very impressive.

[Laskos]

Over 3000 Elo at only half a million games? Wow, that's insane.

It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

[dkappe]

I noticed that Lc0 started a new network from scratch, and after just ten iterations it showed a rating around 1500 (now 1466). I have read that these self-play ratings are inflated...

Larry,

The self-play elo used to be determined by playing consecutive nets against each other and calculating using Bayes ELO formula (I think). That approach resulted in rating inflation over time.

Nowadays a new net plays against a plethora of past nets and the ratings are calculated by running all games (between all nets) through ordo (https://github.com/michiguel/Ordo), with the first net anchored at zero elo. This results in much less rating inflation.

[jp]

Fritz GUI (technically Komodo 12 ChessBase GUI), and yes, new engine. It was obviously not playing like normal Lc0 versions. Evaluations of positions with material lost were drastically different.

Not normal & different in a good way or a bad way?

[mwyoung]

Over 3000 Elo at only half a million games? Wow, that's insane.

It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.

[lkaufman]

Fritz GUI (technically Komodo 12 ChessBase GUI), and yes, new engine. It was obviously not playing like normal Lc0 versions. Evaluations of positions with material lost were drastically different.

Not normal & different in a good way or a bad way?

Well, it is good in the sense that it understands that it is better to be down a piece than to be down two pieces or a queen, which the top rated networks (except the 11xxx ones) have a problem with. So it should be much more efficient in winning a clearly won game and put up much better resistance in a clearly losing game. But presumably this is at the price of playing less precisely in positions where the result is not yet clear.

[lkaufman]

Over 3000 Elo at only half a million games? Wow, that's insane.

It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.

I don't think that flawed thinking and wrong assumptions are the reason I (or any strong player) lose to 3000+ rated engines. The reason is that I struggle to search even one node per second! Of course my thinking and assumptions are far from perfect, I just don't think that they are worse than what engines "think" or "assume". Although I'm not sure I could beat Lc0 in a serious match even with it set to use policy network only, i.e. one node per move, so maybe my last sentence only applies to A/B engines.

[mwyoung]

Over 3000 Elo at only half a million games? Wow, that's insane.

It is insane. Imagine poor Larry, all he learned in dozens of years is well surclassed by this dirt quick net using dirt basic monte carlo search.

This is not a sad thing. This is what many people in chess have strive to achieve for many reasons. As I chess player. I know we are flawed in our thinking when it comes to chess. And are assumptions are our worst enemy in chess play. This only makes us stronger for every class of chess player.

I don't think that flawed thinking and wrong assumptions are the reason I (or any strong player) lose to 3000+ rated engines. The reason is that I struggle to search even one node per second! Of course my thinking and assumptions are far from perfect, I just don't think that they are worse than what engines "think" or "assume". Although I'm not sure I could beat Lc0 in a serious match even with it set to use policy network only, i.e. one node per move, so maybe my last sentence only applies to A/B engines.

I think we make many wrong assumptions in chess. Lets start with the biggest one.

Chess is played out....