LCzero sacs a knight for nothing

[mirek]

do you have an estimate for elo of alphazero ?

Since SF8 4CPU is 3386 CCRL 40/40 thus 64 core setup with ~ 2x faster cores than CCRL reference should be at least 3525+
So A0 + 4xTPU should be 100 elo above, thus 3625+ on CCRL 40/40?

[Robert Pope]

Therefore, just a factor of 4 in time control (or hardware) (2 doublings) gives a boost of 128 Elo points compared to standard A/B engine. Or 64 Elo points per doubling. One can extrapolate:
On one CPU core at 4 s/move, from this match LC0 ID160 is about 2100 CCRL Elo. A top GPU, say Nvidia 1080 Ti, is faster by a factor of 25 compared to 1 CPU core. Tournament TC is about 40 longer than 4s/move. So, all in all, a total of a factor of 1000 time-hardware wise, or 10 doublings. So, ID160 on a top GPU and LTC would be about 2750 CCRL Elo. And on DeepMind hardware used in exhibition match, would be 3100+ CCRL Elo.

so if alphazero elo is 3200, it is only 100 elo stronger than lc0 which means lc0 will soon stall ?

https://imgur.com/a/c04yc

Well, first off, that is a pretty big "if". Whenever you extrapolate upwards like this, there is a real risk that your assumptions won't continue to hold.

do you have an estimate for elo of alphazero ?

Nope. I'm just pointing out that trending out like that is inherently risky, as I'm sure Kai would agree. It does give an interesting data point, but it isn't necessarily meaningful. I prefer to rely on actual match data as it becomes available, than try to extrapolate the impact of a 1000x increase in speed.

[Milos]

Exactly, and what's even more remarkable is that according to the A0 paper (figure 2) 4xTPUs will do about 80k payouts in 1s and at 80k playots A0 is only 100 - 150 elo weaker than at 1 min / move (5000k playots)

Also 1x 1080Ti (11 TFLOPS) vs 4xTPU (180 TFLOPS) means nps gets reduced to 4.8k nps Even if we assumed that the TPU is somehow more effective flops to flops by factor of 4x the resulting 1080Ti playouts would be still close to 80k per minute. Thus to me it seems quite convincing that A0 on 1080Ti would be with good confidence max 150 elo weaker at 1min / move compared to 4xTPU configuration. (and most likely not more than 100 elo weaker)

Gee, you got it almost all wrong. Mainly because figure 2 is totally bogus.
First scaling of SF is bogus.
It can be easily demonstrated that SF8 on 64 cores when going from TC = 1s/move to 1min/move gains at least 40*6 = 240Elo.
They show in figure 60Elo?!???!
Second, A0 might scale better or worse than SF, but will get at least 200Elo, more probably over 300Elo when going from 1s/move to 1min/move.
Third, one second generation TPU is 180TFLOPS so 4 TPUs means 4x180TFLOPS (you are clearly confused by Google's misleading terminology about "Cloud TPUs" where each TPU contains 4 chips, but actual second generation TPU = Cloud TPU).
However, for actual matches Google used first gen TPUs (because 4TPUs for matches give exactly 4x nps that 1TPU used for self-play gives - 800sims in 40ns) that are actually around 92TOPS. So one 1080Ti is approximately 40x slower than 4TPUs used for playing matches with SF.
Finally, A0 on 1080Ti might be 300Elo weaker then current SFdev on 64 cores and 1min/move, but LC0 is at least 800 Elo weaker, which means that LC0 is atm at least 500Elo weaker than A0 on the same hardware.
Will LC0 ever reach performance of A0?
I strongly doubt it, because performance of A0 is simply bogus.

[Werewolf]

[pgn] 1. e4 e5 2. Nf3 f6 3. Ne5 [/pgn]

LCZero ID 160 gets this in 1 min 52 sec on my machine, not bad!

[Werewolf]

This is one LCZero couldn't find at all last time I tested, back on the small network.

ID 160 finds this in 2 seconds!

[pgn] 1.e4 Nf6 2.e5 Nd5 3.c4 Nb6 4.d4 Nc6 5.d5! [/pgn]

For comparison, HIARCS 6 on my old Pentium 166 MHz took 17 seconds back in the day....

[mirek]

Gee, you got it almost all wrong. Mainly because figure 2 is totally bogus.
First scaling of SF is bogus.
It can be easily demonstrated that SF8 on 64 cores when going from TC = 1s/move to 1min/move gains at least 40*6 = 240Elo.
They show in figure 60Elo?!???!

If it can be easily demonstrated then please demonstrated the measured numbers.

Second, A0 might scale better or worse than SF, but will get at least 200Elo, more probably over 300Elo when going from 1s/move to 1min/move.

I like your confidence how first you disregard A0 scaling measured by DeepMind and then provide your own (supposedly better) numbers which as far as I can see you have just pulled out of thin air.

I mean in the other thread you were previously claiming that SF @ depth 1 is stronger than A0 with zero search but the actually measured numbers using just lc0 (which as you claim is at least 500 elo weaker) show that lc0 is most likely already comparable if not stronger.

However, for actual matches Google used first gen TPUs (because 4TPUs for matches give exactly 4x nps that 1TPU used for self-play gives - 800sims in 40ns) that are actually around 92TOPS. So one 1080Ti is approximately 40x slower than 4TPUs used for playing matches with SF.

OK, here I really got confused, but anyways, if I were to take your number (1080Ti = 40x slower) it would still fit comfortably above the 80k playouts per minute thus according to the graph only within 100 - 150 elo weaker.

While in reality the 92TOPS figure is only for 8bit integers so not sure how it would translate to 1080Ti (16bit FLOPS) This should be again best measured by someone who has 1080Ti and also net of the same size as A0 used.

[jp]

I am even beginning to suspect that they didn't release to the general consumer their products because on a normal i7 CPU and average GPU, the performance of their AlphaZero Go and Chess programs would be not that impressive, or even pretty lame (compared to the hype), especially in Chess.

I suspect they could not care less about releasing them as general consumer products, and nor was that ever even on the table.

Kai, I guess you are being extremely polite and didn't need any testing to know that.
Albert, if your "could not care less about releasing " meant "don't want to release", your comment does not contradict Kai's and Kai's comment gives one reason why. But if your "could not care less" meant "could not care less", it's wrong. They definitely care a lot that it is not released. All their actions show that.