Alphazero news

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Nay Lin Tun
Posts: 363
Joined: Mon Jan 16, 2012 5:34 am

Re: Alphazero news

Post by Nay Lin Tun » Fri Dec 07, 2018 4:58 am

mwyoung wrote:
Thu Dec 06, 2018 10:54 pm
Yes their data and match setup with the much faster hardware used by A0. Suggest it is time to get off the Google A0 hype train.

A +52 ELO advantage shown in their results playing Stockfish under Googles testing conditions is nothing.
Because their training is just initial trial run for NN.With these experiences, Leela will soon be better than A0. (Deep blue using massive resources was primitive in chess engines and now Stockfish is much stronger after 20 years of development.)

mwyoung
Posts: 1015
Joined: Wed May 12, 2010 8:00 pm

Re: Alphazero news

Post by mwyoung » Fri Dec 07, 2018 5:38 am

Nay Lin Tun wrote:
Fri Dec 07, 2018 4:58 am
mwyoung wrote:
Thu Dec 06, 2018 10:54 pm
Yes their data and match setup with the much faster hardware used by A0. Suggest it is time to get off the Google A0 hype train.

A +52 ELO advantage shown in their results playing Stockfish under Googles testing conditions is nothing.
Because their training is just initial trial run for NN.With these experiences, Leela will soon be better than A0. (Deep blue using massive resources was primitive in chess engines and now Stockfish is much stronger after 20 years of development.)
A0 results against Stockfish 8 is not that impressive. Under the Google match conditions A0 only a +52 elo advantage over Stockfish 8 with A0 having the hardware advantage. I am sure Lc0 could equal or surpass A0 match results against Stockfish 8 running under the same Google match conditions. And Stockfish 10 is already +112 elo better then Stockfish 8.
Professing themselves to be wise, they became fools,

Albert Silver
Posts: 2779
Joined: Wed Mar 08, 2006 8:57 pm
Location: Rio de Janeiro, Brazil

Re: Alphazero news

Post by Albert Silver » Fri Dec 07, 2018 5:54 am

mwyoung wrote:
Fri Dec 07, 2018 5:38 am
Nay Lin Tun wrote:
Fri Dec 07, 2018 4:58 am
mwyoung wrote:
Thu Dec 06, 2018 10:54 pm
Yes their data and match setup with the much faster hardware used by A0. Suggest it is time to get off the Google A0 hype train.

A +52 ELO advantage shown in their results playing Stockfish under Googles testing conditions is nothing.
Because their training is just initial trial run for NN.With these experiences, Leela will soon be better than A0. (Deep blue using massive resources was primitive in chess engines and now Stockfish is much stronger after 20 years of development.)
A0 results against Stockfish 8 is not that impressive. Under the Google match conditions A0 only a +52 elo advantage over Stockfish 8 with A0 having the hardware advantage. I am sure Lc0 could equal or surpass A0 match results against Stockfish 8 running under the same Google match conditions. And Stockfish 10 is already +112 elo better then Stockfish 8.
The hardware advantage you describe is not there. The speeds they got are roughly equivalent to two 2080Ti GPUs. These are certainly fast, but I would hesitate to describe them as giving an upper edge over the 44 cores Stockfish had. As to Stockfish 10 being stronger today, that may be true, but who says AlphaZero has stood idle all this while?
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."

Gary Internet
Posts: 25
Joined: Thu Jan 04, 2018 6:09 pm

Re: Alphazero news

Post by Gary Internet » Fri Dec 07, 2018 6:09 am

This link will take you to a comment on reddit that in turn has links to Lichess studies containing more AlphaZero games: Click Here

lkaufman
Posts: 3199
Joined: Sun Jan 10, 2010 5:15 am
Location: Maryland USA
Contact:

Re: Alphazero news

Post by lkaufman » Fri Dec 07, 2018 6:42 am

jp wrote:
Thu Dec 06, 2018 9:14 pm
sovaz1997 wrote:
Thu Dec 06, 2018 9:05 pm
1) Match vs. old version of SF.

2) AZ vs. SF + 17,= 75,- 8. It's score with TCEC openings.
And still SF8 in (2), right?
That's plus 32 elo. If the opponent was actually SF9 (does anyone know?) that would be about what I'd expect from SF10 under roughly TCEC conditions. So perhaps they are about equal given $20,000 or so hardware for each. So it's not yet clear that NN plus MCTS has surpassed normal alpha-beta in chess. What they have demonstrated is a good way to utilize the GPU for chess, as Lc0 is also doing. Perhaps there are better ways yet to be found.
Komodo rules!

mwyoung
Posts: 1015
Joined: Wed May 12, 2010 8:00 pm

Re: Alphazero news

Post by mwyoung » Fri Dec 07, 2018 6:51 am

Albert Silver wrote:
Fri Dec 07, 2018 5:54 am
mwyoung wrote:
Fri Dec 07, 2018 5:38 am
Nay Lin Tun wrote:
Fri Dec 07, 2018 4:58 am
mwyoung wrote:
Thu Dec 06, 2018 10:54 pm
Yes their data and match setup with the much faster hardware used by A0. Suggest it is time to get off the Google A0 hype train.

A +52 ELO advantage shown in their results playing Stockfish under Googles testing conditions is nothing.
Because their training is just initial trial run for NN.With these experiences, Leela will soon be better than A0. (Deep blue using massive resources was primitive in chess engines and now Stockfish is much stronger after 20 years of development.)
A0 results against Stockfish 8 is not that impressive. Under the Google match conditions A0 only a +52 elo advantage over Stockfish 8 with A0 having the hardware advantage. I am sure Lc0 could equal or surpass A0 match results against Stockfish 8 running under the same Google match conditions. And Stockfish 10 is already +112 elo better then Stockfish 8.
The hardware advantage you describe is not there. The speeds they got are roughly equivalent to two 2080Ti GPUs. These are certainly fast, but I would hesitate to describe them as giving an upper edge over the 44 cores Stockfish had. As to Stockfish 10 being stronger today, that may be true, but who says AlphaZero has stood idle all this while?
a Core i7 4770K has 45 GFlops per core, a gen 3 TPU has 45 TFlops which equals 1000x faster speed for the chosen task (per core). I would call that a hardware advantage. Sure they are not getting 1000x speed. But for sure they have a huge hardware advantage depending on their implementation . So A0 ran with 44 CPUs + its TPUs. Stockfish 8 run with only 44 CPUs. This is not in dispute by most people. If a 1080 ti is equal to running 40 cpu cores with Lc0. And you say A0 was operating on a equal system with 2 x 2080 ti. That would by like Stockfish 8 running on a 100 core cpu system.
Professing themselves to be wise, they became fools,

User avatar
Laskos
Posts: 8488
Joined: Wed Jul 26, 2006 8:21 pm

Re: Alphazero news

Post by Laskos » Fri Dec 07, 2018 8:05 am

Ok, I browsed quickly the paper and the additional material.

All important results are against SF8, aside the one against SF9, but from 1 standard opening position. The Cerebellum book, "Human Openings" and TCEC openings are used by SF8. The results against SF8 + Cerebellum and against SF8 in TCEC openings suggest that this A0 in these conditions is somewhat weaker than SF10. Very good result is achieved against SF9, but the result is unreliable, being from 1 standard opening position. Very good result against SF8 from "Human Openings", but "Human Openings" in the older preprint were favorable to A0, so I have no very high confidence in this result.

All in all, in their conditions, A0 seems a bit weaker than SF10 from a normal, unbiased set of openings.
Hardware conditions are fair GPU-CPU wise. I have a higher by a factor of ~2.5 "Effective Leela Ratio" than they do, and Lc0 is still a bit weaker than SF10 at LTC. A0 is still significantly better than Leela with the best nets, but in their conditions, seems a bit weaker than SF10 (again, using normal set of openings). I mean, that older A0 in the paper and material presented, probably by today, they improved on it.

User avatar
hgm
Posts: 22666
Joined: Fri Mar 10, 2006 9:06 am
Location: Amsterdam
Full name: H G Muller
Contact:

Re: Alphazero news

Post by hgm » Fri Dec 07, 2018 8:16 am

mwyoung wrote:
Fri Dec 07, 2018 6:51 am
a Core i7 4770K has 45 GFlops per core, a gen 3 TPU has 45 TFlops which equals 1000x faster speed for the chosen task (per core)
But AlphaZero was using gen 1 TPUs, right? IIRC these had 0 GFlops, as the could not do floating point at all.

mwyoung
Posts: 1015
Joined: Wed May 12, 2010 8:00 pm

Re: Alphazero news

Post by mwyoung » Fri Dec 07, 2018 8:34 am

hgm wrote:
Fri Dec 07, 2018 8:16 am
mwyoung wrote:
Fri Dec 07, 2018 6:51 am
a Core i7 4770K has 45 GFlops per core, a gen 3 TPU has 45 TFlops which equals 1000x faster speed for the chosen task (per core)
But AlphaZero was using gen 1 TPUs, right? IIRC these had 0 GFlops, as the could not do floating point at all.
I was told gen 3. But it did not say in the information posted. Here is what was posted on the site.

For the games themselves, Stockfish used 44 CPU (central processing unit) cores and AlphaZero used a single machine with four TPUs and 44 CPU cores. Stockfish had a hash size of 32GB and used syzygy endgame tablebases.

BTW: I need to buy a 2080 ti since 2 TPUs are equal one 2080 ti.
Professing themselves to be wise, they became fools,

jp
Posts: 259
Joined: Mon Apr 23, 2018 5:54 am

Re: Alphazero news

Post by jp » Fri Dec 07, 2018 8:35 am

lkaufman wrote:
Fri Dec 07, 2018 6:42 am
jp wrote:
Thu Dec 06, 2018 9:14 pm
sovaz1997 wrote:
Thu Dec 06, 2018 9:05 pm
1) Match vs. old version of SF.

2) AZ vs. SF + 17,= 75,- 8. It's score with TCEC openings.
And still SF8 in (2), right?
That's plus 32 elo. If the opponent was actually SF9 (does anyone know?) that would be about what I'd expect from SF10 under roughly TCEC conditions.
It was SF8. The Jan 13 SF games were separate with no TCEC openings.

Post Reply