mwyoung wrote: ↑Fri Dec 07, 2018 9:34 am
I was told gen 3. But it did not say in the information posted. Here is what was posted on the site.
For the games themselves, Stockfish used 44 CPU (central processing unit) cores and AlphaZero used a single machine with four TPUs and 44 CPU cores. Stockfish had a hash size of 32GB and used syzygy endgame tablebases.
What do people think would happen if A0 used 4 TPUs and just 1 CPU core? How would that change its strength?
I see no "fairness" reason to give it 44 cores as well as the 4 TPUs.
Laskos wrote: ↑Fri Dec 07, 2018 9:05 am
Ok, I browsed quickly the paper and the additional material.
All important results are against SF8, aside the one against SF9, but from 1 standard opening position. The Cerebellum book, "Human Openings" and TCEC openings are used by SF8. The results against SF8 + Cerebellum and against SF8 in TCEC openings suggest that this A0 in these conditions is somewhat weaker than SF10. Very good result is achieved against SF9, but the result is unreliable, being from 1 standard opening position. Very good result against SF8 from "Human Openings", but "Human Openings" in the older preprint were favorable to A0, so I have no very high confidence in this result.
All in all, in their conditions, A0 seems a bit weaker than SF10 from a normal, unbiased set of openings.
Hardware conditions are fair GPU-CPU wise. I have a higher by a factor of ~2.5 "Effective Leela Ratio" than they do, and Lc0 is still a bit weaker than SF10 at LTC. A0 is still significantly better than Leela with the best nets, but in their conditions, seems a bit weaker than SF10 (again, using normal set of openings). I mean, that older A0 in the paper and material presented, probably by today, they improved on it.
Once the article was sent for publication, it was submitted to the review of two or three referees, whose identity is kept secret, which will send their recommendations or demands to the authors for the article to be published. I suppose that among these recommendations they will have included the topic of the book of openings for Stockfish and the variety in the time control.
Since the article is based on the first version of A0, all the games played should have been done with this version.
If during this time they have advanced and have achieved better results with A0 version 2, this will probably lead to the publication of another article.
mwyoung wrote: ↑Fri Dec 07, 2018 9:34 amI was told gen 3. But it did not say in the information posted. Here is what was posted on the site.
Well, I don't know who 'told you' that, but the newly published paper states:
science paper wrote:Each program was run on the hardware for which it was designed (23): Stockfish and Elmo used 44 central processing unit (CPU) cores (as in the TCEC world championship), whereas AlphaZero and AlphaGo Zero used a single machine with four first-generation TPUs and 44 CPU cores (24).
So it seems in fact that in terms of GFlops there was no advantage at all...
Javier Ros wrote: ↑Fri Dec 07, 2018 10:03 am
If during this time they have advanced and have achieved better results with A0 version 2
It's not clear whether they are actively developing it. Maybe not?
I don't know, but for the amount of games and news in the revised version of the article, I would say that they have been quite busy until the final version has been accepted.
jp wrote: ↑Fri Dec 07, 2018 9:42 amWhat do people think would happen if A0 used 4 TPUs and just 1 CPU core? How would that change its strength?
I see no "fairness" reason to give it 44 cores as well as the 4 TPUs.
I am pretty sure that AlphaZero hardly used any of those cores. There just wouldn't be anything to do for them while it was waiting for the TPU output. The paper doesn't say it used them, just that the machine had them. For simplicity it would make sense to use 4 CPU cores, so that each core controls its own CPU. Using just one core would probably hardly slow it down.
Javier Ros wrote: ↑Fri Dec 07, 2018 10:03 am
If during this time they have advanced and have achieved better results with A0 version 2
It's not clear whether they are actively developing it. Maybe not?
I don't know, but for the amount of games and news in the revised version of the article, I would say that they have been quite busy until the final version has been accepted.
That's not development, though. It's just testing they should have done in the beginning that the reviewers asked them to do.
jp wrote: ↑Fri Dec 07, 2018 9:42 amWhat do people think would happen if A0 used 4 TPUs and just 1 CPU core? How would that change its strength?
I see no "fairness" reason to give it 44 cores as well as the 4 TPUs.
I am pretty sure that AlphaZero hardly used any of those cores. There just wouldn't be anything to do for them while it was waiting for the TPU output. The paper doesn't say it used them, just that the machine had them. For simplicity it would make sense to use 4 CPU cores, so that each core controls its own CPU. Using just one core would probably hardly slow it down.
My guess is it will slow it down by a factor of 4x atleast, according to my experience, when used with aggresive batching. However, that is only the case if TPU latency is as high as the GPU's and there is an actual need for aggressive batching. LC0 uses single threaded batching so it is not helped at all by presence of multiple cpu cores. Scorpio, like alphazero, use multi-threaded batching where the MCTS simulations on the CPU are done in parallel. If I am launching 128--256 threads on a single core CPU, I need to have a 1 millisecond sleep inserted in each threads execution path to get better performance -- even after that I get atleast 4x slowdown compared to using 2 haswell cpus (16 core each). And if you do mix alpha-beta, even qsearch(), the CPUs cores become even more important. I am now settlng on a simplified qsearch() with recaptures only after the first ply to reduce the load on the CPUs -- many testers automatically assume CPU are not needed for a GPU engine based on their experience with LC0...
