Ozymandias wrote: ↑Sun May 27, 2018 2:08 pm
These questions, designed to ridicule the ones quoted before, are unwarranted.
For many years, the recurrent "is there an engine that can use my GPU?" popped in computer chess forums every now and then; now that we have the option to finally use that (previously untapped) HW resource, it'd be nice to see traditional rating lists reacting a little more promptly to the new situation, instead of just keeping testing it as a CPU bound engine.
That is just BS. Maybe you personally had that question but no one ever posed it in this forum because ppl are a little more educated in the matters that what you seems to be.
Lc0 on strongest 8 years ago old GPU is weaker than Lc0 on single core of 8 years old CPU. I.e. before the boom of ML research in last 5 years no one ever thought of running chess engine on GPU. 3 years ago on Srdja was working on running conventional engine on GPU and even Giraffe with full CNN evaluation was running only on CPU. Even most of ML research was done on regular CPUs till 2 years ago.
So again you are just writing BS.
Robert Pope wrote: ↑Tue May 29, 2018 10:03 pm
...when pro-athlete swimsuit designers started coming up with designs with less and less water resistance, the governing bodies didn't just say "we'll only allow results from swimmers wearing 1980s spandex Speedos". No, they sat down, examined the issue and tried to set standards that allowed competitors to use state of the art materials, while not imparting an unnatural advantage to those that used them (e.g. suits that added buoyancy).
There's no reason the "common hardware" can't be defined to include some level of GPU as well as CPU.
Not a good example, because they then screwed up by allowing supersuits & the sport still hasn't completely recovered from the farce of the supersuit era. Mediocre swimmers were smashing world records. Worse, the supersuits favored the brute force swimmers by covering up their defects, so brutes vs technical kings in supersuits was worse for the technical kings.
Robert Pope wrote: ↑Tue May 29, 2018 10:03 pm
...when pro-athlete swimsuit designers started coming up with designs with less and less water resistance, the governing bodies didn't just say "we'll only allow results from swimmers wearing 1980s spandex Speedos". No, they sat down, examined the issue and tried to set standards that allowed competitors to use state of the art materials, while not imparting an unnatural advantage to those that used them (e.g. suits that added buoyancy).
There's no reason the "common hardware" can't be defined to include some level of GPU as well as CPU.
Not a good example, because they then screwed up by allowing supersuits & the sport still hasn't completely recovered from the farce of the supersuit era. Mediocre swimmers were smashing world records. Worse, the supersuits favored the brute force swimmers by covering up their defects, so brutes vs technical kings in supersuits was worse for the technical kings.
Even worse for the example, supersuits gave few % advantage in overall time. Running things on fastest GPU compared to single CPU core gives advantage of 1000x at least. That is even worse than putting jet engine on a bicycle, it's like strapping Saturn V rocket to a bicycle...
Some ppl have really no touch with reality when making analogies and comparisons.
[moderation]
I am getting complaints on the unnecessary offensive tone of some of the postings here. Please try to keep it civilized, even when unfamiliar with the concept, so that I won't have to start deleting posts. I think Ozymanias raises sensible points, which deserve a decent discussion.
Ozymandias wrote: ↑Sun May 27, 2018 2:08 pm
now that we have the option to finally use that (previously untapped) HW resource, it'd be nice to see traditional rating lists reacting a little more promptly to the new situation, instead of just keeping testing it as a CPU bound engine.
The principle of the CCRL list is that every participant is on equal hardware. It is a list for engines that use the CPU, and that can be benchmarked and time control adjusted agaunst the benchmark CPU. Yes at some point in the future this sort of list could become obsolete, but not yet.
Someone like SSDF could test it on GPU. Their list does not have common hardware. They have old dedicated chess machines, and old A1200 right through to latest Ryzen all on the same ratings list. And actual time control, not adjusted ones.
You mention benchmarking, but what is your reference machine and your benchmark? If your reference machine is a 32-bit Pentium I, you would arrive at a completely different TC factor when using a bitboard engine as benchmark, or a mailbox engine. If you would insist on testing everything on a 32-bit architecture, it is clear that all bitboard engines would drop some 100 Elo compared to mailbox engines. And as almost everyone now has 64-bit hardware, the resulting rating list would have little reality value. So you do allow magic-bitboard engines to use the 64-bit instruction-set extension.
I think it would be wise to do the same with GPU. Until now, GPUs were useless to Chess engines; powerful as they were, they did not do anything Chess engines could use to their advantage. So you can redefine the reference computer as having a certain GPU, without any effect on the rating list. And then allow new engines to use that reference GPU, in addition to the CPU. That way your rating list would gradually adapt to the future, as more andd more engines learn to exploit the GPU. The alternative is a persistently growing discrepancy between your list an the reality in people's homes, which could be hard to correct once the discrepancy gets so large that the list has become totally obsolete. Like it would be if you were still only listing the 32-bit performance of all engines.
hgm wrote: ↑Wed May 30, 2018 5:37 pmI think it would be wise to do the same with GPU. Until now, GPUs were useless to Chess engines; powerful as they were, they did not do anything Chess engines could use to their advantage. So you can redefine the reference computer as having a certain GPU, without any effect on the rating list. And then allow new engines to use that reference GPU, in addition to the CPU. That way your rating list would gradually adapt to the future, as more andd more engines learn to exploit the GPU. The alternative is a persistently growing discrepancy between your list an the reality in people's homes, which could be hard to correct once the discrepancy gets so large that the list has become totally obsolete. Like it would be if you were still only listing the 32-bit performance of all engines.
That's the theory, but trying to incorporate a GPU to the CCRL reference HW can be problematic, if they want to keep it historically coherent. They started 12+ years ago, and the names of the lists (40/40 and 40/4) serve as the template for the adjusted TCs. They can only keep with the times as far as CPUs go, by comparing to a CPU of that time, but Leela has a much higher performance when run trough CUDA 9 (which limits you to Kepler or newer architectures). In essence, to test Leela fairly, you can't go back more than 6 years to choose a GPU. A system with an Athlon 64 X2 4600+ and a GeForce GTX 680? It could be, upgrading the VGA is more common (or was), certainly easier, but would they twist the rules to accommodate Leela? Would it be fair to other engines that get TC based on a CPU twice as old? Further adjustments seem to be in order.
When GPU improvements in the future will go at a different pace than CPU improvements, it might indeed be more difficult to emulate an older machine with a newer one. GPU-bound engines might need to use faster TC than CPU-bound programs. But is should not be impossible to test that way.
hgm wrote: ↑Wed May 30, 2018 9:53 pm
When GPU improvements in the future will go at a different pace than CPU improvements,
You just made that one up.
There is zero evidence for such a claim. Both CPU and GPU are subject to exactly same rules of technology scaling, have same power limitations in equivalent technology node, are subject to same leakage current, noise margins, etc.
