Useful only for comparing Lc with A0 (but then you'd also need to reproduce their other conditions and settings, which you probably can only guess at).
Bad name, too. It's got nothing to do with Leela.
Moderators: hgm, Rebel, chrisw
Useful only for comparing Lc with A0 (but then you'd also need to reproduce their other conditions and settings, which you probably can only guess at).
You can change Leela Ratio settings to match conditions you would use for different time controls, as I show in the doc, if you’d like. Bullet vs TCEC time controls, for example. But the changes to nncache, like you mention, won’t affect the ratio much, unless you’re choking Leela by giving way too little. The biggest factor in changing the ratio’s value is backend, as I’m sure you’re aware. As long as that’s correct, you’ll be close. You mentioned a 10-15% margin in other test groups for CPU testing being acceptable. If we can get our GPU to CPU ratios within that margin, why not? That’s a lot better than a 1.5x - 3.8x difference in my rig.Laskos wrote: ↑Thu Feb 28, 2019 8:19 am ... The so called effective "Leela Ratio" is different by a factor of 2 or more going from say 15''+0.25'' to 240''+4'', and maybe a factor of 3 or more compared to this "benchmark". The size of cache also matters...
This is still useful benchmark to determine the general hardware configuration GPU vs CPU in Leela case, but that's pretty much all.
Exactly.
Thanks for the link.brianr wrote: ↑Thu Feb 28, 2019 2:15 pmI have found that when running on a GPU there is significant startup overhead and the NPS increases very rapidly until about 2 seconds (and slowly continues to increase after that until the nncache fills). This is on a 1070. Accordingly, although the games take more time, I no longer test at faster time controls than with an increment of 2 seconds. I have seen match results completely reversed at faster TCs vs 1+2, even well within the margin of error. Leela NPS is also much more variable depending on the position than the NPS for CPU engines. Of course, CPUs have practically no overhead, so the ultra-rapid TC testing is fine. Moreover, the Leela Ratio only applies to nets with a specific size/architecture, as has been pointed out.Laskos wrote: ↑Thu Feb 28, 2019 8:19 am I stressed what I wanted to say because I often use very short time controls, similar to those in Fishtest testing framework, often one needs many games. The so called effective "Leela Ratio" is different by a factor of 2 or more going from say 15''+0.25'' to 240''+4'', and maybe a factor of 3 or more compared to this "benchmark". The size of cache also matters. With CPUs the speeds vary only by at most 10-15%% on these factors, not 300%, that's why CCRL benchmark is of much more use.
This is still useful benchmark to determine the general hardware configuration GPU vs CPU in Leela case, but that's pretty much all.
Just noticed more info in this post:
viewtopic.php?f=7&t=67347
I agree it is meaningless. My ratio is 2950x vs 2080 ti.