The comparison is not fair, emulated GPU is state-of-the-art latest generation, more expensive and consumes more power than my a bit old CPU. Next step in several months will be to have an 8 core CPU AMD machine.
Leela Ratio at equal time control would be about 0.5, at this 4x time control, Leela Ratio is equivalent to ~2.0.
Lc0 is v18.1, ID11261.
Strength-wise, from 4-mover opening book of Adam Hair of regular openings, SF_dev and Lc0 are fairly equal:
- Regular openings, 4-moves:
Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 11 - 10 - 29 [0.510] 50
Elo difference: 6.95 +/- 63.00
Finished match
Having this equal result in the general play, I was curious how they compare in different conditions.
From the balanced middlegame positions, SF_dev is much stronger:
- Middlegame balanced
Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 15 - 1 - 34 [0.640] 50
Elo difference: 99.95 +/- 51.76
Finished match
Another interesting aspect would be how Lc0 behaves in unfamiliar Queenless Chess:
[d]rnb1kbnr/pppppppp/8/8/8/8/PPPPPPPP/RNB1KBNR w KQkq - 0 1
- Queenless Chess:
Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 7 - 1 - 42 [0.560] 50
Elo difference: 41.89 +/- 39.83
Finished match
But not always unfamiliar positions disfavor Lc0. In this low draw-rate variant:
[d]3rqknr/4bpp1/4bpp1/1PP1npp1/1PPN1pp1/1PPB4/1PPB4/RNKQR3 w - - 0 1
The fight already starts in the opening and usually ends in midgame, and Lc0 beats heavily SF_dev:
- Variant
Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 14 - 32 - 4 [0.320] 50
Elo difference: -130.94 +/- 102.19
Finished match
And finally, and endgame variant, which is borderline Black Win / Draw, as expected SF_dev beats heavily Lc0
[d]1nn1k1n1/4p3/8/8/8/PPPPPPPP/8/4K3 w - - 0 1
- Disbalance:
Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 24 - 3 - 23 [0.710] 50
Elo difference: 155.54 +/- 71.95
Finished match
All in all, late midgames and endgames are the main weakness of Lc0, this can explain most of the results in under- or over- performance observed.
welcome to the gpu hog-cycle.