Laskos wrote: ↑Thu Oct 04, 2018 8:10 am
Now, the really great news for Lc0 18.1 Syzygy: from regular openings, it improves substantially due to Syzygy 6-men (WDL + DTZ) on SSD, more than regular engines. It is somehow unexpected to me, as from positions close to the TBs at root, it improves similarly to regular engines. I used Adam Hair's
4-mover PGN openings of pretty regular openings. Time control is short, 0.2s/move. I expect that at longer time control, the result might be even more impressive, as more games will be decided in later stages of the games. I played a gauntlet of 2000 games total, SF8_Syzygy against Lc0 18.1 ID11261 with and without Syzygy. GPU is GTX 1060, CPU is strong 4 core i7. Here is the result:
Code: Select all
Rank Name Elo +/- Games Score Draws
0 SF8_Syzygy 150 13 2000 70.3% 31.4%
1 lc0_v18.1_Syzygy -127 18 1000 32.5% 33.0%
2 lc0_v18.1 -174 19 1000 26.9% 29.8%
A whopping
47 Elo points improvement (
47 +/- 13 with 68% confidence) due to Syzygy 6-men. This is about 3 times larger improvement than what regular engines get from Syzygy from regular openings. In these conditions, in CCCC division with 8 engines, Lc0 might end in top 3, overcoming one of the uber-top 3. Probably not in top 2, though. If CCCC had a round-robin of only top 4 engines (and not 8), Lc0 had a good shot at entering top 2 and the final, because it performs better against stronger engines.
That was against a stronger engine in these conditions, SF8. But Lc0 performs pretty well against stronger engines. Where it underperforms is against weaker engines, it often happens that it fails to convert very favorable middlegames or won endgames. I was curious if v18.1 Syzygy (6-men on SSD) cures a bit of this problem against weaker engines.
No adjudications. GPU --- GTX 1060, CPU --- 4 core i7
The first gauntlet, 1000 games at
0.2s/move, against the weaker Arasan 20.5:
Code: Select all
Rank Name Elo +/- Games Score Draws
0 Arasan205 -135 20 1000 31.4% 25.1%
1 lc0_v18.1_Syzygy 162 29 500 71.8% 22.4%
2 lc0_v17 110 27 500 65.3% 27.8%
+52 Elo points due to Syzygy, with decreasing draw rate when using Syzygy, probably converting won endgames to actual wins
The second gauntlet, 200 games at
1.0s/move, against the same weaker Arasan 20.5:
Code: Select all
Rank Name Elo +/- Games Score Draws
0 Arasan205 -295 52 200 15.5% 21.0%
1 lc0_v18.1_Syzygy 328 83 100 86.8% 16.5%
2 lc0_v17 265 69 100 82.2% 25.5%
Finished match
+63 Elo pints due to Syzygy, but the error margins here are large. Now the draw rate decreased dramatically using Syzygy, while losses rate is pretty much the same, but win rate goes much higher. It means that some (many) games which non-Syzygy version was only able to draw, were actually won by Syzygy Lc0. Probably the usual occurrence with Lc0, not managing to win very advantageous endgames, is somewhat cured with Syzygy usage. I am curious whether in CCCC2, Lc0 will perform better against weaker engines, managing to be less drawish against them.