Lc0 26.2 10x128 nets vs Stockfish 12

Discussion of computer chess matches and engine tournaments.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
MMarco
Posts: 120
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Lc0 26.2 10x128 nets vs Stockfish 12

Post by MMarco » Sat Sep 12, 2020 2:41 am

Gauntlet: Lc0 26.2 10x128 nets vs Stockfish 12
Time control: 25s + 0.25s
Hardware: Ryzen 3750H, GTX 1660 Ti
Openings: Super GM 4 moves (500 pos.)
Bench (1 sec): Stockfish 12 = 1800 knps. Lc0 = 30 knps
Stockfish 12: 1 thread, hash=16, Move Overhead=0, abrok modern compile
Lc0 26.2 + 591226: minibatch-size=192, max-prefetch=86, syzygy-fast-play=false, move-overhead=0
Lc0 26.2 + other: minibatch-size=256, max-prefetch=115, syzygy-fast-play=false, move-overhead=0
MLH: moves-left-max-effect=0.1, moves-left-threshold=0.85, moves-left-slope=0.007, moves-left-quadratic-factor=1, moves-left-constant-factor=0, moves-left-scaled-factor=0
TBs and adj.: Syzygy 5-men, resign movecount=5 score=600, draw movenumber=35 movecount=5 score=5, time forfeit disabled
Games: https://gofile.io/d/l5TfkE
SV-nets: https://www.comp.nus.edu.sg/~sergio-v/new/128x10-t60-2/
jio-3450: https://cdn.discordapp.com/attachments/ ... 0000.pb.gz
tinker-6430: https://cdn.discordapp.com/attachments/ ... 0000.pb.gz

Code: Select all

   # PLAYER                  :  RATING  ERROR   PLAYED  (%)   CFS     W     D    L  D(%)
   1 Stockfish 12            :     0.0   ----    8000  54.1    99  1482  5689  829  71.1
   2 Lc0 26.2 + SV-5300      :   -13.7   11.2    1000  48.0    50   122   717  161  71.7
   3 Lc0 26.2 + SV-2990      :   -13.7   11.8    1000  48.0    79   114   733  153  73.3
   4 Lc0 26.2 + tinker-6430  :   -20.4   11.0    1000  47.1    89   116   710  174  71.0
   5 Lc0 26.2 + 591226       :   -30.2   10.4    1000  45.7    52    99   716  185  71.6
   6 Lc0 26.2 + jio-3450     :   -30.6   10.6    1000  45.6    79    93   727  180  72.7
   7 Lc0 26.2 + 703810 MLH   :   -37.3   12.1    1000  44.7    55   100   694  206  69.4
   8 Lc0 26.2 + 703810       :   -38.4   11.5    1000  44.5    82    94   703  203  70.3
   9 Lc0 26.2 + 721876       :   -45.5   10.7    1000  43.5   ---    91   689  220  68.9

White advantage = 11.44 +/- 2.11
Draw rate (equal opponents) = 72.22 % +/- 0.49
Depth stats:

Code: Select all

Engine                 Time/move  Depth   MIDG   EARLY    ENDG    LATE
Lc0 26.2 + 591226         0.57     9.49  10.28 | 11.05 |  8.77 |  7.77
Lc0 26.2 + 703810         0.58     9.48  10.08 | 10.74 |  9.01 |  7.84
Lc0 26.2 + 703810 MLH     0.58     9.55  10.11 | 10.78 |  9.17 |  7.84
Lc0 26.2 + 721876         0.59     9.34   9.83 | 10.63 |  8.87 |  7.90
Lc0 26.2 + SV-2990        0.58     9.19   9.99 | 10.05 |  8.48 |  7.64
Lc0 26.2 + SV-5300        0.57     9.11   9.83 | 10.35 |  8.40 |  7.71
Lc0 26.2 + jio-3450       0.57     9.15  10.00 | 10.77 |  8.45 |  7.36
Lc0 26.2 + tinker-6430    0.59     9.57  10.25 | 10.97 |  8.87 |  7.91
Stockfish 12              0.56    23.17  20.19 | 20.43 | 23.39 | 33.97

User avatar
Ozymandias
Posts: 1214
Joined: Sun Oct 25, 2009 12:30 am

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by Ozymandias » Sat Sep 12, 2020 8:48 pm

Nicely done, any particular reason for the MLH parameters?

MMarco
Posts: 120
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by MMarco » Sun Sep 13, 2020 1:30 am

Ozymandias wrote:
Sat Sep 12, 2020 8:48 pm
Nicely done, any particular reason for the MLH parameters?
I picked these as they are currently used by a poster ("Trre") who did and published many tests on the T70 series in Lc0 discord. I assumed he must know about it. 🙂

brianr
Posts: 442
Joined: Thu Mar 09, 2006 2:01 pm

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by brianr » Sun Sep 13, 2020 11:41 am

Thank you for including one of my best nets (tinker-6430).
Unfortunately, the yaml and training logs are lost to the mists of time, but I'm fairly certain there was no MLH used, FWIW.
This net was trained using SL with 6.4M steps using T60 games and then finished with 30K RL steps.

MMarco
Posts: 120
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by MMarco » Sun Sep 13, 2020 12:56 pm

brianr wrote:
Sun Sep 13, 2020 11:41 am
Thank you for including one of my best nets (tinker-6430).
Unfortunately, the yaml and training logs are lost to the mists of time, but I'm fairly certain there was no MLH used, FWIW.
This net was trained using SL with 6.4M steps using T60 games and then finished with 30K RL steps.
Thank you for training the net. Very impressive performance and among the best on the planet! Do you have any other 10b or 20b net that you would consider worthwhile testing? Larger than that my GPU suffers.

brianr
Posts: 442
Joined: Thu Mar 09, 2006 2:01 pm

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by brianr » Sun Sep 13, 2020 2:37 pm

I could train another without the rule50 move counter which would be roughly about 20 Elo stronger, but it would take quite a lot of time (between one and three weeks). Before starting that effort, I'm trying to understand things a bit more.
If I cook another net that is significantly better I'll provide an update.

MMarco
Posts: 120
Joined: Sat Apr 11, 2020 11:09 pm
Full name: Marc-O Moisan-Plante

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by MMarco » Sun Sep 13, 2020 11:45 pm

Sounds good! 10x128 are still useful when one doesn't have an RTX card, or for Android devices.

User avatar
Ozymandias
Posts: 1214
Joined: Sun Oct 25, 2009 12:30 am

Re: Lc0 26.2 10x128 nets vs Stockfish 12

Post by Ozymandias » Mon Sep 14, 2020 5:43 am

Or hyper-bullet games.

Post Reply