LC0 11248 gauntlet vs new LC0 generation

Discussion of computer chess matches and engine tournaments.

Moderators: hgm, Harvey Williamson, bob

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: LC0 11248 gauntlet vs new LC0 generation

Post by majortom » Mon Jan 07, 2019 9:14 am

LCZero 11248 (714 nodes per move) vs LCZero 32367 (714 nodes per move)

Code: Select all

     Program               Elo      +      -   Games   Score   Av.Op.  Draws

   1 LC0 11248 N714    :    0.0    0.0    0.0   200    54.8 %    -34   51.5 %
   2 LC0 32367 N714    :  -33.6   35.0   35.0   200    45.3 %      0   51.5 %

Code: Select all

1 LC0 11248 N714 :  0.0 200 (+ 58,=103,- 39), 54.8 %

LC0 32367 N714   : 200 (+ 58,=103,- 39), 54.8 %

2 LC0 32367 N714 :-33.6 200 (+ 39,=103,- 58), 45.3 %

LC0 11248 N714   : 200 (+ 39,=103,- 58), 45.3 %
Games

majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: LC0 11248 gauntlet vs new LC0 generation

Post by majortom » Mon Jan 07, 2019 9:15 am

Code: Select all

     Program               Elo      +      -   Games   Score   Av.Op.  Draws

   1 LC0 11248 N714    :    0.0    0.0    0.0  1400    62.5 %    -91   45.1 %
   2 LC0 32367 N714    :  -33.6   32.5   32.5   200    45.3 %      0   51.5 %
   3 LC0 32177 N714    :  -42.5   34.0   34.0   200    44.0 %      0   46.0 %
   4 LC0 32015 N714    :  -82.6   37.3   37.3   200    38.5 %      0   45.0 %
   5 LC0 31741 N714    :  -88.2   37.4   37.4   200    37.8 %      0   45.5 %
   6 LC0 31885 N714    : -113.0   35.7   35.7   200    34.5 %      0   47.0 %
   7 LC0 31687 N714    : -136.9   36.9   36.9   200    31.5 %      0   42.0 %
   8 LC0 31633 N714    : -143.1   36.4   36.4   200    30.8 %      0   38.5 %

majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: LC0 11248 gauntlet vs new LC0 generation

Post by majortom » Mon Jan 14, 2019 7:31 pm

Number of nodes is increased in 10 times.

LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)

Code: Select all

    Program                Elo      +      -   Games   Score   Av.Op.  Draws

   1 LC0 32458 N7140    :   77.0   81.2   81.2    38    60.5 %      0   63.2 %
   2 LC0 11248 N7140    :    0.0    0.0    0.0    38    39.5 %     77   63.2 %

Code: Select all

1 LC0 32458 N7140 : 77.0 38 (+ 11,= 24,-  3), 60.5 %

LC0 11248 N7140   : 38 (+ 11,= 24,-  3), 60.5 %

2 LC0 11248 N7140 :  0.0 38 (+  3,= 24,- 11), 39.5 %

LC0 32458 N7140   : 38 (+  3,= 24,- 11), 39.5 %
Games

majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: LC0 11248 gauntlet vs new LC0 generation

Post by majortom » Fri Jan 18, 2019 11:30 am

LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)

Code: Select all

     Program                Elo      +      -   Games   Score   Av.Op.  Draws

   1 LC0 32458 N7140    :   54.2   61.9   61.9    60    57.5 %      0   61.7 %
   2 LC0 11248 N7140    :    0.0    0.0    0.0    60    42.5 %     54   61.7 %

Code: Select all

1 LC0 32458 N7140 : 54.2 60 (+ 16,= 37,-  7), 57.5 %

LC0 11248 N7140   : 60 (+ 16,= 37,-  7), 57.5 %

2 LC0 11248 N7140 :  0.0 60 (+  7,= 37,- 16), 42.5 %

LC0 32458 N7140   : 60 (+  7,= 37,- 16), 42.5 %
Games

brianr
Posts: 297
Joined: Thu Mar 09, 2006 2:01 pm

Re: LC0 11248 gauntlet vs new LC0 generation

Post by brianr » Fri Jan 18, 2019 1:07 pm

FWIW:

Code: Select all

Score of ID-32585 vs ID-11248: 61 - 63 - 176  [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.

Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.

Of course, 32585 is not 32458.

brianr
Posts: 297
Joined: Thu Mar 09, 2006 2:01 pm

Re: LC0 11248 gauntlet vs new LC0 generation

Post by brianr » Sun Jan 20, 2019 4:25 pm

brianr wrote:
Fri Jan 18, 2019 1:07 pm
FWIW:

Code: Select all

Score of ID-32585 vs ID-11248: 61 - 63 - 176  [0.497] 300
Elo difference: -2.32 +/- 25.28
At 320 games it was exactly even.

Match with v20.1 net 32585 v 11248 at 0:10+0.4 with 6 piece TBs and 2 move book on a GTX-1070.

Of course, 32585 is not 32458.
Update/correction. I ran a couple more matches. I changed the "new" net to 32747, which I think is the one selected for TCEC. The first was at the same time controls as the earlier one.

Code: Select all

Score of ID-32742 vs ID-11248: 19 - 25 - 56  [0.470] 100
Elo difference: -20.87 +/- 45.34
This result was in line with the earlier match with the other 30 series net.

However, there was talk on the Leela Discord chat about the newer nets scaling better than the older ones, so I did another match with tc=1:00+2.0 (1 min plus 2 seconds per move). The results were somewhat startling.

Code: Select all

Score of ID-32742 vs ID-11248: 27 - 12 - 61  [0.575] 100
Elo difference: 52.51 +/- 42.39
I suspect the difference will be even greater on stronger hardware.
Also, the 11248 net was trained with the 50 move bug.

majortom
Posts: 669
Joined: Mon Nov 04, 2013 9:19 pm
Contact:

Re: LC0 11248 gauntlet vs new LC0 generation

Post by majortom » Sat Jan 26, 2019 8:35 am

LCZero 11248 (7140 nodes per move) vs LCZero 32458(7140 nodes per move)

Code: Select all


     Program                Elo      +      -   Games   Score   Av.Op.  Draws

   1 LC0 32458 N7140    :   66.0   52.1   52.1   100    59.0 %      0   62.0 %
   2 LC0 11248 N7140    :    0.0    0.0    0.0   100    41.0 %     66   62.0 %

Code: Select all

1 LC0 32458 N7140 : 66.0 100 (+ 28,= 62,- 10), 59.0 %

LC0 11248 N7140   : 100 (+ 28,= 62,- 10), 59.0 %

2 LC0 11248 N7140 :  0.0 100 (+ 10,= 62,- 28), 41.0 %

LC0 32458 N7140   : 100 (+ 10,= 62,- 28), 41.0 %
Games

Post Reply