Leela vs Allie

[corres]

I run a match between Lc0 ver.0.22 and Allie-dev ver.0.5. Both engine used J13B.2-136 NET.
Used system: AMD Ryzen 7 1800x 8x4000 MHz and 2 x RTX 2060 Windows 10 64bits
Lc0 parameters:
NET J13B.2-136
Threads 4
Backend Multiplexing
BackendOptions backend=cudnn-fp16,(gpu=0),(gpu=1)
NNCachesIze 10000000
MiniBatchSize 512
MaxPrefetchBatch 64
Others DEFAULT
Allie parameters:
NET J13B.2-136
UseFP16 ON
GPUCores 2
Others DEFAULT
Tablebase syzygy 6 men
Opening positions 50 positions with reversed color
Time control 1 min + 2 sec/move
Results:
Lc0 ver.0.22 : Allie-dev ver.0.5 = 5 : 3 (92 draws, 100 games)
It is interesting Allie won the first 44 games (opened and half opened openings) Allie : Lc0 = 3 : 1 (40 draws)
even in the last 56 games (closed openings) Allie can not win one game either Lc0 : Allie = 4 : 0 (52 draws)

[sovaz1997]

That is why in TCEC finals, Stockfish can win Allie. If Leela were in her place, there would be practically no chance for Stockfish. Allie is now just lucky in the tournament, it is trite worse at a distance in all tests. I hope Leela will be in the finals (with Stockfish, or with Allie).
P. S. Thanks for testing! Of course, the sample is not very large, but it complements other tests.

[corres]

That is why in TCEC finals, Stockfish can win Allie. If Leela were in her place, there would be practically no chance for Stockfish. Allie is now just lucky in the tournament, it is trite worse at a distance in all tests. I hope Leela will be in the finals (with Stockfish, or with Allie).
P. S. Thanks for testing!
Of course, the sample is not very large, but it complements other tests.

Thanks.
Where can we find those other tests?

[sovaz1997]

That is why in TCEC finals, Stockfish can win Allie. If Leela were in her place, there would be practically no chance for Stockfish. Allie is now just lucky in the tournament, it is trite worse at a distance in all tests. I hope Leela will be in the finals (with Stockfish, or with Allie).
P. S. Thanks for testing!
Of course, the sample is not very large, but it complements other tests.

Thanks.
Where can we find those other tests?

In Discord, for example:

(same nets, diff binary)

Code: Select all

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)    W    L    D  CFS(%)
   1 lc0.net.LS10.1      :     0.0   ----   434.5     800    54  119   50  631     100
   2 Allie.net.LS10.1    :   -30.9   10.7   365.5     800    46   50  119  631     ---

(diff nets, diff binary)

Code: Select all

   # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)    W    L    D  CFS(%)
   1 lc0.net.T40B.4-160    :     0.0   ----   455.0     800    57  161   51  588     100
   2 Allie.net.LS10.1      :   -49.0   11.7   345.0     800    43   51  161  588  

Same net, diff binary.

[corres]

One can download the games from
http://wikisend.com/download
File ID: 596782
Password: allie
File: Lc0-Allie.zip

[corres]

That is why in TCEC finals, Stockfish can win Allie. If Leela were in her place, there would be practically no chance for Stockfish. Allie is now just lucky in the tournament, it is trite worse at a distance in all tests. I hope Leela will be in the finals (with Stockfish, or with Allie).
P. S. Thanks for testing!
Of course, the sample is not very large, but it complements other tests.

Thanks.
Where can we find those other tests?

In Discord, for example:
(same nets, diff binary)

Code: Select all

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)    W    L    D  CFS(%)
   1 lc0.net.LS10.1      :     0.0   ----   434.5     800    54  119   50  631     100
   2 Allie.net.LS10.1    :   -30.9   10.7   365.5     800    46   50  119  631     ---

(diff nets, diff binary)

Code: Select all

   # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)    W    L    D  CFS(%)
   1 lc0.net.T40B.4-160    :     0.0   ----   455.0     800    57  161   51  588     100
   2 Allie.net.LS10.1      :   -49.0   11.7   345.0     800    43   51  161  588  

Same net, diff binary.

Thanks.
What kind of net is LS10.1 and where it is?

[sovaz1997]

As far as I understand, this network allows Allie to be considered the original engine in TCEC (although I don't consider it original, having studied the code). You can find out everything in Discord: https://discord.gg/rHQ75Bn

[mwyoung]

That is why in TCEC finals, Stockfish can win Allie. If Leela were in her place, there would be practically no chance for Stockfish. Allie is now just lucky in the tournament, it is trite worse at a distance in all tests. I hope Leela will be in the finals (with Stockfish, or with Allie).
P. S. Thanks for testing!
Of course, the sample is not very large, but it complements other tests.

Thanks.
Where can we find those other tests?

In Discord, for example:
(same nets, diff binary)

Code: Select all

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)    W    L    D  CFS(%)
   1 lc0.net.LS10.1      :     0.0   ----   434.5     800    54  119   50  631     100
   2 Allie.net.LS10.1    :   -30.9   10.7   365.5     800    46   50  119  631     ---

(diff nets, diff binary)

Code: Select all

   # PLAYER                :  RATING  ERROR  POINTS  PLAYED   (%)    W    L    D  CFS(%)
   1 lc0.net.T40B.4-160    :     0.0   ----   455.0     800    57  161   51  588     100
   2 Allie.net.LS10.1      :   -49.0   11.7   345.0     800    43   51  161  588  

Same net, diff binary.

Thanks.
What kind of net is LS10.1 and where it is?

I can tell for the draw rate this test is meaningless. Way to low. What were the settings for Lc0, and Allie?

[corres]

...
I can tell for the draw rate this test is meaningless. Way to low. What were the settings for Lc0, and Allie?

Before you make a comment read the posts out and you can find the parameters.
These short time tests do not replace the events of TCEC, obviously.
However the TCEC/CCC events are not appropriate to measure the ELO of the participants either.
These are only a kind of information about the rank of the engines.

[mwyoung]

...
I can tell for the draw rate this test is meaningless. Way to low. What were the settings for Lc0, and Allie?

Before you make a comment read the posts out and you can find the parameters.
These short time tests do not replace the events of TCEC, obviously.
However the TCEC/CCC events are not appropriate to measure the ELO of the participants either.
These are only a kind of information about the rank of the engines.

I stand by my comments. The ranking and elo is meaningless. As I do my own testing also.