Sergio NN 40x512

[Hai]

How many days do we need to wait for the next 40x512 net?

[Hai]

A 512x40 test is running here:

https://www.comp.nus.edu.sg/~sergio-v/test/

Thank you very much. It looks much better running LC0 Sergio 40x512 net with my 2x RTX 2080 Ti.

This is amazing

Hello Hai. What speed do you get in this FEN position. I'm asking because i want to compare the speed difference. I'm running 2x RTX 2080 Super OC (ASUS ROG STRIX) Identical cards.

[d]1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w

Analyse for 1 minute with 512x40 net.

I'm using following settings tweaked from the defaults:

<------->
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
CPuct: 3.4
CPuctBase: 10000
MiniBatchSize: 512
<------->

Tablebases disabled.

Assuming you have two identical cards try same settings or multiplexing if they're not intentical and run that test for speed comparison

1 minute search:

1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w - - 0 1

Analysis by Lc0 v0.23.2+git.c8d9095:

1.Qg4 exf5 2.Qxg7+ Ke6 3.Qg6 Qe5 4.Qe8+ Re7 5.Qg8+ Rf7 6.Re1 Kd6 7.b6 Rd7 8.Qf8+ Re7 9.Qb8+ Ke6 10.Qc8+ Kd6 11.Kh1 Bxg2+ 12.Kxg2 Qe4+ 13.Kf2 Qe3+ 14.Kf1
White is better: +/- (1.14) Depth: 12/35 00:01:06 329kN
(12.01.2020)

2x 2080 Super output: speed 5544 N/s after about one minute.

Hi Nordlandia,

we have nearly the same N/s and kN.

I'm using default settings.
And also:
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
+ 64 GB RAM

You have 2x 2080 Super and OC.
I have 2x 2080 Ti without Super and without OC.
Also I use power limit of 95% per GPU.
Also I use temperature limit of 70 per GPU.
So I've got lower power consumption, lower heat and it's also really quiet .
But GPU load is jumping between 60 to 90% on both GPUs when running your position.

When I run chess start position I have GPU load of 93% on both GPUs.
And CPU heat is 50.

I guess the question is does this network 512x40 need 1, 2,3,4....High end graphic cards. Assuming it is a good net to be the best. Have you run any game test with this Fat Network. I will download it for a full round of testing. Have we now reached the limits of 1 RTX graphics card?

I can test if this Fat Network can top the best on 1 RTX....

I'm interested in chess analysis and not in engine-super-lightning-bullet-tc-chess .

Like for every big net, at the beginning you should have 2x RTX GPUs, 2080 Ti if possible.
Like for every big net, later one RTX GPU, 2080 Ti if possible, will be enough.
-30x384-T60-2, 131 MB, : Ongoing, at 3rd LR (0.002) - That explains why 1 RTX GPU is enough.
-40x512, 318 MB, already started to train = less knowledge at the moment = to compensate it you sould use 2x RTX GPUs. But it also benefit much more from 3 or 4 RTX GPUs than the other nets.

-I can do now much longer analysis, when using the LC0 Sergio 40x512 net.
-I need much lesser RAM when running the 40x512 net.
-It is also possible to run it much longer than a 30x384 net before I run out of memory with my 64 GB RAM.

I expect the 40x512 net to get ~60 points in the ERET Test, because it's an untrained net at the moment.
Of course when using longer time control, the result will be better.
I have tested it also for analysis and games playing style.
It can already find some very difficult moves when using it on some fen positions to analyse. That's really amazing.
Strategically it is weaker at the moment and the positional style is also weaker at the moment.
But tactics are better .

Yes I don't know why the testing sites only play super lightning blitz tournament on only default settings and not the best settings except for me to rate NN engines. I am glad I do not. I guess they stupidly assume that all NN are created equal in regards to the default settings.
I have found the bigger nets to be better at shorter time controls and longer time controls. NPS is a very poor gauge on what is the best NN at any time control. I have show this in my testing.

And I assume you know that every position in a chess game is a series of positions to analyze. And the engine that is most correct general wins the game. That is why we test. General the strongest chess playing engine will give you the best analysis.

Lets see where Sergio 40x512 stands in regards to the other engines and NN.

I have very very very good news .
I've tested the frist 40x512 net with the ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 59 of 111 points.
Second result with 64 GB RAM and a limit of 300 seconds per move: 82 of 111 points.

This is an improvement of 23 points holy shit
83 / 111 = 74.7%.
Mean time = 36.40s / 9.97
Usually when doing the same test as described before with the smaller nets, the improvement was only ~2 points more than before.
This is the first net which have the capability to solve the complete ERET Test Suite after it is trained.

Second LC0 Sergio 40x512 net.
https://www.comp.nus.edu.sg/~sergio-v/test/512x40/
512x40-175.pb.gz 2020-01-24 13:49 309M
ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 62 of 111 points.

= 40x512 net needs only 4 days of training to solve one more of these extrem hard positions.
After only 3 months of training it should reach 85 of 111 points.
Maybe then we would see the first LR-Drop.

[Nordlandia]

2st LR is not long away. Sergio told he'll probably release it once reached 200k steps. Currently soon at 180k steps.

[Hai]

2st LR is not long away. Sergio told he'll probably release it once reached 200k steps. Currently soon at 180k steps.

LR-Drops should be done, when after some training, the elo graph shows more or less no improvement.
I think it doesn't make sense to do LR-Drop when the net is still improving elo after 200k step is reached.

[Hai]

LC0 Sergio 40x512 (50)
Hard Talkchess 2020 set

First result with 8 GB RAM and a limit of 15 seconds per move: 51 of 213 points = 23.9%.

[Hai]

LC0 Sergio 40x512 (175)
Hard Talkchess 2020 set

First result with 8 GB RAM and a limit of 15 seconds per move: 43 of 213 points = 20.1%.

But it is better in the ERET test suite.

[Hai]

A 512x40 test is running here:

https://www.comp.nus.edu.sg/~sergio-v/test/

Thank you very much. It looks much better running LC0 Sergio 40x512 net with my 2x RTX 2080 Ti.

This is amazing

Hello Hai. What speed do you get in this FEN position. I'm asking because i want to compare the speed difference. I'm running 2x RTX 2080 Super OC (ASUS ROG STRIX) Identical cards.

[d]1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w

Analyse for 1 minute with 512x40 net.

I'm using following settings tweaked from the defaults:

<------->
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
CPuct: 3.4
CPuctBase: 10000
MiniBatchSize: 512
<------->

Tablebases disabled.

Assuming you have two identical cards try same settings or multiplexing if they're not intentical and run that test for speed comparison

1 minute search:

1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w - - 0 1

Analysis by Lc0 v0.23.2+git.c8d9095:

1.Qg4 exf5 2.Qxg7+ Ke6 3.Qg6 Qe5 4.Qe8+ Re7 5.Qg8+ Rf7 6.Re1 Kd6 7.b6 Rd7 8.Qf8+ Re7 9.Qb8+ Ke6 10.Qc8+ Kd6 11.Kh1 Bxg2+ 12.Kxg2 Qe4+ 13.Kf2 Qe3+ 14.Kf1
White is better: +/- (1.14) Depth: 12/35 00:01:06 329kN
(12.01.2020)

2x 2080 Super output: speed 5544 N/s after about one minute.

Hi Nordlandia,

we have nearly the same N/s and kN.

I'm using default settings.
And also:
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
+ 64 GB RAM

You have 2x 2080 Super and OC.
I have 2x 2080 Ti without Super and without OC.
Also I use power limit of 95% per GPU.
Also I use temperature limit of 70 per GPU.
So I've got lower power consumption, lower heat and it's also really quiet .
But GPU load is jumping between 60 to 90% on both GPUs when running your position.

When I run chess start position I have GPU load of 93% on both GPUs.
And CPU heat is 50.

I guess the question is does this network 512x40 need 1, 2,3,4....High end graphic cards. Assuming it is a good net to be the best. Have you run any game test with this Fat Network. I will download it for a full round of testing. Have we now reached the limits of 1 RTX graphics card?

I can test if this Fat Network can top the best on 1 RTX....

I'm interested in chess analysis and not in engine-super-lightning-bullet-tc-chess .

Like for every big net, at the beginning you should have 2x RTX GPUs, 2080 Ti if possible.
Like for every big net, later one RTX GPU, 2080 Ti if possible, will be enough.
-30x384-T60-2, 131 MB, : Ongoing, at 3rd LR (0.002) - That explains why 1 RTX GPU is enough.
-40x512, 318 MB, already started to train = less knowledge at the moment = to compensate it you sould use 2x RTX GPUs. But it also benefit much more from 3 or 4 RTX GPUs than the other nets.

-I can do now much longer analysis, when using the LC0 Sergio 40x512 net.
-I need much lesser RAM when running the 40x512 net.
-It is also possible to run it much longer than a 30x384 net before I run out of memory with my 64 GB RAM.

I expect the 40x512 net to get ~60 points in the ERET Test, because it's an untrained net at the moment.
Of course when using longer time control, the result will be better.
I have tested it also for analysis and games playing style.
It can already find some very difficult moves when using it on some fen positions to analyse. That's really amazing.
Strategically it is weaker at the moment and the positional style is also weaker at the moment.
But tactics are better .

Yes I don't know why the testing sites only play super lightning blitz tournament on only default settings and not the best settings except for me to rate NN engines. I am glad I do not. I guess they stupidly assume that all NN are created equal in regards to the default settings.
I have found the bigger nets to be better at shorter time controls and longer time controls. NPS is a very poor gauge on what is the best NN at any time control. I have show this in my testing.

And I assume you know that every position in a chess game is a series of positions to analyze. And the engine that is most correct general wins the game. That is why we test. General the strongest chess playing engine will give you the best analysis.

Lets see where Sergio 40x512 stands in regards to the other engines and NN.

I have very very very good news .
I've tested the frist 40x512 net with the ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 59 of 111 points.
Second result with 64 GB RAM and a limit of 300 seconds per move: 82 of 111 points.

This is an improvement of 23 points holy shit
83 / 111 = 74.7%.
Mean time = 36.40s / 9.97
Usually when doing the same test as described before with the smaller nets, the improvement was only ~2 points more than before.
This is the first net which have the capability to solve the complete ERET Test Suite after it is trained.

Second LC0 Sergio 40x512 net.
https://www.comp.nus.edu.sg/~sergio-v/test/512x40/
512x40-175.pb.gz 2020-01-24 13:49 309M
ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 62 of 111 points.

= 40x512 net needs only 4 days of training to solve one more of these extrem hard positions.
After only 3 months of training it should reach 85 of 111 points.
Maybe then we would see the first LR-Drop.

Third LC0 Sergio 40x512 net.
https://www.comp.nus.edu.sg/~sergio-v/test/512x40/
512x40-320.pb.gz 2020-02-10 23:45 306M

ERET test suite
Result with 8GB RAM and a limit of 15 seconds per move: 69 of 111 points.

Improved from second to third net, from 62 to 69 points of 111 points. = +7.
Improved from second to third net by +11.29% .
Maybe we will see +15 from the next net.

[Hai]

LC0 Sergio 40x512 (175)
Hard Talkchess 2020 set

First result with 8 GB RAM and a limit of 15 seconds per move: 43 of 213 points = 20.1%.

But it is better in the ERET test suite.

LC0 Sergio 40x512 (320)
Hard Talkchess 2020 set
8 GB RAM and a limit of 15 seconds per move: 59 of 213 points = 27.7%.
From second (175) to third (320) net, from 43 to 59 solved positions means an improvement of +37.20%!!
We need some engine matches.

I think the fourth net will be ready in 2 weeks and much stronger .

[corres]

trained with some Chess960 opening/games)
that would be simply awesome and blow away the competition.

I think it would be more better if somebody would make a different NN for Chess960.
It would be very sad if it really would blow away the competition. It would be very bad for NN community and AB community also.

[corres]

After only 3 months of training it should reach 85 of 111 points.

The short Training time proves the new net strongly based on the earlier Lc0 training and because of this the weakpoints of the "new" net are the same as they are in Lc0 nets. So the new net in the reality an old net with some modification only. We can expect really improvement only from a fully new net I think.