Sergio NN 40x512
Moderators: hgm, Rebel, chrisw
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
How many days do we need to wait for the next 40x512 net?
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
Second LC0 Sergio 40x512 net.Hai wrote: ↑Mon Jan 13, 2020 7:31 amI have very very very good news .mwyoung wrote: ↑Sun Jan 12, 2020 11:32 pmYes I don't know why the testing sites only play super lightning blitz tournament on only default settings and not the best settings except for me to rate NN engines. I am glad I do not. I guess they stupidly assume that all NN are created equal in regards to the default settings.Hai wrote: ↑Sun Jan 12, 2020 11:07 pmI'm interested in chess analysis and not in engine-super-lightning-bullet-tc-chess .mwyoung wrote: ↑Sun Jan 12, 2020 10:07 pmI guess the question is does this network 512x40 need 1, 2,3,4....High end graphic cards. Assuming it is a good net to be the best. Have you run any game test with this Fat Network. I will download it for a full round of testing. Have we now reached the limits of 1 RTX graphics card?Hai wrote: ↑Sun Jan 12, 2020 5:23 pmHi Nordlandia,Nordlandia wrote: ↑Sun Jan 12, 2020 4:32 pmThis is amazingHai wrote: ↑Sun Jan 12, 2020 4:06 pmThank you very much. It looks much better running LC0 Sergio 40x512 net with my 2x RTX 2080 Ti.Eduard wrote: ↑Sun Jan 12, 2020 3:02 pm A 512x40 test is running here:
https://www.comp.nus.edu.sg/~sergio-v/test/
Hello Hai. What speed do you get in this FEN position. I'm asking because i want to compare the speed difference. I'm running 2x RTX 2080 Super OC (ASUS ROG STRIX) Identical cards.
[d]1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w
Analyse for 1 minute with 512x40 net.
I'm using following settings tweaked from the defaults:
<------->
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
CPuct: 3.4
CPuctBase: 10000
MiniBatchSize: 512
<------->
Tablebases disabled.
Assuming you have two identical cards try same settings or multiplexing if they're not intentical and run that test for speed comparison
1 minute search:
1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w - - 0 1
Analysis by Lc0 v0.23.2+git.c8d9095:
1.Qg4 exf5 2.Qxg7+ Ke6 3.Qg6 Qe5 4.Qe8+ Re7 5.Qg8+ Rf7 6.Re1 Kd6 7.b6 Rd7 8.Qf8+ Re7 9.Qb8+ Ke6 10.Qc8+ Kd6 11.Kh1 Bxg2+ 12.Kxg2 Qe4+ 13.Kf2 Qe3+ 14.Kf1
White is better: +/- (1.14) Depth: 12/35 00:01:06 329kN
(12.01.2020)
2x 2080 Super output: speed 5544 N/s after about one minute.
we have nearly the same N/s and kN.
I'm using default settings.
And also:
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
+ 64 GB RAM
You have 2x 2080 Super and OC.
I have 2x 2080 Ti without Super and without OC.
Also I use power limit of 95% per GPU.
Also I use temperature limit of 70 per GPU.
So I've got lower power consumption, lower heat and it's also really quiet .
But GPU load is jumping between 60 to 90% on both GPUs when running your position.
When I run chess start position I have GPU load of 93% on both GPUs.
And CPU heat is 50.
I can test if this Fat Network can top the best on 1 RTX....
Like for every big net, at the beginning you should have 2x RTX GPUs, 2080 Ti if possible.
Like for every big net, later one RTX GPU, 2080 Ti if possible, will be enough.
-30x384-T60-2, 131 MB, : Ongoing, at 3rd LR (0.002) - That explains why 1 RTX GPU is enough.
-40x512, 318 MB, already started to train = less knowledge at the moment = to compensate it you sould use 2x RTX GPUs. But it also benefit much more from 3 or 4 RTX GPUs than the other nets.
-I can do now much longer analysis, when using the LC0 Sergio 40x512 net.
-I need much lesser RAM when running the 40x512 net.
-It is also possible to run it much longer than a 30x384 net before I run out of memory with my 64 GB RAM.
I expect the 40x512 net to get ~60 points in the ERET Test, because it's an untrained net at the moment.
Of course when using longer time control, the result will be better.
I have tested it also for analysis and games playing style.
It can already find some very difficult moves when using it on some fen positions to analyse. That's really amazing.
Strategically it is weaker at the moment and the positional style is also weaker at the moment.
But tactics are better .
I have found the bigger nets to be better at shorter time controls and longer time controls. NPS is a very poor gauge on what is the best NN at any time control. I have show this in my testing.
And I assume you know that every position in a chess game is a series of positions to analyze. And the engine that is most correct general wins the game. That is why we test. General the strongest chess playing engine will give you the best analysis.
Lets see where Sergio 40x512 stands in regards to the other engines and NN.
I've tested the frist 40x512 net with the ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 59 of 111 points.
Second result with 64 GB RAM and a limit of 300 seconds per move: 82 of 111 points.
This is an improvement of 23 points holy shit
83 / 111 = 74.7%.
Mean time = 36.40s / 9.97
Usually when doing the same test as described before with the smaller nets, the improvement was only ~2 points more than before.
This is the first net which have the capability to solve the complete ERET Test Suite after it is trained.
https://www.comp.nus.edu.sg/~sergio-v/test/512x40/
512x40-175.pb.gz 2020-01-24 13:49 309M
ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 62 of 111 points.
= 40x512 net needs only 4 days of training to solve one more of these extrem hard positions.
After only 3 months of training it should reach 85 of 111 points.
Maybe then we would see the first LR-Drop.
-
- Posts: 2821
- Joined: Fri Sep 25, 2015 9:38 pm
- Location: Sortland, Norway
Re: Sergio NN 40x512
2st LR is not long away. Sergio told he'll probably release it once reached 200k steps. Currently soon at 180k steps.
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
LR-Drops should be done, when after some training, the elo graph shows more or less no improvement.Nordlandia wrote: ↑Fri Jan 24, 2020 5:18 pm 2st LR is not long away. Sergio told he'll probably release it once reached 200k steps. Currently soon at 180k steps.
I think it doesn't make sense to do LR-Drop when the net is still improving elo after 200k step is reached.
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
LC0 Sergio 40x512 (50)
Hard Talkchess 2020 set
First result with 8 GB RAM and a limit of 15 seconds per move: 51 of 213 points = 23.9%.
Hard Talkchess 2020 set
First result with 8 GB RAM and a limit of 15 seconds per move: 51 of 213 points = 23.9%.
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
LC0 Sergio 40x512 (175)
Hard Talkchess 2020 set
First result with 8 GB RAM and a limit of 15 seconds per move: 43 of 213 points = 20.1%.
But it is better in the ERET test suite.
Hard Talkchess 2020 set
First result with 8 GB RAM and a limit of 15 seconds per move: 43 of 213 points = 20.1%.
But it is better in the ERET test suite.
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
Third LC0 Sergio 40x512 net.Hai wrote: ↑Fri Jan 24, 2020 10:36 amSecond LC0 Sergio 40x512 net.Hai wrote: ↑Mon Jan 13, 2020 7:31 amI have very very very good news .mwyoung wrote: ↑Sun Jan 12, 2020 11:32 pmYes I don't know why the testing sites only play super lightning blitz tournament on only default settings and not the best settings except for me to rate NN engines. I am glad I do not. I guess they stupidly assume that all NN are created equal in regards to the default settings.Hai wrote: ↑Sun Jan 12, 2020 11:07 pmI'm interested in chess analysis and not in engine-super-lightning-bullet-tc-chess .mwyoung wrote: ↑Sun Jan 12, 2020 10:07 pmI guess the question is does this network 512x40 need 1, 2,3,4....High end graphic cards. Assuming it is a good net to be the best. Have you run any game test with this Fat Network. I will download it for a full round of testing. Have we now reached the limits of 1 RTX graphics card?Hai wrote: ↑Sun Jan 12, 2020 5:23 pmHi Nordlandia,Nordlandia wrote: ↑Sun Jan 12, 2020 4:32 pmThis is amazingHai wrote: ↑Sun Jan 12, 2020 4:06 pmThank you very much. It looks much better running LC0 Sergio 40x512 net with my 2x RTX 2080 Ti.Eduard wrote: ↑Sun Jan 12, 2020 3:02 pm A 512x40 test is running here:
https://www.comp.nus.edu.sg/~sergio-v/test/
Hello Hai. What speed do you get in this FEN position. I'm asking because i want to compare the speed difference. I'm running 2x RTX 2080 Super OC (ASUS ROG STRIX) Identical cards.
[d]1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w
Analyse for 1 minute with 512x40 net.
I'm using following settings tweaked from the defaults:
<------->
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
CPuct: 3.4
CPuctBase: 10000
MiniBatchSize: 512
<------->
Tablebases disabled.
Assuming you have two identical cards try same settings or multiplexing if they're not intentical and run that test for speed comparison
1 minute search:
1q6/1r2k1p1/4pp1p/1P1b1P2/3Q4/7P/4B1P1/2R3K1 w - - 0 1
Analysis by Lc0 v0.23.2+git.c8d9095:
1.Qg4 exf5 2.Qxg7+ Ke6 3.Qg6 Qe5 4.Qe8+ Re7 5.Qg8+ Rf7 6.Re1 Kd6 7.b6 Rd7 8.Qf8+ Re7 9.Qb8+ Ke6 10.Qc8+ Kd6 11.Kh1 Bxg2+ 12.Kxg2 Qe4+ 13.Kf2 Qe3+ 14.Kf1
White is better: +/- (1.14) Depth: 12/35 00:01:06 329kN
(12.01.2020)
2x 2080 Super output: speed 5544 N/s after about one minute.
we have nearly the same N/s and kN.
I'm using default settings.
And also:
Roundrobin
(backend=cudnn-fp16,gpu=0),(backend=cudnn-fp16,gpu=1)
+ 64 GB RAM
You have 2x 2080 Super and OC.
I have 2x 2080 Ti without Super and without OC.
Also I use power limit of 95% per GPU.
Also I use temperature limit of 70 per GPU.
So I've got lower power consumption, lower heat and it's also really quiet .
But GPU load is jumping between 60 to 90% on both GPUs when running your position.
When I run chess start position I have GPU load of 93% on both GPUs.
And CPU heat is 50.
I can test if this Fat Network can top the best on 1 RTX....
Like for every big net, at the beginning you should have 2x RTX GPUs, 2080 Ti if possible.
Like for every big net, later one RTX GPU, 2080 Ti if possible, will be enough.
-30x384-T60-2, 131 MB, : Ongoing, at 3rd LR (0.002) - That explains why 1 RTX GPU is enough.
-40x512, 318 MB, already started to train = less knowledge at the moment = to compensate it you sould use 2x RTX GPUs. But it also benefit much more from 3 or 4 RTX GPUs than the other nets.
-I can do now much longer analysis, when using the LC0 Sergio 40x512 net.
-I need much lesser RAM when running the 40x512 net.
-It is also possible to run it much longer than a 30x384 net before I run out of memory with my 64 GB RAM.
I expect the 40x512 net to get ~60 points in the ERET Test, because it's an untrained net at the moment.
Of course when using longer time control, the result will be better.
I have tested it also for analysis and games playing style.
It can already find some very difficult moves when using it on some fen positions to analyse. That's really amazing.
Strategically it is weaker at the moment and the positional style is also weaker at the moment.
But tactics are better .
I have found the bigger nets to be better at shorter time controls and longer time controls. NPS is a very poor gauge on what is the best NN at any time control. I have show this in my testing.
And I assume you know that every position in a chess game is a series of positions to analyze. And the engine that is most correct general wins the game. That is why we test. General the strongest chess playing engine will give you the best analysis.
Lets see where Sergio 40x512 stands in regards to the other engines and NN.
I've tested the frist 40x512 net with the ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 59 of 111 points.
Second result with 64 GB RAM and a limit of 300 seconds per move: 82 of 111 points.
This is an improvement of 23 points holy shit
83 / 111 = 74.7%.
Mean time = 36.40s / 9.97
Usually when doing the same test as described before with the smaller nets, the improvement was only ~2 points more than before.
This is the first net which have the capability to solve the complete ERET Test Suite after it is trained.
https://www.comp.nus.edu.sg/~sergio-v/test/512x40/
512x40-175.pb.gz 2020-01-24 13:49 309M
ERET test suite.
First result with 8GB RAM and a limit of 15 seconds per move: 62 of 111 points.
= 40x512 net needs only 4 days of training to solve one more of these extrem hard positions.
After only 3 months of training it should reach 85 of 111 points.
Maybe then we would see the first LR-Drop.
https://www.comp.nus.edu.sg/~sergio-v/test/512x40/
512x40-320.pb.gz 2020-02-10 23:45 306M
ERET test suite
Result with 8GB RAM and a limit of 15 seconds per move: 69 of 111 points.
Improved from second to third net, from 62 to 69 points of 111 points. = +7.
Improved from second to third net by +11.29% .
Maybe we will see +15 from the next net.
-
- Posts: 598
- Joined: Sun Aug 04, 2013 1:19 pm
Re: Sergio NN 40x512
LC0 Sergio 40x512 (320)
Hard Talkchess 2020 set
8 GB RAM and a limit of 15 seconds per move: 59 of 213 points = 27.7%.
From second (175) to third (320) net, from 43 to 59 solved positions means an improvement of +37.20%!!
We need some engine matches.
I think the fourth net will be ready in 2 weeks and much stronger .
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Sergio NN 40x512
I think it would be more better if somebody would make a different NN for Chess960.
It would be very sad if it really would blow away the competition. It would be very bad for NN community and AB community also.
-
- Posts: 3657
- Joined: Wed Nov 18, 2015 11:41 am
- Location: hungary
Re: Sergio NN 40x512
The short Training time proves the new net strongly based on the earlier Lc0 training and because of this the weakpoints of the "new" net are the same as they are in Lc0 nets. So the new net in the reality an old net with some modification only. We can expect really improvement only from a fully new net I think.