In anticipation of my RTX 2070 GPU :)

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Laskos
Posts: 9496
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

In anticipation of my RTX 2070 GPU :)

Post by Laskos » Sun Oct 28, 2018 11:29 am

In at most a month I will have RTX 2070 GPU working on my desktop PC. I am just waiting to have a more diversified offer, maybe better cooling and maybe better prices. RTX 2070 gives a huge boost, a factor of 4 (!!!) compared to my GTX 1060 6GB. NVidia by allowing fp16 support, just blew out even my most optimistic expectations for AI/Machine Learning. As I am impatient to see how Leela would fare in new conditions against my top CPU engines, with a bit old i7 4790 4-core 3.8 GHz CPU, which is not a slouch, I played in varied conditions Lc0 ID11261 against SF_dev, by far the best CPU engine in 4:1 time handicap conditions with my "old" GTX 1060. So, Lc0 is at 240''+ 4'' time control on GTX 1060, SF_dev is on 4 i7 cores at 60''+ 1''.
The comparison is not fair, emulated GPU is state-of-the-art latest generation, more expensive and consumes more power than my a bit old CPU. Next step in several months will be to have an 8 core CPU AMD machine.

Leela Ratio at equal time control would be about 0.5, at this 4x time control, Leela Ratio is equivalent to ~2.0.
Lc0 is v18.1, ID11261.

Strength-wise, from 4-mover opening book of Adam Hair of regular openings, SF_dev and Lc0 are fairly equal:
  • Regular openings, 4-moves:
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 11 - 10 - 29 [0.510] 50
    Elo difference: 6.95 +/- 63.00
    Finished match

Having this equal result in the general play, I was curious how they compare in different conditions.


From the balanced middlegame positions, SF_dev is much stronger:
  • Middlegame balanced
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 15 - 1 - 34 [0.640] 50
    Elo difference: 99.95 +/- 51.76
    Finished match
The result is very skewed, and it means that if Lc0 doesn't get a serious advantage until the middlegame, it will either draw or lose, almost never win.


Another interesting aspect would be how Lc0 behaves in unfamiliar Queenless Chess:

  • Queenless Chess:
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 7 - 1 - 42 [0.560] 50
    Elo difference: 41.89 +/- 39.83
    Finished match
The result is again skewed, SF_dev winning decisively, with Lc0 having difficulty winning any game. Very drawish too, without Queens.


But not always unfamiliar positions disfavor Lc0. In this low draw-rate variant:


The fight already starts in the opening and usually ends in midgame, and Lc0 beats heavily SF_dev:
  • Variant
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 14 - 32 - 4 [0.320] 50
    Elo difference: -130.94 +/- 102.19
    Finished match
Probably, Lc0 is the strongest engine in this variant (say, with Leela Ratio of about 1 and not too short TC).


And finally, and endgame variant, which is borderline Black Win / Draw, as expected SF_dev beats heavily Lc0

  • Disbalance:
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 24 - 3 - 23 [0.710] 50
    Elo difference: 155.54 +/- 71.95
    Finished match


All in all, late midgames and endgames are the main weakness of Lc0, this can explain most of the results in under- or over- performance observed.

Werewolf
Posts: 1198
Joined: Thu Sep 18, 2008 8:24 pm

Re: In anticipation of my RTX 2070 GPU :)

Post by Werewolf » Sun Oct 28, 2018 12:35 pm

On SF vs Lc0 my hunch is that NNs perform best at summing up very complex situations with a good approximation, but more poorly (compared to alpha-betas) at exact move sequences where a string of best moves need calculation.

Therefore, I would guess that Lc0 will excel in positions which are so complicated they cannot be understood by current calculation depths: complex openings, early middle-games and maybe fortresses, perhaps? But in all other scenarios I would expect alpha-beta to be superior.

You're going to have a lot of fun with your 2070.

Javier Ros
Posts: 182
Joined: Fri Oct 12, 2012 10:48 am
Location: Seville (SPAIN)
Full name: Javier Ros

Re: In anticipation of my RTX 2070 GPU :)

Post by Javier Ros » Sun Oct 28, 2018 7:29 pm

Laskos wrote:
Sun Oct 28, 2018 11:29 am
In at most a month I will have RTX 2070 GPU working on my desktop PC. I am just waiting to have a more diversified offer, maybe better cooling and maybe better prices. RTX 2070 gives a huge boost, a factor of 4 (!!!) compared to my GTX 1060 6GB. NVidia by allowing fp16 support, just blew out even my most optimistic expectations for AI/Machine Learning. As I am impatient to see how Leela would fare in new conditions against my top CPU engines, with a bit old i7 4790 4-core 3.8 GHz CPU, which is not a slouch, I played in varied conditions Lc0 ID11261 against SF_dev, by far the best CPU engine in 4:1 time handicap conditions with my "old" GTX 1060. So, Lc0 is at 240''+ 4'' time control on GTX 1060, SF_dev is on 4 i7 cores at 60''+ 1''.
The comparison is not fair, emulated GPU is state-of-the-art latest generation, more expensive and consumes more power than my a bit old CPU. Next step in several months will be to have an 8 core CPU AMD machine.

Leela Ratio at equal time control would be about 0.5, at this 4x time control, Leela Ratio is equivalent to ~2.0.
Lc0 is v18.1, ID11261.

Strength-wise, from 4-mover opening book of Adam Hair of regular openings, SF_dev and Lc0 are fairly equal:
  • Regular openings, 4-moves:
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 11 - 10 - 29 [0.510] 50
    Elo difference: 6.95 +/- 63.00
    Finished match

Having this equal result in the general play, I was curious how they compare in different conditions.


From the balanced middlegame positions, SF_dev is much stronger:
  • Middlegame balanced
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 15 - 1 - 34 [0.640] 50
    Elo difference: 99.95 +/- 51.76
    Finished match
The result is very skewed, and it means that if Lc0 doesn't get a serious advantage until the middlegame, it will either draw or lose, almost never win.


Another interesting aspect would be how Lc0 behaves in unfamiliar Queenless Chess:

  • Queenless Chess:
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 7 - 1 - 42 [0.560] 50
    Elo difference: 41.89 +/- 39.83
    Finished match
The result is again skewed, SF_dev winning decisively, with Lc0 having difficulty winning any game. Very drawish too, without Queens.


But not always unfamiliar positions disfavor Lc0. In this low draw-rate variant:


The fight already starts in the opening and usually ends in midgame, and Lc0 beats heavily SF_dev:
  • Variant
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 14 - 32 - 4 [0.320] 50
    Elo difference: -130.94 +/- 102.19
    Finished match
Probably, Lc0 is the strongest engine in this variant (say, with Leela Ratio of about 1 and not too short TC).


And finally, and endgame variant, which is borderline Black Win / Draw, as expected SF_dev beats heavily Lc0

  • Disbalance:
    Score of SF_dev_Syzygy vs lc0_v18.1_Syzygy: 24 - 3 - 23 [0.710] 50
    Elo difference: 155.54 +/- 71.95
    Finished match


All in all, late midgames and endgames are the main weakness of Lc0, this can explain most of the results in under- or over- performance observed.
How is the ratio lc0 speed/ price of RTX 2070 compared with 2080 and 2080 Ti?
The love relationship between a chess engine tester and his computer can be summarized in one sentence:
Until heat do us part.

User avatar
Laskos
Posts: 9496
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: In anticipation of my RTX 2070 GPU :)

Post by Laskos » Sun Oct 28, 2018 7:44 pm

Javier Ros wrote:
Sun Oct 28, 2018 7:29 pm
How is the ratio lc0 speed/ price of RTX 2070 compared with 2080 and 2080 Ti?
Although there are a bit contradicting benches, according to all the data I have, RTX 2070 fares the best.
2080 is 1.4-1.6 times more expensive (at least here), but only at most 1.3 times faster.
2080 Ti is 2.2-2.4 times more expensive (here), but only at most 1.7 times faster.
I guess 2 x RTX 2070 are better price-wise and speed-wise than one RTX 2080 Ti, so I will have the possibility even to upgrade to a top-level GPU capacity at reasonable price.

User avatar
Laskos
Posts: 9496
Joined: Wed Jul 26, 2006 8:21 pm
Full name: Kai Laskos

Re: In anticipation of my RTX 2070 GPU :)

Post by Laskos » Tue Oct 30, 2018 8:47 am

Werewolf wrote:
Sun Oct 28, 2018 12:35 pm
On SF vs Lc0 my hunch is that NNs perform best at summing up very complex situations with a good approximation, but more poorly (compared to alpha-betas) at exact move sequences where a string of best moves need calculation.

Therefore, I would guess that Lc0 will excel in positions which are so complicated they cannot be understood by current calculation depths: complex openings, early middle-games and maybe fortresses, perhaps? But in all other scenarios I would expect alpha-beta to be superior.

You're going to have a lot of fun with your 2070.
Fortresses are not a strong point of Lc0. Out of 11 endgame fortresses, in these time control conditions, SF_dev broke 3/11 of them against Lc0, Lc0 didn't break any against SF_dev. Still, SF_dev broke 5/11 against Fruit 2.1, so Lc0 is at least not a disaster in these endgame fortresses.

shrapnel
Posts: 1245
Joined: Fri Nov 02, 2012 8:43 am
Location: New Delhi, India

Re: In anticipation of my RTX 2070 GPU :)

Post by shrapnel » Tue Oct 30, 2018 5:53 pm

I will watch your Test Results with the RTX 2070 with interest, Mr Laskos.
The day Lc0 starts beating the mickey out of Stockfish ( as AlphaZero did so memorably) using the 2070, I'll go out and grab myself a 2070 Ti, price be damned !
i7 5960X @ 4.1 Ghz, 64 GB G.Skill RipJaws RAM, Twin Asus ROG Strix OC 11 GB Geforce 2080 Tis

Werewolf
Posts: 1198
Joined: Thu Sep 18, 2008 8:24 pm

Re: In anticipation of my RTX 2070 GPU :)

Post by Werewolf » Wed Oct 31, 2018 9:15 am

shrapnel wrote:
Tue Oct 30, 2018 5:53 pm
I will watch your Test Results with the RTX 2070 with interest, Mr Laskos.
The day Lc0 starts beating the mickey out of Stockfish ( as AlphaZero did so memorably) using the 2070, I'll go out and grab myself a 2070 Ti, price be damned !
Yes this is my position also. At the moment I'm sticking to my humble 1060, but if Lc0 gets really good I'll be applying for a loan to get a dual 2080 Ti etc. etc. lol

smatovic
Posts: 932
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: In anticipation of my RTX 2070 GPU :)

Post by smatovic » Wed Oct 31, 2018 9:51 am

Werewolf wrote:
Wed Oct 31, 2018 9:15 am
shrapnel wrote:
Tue Oct 30, 2018 5:53 pm
I will watch your Test Results with the RTX 2070 with interest, Mr Laskos.
The day Lc0 starts beating the mickey out of Stockfish ( as AlphaZero did so memorably) using the 2070, I'll go out and grab myself a 2070 Ti, price be damned !
Yes this is my position also. At the moment I'm sticking to my humble 1060, but if Lc0 gets really good I'll be applying for a loan to get a dual 2080 Ti etc. etc. lol
:) welcome to the gpu hog-cycle.

Keep in mind that 7nm is at the door.

AMD is working on 7nm Navi gpu for 2019.
Maybe this will cause enough pressure on Nvidia to release 7 nm Ampere next year.

Who knows what happens with Intel entering the dedicated gpu market in 2020....

--
Srdja

Milos
Posts: 3387
Joined: Wed Nov 25, 2009 12:47 am

Re: In anticipation of my RTX 2070 GPU :)

Post by Milos » Wed Oct 31, 2018 10:09 am

smatovic wrote:
Wed Oct 31, 2018 9:51 am
Werewolf wrote:
Wed Oct 31, 2018 9:15 am
shrapnel wrote:
Tue Oct 30, 2018 5:53 pm
I will watch your Test Results with the RTX 2070 with interest, Mr Laskos.
The day Lc0 starts beating the mickey out of Stockfish ( as AlphaZero did so memorably) using the 2070, I'll go out and grab myself a 2070 Ti, price be damned !
Yes this is my position also. At the moment I'm sticking to my humble 1060, but if Lc0 gets really good I'll be applying for a loan to get a dual 2080 Ti etc. etc. lol
:) welcome to the gpu hog-cycle.

Keep in mind that 7nm is at the door.

AMD is working on 7nm Navi gpu for 2019.
Maybe this will cause enough pressure on Nvidia to release 7 nm Ampere next year.

Who knows what happens with Intel entering the dedicated gpu market in 2020....

--
Srdja
Lol, 7nm has nothing to do with AMD, AMD is fabless for god sake. Yes both AMD and IBM moved from GF to TSMC, but TSMC is not yet there for general purpose 7nm process, so everything related to 7nm release dates atm is just a rumor.

smatovic
Posts: 932
Joined: Wed Mar 10, 2010 9:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic
Contact:

Re: In anticipation of my RTX 2070 GPU :)

Post by smatovic » Wed Oct 31, 2018 10:42 am

Milos wrote:
Wed Oct 31, 2018 10:09 am
smatovic wrote:
Wed Oct 31, 2018 9:51 am
Werewolf wrote:
Wed Oct 31, 2018 9:15 am
shrapnel wrote:
Tue Oct 30, 2018 5:53 pm
I will watch your Test Results with the RTX 2070 with interest, Mr Laskos.
The day Lc0 starts beating the mickey out of Stockfish ( as AlphaZero did so memorably) using the 2070, I'll go out and grab myself a 2070 Ti, price be damned !
Yes this is my position also. At the moment I'm sticking to my humble 1060, but if Lc0 gets really good I'll be applying for a loan to get a dual 2080 Ti etc. etc. lol
:) welcome to the gpu hog-cycle.

Keep in mind that 7nm is at the door.

AMD is working on 7nm Navi gpu for 2019.
Maybe this will cause enough pressure on Nvidia to release 7 nm Ampere next year.

Who knows what happens with Intel entering the dedicated gpu market in 2020....

--
Srdja
Lol, 7nm has nothing to do with AMD, AMD is fabless for god sake. Yes both AMD and IBM moved from GF to TSMC, but TSMC is not yet there for general purpose 7nm process, so everything related to 7nm release dates atm is just a rumor.
Then take it as a rumor.

--
Srdja

Post Reply