Leela Chess Zero Makes Mind-Blowing Piece Sacrifice!

[OliverBr]

If you wish you can enlighten us and write the winning line for white after 9...h6.

I let Leela find it herself. She wins easily against Stockfish after 9...h6:

[pgn][Event "Computer Chess Game"] [Site "OliversMacBook2.local"] [Date "2020.05.31"] [Round "-"] [White "Lc0 v0.25.1+git.unknown"] [Black "Stockfish 11 64"] [Result "1-0"] [TimeControl "40/180"] [Annotator "5. +0.54 5... -1.03"] 1. e4 {book} 1... e5 {book} 2. Nf3 {book} 2... Nc6 {book} 3. Bb5 {book} 3... Bc5 {book} 4. c3 {book} 4... Bb6 {book} 5. d4 {+0.54/13 1.0} 5... exd4 {-1.03/26 8} 6. cxd4 {+0.46/13 1.4} 6... Nce7 {-1.02/27 1.9} 7. O-O {+0.57/16 2.4} 7... c6 {-0.78/25 7} 8. d5 {+0.30/12 2.9} cxb5 {-0.50/23 6} 9. d6 {+0.58/16 9} h6 {+0.53/25 1.3} 10. e5 {+0.27/16} Nc6 {+1.09/25 3} 11. Nc3 {+0.40/17 4} Kf8 {+0.67/28 4} 12. Nd5 {+0.82/14 7} Qe8 {+1.62/23 2.4} 13. b4 {+1.39/15 5} f6 {-1.31/28 32} 14. a4 {+2.75/11 6} Nxe5 {-1.30/26 2.2} 15. axb5 {+3.00/12 2.8} Bd8 {-1.68/26 4} 16. b6 {+3.23/11 3} a5 {-2.14/27 16} 17. Nxe5 {+3.28/13 13} fxe5 {-0.69/23 1.6} 18. Rxa5 {+3.54/15 0.6} Ra6 {-3.29/33 21} 19. Qc2 {+6.61/12 7} Bxb6 {-3.45/26 3} 20. Nxb6 {+7.03/11 4} Rxb6 {-3.03/29 0.6} 21. b5 {+8.56/10 3} Kf7 {-2.85/26 0.3} 22. Qc7 {+18.79/8 9} Rxd6 {-4.26/28 0.2} 23. Qxd6 {+13.24/8 6} Qe6 {-4.35/30 0.1} 24. Ba3 {+13.73/8 9} Qxd6 {-4.69/31 0.7} 25. Bxd6 {+9.76/8 4} Nf6 {-4.69/29 0.3} 26. Bxe5 {+15.20/6 12} Nd5 {-4.30/29 6} 27. Rd1 {+40.57/5 7} Ne7 {-4.40/32 0.1} 28. Bd6 {+53.26/5 7} Nf5 {-4.57/30 0.4} 29. b6 {+50.28/5 6} Nxd6 {-6.57/32 29} 30. Rxd6 {+38.10/7 1.1} Re8 {-6.89/30 12} 31. Ra1 {+40.64/6 10} Ke7 {-5.97/24 3} 32. Rad1 {+35.49/6 5} {Black Resigns} 1-0[/pgn]

@Milos, your claims for this match are just based just on Stockfish analysis, the losing side. This shows that you are not very smart.

[zullil]

Analysis is not just press a button and forget (even though you might be doing it that way).
Lc0 doesn't have hash (its cashed moves are in no way replacement for hash just for faster running) and that is a total killer in any analysis (and it's not only Lc0, MCTS engines are not useful for analysis, period). Since you can't go forward and backward and get any useful result of it.

And if you do just press a button and forget, Lc0 needs huge amounts of memory or it will run out and abort. Not everyone has 512GB RAM (but good luck to those who do!).

Hö? What?
Leela, compiled by me with CUDA libraries, runs reliably on a 32GB RAM System and doesn't use any significant part of it.

Lc0 will quantify its memory needs for all to see in its log file. For example, using the (default) value of 200000 for NNCacheSize, a RAM limit of 32GB allows for 163,520,000 nodes.

0531 08:29:43.532464 140114558284224 ../../src/chess/uciloop.cc:222] << uciok
0531 08:30:04.139995 140114558284224 ../../src/chess/uciloop.cc:135] >> setoption name LogFile value lczero.log
0531 08:31:52.406748 140114558284224 ../../src/chess/uciloop.cc:135] >> setoption name RamLimitMb value 32768
0531 08:32:09.191241 140114558284224 ../../src/chess/uciloop.cc:135] >> ucinewgame
0531 08:32:09.192095 140114558284224 ../../src/neural/loader.cc:230] Found pb network file: ./384x30-swa-3350000.pb
0531 08:32:09.762279 140114558284224 ../../src/neural/factory.cc:91] Creating backend [cudnn-auto]...
0531 08:32:09.805644 140114558284224 ../../src/neural/cuda/network_cudnn.cc:1039] Switching to [cudnn-fp16]...
0531 08:32:10.026488 140114558284224 ../../src/neural/cuda/network_cudnn.cc:913] CUDA Runtime version: 10.2.0
0531 08:32:10.026569 140114558284224 ../../src/neural/cuda/network_cudnn.cc:926] Cudnn version: 7.6.5
0531 08:32:10.026581 140114558284224 ../../src/neural/cuda/network_cudnn.cc:936] Latest version of CUDA supported by the driver: 10.2.0
0531 08:32:10.026760 140114558284224 ../../src/neural/cuda/network_cudnn.cc:944] GPU: GeForce RTX 2080 Ti
0531 08:32:10.026776 140114558284224 ../../src/neural/cuda/network_cudnn.cc:945] GPU memory: 10.7534 Gb
0531 08:32:10.026803 140114558284224 ../../src/neural/cuda/network_cudnn.cc:947] GPU clock frequency: 1635 MHz
0531 08:32:10.026815 140114558284224 ../../src/neural/cuda/network_cudnn.cc:948] GPU compute capability: 7.5
0531 08:32:21.703573 140114558284224 ../../src/chess/uciloop.cc:135] >> go nodes 1000
0531 08:32:21.703707 140114558284224 ../../src/mcts/stoppers/stoppers.cc:104] RAM limit 32768MB. Cache takes 64MB. Remaining memory is enough for 163520000 nodes.

[OliverBr]

Lc0 will quantify its memory needs for all to see in its log file. For example, using the (default) value of 200000 for NNCacheSize, a RAM limit of 32GB allows for 163,520,000 nodes.

Not on my machine, see screenshot. Leela (CUDA) is using about 700Mbyte of Ram, no matter how long she is calculating.
Perhaps you are running her with other options?

PS: I don't use any extra options, except the CUDnn backend.
PPS: I cannot attach the screenshot, some board quota is exceeded.

[zullil]

Lc0 will quantify its memory needs for all to see in its log file. For example, using the (default) value of 200000 for NNCacheSize, a RAM limit of 32GB allows for 163,520,000 nodes.

Not on my machine, see screenshot. Leela (CUDA) is using about 700Mbyte of Ram, no matter how long she is calculating.
Perhaps you are running her with other options?

PS: I don't use any extra options, except the CUDnn backend.
PPS: I cannot attach the screenshot, some board quota is exceeded.

You are misinterpreting something. The search tree is kept in RAM, so RAM usage grows continually.

You can see the two options I used in my previous post. All others are at default values.

[OliverBr]

You are misinterpreting something. The search tree is kept in RAM, so RAM usage grows continually.

You can see the two options I used in my previous post. All others are at default values.

I understand. I am observing the RAM usage and indeed, it's looks as it is growing, albeit slowly. It's 800MB now.
EDIT: But now it's staying at 800MB for quite some time.

PS: She is winning against Stockfish 11. Again. With my hardware it's about:
50% Leela wins
50% draws
0% Stockfish wins

[zullil]

You are misinterpreting something. The search tree is kept in RAM, so RAM usage grows continually.

You can see the two options I used in my previous post. All others are at default values.

I understand. I am observing the RAM usage and indeed, it's looks as it is growing, albeit slowly. It's 800MB now.
PS: She is winning against Stockfish 11. Again.

See here: https://lczero.org/play/troubleshoot/

[OliverBr]

See here: https://lczero.org/play/troubleshoot/

I see. Strangely enough the RAM usage on my computer didn't exceed 800MB for a whole match against Stockfish. Is it possible that Leela behaves different on MacOSX oder Unix in general?

[pgn][Event "Computer Chess Game"] [Site "OliversMacBook2.local"] [Date "2020.05.31"] [Round "-"] [White "Lc0 v0.25.1+git.unknown"] [Black "Stockfish 11 64"] [Result "1-0"] [TimeControl "40/180"] [Annotator "1. +0.13 1... -0.08"] 1. e4 {+0.13/15} c5 {-0.08/26 9} 2. Nf3 {+0.16/16 4} e6 {+0.03/23 1.6} 3. d4 {+0.19/16 4} cxd4 {+0.25/21 2.0} 4. Nxd4 {+0.20/21 0.8} Nf6 {+0.15/23 3} 5. Nc3 {+0.20/23 1.9} Nc6 {+0.00/23 0.8} 6. Nxc6 {+0.19/23 10} bxc6 {+0.43/22 1.4} 7. e5 {+0.19/28 0.3} Nd5 {+0.37/21 1.8} 8. Ne4 {+0.21/25 0.4} Qc7 {+0.26/24 11} 9. f4 {+0.20/25 0.2} Qb6 {+0.17/22 1.7} 10. c4 {+0.20/24 0.3} Ne3 {-0.05/25 9} 11. Qd3 {+0.38/14 11} Nf5 {-0.34/25 0.1} 12. g4 {+0.36/14 11} Nd4 {-0.44/21 2.6} 13. Rb1 {+0.38/17 4} a5 {-0.35/25 4} 14. Bg2 {+0.25/13 8} h5 {+0.00/27 0.1} 15. g5 {+0.24/16 0.9} Nf5 {+0.00/27 1.2} 16. Ke2 {+0.17/15 10} Ba6 {+0.00/30 0.1} 17. b3 {+0.16/17 6} O-O-O {+0.00/30 1.5} 18. Nd6+ {+0.15/13 6} Bxd6 {+1.04/26 2.9} 19. exd6 {+0.09/14 0.3} c5 {+1.13/26 4} 20. Rd1 {+0.23/15 3} Bb7 {+0.57/27 13} 21. Bxb7+ {+0.20/19 0.2} Qxb7 {+0.38/26 5} 22. Kf2 {+0.19/21 0.3} h4 {+0.77/25 8} 23. h3 {+0.16/25 0.3} f6 {+1.01/23 3} 24. a3 {+0.75/15 10} Qa7 {-0.23/28 28} 25. b4 {+1.05/14 7} cxb4+ {-0.51/26 0.1} 26. Kg2 {+1.01/14 2.7} fxg5 {+0.00/27 0.1} 27. axb4 {+0.97/14 9} a4 {-0.84/27 10} 28. c5 {+1.09/14 0.2} Rhf8 {-0.89/25 2.8} 29. Kh2 {+1.22/18 2.0} gxf4 {-0.69/24 5} 30. Bxf4 {+1.24/17 2.1} Ng3 {-1.27/28 5} 31. Qe3 {+1.18/18 0.7} Kb8 {-0.86/29 9} 32. b5 {+1.09/20 1.1} Rxf4 {-1.00/27 1.0} 33. Qxf4 {+1.63/18 11} Qxc5 {-1.57/29 1.9} 34. Rbc1 {+1.51/24 6} Qf5 {-1.47/28 2.1} 35. Qe3 {+1.68/21 7} Rf8 {-1.46/26 2.9} 36. Rc7 {+1.68/22 0.3} Qf2+ {-1.98/28 7} 37. Qxf2 {+1.79/19 0.2} Rxf2+ {-2.06/28 5} 38. Kg1 {+1.88/17 0.2} Ra2 {-1.96/29 7} 39. Rd3 {+1.69/24 7} Ne2+ {-2.13/27 0.1} 40. Kf1 {+1.83/18 4} a3 {-2.18/27 2.2} 41. Rxd7 {+2.00/21 15} Ng3+ {-2.67/30 7} 42. Ke1 {+2.51/19 5} Ra1+ {-2.49/30 0.1} 43. Kd2 {+2.57/18 2.9} a2 {-2.41/29 2.7} 44. Ra3 {+2.84/18 6} Rg1 {-2.00/31 6} 45. Rd8+ {+4.01/14 23} Kb7 {-2.80/31 7} 46. d7 {+4.62/14 2.4} Kc7 {-2.77/31 1.4} 47. Rda8 {+4.49/14 9} Kxd7 {-2.76/31 0.2} 48. Rxa2 {+4.71/13 6} Kd6 {-2.65/29 4} 49. Ra1 {+5.60/13 18} Rg2+ {-3.66/29 26} 50. Kc3 {+5.91/13 9} Nf5 {-4.07/30 20} 51. Rb1 {+6.98/11 14} Rg3+ {-3.86/23 3} 52. Kb4 {+7.70/11 11} Ne3 {-3.85/29 1.0} 53. b6 {+9.84/11 14} Nd5+ {-4.28/35 0.1} 54. Kc4 {+17.08/8 10} Nxb6+ {-4.58/31 0.1} 55. Rxb6+ {+13.04/8 9} Ke5 {-5.22/32 16} 56. Rb3 {+12.33/8 14} g5 {-5.36/30 1.3} 57. Rf8 {+14.14/8 4} Ke4 {-5.64/28 0.5} 58. Rc3 {+17.51/8 5} e5 {-7.09/31 17} 59. Rf1 {+20.91/8 0.7} Rg2 {-7.12/30 2.5} 60. Re1+ {+24.39/7 5} Kf5 {-4.17/24 2.3} 61. Rf3+ {+41.48/7 2.6} Ke6 {-5.10/27 12} 62. Rfe3 {+46.71/6 3} Rc2+ {-5.81/28 9} 63. Kd3 {+43.78/6 2.4} Rc5 {-5.92/25 0.7} 64. Ke2 {+32.06/5 1.7} Ra5 {-5.90/26 6} 65. Kf3 {+29.66/5 1.4} Kf5 {-6.02/26 1.4} 66. Kg2 {+38.30/5 1.2} Rc5 {-6.03/24 1.2} 67. R1e2 {+48.07/5 1.1} Ke6 {-6.15/28 7} {Black Resigns} 1-0[/pgn]

[OliverBr]

It's really strange that there are people who doubt Leela's quality.

8. d5 is a little chess miracle. This opening has been played thousands of times before and no human player or chess engine saw this move before.

[jp]

What are the reported nodes per second TCEC gets for Leela on those 4 GPUs and SF on those 176 cores?

You do realize, that the NPS figures for AB and NN programs have nothing to do with each other.

Yes, of course, I realize. (We have been in discussions with others in other threads about this topic before, in case you don't remember.) But for a given NN size, the figures will reveal whether the hardware is "fair" according to different criteria people have suggested. (Not that any of them is my first preference.)

Thanks for the numbers. I'll have to see if I can do the comparison. (I have numbers for SF, not K, and regular NN size.) My guess is the the NN engine is being given far "too much" hardware power (by those criteria), which I guess is not surprising.

[jp]

Analysis is not just press a button and forget (even though you might be doing it that way).
Lc0 doesn't have hash (its cashed moves are in no way replacement for hash just for faster running) and that is a total killer in any analysis (and it's not only Lc0, MCTS engines are not useful for analysis, period). Since you can't go forward and backward and get any useful result of it.

And if you do just press a button and forget, Lc0 needs huge amounts of memory or it will run out and abort. Not everyone has 512GB RAM (but good luck to those who do!).

Hö? What?
Leela, compiled by me with CUDA libraries, runs reliably on a 32GB RAM System and doesn't use any significant part of it.

You are not doing very long analysis, like someone playing CC. If you analyze long enough, that 32GB will be used up easily.

For engine-engine games at normal TC, 32GB is fine. For analysis, it's not.