LCZero update

[jkiliani]

As of today, the network match system is active, and a new neural net is currently being tested which will almost certainly pass. The individual match games can be viewed online like the training games on http://lczero.org/match/13, but unlike training games, match games are played without temperature=1. They contain way less blunders and look almost like chess already

[nabildanial]

Wouldn't it be faster if initially you train against Stockfish? Once the learning curve levels you can continue with self-play.

Yes, it would be faster, way faster. And it was already tried and it worked fine and Leela was indeed quite strong. But that's not the idea of the project, it was just made as a test to see if all was working ok in a quick manner.

The idea of the project is to have an engine that learsn from zero, by playing itself. Self-learning, not reinforced learning. That's why the "Zero". It starts from zero knowledge. If you introduce Stockfish, then it's not zero anymore.

May I know what is the motivation behind starting the project from zero instead of doing reinforced learning first then self-plays? As a contributor of computation time, the idea behind that doesn't look attractive to me at all.

[jkiliani]

May I know what is the motivation behind starting the project from zero instead of doing reinforced learning first then self-plays? As a contributor of computation time, the idea behind that doesn't look attractive to me at all.

I'm going to assume you mean doing supervised learning from Stockfish games first, and then self-play reinforcement learning? Otherwise I don't understand the question.

The concept is to avoid imparting assumptions on the neural net that could potentially turn out to be wrong on closer inspection. Personally I think it would not make much of a difference, since reinforcement learning tends to be quite robust in eventually finding the global optimum, regardless of where you start from. At this point it actually isn't that much of an issue anymore either way, since the neural net is rapidly approaching the strength of the supervised net from the Kingbase database kbb1-64x6-796000.zip (which you can find here: https://github.com/glinscott/lczero-weights). So the initial investment is mostly done already anyway.

[koedem]

Wouldn't it be faster if initially you train against Stockfish? Once the learning curve levels you can continue with self-play.

Yes, it would be faster, way faster. And it was already tried and it worked fine and Leela was indeed quite strong. But that's not the idea of the project, it was just made as a test to see if all was working ok in a quick manner.

The idea of the project is to have an engine that learsn from zero, by playing itself. Self-learning, not reinforced learning. That's why the "Zero". It starts from zero knowledge. If you introduce Stockfish, then it's not zero anymore.

May I know what is the motivation behind starting the project from zero instead of doing reinforced learning first then self-plays? As a contributor of computation time, the idea behind that doesn't look attractive to me at all.

The motivation is a) to "verify" that a zero approach can work in chess and
b) the fact that AlphaGoZero eventually surpassed AlphaGoMaster implying that the zero approach actually leads to better results in the long run, see also wiki:
"Training artificial intelligence (AI) without datasets derived from human experts has significant implications for the development of AI with superhuman skills because expert data is "often expensive, unreliable or simply unavailable."[3] Demis Hassabis, the co-founder and CEO of DeepMind, said that AlphaGo Zero was so powerful because it was "no longer constrained by the limits of human knowledge"."

[jkiliani]

The motivation is a) to "verify" that a zero approach can work in chess and b) the fact that AlphaGoZero eventually surpassed AlphaGoMaster implying that the zero approach actually leads to better results in the long run, see also wiki: "Training artificial intelligence (AI) without datasets derived from human experts has significant implications for the development of AI with superhuman skills because expert data is "often expensive, unreliable or simply unavailable."[3] Demis Hassabis, the co-founder and CEO of DeepMind, said that AlphaGo Zero was so powerful because it was "no longer constrained by the limits of human knowledge"."

I agree with a), but I think b) is actually problematic. There are strong indications that Alphago Zero only surpassed Alphago Master because Deepmind wanted that message for their paper, and that they achieved it simply by giving Zero a deeper net (more residual blocks) than Master. David Silver actually confirmed in a talk that Master used 40 layers i.e. 20 blocks, while Zero was eventually given 40 residual blocks.

[nabildanial]

I'm going to assume you mean doing supervised learning from Stockfish games first, and then self-play reinforcement learning? Otherwise I don't understand the question.

Yes, that's what I wanted to say. Thanks for the explanation.

[lucasart]

No one has a clue how much random play is. There just aren't enough intermediate opponents to bridge the gap between random movers and searching engines. The weak engines that seem to fit in there are all very buggy, and do not behave according to the Elo model: they have a fixed finite probability to lose against any opponent, no matter how weak, because they crash.

So random play could be 3000 Elo below Stockfish. Or 30,000 Elo. We just don't know.

The CCRL list in this region is a bit suspect. E.g. RAM is supposed to be a random mover. So how can it be at the level of NEG, ~300 Elo above Brutus RND? I am also pretty sure that NEG scores 99% against a random mover, much more than the Elo difference suggests. The rating of the random mover must be highly inflated because it gets free points from buggy engine higher in the list. You cannot make a sensible rating list with engines that hand out free points, irrespective of opponent performance.

Completely agree.

I think the gap between random play and SF is immense. Much higher than what rating lists indicate.

I am looking at lczero.org now, and I can see that:
* game play is indistinguishable from random (to my untrained eye)
* yet, it is showing +3,100 elo vs. random play

By this metric, SF could easily be 30,000 elo.

[jhellis3]

Most of the games on the site are not a good reflection of the network strength because they are played with a high level of noise.

If you click on Matches and then the ID for the match you can see more representative games. The current network which is self-rated at ~3100 Elo is probably ~1050 "real world" Elo from what I have seen of others' tests.

[Nay Lin Tun]

[pgn]1. g3 e5 2. Bg2 a5 3. h4 Nf6 4. Nc3 d6 5. e4 h5 6. d3 Bg4 7. f3 c6 8. fxg4 hxg4 9. Bg5 Be7 10. Bxf6 Bxf6 11. Qxg4 Na6 12. Qd1 Kf8 13. Nh3 Rh6 14. Nf2 Nc5 15. Rb1 b5 16. Ng4 Rh7 17. a3 b4 18. axb4 axb4 19. Ne2 d5 20. O-O d4 21. Nxf6 gxf6 22. Ra1 Rxa1 23. Qxa1 b3 24. Qd1 Rg7 25. cxb3 Na6 26. h5 Rh7 27. Kh2 Rxh5+ 28. Kg1 Nb4 29. g4 Rh4 30. Qe1 Rh6 31. Qxb4+ Ke8 32. Qd2 Rg6 33. Bf3 Qb6 34. Rc1 Rg8 35. b4 Kd7 36. b3 Ra8 37. g5 Rg8 38. Rc5 fxg5 39. Rxe5 g4 40. Bh1 f6 41. Rc5 g3 42. Nxd4 g2 43. Nxc6 gxh1=Q+ 44. Kxh1 Kd6 45. d4 Re8 46. Qf4+ Ke6 47. Ra5 Rf8 48. Ra7 Rh8+ 49. Kg2 Rg8+ 50. Kf3 Qxc6 51. d5+ Qxd5 52. exd5+ Kxd5 53. Qc4+ Ke5 54. Qxg8 f5 55. b5 f4 56. b6 Kf6 57. b7 Ke5 58. b8=Q+ Kf5 59. Qxf4# 1-0[/pgn]

The first game is from match 13 and the second one is the latest game from my machine. The blunder, dropping pieces, are still frequent, but i agreed that these ugly moves are much less than yesterday.

[pgn]1. d3 d5 2. h4 Nc6 3. Na3 Nf6 4. Nh3 Bxh3 5. Rxh3 a5 6. e3 Ng8 7. Nb5 Rb8 8. Kd2 e5 9. Ke1 Nd4 10. exd4 c6 11. dxe5 cxb5 12. a4 Bb4+ 13. c3 Be7 14. axb5 Bf8 15. Qa4 Qc7 16. b6+ Qc6 17. Qxc6+ bxc6 18. Bf4 Kd8 19. Rg3 Rb7 20. b3 c5 21. Rxa5 f6 22. Ra7 Rxb6 23. Rf3 d4 24. Kd2 h6 25. cxd4 Ne7 26. exf6 Rxf6 27. Be5 Rxf3 28. gxf3 Nc8 29. Ra2 Rh7 30. Ra4 Na7 31. Rxa7 cxd4 32. Bxd4 Ke8 33. Ra6 Kf7 34. Ra7+ Ke6 35. f4 Bd6 36. Rxg7 Bxf4+ 37. Kc2 Rh8 38. Be2 Rg8 39. Rxg8 Kd7 40. Rg7+ Kd6 41. Rh7 Bh2 42. Bf1 Bg1 43. Rxh6+ Kc7 44. Rh8 Bxf2 45. Bb2 Kb7 46. Bg7 Bxh4 47. b4 Bf2 48. Bg2+ Kc7 49. Bc3 Bb6 50. Ba8 Kd6 51. Bh1 Kd7 52. Be5 Be3 53. b5 Bh6 54. Kd1 Bg5 55. Kc2 Be7 56. Bb8 Bf6 57. Bf4 Bxh8 58. Bh2 Bf6 59. Kd1 Bd4 60. Bg3 Bb6 61. Be4 Ba7 62. Ke1 Bc5 63. Bb8 Bb4+ 64. Kf2 Kc8 65. Kg1 Kxb8 66. Kh1 Ba5 67. d4 Kc7 68. Bg2 Kb6 69. Bc6 Kc7 70. Bg2 Kb6 71. Bf1 Bb4 72. Kg1 Kb7 73. Kh1 Be7 74. Kh2 Bd8 75. Be2 Kc8 76. Kh1 Kc7 77. d5 Kb7 78. Bf3 Kb6 79. Bg2 Kc5 80. Bf3 Kxb5 81. Bh5 Ba5 82. Bf7 Kb4 83. Be6 Bd8 84. Bg8 Kc5 85. Bf7 Ba5 86. Kg2 Bb4 87. Kh3 Kd6 88. Kh4 Ke7 89. d6+ Kd7 90. Kg4 Be1 91. Bg6 Bb4 92. Bh5 Bxd6 93. Bf7 Bb8 94. Kf5 Kd6 95. Bh5 Ba7 96. Bf3 Kc5 97. Ba8 Kb6 98. Ke6 Bb8 99. Kf5 Ka5 100. Kg6 Kb4 101. Bb7 Ba7 102. Ba6 Bb6 103. Kf6 Bd8+ 104. Ke6 Ka5 105. Bb5 Kb6 106. Kf5 Kxb5[/pgn]

[CMCanavessi]

It's looking MUCH better than a couple of days ago... I'm impressed by the speed of the improvement. I expected way less.

[pgn][Event "021 - LCZero Gen 12 Gauntlet II"]
[Site "RYZEN"]
[Date "2018.03.20"]
[Round "2"]
[White "Sabrina 3.1.25 x64"]
[Black "Leela Chess Zero Gen 12 x64"]
[Result "0-1"]
[ECO "D15"]
[Opening "Slav"]
[Time "20:20:04"]
[Variation "Accepted, Alekhine: 5...b5 6.a4 b4 7.Nb1"]
[TimeControl "60+1"]
[Termination "normal"]
[PlyCount "140"]
[WhiteType "program"]
[BlackType "program"]

1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3 dxc4 5. e3 b5 6. a4 b4 7. Nb1 Ba6 8.
Nbd2 c3 9. Nb1 {(9.Nb1) -1.76/12 1} Bxf1 {(9. ... Bxf1 10.Kxf1 e5 11.bxc3
e4 12.cxb4 exf3 13.gxf3 Bxb4 14.Bd2 Bxd2) +1.42/19 2} 10. Kxf1 {(10.Kxf1)
-1.60/12 1} cxb2 {(10. ... cxb2 11.Bxb2 a5 12.Ne5 e6 13.f3 Nd5) +1.28/19 2}
11. Bxb2 {(Bc1xPb2) -1.46/12 1} a5 {(11. ... a5 12.Nbd2 Ng4 13.h3 Nxf2
14.Kxf2 f6 15.Nb3) +0.99/19 2} 12. Nbd2 {(12.Nbd2) -1.17/10 1} e6 {(12. ...
e6 13.h3 Qd5 14.e4 Nxe4 15.Nxe4 Qxe4) +0.82/19 2} 13. Ke2 {(13.Ke2)
-1.17/10 1} Be7 {(13. ... Be7 14.Nc4 Nbd7 15.Nce5 Qc7 16.Qb1 Bd6) +2.14/19
2} 14. Qc2 {(14.Qc2) -1.34/10 1} Nbd7 {(14. ... Nbd7 15.Rad1 h6 16.h3 Qb8
17.Rhg1) +2.43/19 2} 15. e4 {(Pe3-e4) -1.17/10 1} h6 {(15. ... h6 16.e5 Nd5
17.Qb3 Rb8 18.h4 Qc7) +2.46/19 2} 16. e5 {(Pe4-e5) -1.30/10 1} Nd5 {(16.
... Nd5 17.Rad1 Qb8 18.Ba1 g5 19.Rhg1) +2.16/19 2} 17. g4 {(Pg2-g4)
-1.30/10 1} h5 {(17. ... h5 18.g5 h4 19.Qb3 Qb8 20.h3 Rg8) +2.75/19 2} 18.
gxh5 {(Pg4xPh5) -1.64/12 1} Rxh5 {(18. ... Rxh5 19.Rab1 Rf5 20.Rhg1 Qb8
21.Qxf5 exf5 22.h4) +2.56/19 2} 19. Rag1 {(19.Rag1) -1.69/10 1} Rh8 {(19.
... Rh8 20.h4 Qb8 21.Nc4 Rg8 22.Nxa5) +2.02/19 2} 20. Rxg7 {(Rg1xPg7)
-0.52/10 1} Kf8 {(20. ... Kf8 21.Rh7 Rxh7 22.Qxh7 N7f6 23.Qh8+ Ng8 24.Rc1
Rb8 25.Qh7) +1.90/19 2} 21. Rg3 {(21.Rg3) -0.15/10 1} Qe8 {(21. ... Qe8
22.h4 f6 23.exf6 N7xf6 24.Ne5 Rb8 25.Ndc4) +2.41/19 2} 22. Kd3 {(22.Kd3)
-0.60/10 1} N7b6 {(22. ... N7b6 23.Bc1 Nxa4 24.Ne4 Nac3) +3.12/19 2} 23.
Rd1 {(23.Rd1) -0.13/10 1} Rb8 {(23. ... Rb8 24.Ne4 Nxa4 25.Bc1 b3 26.Qb1)
+3.22/19 2} 24. Qb3 {(24.Qb3) -0.11/10 1} c5 {(24. ... c5 25.Ba1 Nxa4
26.Rdg1 cxd4 27.Nxd4 Nac3 28.Nc4 a4) +3.46/19 2} 25. dxc5 {(Pd4xPc5)
-1.58/10 1} Nxa4 {(25. ... Nxa4 26.Kc4 Nxb2+ 27.Qxb2 Nc3 28.Rc1 a4 29.Nd4
a3 30.Qb3) +4.93/19 2} 26. Bd4 {(26.Bd4) -1.81/10 1} Nac3 {(26. ... Nac3
27.Ra1 a4 28.Qc4 Nf4+ 29.Ke3 Nfd5+ 30.Kd3 f5 31.c6 f4) +3.74/19 2} 27. Re1
{(27.Re1) -1.75/10 1} a4 {(27. ... a4 28.Qc4 a3 29.Ra1 Na4 30.Qa6 Nac3
31.Qa7 Ra8 32.Qxa8) +4.47/19 2} 28. Qc4 {(28.Qc4) -1.65/10 1} a3 {(28. ...
a3 29.Ra1 Ra8 30.c6 a2 31.c7 Nxc7 32.Bxc3 bxc3 33.Ne4 c2) +3.77/20 2} 29.
Bxc3 {(29.Bxc3) -1.69/10 1} bxc3 {(29. ... bxc3 30.Nb1 Rb4 31.Qa6 Rb8
32.Nxa3 Nb4+ 33.Kxc3 Nxa6 34.h4 Nxc5) +4.04/19 2} 30. Qg4 {(30.Qg4)
-1.69/10 1} Bxc5 {(30. ... Bxc5 31.Qg7+ Ke7 32.Ne4 a2 33.Nxc5 a1Q 34.Rxa1)
+7.65/19 2} 31. Qg7+ {(31.Qg7+) -1.30/12 1} Ke7 {(31. ... Ke7 32.Nc4 Rb2
33.Nxb2 cxb2 34.Rb1 a2 35.Rxb2 a1Q 36.Rc2 Rg8) +4.86/19 2} 32. Qg5+
{(32.Qg5+) -3.28/12 1} Kf8 {(32. ... Kf8 33.Nc4 Rb2 34.Ra1 c2 35.Nxb2 axb2
36.Kxc2 bxa1Q) +6.49/19 2} 33. Qg7+ {(33.Qg7+) -1.30/12 1} Ke7 {(33. ...
Ke7 34.Ra1 cxd2 35.Nxd2 Rxh2 36.Qg5+ Kd7 37.Ne4 Bxf2) +4.67/20 2} 34. Qg5+
{(34.Qg5+) -3.28/12 1} Kd7 {(34. ... Kd7 35.Nc4 Kc6 36.h4 a2 37.Ra1 Rb2
38.Nxb2) +5.73/19 2} 35. Ne4 {(35.Ne4) -3.49/10 1} Be7 {(35. ... Be7 36.Qg7
Rb2 37.Nf6+ Bxf6 38.exf6 a2 39.Ra1 Rxf2 40.Ng5) +5.72/19 2} 36. Nf6+
{(36.Nf6+) -3.18/10 1} Bxf6 {(36. ... Bxf6 37.exf6 Rb2 38.Qg7 a2 39.Ra1 c2
40.Rxa2 Rxa2) +6.45/19 2} 37. exf6 {(Pe5xBf6) -3.18/10 1} Rb2 {(37. ... Rb2
38.Ne5+ Kc7 39.Qg7 Rf8 40.Nc4 a2 41.Nxb2 cxb2 42.Qh7) +6.11/20 2} 38. Qg4
{(38.Qg4) -1.98/10 1} a2 {(38. ... a2 39.Qg7 Rh5 40.Ra1 Rxh2 41.Nxh2)
+8.48/19 2} 39. Qa4+ {(39.Qa4+) -5.12/10 1} Kd6 {(39. ... Kd6 40.Qa3+ Kc6
41.h4 Rxf2 42.Qxa2 Rxa2 43.Ne5+) +5.78/19 2} 40. Qxe8 {(40.Qxe8) -5.76/10
1} Rxe8 {(40. ... Rxe8 41.h4 Reb8 42.h5 a1Q 43.Rxa1 c2 44.Rc1) +9.68/18 2}
41. Ra1 {(41.Ra1) -5.91/10 1} Reb8 {(41. ... Reb8 42.h4 R8b3 43.Nd4 c2+
44.Nxb3 c1Q) +8.49/18 2} 42. Rgg1 {(42.Rgg1) -5.10/10 1} Rxf2 {(42. ...
Rxf2 43.Ng5 Rbb2 44.Ne4+ Ke5 45.Nxf2 Rxf2 46.Raf1) +8.30/18 2} 43. Ng5
{(43.Ng5) -5.10/12 1} Rd2+ {(43. ... Rd2+ 44.Ke4 Nxf6+ 45.Ke3 Rdb2 46.Kd3
Nd5 47.Nxf7+) +7.80/18 2} 44. Ke4 {(44.Ke4) -3.93/4 0} Nxf6+ {(44. ...
Nxf6+ 45.Ke3 Rdb2 46.Kd3 c2 47.Nxf7+ Kd7) +8.00/18 1} 45. Kf3 {(45.Kf3)
-6.13/10 1} Rdb2 {(45. ... Rdb2 46.Rgc1 Nd5 47.Nxf7+ Ke7 48.h4 Kxf7)
+8.52/18 1} 46. Nxf7+ {(Ng5xPf7) -3.52/12 1} Ke7 {(46. ... Ke7 47.Rgc1 Kxf7
48.Rxc3 R8b6 49.h3 Nd5 50.Rcc1) +8.64/18 1} 47. Ne5 {(47.Ne5) -4.01/10 1}
R8b3 {(47. ... R8b3 48.Nc4 Nd5 49.Ke4 c2 50.Ke5 Rc3) +8.06/18 1} 48. Rgc1
{(48.Rgc1) -5.91/10 1} Rb1 {(48. ... Rb1 49.Nc4 Rxc1 50.Rxc1 Rb2 51.Rxc3
a1Q 52.Nxb2 Qxb2 53.Rd3 Qc2) +8.07/17 1} 49. Ng6+ {(49.Ng6+) -6.21/12 1}
Kd6 {(49. ... Kd6 50.Rxa2 Rxc1 51.Ra8 c2+ 52.Ke2 Ra1 53.Rxa1) +9.43/17 1}
50. Rcxb1 {(50.Raxb1) -4.53/12 1} axb1=Q {(50. ... axb1Q 51.Rxb1 Rxb1
52.Ke2 c2 53.Kd2 c1Q+ 54.Kd3 Ra1 55.Kd4 Ra3) +12.01/17 1} 51. Rxb1
{(51.Rxb1) -14.70/10 1} Rxb1 {(51. ... Rxb1 52.Ke2 c2 53.Kd2 c1Q+ 54.Kd3
Ra1 55.h3 Nh5 56.h4) +12.30/18 1} 52. Nf4 {(52.Nf4) -10.19/10 1} c2 {(52.
... c2 53.h4 c1Q 54.h5 Nxh5 55.Nxh5 Ra1 56.Ke4 Ra3) +13.97/17 1} 53. Nd3
{(53.Nd3) -13.02/12 1} Ra1 {(53. ... Ra1 54.h4 c1Q 55.Nxc1 Rxc1 56.h5 Nxh5
57.Ke4) +14.26/17 1} 54. Ke2 {(54.Ke2) -9.42/12 1} Ne4 {(54. ... Ne4 55.h4
c1Q 56.Nxc1 Rxc1 57.h5 Rh1) +13.00/17 1} 55. h4 {(Ph2-h4) -9.44/12 1} Nc5
{(55. ... Nc5 56.Nxc5 Kxc5 57.Kd2 c1Q+ 58.Kd3 Ra3+ 59.Ke2) +12.55/16 1} 56.
Kd2 {(56.Kd2) -9.33/12 1} Nxd3 {(56. ... Nxd3 57.Kxc2 Rh1 58.Kxd3 Rxh4
59.Ke3 e5 60.Kf3 Kd5) +11.73/16 1} 57. Kxc2 {(56. ... Rxd2) -9.44/14 1} Rh1
{(57. ... Rh1 58.Kxd3 Rxh4 59.Ke3 e5 60.Kf3 Kd5 61.Kg3) +11.30/17 1} 58.
Kxd3 {(58.Kxd3) -6.59/12 1} Rxh4 {(58. ... Rxh4 59.Ke3 e5 60.Kf3 Kd5 61.Ke3
e4) +12.48/16 1} 59. Ke3 {(59.Ke3) -6.59/12 1} Kd5 {(59. ... Kd5 60.Kf3 e5
61.Ke3 e4 62.Kf2 Kd4) +12.93/17 1} 60. Kd3 {(60.Kd3) -6.64/12 1} e5 {(60.
... e5 61.Ke3 e4 62.Ke2 Kd4 63.Kf2 Kd3) +13.86/16 1} 61. Ke3 {(61.Ke3)
-6.68/12 1} Rh3+ {(61. ... Rh3+ 62.Ke2 Ke4 63.Kf2 Rh2+ 64.Kg3) +13.58/16 1}
62. Kf2 {(62.Kf2) -6.79/10 1} e4 {(62. ... e4 63.Ke2 Kd4 64.Kf2 Kd3 65.Kg2)
+13.89/16 1} 63. Ke2 {(63.Ke2) -6.88/12 1} Kd4 {(63. ... Kd4 64.Kf2 Kd3
65.Kg2 e3 66.Kxh3 e2) +14.05/16 1} 64. Kf2 {(64.Kf2) -14.82/12 1} e3+ {(64.
... e3+ 65.Ke2 Ke4 66.Kf1 Kf3 67.Ke1 Rh2) +14.83/15 1} 65. Kf1 {(65.Kf1)
-14.90/12 1} Rh2 {(65. ... Rh2 66.Kg1 Rb2 67.Kf1 Rc2 68.Ke1) +14.34/15 1}
66. Ke1 {(66.Ke1) -M381/10 0} Kd3 {(66. ... Kd3 67.Kf1 Rh1+ 68.Kg2 e2
69.Kf2 e1Q+ 70.Kg2) +15.16/16 1} 67. Kf1 {(67.Kf1) -M381/10 0} Ra2 {(67.
... Ra2 68.Kg1 e2 69.Kf2 Kd2 70.Kf3 e1Q 71.Kf4 Rb2 72.Kf3) +15.79/16 1} 68.
Kg1 {(68.Kg1) -14.83/4 0} Rf2 {(68. ... Rf2 69.Kh1 e2 70.Kg1 e1Q+)
+24.05/16 1} 69. Kh1 {(69.Kh1) -7.26/2 0} e2 {(69. ... e2 70.Kg1 e1Q+)
+35.55/16 1} 70. Kg1 {(70.Kg1) -15.22/2} e1=Q# {(70. ... e1Q+) +44.92/16 1}
0-1[/pgn]

Mind you, this game was played at 1 min + 1 second TC, with longer time controls the strenght goes up DRAMATICALLY. At real 40/18 it's on par with Stockfish 8 level 3 right now. Last weekend it couldn't even keep up with level 1.