LCZero update

[Nay Lin Tun]

My games vs Leela zero.

Got 1-1 in 5 mins blitz. Here is the second game I won, I kept pushing without knowing that leela resigned already. lol ( I lost my first game by dropping pieces). My 5 mins blitz rating is 2000+ in chess.com.
https://www.chess.com/member/drmrboss

My guess is that Leela is around 1600-1800 in blitz now.

[pgn][Event "Computer chess game"]
[Site "DESKTOP-U9VVTOA"]
[Date "2018.03.22"]
[Round "?"]
[White "Nay Lin Tun"]
[Black "Play"]
[Result "1/2-1/2"]
[BlackElo "2000"]
[ECO "A07"]
[Opening "Reti Opening"]
[Time "00:13:07"]
[Variation "KIA, 2.g3"]
[WhiteElo "2400"]
[TimeControl "300"]
[Termination "normal"]
[PlyCount "129"]
[WhiteType "human"]
[BlackType "program"]

1. Nf3 d5 {(d7-d5 h2-h3 Ng8-f6 a2-a4 a7-a5 Nb1-c3) +0.16/21 6} 2. g3 e5
{(e7-e5 Nf3xe5 f7-f6 Ne5-f3 Nb8-c6 a2-a4 Ng8-h6 Nb1-c3) +0.30/22 6} 3. Nxe5
f6 {(f7-f6 Ne5-f3 Nb8-c6 a2-a4 Ng8-h6 Nb1-c3 d5-d4 Nc3-b5) +0.17/22 6} 4.
Nf3 Nc6 {(Nb8-c6 a2-a4 Ng8-h6 Nb1-c3 d5-d4 Nc3-b5 a7-a6 Nb5xd4) 0.00/22 6}
5. d4 Bd7 {(Bc8-d7 c2-c4 d5xc4 d4-d5 Nc6-b4 a2-a3 Nb4xd5 e2-e4 Nd5-b6)
-0.01/22 5} 6. Bg2 Qc8 {(Qd8-c8 Nb1-c3 Bd7-e6 Nf3-h4 g7-g5 Nh4-f3 g5-g4
Nf3-h4 Nc6xd4) +0.62/22 5} 7. O-O Nge7 {(Ng8-e7 c2-c4 d5xc4 Nb1-c3 b7-b5
d4-d5 a7-a6 d5xc6) +0.54/22 5} 8. Nbd2 Be6 {(Bd7-e6 c2-c4 d5xc4 e2-e4 b7-b5
d4-d5 Ne7xd5 e4xd5 Be6xd5 a2-a4 b5-b4 Nd2xc4) +0.33/21 5} 9. e4 dxe4
{(d5xe4 Nd2xe4 Be6-g4 d4-d5 Ne7xd5 Qd1xd5 Bg4-e6 Qd5-d1 Be6-g4 h2-h3)
-0.23/22 5} 10. Nxe4 Bc4 {(Be6-c4 Nf3-h4 Bc4xf1 Kg1xf1 Ne7-g6 d4-d5 Nc6-e5
Nh4xg6 h7xg6 f2-f4 Ne5-g4 h2-h3) -0.68/22 5} 11. Re1 Bd5 {(Bc4-d5 Nf3-h4
Bd5-f7 Ne4-c5 g7-g5 Nc5-e6 Bf7xe6) -0.99/22 5} 12. b3 Bxe4 {(Bd5xe4 Re1xe4
Ra8-b8 Nf3-h4 g7-g5 Qd1-h5+ Ke8-d7) -0.65/22 5} 13. Rxe4 Nb4 {(Nc6-b4
Bc1-a3 Nb4-d5 Nf3-h4 Nd5-c3 Re4xe7+ Bf8xe7 Ba3xe7 Nc3xd1) -1.91/22 5} 14.
Ba3 Nd5 {(Nb4-d5 Qd1-e1 a7-a5 c2-c4 Nd5-b4 Ba3xb4 a5xb4 Nf3-h4 g7-g5 Nh4-f5
Qc8xf5) -2.38/22 5} 15. c4 Nc3 {(Nd5-c3 Qd1-d3 Nc3xe4 Qd3xe4 a7-a6 d4-d5
c7-c6 d5-d6 c6-c5 d6xe7 Bf8xe7 Ra1-e1 Ke8-f7 Qe4xe7+ Kf7-g6) -1.95/22 5}
16. Rxe7+ Bxe7 {(Bf8xe7 Qd1-e1 c7-c5 d4xc5 Ke8-f7 b3-b4 a7-a5 Qe1xc3 a5xb4
Ba3xb4 Be7xc5 Bb4xc5 Qc8xc5 Ra1-d1 Rh8-e8 Nf3-d4) -3.85/22 5} 17. Qe1 c5
{(c7-c5 d4xc5 Ke8-f7 b3-b4 a7-a5 Nf3-d4 a5xb4 Ba3xb4 Nc3xa2 Ra1xa2 Ra8xa2
Bg2-d5+ Kf7-g6 Qe1xe7) -3.96/23 5} 18. Bxc5 Qd7 {(Qc8-d7 Qe1xc3 Be7xc5
d4xc5 Qd7-c6 Nf3-d4 Qc6xc5 b3-b4 Qc5-b6 c4-c5 Qb6-c7 Nd4-e6 Qc7-c6)
-4.16/22 5} 19. Bxe7 Qxe7 {(Qd7xe7 Qe1xc3 Qe7-c7 d4-d5 a7-a6 Nf3-d4 Ke8-f7
a2-a4 Rh8-d8 Nd4-e6) -4.28/22 5} 20. Qxc3 Qc7 {(Qe7-c7 d4-d5 Ke8-f7 a2-a4
Ra8-c8 Nf3-d4 a7-a5 Nd4-e6 Qc7-d7) -4.36/23 5} 21. Re1+ Kf7 {(Ke8-f7 d4-d5
Ra8-d8 Nf3-d4 Qc7-b6 Nd4-e6 Rd8-c8 c4-c5 Qb6-a6 c5-c6 b7xc6) -4.44/22 5}
22. d5 Qd8 {(Qc7-d8 Re1-e6 Ra8-c8 Nf3-d4 Rh8-e8 Re6xe8 Qd8xe8 a2-a4 Qe8-d8
Nd4-e6) -4.73/23 5} 23. Nd4 a5 {(a7-a5 Re1-e6 Rh8-e8 h2-h4 Re8xe6 d5xe6+
Kf7-f8 Bg2xb7 Ra8-b8) -5.77/22 5} 24. Ne6 Qd7 {(Qd8-d7 Qc3-f3 Rh8-c8 c4-c5
Qd7-e8 Qf3-h5+ g7-g6 Qh5xh7+) -5.95/22 5} 25. Bh3 Qe8 {(Qd7-e8 Ne6-c7
Qe8-d8 Nc7xa8 h7-h6 Bh3-e6+ Kf7-g6 Na8-b6 Qd8xb6 c4-c5) -5.69/22 5} 26.
Ng5+ fxg5 {(f6xg5 Re1xe8 Rh8xe8 Qc3-f3+ Kf7-g6 Bh3-g4 Re8-e5 Bg4-e6 Ra8-e8
Qf3-f7+ Kg6-h6 Qf7xe8) -8.56/21 5} 27. Rxe8 Rhxe8 {(Rh8xe8 Qc3-f3+ Kf7-g6
Bh3-f5+ Kg6-f6 Bf5-d7+ Kf6-e7 Bd7xe8 Ra8xe8 Qf3-h5 g7-g6 Qh5xh7+ Ke7-d6
Qh7xg6+) -8.93/22 5} 28. Be6+ Kg6 {(Kf7-g6 Qc3-d3+ Kg6-h6 Kg1-g2 g7-g6
Qd3-d4 Re8-e7 c4-c5 Re7-e8 d5-d6) -8.77/21 5} 29. Qd3+ Kh6 {(Kg6-h6 Kg1-g2
Ra8-d8 Qd3-c2 g7-g6 Qc2-c3 Re8-e7 Qc3xa5 Rd8-b8 d5-d6 Re7xe6) -8.99/22 5}
30. Qf5 Rf8 {(Re8-f8 Qf5-e4 Rf8-f6 c4-c5 g7-g6 d5-d6 Ra8-d8 Qe4xb7 Rf6xe6
c5-c6 Re6xd6) -8.47/21 5} 31. Qh3+ Kg6 {(Kh6-g6 f2-f4 h7-h6 f4-f5+ Kg6-f6
Qh3-h5 Ra8-e8 Qh5-g6+ Kf6-e5 Qg6xg7+ Ke5-d6 Qg7xh6 Rf8-h8 Qh6xh8) -7.74/21
5} 32. g4 h6 {(h7-h6 Qh3-c3 b7-b6 Be6-f5+ Kg6-f7 d5-d6 Rf8-d8 Qc3-e5 Rd8-f8
d6-d7) -6.98/21 5} 33. Qd3+ Kf6 {(Kg6-f6 Qd3-d4+ Kf6-g6 Qd4-e4+ Kg6-f6
Be6-d7 g7-g6 Qe4-e6+ Kf6-g7 c4-c5 Ra8-d8 c5-c6 b7xc6 Bd7xc6 Rf8-f6)
-7.86/21 4} 34. Qf5+ Ke7 {(Kf6-e7 Qf5-g6 Ke7-d6 Qg6xg7 Rf8-h8 Qg7-d7+
Kd6-c5 Qd7xb7 Ra8-b8 Qb7xb8 Rh8xb8 f2-f4 g5xf4) -8.76/21 4} 35. Qg6 Kd6
{(Ke7-d6 Qg6xg7 b7-b6 Kg1-f1 Rf8-h8 Qg7-d7+ Kd6-c5 Qd7-f7 Rh8-f8 Qf7xf8+
Ra8xf8 Kf1-e1) -9.10/22 4} 36. Qxg7 b6 {(b7-b6 Kg1-f1 Rf8-h8 Qg7-d7+ Kd6-e5
Qd7-f7 Rh8-f8 Qf7xf8 Ra8xf8 Kf1-e1 Ke5-d6) -9.18/22 4} 37. Qxh6 Kc7
{(Kd6-c7 Qh6xg5 Kc7-d6 Qg5-h6 Rf8-h8 Qh6xh8 Ra8xh8 g4-g5 Kd6-e7) -9.34/21
4} 38. Qxg5 Kd6 {(Kc7-d6 Kg1-g2 Kd6-c5 Qg5-h6 Rf8-e8 g4-g5 Re8-e7)
-10.23/21 4} 39. h4 Rh8 {(Rf8-h8 Kg1-g2 Ra8-f8 f2-f3 Rf8-e8 Qg5-g6 Re8-e7
h4-h5) -10.26/21 4 Black resigns} 40. h5 Kc7 {(Kd6-c7 Qg5-g6 Kc7-d6 Qg6-g7
Rh8-e8 h5-h6 Re8-g8 Qg7xg8) -10.89/21 3 Black resigns} 41. Qg7+ Kd6
{(Kc7-d6 Kg1-g2 Ra8-b8 h5-h6 Rb8-e8 g4-g5 Rh8-g8 Qg7xg8) -11.99/21 3 Black
resigns} 42. h6 Kc5 {(Kd6-c5 h6-h7 Kc5-b4 d5-d6 a5-a4 b3xa4 Ra8xa4)
-12.13/21 3 Black resigns} 43. h7 Kb4 {(Kc5-b4 f2-f4 Kb4-a3 f4-f5 Ka3xa2
d5-d6 Ka2xb3 Qg7xh8 Ra8xh8 d6-d7) -12.61/20 3 Black resigns} 44. g5 Ka3
{(Kb4-a3 d5-d6 Ka3xa2 d6-d7 Ra8-d8 Qg7xh8 Rd8xh8 d7-d8Q Rh8xh7 g5-g6)
-11.92/20 3 Black resigns} 45. g6 Kxa2 {(Ka3xa2 d5-d6 Ka2xb3 Be6-d5 Ra8-d8
Qg7-f7 Kb3-b4 g6-g7) -12.40/20 3 Black resigns} 46. Qf6 Kxb3 {(Ka2xb3 d5-d6
a5-a4 c4-c5+ Kb3-c2 c5xb6 a4-a3 b6-b7 Ra8-b8 g6-g7 Rh8xh7) -11.77/20 2
Black resigns} 47. Bg8 Raxg8 {(Ra8xg8 h7xg8N Rh8xg8 Qf6-f7 Rg8xg6+ Qf7xg6
Kb3xc4 d5-d6 b6-b5 d6-d7 b5-b4 d7-d8Q b4-b3 Qd8xa5) -11.07/20 2 Black
resigns} 48. hxg8=Q Rxg8 {(Rh8xg8 Qf6-f7 Rg8-d8 g6-g7 a5-a4 g7-g8Q Rd8xg8+
Qf7xg8 a4-a3 d5-d6 a3-a2 d6-d7 a2-a1Q+ Kg1-g2) -11.27/20 2 Black resigns}
49. g7 a4 {(a5-a4 Qf6-f7 Rg8-d8 Qf7-b7 Rd8-g8 d5-d6 Kb3xc4 d6-d7 Rg8xg7+
Kg1-h2 Rg7xd7 Qb7xd7 a4-a3) -10.89/20 2 Black resigns} 50. d6 a3 {(a4-a3
Qf6-f7 a3-a2 Qf7xg8 a2-a1Q+ Kg1-g2 Qa1-f6 Qg8-h7 Qf6xd6 g7-g8Q Qd6-c6+
Kg2-g3 b6-b5 c4xb5+) -10.46/20 2 Black resigns} 51. d7 a2 {(a3-a2 Kg1-h2
Rg8-b8 Qf6-a1 Rb8-d8 Kh2-g3 Rd8xd7 g7-g8Q Rd7-d2 c4-c5+ Kb3-c2 c5xb6
Kc2-d3) -10.44/19 2 Black resigns} 52. d8=Q Rxd8 {(Rg8xd8 Qf6xb6+ Kb3-c2
Qb6xd8 a2-a1Q+ Kg1-h2 Qa1xg7 c4-c5 Qg7-c3 Qd8-e7 Qc3-d3 c5-c6 Qd3-d2 c6-c7)
-5.11/19 2} 53. Qxd8 a1=Q+ {(a2-a1Q+ Kg1-h2 Qa1xg7 Qd8xb6+ Kb3xc4 Qb6-a6+
Kc4-b3 Qa6-b5+ Kb3-c3 Qb5-a6 Qg7-d4 Kh2-g3 Qd4-d2) -2.00/19 2} 54. Kh2 Qxg7
{(Qa1xg7 Qd8xb6+ Kb3xc4 Qb6-e6+ Kc4-d3 Qe6-a6+ Kd3-d2 Kh2-h3 Kd2-c3 Kh3-h2
Qg7-d4) -1.86/21 2} 55. Qh4 Qg6 {(Qg7-g6 Qh4-d4 Kb3-c2 c4-c5 b6xc5 Qd4xc5+
Kc2-d3 Qc5-d5+ Kd3-c3 Qd5-e5+) +0.54/19 1} 56. Qg3+ Qxg3+ {(Qg6xg3+ f2xg3
Kb3xc4 g3-g4 Kc4-d4 Kh2-g3 Kd4-e4 Kg3-h4 b6-b5 g4-g5 Ke4-f5 Kh4-h5 b5-b4
g5-g6 b4-b3 g6-g7 b3-b2 g7-g8Q b2-b1Q Qg8-f7+ Kf5-e5) +1.88/18 1} 57. fxg3
Kxc4 {(Kb3xc4 g3-g4 Kc4-d4 Kh2-g3 Kd4-e4 Kg3-h4 b6-b5 g4-g5 Ke4-f5 Kh4-h5
b5-b4 Kh5-h6 b4-b3 g5-g6 b3-b2 g6-g7 b2-b1Q g7-g8Q Qb1-b7 Qg8-f8+ Kf5-e6
Kh6-g6 Qb7-d7) +1.88/19 1} 58. g4 Kd5 {(Kc4-d5 Kh2-g3 b6-b5 g4-g5 Kd5-e5
g5-g6 Ke5-f6 Kg3-f4 Kf6xg6 Kf4-e5 b5-b4 Ke5-d4 b4-b3 Kd4-c3) +2.08/20 1}
59. Kg3 b5 {(b6-b5 Kg3-f4 b5-b4 Kf4-e3 Kd5-e5 Ke3-d3 Ke5-f4 Kd3-c4 Kf4xg4
Kc4xb4) +1.27/20 1} 60. Kf4 b4 {(b5-b4 Kf4-e3 Kd5-e5 Ke3-d3 Ke5-f4 Kd3-c4
Kf4xg4 Kc4xb4) +0.15/19 1} 61. Ke3 Ke5 {(Kd5-e5 Ke3-d3 Ke5-f4 Kd3-c4 Kf4xg4
Kc4xb4) 0.00/19 1} 62. Kd2 Kf4 {(Ke5-f4 Kd2-d3 Kf4xg4 Kd3-c4 Kg4-f4 Kc4xb4)
+0.14/19 1} 63. Kc2 Kxg4 {(Kf4xg4 Kc2-b3 Kg4-f4 Kb3xb4) +0.06/19 1} 64. Kb3
Kf4 {(Kg4-f4 Kb3xb4) +0.04/18 1} 65. Kxb4 {Insufficient material} 1/2-1/2
[/pgn][/url]

At move 15. Leela did not expect the tactical shot 16. Rxe7! I did not see that tactical shot before I did 14.c4., I was panic for a few seconds but i saw it.

[noobpwnftw]

Is it correct to conclude that a 6-block network will reach its ceiling quite soon since there is already a decreased rate of progression in ELO and more chances that a new generation would perform worse(or in the error bars, whichever sounds better)?

Then can we move on to discover the relationship among number of blocks, number of games needed to train them and their relative strength when they reach certain progression rates?

After that we can then decide what number of blocks is more "efficient" to get a reasonable playing strength, or even if there is a cap there that with more blocks and its corresponding number of games, it wouldn't do any better.

[David Xu]

Judging by the progress of Leela Zero, it's unlikely that we've hit the ceiling of 6x64 yet. I wouldn't worry about network saturation until several generations in a row fail to pass.

[CMCanavessi]

Is it correct to conclude that a 6-block network will reach its ceiling quite soon since there is already a decreased rate of progression in ELO and more chances that a new generation would perform worse(or in the error bars, whichever sounds better)?

Then can we move on to discover the relationship among number of blocks, number of games needed to train them and their relative strength when they reach certain progression rates?

After that we can then decide what number of blocks is more "efficient" to get a reasonable playing strength, or even if there is a cap there that with more blocks and its corresponding number of games, it wouldn't do any better.

Decreased rate?

Code: Select all

186 Leela Chess Zero Gen 12 x64            :  1097.6     246   61   25  160    30    10  1298.7    48    39.8
198 Leela Chess Zero Gen 10 x64            :   862.1      92   53   11   28    64    12   656.1    23    23.0
201 Leela Chess Zero Gen 8 x64             :   793.3      92   45   17   30    58    18   656.1    23    23.0
206 Leela Chess Zero Gen 6 x64             :   598.5      92   31   18   43    43    20   656.1    23    23.0
210 Leela Chess Zero Gen 4 x64             :   369.6     150   43   18   89    35    12   623.6    15    15.0

[jkiliani]

I think that the ceiling of the current 64x6 net (64 filters, 6 residual blocks) is still quite far away. I would expect at least human master play before progress stalls. At this point, a bootstrap to a larger network would be in order. Judging by Leela Zero's progress, a good choice for the next network architecture would be 128 filters, 10 residual blocks.

[Uri Blass]

At least one advantage of this approach is that you can create human-like weak levels, which is not really possible with alpha beta. Snapshots of the NN weights along the way would give you that.

Although you may end up with unhuman play in the opposite direction: reasonably good positional play but horrendous tactics. Basically too human like to be credible.

The thing with humans is that they play with an erratic strength, you may have some 1200 rating player that plays like a 2900 elo entity 30% of its moves, 2500 20% of its moves, 1800 40% of its moves and ???? 10% of its moves. This unknown rating can make this player give up their queen or miss a mate in 1, or basically just be completely random now and then, and it's a question of time when the blunder will be played and would allow it to lose a game against anybody else, making the strength of the rest of the moves irrelevant.

This explains why the weaker the players the more erractic the strength, and you see more often huge swings of evaluation, while they get rarer the stronger the players (but even GMs will have some chance of making those ???? elo moves that you'd not see an engine with the same elo to make.)

So I believe the best shot at emulating weak human play by NNs is to mix them up and use different ones for each move. Adversarial Networks could work for this, as you have a NN trying to emulate human play by switching the elo per move as humans do, while another NN is feed those games and gets better at identifying the ones player by real humans.

Eventually you have a NN that can tell if a side was played by a human most of the time, and another that can produce moves that can trick the former. My claim is this latter one would be able to fool most humans into thinking it's a human playing, since the other NN would already be better at identifying computer play than humans.

I do not know what do you mean
"plays like a 2900 elo entity 30% of its moves" and I think that even weak chess programs with rating 1200 have no problem to do it(assuming we talk about something like stockfish at depth 2 or 3) because
there are a lot of cases when searching to depth of 1 or 2 plies lead to the same move as searching to depth of 30 plies.

The way to identify weak humans is not by the fact that they play some strong moves but by the special blunders that they do that are not random blunders.

I believe that people can easily detect LCZero at level of 1200 as a a computer if they get enough games because it does not blunder in a way that is typical to weak humans at level of 1200.

[noobpwnftw]

Decreased rate?

Code: Select all

186 Leela Chess Zero Gen 12 x64            :  1097.6     246   61   25  160    30    10  1298.7    48    39.8
198 Leela Chess Zero Gen 10 x64            :   862.1      92   53   11   28    64    12   656.1    23    23.0
201 Leela Chess Zero Gen 8 x64             :   793.3      92   45   17   30    58    18   656.1    23    23.0
206 Leela Chess Zero Gen 6 x64             :   598.5      92   31   18   43    43    20   656.1    23    23.0
210 Leela Chess Zero Gen 4 x64             :   369.6     150   43   18   89    35    12   623.6    15    15.0

Care to provide a title to your figures?

According to the bible every learning curve had a decreased rate after the initial breakthrough, so it is either they faked it or you are doing it wrong then.

Although it is rather a brief match against previous generation here http://lczero.org/matches, can you calculate the ratio of W/L/D and tell me how it is not improving at a a slower rate from the number of games between generations vs their corresponding stats?
Is it not obvious to you seeing higher draw rates in recent generations than it was before and since it is a small network anyway is there any good reason not to try a bigger one but struggling with the remaining few spaces it may have?

EDIT:
What is more important here anyway is to get to know how to do reinforcement learning in terms of generally acceptable resource vs relative outcome as a verifiable reference, or just to build a chess engine that is strong, according to the project brief. Either way it is not likely to work with 6-block networks, you'd probably get a 1800 rating if not lower.

Of course you can get higher ratings if other parts like search are improved, but it is not related to how you define the network, the bible didn't say too much about failed experiments but those are exactly what we should do, I think.

[CheckersGuy]

Decreased rate?

Code: Select all

186 Leela Chess Zero Gen 12 x64            :  1097.6     246   61   25  160    30    10  1298.7    48    39.8
198 Leela Chess Zero Gen 10 x64            :   862.1      92   53   11   28    64    12   656.1    23    23.0
201 Leela Chess Zero Gen 8 x64             :   793.3      92   45   17   30    58    18   656.1    23    23.0
206 Leela Chess Zero Gen 6 x64             :   598.5      92   31   18   43    43    20   656.1    23    23.0
210 Leela Chess Zero Gen 4 x64             :   369.6     150   43   18   89    35    12   623.6    15    15.0

Care to provide a title to your figures?

According to the bible every learning curve had a decreased rate after the initial breakthrough, so it is either they faked it or you are doing it wrong then.

Although it is rather a brief match against previous generation here http://lczero.org/matches, can you calculate the ratio of W/L/D and tell me how it is not improving at a a slower rate from the number of games between generations vs their corresponding stats?
Is it not obvious to you seeing higher draw rates in recent generations than it was before and since it is a small network anyway is there any good reason not to try a bigger one but struggling with the remaining few spaces it may have?

We dont need to try a bigger one yet since ones the training process becomes to slow (networks dont pass anymore) we can use the net2net approach and switch to 10 blocks and continue from there. They did so with Go (LeelaZero) and with great sucess

[noobpwnftw]

We dont need to try a bigger one yet since ones the training process becomes to slow (networks dont pass anymore) we can use the net2net approach and switch to 10 blocks and continue from there. They did so with Go (LeelaZero) and with great sucess

The point is that what if we start from scratch with a bigger network, does it require less or more resources to achieve the same level of strength? Do we know any relationships in between? At what size it is more suitable for chess in particular? This is something can be observed by briefly training the networks past their initial breakthrough stages, the way I see it is what we already have with 6-blocks.

Can you enlighten me if there are any attempts on this has been made before? Thanks.

[Robert Pope]

According to the bible every learning curve had a decreased rate after the initial breakthrough, so it is either they faked it or you are doing it wrong then.

If you look at the elo graphs in the Alpha Zero paper (figure 1), the improvement they saw was linear for quite a long while. That is true for chess, go, and Shogi, though their chess progression did have a period of slower progression before returning to the faster improvement rate.