Historic Milestone: AlphaZero

[smatovic]

Go Matthew, go Matthew

--
Srdja

[Milos]

So basically they played 400 cores engine vs 60 cores engine. Hmm..

From the paper:

Code: Select all

AlphaZero and the previous AlphaGo Zero used a single machine with 4 TPUs

I guess it depends on how you count the TPUs

Since no one of us has access to TPU it's only fair to count in terms of what is available (for example if we had Alpha0 x64 binary compile and wanted to run it at home).
1TPU ~ 30xE5-2699v3 (18 cores machine).
4TPUs ~ 2000 Haswell cores

[Vinvin]

the 10 games in PGN format : https://www.dropbox.com/s/an9gkh38jucuk ... F.pgn?dl=0

[pgn]
[Event "AlphaZero-SF"]
[Site "Google"]
[White "Stockfish8"]
[Black "AlphaZero"]
[Result "0-1"]
1. e4 e5 2. Nf3 Nc6 3. Bb5 Nf6 4. d3 Bc5 5. Bxc6 dxc6 6. 0-0 Nd7 7. Nbd2 0-0 8. Qe1 f6 9. Nc4 Rf7
10. a4 Bf8 11. Kh1 Nc5 12. a5 Ne6 13. Ncxe5 fxe5 14. Nxe5 Rf6 15. Ng4 Rf7 16. Ne5 Re7 17. a6 c5
18. f4 Qe8 19. axb7 Bxb7 20. Qa5 Nd4 21. Qc3 Re6 22. Be3 Rb6 23. Nc4 Rb4 24. b3 a5 25. Rxa5
Rxa5 26. Nxa5 Ba6 27. Bxd4 Rxd4 28. Nc4 Rd8 29. g3 h6 30. Qa5 Bc8 31. Qxc7 Bh3 32. Rg1 Rd7
33. Qe5 Qxe5 34. Nxe5 Ra7 35. Nc4 g5 36. Rc1 Bg7 37. Ne5 Ra8 38. Nf3 Bb2 39. Rb1 Bc3 40. Ng1
Bd7 41. Ne2 Bd2 42. Rd1 Be3 43. Kg2 Bg4 44. Re1 Bd2 45. Rf1 Ra2 46. h3 Bxe2 47. Rf2 Bxf4 48.
Rxe2 Be5 49. Rf2 Kg7 50. g4 Bd4 51. Re2 Kf6 52. e5+ Bxe5 53. Kf3 Ra1 54. Rf2 Re1 55. Kg2+ Bf4
56. c3 Rc1 57. d4 Rxc3 58. dxc5 Rxc5 59. b4 Rc3 60. h4 Ke5 61. hxg5 hxg5 62. Re2+ Kf6 63. Kf2
Be5 64. Ra2 Rc4 65. Ra6+ Ke7 66. Ra5 Ke6 67. Ra6+ Bd6 0-1

[Event "AlphaZero-SF"]
[Site "Google"]
[White "Stockfish8"]
[Black "AlphaZero"]
[Result "0-1"]
1. e4 e5 2. Nf3 Nc6 3. Bb5 Nf6 4. d3 Bc5 5. Bxc6 dxc6 6. 0-0 Nd7 7. c3 0-0 8. d4 Bd6 9. Bg5 Qe8 10.
Re1 f6 11. Bh4 Qf7 12. Nbd2 a5 13. Bg3 Re8 14. Qc2 Nf8 15. c4 c5 16. d5 b6 17. Nh4 g6 18. Nhf3
Bd7 19. Rad1 Re7 20. h3 Qg7 21. Qc3 Rae8 22. a3 h6 23. Bh4 Rf7 24. Bg3 Rfe7 25. Bh4 Rf7 26. Bg3
a4 27. Kh1 Rfe7 28. Bh4 Rf7 29. Bg3 Rfe7 30. Bh4 g5 31. Bg3 Ng6 32. Nf1 Rf7 33. Ne3 Ne7 34. Qd3
h5 35. h4 Nc8 36. Re2 g4 37. Nd2 Qh7 38. Kg1 Bf8 39. Nb1 Nd6 40. Nc3 Bh6 41. Rf1 Ra8 42. Kh2
Kf8 43. Kg1 Qg6 44. f4 gxf3 45. Rxf3 Bxe3+ 46. Rfxe3 Ke7 47. Be1 Qh7 48. Rg3 Rg7 49. Rxg7+
Qxg7 50. Re3 Rg8 51. Rg3 Qh8 52. Nb1 Rxg3 53. Bxg3 Qh6 54. Nd2 Bg4 55. Kh2 Kd7 56. b3 axb3
57. Nxb3 Qg6 58. Nd2 Bd1 59. Nf3 Ba4 60. Nd2 Ke7 61. Bf2 Qg4 62. Qf3 Bd1 63. Qxg4 Bxg4 64. a4
Nb7 65. Nb1 Na5 66. Be3 Nxc4 67. Bc1 Bd7 68. Nc3 c6 69. Kg1 cxd5 70. exd5 Bf5 71. Kf2 Nd6 72.
Be3 Ne4+ 73. Nxe4 Bxe4 74. a5 bxa5 75. Bxc5+ Kd7 76. d6 Bf5 77. Ba3 Kc6 78. Ke1 Kd5 79. Kd2
Ke4 80. Bb2 Kf4 81. Bc1 Kg3 82. Ke2 a4 83. Kf1 Kxh4 84. Kf2 Kg4 85. Ba3 Bd7 86. Bc1 Kf5 87.
Ke3 Ke6 0-1

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. Nf3 Nf6 2. c4 b6 3. d4 e6 4. g3 Ba6 5. Qc2 c5 6. d5 exd5 7. cxd5 Bb7 8. Bg2 Nxd5 9. 0-0 Nc6 10.
Rd1 Be7 11. Qf5 Nf6 12. e4 g6 13. Qf4 0-0 14. e5 Nh5 15. Qg4 Re8 16. Nc3 Qb8 17. Nd5 Bf8 18.
Bf4 Qc8 19. h3 Ne7 20. Ne3 Bc6 21. Rd6 Ng7 22. Rf6 Qb7 23. Bh6 Nd5 24. Nxd5 Bxd5 25. Rd1 Ne6
26. Bxf8 Rxf8 27. Qh4 Bc6 28. Qh6 Rae8 29. Rd6 Bxf3 30. Bxf3 Qa6 31. h4 Qa5 32. Rd1 c4 33. Rd5
Qe1+ 34. Kg2 c3 35. bxc3 Qxc3 36. h5 Re7 37. Bd1 Qe1 38. Bb3 Rd8 39. Rf3 Qe4 40. Qd2 Qg4 41.
Bd1 Qe4 42. h6 Nc7 43. Rd6 Ne6 44. Bb3 Qxe5 45. Rd5 Qh8 46. Qb4 Nc5 47. Rxc5 bxc5 48. Qh4
Rde8 49. Rf6 Rf8 50. Qf4 a5 51. g4 d5 52. Bxd5 Rd7 53. Bc4 a4 54. g5 a3 55. Qf3 Rc7 56. Qxa3 Qxf6
57. gxf6 Rfc8 58. Qd3 Rf8 59. Qd6 Rfc8 60. a4 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. d4 e6 2. Nc3 Nf6 3. e4 d5 4. e5 Nfd7 5. f4 c5 6. Nf3 Nc6 7. Be3 Be7 8. Qd2 a6 9. Bd3 c4 10. Be2 b5
11. a3 Rb8 12. 0-0 0-0 13. f5 a5 14. fxe6 fxe6 15. Bd1 b4 16. axb4 axb4 17. Ne2 c3 18. bxc3 Nb6 19.
Qe1 Nc4 20. Bc1 bxc3 21. Qxc3 Qb6 22. Kh1 Nb2 23. Nf4 Nxd1 24. Rxd1 Bd7 25. h4 Ra8 26. Bd2
Rfb8 27. h5 Rxa1 28. Rxa1 Qb2 29. Qxb2 Rxb2 30. c3 Rb3 31. Ra8+ Rb8 32. Ra2 Rb3 33. g4 Ra3 34.
Rb2 Kf7 35. Kg2 Bc8 36. Rb6 Ra6 37. Rb1 Ke8 38. Kg3 h6 39. Ng6 Ra3 40. Rb6 Bd7 41. g5 hxg5 42.
Kg4 Bd8 43. Rb2 Bc8 44. Nxg5 Ra1 45. Nf3 Ra3 46. Be1 Ba5 47. Rf2 Ra1 48. Bd2 Bd8 49. Rh2 Ne7
50. Bg5 Nf5 51. Bxd8 Kxd8 52. Rb2 Rc1 53. Ngh4 Nxh4 54. Nxh4 Bd7 55. Rb8+ Bc8 56. Ng2 Rxc3
57. Nf4 Rc1 58. Ra8 Kd7 59. Kf3 Rc3+ 60. Kf2 Ke7 61. Kg2 Kf7 62. Ng6 Ke8 63. Ra1 Rc7 64. Kh3
Rf7 65. Kg4 Kd8 66. Nf4 Bd7 67. Ra7 Kc8 68. Kg3 Re7 69. Nd3 Kb8 70. Ra6 Bc8 71. Rb6+ Kc7 72.
Rd6 Kb8 73. Nc5 g6 74. h6 Rh7 75. Nxe6 Rxh6 76. Nf4 Rh1 77. Nxd5 Rh3+ 78. Kf4 Rh4+ 79. Ke3
Rh3+ 80. Kd2 Bf5 81. Ne7 Rh2+ 82. Ke3 Bh3 83. Nxg6 Rh1 84. Nf4 Bg4 85. Rf6 Kc7 86. Nd3 Bd7
87. d5 Bb5 88. Nf4 Ba4 89. Kd4 Be8 90. Rf8 Rd1+ 91. Kc5 Rc1+ 92. Kb4 Rb1+ 93. Kc3 Bb5 94. Kd4
Ba6 95. Rf7+ 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Bb7 5. Bg2 Be7 6. 0-0 0-0 7. d5 exd5 8. Nh4 c6 9. cxd5 Nxd5 10.
Nf5 Nc7 11. e4 Bf6 12. Nd6 Ba6 13. Re1 Ne8 14. e5 Nxd6 15. exf6 Qxf6 16. Nc3 Nb7 17. Ne4 Qg6
18. h4 h6 19. h5 Qh7 20. Qg4 Kh8 21. Bg5 f5 22. Qf4 Nc5 23. Be7 Nd3 24. Qd6 Nxe1 25. Rxe1 fxe4
26. Bxe4 Rf5 27. Bh4 Bc4 28. g4 Rd5 29. Bxd5 Bxd5 30. Re8+ Bg8 31. Bg3 c5 32. Qd5 d6 33. Qxa8
Nd7 34. Qe4 Nf6 35. Qxh7+ Kxh7 36. Re7 Nxg4 37. Rxa7 Nf6 38. Bxd6 Be6 39. Be5 Nd7 40. Bc3 g6
41. Bd2 gxh5 42. a3 Kg6 43. Bf4 Kf5 44. Bc7 h4 45. Ra8 h5 46. Rh8 Kg6 47. Rd8 Kf7 48. f3 Bf5 49.
Bh2 h3 50. Rh8 Kg6 51. Re8 Kf7 52. Re1 Be6 53. Bc7 b5 54. Kh2 Kf6 55. Re3 Ke7 56. Re4 Kf7 57.
Bd6 Kf6 58. Kg3 Kf7 59. Kf2 Bf5 60. Re1 Kg6 61. Kg1 c4 62. Kh2 h4 63. Be7 Nb6 64. Bxh4 Na4 65.
Re2 Nc5 66. Re5 Nb3 67. Rd5 Be6 68. Rd6 Kf5 69. Be1 Ke5 70. Rb6 Bd7 71. Kg3 Nc1 72. Rh6 Kd5
73. Bc3 Bf5 74. Rh5 Ke6 75. Kf2 Nd3+ 76. Kg1 Nf4 77. Rh6+ Ke7 78. Kh2 Nd5 79. Kg3 Be6 80. Rh5
Ke8 81. Re5 Kf7 82. Bd2 Ne7 83. Bb4 Nd5 84. Bc3 Ke7 85. Bd2 Kf6 86. f4 Ne7 87. Rxb5 Nf5+ 88.
Kh2 Ke7 89. Ra5 Nh4 90. Bb4+ Kf7 91. Rh5 Nf3+ 92. Kg3 Kg6 93. Rh8 Nd4 94. Bc3 Nf5+ 95. Kxh3
Bd7 96. Kh2 Kf7 97. Rb8 Ke6 98. Kg1 Bc6 99. Rb6 Kd5 100. Kf2 Bd7 101. Ke1 Ke4 102. Bd2 Kd5
103. Rf6 Nd6 104. Rh6 Nf5 105. Rh8 Ke4 106. Rh7 Bc8 107. Rc7 Ba6 108. Rc6 Bb5 109. Rc5 Bd7
110. Rxc4+ Kd5 111. Rc7 Kd6 112. Rc3 Ke6 113. Rc5 Nd4 114. Be3 Nf5 115. Bf2 Nd6 116. Rc3 Ne4
117. Rd3 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. d4 Nf6 2. Nf3 e6 3. c4 b6 4. g3 Be7 5. Bg2 Bb7 6. 0-0 0-0 7. d5 exd5 8. Nh4 c6 9. cxd5 Nxd5 10.
Nf5 Nc7 11. e4 Bf6 12. Nd6 Ba6 13. Re1 Ne8 14. e5 Nxd6 15. exf6 Qxf6 16. Nc3 Bc4 17. h4 h6 18.
b3 Qxc3 19. Bf4 Nb7 20. bxc4 Qf6 21. Be4 Na6 22. Be5 Qe6 23. Bd3 f6 24. Bd4 Qf7 25. Qg4 Rfd8
26. Re3 Nac5 27. Bg6 Qf8 28. Rd1 Rab8 29. Kg2 Ne6 30. Bc3 Nbc5 31. Rde1 Na4 32. Bd2 Kh8 33.
f4 Qd6 34. Bc1 Nd4 35. Re7 f5 36. Bxf5 Nxf5 37. Qxf5 Rf8 38. Rxd7 Rxf5 39. Rxd6 Rf7 40. g4 Kg8
41. g5 hxg5 42. hxg5 Nc5 43. Kf3 Nb7 44. Rdd1 Na5 45. Re4 c5 46. Bb2 Nc6 47. g6 Rc7 48. Kg4
Nd4 49. Rd2 Rf8 50. Bxd4 cxd4 51. Rdxd4 Rfc8 52. Kg5 Rf8 53. Rd2 Rc6 54. Rd5 Rc7 55. f5 Rb7 56.
a3 Rc7 57. a4 a6 58. Red4 Rcc8 59. Re5 Rc7 60. a5 Rc5 61. Rxc5 bxc5 62. Rd6 Ra8 63. Re6 Kf8 64.
Rc6 Ke7 65. Kf4 Kd7 66. Rxc5 Rh8 67. Rd5+ Ke7 68. Re5+ Kd7 69. Re6 Rh4+ 70. Kg5 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Bb7 5. Bg2 Bb4+ 6. Bd2 Bxd2+ 7. Qxd2 d5 8. 0-0 0-0 9. cxd5 exd5
10. Nc3 Nbd7 11. b4 c6 12. Qb2 a5 13. b5 c5 14. Rac1 Qe7 15. Na4 Rab8 16. Rfd1 c4 17. Ne5 Qe6
18. f4 Rfd8 19. Qd2 Nf8 20. Nc3 Ng6 21. Rf1 Qd6 22. a4 Rbc8 23. e3 Ne7 24. g4 Ne8 25. f5 f6 26.
Nf3 Qd7 27. Qf2 Nd6 28. Nd2 Rf8 29. Qg3 Rcd8 30. Rf4 Nf7 31. Rf2 Rfe8 32. h3 Qd6 33. Nf1 Qa3
34. Rcc2 h5 35. Qc7 Qd6 36. Qxd6 Rxd6 37. Ng3 h4 38. Nh5 Ng5 39. Rf1 Kh7 40. Nf4 Rdd8 41. Kh2
Rd7 42. Bh1 Rd6 43. Ng2 g6 44. Nxh4 gxf5 45. gxf5 Rh8 46. Nf3 Kg7 47. Nxg5 fxg5 48. Rg2 Kf6 49.
Rg3 Re8 50. Bf3 Rdd8 51. Be2 Rf8 52. Bg4 Nc8 53. Bf3 Rfe8 54. h4 Rh8 55. h5 Rhe8 56. Bg2 Ne7
57. h6 Rh8 58. Rh3 Rh7 59. Kg1 Ba8 60. Nd1 g4 61. Rh5 g3 62. Nc3 Ng8 63. Ne2 Rxh6 64. Nxg3
Rxh5 65. Nxh5+ Kf7 66. Kf2 Nf6 67. Nxf6 Kxf6 68. Rh1 c3 69. Rc1 Rh8 70. Rxc3 Kxf5 71. Rc7 Kf6
72. Bf3 Rg8 73. Rh7 Rg6 74. Bd1 Rg8 75. Rh6+ Ke7 76. Rxb6 Kd7 77. Rf6 Ke7 78. Rh6 Rg7 79. Rh8
Bb7 80. Rh5 Kd6 81. Rh3 Rf7+ 82. Ke1 Bc8 83. Rh6+ Kc7 84. Rc6+ Kb8 85. Rd6 Bb7 86. b6 Ba6 87.
Rxd5 Rf6 88. Rxa5 Rxb6 89. Kd2 Bb7 90. Rb5 Rf6 91. Bb3 Kc7 92. Re5 Ba6 93. Kc3 Rf1 94. Bc2
Rh1 95. a5 Kd6 96. e4 Bf1 97. Rf5 Bg2 98. Rf4 Rc1 99. Kb2 Rh1 100. a6 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. g3 Bb7 5. Bg2 Bb4+ 6. Bd2 Be7 7. Nc3 c6 8. e4 d5 9. e5 Ne4 10. 0-0
Ba6 11. b3 Nxc3 12. Bxc3 dxc4 13. b4 b5 14. Nd2 0-0 15. Ne4 Bb7 16. Qg4 Nd7 17. Nc5 Nxc5 18.
dxc5 a5 19. a3 axb4 20. axb4 Rxa1 21. Rxa1 Qd3 22. Rc1 Ra8 23. h4 Qd8 24. Be4 Qc8 25. Kg2 Qc7
26. Qh5 g6 27. Qg4 Bf8 28. h5 Rd8 29. Qh4 Qe7 30. Qf6 Qe8 31. Rh1 Rd7 32. hxg6 fxg6 33. Qh4
Qe7 34. Qg4 Rd8 35. Bb2 Qf7 36. Bc1 c3 37. Be3 Be7 38. Qe2 Bf8 39. Qc2 Bg7 40. Qxc3 Qd7 41.
Rc1 Qc7 42. Bg5 Rf8 43. f4 h6 44. Bf6 Bxf6 45. exf6 Qf7 46. Ra1 Qxf6 47. Qxf6 Rxf6 48. Ra7 Rf7
49. Bxg6 Rd7 50. Kf2 Kf8 51. g4 Bc8 52. Ra8 Rc7 53. Ke3 h5 54. gxh5 Kg7 55. Ra2 Re7 56. Be4 e5
57. Bxc6 exf4+ 58. Kxf4 Rf7+ 59. Ke5 Rf5+ 60. Kd6 Rxh5 61. Rg2+ Kf6 62. Kc7 Bf5 63. Kb6 Rh4
64. Ka5 Bg4 65. Bxb5 Ke7 66. Rg3 Bc8 67. Re3+ Kf7 68. Be2 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. d4 e6 2. e4 d5 3. Nc3 Nf6 4. e5 Nfd7 5. f4 c5 6. Nf3 cxd4 7. Nb5 Bb4+ 8. Bd2 Bc5 9. b4 Be7 10.
Nbxd4 Nc6 11. c3 a5 12. b5 Nxd4 13. cxd4 Nb6 14. a4 Nc4 15. Bd3 Nxd2 16. Kxd2 Bd7 17. Ke3 b6
18. g4 h5 19. Qg1 hxg4 20. Qxg4 Bf8 21. h4 Qe7 22. Rhc1 g6 23. Rc2 Kd8 24. Rac1 Qe8 25. Rc7 Rc8
26. Rxc8+ Bxc8 27. Rc6 Bb7 28. Rc2 Kd7 29. Ng5 Be7 30. Bxg6 Bxg5 31. Qxg5 fxg6 32. f5 Rg8 33.
Qh6 Qf7 34. f6 Kd8 35. Kd2 Kd7 36. Rc1 Kd8 37. Qe3 Qf8 38. Qc3 Qb4 39. Qxb4 axb4 40. Rg1 b3
41. Kc3 Bc8 42. Kxb3 Bd7 43. Kb4 Be8 44. Ra1 Kc7 45. a5 Bd7 46. axb6+ Kxb6 47. Ra6+ Kb7 48.
Kc5 Rd8 49. Ra2 Rc8+ 50. Kd6 Be8 51. Ke7 g5 52. hxg5 1-0

[Event "AlphaZero-SF"]
[Site "Google"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
1. Nf3 Nf6 2. d4 e6 3. c4 b6 4. g3 Bb7 5. Bg2 Be7 6. 0-0 0-0 7. d5 exd5 8. Nh4 c6 9. cxd5 Nxd5 10.
Nf5 Nc7 11. e4 d5 12. exd5 Nxd5 13. Nc3 Nxc3 14. Qg4 g6 15. Nh6+ Kg7 16. bxc3 Bc8 17. Qf4 Qd6
18. Qa4 g5 19. Re1 Kxh6 20. h4 f6 21. Be3 Bf5 22. Rad1 Qa3 23. Qc4 b5 24. hxg5+ fxg5 25. Qh4+
Kg6 26. Qh1 Kg7 27. Be4 Bg6 28. Bxg6 hxg6 29. Qh3 Bf6 30. Kg2 Qxa2 31. Rh1 Qg8 32. c4 Re8 33.
Bd4 Bxd4 34. Rxd4 Rd8 35. Rxd8 Qxd8 36. Qe6 Nd7 37. Rd1 Nc5 38. Rxd8 Nxe6 39. Rxa8 Kf6 40.
cxb5 cxb5 41. Kf3 Nd4+ 42. Ke4 Nc6 43. Rc8 Ne7 44. Rb8 Nf5 45. g4 Nh6 46. f3 Nf7 47. Ra8 Nd6+
48. Kd5 Nc4 49. Rxa7 Ne3+ 50. Ke4 Nc4 51. Ra6+ Kg7 52. Rc6 Kf7 53. Rc5 Ke6 54. Rxg5 Kf6 55.
Rc5 g5 56. Kd4 1-0
[/pgn]

[smatovic]

Since no one of us has access to TPU it's only fair to count in terms of what is available (for example if we had Alpha0 x64 binary compile and wanted to run it at home).
1TPU ~ 30xE5-2699v3 (18 cores machine).
4TPUs ~ 2000 Haswell cores

Apples and Bananas,

Stockfish is not able to make use of these TPUs,
and AlphaZero depends probably heavily on floating point operations (maybe half precision) to query the neural network.

So the question might be, if an stripped down x86-64 version of AlphaZero, with only some hundred or thousand of nps, is still able to beat Stockfish.....dunno.



--
Srdja

[Vinvin]

The 10 games with graph eval on Lichess :
https://lichess.org/zbCOTavB
https://lichess.org/CXpk8N5X
https://lichess.org/zDJ4IyBP
https://lichess.org/tnbEgxLG
https://lichess.org/ZMuecDNu
https://lichess.org/Y6tIfcqu
https://lichess.org/KRYcCvtd
https://lichess.org/tJzewmVJ
https://lichess.org/SfaOcHXM
https://lichess.org/B6ebUyzC

[Guenther]

The 10 games with graph eval on Lichess :
https://lichess.org/zbCOTavB
https://lichess.org/CXpk8N5X
https://lichess.org/zDJ4IyBP
https://lichess.org/tnbEgxLG
https://lichess.org/ZMuecDNu
https://lichess.org/Y6tIfcqu
https://lichess.org/KRYcCvtd
https://lichess.org/tJzewmVJ
https://lichess.org/SfaOcHXM
https://lichess.org/B6ebUyzC

What is the usual time or depth lichess-SF8 uses for those annotations?

At a first quick glance it seems impossible that SF on good hardware
is even able to make some of the bad moves that are 'colported' from blank pgn.
From the annotations it looks more likely that in reality SF8 was AlphaChess and vice versa ;-)

[Guenther]

[pgn][Event "AlphaZero-SF"]
[Site "https://lichess.org/zbCOTavB"]
[Date "2017.12.06"]
[Round "-"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
[UTCDate "2017.12.06"]
[UTCTime "09:29:15"]
[WhiteElo "?"]
[BlackElo "?"]
[Variant "Standard"]
[TimeControl "-"]
[ECO "E17"]
[Opening "Queen's Indian Defense: Classical Variation, Polugaevsky Gambit"]
[Termination "Normal"]
[Annotator "lichess.org"]

1. Nf3 Nf6 2. d4 e6 3. c4 b6 4. g3 Bb7 5. Bg2 Be7 6. O-O O-O 7. d5 exd5 8. Nh4 { E17 Queen's Indian Defense: Classical Variation, Polugaevsky Gambit } c6 9. cxd5 Nxd5 10. Nf5 Nc7 11. e4 d5 12. exd5 Nxd5 13. Nc3 Nxc3 14. Qg4 g6 15. Nh6+ Kg7 16. bxc3 Bc8 17. Qf4 Qd6 18. Qa4 g5 19. Re1 Kxh6 20. h4 f6 21. Be3?! { (-0.64 → -1.48) Inaccuracy. Best move was Rxe7. } (21. Rxe7 Qxe7 22. Ba3 c5 23. Bxa8 Bf5 24. Bg2 Rd8 25. hxg5+ fxg5 26. Bc1 Kg7 27. Be3 h6) 21... Bf5? { (-1.48 → -0.25) Mistake. Best move was Kg7. } (21... Kg7 22. Rad1 Qe6 23. Bxg5 Qf7 24. Be3 b5 25. Qe4 a5 26. a4 bxa4 27. Qxa4 Re8 28. Bf1) 22. Rad1 Qa3?! { (-0.61 → 0.00) Inaccuracy. Best move was Qc7. } (22... Qc7 23. Be4) 23. Qc4 b5 24. hxg5+ fxg5 25. Qh4+ Kg6 26. Qh1 Kg7 27. Be4 Bg6 28. Bxg6 hxg6 29. Qh3 Bf6 30. Kg2 Qxa2 31. Rh1 Qg8 32. c4 Re8 33. Bd4? { (0.00 → -1.07) Mistake. Best move was Rd6. } (33. Rd6 bxc4 34. Qh6+ Kf7 35. Rxf6+ Kxf6 36. Qxg5+ Ke6 37. Rd1 Qf7 38. Qg4+ Qf5 39. Qxc4+ Ke7) 33... Bxd4? { (-1.07 → 0.00) Mistake. Best move was Rd8. } (33... Rd8 34. Bxf6+ Kxf6 35. cxb5 Rd5 36. Qg4 Nd7 37. Qf3+ Kg7 38. bxc6 Nf6 39. Rb1 Rf8 40. Rb7+) 34. Rxd4 Rd8? { (-0.24 → 1.59) Mistake. Best move was Re5. } (34... Re5 35. cxb5 cxb5 36. Rd3 Re6 37. Qh6+ Kf7 38. Qxg5 Nc6 39. Rf3+ Ke8 40. Qxb5 Rb8 41. Qc5) 35. Rxd8 Qxd8 36. Qe6 Nd7 37. Rd1 Nc5 38. Rxd8 Nxe6 39. Rxa8 Kf6 40. cxb5 cxb5 41. Kf3 Nd4+?! { (1.29 → 2.10) Inaccuracy. Best move was Ke5. } (41... Ke5 42. Ke3 Ng7 43. Rxa7 Nf5+ 44. Kd3 Kd5 45. Rb7 Nd6 46. Rb8 g4 47. Rg8 Nc4 48. Kc3) 42. Ke4 Nc6 43. Rc8 Ne7? { (1.60 → 3.92) Mistake. Best move was Nb4. } (43... Nb4 44. Rc5) 44. Rb8 Nf5?! { (3.50 → 4.33) Inaccuracy. Best move was b4. } (44... b4 45. Rxb4) 45. g4 Nh6? { (3.92 → 4.98) Mistake. Best move was Nd6+. } (45... Nd6+ 46. Kd5 Nf7 47. Rxb5 Ne5 48. Rb4 Nf3 49. Ra4 Nh2 50. Ra6+ Kg7 51. Rxa7+ Kf6 52. Ra4) 46. f3 Nf7 47. Ra8 Nd6+ 48. Kd5 Nc4 49. Rxa7 Ne3+ 50. Ke4 Nc4 51. Ra6+ Kg7 52. Rc6 Kf7?! { (4.23 → 4.78) Inaccuracy. Best move was Nd2+. } (52... Nd2+ 53. Ke3 Nb1 54. Kd3 b4 55. Rb6 Nc3 56. Rxb4 Nd5 57. Rb5 Nf4+ 58. Ke3 Nh3 59. Ke4) 53. Rc5 Ke6?! { (4.01 → 4.87) Inaccuracy. Best move was Kf6. } (53... Kf6 54. Rd5) 54. Rxg5 Kf6 55. Rc5 g5 56. Kd4 { Black resigns. } 1-0
[/pgn]

I tried with the 'old' SF8 here and even with 1 cpu setting it would never play 21. Be3??
Also it would always go for a dead draw by repetition with 16. Nf5+ (=) instead of the weaker 16. bxc3?

This raises some questions. Did they perhaps add extreme contempt to SF? Of course the whole setup could be erroneous anyways.

Thanks to Vincent for the lichess analysis pgn.

[Vinvin]

The 10 games with graph eval on Lichess :
https://lichess.org/zbCOTavB
https://lichess.org/CXpk8N5X
https://lichess.org/zDJ4IyBP
https://lichess.org/tnbEgxLG
https://lichess.org/ZMuecDNu
https://lichess.org/Y6tIfcqu
https://lichess.org/KRYcCvtd
https://lichess.org/tJzewmVJ
https://lichess.org/SfaOcHXM
https://lichess.org/B6ebUyzC

What is the usual time or depth lichess-SF8 uses for those annotations?

At a first quick glance it seems impossible that SF on good hardware
is even able to make some of the bad moves that are 'colported' from blank pgn.
From the annotations it looks more likely that in reality SF8 was AlphaChess and vice versa

I don't remember exactly but from my memory the analyse uses around 3 sec per move on a 2 GHz CPU.
The moves in the game has been played with Stockfish 8 on 64 CPUs 1 min/per move (quote from the paper : "In this section we include 10 example games played by AlphaZero against Stockfish during the 100 game match using 1 minute per move." )
70 Mnodes/sec for SF8 vs 80Knodes/sec for AlphaZero, see tables S4 in the paper : https://arxiv.org/pdf/1712.01815.pdf

[zenpawn]

[pgn][Event "AlphaZero-SF"]
[Site "https://lichess.org/zbCOTavB"]
[Date "2017.12.06"]
[Round "-"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
[UTCDate "2017.12.06"]
[UTCTime "09:29:15"]
[WhiteElo "?"]
[BlackElo "?"]
[Variant "Standard"]
[TimeControl "-"]
[ECO "E17"]
[Opening "Queen's Indian Defense: Classical Variation, Polugaevsky Gambit"]
[Termination "Normal"]
[Annotator "lichess.org"]

1. Nf3 Nf6 2. d4 e6 3. c4 b6 4. g3 Bb7 5. Bg2 Be7 6. O-O O-O 7. d5 exd5 8. Nh4 { E17 Queen's Indian Defense: Classical Variation, Polugaevsky Gambit } c6 9. cxd5 Nxd5 10. Nf5 Nc7 11. e4 d5 12. exd5 Nxd5 13. Nc3 Nxc3 14. Qg4 g6 15. Nh6+ Kg7 16. bxc3 Bc8 17. Qf4 Qd6 18. Qa4 g5 19. Re1 Kxh6 20. h4 f6 21. Be3?! { (-0.64 → -1.48) Inaccuracy. Best move was Rxe7. } (21. Rxe7 Qxe7 22. Ba3 c5 23. Bxa8 Bf5 24. Bg2 Rd8 25. hxg5+ fxg5 26. Bc1 Kg7 27. Be3 h6) 21... Bf5? { (-1.48 → -0.25) Mistake. Best move was Kg7. } (21... Kg7 22. Rad1 Qe6 23. Bxg5 Qf7 24. Be3 b5 25. Qe4 a5 26. a4 bxa4 27. Qxa4 Re8 28. Bf1) 22. Rad1 Qa3?! { (-0.61 → 0.00) Inaccuracy. Best move was Qc7. } (22... Qc7 23. Be4) 23. Qc4 b5 24. hxg5+ fxg5 25. Qh4+ Kg6 26. Qh1 Kg7 27. Be4 Bg6 28. Bxg6 hxg6 29. Qh3 Bf6 30. Kg2 Qxa2 31. Rh1 Qg8 32. c4 Re8 33. Bd4? { (0.00 → -1.07) Mistake. Best move was Rd6. } (33. Rd6 bxc4 34. Qh6+ Kf7 35. Rxf6+ Kxf6 36. Qxg5+ Ke6 37. Rd1 Qf7 38. Qg4+ Qf5 39. Qxc4+ Ke7) 33... Bxd4? { (-1.07 → 0.00) Mistake. Best move was Rd8. } (33... Rd8 34. Bxf6+ Kxf6 35. cxb5 Rd5 36. Qg4 Nd7 37. Qf3+ Kg7 38. bxc6 Nf6 39. Rb1 Rf8 40. Rb7+) 34. Rxd4 Rd8? { (-0.24 → 1.59) Mistake. Best move was Re5. } (34... Re5 35. cxb5 cxb5 36. Rd3 Re6 37. Qh6+ Kf7 38. Qxg5 Nc6 39. Rf3+ Ke8 40. Qxb5 Rb8 41. Qc5) 35. Rxd8 Qxd8 36. Qe6 Nd7 37. Rd1 Nc5 38. Rxd8 Nxe6 39. Rxa8 Kf6 40. cxb5 cxb5 41. Kf3 Nd4+?! { (1.29 → 2.10) Inaccuracy. Best move was Ke5. } (41... Ke5 42. Ke3 Ng7 43. Rxa7 Nf5+ 44. Kd3 Kd5 45. Rb7 Nd6 46. Rb8 g4 47. Rg8 Nc4 48. Kc3) 42. Ke4 Nc6 43. Rc8 Ne7? { (1.60 → 3.92) Mistake. Best move was Nb4. } (43... Nb4 44. Rc5) 44. Rb8 Nf5?! { (3.50 → 4.33) Inaccuracy. Best move was b4. } (44... b4 45. Rxb4) 45. g4 Nh6? { (3.92 → 4.98) Mistake. Best move was Nd6+. } (45... Nd6+ 46. Kd5 Nf7 47. Rxb5 Ne5 48. Rb4 Nf3 49. Ra4 Nh2 50. Ra6+ Kg7 51. Rxa7+ Kf6 52. Ra4) 46. f3 Nf7 47. Ra8 Nd6+ 48. Kd5 Nc4 49. Rxa7 Ne3+ 50. Ke4 Nc4 51. Ra6+ Kg7 52. Rc6 Kf7?! { (4.23 → 4.78) Inaccuracy. Best move was Nd2+. } (52... Nd2+ 53. Ke3 Nb1 54. Kd3 b4 55. Rb6 Nc3 56. Rxb4 Nd5 57. Rb5 Nf4+ 58. Ke3 Nh3 59. Ke4) 53. Rc5 Ke6?! { (4.01 → 4.87) Inaccuracy. Best move was Kf6. } (53... Kf6 54. Rd5) 54. Rxg5 Kf6 55. Rc5 g5 56. Kd4 { Black resigns. } 1-0
[/pgn]

I tried with the 'old' SF8 here and even with 1 cpu setting it would never play 21. Be3??
Also it would always go for a dead draw by repetition with 16. Nf5+ (=) instead of the weaker 16. bxc3?

This raises some questions. Did they perhaps add extreme contempt to SF? Of course the whole setup could be erroneous anyways.

Thanks to Vincent for the lichess analysis pgn.

Those are AlphaZero moves you've called out as odd.

[Vinvin]

[pgn][Event "AlphaZero-SF"]
[Site "https://lichess.org/zbCOTavB"]
[Date "2017.12.06"]
[Round "-"]
[White "AlphaZero"]
[Black "Stockfish8"]
[Result "1-0"]
[UTCDate "2017.12.06"]
[UTCTime "09:29:15"]
[WhiteElo "?"]
[BlackElo "?"]
[Variant "Standard"]
[TimeControl "-"]
[ECO "E17"]
[Opening "Queen's Indian Defense: Classical Variation, Polugaevsky Gambit"]
[Termination "Normal"]
[Annotator "lichess.org"]

1. Nf3 Nf6 2. d4 e6 3. c4 b6 4. g3 Bb7 5. Bg2 Be7 6. O-O O-O 7. d5 exd5 8. Nh4 { E17 Queen's Indian Defense: Classical Variation, Polugaevsky Gambit } c6 9. cxd5 Nxd5 10. Nf5 Nc7 11. e4 d5 12. exd5 Nxd5 13. Nc3 Nxc3 14. Qg4 g6 15. Nh6+ Kg7 16. bxc3 Bc8 17. Qf4 Qd6 18. Qa4 g5 19. Re1 Kxh6 20. h4 f6 21. Be3?! { (-0.64 → -1.48) Inaccuracy. Best move was Rxe7. } (21. Rxe7 Qxe7 22. Ba3 c5 23. Bxa8 Bf5 24. Bg2 Rd8 25. hxg5+ fxg5 26. Bc1 Kg7 27. Be3 h6) 21... Bf5? { (-1.48 → -0.25) Mistake. Best move was Kg7. } (21... Kg7 22. Rad1 Qe6 23. Bxg5 Qf7 24. Be3 b5 25. Qe4 a5 26. a4 bxa4 27. Qxa4 Re8 28. Bf1) 22. Rad1 Qa3?! { (-0.61 → 0.00) Inaccuracy. Best move was Qc7. } (22... Qc7 23. Be4) 23. Qc4 b5 24. hxg5+ fxg5 25. Qh4+ Kg6 26. Qh1 Kg7 27. Be4 Bg6 28. Bxg6 hxg6 29. Qh3 Bf6 30. Kg2 Qxa2 31. Rh1 Qg8 32. c4 Re8 33. Bd4? { (0.00 → -1.07) Mistake. Best move was Rd6. } (33. Rd6 bxc4 34. Qh6+ Kf7 35. Rxf6+ Kxf6 36. Qxg5+ Ke6 37. Rd1 Qf7 38. Qg4+ Qf5 39. Qxc4+ Ke7) 33... Bxd4? { (-1.07 → 0.00) Mistake. Best move was Rd8. } (33... Rd8 34. Bxf6+ Kxf6 35. cxb5 Rd5 36. Qg4 Nd7 37. Qf3+ Kg7 38. bxc6 Nf6 39. Rb1 Rf8 40. Rb7+) 34. Rxd4 Rd8? { (-0.24 → 1.59) Mistake. Best move was Re5. } (34... Re5 35. cxb5 cxb5 36. Rd3 Re6 37. Qh6+ Kf7 38. Qxg5 Nc6 39. Rf3+ Ke8 40. Qxb5 Rb8 41. Qc5) 35. Rxd8 Qxd8 36. Qe6 Nd7 37. Rd1 Nc5 38. Rxd8 Nxe6 39. Rxa8 Kf6 40. cxb5 cxb5 41. Kf3 Nd4+?! { (1.29 → 2.10) Inaccuracy. Best move was Ke5. } (41... Ke5 42. Ke3 Ng7 43. Rxa7 Nf5+ 44. Kd3 Kd5 45. Rb7 Nd6 46. Rb8 g4 47. Rg8 Nc4 48. Kc3) 42. Ke4 Nc6 43. Rc8 Ne7? { (1.60 → 3.92) Mistake. Best move was Nb4. } (43... Nb4 44. Rc5) 44. Rb8 Nf5?! { (3.50 → 4.33) Inaccuracy. Best move was b4. } (44... b4 45. Rxb4) 45. g4 Nh6? { (3.92 → 4.98) Mistake. Best move was Nd6+. } (45... Nd6+ 46. Kd5 Nf7 47. Rxb5 Ne5 48. Rb4 Nf3 49. Ra4 Nh2 50. Ra6+ Kg7 51. Rxa7+ Kf6 52. Ra4) 46. f3 Nf7 47. Ra8 Nd6+ 48. Kd5 Nc4 49. Rxa7 Ne3+ 50. Ke4 Nc4 51. Ra6+ Kg7 52. Rc6 Kf7?! { (4.23 → 4.78) Inaccuracy. Best move was Nd2+. } (52... Nd2+ 53. Ke3 Nb1 54. Kd3 b4 55. Rb6 Nc3 56. Rxb4 Nd5 57. Rb5 Nf4+ 58. Ke3 Nh3 59. Ke4) 53. Rc5 Ke6?! { (4.01 → 4.87) Inaccuracy. Best move was Kf6. } (53... Kf6 54. Rd5) 54. Rxg5 Kf6 55. Rc5 g5 56. Kd4 { Black resigns. } 1-0
[/pgn]

I tried with the 'old' SF8 here and even with 1 cpu setting it would never play 21. Be3??
Also it would always go for a dead draw by repetition with 16. Nf5+ (=) instead of the weaker 16. bxc3?

This raises some questions. Did they perhaps add extreme contempt to SF? Of course the whole setup could be erroneous anyways.

Thanks to Vincent for the lichess analysis pgn.

White is AlphaZero in this game and white won