Lc0 with two Networks?

[Eduard]

I read that Scorpio can use two networks. Lc0 with two nets (one for the endgame) would be strong. Especially small networks, would play better (with a little ender like scorpio).

[jjoshua2]

There was a fork of lc0 that is old by now, that allowed a separate net to run in the endgame to switch over at a set piece amount. Like at 16 pieces switch to ender net. I think it was found to gain some elo overall, but I guess it wasn't merged due to complexity, and nets have improved in endgame a lot since then. Dkappe has talked about reviving it.

[the_real_greco]

Scorpio 3.0.5 has two networks.

Oh wait you said that. Ignore me.

[crem]

It's possible to switch network file between moves in Lc0 with uci parameter.

However it's not currently possible to use several networks within one search (e.g. depending on depth or number of pieces).
But won't that suffer from Evaluation Discontinuity?

[dkappe]

Getting a smooth switch is still a bit tricky. Still, the results are promising. Here a game between FF+LE and 42850 at 3 seconds per move on a 1070. Using Noomen 3. Switch is at 16 men.

[pgn] [Event "?"] [Site "?"] [Date "????.??.??"] [Round "3"] [White "FF+LE"] [Black "42850"] [Result "1-0"] 1. d4 Nf6 2. Bg5 e6 3. e4 c5 4. d5 d6 5. Bb5+ Bd7 6. Bxd7+ Nbxd7 7. Nc3 exd5 8. Nxd5 Be7 9. Bxf6 Nxf6 10. Ne2 Nxe4 11. Qd3 Nf6 12. Qb5+ Kf8 13. Nec3 Nxd5 14. Nxd5 Rb8 15. O-O-O h5 16. Rhe1 Bf6 17. Qd3 Rh6 18. Re4 Qd7 19. Rde1 b5 20. h3 b4 21. g4 Bh4 22. Qf3 a5 23. gxh5 Bg5+ 24. Kb1 Bf6 25. Qg4 Qxg4 26. hxg4 Bh4 27. R1e2 g6 28. hxg6 Rxg6 29. f3 Kg7 30. Ne3 Kf8 31. Nd5 Kg7 32. Ne3 Kf8 33. Kc1 Rd8 34. Nd5 Rg5 35. Nf6 Rg6 36. Re8+ Rxe8 37. Rxe8+ Kg7 38. Nh5+ Kh6 39. Rh8+ Kg5 40. Rf8 Re6 41. Rxf7 Re1+ 42. Kd2 Rf1 43. b3 Kg6 44. Rg7+ Kh6 45. Ke3 Bf6 46. Rf7 Bc3 47. Nf4 Rc1 48. Ke4 Kg5 49. Rf5+ Kh6 50. g5+ Kg7 51. g6 Re1+ 52. Kd5 Rd1+ 53. Ke6 Be5 54. Rf7+ Kh6 55. Kf5 Bg7 56. Ne6 Rd5+ 57. Kg4 Bc3 58. g7 Rd2 59. Kf5 Rg2 60. Rd7 Kh7 61. Ng5+ Kg8 62. Kg6 Bxg7 63. Rxg7+ Kf8 64. Kf6 Rxg5 65. Rxg5 d5 66. Rxd5 Ke8 67. Rxc5 Kd7 68. Rxa5 Kc6 69. f4 Kb6 70. Ra8 Kb7 71. Rd8 Kc6 72. Re8 Kb7 73. Rh8 Ka7 74. f5 Ka6 75. Rg8 Kb7 76. Rh8 Ka6 77. Ke5 Kb7 78. Kd4 Ka7 79. Kc5 Ka6 80. Ra8+ Kb7 81. Re8 Kc7 82. Re7+ Kc8 83. Kc6 Kb8 84. Rh7 Ka8 85. Kb6 Kb8 86. Rh8# 1-0 [/pgn]

[corres]

It's possible to switch network file between moves in Lc0 with uci parameter.
However it's not currently possible to use several networks within one search (e.g. depending on depth or number of pieces).
But won't that suffer from Evaluation Discontinuity?

Yes, it suffer from it.
But if you use nets as a kind of book, this disturbance may be minimal. So I think during the opening phase and in the endgame phase you can use (appropriate) specialized net together with Leela standard net. The system for this may be the same as is in those AB engines what use more book: If the standard net do not contain "Value" for a position, Leela should search for it in the another net. Obviously the used nets should be tuned well to get real enhancement in Elo and at least the auxiliary net would be small to decrease the waste of time.