Daniel Shawul wrote:
They had no option but to use MCTS not because it is better.
That is because it was getting 80,000 nodes/s even on 4-TPUs. With that nps a full width alpha-beta search you are stuck with search depth engines used to get in the 90's. That brings up tactical problems which they minimized with massive hardware -- it annoys me that there is no mention of this in the arxiv paper. They could have said yes their is a problem that could be exploited by a tactical engine, but we solved it with massive hardware woud be enough. The kind of tactical mistakes Leala zero made on a 48 core tcec machine speaks loud about this problem.
The tactical problem of A0 is there only if we are speaking of very short time controls. And also anyone can see the strength scaling graphs in the paper. From there it's mostly clear how strength of SF8 and A0 scales based on thinking time and also based on total nodes searched. So it's not like they made it a secret that strength of A0 goes down rapidly as total nodes searched approaches zero. Or that A0 on 1sec / move + 1080Ti would be much weaker compared to SF8 on 64 cores, while at 1 min / move they would be similar in strength. The only thing that was not explicitly mentioned is that the reason of such scaling at low nodes count is due to "tactical vulnerability" - but that should be more or less given.
Also speaking of details that are "not being explicitly mentioned" it seems to me you are overly concerned with tactical vulnerabilities present only at short time controls and ignoring the fact that A0 + 1080Ti at 1 min / move or above doesn't suffer with tactical vulnerabilities (unless you want also to call 64 core SF8, 1min / move tactically vulnerable, or assume that A0 can match SF8 strength and still be somehow tactically "inferior")
And you can't compare level at which LC0 is at the moment with A0. 1 month ago LC0 was doing even much more horrible tactical blunders, so now you extrapolate to the future and I think it should be clear what the correct conclusions should be.
So if 1s / move or engine bullet games is your thing than sure, A0 will suck there on consumer HW for quiet some time. If on the other hand you are more inclined towards LTC, then clearly A0 approach is the way to go. I mean if they made it a regular 120min / 40 moves + 30 sec increment match + proper time management SF8 would probably lose even much worse than just by 100 elo.