Guenther wrote: ↑Wed Sep 26, 2018 8:14 am
This is the table after some cleaning up (not only the unterminated, but also the wrong game headers)
Game 1 which was still testing and was played with no opening moves at all still counts...
Andscacs instead of 1. e4 c5 2. Nf3 h6 3. c3 vs. SF played from the start position and the testing game
was elevated to a real stage 2 game later.
I want to suggest the elo of LC0 somehow manages to match the elo of whatever it is playing against. Well, that's one "explanation" of its rather curious behaviour:
If one looks at the loss count of the second program, Houdini against the final ranking of its opponents, we get, as would be expected, an decreasing gradient: 5,1,0,0,0,0,0
Komodo gets: 2,3,2,0,1,0,0
Ethereal: 6,3,3,1,2,0,1
Fire: 6,6,3,3,1,0,1
Booot: 5,4,4,3,1,3,0
Andsacs: 7,4,4,3,5,1,1
Lc0 is different: 1,1,1,1,1,0,2, almost irrelevent who the opponent is, the loss rate remains almost constant.
Obviously, "non-losses", counting wins and draws together, shows the same pattern in reverse. Which suggests, well, to me, that LC0 doesn't really have an elo that can be mapped onto any particular opponent. It's not behaving itself properly according to the laws of elo ratings.
This constant loss rate is mainly due to the tactical weakness of lc0 that, when it appears against any alpha-beta program always leads to defeat.
On the other hand, this tactical weakness is practically irrelevant against humans, unable to detect it and take advantage of it in most cases.
So, perhaps we should say that lc0 is not behaving itself according to the laws of elo of the classical alpha-beta chess programs.
Note that stage 1 and stage 2 are not cleaned up files and adjudication results are missing for the
unterminated games. Also one game is missing per file.
Robert Pope wrote: ↑Wed Sep 26, 2018 5:15 pm
I think the issue is that LC0 plays very well, except for a glaring hole that almost any opponent can capitalize on. Other engines have their own holes, but they are subtle, so weaker engines are less likely to discover them.
So we can say LC0 capitalize the weaker positional knowledge of AB engines and AB engines capitalize the weaker tactical knowledge of AB engines.
Where is the engine with positional knowledge of LC0 and tactical knowledge of AB engines?
Not far off. Will continue to be shocked there isn’t an adequate if not necessarily legal jerryrigged hybrid floating around. If lc0 + 3rd party doesn’t do it my money would be on Komodo or Houdini making a splash commercially.
