is Leela still the best engine at Knight handicap ?

[JJJ]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

[lkaufman]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

Early Leela nets (starting with "1") were indeed quite strong at knight odds, then for a long time later ones were simply awful, recently they have become good again. I'm reasonably confident that current Leela is much better than recent Stockfish at giving knight odds, but whether it is as good as Dragon at this is questionable. It would have to prove that it can draw or win over 50% of the games at 15' + 10" vs 2500 level GMs, as Dragon has done in three matches totaling 21 games.

[JJJ]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

Early Leela nets (starting with "1") were indeed quite strong at knight odds, then for a long time later ones were simply awful, recently they have become good again. I'm reasonably confident that current Leela is much better than recent Stockfish at giving knight odds, but whether it is as good as Dragon at this is questionable. It would have to prove that it can draw or win over 50% of the games at 15' + 10" vs 2500 level GMs, as Dragon has done in three matches totaling 21 games.

Are they better now than before ? Can you give me an exemple of a strong recent net at this handicap ? Is it possible to run a match of knight handicap against a human like bot to determine the strongest chess engine ?

[Uri Blass]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

Early Leela nets (starting with "1") were indeed quite strong at knight odds, then for a long time later ones were simply awful, recently they have become good again. I'm reasonably confident that current Leela is much better than recent Stockfish at giving knight odds, but whether it is as good as Dragon at this is questionable. It would have to prove that it can draw or win over 50% of the games at 15' + 10" vs 2500 level GMs, as Dragon has done in three matches totaling 21 games.

Are they better now than before ? Can you give me an exemple of a strong recent net at this handicap ? Is it possible to run a match of knight handicap against a human like bot to determine the strongest chess engine ?

It is possible to run a tournament against relatively weak engines to find the best engine with knight handicap against chess engines.
I do not know if the best against chess engines is also the best against humans.

[lkaufman]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

Early Leela nets (starting with "1") were indeed quite strong at knight odds, then for a long time later ones were simply awful, recently they have become good again. I'm reasonably confident that current Leela is much better than recent Stockfish at giving knight odds, but whether it is as good as Dragon at this is questionable. It would have to prove that it can draw or win over 50% of the games at 15' + 10" vs 2500 level GMs, as Dragon has done in three matches totaling 21 games.

Are they better now than before ? Can you give me an exemple of a strong recent net at this handicap ? Is it possible to run a match of knight handicap against a human like bot to determine the strongest chess engine ?

It is possible to run a tournament against relatively weak engines to find the best engine with knight handicap against chess engines.
I do not know if the best against chess engines is also the best against humans.

Dragon MCTS consistently does worse than standard dragon giving knight odds to engines, but better against humans. So there is strong evidence that results against engines giving knight odds are not closely correlated with results vs. humans.

[Marcus9]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

Early Leela nets (starting with "1") were indeed quite strong at knight odds, then for a long time later ones were simply awful, recently they have become good again. I'm reasonably confident that current Leela is much better than recent Stockfish at giving knight odds, but whether it is as good as Dragon at this is questionable. It would have to prove that it can draw or win over 50% of the games at 15' + 10" vs 2500 level GMs, as Dragon has done in three matches totaling 21 games.

Are they better now than before ? Can you give me an exemple of a strong recent net at this handicap ? Is it possible to run a match of knight handicap against a human like bot to determine the strongest chess engine ?

It is possible to run a tournament against relatively weak engines to find the best engine with knight handicap against chess engines.
I do not know if the best against chess engines is also the best against humans.

Dragon MCTS consistently does worse than standard dragon giving knight odds to engines, but better against humans. So there is strong evidence that results against engines giving knight odds are not closely correlated with results vs. humans.

Of course, however a "knight odd rating list" would be very interesting.
It will be possible to analyze how an engine, even with different settings, performs against other engines.
Just choose: Time Control, an appropriate book,an opponent engines pool, like leela with few fixed nodes/s (Dragon MCTS will perform better?) and other indipendent engines with a few hundred elo differen each other.

[lkaufman]

All in the title. I know the first leela net were very strong at knight handicap because they trained even being down material. But perhaps, Komodo is the now better than Leela ?

Early Leela nets (starting with "1") were indeed quite strong at knight odds, then for a long time later ones were simply awful, recently they have become good again. I'm reasonably confident that current Leela is much better than recent Stockfish at giving knight odds, but whether it is as good as Dragon at this is questionable. It would have to prove that it can draw or win over 50% of the games at 15' + 10" vs 2500 level GMs, as Dragon has done in three matches totaling 21 games.

Are they better now than before ? Can you give me an exemple of a strong recent net at this handicap ? Is it possible to run a match of knight handicap against a human like bot to determine the strongest chess engine ?

It is possible to run a tournament against relatively weak engines to find the best engine with knight handicap against chess engines.
I do not know if the best against chess engines is also the best against humans.

Dragon MCTS consistently does worse than standard dragon giving knight odds to engines, but better against humans. So there is strong evidence that results against engines giving knight odds are not closely correlated with results vs. humans.

Of course, however a "knight odd rating list" would be very interesting.
It will be possible to analyze how an engine, even with different settings, performs against other engines.
Just choose: Time Control, an appropriate book,an opponent engines pool, like leela with few fixed nodes/s (Dragon MCTS will perform better?) and other indipendent engines with a few hundred elo differen each other.

Ed Schroder did such a knight odds test of top engines vs. weaker ones (like 2500 to 2700 CCRL or so) a few months back, but I don't think he tested Lc0 due to only being able to play one game at a time with it using the GPU. Dragon MCTS wouldn't be suitable as the odds receiver since if you limit its time or nodes severely enough to drop it to that elo range it becomes essentially regular Dragon doing a short search. Regular Dragon with a short fixed depth (or a suitable Elo setting) would be ok as the odds receiver, but then when full strength Dragon plays against it it might do much better or much worse than against a totally unrelated engine. You could use a recent Lc0 CPU version as the odds receiver with a tiny NPS limit; some tests I ran about a year ago suggested than somewhere in the 1.5 to 2 NPS range would make it a fair opponent for full strength Dragon giving knight odds, but of course the exact figure depends on the version, the hardware, and the time limit. But then we have the same problem if regular Lc0 is giving knight odds to the crippled Lc0, it might do much better or worse due to the similarity. My best suggestion is to use Critter 1.6a on one thread at some fixed depth, maybe around 11 plies, as the odds receiver, because Critter is obsolete and not similar to any modern top engine, but was very good at playing with a handicap, and is strong enough to challenge top engines at knight odds without needing more than some fraction of a second per move. There is still the question of the opening book; the knight odds book by chrisW offers sufficient variety, but it isn't quite "pure" knight odds since both sides play suboptimal moves in most lines. It's probably the best available book for the idea though.