I toast to LeelaqueenOdds and its creators. I raise my wine glass high and shout: Long live the queen. We have a queen!!

[lkaufman]

Here is another video of a strong player trying the QueenOddsBot

What is certain is that this BOT is NOT playing just with the SF NN. It annoyingly avoids trading down even when the trade is to the advantage of the BOT and seems forced. Unlike the normal NN from SF ... which quickly allows trading when it is advantageous ... this one will hold on to material even if it means that it is detrimental to the position. It just avoids trades to keep as many pieces on the board as possible ... and that way us humans will have more chances of making mistakes.

I do wish that playing this BOT is easier as you have to do quite a bit just to play a game and you cannot rematch it. Once that happens, most likely systems can be developed to easily beat it. I mean a Queen is a Queen after all. But then again maybe the NN can also change and create more challenges.

I thought you were knowledgeable about computer chess, based on past posts, so I am shocked that you don't know the difference between Stockfish and Leela. They are completely different engines, Stockfish is CPU based and does normal Alpha-beta search, while Leela is GPU based and uses a version of Monte-Carlo search. Your question asked about playing Stockfish, so that's what I answered. The Leela odds bots are trained specifically for odds chess, so they are vastly superior to Stockfish at doing this.
Rematch is being worked on, may be available soon. The problem is that normally rematch flips colors. It is true that with experience good players do learn and improve their results, but it's not really "systems", just learning to be careful, how to simplify, etc. The bot does get upgraded frequently and becomes more challenging each time, but of course there are limits, with enough time strong players can win with an extra queen. At the present time, the level needed to exceed 50% in blitz (5'0" or 3'2") with queen odds (Leela White) is strong FM level, maybe 2350 FIDE or so. At 1'1" it is good GM level, about 2550 FIDE. At 1'0" it is Hikaru Nakamura level, no one else has reached 40% (and he hasn't played enough to prove that he can score 50% or more).

Orichess88 won the last 9 games against the queen odds bot at 2+3 and he is only FM so I guess weaker players than him can exceed 50% in blitz.

Orichess88's fide rating is 2340

I think that it is not exactly chess so it may be possible that some players with fide rating below 2200 can get more than 50% at blitz and at the same time some players with fide rating above 2400 do not get more than 50%

Yes, results in odds games don't correlate perfectly with FIDE rating, but do correlate pretty well. Ori seems to be rather good at odds chess; on the other side one Russian GM with a 2400 FIDE rating got only half a point out of 15 games at queen odds at 3'2"!! On average I think FIDE 2350 players get about 50%, but some below 2300 do make a plus score. Below 2200 it is probably quite rare, though in general we don't know the FIDE ratings of LiChess players without FIDE titles. What I have noticed is that odds results, including queen odds, correlate much better with FIDE ratings than they do with LiChess blitz ratings, even though the odds games are blitz games! Seems strange, but maybe LiChess blitz ratings reflect skills that are useless against these bots, like playing to flag or setting traps.

[M ANSARI]

[/quote]
I thought you were knowledgeable about computer chess, based on past posts, so I am shocked that you don't know the difference between Stockfish and Leela. They are completely different engines, Stockfish is CPU based and does normal Alpha-beta search, while Leela is GPU based and uses a version of Monte-Carlo search. Your question asked about playing Stockfish, so that's what I answered. The Leela odds bots are trained specifically for odds chess, so they are vastly superior to Stockfish at doing this.
Rematch is being worked on, may be available soon. The problem is that normally rematch flips colors. It is true that with experience good players do learn and improve their results, but it's not really "systems", just learning to be careful, how to simplify, etc. The bot does get upgraded frequently and becomes more challenging each time, but of course there are limits, with enough time strong players can win with an extra queen. At the present time, the level needed to exceed 50% in blitz (5'0" or 3'2") with queen odds (Leela White) is strong FM level, maybe 2350 FIDE or so. At 1'1" it is good GM level, about 2550 FIDE. At 1'0" it is Hikaru Nakamura level, no one else has reached 40% (and he hasn't played enough to prove that he can score 50% or more).
[/quote]

I have to admit I haven't checked on chess engines for a few years. But every once in a while something comes up that makes me want to check up on chess engines again ... in this case it is the QueensOdd thing. I had thought that since new SF was using NN, that Leela and SF are basically morphed into similar engines now, and that somehow SF with a NN use in shaping its evaluation makes it similar to Leela. I do remember watching the epic matches between Lc0 and SF and even Komodo... and how Lc0 would exploit some weaknesses in engines without a NN, but pretty much once SF started using NN, that covered weaknesses in its play that Lc0 would exploit. I actually paid a lot of money to get a 2080Ti GPU at the time just to see what all the fuss with Lc0 was, but haven't kept up with how things have evolved.

From what I can tell, although this QueenOdds engine (that is extremely impressive by the way) is Lc0 based, I don't see why SF can't have a NN that can be taught to do the same thing. Or am I missing something? Is SF today using only Alpha Beta search and no NN?

[lkaufman]

I thought you were knowledgeable about computer chess, based on past posts, so I am shocked that you don't know the difference between Stockfish and Leela. They are completely different engines, Stockfish is CPU based and does normal Alpha-beta search, while Leela is GPU based and uses a version of Monte-Carlo search. Your question asked about playing Stockfish, so that's what I answered. The Leela odds bots are trained specifically for odds chess, so they are vastly superior to Stockfish at doing this.
Rematch is being worked on, may be available soon. The problem is that normally rematch flips colors. It is true that with experience good players do learn and improve their results, but it's not really "systems", just learning to be careful, how to simplify, etc. The bot does get upgraded frequently and becomes more challenging each time, but of course there are limits, with enough time strong players can win with an extra queen. At the present time, the level needed to exceed 50% in blitz (5'0" or 3'2") with queen odds (Leela White) is strong FM level, maybe 2350 FIDE or so. At 1'1" it is good GM level, about 2550 FIDE. At 1'0" it is Hikaru Nakamura level, no one else has reached 40% (and he hasn't played enough to prove that he can score 50% or more).
[/quote]

I have to admit I haven't checked on chess engines for a few years. But every once in a while something comes up that makes me want to check up on chess engines again ... in this case it is the QueensOdd thing. I had thought that since new SF was using NN, that Leela and SF are basically morphed into similar engines now, and that somehow SF with a NN use in shaping its evaluation makes it similar to Leela. I do remember watching the epic matches between Lc0 and SF and even Komodo... and how Lc0 would exploit some weaknesses in engines without a NN, but pretty much once SF started using NN, that covered weaknesses in its play that Lc0 would exploit. I actually paid a lot of money to get a 2080Ti GPU at the time just to see what all the fuss with Lc0 was, but haven't kept up with how things have evolved.

From what I can tell, although this QueenOdds engine (that is extremely impressive by the way) is Lc0 based, I don't see why SF can't have a NN that can be taught to do the same thing. Or am I missing something? Is SF today using only Alpha Beta search and no NN?
[/quote]

Stockfish uses both traditional Alpha Beta search, and NNs for eval, using a technique called "NNUE" (Neural Net updated efficiently). Yes, Stockfish could train NNs specifically for odds play as Leela did, though NNs in SF are tiny compared to Leela (necessary if you do alpha-beta search). Whether Alpha-Beta would ever be as good as Monte Carlo for odds play is questionable. Certainly SF with nets trained for odds play would be vastly stronger than current Stockfish when giving those odds, but perhaps not close to Leela. But until someone tries this, we won't know. We do know that Komodo Dragon with a small percentage of odds training thrown in was much better at odds play than Komodo Dragon without the odds training.

[M ANSARI]

I have played that Queen Odds engine a few more times and did manage 2 wins at 10 minute time control. What is certain is that it is "trained" against human weaknesses ... humans in general have a good concept about material and how extra material will eventually win a game. However, humans are extremely weak at being able to comprehend the dynamics of a position and possible counterplay, when material is sacrificed. Sure they can do a good job in the opening where they have used engines to analyze the opening extensively, but when put in a situation on their own, with not much time to calculate all the possible consequences ... then you really need to be extremely strong with strong intuition to avoid the pitfalls. I found that you can play differently ... accept a loss of an exchange or even shed a pawn or two, if it means that you can get pieces off the board. Once that happens and the lots of pieces are exchanged, even a slight material advantage can be an easy win for the human. What is interesting is that this BOT will not accept these offers on many occasions, even if it means it is giving up some things positionally. It knows that just complicating things with more pieces off the board will work well against humans! Now that is pretty scary as I can see how this type of AI tuning can be used nefariously in many other different fields that have nothing to do with Chess!

[lkaufman]

I have played that Queen Odds engine a few more times and did manage 2 wins at 10 minute time control. What is certain is that it is "trained" against human weaknesses ... humans in general have a good concept about material and how extra material will eventually win a game. However, humans are extremely weak at being able to comprehend the dynamics of a position and possible counterplay, when material is sacrificed. Sure they can do a good job in the opening where they have used engines to analyze the opening extensively, but when put in a situation on their own, with not much time to calculate all the possible consequences ... then you really need to be extremely strong with strong intuition to avoid the pitfalls. I found that you can play differently ... accept a loss of an exchange or even shed a pawn or two, if it means that you can get pieces off the board. Once that happens and the lots of pieces are exchanged, even a slight material advantage can be an easy win for the human. What is interesting is that this BOT will not accept these offers on many occasions, even if it means it is giving up some things positionally. It knows that just complicating things with more pieces off the board will work well against humans! Now that is pretty scary as I can see how this type of AI tuning can be used nefariously in many other different fields that have nothing to do with Chess!

Yes, the bot is tuned to score as well as possible giving queen odds to bots that simulate strong human blitz play. It uses a "human node limit" to estimate what a human of the intended level will or will not see. The next step (in progress right now!) is to actually train it directly from the many games it has played already against actual humans rather than simulated ones! Whether this helps or not remains to be seen; the number of games played against humans strong enough to have chances may not yet be quite enough, we'll see. Yes, it could indeed be used nefariously in non-chess areas, but it is easier with chess since we have the data to train on. It's not so easy to get data on what swindles might work on humans (for example) in ordinary life. But eventually it will be a problem.

[Father]

I have played that Queen Odds engine a few more times and did manage 2 wins at 10 minute time control. What is certain is that it is "trained" against human weaknesses ... humans in general have a good concept about material and how extra material will eventually win a game. However, humans are extremely weak at being able to comprehend the dynamics of a position and possible counterplay, when material is sacrificed. Sure they can do a good job in the opening where they have used engines to analyze the opening extensively, but when put in a situation on their own, with not much time to calculate all the possible consequences ... then you really need to be extremely strong with strong intuition to avoid the pitfalls. I found that you can play differently ... accept a loss of an exchange or even shed a pawn or two, if it means that you can get pieces off the board. Once that happens and the lots of pieces are exchanged, even a slight material advantage can be an easy win for the human. What is interesting is that this BOT will not accept these offers on many occasions, even if it means it is giving up some things positionally. It knows that just complicating things with more pieces off the board will work well against humans! Now that is pretty scary as I can see how this type of AI tuning can be used nefariously in many other different fields that have nothing to do with Chess!

Yes, the bot is tuned to score as well as possible giving queen odds to bots that simulate strong human blitz play. It uses a "human node limit" to estimate what a human of the intended level will or will not see. The next step (in progress right now!) is to actually train it directly from the many games it has played already against actual humans rather than simulated ones! Whether this helps or not remains to be seen; the number of games played against humans strong enough to have chances may not yet be quite enough, we'll see. Yes, it could indeed be used nefariously in non-chess areas, but it is easier with chess since we have the data to train on. It's not so easy to get data on what swindles might work on humans (for example) in ordinary life. But eventually it will be a problem.

Mr. Larry Kaufman, good evening.
I have no gifts or achievements in "science"
or "expertise" with computers; I am guided only by my "natural knowledge and experience with computers."
Therefore, I wonder: What do you experts think would be the probable limit that "LeelaQueenOdds" could reach in Fide Elo at levels 1-0, 1+1, 2-0, 2+1, 3-0, 3+2, 5-0, 10-0, 10+5, etc.?
Is there really an insurmountable limit?
My own assessment is optimistic and promising. I believe the robot will reach an "extremely high" limit.
Time will tell... this machine reminds me of human beings who overcome situations of incapacity and achieve unimaginable goals. Truly, where there is will, there is power.

[lkaufman]

I have played that Queen Odds engine a few more times and did manage 2 wins at 10 minute time control. What is certain is that it is "trained" against human weaknesses ... humans in general have a good concept about material and how extra material will eventually win a game. However, humans are extremely weak at being able to comprehend the dynamics of a position and possible counterplay, when material is sacrificed. Sure they can do a good job in the opening where they have used engines to analyze the opening extensively, but when put in a situation on their own, with not much time to calculate all the possible consequences ... then you really need to be extremely strong with strong intuition to avoid the pitfalls. I found that you can play differently ... accept a loss of an exchange or even shed a pawn or two, if it means that you can get pieces off the board. Once that happens and the lots of pieces are exchanged, even a slight material advantage can be an easy win for the human. What is interesting is that this BOT will not accept these offers on many occasions, even if it means it is giving up some things positionally. It knows that just complicating things with more pieces off the board will work well against humans! Now that is pretty scary as I can see how this type of AI tuning can be used nefariously in many other different fields that have nothing to do with Chess!

Yes, the bot is tuned to score as well as possible giving queen odds to bots that simulate strong human blitz play. It uses a "human node limit" to estimate what a human of the intended level will or will not see. The next step (in progress right now!) is to actually train it directly from the many games it has played already against actual humans rather than simulated ones! Whether this helps or not remains to be seen; the number of games played against humans strong enough to have chances may not yet be quite enough, we'll see. Yes, it could indeed be used nefariously in non-chess areas, but it is easier with chess since we have the data to train on. It's not so easy to get data on what swindles might work on humans (for example) in ordinary life. But eventually it will be a problem.

Mr. Larry Kaufman, good evening.
I have no gifts or achievements in "science"
or "expertise" with computers; I am guided only by my "natural knowledge and experience with computers."
Therefore, I wonder: What do you experts think would be the probable limit that "LeelaQueenOdds" could reach in Fide Elo at levels 1-0, 1+1, 2-0, 2+1, 3-0, 3+2, 5-0, 10-0, 10+5, etc.?
Is there really an insurmountable limit?
My own assessment is optimistic and promising. I believe the robot will reach an "extremely high" limit.
Time will tell... this machine reminds me of human beings who overcome situations of incapacity and achieve unimaginable goals. Truly, where there is will, there is power.

I can say that it has already exceeded the limit of what I and most strong players and computer chess experts thought to be possible. In the 1800s, when odds chess was very popular, rook odds was generally considered to be the maximum handicap one could give to any experienced club player; queen odds was something the best players gave to novice players. I suppose they were generally playing at a tempo we would call "rapid", maybe like 15'+10" or so. Now it seems that the only player of the mid 1800s strong enough to beat LeelaQueenOdds in a Rapid match was probably Paul Morphy! Currently I think the bot (playing White) is about FIDE 2350 at 3'2" or 5'0", 2400 at 3'1" (Titled Tuesday time limit), 2450 at 3'0" or 2'1", 2550 at 1'1" or 2'0", and perhaps 2850 or so at 1'0". Not so much data on Rapid games with players of known reliable FIDE ratings, maybe 10'0" would be about 2200 and 10'5" about 2150 or so. How much can these numbers be raised? It's not obvious how that will happen, the current project might add 50 elo if we are lucky. There are some fairly obvious likely future improvements, but they are only likely to add a small number of elo points, things like making sure we win won games, and possibly modifying play based on opponent's remaining time, which would mostly just help no increment fast games. When the 5090 becomes available for the bot, it will allow more nodes or the same number in less time (reducing ponder time for the human), but that's also a small elo gain. I can't see how we will increase the numbers by more than 100, but someone may have another brilliant idea that pushes beyond that. It is hard for me to believe that the current bot should earn a solid IM rating (2400 FIDE, maybe 2700 chess.com blitz) if it were allowed to play in Titled Tuesday (3'1") on chess.com, such players don't consider themselves to be queen-odds level (!!), but that is what the results are telling us.

[Father]

I have played that Queen Odds engine a few more times and did manage 2 wins at 10 minute time control. What is certain is that it is "trained" against human weaknesses ... humans in general have a good concept about material and how extra material will eventually win a game. However, humans are extremely weak at being able to comprehend the dynamics of a position and possible counterplay, when material is sacrificed. Sure they can do a good job in the opening where they have used engines to analyze the opening extensively, but when put in a situation on their own, with not much time to calculate all the possible consequences ... then you really need to be extremely strong with strong intuition to avoid the pitfalls. I found that you can play differently ... accept a loss of an exchange or even shed a pawn or two, if it means that you can get pieces off the board. Once that happens and the lots of pieces are exchanged, even a slight material advantage can be an easy win for the human. What is interesting is that this BOT will not accept these offers on many occasions, even if it means it is giving up some things positionally. It knows that just complicating things with more pieces off the board will work well against humans! Now that is pretty scary as I can see how this type of AI tuning can be used nefariously in many other different fields that have nothing to do with Chess!

Yes, the bot is tuned to score as well as possible giving queen odds to bots that simulate strong human blitz play. It uses a "human node limit" to estimate what a human of the intended level will or will not see. The next step (in progress right now!) is to actually train it directly from the many games it has played already against actual humans rather than simulated ones! Whether this helps or not remains to be seen; the number of games played against humans strong enough to have chances may not yet be quite enough, we'll see. Yes, it could indeed be used nefariously in non-chess areas, but it is easier with chess since we have the data to train on. It's not so easy to get data on what swindles might work on humans (for example) in ordinary life. But eventually it will be a problem.

Mr. Larry Kaufman, good evening.
I have no gifts or achievements in "science"
or "expertise" with computers; I am guided only by my "natural knowledge and experience with computers."
Therefore, I wonder: What do you experts think would be the probable limit that "LeelaQueenOdds" could reach in Fide Elo at levels 1-0, 1+1, 2-0, 2+1, 3-0, 3+2, 5-0, 10-0, 10+5, etc.?
Is there really an insurmountable limit?
My own assessment is optimistic and promising. I believe the robot will reach an "extremely high" limit.
Time will tell... this machine reminds me of human beings who overcome situations of incapacity and achieve unimaginable goals. Truly, where there is will, there is power.

I can say that it has already exceeded the limit of what I and most strong players and computer chess experts thought to be possible. In the 1800s, when odds chess was very popular, rook odds was generally considered to be the maximum handicap one could give to any experienced club player; queen odds was something the best players gave to novice players. I suppose they were generally playing at a tempo we would call "rapid", maybe like 15'+10" or so. Now it seems that the only player of the mid 1800s strong enough to beat LeelaQueenOdds in a Rapid match was probably Paul Morphy! Currently I think the bot (playing White) is about FIDE 2350 at 3'2" or 5'0", 2400 at 3'1" (Titled Tuesday time limit), 2450 at 3'0" or 2'1", 2550 at 1'1" or 2'0", and perhaps 2850 or so at 1'0". Not so much data on Rapid games with players of known reliable FIDE ratings, maybe 10'0" would be about 2200 and 10'5" about 2150 or so. How much can these numbers be raised? It's not obvious how that will happen, the current project might add 50 elo if we are lucky. There are some fairly obvious likely future improvements, but they are only likely to add a small number of elo points, things like making sure we win won games, and possibly modifying play based on opponent's remaining time, which would mostly just help no increment fast games. When the 5090 becomes available for the bot, it will allow more nodes or the same number in less time (reducing ponder time for the human), but that's also a small elo gain. I can't see how we will increase the numbers by more than 100, but someone may have another brilliant idea that pushes beyond that. It is hard for me to believe that the current bot should earn a solid IM rating (2400 FIDE, maybe 2700 chess.com blitz) if it were allowed to play in Titled Tuesday (3'1") on chess.com, such players don't consider themselves to be queen-odds level (!!), but that is what the results are telling us.

Good evening, Mr. Larry Kaufman. Your historical account, current state of the art, and projection estimates perfectly connect the past, present, and future of Odds Chess. Since the 19th century with Paul Morphy, the conception and surprise with admiration at the advances of bots by scientists and strong players, and the "titled Tuesdays" of chess.com, and the prospect of "LeelaQueenOdds" participating in the Tuesday Open Titles. If I were allowed to bet on the bot, I would bet that the leelaqueenodds bot would outperform the top 50% of titled players. Chess.com should open the space for LeelaQueenOdds. I think chess would be doing a favor if it took the initiative, remembering the saying: "If the mountain won't go to Mohammed, Mohammed goes to the mountain." It means not to wait for things to happen, but to take the initiative.

[lkaufman]

Here is a summary analysis of all the results for the main odds bots since the Feb. 27 upgrade at the specific time control of 5'3". This is the longest one for which we have enough data to be fairly accurate, and is also at the top end of the blitz range, which is important because the Rapid ratings on LiChess are often based on few games or missing entirely. I only considered games with players rated above 2400 LiChess blitz (2600 for knight odds), 5 move minimum, and discarded games lost by disconnection. Minimum sample size 92 games. Here are the performance ratings calculated: LeelaQueenOdds + move (Leela Black) 2544. LeelaQueenOdds (Leela White 2663. LeelaRookOdds + move (Leela Black) 2960. LeelaRookOdds (Leela White) 3067. LeelaKnightOdds (Leela White) 3208. To convert to FIDE ratings, I now think a deduction of about 300 is appropriate at these levels; for titled players statistically 200 is more than enough, but they are a set of players known to excel in classical chess, so I think the 300 figure is more generally reliable. We should probably also round down to the nearest multiple of fifty to allow for the fact that not all of the players in these games were playing seriously, some moved too fast or resigned too soon. That would give LQO + move 2200 FIDE (Candidate Master standard), LQO 2350 (strong FM), LRO + move 2650 (strong GM), LRO 2750 (top ten level), LKO 2900 (well above Magnus). In every case, restricting the elo range to 2600 and above would have raised the numbers, so there is no reason to think that facing stronger opposition would lower the performance. The queen odds numbers (both colors) seem higher than expected relative to faster games, but the sample size was quite large for queen odds (665 games for White!). I also ran Queen for Knight (Leela White), with a 3060 result (nearly equal to rook odds), but the sample size was tiny (18 games) so I didn't include that above.

[Father]

Here is a summary analysis of all the results for the main odds bots since the Feb. 27 upgrade at the specific time control of 5'3". This is the longest one for which we have enough data to be fairly accurate, and is also at the top end of the blitz range, which is important because the Rapid ratings on LiChess are often based on few games or missing entirely. I only considered games with players rated above 2400 LiChess blitz (2600 for knight odds), 5 move minimum, and discarded games lost by disconnection. Minimum sample size 92 games. Here are the performance ratings calculated: LeelaQueenOdds + move (Leela Black) 2544. LeelaQueenOdds (Leela White 2663. LeelaRookOdds + move (Leela Black) 2960. LeelaRookOdds (Leela White) 3067. LeelaKnightOdds (Leela White) 3208. To convert to FIDE ratings, I now think a deduction of about 300 is appropriate at these levels; for titled players statistically 200 is more than enough, but they are a set of players known to excel in classical chess, so I think the 300 figure is more generally reliable. We should probably also round down to the nearest multiple of fifty to allow for the fact that not all of the players in these games were playing seriously, some moved too fast or resigned too soon. That would give LQO + move 2200 FIDE (Candidate Master standard), LQO 2350 (strong FM), LRO + move 2650 (strong GM), LRO 2750 (top ten level), LKO 2900 (well above Magnus). In every case, restricting the elo range to 2600 and above would have raised the numbers, so there is no reason to think that facing stronger opposition would lower the performance. The queen odds numbers (both colors) seem higher than expected relative to faster games, but the sample size was quite large for queen odds (665 games for White!). I also ran Queen for Knight (Leela White), with a 3060 result (nearly equal to rook odds), but the sample size was tiny (18 games) so I didn't include that above.

Congratulations on such great work. I believe that just as the current power of the bot represents a pleasant surprise, there's no reason to rule out the superpowered "LeelaQueenOdds bot" in the near future! Congratulations!