Dragon versus Nakamura

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Marcus9
Posts: 85
Joined: Mon May 13, 2019 6:05 pm
Full name: Marco Giorgio

Re: Dragon versus Nakamura

Post by Marcus9 »

lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
Have you found a way on how to simulate human play? Were you able to answer some questions you asked yourself about lc0 j92-330? It would be great if we could agree on the same engine to use to simulate the human and share the results together here
Cornfed
Posts: 511
Joined: Sun Apr 26, 2020 11:40 pm
Full name: Brian D. Smith

Re: Dragon versus Nakamura

Post by Cornfed »

jp wrote: Wed Nov 25, 2020 5:26 am
Cornfed wrote: Tue Nov 24, 2020 2:50 pm
jp wrote: Tue Nov 24, 2020 8:48 am For handicap matches, the handicap-giver is forced to bluff, but in prep/analysis you are not looking to bluff. e.g. for White opening preparation you are looking for awkward lines for the unbooked opponent among all the +/= lines.
Actually, the point above I put in 'red' just now is not always the case when it comes to prep/analysis. Lets say you are playing a truly speculative opening/defense - The Scandanvian Portuguese variation for example, you do not always want to play (as black) the 'best computer moves' simply because there are lines for white that are easy to find and keep/increase his edge with ease.

What you want to be able to do in that case if find...2nd or 3rd best moves which are not too far off the reservation, but offer white MORE chances to go wrong (not talking just outright blunders here) in the rough and tumble of actual play...particularly OTB, especially at the increasingly common quicker time controls.
It's a matter of what we mean by "bluff". If you choose e.g. a really unsound gambit opening, you could say you've already bluffed by playing that opening. Then, further down the opening, choosing a variation within it that has comp eval e.g. -0.8 instead of a line with comp eval -0.65 is not what I'd call bluffing. (It seems this is your "not too far off the reservation" move.)

The other type of bluffing is not really knowing the eval or lines and just playing/evaluating moves semi-blind. That's not the type of bluffing I'd want the computer to give me, because I'd want to know what sort of risk playing that line involves.

I'll even go this far...which I think fits your definition of a 'bluff' straight from move 1: For kicks, In blitz, I often play as black 1.d4 e5 which usually goes 2.de5 Bc5 when very common is 3.Nf3 f6, 4. ef6 Nf6.

Now, of course this is bad....but takes everyone out of their comfort zone and I even have one Canadian IM's scalp with it...directly related to the opening.

In such lines having an engine show you in some of the sidelines (which are still objectively 'bad') where choices for you as Black simply give White more chances to make second rate moves/go in the wrong direction ...giving you real chances for compensation, well that would be valuable for preparation at this time control. Granted...I'm taking an extreme example with a poor defense/gambit, but in blitz lots of things become borderline playable against the unsuspecting.
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Dragon versus Nakamura

Post by lkaufman »

Marcus9 wrote: Wed Nov 25, 2020 10:36 am
lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
Have you found a way on how to simulate human play? Were you able to answer some questions you asked yourself about lc0 j92-330? It would be great if we could agree on the same engine to use to simulate the human and share the results together here
I think that using Lc0 with a net like j92-330 and a max nps rate, as suggested in this forum, is indeed a good way to simulate top human play, but I think that the proper nps rate was overestimated. Using Lc0 this way has a very nice property; it is well known that the performance of humans against standard A/B engines drops significantly as the time limit shrinks, but the same is also true of Lc0 with a tiny max NPS setting, so it looks quite likely that if the NPS rate is set properly to simulate a given human at one time control, it may also be reasonably accurate for a significantly faster or slower time control. What NPS max best simulates a top GM (Carlsen, Nakamura, MVL, So) in Rapid play (and perhaps blitz and standard as well)? Incredible as it may seem, the answer appears to be not much higher than 1! I'm still gathering data to refine this estimate, but I'm reasonably sure now that no human can compete on even terms with a max NPS speed of 2!
Komodo rules!
mwyoung
Posts: 2727
Joined: Wed May 12, 2010 10:00 pm

Re: Dragon versus Nakamura

Post by mwyoung »

lkaufman wrote: Wed Nov 25, 2020 6:21 pm
Marcus9 wrote: Wed Nov 25, 2020 10:36 am
lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
Have you found a way on how to simulate human play? Were you able to answer some questions you asked yourself about lc0 j92-330? It would be great if we could agree on the same engine to use to simulate the human and share the results together here
I think that using Lc0 with a net like j92-330 and a max nps rate, as suggested in this forum, is indeed a good way to simulate top human play, but I think that the proper nps rate was overestimated. Using Lc0 this way has a very nice property; it is well known that the performance of humans against standard A/B engines drops significantly as the time limit shrinks, but the same is also true of Lc0 with a tiny max NPS setting, so it looks quite likely that if the NPS rate is set properly to simulate a given human at one time control, it may also be reasonably accurate for a significantly faster or slower time control. What NPS max best simulates a top GM (Carlsen, Nakamura, MVL, So) in Rapid play (and perhaps blitz and standard as well)? Incredible as it may seem, the answer appears to be not much higher than 1! I'm still gathering data to refine this estimate, but I'm reasonably sure now that no human can compete on even terms with a max NPS speed of 2!
You are correct I was using 2.5 with Naka's Blitz rating of 2900, and 2.3 for his rapid rating. And that was too much. But we will get better at dialing in the correct number.

And I may have used too smart of a NN. It maybe better to use a smaller NN. Still a work in progress.
Last edited by mwyoung on Wed Nov 25, 2020 6:36 pm, edited 1 time in total.
"The worst thing that can happen to a forum is a running wild attacking moderator(HGM) who is not corrected by the community." - Ed Schröder
But my words like silent raindrops fell. And echoed in the wells of silence.
dkappe
Posts: 1631
Joined: Tue Aug 21, 2018 7:52 pm
Full name: Dietrich Kappe

Re: Dragon versus Nakamura

Post by dkappe »

I’d suggest using a smaller net. You can turn the knob a lot more in terms of nodes and nps at the bottom end. For low level play I even use 16x2 nets that at 4500 nodes will get you around 2200 blitz.
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Dragon versus Nakamura

Post by Laskos »

lkaufman wrote: Wed Nov 25, 2020 6:21 pm
Marcus9 wrote: Wed Nov 25, 2020 10:36 am
lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
Have you found a way on how to simulate human play? Were you able to answer some questions you asked yourself about lc0 j92-330? It would be great if we could agree on the same engine to use to simulate the human and share the results together here
I think that using Lc0 with a net like j92-330 and a max nps rate, as suggested in this forum, is indeed a good way to simulate top human play, but I think that the proper nps rate was overestimated. Using Lc0 this way has a very nice property; it is well known that the performance of humans against standard A/B engines drops significantly as the time limit shrinks, but the same is also true of Lc0 with a tiny max NPS setting, so it looks quite likely that if the NPS rate is set properly to simulate a given human at one time control, it may also be reasonably accurate for a significantly faster or slower time control. What NPS max best simulates a top GM (Carlsen, Nakamura, MVL, So) in Rapid play (and perhaps blitz and standard as well)? Incredible as it may seem, the answer appears to be not much higher than 1! I'm still gathering data to refine this estimate, but I'm reasonably sure now that no human can compete on even terms with a max NPS speed of 2!
I estimated 30 - 100 nodes per move at 15m + 10s TC the range of Nakamura's strength, and for 50 nodes per move I got +3 -0 =5 score for Dragon, which is not much off. I consider that Naka would have gotten a score in this ballpark, had he had more composure and was less distracted. Also, had he prepared better the openings. In terms of nodes per second, 2 nodes per second should match pretty well Naka with this net at a variety of TC. I have chosen Lc0 also because its scaling with TC is more similar to humans than the scaling of AB engines and yes, you are right, we can use the constant NPS speed to simulate Nakamura at variety of TCs. But I do use max NPS Lc0 to actually simulate because it's much faster to simulate, one side (simulated human) plays practically instantly. Also, using some 50 or 70 nodes per move for Nakamura and 10 nodes per move for a much weaker GM comes handily to simulate a variety of GMs, while with nodes per second = 1 one can use only 1, 2, and 3 as values for a human (GM).

Btw, for some reason I am also getting that 2 pawns + move is playable and fair against Nakamura for Dragon MCTS (enabled with a small book), and that Knight odds at 15 + 10 are playable against ~2400-2450 GMs (although Dragon should lose in this case) if the prepared human openings are neutralized (Dragon to have a good small book).
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Dragon versus Nakamura

Post by lkaufman »

Laskos wrote: Wed Nov 25, 2020 7:13 pm
lkaufman wrote: Wed Nov 25, 2020 6:21 pm
Marcus9 wrote: Wed Nov 25, 2020 10:36 am
lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
Have you found a way on how to simulate human play? Were you able to answer some questions you asked yourself about lc0 j92-330? It would be great if we could agree on the same engine to use to simulate the human and share the results together here
I think that using Lc0 with a net like j92-330 and a max nps rate, as suggested in this forum, is indeed a good way to simulate top human play, but I think that the proper nps rate was overestimated. Using Lc0 this way has a very nice property; it is well known that the performance of humans against standard A/B engines drops significantly as the time limit shrinks, but the same is also true of Lc0 with a tiny max NPS setting, so it looks quite likely that if the NPS rate is set properly to simulate a given human at one time control, it may also be reasonably accurate for a significantly faster or slower time control. What NPS max best simulates a top GM (Carlsen, Nakamura, MVL, So) in Rapid play (and perhaps blitz and standard as well)? Incredible as it may seem, the answer appears to be not much higher than 1! I'm still gathering data to refine this estimate, but I'm reasonably sure now that no human can compete on even terms with a max NPS speed of 2!
I estimated 30 - 100 nodes per move at 15m + 10s TC the range of Nakamura's strength, and for 50 nodes per move I got +3 -0 =5 score for Dragon, which is not much off. I consider that Naka would have gotten a score in this ballpark, had he had more composure and was less distracted. Also, had he prepared better the openings. In terms of nodes per second, 2 nodes per second should match pretty well Naka with this net at a variety of TC. I have chosen Lc0 also because its scaling with TC is more similar to humans than the scaling of AB engines and yes, you are right, we can use the constant NPS speed to simulate Nakamura at variety of TCs. But I do use max NPS Lc0 to actually simulate because it's much faster to simulate, one side (simulated human) plays practically instantly. Also, using some 50 or 70 nodes per move for Nakamura and 10 nodes per move for a much weaker GM comes handily to simulate a variety of GMs, while with nodes per second = 1 one can use only 1, 2, and 3 as values for a human (GM).

Btw, for some reason I am also getting that 2 pawns + move is playable and fair against Nakamura for Dragon MCTS (enabled with a small book), and that Knight odds at 15 + 10 are playable against ~2400-2450 GMs (although Dragon should lose in this case) if the prepared human openings are neutralized (Dragon to have a good small book).
I'm using Ed Schroder's "Benjamin" program as a benchmark to compare Lc0 at low NPS. Benjamin is rated about 2700 CCRL 40/15, and is a logical modern engine to use as it is an improved version of one that played a match with Anand back in 1998 (Ed, if you read this, how much stronger is Benjamin than that 1998 Rebel?). Running on a modern i7 (the reference engine for CCRL), I would expect it to play close to 2800 FIDE at standard time limit, and to be slightly stronger than Nakamura or even Carlsen at Rapid. According to my tests, Lc0 with the j92-330 net needs about 1.6 NPS to break even with "Benjamin" at 15' + 10". So I would estimate that a setting somewhere in the 1 to 1.5 range would best simulate Nakamura in Rapid, and possibly also at other time controls.
Some questions: 1. Why do you say nps values have to be integers, it seems to accept decimal values with no problem?
2. I don't see a place to set nodes per move, only nodes per second, in the 26.3 Lc0 gui. How do you do that?
3. Are you able to give any comparative results for Dragon in standard mode compared to Dragon MCTS in giving handicaps to simulated human?
Komodo rules!
User avatar
Laskos
Posts: 10948
Joined: Wed Jul 26, 2006 10:21 pm
Full name: Kai Laskos

Re: Dragon versus Nakamura

Post by Laskos »

lkaufman wrote: Wed Nov 25, 2020 8:55 pm
Laskos wrote: Wed Nov 25, 2020 7:13 pm
I estimated 30 - 100 nodes per move at 15m + 10s TC the range of Nakamura's strength, and for 50 nodes per move I got +3 -0 =5 score for Dragon, which is not much off. I consider that Naka would have gotten a score in this ballpark, had he had more composure and was less distracted. Also, had he prepared better the openings. In terms of nodes per second, 2 nodes per second should match pretty well Naka with this net at a variety of TC. I have chosen Lc0 also because its scaling with TC is more similar to humans than the scaling of AB engines and yes, you are right, we can use the constant NPS speed to simulate Nakamura at variety of TCs. But I do use max NPS Lc0 to actually simulate because it's much faster to simulate, one side (simulated human) plays practically instantly. Also, using some 50 or 70 nodes per move for Nakamura and 10 nodes per move for a much weaker GM comes handily to simulate a variety of GMs, while with nodes per second = 1 one can use only 1, 2, and 3 as values for a human (GM).

Btw, for some reason I am also getting that 2 pawns + move is playable and fair against Nakamura for Dragon MCTS (enabled with a small book), and that Knight odds at 15 + 10 are playable against ~2400-2450 GMs (although Dragon should lose in this case) if the prepared human openings are neutralized (Dragon to have a good small book).
I'm using Ed Schroder's "Benjamin" program as a benchmark to compare Lc0 at low NPS. Benjamin is rated about 2700 CCRL 40/15, and is a logical modern engine to use as it is an improved version of one that played a match with Anand back in 1998 (Ed, if you read this, how much stronger is Benjamin than that 1998 Rebel?). Running on a modern i7 (the reference engine for CCRL), I would expect it to play close to 2800 FIDE at standard time limit, and to be slightly stronger than Nakamura or even Carlsen at Rapid. According to my tests, Lc0 with the j92-330 net needs about 1.6 NPS to break even with "Benjamin" at 15' + 10". So I would estimate that a setting somewhere in the 1 to 1.5 range would best simulate Nakamura in Rapid, and possibly also at other time controls.
Some questions: 1. Why do you say nps values have to be integers, it seems to accept decimal values with no problem?
2. I don't see a place to set nodes per move, only nodes per second, in the 26.3 Lc0 gui. How do you do that?
3. Are you able to give any comparative results for Dragon in standard mode compared to Dragon MCTS in giving handicaps to simulated human?

1. I was probably wrong and one can use decimal and centesimal values like 1.62 nps. I don't use nodes per second as this slows down my experiment by a factor of 2 compared to specifying the nodes per move (and instant move).

2. I don't know what Lc0 GUI is. Something like Nibbler? In Cutechess - Cli or GUI one can specify the number of nodes per move used (Cutechess sends "go nodes" command).

3. I seem to get no better results with Dragon MCTS than with standard Dragon against simulated human as Lc0, unlike of what apparently happened against Nakamura. I am not sure why, maybe MCTS is not that much stronger against Naka as it seemed?

It is very interesting that you came up with 1.6 nps starting with Benjamin plausible human rating, Mark came with 2.3 nps with some extrapolations, and I came with 50 nodes per move or about 2.0 nps starting with our older elaborate estimation of a 1 million nodes of Komodo 10 matching a 2800 GM at 45 + 15. All estimates and extrapolations come amazingly close, not even a factor of 2 off, while unknowns like scaling issues and Elo behavior are very serious.
Chessqueen
Posts: 5590
Joined: Wed Sep 05, 2018 2:16 am
Location: Moving
Full name: Jorge Picado

Re: Dragon versus Nakamura

Post by Chessqueen »

lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
I have been watching the Skilling Open and Noticed that in most 4 or 5 men endgames these super GMs do NOT make the best moves according to engines that are analyzing it, including Carlsen has made several inaccurate moves, so my question here is why do Engines need 6 and 7 men Endgames when playing versus any Super GM, especially in Rapid or Blitz games like that match Dragon MCTS Vs Nakamura?

NOTE: What would be a fair time control for Nakamura 20 minutes and 10 sec increment or 30 Minutes and 10 sec increment with the two pawn Odds Vs Dragon MCTS ?
Who is 17 years old GM Gukesh 2nd at the Candidate in Toronto?
https://indianexpress.com/article/sport ... t-9281394/
lkaufman
Posts: 5960
Joined: Sun Jan 10, 2010 6:15 am
Location: Maryland USA

Re: Dragon versus Nakamura

Post by lkaufman »

Chessqueen wrote: Wed Nov 25, 2020 10:17 pm
lkaufman wrote: Wed Nov 25, 2020 7:22 am
Chessqueen wrote: Wed Nov 25, 2020 5:45 am
Chessqueen wrote: Mon Nov 23, 2020 4:49 am
lkaufman wrote: Sun Nov 22, 2020 6:38 pm
Nordlandia wrote: Sun Nov 22, 2020 3:39 pm One way to justify more handicap is by playing FRC. In Fischer Random,the human is out of the comfort zone and has to extemporize from scratch. Based on my testing, Dragon is nearly on par Stockfish in FRC. So Dragon is not necessarily weaker in Fischer Random.
Yes, CCRL has already confirmed our own findings that Dragon is stronger than SF12 in FRC. We did indeed demonstrate that it is possible even for Komodo MCTS (pre-Dragon) to give knight odds in FRC Rapid to a fairly strong GM (Alex Lenderman), even though in normal chess Komodo needed some compensation for the knight (a pawn or castling rights). Perhaps Dragon MCTS could now give knight odds in FRC Rapid to any human with decent chances. The ideal opponent would be Wesley So as he is still the reigning FRC World Champion.
What a Great Idea! GM SO is also playing LIVE Now in the Skilling Open, can you imagine all the headlines around the World Komodo Beat the Current FRC Champion giving him a Knight Odds ! But you will have to find a good sponsor, since GM SO is NOT going to play cheap to risk being embarrassed like this.
If any of the these previous cham[ions would be alive, Capablanca, Alekhine, or Fischer which of these would do better against Komodo MCTS with the same 2 pawns Odds that Nakamura lost 4 straight games the second day they played?
Well, the obvious answer would be Capablanca, since he was known for technique and almost never losing, while Alekhine took lots of risks. But the level of play advanced over time, so Fischer would be my pick, and probably Carlsen today would be the best of all time at playing odds vs. engines. But I think that the time limit is too fast for any human to win a match from Dragon at the two White pawn handicap used here. I've learned a lot from this match and from the comments about how to simulate the play of top human players in situations like this; in my opinion the simulations overestimated the strength of Nakamura (or any human) at this time limit. I think this was the bigger factor in the result rather than the MCTS choice; in most critical positions regular Dragon would choose the same move. Humans just don't play very well with short time.
I have been watching the Skilling Open and Noticed that in most 4 or 5 men endgames these super GMs do NOT make the best moves according to engines that are analyzing it, including Carlsen has made several inaccurate moves, so my question here is why do Engines need 6 and 7 men Endgames when playing versus any Super GM, especially in Rapid or Blitz games like that match Dragon MCTS Vs Nakamura?

NOTE: What would be a fair time control for Nakamura 20 minutes and 10 sec increment or 30 Minutes and 10 sec increment with the two pawn Odds Vs Dragon MCTS ?
I would say the Engines don't "need" 6 man TBs (we don't use 7) vs. humans, and for handicap play we might do better not to use them. Playing the best practical choice makes more sense vs. human than playing "perfectly". I suppose it is a slight advantage to know whether some 5 or 6 man endgame is won or drawn before reaching it, but even here it might be better vs. human to go by the eval.
Probably Naka would need 30 min plus 10 sec to have good chances at the two pawn handicap. But as Kai indicates he should do fine with 15' + 10" if he also gets White, in my tests that's just a huge swing. Especially c7 + f7, that's just a huge handicap.
Komodo rules!