Question for Ed Shroder

[mwyoung]

Since Rebel played several matches with strong GMs at standard time controls around the turn of the century, performing a bit below 2700 I would estimate if we rate them all together, my question is this: What FIDE rating would you expect "Benjamin" to earn in similar matches now, running on a typical i7 (say the reference hardware for CCRL), given that Benjamin is an improved version of those older engines and the hardware is much better now (even if perhaps some of those old Rebels ran on 2 or even 4 threads for those matches, I don't have that info)? How much better is Benjamin, and how much better is the hardware (allowing for Benjamin using only one thread)? This would help determine the human equivalent (at 40 moves in two hours) of the CCRL ratings of engines in the ballpark of 2700. You would probably be the best person to make this judgment.

Here is Ed's reply. Ed will not post here. As many of us are transitioning to a better forum.

Since I no longer post there but want to answer Larry anyway - Rebel in 2003 (IIRC on an AMD 450 MHz PC) played 4 games at tournament time control (40/2h) against GM Loek van Wely then rated 2700+ and won 2, lost 2, a tie. I remember the games very well, all 4 were decided notable on attacking the King, the specialty of the human, that was a new experience for me.

Also the first tournament game against Anand in 1998 Rebel had a winning position attacking the King but failed to cash a full point missing a simple continuation due to the lack of depth, something Benjamin sees instantly. Instead Rebel decided to enter the endgame (with material advantage) too early which Anand brilliantly defended into a draw.

It's hard to make a prediction, but for an educated guess, I doubt GM's rated 2750 will have a chance in a 6 game match, and I wouldn't be surprised if Benjamin would win a 6 game match against 2800 players.

If you have any more questions Larry. Ed has a thread started here.

OK, so Ed is confirming my belief that to estimate FIDE ratings at 40 moves in 2 hours, the CCRL 40/15 ratings of engines rated around 2700 should be increased by 100 elo at least. Since engines perform at least a hundred elo better in Rapid than in Classical vs. humans, we should add 200 to the CCRL 40/15 ratings to estimate performance at 15' + 10" vs. the top human players. So for Nakamura, Rapid 2829, we want a CCRL rating around 2629. I'll try to determine what nps setting for Lc0 j92-330 would get about that rating.

https://prodeo.actieforum.com/t60-larry

Thanks. And yes I am also sure now it will be less then 2 nps.

One has to specify the MinibatchSize when talking of (low) nodes per move with Lc0. For nodes per move in dozens, the strongest setting is MinibatchSize = 1.

So, with the default MinibatchSize = 256, I am still getting nps = 2.0 both from my older calibration and the new Nakamura adjusted for distraction result (I take it to be the predicted at 50 nodes per move +3 -0 = 5 for Dragon instead of "distracted" +5 -0 = 3)

With MinibatchSize = 1, this nps = 1.2.

I had the same issue. But I just changed to the CPU version. Doing the same thing.

I didn't know about this issue, but since I'm using the CPU version for this I suppose this means we're pretty much in agreement. So far 1.5 nps is coming out pretty close to the 2629 target value, I'll post when I have a final number for it.

Not a big deal. The RTX GPU version of Lc0 had a hard time throttling the speed down that low. The CPU version fixed this issue nicely, since it runs much slower already.

Thanks for the update!

[lkaufman]

Since Rebel played several matches with strong GMs at standard time controls around the turn of the century, performing a bit below 2700 I would estimate if we rate them all together, my question is this: What FIDE rating would you expect "Benjamin" to earn in similar matches now, running on a typical i7 (say the reference hardware for CCRL), given that Benjamin is an improved version of those older engines and the hardware is much better now (even if perhaps some of those old Rebels ran on 2 or even 4 threads for those matches, I don't have that info)? How much better is Benjamin, and how much better is the hardware (allowing for Benjamin using only one thread)? This would help determine the human equivalent (at 40 moves in two hours) of the CCRL ratings of engines in the ballpark of 2700. You would probably be the best person to make this judgment.

Here is Ed's reply. Ed will not post here. As many of us are transitioning to a better forum.

Since I no longer post there but want to answer Larry anyway - Rebel in 2003 (IIRC on an AMD 450 MHz PC) played 4 games at tournament time control (40/2h) against GM Loek van Wely then rated 2700+ and won 2, lost 2, a tie. I remember the games very well, all 4 were decided notable on attacking the King, the specialty of the human, that was a new experience for me.

Also the first tournament game against Anand in 1998 Rebel had a winning position attacking the King but failed to cash a full point missing a simple continuation due to the lack of depth, something Benjamin sees instantly. Instead Rebel decided to enter the endgame (with material advantage) too early which Anand brilliantly defended into a draw.

It's hard to make a prediction, but for an educated guess, I doubt GM's rated 2750 will have a chance in a 6 game match, and I wouldn't be surprised if Benjamin would win a 6 game match against 2800 players.

If you have any more questions Larry. Ed has a thread started here.

OK, so Ed is confirming my belief that to estimate FIDE ratings at 40 moves in 2 hours, the CCRL 40/15 ratings of engines rated around 2700 should be increased by 100 elo at least. Since engines perform at least a hundred elo better in Rapid than in Classical vs. humans, we should add 200 to the CCRL 40/15 ratings to estimate performance at 15' + 10" vs. the top human players. So for Nakamura, Rapid 2829, we want a CCRL rating around 2629. I'll try to determine what nps setting for Lc0 j92-330 would get about that rating.

https://prodeo.actieforum.com/t60-larry

Thanks. And yes I am also sure now it will be less then 2 nps.

One has to specify the MinibatchSize when talking of (low) nodes per move with Lc0. For nodes per move in dozens, the strongest setting is MinibatchSize = 1.

So, with the default MinibatchSize = 256, I am still getting nps = 2.0 both from my older calibration and the new Nakamura adjusted for distraction result (I take it to be the predicted at 50 nodes per move +3 -0 = 5 for Dragon instead of "distracted" +5 -0 = 3)

With MinibatchSize = 1, this nps = 1.2.

I had the same issue. But I just changed to the CPU version. Doing the same thing.

I didn't know about this issue, but since I'm using the CPU version for this I suppose this means we're pretty much in agreement. So far 1.5 nps is coming out pretty close to the 2629 target value, I'll post when I have a final number for it.

Not a big deal. The RTX GPU version of Lc0 had a hard time throttling the speed down that low. The CPU version fixed this issue nicely, since it runs much slower already.

I got a 2618 CCRL 40/15 "rating" for Lc0 j92-330 cpu set to 1.5 nps limit after 758 games at 15' + 10". I would estimate that increasing that to 1.6 nps would make it as close to the 2629 target rating as I could ask for, so I'll use 1.6 nps as the simulation for Nakamura in Rapid. It has the nice property that I can divide by 2 repeatedly to get elo values for lower nps numbers; I'd like an estimated nps for 2500, the minimum initial rating for GM (with rare exceptions like me!).

Thanks for the update!

[mwyoung]

Since Rebel played several matches with strong GMs at standard time controls around the turn of the century, performing a bit below 2700 I would estimate if we rate them all together, my question is this: What FIDE rating would you expect "Benjamin" to earn in similar matches now, running on a typical i7 (say the reference hardware for CCRL), given that Benjamin is an improved version of those older engines and the hardware is much better now (even if perhaps some of those old Rebels ran on 2 or even 4 threads for those matches, I don't have that info)? How much better is Benjamin, and how much better is the hardware (allowing for Benjamin using only one thread)? This would help determine the human equivalent (at 40 moves in two hours) of the CCRL ratings of engines in the ballpark of 2700. You would probably be the best person to make this judgment.

Here is Ed's reply. Ed will not post here. As many of us are transitioning to a better forum.

Since I no longer post there but want to answer Larry anyway - Rebel in 2003 (IIRC on an AMD 450 MHz PC) played 4 games at tournament time control (40/2h) against GM Loek van Wely then rated 2700+ and won 2, lost 2, a tie. I remember the games very well, all 4 were decided notable on attacking the King, the specialty of the human, that was a new experience for me.

Also the first tournament game against Anand in 1998 Rebel had a winning position attacking the King but failed to cash a full point missing a simple continuation due to the lack of depth, something Benjamin sees instantly. Instead Rebel decided to enter the endgame (with material advantage) too early which Anand brilliantly defended into a draw.

It's hard to make a prediction, but for an educated guess, I doubt GM's rated 2750 will have a chance in a 6 game match, and I wouldn't be surprised if Benjamin would win a 6 game match against 2800 players.

If you have any more questions Larry. Ed has a thread started here.

OK, so Ed is confirming my belief that to estimate FIDE ratings at 40 moves in 2 hours, the CCRL 40/15 ratings of engines rated around 2700 should be increased by 100 elo at least. Since engines perform at least a hundred elo better in Rapid than in Classical vs. humans, we should add 200 to the CCRL 40/15 ratings to estimate performance at 15' + 10" vs. the top human players. So for Nakamura, Rapid 2829, we want a CCRL rating around 2629. I'll try to determine what nps setting for Lc0 j92-330 would get about that rating.

https://prodeo.actieforum.com/t60-larry

Thanks. And yes I am also sure now it will be less then 2 nps.

One has to specify the MinibatchSize when talking of (low) nodes per move with Lc0. For nodes per move in dozens, the strongest setting is MinibatchSize = 1.

So, with the default MinibatchSize = 256, I am still getting nps = 2.0 both from my older calibration and the new Nakamura adjusted for distraction result (I take it to be the predicted at 50 nodes per move +3 -0 = 5 for Dragon instead of "distracted" +5 -0 = 3)

With MinibatchSize = 1, this nps = 1.2.

I had the same issue. But I just changed to the CPU version. Doing the same thing.

I didn't know about this issue, but since I'm using the CPU version for this I suppose this means we're pretty much in agreement. So far 1.5 nps is coming out pretty close to the 2629 target value, I'll post when I have a final number for it.

Not a big deal. The RTX GPU version of Lc0 had a hard time throttling the speed down that low. The CPU version fixed this issue nicely, since it runs much slower already.

I got a 2618 CCRL 40/15 "rating" for Lc0 j92-330 cpu set to 1.5 nps limit after 758 games at 15' + 10". I would estimate that increasing that to 1.6 nps would make it as close to the 2629 target rating as I could ask for, so I'll use 1.6 nps as the simulation for Nakamura in Rapid. It has the nice property that I can divide by 2 repeatedly to get elo values for lower nps numbers; I'd like an estimated nps for 2500, the minimum initial rating for GM (with rare exceptions like me!).

Thanks for the update!

Larry won the 2008 Fide World Senior Championship. Doing so earned Larry the title of Grandmaster. That is the meaning of rare exception.