Benjamin 1.1 for the GRL

[Rebel]

Started Benjamin 1.1 for the GRL.

Games : 1000

Elo pool : 2752

http://rebel13.nl/b/grl.htm

Curious how much elo remains from the +100 against version 1.0

[Rebel]

Benjamin 1.1 +53

Details - https://prodeo.actieforum.com/t358p50-g ... sults#4814

[amanjpro]

Congrats Ed, happy to see a new version of Benjamin

[lkaufman]

Benjamin 1.1 +53

Details - https://prodeo.actieforum.com/t358p50-g ... sults#4814

This is interesting; I've generally found that results in direct matches against older versions hold up pretty well against unrelated engines, maybe with just a small dropoff, but you are finding the gain to be nearly cut in half. I wonder why. Perhaps it is somehow different with 3700 level engines than with 2700 level engines, but it's not obvious why that should be so. What has been the experience of others on this question?

[Rebel]

Benjamin 1.1 +53

Details - https://prodeo.actieforum.com/t358p50-g ... sults#4814

This is interesting; I've generally found that results in direct matches against older versions hold up pretty well against unrelated engines, maybe with just a small dropoff, but you are finding the gain to be nearly cut in half. I wonder why. Perhaps it is somehow different with 3700 level engines than with 2700 level engines, but it's not obvious why that should be so. What has been the experience of others on this question?

In the early 80's Frans Morsch and I cooperated a lot inventing all the wheels ourselves and at some moment Frans said - with you everything is different. I took it as a compliment and it's also an answer to your question

Kidding aside, I more or less expected something like this, as announced in the OP. After all Benjamin = ProDeo with some code changes (King Attack mainly) plus more aggressive tuning. I ran a provisional rating list and we see Benjamin 1.1 rated higher than ProDeo 3.1 which is impossible IMO playing normal openings, normal rating lists.

Code: Select all

 125 Benjamin 1.1           :  2796.3   19.9   560.5    1000    56      65   435   251   314    25
 126 Nalwald 1.14           :  2792.1   12.1   426.0    1000    43      77   305   242   453    24
 127 GreKo 2021_08          :  2783.7   21.4   617.5    1200    51      57   466   303   431    25
 128 ProDeo 3.1             :  2782.1    9.8   836.0    1800    46      55   610   452   738    25

I don't have a comp free at the moment but I will test ProDeo 3.1 vs Benjamin 1.1 with normal openings for the sake of knowing.

[lkaufman]

Started Benjamin 1.1 for the GRL.

Games : 1000

Elo pool : 2752

http://rebel13.nl/b/grl.htm

Curious how much elo remains from the +100 against version 1.0

I am interested in the engines of around the year 2000 that were performing at roughly 2800 level against humans (in Rapid even before 2000, in Classical shortly after) as proxies for human top-ten players. Naturally your programs are prime candidates. I know that Rebel (10 I believe) played a match with Anand in 1998 where it scored evenly in the four classical and Rapid games combined (ignoring the 3 to 1 blitz victory for Rebel). So a few related questions:
1. What was the hardware in that match (how many cores, what speed?)
2. Is the RebelCentury UCI engine pretty close to the one that played Anand? If it is improved, how much?
3. I see a RebelCentury 3 and 4 in the SSDF list. Which one (if either) is the UCI version?
4. How different is Benjamin 1.1 from RebelCentury UCI, aside from the gambit style? I can look at rating lists, I'm just asking if it is substantially different or just modestly, and if the search is substantially modernized or fairly similar?
5. Any other data that would suggest how RebelCentury UCI would perform against top human players on a modern i7, either at Classical time limit or Rapid?
I realize it's been more than two decades since RebelCentury was written and competed, so I'll certainly understand if you can't remember details.

[Rebel]

The 4 matches I played :

1997 - Rebel9 - Arthur Yusopov, on that moment rated 10th on the FIDE rating list
1998 - Rebel10 - Vishy Anand, second place FIDE rating list
2001 - Rebel Century - GM John van der Wiel who never lost a game to a computer.
2002 - Rebel Century 4 - GM Loek van Wely, at the time rated 2700+

The 2001 and 2002 matches the time control was 40 moves in 2 hours, shorter time controls were no longer an issue.

Regarding your questions -

1. What was the hardware in that match (how many cores, what speed?)
2. Is the RebelCentury UCI engine pretty close to the one that played Anand? If it is improved, how much?
3. I see a RebelCentury 3 and 4 in the SSDF list. Which one (if either) is the UCI version?

All 4 programs were DOS, hardware, NPS, TC etc. is listed in the 4 links.

4. How different is Benjamin 1.1 from RebelCentury UCI, aside from the gambit style? I can look at rating lists, I'm just asking if it is substantially different or just modestly, and if the search is substantially modernized or fairly similar?

After the 2002 match I retired and lost interest, only so now and then did something minor. I became interested in comp-chess again after the crazy ICGA verdict in 2011. Even started to improve the engine, not so much to compete, but to understand the latests inventions. ProDeo still is and will always remain an old school 32-bit assembler engine, the idea to change it into a modern competitive engine died in 2002, my grand children and other things became more important. Being blackmailed by a 4 year old little lady with big eyes and curly hair for a new doll gives ultimate pleasure. I am drifting away, coming to your question, CCRL tested the UCI version of Rebel Century (rated 2542) that played Loek van Wely. Nowadays ProDeo is ~2800, so with limited effort it's substantially stronger, not similar any longer and not modernized.

5. Any other data that would suggest how RebelCentury UCI would perform against top human players on a modern i7, either at Classical time limit or Rapid?I realize it's been more than two decades since RebelCentury was written and competed, so I'll certainly understand if you can't remember details.

Faster hardware certainly would have avoided throwing away a won position against Anand on classical tournament time control ending in just a draw. While Deep Blue got all the credit beating Kasparov in 1997, one year later a PC program showed all the signs it was capable also, not so many years later confirmed by Fritz vs Kramnik and Junior vs Kasparov.

[lkaufman]

The 4 matches I played :

1997 - Rebel9 - Arthur Yusopov, on that moment rated 10th on the FIDE rating list
1998 - Rebel10 - Vishy Anand, second place FIDE rating list
2001 - Rebel Century - GM John van der Wiel who never lost a game to a computer.
2002 - Rebel Century 4 - GM Loek van Wely, at the time rated 2700+

The 2001 and 2002 matches the time control was 40 moves in 2 hours, shorter time controls were no longer an issue.

Regarding your questions -

1. What was the hardware in that match (how many cores, what speed?)
2. Is the RebelCentury UCI engine pretty close to the one that played Anand? If it is improved, how much?
3. I see a RebelCentury 3 and 4 in the SSDF list. Which one (if either) is the UCI version?

All 4 programs were DOS, hardware, NPS, TC etc. is listed in the 4 links.

4. How different is Benjamin 1.1 from RebelCentury UCI, aside from the gambit style? I can look at rating lists, I'm just asking if it is substantially different or just modestly, and if the search is substantially modernized or fairly similar?

After the 2002 match I retired and lost interest, only so now and then did something minor. I became interested in comp-chess again after the crazy ICGA verdict in 2011. Even started to improve the engine, not so much to compete, but to understand the latests inventions. ProDeo still is and will always remain an old school 32-bit assembler engine, the idea to change it into a modern competitive engine died in 2002, my grand children and other things became more important. Being blackmailed by a 4 year old little lady with big eyes and curly hair for a new doll gives ultimate pleasure. I am drifting away, coming to your question, CCRL tested the UCI version of Rebel Century (rated 2542) that played Loek van Wely. Nowadays ProDeo is ~2800, so with limited effort it's substantially stronger, not similar any longer and not modernized.

5. Any other data that would suggest how RebelCentury UCI would perform against top human players on a modern i7, either at Classical time limit or Rapid?I realize it's been more than two decades since RebelCentury was written and competed, so I'll certainly understand if you can't remember details.

Faster hardware certainly would have avoided throwing away a won position against Anand on classical tournament time control ending in just a draw. While Deep Blue got all the credit beating Kasparov in 1997, one year later a PC program showed all the signs it was capable also, not so many years later confirmed by Fritz vs Kramnik and Junior vs Kasparov.

Thanks, Ed. This is quite helpful. My analysis of the data suggests that RebelCentury would be a good anchor engine for a rating list that claims to correspond to human FIDE ratings. I would estimate from the data that RebelCentury running on an I-7 (say the reference hardware of CCRL) playing matches against current human GMS would perform about 2700 FIDE at Classical time limit, 2800 FIDE at Rapid (15' + 10"), and perhaps 2950 at blitz (3' + 2"). At 2' + 1" (which is technically bullet chess) it would probably be 3000. If these seem a bit low, I'm allowing for some rating deflation in recent years (Carlsen only player over 2800 now despite general increase in playing strength), much more experience for GMs in Rapid play online, and much more familiarity with engines compared to twenty years ago. I'm using this partly as a guide for Dragon Skill levels.

[Rebel]

The 4 matches I played :

1997 - Rebel9 - Arthur Yusopov, on that moment rated 10th on the FIDE rating list
1998 - Rebel10 - Vishy Anand, second place FIDE rating list
2001 - Rebel Century - GM John van der Wiel who never lost a game to a computer.
2002 - Rebel Century 4 - GM Loek van Wely, at the time rated 2700+

The 2001 and 2002 matches the time control was 40 moves in 2 hours, shorter time controls were no longer an issue.

Regarding your questions -

1. What was the hardware in that match (how many cores, what speed?)
2. Is the RebelCentury UCI engine pretty close to the one that played Anand? If it is improved, how much?
3. I see a RebelCentury 3 and 4 in the SSDF list. Which one (if either) is the UCI version?

All 4 programs were DOS, hardware, NPS, TC etc. is listed in the 4 links.

4. How different is Benjamin 1.1 from RebelCentury UCI, aside from the gambit style? I can look at rating lists, I'm just asking if it is substantially different or just modestly, and if the search is substantially modernized or fairly similar?

After the 2002 match I retired and lost interest, only so now and then did something minor. I became interested in comp-chess again after the crazy ICGA verdict in 2011. Even started to improve the engine, not so much to compete, but to understand the latests inventions. ProDeo still is and will always remain an old school 32-bit assembler engine, the idea to change it into a modern competitive engine died in 2002, my grand children and other things became more important. Being blackmailed by a 4 year old little lady with big eyes and curly hair for a new doll gives ultimate pleasure. I am drifting away, coming to your question, CCRL tested the UCI version of Rebel Century (rated 2542) that played Loek van Wely. Nowadays ProDeo is ~2800, so with limited effort it's substantially stronger, not similar any longer and not modernized.

5. Any other data that would suggest how RebelCentury UCI would perform against top human players on a modern i7, either at Classical time limit or Rapid?I realize it's been more than two decades since RebelCentury was written and competed, so I'll certainly understand if you can't remember details.

Faster hardware certainly would have avoided throwing away a won position against Anand on classical tournament time control ending in just a draw. While Deep Blue got all the credit beating Kasparov in 1997, one year later a PC program showed all the signs it was capable also, not so many years later confirmed by Fritz vs Kramnik and Junior vs Kasparov.

Thanks, Ed. This is quite helpful. My analysis of the data suggests that RebelCentury would be a good anchor engine for a rating list that claims to correspond to human FIDE ratings. I would estimate from the data that RebelCentury running on an I-7 (say the reference hardware of CCRL) playing matches against current human GMS would perform about 2700 FIDE at Classical time limit, 2800 FIDE at Rapid (15' + 10"), and perhaps 2950 at blitz (3' + 2"). At 2' + 1" (which is technically bullet chess) it would probably be 3000. If these seem a bit low, I'm allowing for some rating deflation in recent years (Carlsen only player over 2800 now despite general increase in playing strength), much more experience for GMs in Rapid play online, and much more familiarity with engines compared to twenty years ago. I'm using this partly as a guide for Dragon Skill levels.

I think your estimations are pretty good. But if we take a good look at the 2 tournament games Rebel 10 played against Anand one year after the famous victory of Deep Blue over Kasparov in 1997 one may conclude PC programs (Rebel 10 in this case) were very near and did not need 200 million NPS but only a NPS of 200,000-250,000 (a factor 800 less) to put a player like Anand into big trouble.

First game

[pgn][Event "game-7 (40/2:00 all 1:00)"]
[Site "8 game match"]
[Date "1998.07.22"]
[Round "?"]
[White "Anand,V"]
[Black "Rebel10 (exp)"]
[Result "1/2-1/2"]
[WhiteElo "2795"]
[BlackElo "?"]
[ECO "D07"]

1. d4 d5 2. c4 Nc6 3. e3 e5 4. Nf3 Bg4 5. Nc3 exd4 6. Nxd5 Nf6 7. Qb3 Bxf3
8. gxf3 Rb8 9. Bd2 Bd6 10. O-O-O O-O 11. Kb1 Nxd5 12. cxd5 Ne5 13. Be2
Re8 14. h4 c5 15. f4 Nd7 16. Bf3 b5 17. Qd3 Qf6 18. Bc1 dxe3 19. fxe3 c4
20. Qc2 Nc5 21. h5 Nd3 22. Rxd3 cxd3 23. Qxd3 Rec8 24. Bd2 b4 25. Be4 Qh6
26. Bg2 Rc7 27. Rf1 b3 28. e4 bxa2+ 29. Ka1 Rcb7 30. Bc1 Bc5 31. e5 Rb3
32. Qc2 Bd4 33. d6 Bxb2+ 34. Bxb2 Rxb2 35. Qxb2 Rxb2 36. Kxb2 g5 37. Bf3
gxf4 38. Rd1 Qe6 39. d7 Qxe5+ 40. Kxa2 Qa5+ 1/2-1/2[/pgn]
The position after 33.d6 black is won for black. I was much surprised. Instead of playing 33..Qe6 Rebel went for the drawish endgame exchange with 33..Bxb2+? One iteration deeper and Rebel 10 would have seen the winning 33..Qe6.

Second game

[pgn][Event "game-8 (40/2:00 all 1:00)"]
[Site "8 game match"]
[Date "1998.07.23"]
[Round "?"]
[White "Rebel10 (exp)"]
[Black "Anand,V"]
[Result "0-1"]
[WhiteElo "?"]
[BlackElo "2795"]
[ECO "E12"]

1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. a3 Bb7 5. Nc3 d5 6. cxd5 Nxd5 7. Qc2 Nxc3
8. bxc3 Nd7 9. e4 c5 10. Bf4 Qc8 11. Bb5 a6 12. Bxd7+ Qxd7 13. Ne5 Qc8
14. Qd3 b5 15. Qf3 Bd6 16. Nxf7 Bxf4 17. Nxh8 Qc7 18. Qh5+ g6 19. Nxg6
hxg6 20. Qxg6+ Qf7 21. Qxf7+ Kxf7 22. g3 Bh6 23. f3 cxd4 24. cxd4 Rc8 25.
h4 Rc2 26. g4 Be3 27. h5 Kg7 28. Rd1 a5 29. d5 exd5 30. Rh3 b4 31. axb4
axb4 32. f4 Bf2+ 33. Kf1 Bc5 34. Ke1 d4 35. e5 Rc3 36. Rh2 b3 37. h6+ Kh7
38. Rb2 0-1[/pgn]
After 17 moves Anand already is in trouble, then Rebel went wrong with 18.Qh5+ and blew its material advantage in the ending. 18. g3 or 18. 0-0 would have given Rebel 10 good chances on a victory, a draw at least.

It was surprising to see an 1998 engine doing only 200,000-250,000 NPS causing the world number 2 after Kasparov with a rating of 2795 so much trouble on the last bastion (40/2:00 all 1:00), a sign the end of the human domination over comps was inevitable in the near future.

[lkaufman]

The 4 matches I played :

1997 - Rebel9 - Arthur Yusopov, on that moment rated 10th on the FIDE rating list
1998 - Rebel10 - Vishy Anand, second place FIDE rating list
2001 - Rebel Century - GM John van der Wiel who never lost a game to a computer.
2002 - Rebel Century 4 - GM Loek van Wely, at the time rated 2700+

The 2001 and 2002 matches the time control was 40 moves in 2 hours, shorter time controls were no longer an issue.

Regarding your questions -

1. What was the hardware in that match (how many cores, what speed?)
2. Is the RebelCentury UCI engine pretty close to the one that played Anand? If it is improved, how much?
3. I see a RebelCentury 3 and 4 in the SSDF list. Which one (if either) is the UCI version?

All 4 programs were DOS, hardware, NPS, TC etc. is listed in the 4 links.

4. How different is Benjamin 1.1 from RebelCentury UCI, aside from the gambit style? I can look at rating lists, I'm just asking if it is substantially different or just modestly, and if the search is substantially modernized or fairly similar?

After the 2002 match I retired and lost interest, only so now and then did something minor. I became interested in comp-chess again after the crazy ICGA verdict in 2011. Even started to improve the engine, not so much to compete, but to understand the latests inventions. ProDeo still is and will always remain an old school 32-bit assembler engine, the idea to change it into a modern competitive engine died in 2002, my grand children and other things became more important. Being blackmailed by a 4 year old little lady with big eyes and curly hair for a new doll gives ultimate pleasure. I am drifting away, coming to your question, CCRL tested the UCI version of Rebel Century (rated 2542) that played Loek van Wely. Nowadays ProDeo is ~2800, so with limited effort it's substantially stronger, not similar any longer and not modernized.

5. Any other data that would suggest how RebelCentury UCI would perform against top human players on a modern i7, either at Classical time limit or Rapid?I realize it's been more than two decades since RebelCentury was written and competed, so I'll certainly understand if you can't remember details.

Faster hardware certainly would have avoided throwing away a won position against Anand on classical tournament time control ending in just a draw. While Deep Blue got all the credit beating Kasparov in 1997, one year later a PC program showed all the signs it was capable also, not so many years later confirmed by Fritz vs Kramnik and Junior vs Kasparov.

Thanks, Ed. This is quite helpful. My analysis of the data suggests that RebelCentury would be a good anchor engine for a rating list that claims to correspond to human FIDE ratings. I would estimate from the data that RebelCentury running on an I-7 (say the reference hardware of CCRL) playing matches against current human GMS would perform about 2700 FIDE at Classical time limit, 2800 FIDE at Rapid (15' + 10"), and perhaps 2950 at blitz (3' + 2"). At 2' + 1" (which is technically bullet chess) it would probably be 3000. If these seem a bit low, I'm allowing for some rating deflation in recent years (Carlsen only player over 2800 now despite general increase in playing strength), much more experience for GMs in Rapid play online, and much more familiarity with engines compared to twenty years ago. I'm using this partly as a guide for Dragon Skill levels.

I think your estimations are pretty good. But if we take a good look at the 2 tournament games Rebel 10 played against Anand one year after the famous victory of Deep Blue over Kasparov in 1997 one may conclude PC programs (Rebel 10 in this case) were very near and did not need 200 million NPS but only a NPS of 200,000-250,000 (a factor 800 less) to put a player like Anand into big trouble.

First game

[pgn][Event "game-7 (40/2:00 all 1:00)"]
[Site "8 game match"]
[Date "1998.07.22"]
[Round "?"]
[White "Anand,V"]
[Black "Rebel10 (exp)"]
[Result "1/2-1/2"]
[WhiteElo "2795"]
[BlackElo "?"]
[ECO "D07"]

1. d4 d5 2. c4 Nc6 3. e3 e5 4. Nf3 Bg4 5. Nc3 exd4 6. Nxd5 Nf6 7. Qb3 Bxf3
8. gxf3 Rb8 9. Bd2 Bd6 10. O-O-O O-O 11. Kb1 Nxd5 12. cxd5 Ne5 13. Be2
Re8 14. h4 c5 15. f4 Nd7 16. Bf3 b5 17. Qd3 Qf6 18. Bc1 dxe3 19. fxe3 c4
20. Qc2 Nc5 21. h5 Nd3 22. Rxd3 cxd3 23. Qxd3 Rec8 24. Bd2 b4 25. Be4 Qh6
26. Bg2 Rc7 27. Rf1 b3 28. e4 bxa2+ 29. Ka1 Rcb7 30. Bc1 Bc5 31. e5 Rb3
32. Qc2 Bd4 33. d6 Bxb2+ 34. Bxb2 Rxb2 35. Qxb2 Rxb2 36. Kxb2 g5 37. Bf3
gxf4 38. Rd1 Qe6 39. d7 Qxe5+ 40. Kxa2 Qa5+ 1/2-1/2[/pgn]
The position after 33.d6 black is won for black. I was much surprised. Instead of playing 33..Qe6 Rebel went for the drawish endgame exchange with 33..Bxb2+? One iteration deeper and Rebel 10 would have seen the winning 33..Qe6.

Second game

[pgn][Event "game-8 (40/2:00 all 1:00)"]
[Site "8 game match"]
[Date "1998.07.23"]
[Round "?"]
[White "Rebel10 (exp)"]
[Black "Anand,V"]
[Result "0-1"]
[WhiteElo "?"]
[BlackElo "2795"]
[ECO "E12"]

1. d4 Nf6 2. c4 e6 3. Nf3 b6 4. a3 Bb7 5. Nc3 d5 6. cxd5 Nxd5 7. Qc2 Nxc3
8. bxc3 Nd7 9. e4 c5 10. Bf4 Qc8 11. Bb5 a6 12. Bxd7+ Qxd7 13. Ne5 Qc8
14. Qd3 b5 15. Qf3 Bd6 16. Nxf7 Bxf4 17. Nxh8 Qc7 18. Qh5+ g6 19. Nxg6
hxg6 20. Qxg6+ Qf7 21. Qxf7+ Kxf7 22. g3 Bh6 23. f3 cxd4 24. cxd4 Rc8 25.
h4 Rc2 26. g4 Be3 27. h5 Kg7 28. Rd1 a5 29. d5 exd5 30. Rh3 b4 31. axb4
axb4 32. f4 Bf2+ 33. Kf1 Bc5 34. Ke1 d4 35. e5 Rc3 36. Rh2 b3 37. h6+ Kh7
38. Rb2 0-1[/pgn]
After 17 moves Anand already is in trouble, then Rebel went wrong with 18.Qh5+ and blew its material advantage in the ending. 18. g3 or 18. 0-0 would have given Rebel 10 good chances on a victory, a draw at least.

It was surprising to see an 1998 engine doing only 200,000-250,000 NPS causing the world number 2 after Kasparov with a rating of 2795 so much trouble on the last bastion (40/2:00 all 1:00), a sign the end of the human domination over comps was inevitable in the near future.

Yes, the Anand match was very impressive for Rebel. If I only had that to go by, I would have estimated the ratings higher. But the results vs Yusupov and van der Wiel were significantly worse in elo terms, though still quite good, although I know the hardware was slower for Yusupov and that van der Wiel was an anti-computer specialist. I think that the combined result of the RebelCentury matches (with van der Wiel and van Wely) gave about a 2625 performance rating at classical chess, combining results from a 1.4 GHz and 1.9 Ghz machines using RebelCentury versions 3 and 4. If we pretend that all games used the slower hardware but the later program version (which is probably roughly fair), then the current RebelCentury UCI on the ccrl reference i7 runs roughly four times as fast as the 1.4 GHz Athlon (as best I can determine). Doubling speed appears to be worth about 45 elo at 40/2 hours (for engines in that range), so that gives us 2715. Van der Wiel's anti-computer expertise probably offsets the fact that all GMs are now pretty computer-savvy, and allowing a bit for recent rating deflation, 2700 for RebelCentury on i7 at classical TC seems about right.