Rook odds match: human vs engines.

[lkaufman]

I ran a match between Dragon 2.6 1500 elo vs Komodo 12.1.1 Skill 10, after 1000 games whoever had the worst increased his strength, +100 elo for Dragon, +1 level for Komodo, and ran another 1000 games until Komodo 12 lvl 19 is beaten by Dragon. Book used are the first 500 openings of 8moves_v3

dr 1500 vs km 10:
WDL: [156, 529, 315]
Time: 277.91 seconds
elo performance: -55.71 ± 14.76

dr 1600 vs km 10:
WDL: [227, 451, 322]
Time: 294.96 seconds
elo performance: -33.11 ± 15.98

dr 1700 vs km 10:
WDL: [297, 418, 285]
Time: 305.26 seconds
elo performance: 4.17 ± 16.44

dr 1700 vs km 11:
WDL: [169, 418, 413]
Time: 333.98 seconds
elo performance: -86.52 ± 16.57

dr 1800 vs km 11:
WDL: [253, 382, 365]
Time: 343.85 seconds
elo performance: -39.08 ± 16.98

dr 1900 vs km 11:
WDL: [352, 343, 305]
Time: 361.04 seconds
elo performance: 16.34 ± 17.48

dr 1900 vs km 12:
WDL: [261, 334, 405]
Time: 352.78 seconds
elo performance: -50.38 ± 17.68

dr 2000 vs km 12:
WDL: [410, 320, 270]
Time: 388.43 seconds
elo performance: 48.96 ± 17.87

dr 2000 vs km 13:
WDL: [327, 326, 347]
Time: 423.55 seconds
elo performance: -6.95 ± 17.70

dr 2100 vs km 13:
WDL: [458, 283, 259]
Time: 446.74 seconds
elo performance: 70.07 ± 18.47

dr 2100 vs km 14:
WDL: [351, 275, 374]
Time: 510.63 seconds
elo performance: -7.99 ± 18.36

dr 2200 vs km 14:
WDL: [464, 244, 292]
Time: 535.46 seconds
elo performance: 60.36 ± 18.93

dr 2200 vs km 15:
WDL: [385, 260, 355]
Time: 533.94 seconds
elo performance: 10.43 ± 18.55

dr 2200 vs km 16:
WDL: [224, 297, 479]
Time: 571.36 seconds
elo performance: -90.59 ± 18.42

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

dr 2300 vs km 17:
WDL: [235, 262, 503]
Time: 721.49 seconds
elo performance: -95.44 ± 18.96

dr 2400 vs km 17:
WDL: [430, 252, 318]
Time: 841.09 seconds
elo performance: 39.08 ± 18.72

dr 2400 vs km 18:
WDL: [298, 267, 435]
Time: 988.44 seconds
elo performance: -47.90 ± 18.57

dr 2500 vs km 18:
WDL: [474, 256, 270]
Time: 1111.39 seconds
elo performance: 71.89 ± 18.85

dr 2500 vs km 19:
WDL: [271, 274, 455]
Time: 1524.66 seconds
elo performance: -64.66 ± 18.56

dr 2600 vs km 19:
WDL: [493, 285, 222]
Time: 1803.14 seconds
elo performance: 96.57 ± 18.64

Total time: 13311.54

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

This test is especially interesting to me. In the last couple of days, I tried my luck against different versions of Komodo at 7 ply search. I played against Komodo 13.02 level 16, Komodo 14.1 level 20 and Komodo 12.1.1 level 16, all at 15+10 time control. My results are:

3-3 vs. Komodo 13.02 level 16 (2 wins, 2 losses, 2 draws)
1-2 vs. Komodo 14.1 level 20 (1 win, 2 losses)
1-3 vs. Komodo 12.1.1 level 16 (1 win, 3 losses)

It seems that I can put a solid resistance to Komodos at 7 ply. I estimate that they are about 2200 Elo at rapid 15+10, maybe a bit less. 5 out of 13 is 38,5%, that is, -81 ELo. So my performance in these 13 games is at least 2100 Elo.

I think this is a very good method for measuring the elo differences between different Elo settings on Dragon, even if the human equivalence of the Komodo 12 skill levels is unclear. I'm using it myself with Dragon 3 beta to see if the Elo levels are at least reasonably spaced. I find that the 2200 Elo setting is a good match for me at 15' + 10", maybe even a bit stronger than I am at that level. On the other hand, I just had it play a couple games with a student rated only 320 chess.com Rapid, and I had to raise the Elo setting to 1000 to get a good matchup. So something needs adjusting still. One question: when you play against the engines yourself, what opening book do you use? Note that if you are not using the free book on the komododhess website, but use an external book, the book settings for the levels won't work, you'll have to limit book depth manually (depth in moves = first two digits of elo rating /2 - 2). If you are using no book at all that might weaken it a level or so.

[pepechuy]

Why do you refer to "the future" for Dragon 3 if you expect it to be out in two weeks? Do you just mean "when released"? If so nothing has changed, I doubt that Dragon 3 will be noticeably better at giving piece odds than 2.6 was. 30 minutes per game is just 20% more than 15' + 10" (assuming 60 moves), so Dragon 3 should still be able to give knight odds to a par (2400 FIDE) IM with even results at 30 min. For a borderline master (USCF 2200, about 2140 FIDE or so) knight and move (i.e. Black knight odds) would be about right. If you are asking about the distant future when we might have radically different chess programs, then it's just speculation, maybe rook odds to an IM might be possible in 20 years, who knows?

Hi.

If 30 years ago somebody had told me that I would be taking rook odds from an engine (we called them "programs" back then), or that grandmaster would be taking two-pawn odds, I would have had a hard time believing it.

Greetings.

[pepechuy]

Hi.

The second game against Komodo, removing the a1 rook.
¡My first win against Komodo in this challenge!

Komodo deviated from the first game with 3. d4 (instead of 3. Ne5)
I felt confident about the win after the queen trade on move 28.
But even two moves before that, I felt that Komodo was not giving me any respite, it had some kind of threat every move. But I was never in danger of losing, just had to play very carefully.

[pgn]
[Event "banksia game"]
[Site "?"]
[Date "2022.04.23"]
[Round "?"]
[White "Komodo 12.1.1 64-bit"]
[Black "hm::Garcia Ruvalcaba, Jose de Jesus"]
[Result "0-1"]
[WhiteElo "3500"]
[Time "18:01:39"]
[TimeControl "900+10"]
[SetUp "1"]
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/1NBQKBNR w Kkq - 0 1"]
[Termination "mate"]

1. e4 {-3.27/22 27373 22532299} 1... e5 2. Nf3 {-3.26/24 40903 35484185} 2...
Nf6 3. d4 {-3.43/25 61474 57390008} 3... Nxe4 4. Nxe5 {-3.55/23 32906 41511471}
4... d5 5. Bd3 {-3.58/24 30165 35112902} 5... Nd7 6. Nf3 {-3.67/25 48729
65886913} 6... Ndf6 7. Ne5 {-3.48/24 47850 45435724} 7... Be6 8. O-O {-3.54/23
40143 37826803} 8... Be7 9. c3 {-3.59/23 34452 39680054} 9... O-O 10. Bc2
{-3.65/23 37489 43444457} 10... Re8 11. Nd2 {-3.82/24 81209 86461408} 11... Nxd2
12. Bxd2 {-3.88/22 32402 29015193} 12... Qc8 13. Qb1 {-4.04/22 32384 30051263}
13... c6 14. Bf4 {-4.10/22 53422 51322912} 14... h6 15. Re1 {-4.13/24 90208
77608100} 15... Nd7 16. a3 {-3.88/22 42533 40275683} 16... Nxe5 17. dxe5
{-4.16/22 31030 28350085} 17... Qd7 18. Be3 {-3.91/23 26852 25733856} 18... f6
19. Bh7+ {-4.26/22 70846 64533183} 19... Kh8 20. Bg6 {-4.36/23 15855 18530397}
20... Rf8 21. exf6 {-4.46/23 18531 17763288} 21... Bxf6 22. Qd3 {-4.34/24 70434
67982855} 22... Bf7 23. Bf5 {-4.58/23 10 30477316} 23... Qc7 24. Qd2 {-4.50/23
84347 77592963} 24... Rfe8 25. Rf1 {-4.61/21 18355 17684572} 25... Qe7 26. Bg4
{-5.04/25 28705 29534373} 26... Bg5 27. Bxg5 {-5.40/29 10 52952333} 27... Qxg5
28. Qxg5 {-5.48/26 19613 25734163} 28... hxg5 29. f4 {-5.43/26 10311 13866441}
29... gxf4 30. Rxf4 {-5.48/26 10 16369199} 30... Kg8 31. Rd4 {-5.51/25 29936
33816129} 31... Re4 32. Rxe4 {-5.60/28 14147 18861343} 32... dxe4 33. Kf2
{-5.89/28 8387 16180315} 33... Rd8 34. Ke3 {-5.79/24 11423 15194953} 34... Bg6
35. Be2 {-6.15/23 14268 19084501} 35... Kf7 36. g3 {-6.65/25 23886 42807498}
36... Ke6 37. b3 {-6.98/25 28182 36293010} 37... Ke5 38. h3 {-7.25/24 10
21890295} 38... Rh8 39. Bf1 {-7.45/24 10537 12775796} 39... Bf7 40. c4 {-8.38/23
13077 14483683} 40... c5 41. Bg2 {-8.40/25 15423 20143793} 41... Bg6 42. Bf1
{-8.97/26 10 20432310} 42... Rd8 43. Be2 {-9.30/26 10 21763875} 43... Rd4 44. h4
{-9.37/23 12338 16887908} 44... Be8 45. g4 {-9.66/24 16028 20606445} 45... Bc6
46. g5 {-8.03/25 22785 28434875} 46... g6 47. b4 {-9.11/23 10511 15224854} 47...
cxb4 48. axb4 {-7.82/22 10022 14790009} 48... b6 49. Bf1 {-8.98/22 10001
14701502} 49... Rd8 50. Be2 {-9.64/23 9998 15413945} 50... Rh8 51. b5 {-9.87/23
10 4865182} 51... Bb7 52. h5 {-11.55/24 10 20225255} 52... gxh5 53. c5
{-15.20/23 16497 34013091} 53... bxc5 54. g6 {-17.23/21 8806 22681400} 54... h4
55. Kf2 {-19.25/19 11203 24358733} 55... Kf6 56. Kg1 {-16.22/22 22933 40193002}
56... Kxg6 57. Kg2 {-16.38/20 10515 17868119} 57... e3+ 58. Kg1 {M-8/18 1435
2611775} 58... Kg5 59. b6 {M-8/22 1136 2116281} 59... axb6 60. Kf1 {M-7/23 1022
2216668} 60... Kf4 61. Bg4 {M-6/27 1175 2886137} 61... h3 62. Ke1 {M-5/36 1054
2935244} 62... h2 63. Kd1 {M-4/99 943 3072656} 63... h1=Q+ 64. Kc2 {M-3/99 10
14855} 64... Be4+ 65. Kc3 {M-2/99 10 6870} 65... Qc1+ 66. Kb3 {M-2/5 10 220}
66... Bd5+ 67. Ka4 {M-2/5 10 264} 67... Qc4+ 68. Ka3 {M-1/99 10 197} 68... Qb3#
0-1
[/pgn]

After two rounds, the results (all from my perspective)

Crafty, a1 rook: 1/2, 1
Komodo, h1 rook: 0, 1/2
Lc0, a1 rook: 0, 1
Stockfish, h1 rook: 0, 1
Crafty, h1 rook: 1
Komodo, a1 rook: 0, 1
Lc0, h1 rook: 1
Stockfish, a1 rook: 1

I have dropped 5 points out of 13 games.
Taking the a1 rook: 2.5 dropped points in 7 games.
Taking the h1 rook: 2.5 dropped points in 6 games.
Against Crafty: 0.5 dropped points in 3 games.
Against Komodo: 2.5 dropped points in 4 games.
Against Lc0: 1 dropped point in 3 games.
Against Stockfish: 1 dropped point in 3 games.

My results in the second round (4.5 points in 5 games) were much better than in the first round (3.5 points in 8 games).
But even in the first round, I scored much better in the second half (first half: 0.5 points in 4 games, second half: 3 points in 4 games).
I do not think that I have improved that much. Rather, I have quickly adapted my play to the peculiarities of this challenge.

Next game, against Komodo removing the h1 rook, as soon as I get the chance.

Greetings.

[Fritz 0]

I ran a match between Dragon 2.6 1500 elo vs Komodo 12.1.1 Skill 10, after 1000 games whoever had the worst increased his strength, +100 elo for Dragon, +1 level for Komodo, and ran another 1000 games until Komodo 12 lvl 19 is beaten by Dragon. Book used are the first 500 openings of 8moves_v3

dr 1500 vs km 10:
WDL: [156, 529, 315]
Time: 277.91 seconds
elo performance: -55.71 ± 14.76

dr 1600 vs km 10:
WDL: [227, 451, 322]
Time: 294.96 seconds
elo performance: -33.11 ± 15.98

dr 1700 vs km 10:
WDL: [297, 418, 285]
Time: 305.26 seconds
elo performance: 4.17 ± 16.44

dr 1700 vs km 11:
WDL: [169, 418, 413]
Time: 333.98 seconds
elo performance: -86.52 ± 16.57

dr 1800 vs km 11:
WDL: [253, 382, 365]
Time: 343.85 seconds
elo performance: -39.08 ± 16.98

dr 1900 vs km 11:
WDL: [352, 343, 305]
Time: 361.04 seconds
elo performance: 16.34 ± 17.48

dr 1900 vs km 12:
WDL: [261, 334, 405]
Time: 352.78 seconds
elo performance: -50.38 ± 17.68

dr 2000 vs km 12:
WDL: [410, 320, 270]
Time: 388.43 seconds
elo performance: 48.96 ± 17.87

dr 2000 vs km 13:
WDL: [327, 326, 347]
Time: 423.55 seconds
elo performance: -6.95 ± 17.70

dr 2100 vs km 13:
WDL: [458, 283, 259]
Time: 446.74 seconds
elo performance: 70.07 ± 18.47

dr 2100 vs km 14:
WDL: [351, 275, 374]
Time: 510.63 seconds
elo performance: -7.99 ± 18.36

dr 2200 vs km 14:
WDL: [464, 244, 292]
Time: 535.46 seconds
elo performance: 60.36 ± 18.93

dr 2200 vs km 15:
WDL: [385, 260, 355]
Time: 533.94 seconds
elo performance: 10.43 ± 18.55

dr 2200 vs km 16:
WDL: [224, 297, 479]
Time: 571.36 seconds
elo performance: -90.59 ± 18.42

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

dr 2300 vs km 17:
WDL: [235, 262, 503]
Time: 721.49 seconds
elo performance: -95.44 ± 18.96

dr 2400 vs km 17:
WDL: [430, 252, 318]
Time: 841.09 seconds
elo performance: 39.08 ± 18.72

dr 2400 vs km 18:
WDL: [298, 267, 435]
Time: 988.44 seconds
elo performance: -47.90 ± 18.57

dr 2500 vs km 18:
WDL: [474, 256, 270]
Time: 1111.39 seconds
elo performance: 71.89 ± 18.85

dr 2500 vs km 19:
WDL: [271, 274, 455]
Time: 1524.66 seconds
elo performance: -64.66 ± 18.56

dr 2600 vs km 19:
WDL: [493, 285, 222]
Time: 1803.14 seconds
elo performance: 96.57 ± 18.64

Total time: 13311.54

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

This test is especially interesting to me. In the last couple of days, I tried my luck against different versions of Komodo at 7 ply search. I played against Komodo 13.02 level 16, Komodo 14.1 level 20 and Komodo 12.1.1 level 16, all at 15+10 time control. My results are:

3-3 vs. Komodo 13.02 level 16 (2 wins, 2 losses, 2 draws)
1-2 vs. Komodo 14.1 level 20 (1 win, 2 losses)
1-3 vs. Komodo 12.1.1 level 16 (1 win, 3 losses)

It seems that I can put a solid resistance to Komodos at 7 ply. I estimate that they are about 2200 Elo at rapid 15+10, maybe a bit less. 5 out of 13 is 38,5%, that is, -81 ELo. So my performance in these 13 games is at least 2100 Elo.

I think this is a very good method for measuring the elo differences between different Elo settings on Dragon, even if the human equivalence of the Komodo 12 skill levels is unclear. I'm using it myself with Dragon 3 beta to see if the Elo levels are at least reasonably spaced. I find that the 2200 Elo setting is a good match for me at 15' + 10", maybe even a bit stronger than I am at that level. On the other hand, I just had it play a couple games with a student rated only 320 chess.com Rapid, and I had to raise the Elo setting to 1000 to get a good matchup. So something needs adjusting still. One question: when you play against the engines yourself, what opening book do you use? Note that if you are not using the free book on the komododhess website, but use an external book, the book settings for the levels won't work, you'll have to limit book depth manually (depth in moves = first two digits of elo rating /2 - 2). If you are using no book at all that might weaken it a level or so.

I always use Fritz 16 opening book set to the "handicap" mode. It gives almost the maximum variety and allows following the book up to 20 moves, although the engine usually only follows it for the first 8-10 moves (of course i don't look at the book during the games, but only when I go over them afterwards to see what idiotic mistakes I made).

Do you think that 7 ply Komodos are in the ballpark of 2200 at 15+10?

[lkaufman]

I ran a match between Dragon 2.6 1500 elo vs Komodo 12.1.1 Skill 10, after 1000 games whoever had the worst increased his strength, +100 elo for Dragon, +1 level for Komodo, and ran another 1000 games until Komodo 12 lvl 19 is beaten by Dragon. Book used are the first 500 openings of 8moves_v3

dr 1500 vs km 10:
WDL: [156, 529, 315]
Time: 277.91 seconds
elo performance: -55.71 ± 14.76

dr 1600 vs km 10:
WDL: [227, 451, 322]
Time: 294.96 seconds
elo performance: -33.11 ± 15.98

dr 1700 vs km 10:
WDL: [297, 418, 285]
Time: 305.26 seconds
elo performance: 4.17 ± 16.44

dr 1700 vs km 11:
WDL: [169, 418, 413]
Time: 333.98 seconds
elo performance: -86.52 ± 16.57

dr 1800 vs km 11:
WDL: [253, 382, 365]
Time: 343.85 seconds
elo performance: -39.08 ± 16.98

dr 1900 vs km 11:
WDL: [352, 343, 305]
Time: 361.04 seconds
elo performance: 16.34 ± 17.48

dr 1900 vs km 12:
WDL: [261, 334, 405]
Time: 352.78 seconds
elo performance: -50.38 ± 17.68

dr 2000 vs km 12:
WDL: [410, 320, 270]
Time: 388.43 seconds
elo performance: 48.96 ± 17.87

dr 2000 vs km 13:
WDL: [327, 326, 347]
Time: 423.55 seconds
elo performance: -6.95 ± 17.70

dr 2100 vs km 13:
WDL: [458, 283, 259]
Time: 446.74 seconds
elo performance: 70.07 ± 18.47

dr 2100 vs km 14:
WDL: [351, 275, 374]
Time: 510.63 seconds
elo performance: -7.99 ± 18.36

dr 2200 vs km 14:
WDL: [464, 244, 292]
Time: 535.46 seconds
elo performance: 60.36 ± 18.93

dr 2200 vs km 15:
WDL: [385, 260, 355]
Time: 533.94 seconds
elo performance: 10.43 ± 18.55

dr 2200 vs km 16:
WDL: [224, 297, 479]
Time: 571.36 seconds
elo performance: -90.59 ± 18.42

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

dr 2300 vs km 17:
WDL: [235, 262, 503]
Time: 721.49 seconds
elo performance: -95.44 ± 18.96

dr 2400 vs km 17:
WDL: [430, 252, 318]
Time: 841.09 seconds
elo performance: 39.08 ± 18.72

dr 2400 vs km 18:
WDL: [298, 267, 435]
Time: 988.44 seconds
elo performance: -47.90 ± 18.57

dr 2500 vs km 18:
WDL: [474, 256, 270]
Time: 1111.39 seconds
elo performance: 71.89 ± 18.85

dr 2500 vs km 19:
WDL: [271, 274, 455]
Time: 1524.66 seconds
elo performance: -64.66 ± 18.56

dr 2600 vs km 19:
WDL: [493, 285, 222]
Time: 1803.14 seconds
elo performance: 96.57 ± 18.64

Total time: 13311.54

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

This test is especially interesting to me. In the last couple of days, I tried my luck against different versions of Komodo at 7 ply search. I played against Komodo 13.02 level 16, Komodo 14.1 level 20 and Komodo 12.1.1 level 16, all at 15+10 time control. My results are:

3-3 vs. Komodo 13.02 level 16 (2 wins, 2 losses, 2 draws)
1-2 vs. Komodo 14.1 level 20 (1 win, 2 losses)
1-3 vs. Komodo 12.1.1 level 16 (1 win, 3 losses)

It seems that I can put a solid resistance to Komodos at 7 ply. I estimate that they are about 2200 Elo at rapid 15+10, maybe a bit less. 5 out of 13 is 38,5%, that is, -81 ELo. So my performance in these 13 games is at least 2100 Elo.

I think this is a very good method for measuring the elo differences between different Elo settings on Dragon, even if the human equivalence of the Komodo 12 skill levels is unclear. I'm using it myself with Dragon 3 beta to see if the Elo levels are at least reasonably spaced. I find that the 2200 Elo setting is a good match for me at 15' + 10", maybe even a bit stronger than I am at that level. On the other hand, I just had it play a couple games with a student rated only 320 chess.com Rapid, and I had to raise the Elo setting to 1000 to get a good matchup. So something needs adjusting still. One question: when you play against the engines yourself, what opening book do you use? Note that if you are not using the free book on the komododhess website, but use an external book, the book settings for the levels won't work, you'll have to limit book depth manually (depth in moves = first two digits of elo rating /2 - 2). If you are using no book at all that might weaken it a level or so.

I always use Fritz 16 opening book set to the "handicap" mode. It gives almost the maximum variety and allows following the book up to 20 moves, although the engine usually only follows it for the first 8-10 moves (of course i don't look at the book during the games, but only when I go over them afterwards to see what idiotic mistakes I made).

Do you think that 7 ply Komodos are in the ballpark of 2200 at 15+10?

It varied from version to version (not always upward at fixed depth), but I recall that for one of the later versions of Komodo, after it switched to 25 levels, I published the estimated formula FIDE rating (at Rapid) = 114 X level. Level 20 was the 7 ply level in those versions, so my estimate was 2280. It's hard to explain why you do so well against it compared to your chess.com Rapid rating. Perhaps the Fritz "handicap" opening book could be the problem; I'm not familiar with it, but you should be using a normal variety book (depth limited, weighted more or less by frequency), not a handicap book, for these games. For 2100 Elo, 8 moves of book would be about right. If you are coming out of book with favorable evals (on average, combining White and Black) that might be the explanation.

[Fritz 0]

I ran a match between Dragon 2.6 1500 elo vs Komodo 12.1.1 Skill 10, after 1000 games whoever had the worst increased his strength, +100 elo for Dragon, +1 level for Komodo, and ran another 1000 games until Komodo 12 lvl 19 is beaten by Dragon. Book used are the first 500 openings of 8moves_v3

dr 1500 vs km 10:
WDL: [156, 529, 315]
Time: 277.91 seconds
elo performance: -55.71 ± 14.76

dr 1600 vs km 10:
WDL: [227, 451, 322]
Time: 294.96 seconds
elo performance: -33.11 ± 15.98

dr 1700 vs km 10:
WDL: [297, 418, 285]
Time: 305.26 seconds
elo performance: 4.17 ± 16.44

dr 1700 vs km 11:
WDL: [169, 418, 413]
Time: 333.98 seconds
elo performance: -86.52 ± 16.57

dr 1800 vs km 11:
WDL: [253, 382, 365]
Time: 343.85 seconds
elo performance: -39.08 ± 16.98

dr 1900 vs km 11:
WDL: [352, 343, 305]
Time: 361.04 seconds
elo performance: 16.34 ± 17.48

dr 1900 vs km 12:
WDL: [261, 334, 405]
Time: 352.78 seconds
elo performance: -50.38 ± 17.68

dr 2000 vs km 12:
WDL: [410, 320, 270]
Time: 388.43 seconds
elo performance: 48.96 ± 17.87

dr 2000 vs km 13:
WDL: [327, 326, 347]
Time: 423.55 seconds
elo performance: -6.95 ± 17.70

dr 2100 vs km 13:
WDL: [458, 283, 259]
Time: 446.74 seconds
elo performance: 70.07 ± 18.47

dr 2100 vs km 14:
WDL: [351, 275, 374]
Time: 510.63 seconds
elo performance: -7.99 ± 18.36

dr 2200 vs km 14:
WDL: [464, 244, 292]
Time: 535.46 seconds
elo performance: 60.36 ± 18.93

dr 2200 vs km 15:
WDL: [385, 260, 355]
Time: 533.94 seconds
elo performance: 10.43 ± 18.55

dr 2200 vs km 16:
WDL: [224, 297, 479]
Time: 571.36 seconds
elo performance: -90.59 ± 18.42

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

dr 2300 vs km 17:
WDL: [235, 262, 503]
Time: 721.49 seconds
elo performance: -95.44 ± 18.96

dr 2400 vs km 17:
WDL: [430, 252, 318]
Time: 841.09 seconds
elo performance: 39.08 ± 18.72

dr 2400 vs km 18:
WDL: [298, 267, 435]
Time: 988.44 seconds
elo performance: -47.90 ± 18.57

dr 2500 vs km 18:
WDL: [474, 256, 270]
Time: 1111.39 seconds
elo performance: 71.89 ± 18.85

dr 2500 vs km 19:
WDL: [271, 274, 455]
Time: 1524.66 seconds
elo performance: -64.66 ± 18.56

dr 2600 vs km 19:
WDL: [493, 285, 222]
Time: 1803.14 seconds
elo performance: 96.57 ± 18.64

Total time: 13311.54

dr 2300 vs km 16:
WDL: [398, 228, 374]
Time: 641.44 seconds
elo performance: 8.34 ± 18.94

This test is especially interesting to me. In the last couple of days, I tried my luck against different versions of Komodo at 7 ply search. I played against Komodo 13.02 level 16, Komodo 14.1 level 20 and Komodo 12.1.1 level 16, all at 15+10 time control. My results are:

3-3 vs. Komodo 13.02 level 16 (2 wins, 2 losses, 2 draws)
1-2 vs. Komodo 14.1 level 20 (1 win, 2 losses)
1-3 vs. Komodo 12.1.1 level 16 (1 win, 3 losses)

It seems that I can put a solid resistance to Komodos at 7 ply. I estimate that they are about 2200 Elo at rapid 15+10, maybe a bit less. 5 out of 13 is 38,5%, that is, -81 ELo. So my performance in these 13 games is at least 2100 Elo.

I think this is a very good method for measuring the elo differences between different Elo settings on Dragon, even if the human equivalence of the Komodo 12 skill levels is unclear. I'm using it myself with Dragon 3 beta to see if the Elo levels are at least reasonably spaced. I find that the 2200 Elo setting is a good match for me at 15' + 10", maybe even a bit stronger than I am at that level. On the other hand, I just had it play a couple games with a student rated only 320 chess.com Rapid, and I had to raise the Elo setting to 1000 to get a good matchup. So something needs adjusting still. One question: when you play against the engines yourself, what opening book do you use? Note that if you are not using the free book on the komododhess website, but use an external book, the book settings for the levels won't work, you'll have to limit book depth manually (depth in moves = first two digits of elo rating /2 - 2). If you are using no book at all that might weaken it a level or so.

I always use Fritz 16 opening book set to the "handicap" mode. It gives almost the maximum variety and allows following the book up to 20 moves, although the engine usually only follows it for the first 8-10 moves (of course i don't look at the book during the games, but only when I go over them afterwards to see what idiotic mistakes I made).

Do you think that 7 ply Komodos are in the ballpark of 2200 at 15+10?

It varied from version to version (not always upward at fixed depth), but I recall that for one of the later versions of Komodo, after it switched to 25 levels, I published the estimated formula FIDE rating (at Rapid) = 114 X level. Level 20 was the 7 ply level in those versions, so my estimate was 2280. It's hard to explain why you do so well against it compared to your chess.com Rapid rating. Perhaps the Fritz "handicap" opening book could be the problem; I'm not familiar with it, but you should be using a normal variety book (depth limited, weighted more or less by frequency), not a handicap book, for these games. For 2100 Elo, 8 moves of book would be about right. If you are coming out of book with favorable evals (on average, combining White and Black) that might be the explanation.

I've just played a 15+10 game against Dragon 2.6 Elo 2200, Fritz 16 opening book set to "normal", and drew as Black. Of course, one game means nothing, but my impression is that Komodo 12 level 16 is stronger than it.

Probably I should have said that I haven't played on chess.com and lichess for a while, due to occasional problems with the connection and cheaters, so I don't really know how reliable my online ratings are.

[pepechuy]

Hi.

The third game against Komodo, removing the h1 rook.
I lost.

Komodo deviated from the two previous games with 5. ba6 (first round: 5. Nc3; second round: 5. e3).
After 13... Rfb8 (activating my extra rook) I felt that I had a very good position, ¡and I did!
Somehow, I misplayed it bit by bit, until I lost.
I still do not find a way to resign in Banksia, had to play until I got checkmated.

[pgn]
[Event "banksia game"]
[Site "?"]
[Date "2022.04.24"]
[Round "?"]
[White "Komodo 12.1.1 64-bit"]
[Black "hm::Garcia Ruvalcaba, Jose de Jesus"]
[Result "1-0"]
[WhiteElo "3500"]
[Time "18:18:36"]
[TimeControl "900+10"]
[SetUp "1"]
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBN1 w Qkq - 0 1"]
[Termination "mate"]

1. d4 {-3.64/24 65576 47970336} 1... Nf6 2. c4 {-3.63/23 66834 55477510} 2... c5
3. d5 {-3.29/25 35530 31309756} 3... b5 4. cxb5 {-3.42/22 13026 13972079} 4...
a6 5. bxa6 {-3.58/23 69648 65778288} 5... Bxa6 6. Nc3 {-3.31/23 45955 41370896}
6... g6 7. e4 {-3.45/23 42182 36412003} 7... Bxf1 8. Kxf1 {-3.16/24 28033
22705425} 8... d6 9. Nge2 {-3.20/25 25758 26295012} 9... Bg7 10. Kg1 {-3.27/25
51100 52508246} 10... O-O 11. Rb1 {-3.37/24 82703 76966237} 11... Na6 12. Bg5
{-3.30/21 31501 29871718} 12... Qd7 13. f3 {-3.23/25 53971 51118053} 13... Rfb8
14. b3 {-3.41/24 78101 78536522} 14... Nc7 15. a4 {-3.26/24 28657 26206782}
15... Na6 16. Rc1 {-3.39/23 66534 60858709} 16... c4 17. bxc4 {-3.04/22 11648
10463149} 17... Nc5 18. Nb5 {-3.23/22 20951 20848844} 18... Rxa4 19. h3
{-3.52/25 22705 52609633} 19... Rba8 20. Kh2 {-3.66/24 69068 70770547} 20... Ne8
21. Ned4 {-3.75/25 47334 45215127} 21... Nc7 22. Nc6 {-2.73/21 21651 21860901}
22... Ra1 23. Nxe7+ {-1.95/22 16031 13640099} 23... Kf8 24. Nc6 {-1.55/23 26569
25043489} 24... Nxb5 25. cxb5 {-1.76/22 13179 13461152} 25... Nb7 26. b6
{-1.54/23 60379 57974208} 26... Rxc1 27. Qxc1 {-1.78/21 11965 12216867} 27...
Qc8 28. Be7+ {-0.99/23 34718 32187113} 28... Ke8 29. Bg5 {-0.97/22 10 3386689}
29... Na5 30. Bf4 {+0.00/25 9349 10521963} 30... Be5 31. Bxe5 {+6.05/22 6849
8824377} 31... dxe5 32. Qc5 {+7.60/24 9774 12405909} 32... Nxc6 33. dxc6
{+8.78/25 7811 12033456} 33... Rb8 34. Qxe5+ {+9.24/27 10 68960281} 34... Kf8
35. b7 {+9.30/25 3128 18417415} 35... Rxb7 36. Qh8+ {M+20/19 14170 23635107}
36... Ke7 37. Qxc8 {M+16/20 10 21184787} 37... Rb1 38. Qd7+ {M+11/16 1602
2947498} 38... Kf8 39. Qd3 {M+10/19 10 3434493} 39... Rc1 40. Qa3+ {M+8/21 1736
3608105} 40... Ke8 41. Qxc1 {M+7/23 1648 3482505} 41... Kd8 42. Qd2+ {M+6/27 62
3374161} 42... Kc7 43. Qd7+ {M+5/69 10 8256463} 43... Kb6 44. c7 {M+4/99 10
531228} 44... Kc5 45. c8=Q+ {M+3/99 66 115838} 45... Kb4 46. Qdb7+ {M+2/99 10
4737} 46... Ka5 47. Qca8# {M+1/99 29 4158} 1-0
[/pgn]


After three rounds, the results (all from my perspective)

Crafty, a1 rook: 1/2, 1
Komodo, h1 rook: 0, 1/2, 0
Lc0, a1 rook: 0, 1
Stockfish, h1 rook: 0, 1
Crafty, h1 rook: 1
Komodo, a1 rook: 0, 1
Lc0, h1 rook: 1
Stockfish, a1 rook: 1

I have dropped 6 points out of 14 games.
Taking the a1 rook: 2.5 dropped points in 7 games.
Taking the h1 rook: 3.5 dropped points in 7 games.
Against Crafty: 0.5 dropped points in 3 games.
Against Komodo: 3.5 dropped points in 5 games.
Against Lc0: 1 dropped point in 3 games.
Against Stockfish: 1 dropped point in 3 games.

Next game, against again Komodo removing the h1 rook, after I recover from this heartbreaking loss.

Greetings.

[Fritz 0]

I don't use any GUI and i testing engines with 'chess' python library, I guess some GUI setting slow down the game with useless animation move.
I can confirm, in my slow 2Ghz laptop, a single game Dragon2.6 vs Dragon2.6 about half second
Thread: 1
UCI elo: 2000
Games: 100
Total time: 49s
Average: 0.49s

I have also try with 2300 UCI elo setting:
100 games in 76.8 seconds

And level 17 komodo 12.1.1 against itself:
100 games in 65.7 seconds

I haven't said it before, I always set 'varierty' to 10

You mean, you set "variety" to 10 AND a skill level? Do parameters even work with the levels, or are they set automatically with a certain level?

[Marcus9]

I don't use any GUI and i testing engines with 'chess' python library, I guess some GUI setting slow down the game with useless animation move.
I can confirm, in my slow 2Ghz laptop, a single game Dragon2.6 vs Dragon2.6 about half second
Thread: 1
UCI elo: 2000
Games: 100
Total time: 49s
Average: 0.49s

I have also try with 2300 UCI elo setting:
100 games in 76.8 seconds

And level 17 komodo 12.1.1 against itself:
100 games in 65.7 seconds

I haven't said it before, I always set 'varierty' to 10

You mean, you set "variety" to 10 AND a skill level? Do parameters even work with the levels, or are they set automatically with a certain level?

Yes, but skill levels already have inherent variety

[lkaufman]

I don't use any GUI and i testing engines with 'chess' python library, I guess some GUI setting slow down the game with useless animation move.
I can confirm, in my slow 2Ghz laptop, a single game Dragon2.6 vs Dragon2.6 about half second
Thread: 1
UCI elo: 2000
Games: 100
Total time: 49s
Average: 0.49s

I have also try with 2300 UCI elo setting:
100 games in 76.8 seconds

And level 17 komodo 12.1.1 against itself:
100 games in 65.7 seconds

I haven't said it before, I always set 'varierty' to 10

You mean, you set "variety" to 10 AND a skill level? Do parameters even work with the levels, or are they set automatically with a certain level?

Yes, but skill levels already have inherent variety

It seems that the variety option has no effect when an Elo level is set; the Elo level has a defined variety value which will override whatever value you set for variety.