Knight odds match

[lkaufman]

Hi larry.
We should make a section on bet.

This is all just publicity of Komodo, which is a commercial product.
It has no other value.
Please!!

The publicity is what justifies the cost of running these matches. I think it is of some general interest to learn whether the gap between the top engine and a real (but borderline) chess master has reached knight odds or not. It is also useful for people who want to play their own engine (whether Komodo or another one) at some handicap. I (like many others) found the Nakamura vs. Stockfish handicap match to be interesting too, and I encourage others to try such matches with other engines vs. human masters.

Then the opponent of your engine,
¿do have a salary? ¿you pay that salary?

Yes, we pay the opponent for every win and draw. For FMs (who are amateurs), it's small amount. For example in this knight odds match we will pay $25 per win and just $5 per draw. But for GMs it is substantial. Movsesian got $100 base fee plus $100 per draw and $250 per win. It only cost us $400 (plus expenses) since he only scored three draws, but it could have been over a thousand dollars. I don't know whether these matches increase sales enough to offset these costs, but even if it's a slight net loss we're glad to do it.

[Peter Berger]

These odd matches are interesting and incredibly hard to judge IMHO.

The rating you give for the opponent is pretty close to what I could personally do in my best years, and I am sure that the best I could ever have got is a lucky draw given reasonable settings for the engine.

The FM player shouldn't be able to control the tactics IMHO and Komodo will toast him IMHO.

This is what an incredibly strong engine like Crafty can do against latest Stockfish on my computer in a random test game:

[Event "45/15 Fischer 2CPU ponder=off"]
[Site "Testgame Peter's comp"]
[Date "2015.09.26"]
[White "Stockfish 190915"]
[Black "Crafty 24.1"]
[Result "1/2-1/2"]
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKB1R w KQkq -"]
[SetUp "1"]

1. d4 d5 2. c4 e6 3. Nc3 Nc6 4. e3 Nf6 5. a3 Be7 6. Bd3 O-O
7. O-O dxc4 8. Bxc4 b6 9. b4 Bb7 10. Qe2 a5 11. b5 Na7 12. Bb2
Rc8 13. Rfd1 Qd7 14. e4 c6 15. bxc6 Qxc6 16. Bd3 Rfd8 17. Rac1
Qd7 18. Qe3 Kf8 19. Bf1 b5 20. d5 Bc5 21. Qg3 e5 22. Re1 Bd4
23. Rb1 Qc7 24. Nd1 Qc2 25. Bd3 Qc5 26. Bc1 h6 27. Ne3 Nh5 28. Qf3
Nf4 29. Bf1 Qb6 30. Bd2 Kg8 31. g3 Ng6 32. Nc4 Qa6 33. Nxa5
Rd6 34. Qd1 Rf6 35. Re2 Ba8 36. Bb4 Bb6 37. Qe1 Rc7 38. Nb3
Qc8 39. Ba5 Bxa5 40. Nxa5 Nf8 41. Reb2 Nd7 42. a4 bxa4 43. Qb4
Rb6 44. Qxa4 Rxb2 45. Rxb2 Kh7 46. Nc4 Nc5 47. Qc2 Bb7 48. Qb1
f6 49. Qa2 Nxe4 50. d6 Rxc4 51. Qxa7 Nxd6 52. Rxb7 Qxb7 53. Qxb7
Nxb7 54. Bxc4 Nd6 55. Bb3 f5 56. h4 g5 57. hxg5 hxg5 58. Kf1
Kg6 59. Ke2 g4 60. Ke3 Kg5 61. Bd5 f4+ 62. gxf4+ exf4+ 63. Kd4
g3 64. fxg3 fxg3 65. Ke3 Kh4 66. Bg2 Kg4 67. Ke2 Nf5 68. Kf1
Ne3+ 69. Kg1 Kf4 70. Bc6 Kf5 71. Bb7 Ke5 72. Bc6 Kf4 73. Bb7
Kf5 74. Bc6 Kg6 75. Bh1 Kh5 76. Bg2 Kg5 77. Bf3 Kh6 78. Be4
Kg7 79. Bc6 Kf6 80. Kh1 Ke6 81. Bb7 Kd6 82. Kg1 Kd7 83. Be4
Kc8 84. Bc6 Kc7 85. Be4 Kb6 86. Bf3 Kc5 87. Be4 Kd4 88. Bb7
Kd3 89. Kh1 Ke2 90. Kg1 Ke1 91. Bf3 Kd2 92. Bb7 Kc3 93. Kh1
Kc2 94. Be4+ Kd2 95. Kg1 Kd1 96. Kh1 Kc1 97. Kg1 Kb2 98. Kh1
Kb3 99. Kg1 Kc3 100. Bh1 Kc4 101. Be4 Kb3 102. Kh1 Kb2 103. Kg1
Ka1 104. Kh1 Ka2 105. Kg1 Ka3 106. Bb7 Ka2 107. Be4 Nc4 108. Bd5
Kb3 109. Kg2 Kc3 110. Bxc4 Kxc4 111. Kxg3 Kb3 1/2

[lkaufman]

These odd matches are interesting and incredibly hard to judge IMHO.

The rating you give for the opponent is pretty close to what I could personally do in my best years, and I am sure that the best I could ever have got is a lucky draw given reasonable settings for the engine.

The FM player shouldn't be able to control the tactics IMHO and Komodo will toast him IMHO.

This is what an incredibly strong engine like Crafty can do against latest Stockfish on my computer in a random test game:

[Event "45/15 Fischer 2CPU ponder=off"]
[Site "Testgame Peter's comp"]
[Date "2015.09.26"]
[White "Stockfish 190915"]
[Black "Crafty 24.1"]
[Result "1/2-1/2"]
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKB1R w KQkq -"]
[SetUp "1"]

1. d4 d5 2. c4 e6 3. Nc3 Nc6 4. e3 Nf6 5. a3 Be7 6. Bd3 O-O
7. O-O dxc4 8. Bxc4 b6 9. b4 Bb7 10. Qe2 a5 11. b5 Na7 12. Bb2
Rc8 13. Rfd1 Qd7 14. e4 c6 15. bxc6 Qxc6 16. Bd3 Rfd8 17. Rac1
Qd7 18. Qe3 Kf8 19. Bf1 b5 20. d5 Bc5 21. Qg3 e5 22. Re1 Bd4
23. Rb1 Qc7 24. Nd1 Qc2 25. Bd3 Qc5 26. Bc1 h6 27. Ne3 Nh5 28. Qf3
Nf4 29. Bf1 Qb6 30. Bd2 Kg8 31. g3 Ng6 32. Nc4 Qa6 33. Nxa5
Rd6 34. Qd1 Rf6 35. Re2 Ba8 36. Bb4 Bb6 37. Qe1 Rc7 38. Nb3
Qc8 39. Ba5 Bxa5 40. Nxa5 Nf8 41. Reb2 Nd7 42. a4 bxa4 43. Qb4
Rb6 44. Qxa4 Rxb2 45. Rxb2 Kh7 46. Nc4 Nc5 47. Qc2 Bb7 48. Qb1
f6 49. Qa2 Nxe4 50. d6 Rxc4 51. Qxa7 Nxd6 52. Rxb7 Qxb7 53. Qxb7
Nxb7 54. Bxc4 Nd6 55. Bb3 f5 56. h4 g5 57. hxg5 hxg5 58. Kf1
Kg6 59. Ke2 g4 60. Ke3 Kg5 61. Bd5 f4+ 62. gxf4+ exf4+ 63. Kd4
g3 64. fxg3 fxg3 65. Ke3 Kh4 66. Bg2 Kg4 67. Ke2 Nf5 68. Kf1
Ne3+ 69. Kg1 Kf4 70. Bc6 Kf5 71. Bb7 Ke5 72. Bc6 Kf4 73. Bb7
Kf5 74. Bc6 Kg6 75. Bh1 Kh5 76. Bg2 Kg5 77. Bf3 Kh6 78. Be4
Kg7 79. Bc6 Kf6 80. Kh1 Ke6 81. Bb7 Kd6 82. Kg1 Kd7 83. Be4
Kc8 84. Bc6 Kc7 85. Be4 Kb6 86. Bf3 Kc5 87. Be4 Kd4 88. Bb7
Kd3 89. Kh1 Ke2 90. Kg1 Ke1 91. Bf3 Kd2 92. Bb7 Kc3 93. Kh1
Kc2 94. Be4+ Kd2 95. Kg1 Kd1 96. Kh1 Kc1 97. Kg1 Kb2 98. Kh1
Kb3 99. Kg1 Kc3 100. Bh1 Kc4 101. Be4 Kb3 102. Kh1 Kb2 103. Kg1
Ka1 104. Kh1 Ka2 105. Kg1 Ka3 106. Bb7 Ka2 107. Be4 Nc4 108. Bd5
Kb3 109. Kg2 Kc3 110. Bxc4 Kxc4 111. Kxg3 Kb3 1/2

I have pointed out before that for some reason Komodo is able to give surprisingly large handicaps to very strong unrelated engines, but it has trouble giving handicaps to weaker versions of itself or to itself with much less time. My theory is that when two engines have widely differing evals, they will often both head for some complicated variation that they each think favors them, but with similar evals they tend to keep things simple. Humans are smart enough to avoid unnecessary complications when well ahead, so they do much better than unrelated engines of the same elo. For example, Komodo lost badly giving c2 + f2 two pawn handicap to a GM, but would crush any unrelated engine of around his elo (2455) at this handicap.
This doesn't mean that you are wrong, only that the fact that the top two engines can do well giving big handicaps to much weaker (but still GM+ level) engines doesn't tell us how humans will do.

[Peter Berger]

[quote="lkaufman"
I have pointed out before that for some reason Komodo is able to give surprisingly large handicaps to very strong unrelated engines, but it has trouble giving handicaps to weaker versions of itself or to itself with much less time. My theory is that when two engines have widely differing evals, they will often both head for some complicated variation that they each think favors them, but with similar evals they tend to keep things simple. Humans are smart enough to avoid unnecessary complications when well ahead, so they do much better than unrelated engines of the same elo. For example, Komodo lost badly giving c2 + f2 two pawn handicap to a GM, but would crush any unrelated engine of around his elo (2455) at this handicap.
This doesn't mean that you are wrong, only that the fact that the top two engines can do well giving big handicaps to much weaker (but still GM+ level) engines doesn't tell us how humans will do.[/quote]

Maybe you don't know yet ( I haven't read it here so far, and I am in a useful mode today).

A great engine to use in engine tests for odds match is ProDeo 1.86 - you simply can't give knight odds to it, as it will toast you. It will win against latest Stockfish in like 100% of the games on my computer, although it clearly plays at about Crafty level in ordinary chess games.

I hope this is a useful piece of information for you.

Peter

[lkaufman]

[quote="lkaufman"
I have pointed out before that for some reason Komodo is able to give surprisingly large handicaps to very strong unrelated engines, but it has trouble giving handicaps to weaker versions of itself or to itself with much less time. My theory is that when two engines have widely differing evals, they will often both head for some complicated variation that they each think favors them, but with similar evals they tend to keep things simple. Humans are smart enough to avoid unnecessary complications when well ahead, so they do much better than unrelated engines of the same elo. For example, Komodo lost badly giving c2 + f2 two pawn handicap to a GM, but would crush any unrelated engine of around his elo (2455) at this handicap.
This doesn't mean that you are wrong, only that the fact that the top two engines can do well giving big handicaps to much weaker (but still GM+ level) engines doesn't tell us how humans will do.

Maybe you don't know yet ( I haven't read it here so far, and I am in a useful mode today).

A great engine to use in engine tests for odds match is ProDeo 1.86 - you simply can't give knight odds to it, as it will toast you. It will win against latest Stockfish in like 100% of the games on my computer, although it clearly plays at about Crafty level in ordinary chess games.

I hope this is a useful piece of information for you.

Peter[/quote]

It's a bit too high rated for knight odds, especially at non-bullet time controls. At game in one minute I would expect it to lose some games to SF. What time limit did you test? Also, is it just a single core engine? If so it's not very interesting to use for this purpose. My tests showed that at bullet or near-bullet speeds Komodo giving knight odds is too strong for Nebula 2 but not strong enough for Arasan 17.5, all engines running four threads. One thing I observed is that Komodo scores better giving the g1 knight than giving the b1 knight.

[Laskos]

Maybe you don't know yet ( I haven't read it here so far, and I am in a useful mode today).

A great engine to use in engine tests for odds match is ProDeo 1.86 - you simply can't give knight odds to it, as it will toast you. It will win against latest Stockfish in like 100% of the games on my computer, although it clearly plays at about Crafty level in ordinary chess games.

I hope this is a useful piece of information for you.

Peter

There seems to be nothing special with ProDeo 1.86. I played at knight odds 2'+1'' games, Komodo managed 2 wins and 2 draws in 8 games. Against similarly rated Shredder 9.1, Komodo managed only 1 win and 1 draw in 8 games. These results seem to imply a 700 ELO points handicap, while self-games at time odds give much more and closer to real 1100 handicap at say 45'+15''.

[lkaufman]

So this time Komodo did worse than expected, scoring two draws only. If we had stated that draws counted as losses (as we did with FM Bolzoni) it would have been a tie match, since Meyer played almost as if they were that way due to the prize per win being 5x the prize per draw. It seems that it's very hard to find handicaps that are truly even for any two players, one side always wins big so far.