Komodo-Dragon-2 vs Stockfish 14 at knight odss

[lkaufman]

So bishops are indeed worth more than knights (at least when bishop pair is broken for the side losing the bishop), no surprise there. But it is interesting that Stockfish lost much more than Komodo from this, SF score was nearly cut in half going from knight odds to bishop odds! Regarding rook odds, it is roughly a class (200 elo) larger handicap than knight odds, so a field in the 2500 to 2530 range for opponents might be more balanced, but anyway it will be interesting.

At the moment I am doing queen odds, just to be complete.

The rook epd is not good, see:

Code: Select all

rnbqkb1r/ppp1pppp/3p4/3nP3/3P4/8/PPP2PPP/1NBQKBNR w KQkq - 0 4; v=-526
r1bqkb1r/pppnpppp/3p1n2/8/2PP4/2N5/PP2PPPP/2BQKBNR w KQkq - 2 4; v=-529
rnbqk1nr/ppp1ppbp/3p2p1/8/3PP3/5N2/PPP2PPP/1NBQKB1R w KQkq - 0 4; v=-536
r1bqkb1r/ppp1pppp/2n2n2/3p4/2PP4/4P3/PP3PPP/1NBQKBNR w KQkq - 1 4; v=-538

Castling flags are wrong and positions are ignored by cute.

Does somebody has a good rook odds epd of (at least) 100 positions?

At queen odds, if you keep the same field, I don't think that either Dragon or Stockfish will get more than a few draws in 700 games, maybe not even that. Probably you need engines about a thousand elo lower than these for a reasonably close match at queen odds even at this bullet tc.

In order to get an even score with Dragon2, I tested more than 1000 games at an average of 1 seconds per move with these field with Queen Odds

462 Casper rev4 64-bit 1579 +24 −24 50.2% −2.6 23.9% 624
60.5%
463 PolarChess 1.3 1574 +25 −25 49.3% +3.5 16.9% 629
56.3%
464‑465 Darky 0.5d 1571 +24 −24 43.5% +56.2 20.8% 677
49.7%
464‑465 Storm 0.6 1571 +21 −21 38.3% +95.3 15.0% 925
74.3%
466 Damas 9 1560 +25 −25 47.8% +17.1 17.6% 626
76.1%
467 IQ23.003 1547 +21 −21 33.6% +118.6 32.6% 854
72.6%
468 Cicada 0.1 64-bit 1536 +24 −25 44.1% +48.6 19.7% 636
65.6%


In order to get an even score with Dragon2, I tested more than 1000 games at an average of 1 seconds per move with these field with Rook Odds

357 Kurt 0.9.2.2 64-bit 2166 +21 −21 47.7% +18.0 24.2% 828
56.9%
358 ProChess 1.02AD 2164 +19 −19 48.4% +6.3 22.1% 1043
51.8%
359 Chesley r323 64-bit 2163 +22 −22 48.1% +14.4 20.8% 804
51.3%
360 Micah 1.0 64-bit 2162 +25 −25 44.0% +45.9 25.4% 566
73.2%
361 KnockOut 0.7.1 2153 +15 −15 51.1% −8.0 26.6% 1574
81.4%

Something seems wrong with your rook odds results. You are saying that engines with CCRL blitz ratings in the 2160s scored about 47% with Dragon2 at rook odds at 40 moves in 40 seconds or something similar? Is this on one thread? This seems totally out of line with the performance ratings Ed reports at rook odds for Dragon2. He was getting 2545 at rook odds, you are getting under 2200. This is crazy.

Something seems to be wrong with Ed's results because looking at the games fruit2.1 get clearly bigger depths at the same time with my hardware
and I do not have a fast hardware.

I did not look only at games when fruit lost but also at games when fruit won.

I see that it is probably my mistake because arena simply wrote wrong times when I copied a game to arena to look at it.

It could also be the Ed is using more than 1 thread whereas I am only using 1 thread, to make it fair to the inferior engines competing against either Komodo Dragon or Stockfish 14. Also I use time per move 1 second and that could yield a different result, even if it average 1 second per move, at TC Time control : 40/40 .

I'm reasonably certain that Ed is using just one thread for all engines, he would tell us otherwise. It's true that one second per move is not exactly the same as 40 moves in 40 seconds, but results wouldn't differ by hundreds of Elo, maybe by twenty or thirty. I suspect that the GUI you are using does not award the increment before the move is played, so 0" + 1" time setting might result in White losing many games on time on the first move, which would explain everything. If I'm right, you should see a lot of time forfeits and one move games. If that's not so, we have a mystery to solve.

[Rebel]

BISHOP ODDS - 2500 elo engines

Komodo Dragon 2

Code: Select all

BISHOP odds match Komodo Dragon 2 vs a pool of 2500 elo rated engines
Time Control : Time control : 40/40
Games        : 600

Results from file all.pgn:

No. Name             Win Draw Loss Unf.  Score Games       %
------------------------------------------------------------
  1 Komodo-Dragon 2 +435  =46  -61   *0  458.0   542   84.5%
  2 Monolith 0.3     +11  =14  -66   *0   18.0    91   19.8%
  3 Foxsee 7.20.1    +12   =9  -69   *0   16.5    90   18.3%
  4 Nalwald 1.8.1    +14   =5  -71   *0   16.5    90   18.3%
  5 Loki 3.5.0       +11   =9  -71   *0   15.5    91   17.0%
  6 Marvin 2.0        +9   =5  -76   *0   11.5    90   12.8%
  7 CT800 1.43        +4   =4  -82   *0    6.0    90    6.7%

Total Games:     542
White Wins:      165 (30.4%)
Black Wins:      331 (61.1%)
Draws:            46 (8.5%)
Unfinished:        0 (0.0%)

Estimated ratings for this elo 2500 pool

   # PLAYER             :  RATING  POINTS  PLAYED   (%)
   1 Komodo-Dragon 2    :  2763.6   458.0     542    85
   2 Monolith 0.3       :  2518.3    18.0      91    20
   3 Nalwald 1.8.1      :  2501.8    16.5      90    18
   4 Foxsee 7.20.1      :  2501.8    16.5      90    18
   5 Loki 3.5.0         :  2486.2    15.5      91    17
   6 Marvin 2.0         :  2427.0    11.5      90    13
   7 CT800 1.43         :  2301.2     6.0      90     7

Stockfish 14

Code: Select all

BISHOP odds match Stockfish 14 vs a pool of 2500 elo rated engines
Time Control : Time control : 40/40
Games        : 600

Results from file all.pgn:

No. Name           Win Draw Loss Unf.  Score Games       %
----------------------------------------------------------
  1 Stockfish 14  +381  =28 -191   *0  395.0   600   65.8%
  2 Loki 3.5.0     +43   =1  -56   *0   43.5   100   43.5%
  3 Monolith 0.3   +32   =9  -59   *0   36.5   100   36.5%
  4 Nalwald 1.8.1  +34   =3  -63   *0   35.5   100   35.5%
  5 Foxsee 7.20.1  +33   =4  -63   *0   35.0   100   35.0%
  6 Marvin 2.0     +27   =4  -69   *0   29.0   100   29.0%
  7 CT800 1.43     +22   =7  -71   *0   25.5   100   25.5%

Total Games:     600
White Wins:       81 (13.5%)
Black Wins:      491 (81.8%)
Draws:            28 (4.7%)
Unfinished:        0 (0.0%)

Estimated ratings for this elo 2500 pool

   # PLAYER           :  RATING  POINTS  PLAYED   (%)
   1 Stockfish 14     :  2600.1   395.0     600    66
   2 Loki 3.5.0       :  2554.3    43.5     100    44
   3 Monolith 0.3     :  2503.1    36.5     100    37
   4 Nalwald 1.8.1    :  2495.5    35.5     100    36
   5 Foxsee 7.20.1    :  2491.6    35.0     100    35
   6 Marvin 2.0       :  2443.2    29.0     100    29
   7 CT800 1.43       :  2412.2    25.5     100    26

Stockfish : 65.8%
Komodo : 84.5%

Next, rook odds.

[Rebel]

It could also be the Ed is using more than 1 thread whereas I am only using 1 thread, to make it fair to the inferior engines competing against either Komodo Dragon or Stockfish 14. Also I use time per move 1 second and that could yield a different result, even if it average 1 second per move, at TC Time control : 40/40 .

Will you please stay out of my thread?

Thank you.

[Uri Blass]

It could also be the Ed is using more than 1 thread whereas I am only using 1 thread, to make it fair to the inferior engines competing against either Komodo Dragon or Stockfish 14. Also I use time per move 1 second and that could yield a different result, even if it average 1 second per move, at TC Time control : 40/40 .

Will you please stay out of my thread?

Thank you.

1)I see that I posted something in the Prodeo forum and you deleted it.
No complaining about it because I did a mistake to think
something may be wrong with the hardware and edited it to correct it explaining my mistake and it is ok from my point of view if you deleted it to have a clean thread there.
I think maybe you confuse between me and Chessqueen and you thought that chessqueen posted in the prodeo forum.

2)For the question for the reason of different results I can say that I think that the main difference except different time control is simply that
You use different opening positions and not simply the opening position without a queen or without another piece.

I do not know if different opening position help the weaker side or not but it is not normal odd games from my point of view.
Opening book for the odd giver to avoid repetition of the same moves again and again make sense but in this case
I think that only the odd giver should get an opening book when the opponent engine should not get a book.

[Rebel]

It could also be the Ed is using more than 1 thread whereas I am only using 1 thread, to make it fair to the inferior engines competing against either Komodo Dragon or Stockfish 14. Also I use time per move 1 second and that could yield a different result, even if it average 1 second per move, at TC Time control : 40/40 .

Will you please stay out of my thread?

Thank you.

1)I see that I posted something in the Prodeo forum and you deleted it.
No complaining about it because I did a mistake to think
something may be wrong with the hardware and edited it to correct it explaining my mistake and it is ok from my point of view if you deleted it to have a clean thread there.
I think maybe you confuse between me and Chessqueen and you thought that chessqueen posted in the prodeo forum.

Uri, that request was absolutely not directed at you. Secondly I deleted that post because 1) you said it was a mistake and 2) I want the copy thread remain to be clear because as I already indicated when the job is done I want to make a web page of the findings and meanwhile this thread is a maze.

[lkaufman]

2)For the question for the reason of different results I can say that I think that the main difference except different time control is simply that
You use different opening positions and not simply the opening position without a queen or without another piece.

I do not know if different opening position help the weaker side or not but it is not normal odd games from my point of view.
Opening book for the odd giver to avoid repetition of the same moves again and again make sense but in this case
I think that only the odd giver should get an opening book when the opponent engine should not get a book.

Of course using an odds opening book is not the same as playing from the initial odds position, but it is not dramatically different based on my testing. However a lot depends on how the book is trimmed to the desired size (in this case one hundred positions). I always select the middle N positions from the Chris W book, as they are ordered from smallest to largest advantage. Random choice would work too. But if you just use the first 100 positions from the list, that would effectively reduce the handicap by something like a pawn, maybe 300 elo or so. So it's a question for Ed, how did you select the 100 positions from the entire list? If they were chosen by some unbiased method, then the use of the book could not explain the disparity of results, it would more likely be due to a GUI issue when no base time is specified. Ed's results seem right to me, consistent with my own.

[Chessqueen]

2)For the question for the reason of different results I can say that I think that the main difference except different time control is simply that
You use different opening positions and not simply the opening position without a queen or without another piece.

I do not know if different opening position help the weaker side or not but it is not normal odd games from my point of view.
Opening book for the odd giver to avoid repetition of the same moves again and again make sense but in this case
I think that only the odd giver should get an opening book when the opponent engine should not get a book.

Of course using an odds opening book is not the same as playing from the initial odds position, but it is not dramatically different based on my testing. However a lot depends on how the book is trimmed to the desired size (in this case one hundred positions). I always select the middle N positions from the Chris W book, as they are ordered from smallest to largest advantage. Random choice would work too. But if you just use the first 100 positions from the list, that would effectively reduce the handicap by something like a pawn, maybe 300 elo or so. So it's a question for Ed, how did you select the 100 positions from the entire list? If they were chosen by some unbiased method, then the use of the book could not explain the disparity of results, it would more likely be due to a GUI issue when no base time is specified. Ed's results seem right to me, consistent with my own.

I just realized that I was running 4 Applications in the background when I did the test, so re started the Rook Odds from scratch and after 500 games Komodo Dragon2 is winning the field by the same margin that Ed reported, I will let it run until the end but so far it is consistent with Ed result.

[Rebel]

ROOK ODDS - 2500 elo engines

Code: Select all

ROOK odds match Stockfish 14 vs a pool of 2500 elo rated engines
Time Control : Time control : 40/40
Games        : 600

Results from file all.pgn:

No. Name           Win Draw Loss Unf.  Score Games       %
----------------------------------------------------------
  1 Stockfish 14  +416  =25 -159   *0  428.5   600   71.4%
  2 Loki 3.5.0     +33   =1  -66   *0   33.5   100   33.5%
  3 Monolith 0.3   +28  =11  -61   *0   33.5   100   33.5%
  4 Marvin 2.0     +30   =4  -66   *0   32.0   100   32.0%
  5 Foxsee 7.20.1  +25   =6  -69   *0   28.0   100   28.0%
  6 Nalwald 1.8.1  +27   =2  -71   *0   28.0   100   28.0%
  7 CT800 1.43     +16   =1  -83   *0   16.5   100   16.5%

Total Games:     600
White Wins:      116 (19.3%)
Black Wins:      459 (76.5%)
Draws:            25 (4.2%)
Unfinished:        0 (0.0%)

Estimated ratings for this elo 2500 pool

   # PLAYER           :  RATING  POINTS  PLAYED   (%)
   1 Stockfish 14     :  2641.1   428.5     600    71
   2 Loki 3.5.0       :  2520.9    33.5     100    34
   3 Monolith 0.3     :  2520.9    33.5     100    34
   4 Marvin 2.0       :  2509.0    32.0     100    32
   5 Nalwald 1.8.1    :  2475.6    28.0     100    28
   6 Foxsee 7.20.1    :  2475.6    28.0     100    28
   7 CT800 1.43       :  2356.9    16.5     100    17

Komodo Dragon 2

Code: Select all

ROOK odds match Komodo Dragon 2 vs a pool of 2500 elo rated engines
Time Control : Time control : 40/40
Games        : 600

Results from file all.pgn:

No. Name             Win Draw Loss Unf.  Score Games       %
------------------------------------------------------------
  1 Komodo-Dragon 2 +447  =42 -111   *0  468.0   600   78.0%
  2 Loki 3.5.0       +30   =6  -64   *0   33.0   100   33.0%
  3 Marvin 2.0       +22   =6  -72   *0   25.0   100   25.0%
  4 Monolith 0.3     +16  =17  -67   *0   24.5   100   24.5%
  5 Nalwald 1.8.1    +18   =6  -76   *0   21.0   100   21.0%
  6 Foxsee 7.20.1    +18   =4  -78   *0   20.0   100   20.0%
  7 CT800 1.43        +7   =3  -90   *0    8.5   100    8.5%

Total Games:     600
White Wins:      147 (24.5%)
Black Wins:      411 (68.5%)
Draws:            42 (7.0%)
Unfinished:        0 (0.0%)

Estimated ratings for this elo 2500 pool

   # PLAYER             :  RATING  POINTS  PLAYED   (%)
   1 Komodo-Dragon 2    :  2700.8   468.0     600    78
   2 Loki 3.5.0         :  2576.7    33.0     100    33
   3 Marvin 2.0         :  2508.2    25.0     100    25
   4 Monolith 0.3       :  2503.5    24.5     100    25
   5 Nalwald 1.8.1      :  2468.6    21.0     100    21
   6 Foxsee 7.20.1      :  2457.8    20.0     100    20
   7 CT800 1.43         :  2284.3     8.5     100     9

Komodo : 78.0%
Stockfish : 71.4%

Next, queen odds.

[Rebel]

2)For the question for the reason of different results I can say that I think that the main difference except different time control is simply that
You use different opening positions and not simply the opening position without a queen or without another piece.

I do not know if different opening position help the weaker side or not but it is not normal odd games from my point of view.
Opening book for the odd giver to avoid repetition of the same moves again and again make sense but in this case
I think that only the odd giver should get an opening book when the opponent engine should not get a book.

Of course using an odds opening book is not the same as playing from the initial odds position, but it is not dramatically different based on my testing. However a lot depends on how the book is trimmed to the desired size (in this case one hundred positions). I always select the middle N positions from the Chris W book, as they are ordered from smallest to largest advantage. Random choice would work too. But if you just use the first 100 positions from the list, that would effectively reduce the handicap by something like a pawn, maybe 300 elo or so. So it's a question for Ed, how did you select the 100 positions from the entire list? If they were chosen by some unbiased method, then the use of the book could not explain the disparity of results, it would more likely be due to a GUI issue when no base time is specified. Ed's results seem right to me, consistent with my own.

I am using the first 100 positions of Chris tool.

[lkaufman]

2)For the question for the reason of different results I can say that I think that the main difference except different time control is simply that
You use different opening positions and not simply the opening position without a queen or without another piece.

I do not know if different opening position help the weaker side or not but it is not normal odd games from my point of view.
Opening book for the odd giver to avoid repetition of the same moves again and again make sense but in this case
I think that only the odd giver should get an opening book when the opponent engine should not get a book.

Of course using an odds opening book is not the same as playing from the initial odds position, but it is not dramatically different based on my testing. However a lot depends on how the book is trimmed to the desired size (in this case one hundred positions). I always select the middle N positions from the Chris W book, as they are ordered from smallest to largest advantage. Random choice would work too. But if you just use the first 100 positions from the list, that would effectively reduce the handicap by something like a pawn, maybe 300 elo or so. So it's a question for Ed, how did you select the 100 positions from the entire list? If they were chosen by some unbiased method, then the use of the book could not explain the disparity of results, it would more likely be due to a GUI issue when no base time is specified. Ed's results seem right to me, consistent with my own.

I am using the first 100 positions of Chris tool.

Maybe he already randomized them; the original book from him was in order of eval, but you may have something later. Maybe I'll check when I have more time.