Stockfish handicap match - PART II -

Rebel · Post by **Rebel** » Fri May 14, 2021 11:31 am

About a year ago I did the Stockfish 11 handicap match on time control with stunning results. Times have changed, NNUE is the new normal. Time to give Stockfish 13 an even harder time, a handicap match based on material. We feed the beast with 7 positions down in material, invite 10 engines in the elo range ~2400-3200 and watch the slaughter.

Code: Select all

. Stockfish 13 : Handicap Match

. Tournament   : Gauntlet
. Time Control : 40 moves in 10 minutes repeating
. Games        : 7 games each engine

. Opponents    : elo range ~2400-3200
. Arasan 2.22
. Cheng 4.41  
. Fruit 2.1  
. Ethereal 12.75    
. Foxsee 7.8       
. Komodo 3          
. ProDeo 3.1        
. Senpai 1.0        
. Texel 1.7         
. Vajolet2 2.8    

. Software     : Arena 3.0
. Hash table   : Default setting engines

. Openings     : handicap.epd
. rnbqkbnr/pppppppp/8/8/8/8/PPPP1PPP/RNBQKBNR w KQkq - c0 "-wpe2";
. rnbqkbnr/pppppppp/8/8/8/8/PPP2PPP/RNBQKBNR w KQkq - c0 "-wpe2 | wpd2";
. rnbqkbnr/pppppppp/8/8/8/8/PP3PPP/RNBQKBNR w KQkq - c0 "-wpe2 | wpd2 | wpc2";
. rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RN1QKBNR w KQkq - c0 "-wbc1";
. rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/R1BQKBNR w KQkq - c0 "-wnb1";
. rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/1NBQKBNR w Kkq - c0 "-wra1";
. rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNB1KBNR w KQkq - c0 "-wqd1"'

Live broadcast at - http://rebel13.nl/pgn4web-3.05/live-test.html

Current ranking:

Code: Select all

                             Result Table

     Name         Gms  :  Win  : Draw  : Lose  :  Pts   :    S-B

SF13                3  :   1+  :   2=  :   0-  :   2.0  :     0.50
Arasan_2.22         1  :   0+  :   1=  :   0-  :   0.5  :     1.00
Cheng_4.41          1  :   0+  :   1=  :   0-  :   0.5  :     1.00
Fruit 2.1           1  :   0+  :   0=  :   1-  :   0.0  :     0.00

Games played 3, games to go 67

Rebel · Post by **Rebel** » Fri May 14, 2021 11:40 am

Place your bets if SF13 in this setting can keep a 50% score after the 70 games are finished.

I say NO.

My bet is 37%.

RubiChess · Post by **RubiChess** » Fri May 14, 2021 1:13 pm

So the last round SF13 will still play 40/10 while the opponents play 40/100? Can get long matches but at 40/10 SF13 will win against most of the opponents regardless how much time they have. So my bet goes SF13 >50%. Maybe even >60%.

Edit: I just realized that it is not only the time handicap but a missing pawn

But I keep my bet.
Edit2: ... and missing pieces, even queen in the later games... well... I keep my bet > 50%

Guenther · Post by **Guenther** » Fri May 14, 2021 1:25 pm

I did once a kind of similar tests with the strongest available entity at that time, which was asmFish 2017-08-05.

I let it play all permutations of 3P and even 4P (not finished) handicaps vs. AnMon 5.75 (2537 at CCRL Blitz) at 40/180 with W and B.
(asmfish with max contempt 100)

For this I used a tool, which Sven once did for me for creating the possible handicap FEN permutations automatically.
May be you find it useful too for some experiments.
http://www.talkchess.com/forum3/viewtopic.php?t=61003

The results were:

Code: Select all

RWBC CAPPUCCINO 2017 - Big Odds Test

3P Handicap + White
                                          
1   asmFishW_2017-08-05_base        +265  +42/=8/-6 82.14%   46.0/56
2   AnMon 5.75                      -265  +6/=8/-42 17.86%   10.0/56

3P Handicap + Black
                                          
1   asmFishW_2017-08-05_base        +109  +32/=9/-15 65.18%   36.5/56
2   AnMon 5.75                      -109  +15/=9/-32 34.82%   19.5/56

4P Handicap + White
                                          
1   asmFishW_2017-08-05_base        +191  +49/=7/-14 75.00%   52.5/70
2   AnMon 5.75                      -191  +14/=7/-49 25.00%   17.5/70

4P Handicap + Black (unfinished)
                                          12345678901
1   asmFishW_2017-08-05_base         +64  1½½10½10101  59.09%  6.5/11
2   AnMon 5.75                       -64  0½½01½01010  40.91%  5.5/11

Two examples for the last setup 4P +Black

[pgn][Event "RWBC"]
[Site "CAPPUCCINO"]
[Date "2017.08.10"]
[Round "4"]
[Number "186"]
[White "AnMon 5.75"]
[Black "asmFishW_2017-08-05_base (UCI2WB)"]
[Result "0-1"]
[TimeControl "40/180"]
[FEN "rnbqkbnr/3ppp1p/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
[SetUp "1"]

{--------------
r n b q k b n r
. . . p p p . p
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
P P P P P P P P
R N B Q K B N R
white to play
--------------}

1. e4 {+4.18/11} Bb7 {-4.02/20 8}
2. Nc3 {+4.23/11 5} Nf6 {-3.97/20 5}
3. f3 {+4.27/11 5} e5 {-4.13/21 10}
4. d4 {+4.31/11 3} exd4 {-4.00/20 5}
5. Qxd4 Nc6 {-4.03/18 0.7}
6. Qf2 {+4.41/10 5} Bb4 {-3.83/18 1.9}
7. Bd2 {+4.62/10 2.8} d5 {-4.24/17 0.8}
8. Bb5 {+4.53/10 10} d4 {-4.04/19 2.2}
9. Nd1 {+4.53/10 5} Bc5 {-4.49/20 11}
10. Qh4 {+4.78/11 4} Rg8 {-4.19/18 1.8}
11. g4 {+4.92/11 8} h5 {-3.74/18 2.7}
12. g5 {+4.61/11 5} Nd7 {-2.90/19 2.6}
13. Qxh5 {+4.97/11 4} Qb6 {-2.58/21 5}
14. Qh7 {+4.92/10 3} Ke7 {-2.85/22 12}
15. Bxc6 {+5.18/10 4} Bxc6 {-2.55/23 4}
16. Ne2 {+5.14/11 10} d3 {-3.39/21 3}
17. g6 {+4.69/10 10} Ne5 {-0.68/19 1.7}
18. Qh5 {+4.70/9 5} f6 {+0.36/20 5}
19. cxd3 {+4.02/9 12} Rh8 {+0.61/22 5}
20. Bh6 {+4.10/10 2.8} Nxd3+ {+0.76/18 0.8}
21. Kf1 {+4.56/10 4} Rag8 {+0.25/24 7}
22. a4 {+3.97/9 5} Ne5 {+2.14/19 2.1}
23. Rc1 {+2.49/9 5} Rxg6 {+2.38/20 2.2}
24. Be3 {+2.19/10 3} Bxe3 {+2.52/19 0.7}
25. Qxh8 {+0.45/10 4} Bxc1 {+2.40/23 5}
26. Qh7+ {-0.58/10 12} Kf8 {+2.13/23 2.1}
27. Qh8+ {+0.00/9 1.8} Kf7 {+2.59/23 3}
28. Qh7+ {+0.00/9 2.8} Rg7 {+2.77/23 9}
29. Qh5+ {+0.00/10 10} Kg8 {+2.85/24 7}
30. Nxc1 {+0.00/10 1.8} Qd4 {+7.82/22 7}
31. Nf2 {-1.34/9 2.6} Qxa4 {+9.38/23 2.9}
32. Qf5 {-2.13/9 6} Bb5+ {+12.09/21 2.7}
33. Ke1 {-4.76/9 5} Qc2 {+13.66/22 3}
34. Ncd3 Bxd3 {+13.93/20 0.7}
35. Qe6+ {-7.03/7 2.4} Nf7 {+56.66/24 17}
36. Nxd3 Rg2 {+132.65/24 1.9}
37. Qg4+ Rxg4 {+132.67/24 1.0}
38. Nf2 Rg2 {+1000.04/64 2.2}
39. f4 Qxf2+ {+1000.02/127 3}
40. Kd1 Qd2# {+1000.01/127 0.1}
{Xboard adjudication: Checkmate}
0-1

[Event "RWBC"]
[Site "CAPPUCCINO"]
[Date "2017.08.10"]
[Round "5"]
[Number "187"]
[White "AnMon 5.75"]
[Black "asmFishW_2017-08-05_base (UCI2WB)"]
[Result "1-0"]
[TimeControl "40/180"]
[FEN "rnbqkbnr/3pppp1/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - 0 1"]
[SetUp "1"]

{--------------
r n b q k b n r
. . . p p p p .
. . . . . . . .
. . . . . . . .
. . . . . . . .
. . . . . . . .
P P P P P P P P
R N B Q K B N R
white to play
--------------}

1. Nf3 {+3.94/11} Nf6 {-3.70/20 11}
2. b3 {+3.89/11 6} Nc6 {-3.82/20 11}
3. Bb2 {+3.97/11 5} Rh5 {-3.80/22 19}
4. e3 {+4.11/10 3} g5 {-3.66/20 2.2}
5. Ne5 {+4.30/10 4} Rh6 {-3.96/24 17}
6. Nxc6 {+4.36/10 3} dxc6 {-4.01/23 7}
7. Na3 {+4.48/11 5} Bg7 {-4.13/22 4}
8. Nc4 {+4.59/10 2.3} Be6 {-4.16/21 3}
9. Be2 {+4.65/10 4} Kf8 {-3.98/19 3}
10. O-O {+4.63/11 8} Ne4 {-3.66/19 3}
11. d4 {+4.72/10 2.3} c5 {-3.97/20 4}
12. h3 {+4.66/10 6} cxd4 {-3.06/19 2.2}
13. exd4 {+4.83/10 2.3} g4 {-3.79/22 5}
14. Bxg4 {+4.28/10 3} Bxc4 {-3.87/20 0.9}
15. bxc4 Rb6 {-3.53/21 1.8}
16. Qe1 {+3.99/11 5} Nf6 {-3.57/20 3}
17. Bc3 {+4.21/12 5} Nxg4 {-3.73/21 3}
18. hxg4 Re6 {-3.66/21 8}
19. Qd2 {+4.30/12 9} Ra4 {-3.75/21 11}
20. d5 {+4.40/12 8} Rg6 {-4.43/22 4}
21. Bxg7+ {+4.42/11 5} Rxg7 {-4.50/20 0.6}
22. Qf4 {+4.47/11 2.8} Qd6 {-3.97/21 1.3}
23. Qxd6 {+4.39/11 3} exd6 {-3.71/24 1.6}
24. f3 {+4.33/13 4} Rg5 {-4.46/26 10}
25. Rf2 {+4.44/15 4} Rxc4 {-3.83/21 2.0}
26. Rd2 {+4.45/15 5} Ra4 {-3.46/23 5}
27. c3 {+4.45/14 13} f5 {-3.75/23 7}
28. gxf5 {+4.66/13 4} Rxf5 {-4.12/23 2.7}
29. Rc1 {+4.68/12 3} Rff4 {-3.72/19 0.9}
30. Kf2 {+4.83/11 2.2} Kg7 {-4.23/22 8}
31. g4 {+4.95/12 4} Rfc4 {-4.09/20 2.0}
32. Rcc2 {+4.88/12 4} Ra5 {-4.36/21 2.4}
33. Ke3 {+4.84/13 5} Ra3 {-4.54/23 2.8}
34. Rd4 {+4.85/13 2.4} Raxc3+ {-4.26/19 0.6}
35. Rxc3 Rxc3+ {-4.81/17 0.2}
36. Rd3 {+4.95/14 4} Rc2 {-4.37/21 1.0}
37. a4 {+4.96/13 3} Ra2 {-6.39/29 4}
38. Rd4 {+4.96/14 4} Kf7 {-6.70/26 1.7}
39. f4 {+4.97/15 8} Ra3+ {-5.02/21 1.4}
40. Kd2 {+4.96/15 7} Ke8 {-5.55/22 0.9}
41. g5 {+5.00/13 5} Rg3 {-9.13/29 15}
42. Re4+ {+5.03/12 2.3} Kf8 {-11.14/31 19}
43. Re6 {+5.32/12 2.4} Kg7 {-10.47/21 2.8}
44. Rxd6 {+5.76/13 6} Rg2+ {-123.75/28 37}
45. Ke3 {+6.67/12 4} Rg1 {-132.64/34 7}
46. a5 {+7.63/10 2.5} Re1+ {-132.64/32 3}
{User adjudication}
1-0[/pgn]

Rebel · Post by **Rebel** » Fri May 14, 2021 5:17 pm

Round 1 (first 10 games) 8-2 for Stockfish.

Round 2 running...

Results with diagrams at my forum.

Uri Blass · Post by **Uri Blass** » Fri May 14, 2021 6:39 pm

I would like to know if stockfish use contempt=100 or default contempt.
I expect stockfish to get more than 37%
Not sure about 50%

Uri Blass · Post by **Uri Blass** » Fri May 14, 2021 6:44 pm

I can add that I believe that stockfish is not the right program to play the rook and queen handicap matches.
Probably there are engines that can get points without a rook against these engines but I guess stockfish is not going to be one of them so I guess less than 50% but still more than 37%

Rebel · Post by **Rebel** » Fri May 14, 2021 7:12 pm

Uri Blass wrote: ↑Fri May 14, 2021 6:39 pm I would like to know if stockfish use contempt=100 or default contempt.
I expect stockfish to get more than 37%
Not sure about 50%

I always play out of the box.

Uri Blass · Post by **Uri Blass** » Fri May 14, 2021 9:15 pm

Rebel wrote: ↑Fri May 14, 2021 7:12 pm
Uri Blass wrote: ↑Fri May 14, 2021 6:39 pm I would like to know if stockfish use contempt=100 or default contempt.
I expect stockfish to get more than 37%
Not sure about 50%
I always play out of the box.

out of the box is not designed to perform best with odds against weaker opponents and I guess contempt 100 is better.

Rebel · Post by **Rebel** » Sat May 15, 2021 1:04 am

One spectatular game....

[pgn][Event "Round3"]
[Date "2021.05.14"]
[Round "1"]
[White "SF13"]
[Black "Ethereal_12.75"]
[Result "1/2-1/2"]
[BlackElo "2200"]
[Time "23"]
[WhiteElo "2200"]
[TimeControl "40/600:40/600:40/600"]
[SetUp "1"]
[FEN "rnbqkbnr/pppppppp/8/8/8/8/PP3PPP/RNBQKBNR w KQkq - 0 1"]
[Termination "adjudication"]
[PlyCount "166"]
[WhiteType "program"]
[BlackType "program"]
[Comment "c0 \"-wpe2 | wpd2 | wpc2\";"]

1. Bd3 {-4.80/31 70} Nf6 {+3.56/25 33} 2. Nf3 {-4.85/27 4} e6 {+3.61/23 10}
3. Nc3 {-4.63/25 7} c5 {+3.51/24 15} 4. O-O {-4.51/26 16} d5 {+3.65/24 20}
5. Bg5 {-4.36/26 11} Be7 {+3.63/23 9} 6. Re1 {-4.38/25 4} Nc6 {+3.65/24 18}
7. Rc1 {-4.27/25 6} a6 {+3.59/27 38} 8. Bb1 {-4.32/24 8} b6 {+3.62/25 11}
9. a3 {-4.48/26 34} Bb7 {+3.79/24 15} 10. h3 {-4.55/29 64} Nb8 {+3.79/25
15} 11. b4 {-5.03/28 56} c4 {+4.23/25 16} 12. Qd4 {-4.36/25 15} a5
{+4.42/25 13} 13. Qh4 {-4.21/21 4} axb4 {+4.62/25 18} 14. axb4 {-4.05/23 6}
Bxb4 {+3.84/26 11} 15. Nd4 {-3.52/21 5} Nbd7 {+3.86/28 26} 16. Ncb5
{-2.87/23 5} Bxe1 {+3.95/28 22} 17. Rxe1 {-2.79/27 10} Kf8 {+3.85/27 12}
18. Rxe6 {-1.28/24 7} Kg8 {+4.03/28 19} 19. Rd6 {-0.85/23 5} Qe8 {+3.24/28
41} 20. Kh2 {-1.26/26 13} h6 {+2.81/25 15} 21. Bd2 {-1.27/28 21} Ra1
{+3.45/26 15} 22. Bf5 {-1.52/27 9} Nc5 {+3.65/25 23} 23. f3 {-1.54/26 5}
Qe7 {+2.70/28 42} 24. Bc3 {-1.32/27 6} Ra8 {+2.04/28 8} 25. Nc6 {-1.16/29
16} Bxc6 {+2.44/28 11} 26. Rxc6 {-1.02/27 5} Kf8 {+2.85/25 11} 27. Rxf6
{-0.84/31 21} h5 {+2.97/28 24} 28. Bc2 {-0.52/30 8} gxf6 {+3.37/28 8} 29.
Bxf6 {-1.35/30 14} Qd7 {+3.44/26 4} 30. Bxh8 {-2.03/32 39} Qxb5 {+3.22/23
3} 31. Qxh5 {-1.86/26 8} Qc6 {+3.30/25 4} 32. Be5 {-1.87/26 8} Nd3
{+3.69/24 6} 33. Bg3 {-2.07/28 11} Ra2 {+4.02/25 6} 34. Bb1 {-2.72/27 25}
Re2 {+3.92/28 9} 35. Qh7 {-2.68/23 2} Qg6 {+4.63/28 6} 36. Qh8+ {-1.71/24
4} Ke7 {+4.80/26 4} 37. Qa8 {-1.15/27 5} Qf5 {+4.28/27 14} 38. Qa3+
{-1.48/30 10} Kf6 {+5.42/24 2} 39. Qf8 {-1.06/30 8} Qh5 {+5.42/20 0} 40.
Bc7 {-0.62/24 1} Qxf3 {+7.40/21 1} 41. Qh8+ {-0.42/31 8} Kf5 {+2.22/32 58}
42. Bxd3+ {-0.46/33 8} Qxd3 {+1.43/30 48} 43. Qc8+ {-0.30/31 9} Re6
{+1.44/33 36} 44. Qg8 {-0.40/33 16} Re7 {+1.02/32 51} 45. Qc8+ {-0.32/30
10} Kg6 {+1.06/30 27} 46. Qg8+ {-0.21/32 9} Kf5 {+0.68/30 24} 47. Qc8+
{-0.28/33 13} Kf6 {+0.95/33 40} 48. Qh8+ {-0.28/34 19} Kg5 {+0.45/31 42}
49. Bd8 {-0.21/32 10} Qe4 {+0.52/30 42} 50. Qf8 {-0.31/34 13} Qe5+
{+1.34/28 14} 51. Kh1 {-0.21/32 9} Qe1+ {+2.16/28 6} 52. Kh2 {-0.21/30 1}
Kf6 {+1.31/25 7} 53. Qh6+ {-0.19/31 14} Kf5 {+0.52/28 14} 54. g4+ {-0.13/31
13} Ke4 {+1.43/27 11} 55. Bxe7 {-0.13/32 11} Qf2+ {+0.54/23 8} 56. Kh1
{-0.13/32 13} b5 {+0.63/29 27} 57. Qd6 {-0.15/33 23} d4 {+0.38/25 10} 58.
Bg5 {-0.13/33 13} Kd3 {+1.93/29 33} 59. Qb4 {0.00/35 12} f6 {+1.43/25 7}
60. Bh6 {-0.13/35 21} Qg3 {+0.81/25 11} 61. Qb1+ {0.00/36 12} Kc3 {+0.84/25
4} 62. Qa1+ {0.00/40 13} Kb3 {+1.56/23 5} 63. Qd1+ {0.00/40 17} Kb4
{+0.94/28 16} 64. Qb1+ {0.00/40 15} Ka4 {+1.12/21 2} 65. Qa1+ {0.00/38 34}
Qa3 {+0.55/24 6} 66. Qd1+ {0.00/37 14} Ka5 {+0.34/24 4} 67. Bf4 {0.00/39
16} Qc5 {+2.26/25 9} 68. h4 {0.00/41 26} b4 {+1.64/22 3} 69. h5 {0.00/41
11} Qd5+ {+0.41/20 1} 70. Kg1 {0.00/43 25} Qe4 {+1.30/20 3} 71. Bc1
{0.00/44 19} f5 {+0.11/21 4} 72. gxf5 {0.00/37 18} d3 {+0.01/21 0} 73. f6
{0.00/37 24} Qd4+ {+0.01/21 0} 74. Kg2 {0.00/36 12} c3 {+0.01/23 1} 75. h6
{0.00/39 17} d2 {+0.01/24 1} 76. Bxd2 {0.00/38 16} cxd2 {+0.01/24 0} 77.
Qh5+ {0.00/40 39} Ka4 {+0.01/24 2} 78. f7 {0.00/36 2} Qe4+ {+0.01/24 1} 79.
Kg3 {0.00/38 6} Qe1+ {+0.01/23 0} 80. Kg2 {0.00/38 0} d1=Q {+0.01/23 0} 81.
Qxd1+ {0.00/40 13} Qxd1 {0.00/29 13} 82. f8=Q {0.00/40 16} Qg4+ {0.00/30 9}
83. Kf2 {0.00/43 13} Qh4 {0.00/30 10 Arena Adjudication} 1/2-1/2[/pgn]

Stockfish handicap match - PART II -

Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -

Re: Stockfish handicap match - PART II -