PGN - http://kirill-kryukov.com/chess/discuss ... p?id=50726
PGN - http://kirill-kryukov.com/chess/discuss ... p?id=50725
Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
Moderator: Ras
-
Graham Banks
- Posts: 44333
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
PGN - http://kirill-kryukov.com/chess/discuss ... p?id=50726
PGN - http://kirill-kryukov.com/chess/discuss ... p?id=50725
gbanksnz at gmail.com
-
Graham Banks
- Posts: 44333
- Joined: Sun Feb 26, 2006 10:52 am
- Location: Auckland, NZ
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
Code: Select all
CCRL 40/15 Rating List - Custom engine selection
1394344 games played by 3092 programs, run by 25 testers
Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB
Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k.
Computed on May 21, 2022 with Bayeselo based on 1'394'344 games
Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/
Rank Engine Elo + - Score AvOp Games
1 Stockfish 15 64-bit 3528 +16 -16 70.7% -131.1 1204
Stockfish 060122 64-bit 3525 +17 -17 72.4% -139.3 1098
Stockfish 14.1 64-bit 3502 +13 -13 68.3% -110.7 1829
Stockfish 13 64-bit 3500 +19 -18 72.0% -146.3 929
Stockfish 2021-01-11 64-bit 3499 +14 -14 76.2% -173.9 1716
Stockfish 14 64-bit 3497 +16 -16 70.1% -124.2 1167
Stockfish 12 64-bit 3472 +15 -15 74.5% -163.0 1612
Stockfish+NNUE 150720 64-bit 3467 +26 -25 79.2% -202.0 558
Stockfish 2019-12-10 64-bit 3438 +16 -16 76.6% -176.6 1372
Stockfish 11 64-bit 3430 +15 -15 73.4% -151.7 1549
Stockfish 10 64-bit 3383 +9 -9 74.6% -167.5 4812
Stockfish 9 64-bit 3363 +12 -12 71.4% -149.3 2565
Stockfish 8 64-bit 3299 +11 -11 69.7% -128.8 2722
Stockfish 7 64-bit 3244 +13 -13 67.3% -109.1 1889
Stockfish 6 64-bit 3224 +14 -14 67.5% -110.6 1577
Stockfish 5 64-bit 3189 +15 -15 64.4% -88.5 1406
Stockfish DD 64-bit 3162 +15 -15 61.8% -71.6 1373
Stockfish 4 64-bit 3119 +17 -17 58.0% -48.2 1017
Stockfish 3 64-bit 3093 +16 -16 56.5% -38.6 1186
Stockfish 2.3.1 64-bit 3075 +17 -17 57.1% -43.5 1065
Stockfish 2.2.2 64-bit 3070 +20 -19 54.5% -32.3 792
Stockfish 2.2 64-bit 3054 +35 -34 55.2% -31.0 242
Stockfish 2.2.2 32-bit 3051 +13 -13 59.8% -64.6 1759
Stockfish 2.1.1 64-bit 3050 +15 -15 51.2% -7.0 1258
Stockfish 2.3.1 32-bit 3038 +17 -17 57.2% -44.3 1095
Stockfish 1.9.1 64-bit 3032 +21 -20 60.0% -68.3 763
Stockfish 2.0.1 64-bit 3027 +23 -22 57.7% -48.6 611
Stockfish 1.8 64-bit 3026 +21 -21 61.2% -77.5 761
Stockfish 2.1.1 32-bit 3023 +15 -15 61.9% -79.5 1465
Stockfish 2.0.1 32-bit 3012 +16 -16 62.8% -83.9 1302
Stockfish 1.7.1 64-bit 3007 +22 -22 61.3% -74.9 669
Stockfish 1.9.1 32-bit 3002 +15 -15 63.1% -84.8 1382
Stockfish 1.8 32-bit 2990 +16 -16 63.6% -86.4 1268
Stockfish 1.7.1 32-bit 2978 +15 -15 63.7% -90.3 1402
Stockfish 1.6.3 64-bit 2963 +20 -20 58.6% -54.4 801
Stockfish 1.6s 64-bit 2950 +29 -29 52.0% -14.5 346
Stockfish 1.6.3 32-bit 2934 +16 -16 60.0% -63.3 1212
Stockfish 1.5.1 64-bit 2886 +27 -27 54.4% -27.5 428
Stockfish 1.5.1 32-bit 2862 +14 -14 54.5% -29.1 1562
Stockfish 1.4 64-bit 2848 +19 -19 53.1% -17.6 844
Stockfish 1.4 32-bit 2830 +16 -16 53.6% -24.1 1184
Stockfish 1.3.1 32-bit 2793 +25 -25 46.1% +25.0 512
Stockfish 1.2 Default 2780 +21 -22 46.1% +23.8 679
Stockfish 1.01 2746 +31 -31 50.5% -5.5 322Code: Select all
CCRL 40/15 Rating List - Custom engine selection
1394344 games played by 3092 programs, run by 25 testers
Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB
Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k.
Computed on May 21, 2022 with Bayeselo based on 1'394'344 games
Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/
Rank Engine Elo + - Score AvOp Games
1 Stockfish 15 64-bit 4CPU 3540 +17 -17 68.3% -109.7 994
Stockfish 14 64-bit 4CPU 3539 +18 -18 66.9% -101.5 874
Stockfish 13 64-bit 4CPU 3537 +17 -17 75.3% -164.9 1192
Stockfish 2021-01-11 64-bit 4CPU 3537 +18 -17 74.5% -157.2 1088
Stockfish 14.1 64-bit 4CPU 3524 +16 -16 64.1% -81.4 1054
Stockfish 12 64-bit 4CPU 3510 +19 -19 71.8% -139.9 841
Stockfish 2019-10-09 64-bit 4CPU 3485 +23 -23 74.7% -155.8 616
Stockfish 11 64-bit 4CPU 3473 +19 -19 73.0% -143.1 840
Stockfish 10 64-bit 4CPU 3457 +17 -16 73.4% -155.2 1227
Stockfish 270918 64-bit 4CPU 3447 +28 -27 77.9% -188.8 457
Stockfish 9 64-bit 4CPU 3426 +15 -15 72.3% -152.9 1423
Stockfish 8 64-bit 4CPU 3373 +13 -13 64.8% -94.7 1665
Stockfish 7 64-bit 4CPU 3323 +15 -15 66.2% -100.6 1364
Stockfish 6 64-bit 4CPU 3289 +13 -13 63.3% -82.0 1704
Stockfish 5 64-bit 4CPU 3269 +14 -14 64.3% -88.6 1663
Stockfish DD 64-bit 4CPU 3230 +16 -16 61.5% -69.9 1106
Stockfish 4 64-bit 4CPU 3202 +16 -16 59.1% -53.6 1172
Stockfish 3 64-bit 4CPU 3163 +21 -21 56.0% -35.3 663
Stockfish 2.2.2 64-bit 4CPU 3145 +16 -16 58.3% -50.2 1205
Stockfish 2.3.1 64-bit 4CPU 3139 +19 -19 55.4% -32.9 846
Stockfish 2.0.1 64-bit 4CPU 3122 +24 -24 56.4% -40.1 514
Stockfish 1.9.1 64-bit 4CPU 3115 +21 -20 57.3% -50.9 750
Stockfish 2.1.1 64-bit 4CPU 3109 +27 -27 54.7% -29.4 406
Stockfish 1.8 64-bit 4CPU 3107 +17 -17 57.4% -52.0 1058
Stockfish 1.7.1 64-bit 4CPU 3101 +18 -18 58.9% -58.6 972
Stockfish 1.6.3 64-bit 4CPU 3034 +22 -22 57.0% -45.9 667
Stockfish 1.5.1 64-bit 4CPU 2968 +24 -25 53.7% -23.2 495
Stockfish 1.4 64-bit 4CPU 2946 +17 -17 54.2% -28.4 1131gbanksnz at gmail.com
-
lkaufman
- Posts: 6236
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
- Full name: Larry Kaufman
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
Based on the progress from Stockfish 13 to 15 (3 elo) on 4 CPUs, we should expect to wait about forty years to see Stockfish climb from 3540 to 3600, but due to diminishing returns that is probably too optimistic. Seriously, I don't mean this as criticism of Stockfish or CCRL, I think it just means that with normal openings, reasonably long time controls, and four or more CPUs, chess between the top engines is almost a certain draw, and further Elo gains are almost impossible by these criteria. I think it is time to start a discussion of how chess Elo should be measured in the future; there are many possibilities that deserve consideration. It would be a shame for interest in computer chess to die out because it appears to have reached a ceiling. even though engines are far from perfect now, just good enough to draw from normal opening positions.Graham Banks wrote: ↑Sun May 22, 2022 4:34 amCode: Select all
CCRL 40/15 Rating List - Custom engine selection 1394344 games played by 3092 programs, run by 25 testers Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k. Computed on May 21, 2022 with Bayeselo based on 1'394'344 games Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/ Rank Engine Elo + - Score AvOp Games 1 Stockfish 15 64-bit 3528 +16 -16 70.7% -131.1 1204 Stockfish 060122 64-bit 3525 +17 -17 72.4% -139.3 1098 Stockfish 14.1 64-bit 3502 +13 -13 68.3% -110.7 1829 Stockfish 13 64-bit 3500 +19 -18 72.0% -146.3 929 Stockfish 2021-01-11 64-bit 3499 +14 -14 76.2% -173.9 1716 Stockfish 14 64-bit 3497 +16 -16 70.1% -124.2 1167 Stockfish 12 64-bit 3472 +15 -15 74.5% -163.0 1612 Stockfish+NNUE 150720 64-bit 3467 +26 -25 79.2% -202.0 558 Stockfish 2019-12-10 64-bit 3438 +16 -16 76.6% -176.6 1372 Stockfish 11 64-bit 3430 +15 -15 73.4% -151.7 1549 Stockfish 10 64-bit 3383 +9 -9 74.6% -167.5 4812 Stockfish 9 64-bit 3363 +12 -12 71.4% -149.3 2565 Stockfish 8 64-bit 3299 +11 -11 69.7% -128.8 2722 Stockfish 7 64-bit 3244 +13 -13 67.3% -109.1 1889 Stockfish 6 64-bit 3224 +14 -14 67.5% -110.6 1577 Stockfish 5 64-bit 3189 +15 -15 64.4% -88.5 1406 Stockfish DD 64-bit 3162 +15 -15 61.8% -71.6 1373 Stockfish 4 64-bit 3119 +17 -17 58.0% -48.2 1017 Stockfish 3 64-bit 3093 +16 -16 56.5% -38.6 1186 Stockfish 2.3.1 64-bit 3075 +17 -17 57.1% -43.5 1065 Stockfish 2.2.2 64-bit 3070 +20 -19 54.5% -32.3 792 Stockfish 2.2 64-bit 3054 +35 -34 55.2% -31.0 242 Stockfish 2.2.2 32-bit 3051 +13 -13 59.8% -64.6 1759 Stockfish 2.1.1 64-bit 3050 +15 -15 51.2% -7.0 1258 Stockfish 2.3.1 32-bit 3038 +17 -17 57.2% -44.3 1095 Stockfish 1.9.1 64-bit 3032 +21 -20 60.0% -68.3 763 Stockfish 2.0.1 64-bit 3027 +23 -22 57.7% -48.6 611 Stockfish 1.8 64-bit 3026 +21 -21 61.2% -77.5 761 Stockfish 2.1.1 32-bit 3023 +15 -15 61.9% -79.5 1465 Stockfish 2.0.1 32-bit 3012 +16 -16 62.8% -83.9 1302 Stockfish 1.7.1 64-bit 3007 +22 -22 61.3% -74.9 669 Stockfish 1.9.1 32-bit 3002 +15 -15 63.1% -84.8 1382 Stockfish 1.8 32-bit 2990 +16 -16 63.6% -86.4 1268 Stockfish 1.7.1 32-bit 2978 +15 -15 63.7% -90.3 1402 Stockfish 1.6.3 64-bit 2963 +20 -20 58.6% -54.4 801 Stockfish 1.6s 64-bit 2950 +29 -29 52.0% -14.5 346 Stockfish 1.6.3 32-bit 2934 +16 -16 60.0% -63.3 1212 Stockfish 1.5.1 64-bit 2886 +27 -27 54.4% -27.5 428 Stockfish 1.5.1 32-bit 2862 +14 -14 54.5% -29.1 1562 Stockfish 1.4 64-bit 2848 +19 -19 53.1% -17.6 844 Stockfish 1.4 32-bit 2830 +16 -16 53.6% -24.1 1184 Stockfish 1.3.1 32-bit 2793 +25 -25 46.1% +25.0 512 Stockfish 1.2 Default 2780 +21 -22 46.1% +23.8 679 Stockfish 1.01 2746 +31 -31 50.5% -5.5 322Code: Select all
CCRL 40/15 Rating List - Custom engine selection 1394344 games played by 3092 programs, run by 25 testers Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k. Computed on May 21, 2022 with Bayeselo based on 1'394'344 games Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/ Rank Engine Elo + - Score AvOp Games 1 Stockfish 15 64-bit 4CPU 3540 +17 -17 68.3% -109.7 994 Stockfish 14 64-bit 4CPU 3539 +18 -18 66.9% -101.5 874 Stockfish 13 64-bit 4CPU 3537 +17 -17 75.3% -164.9 1192 Stockfish 2021-01-11 64-bit 4CPU 3537 +18 -17 74.5% -157.2 1088 Stockfish 14.1 64-bit 4CPU 3524 +16 -16 64.1% -81.4 1054 Stockfish 12 64-bit 4CPU 3510 +19 -19 71.8% -139.9 841 Stockfish 2019-10-09 64-bit 4CPU 3485 +23 -23 74.7% -155.8 616 Stockfish 11 64-bit 4CPU 3473 +19 -19 73.0% -143.1 840 Stockfish 10 64-bit 4CPU 3457 +17 -16 73.4% -155.2 1227 Stockfish 270918 64-bit 4CPU 3447 +28 -27 77.9% -188.8 457 Stockfish 9 64-bit 4CPU 3426 +15 -15 72.3% -152.9 1423 Stockfish 8 64-bit 4CPU 3373 +13 -13 64.8% -94.7 1665 Stockfish 7 64-bit 4CPU 3323 +15 -15 66.2% -100.6 1364 Stockfish 6 64-bit 4CPU 3289 +13 -13 63.3% -82.0 1704 Stockfish 5 64-bit 4CPU 3269 +14 -14 64.3% -88.6 1663 Stockfish DD 64-bit 4CPU 3230 +16 -16 61.5% -69.9 1106 Stockfish 4 64-bit 4CPU 3202 +16 -16 59.1% -53.6 1172 Stockfish 3 64-bit 4CPU 3163 +21 -21 56.0% -35.3 663 Stockfish 2.2.2 64-bit 4CPU 3145 +16 -16 58.3% -50.2 1205 Stockfish 2.3.1 64-bit 4CPU 3139 +19 -19 55.4% -32.9 846 Stockfish 2.0.1 64-bit 4CPU 3122 +24 -24 56.4% -40.1 514 Stockfish 1.9.1 64-bit 4CPU 3115 +21 -20 57.3% -50.9 750 Stockfish 2.1.1 64-bit 4CPU 3109 +27 -27 54.7% -29.4 406 Stockfish 1.8 64-bit 4CPU 3107 +17 -17 57.4% -52.0 1058 Stockfish 1.7.1 64-bit 4CPU 3101 +18 -18 58.9% -58.6 972 Stockfish 1.6.3 64-bit 4CPU 3034 +22 -22 57.0% -45.9 667 Stockfish 1.5.1 64-bit 4CPU 2968 +24 -25 53.7% -23.2 495 Stockfish 1.4 64-bit 4CPU 2946 +17 -17 54.2% -28.4 1131
Komodo rules!
-
Uri Blass
- Posts: 10822
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
I think that the problem is that stockfish does not try to improve in beating weaker engines.lkaufman wrote: ↑Sun May 22, 2022 6:18 amBased on the progress from Stockfish 13 to 15 (3 elo) on 4 CPUs, we should expect to wait about forty years to see Stockfish climb from 3540 to 3600, but due to diminishing returns that is probably too optimistic. Seriously, I don't mean this as criticism of Stockfish or CCRL, I think it just means that with normal openings, reasonably long time controls, and four or more CPUs, chess between the top engines is almost a certain draw, and further Elo gains are almost impossible by these criteria. I think it is time to start a discussion of how chess Elo should be measured in the future; there are many possibilities that deserve consideration. It would be a shame for interest in computer chess to die out because it appears to have reached a ceiling. even though engines are far from perfect now, just good enough to draw from normal opening positions.Graham Banks wrote: ↑Sun May 22, 2022 4:34 amCode: Select all
CCRL 40/15 Rating List - Custom engine selection 1394344 games played by 3092 programs, run by 25 testers Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k. Computed on May 21, 2022 with Bayeselo based on 1'394'344 games Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/ Rank Engine Elo + - Score AvOp Games 1 Stockfish 15 64-bit 3528 +16 -16 70.7% -131.1 1204 Stockfish 060122 64-bit 3525 +17 -17 72.4% -139.3 1098 Stockfish 14.1 64-bit 3502 +13 -13 68.3% -110.7 1829 Stockfish 13 64-bit 3500 +19 -18 72.0% -146.3 929 Stockfish 2021-01-11 64-bit 3499 +14 -14 76.2% -173.9 1716 Stockfish 14 64-bit 3497 +16 -16 70.1% -124.2 1167 Stockfish 12 64-bit 3472 +15 -15 74.5% -163.0 1612 Stockfish+NNUE 150720 64-bit 3467 +26 -25 79.2% -202.0 558 Stockfish 2019-12-10 64-bit 3438 +16 -16 76.6% -176.6 1372 Stockfish 11 64-bit 3430 +15 -15 73.4% -151.7 1549 Stockfish 10 64-bit 3383 +9 -9 74.6% -167.5 4812 Stockfish 9 64-bit 3363 +12 -12 71.4% -149.3 2565 Stockfish 8 64-bit 3299 +11 -11 69.7% -128.8 2722 Stockfish 7 64-bit 3244 +13 -13 67.3% -109.1 1889 Stockfish 6 64-bit 3224 +14 -14 67.5% -110.6 1577 Stockfish 5 64-bit 3189 +15 -15 64.4% -88.5 1406 Stockfish DD 64-bit 3162 +15 -15 61.8% -71.6 1373 Stockfish 4 64-bit 3119 +17 -17 58.0% -48.2 1017 Stockfish 3 64-bit 3093 +16 -16 56.5% -38.6 1186 Stockfish 2.3.1 64-bit 3075 +17 -17 57.1% -43.5 1065 Stockfish 2.2.2 64-bit 3070 +20 -19 54.5% -32.3 792 Stockfish 2.2 64-bit 3054 +35 -34 55.2% -31.0 242 Stockfish 2.2.2 32-bit 3051 +13 -13 59.8% -64.6 1759 Stockfish 2.1.1 64-bit 3050 +15 -15 51.2% -7.0 1258 Stockfish 2.3.1 32-bit 3038 +17 -17 57.2% -44.3 1095 Stockfish 1.9.1 64-bit 3032 +21 -20 60.0% -68.3 763 Stockfish 2.0.1 64-bit 3027 +23 -22 57.7% -48.6 611 Stockfish 1.8 64-bit 3026 +21 -21 61.2% -77.5 761 Stockfish 2.1.1 32-bit 3023 +15 -15 61.9% -79.5 1465 Stockfish 2.0.1 32-bit 3012 +16 -16 62.8% -83.9 1302 Stockfish 1.7.1 64-bit 3007 +22 -22 61.3% -74.9 669 Stockfish 1.9.1 32-bit 3002 +15 -15 63.1% -84.8 1382 Stockfish 1.8 32-bit 2990 +16 -16 63.6% -86.4 1268 Stockfish 1.7.1 32-bit 2978 +15 -15 63.7% -90.3 1402 Stockfish 1.6.3 64-bit 2963 +20 -20 58.6% -54.4 801 Stockfish 1.6s 64-bit 2950 +29 -29 52.0% -14.5 346 Stockfish 1.6.3 32-bit 2934 +16 -16 60.0% -63.3 1212 Stockfish 1.5.1 64-bit 2886 +27 -27 54.4% -27.5 428 Stockfish 1.5.1 32-bit 2862 +14 -14 54.5% -29.1 1562 Stockfish 1.4 64-bit 2848 +19 -19 53.1% -17.6 844 Stockfish 1.4 32-bit 2830 +16 -16 53.6% -24.1 1184 Stockfish 1.3.1 32-bit 2793 +25 -25 46.1% +25.0 512 Stockfish 1.2 Default 2780 +21 -22 46.1% +23.8 679 Stockfish 1.01 2746 +31 -31 50.5% -5.5 322Code: Select all
CCRL 40/15 Rating List - Custom engine selection 1394344 games played by 3092 programs, run by 25 testers Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k. Computed on May 21, 2022 with Bayeselo based on 1'394'344 games Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/ Rank Engine Elo + - Score AvOp Games 1 Stockfish 15 64-bit 4CPU 3540 +17 -17 68.3% -109.7 994 Stockfish 14 64-bit 4CPU 3539 +18 -18 66.9% -101.5 874 Stockfish 13 64-bit 4CPU 3537 +17 -17 75.3% -164.9 1192 Stockfish 2021-01-11 64-bit 4CPU 3537 +18 -17 74.5% -157.2 1088 Stockfish 14.1 64-bit 4CPU 3524 +16 -16 64.1% -81.4 1054 Stockfish 12 64-bit 4CPU 3510 +19 -19 71.8% -139.9 841 Stockfish 2019-10-09 64-bit 4CPU 3485 +23 -23 74.7% -155.8 616 Stockfish 11 64-bit 4CPU 3473 +19 -19 73.0% -143.1 840 Stockfish 10 64-bit 4CPU 3457 +17 -16 73.4% -155.2 1227 Stockfish 270918 64-bit 4CPU 3447 +28 -27 77.9% -188.8 457 Stockfish 9 64-bit 4CPU 3426 +15 -15 72.3% -152.9 1423 Stockfish 8 64-bit 4CPU 3373 +13 -13 64.8% -94.7 1665 Stockfish 7 64-bit 4CPU 3323 +15 -15 66.2% -100.6 1364 Stockfish 6 64-bit 4CPU 3289 +13 -13 63.3% -82.0 1704 Stockfish 5 64-bit 4CPU 3269 +14 -14 64.3% -88.6 1663 Stockfish DD 64-bit 4CPU 3230 +16 -16 61.5% -69.9 1106 Stockfish 4 64-bit 4CPU 3202 +16 -16 59.1% -53.6 1172 Stockfish 3 64-bit 4CPU 3163 +21 -21 56.0% -35.3 663 Stockfish 2.2.2 64-bit 4CPU 3145 +16 -16 58.3% -50.2 1205 Stockfish 2.3.1 64-bit 4CPU 3139 +19 -19 55.4% -32.9 846 Stockfish 2.0.1 64-bit 4CPU 3122 +24 -24 56.4% -40.1 514 Stockfish 1.9.1 64-bit 4CPU 3115 +21 -20 57.3% -50.9 750 Stockfish 2.1.1 64-bit 4CPU 3109 +27 -27 54.7% -29.4 406 Stockfish 1.8 64-bit 4CPU 3107 +17 -17 57.4% -52.0 1058 Stockfish 1.7.1 64-bit 4CPU 3101 +18 -18 58.9% -58.6 972 Stockfish 1.6.3 64-bit 4CPU 3034 +22 -22 57.0% -45.9 667 Stockfish 1.5.1 64-bit 4CPU 2968 +24 -25 53.7% -23.2 495 Stockfish 1.4 64-bit 4CPU 2946 +17 -17 54.2% -28.4 1131
Stockfish does not have a contempt.
I guess it is possible to get 60 elo improvement in the future in normal chess but making stockfish faster is not enough and you need to set traps for the opponents for this purpose.
-
lkaufman
- Posts: 6236
- Joined: Sun Jan 10, 2010 6:15 am
- Location: Maryland USA
- Full name: Larry Kaufman
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
That might raise the performance against weak engines, but when it is playing engines close to its own strength, setting traps isn't productive, you have to clearly outsearch your opponent for this to work. So whether it would raise the rating much would depend on the choice of opponents by the tester.Uri Blass wrote: ↑Sun May 22, 2022 8:47 amI think that the problem is that stockfish does not try to improve in beating weaker engines.lkaufman wrote: ↑Sun May 22, 2022 6:18 amBased on the progress from Stockfish 13 to 15 (3 elo) on 4 CPUs, we should expect to wait about forty years to see Stockfish climb from 3540 to 3600, but due to diminishing returns that is probably too optimistic. Seriously, I don't mean this as criticism of Stockfish or CCRL, I think it just means that with normal openings, reasonably long time controls, and four or more CPUs, chess between the top engines is almost a certain draw, and further Elo gains are almost impossible by these criteria. I think it is time to start a discussion of how chess Elo should be measured in the future; there are many possibilities that deserve consideration. It would be a shame for interest in computer chess to die out because it appears to have reached a ceiling. even though engines are far from perfect now, just good enough to draw from normal opening positions.Graham Banks wrote: ↑Sun May 22, 2022 4:34 amCode: Select all
CCRL 40/15 Rating List - Custom engine selection 1394344 games played by 3092 programs, run by 25 testers Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k. Computed on May 21, 2022 with Bayeselo based on 1'394'344 games Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/ Rank Engine Elo + - Score AvOp Games 1 Stockfish 15 64-bit 3528 +16 -16 70.7% -131.1 1204 Stockfish 060122 64-bit 3525 +17 -17 72.4% -139.3 1098 Stockfish 14.1 64-bit 3502 +13 -13 68.3% -110.7 1829 Stockfish 13 64-bit 3500 +19 -18 72.0% -146.3 929 Stockfish 2021-01-11 64-bit 3499 +14 -14 76.2% -173.9 1716 Stockfish 14 64-bit 3497 +16 -16 70.1% -124.2 1167 Stockfish 12 64-bit 3472 +15 -15 74.5% -163.0 1612 Stockfish+NNUE 150720 64-bit 3467 +26 -25 79.2% -202.0 558 Stockfish 2019-12-10 64-bit 3438 +16 -16 76.6% -176.6 1372 Stockfish 11 64-bit 3430 +15 -15 73.4% -151.7 1549 Stockfish 10 64-bit 3383 +9 -9 74.6% -167.5 4812 Stockfish 9 64-bit 3363 +12 -12 71.4% -149.3 2565 Stockfish 8 64-bit 3299 +11 -11 69.7% -128.8 2722 Stockfish 7 64-bit 3244 +13 -13 67.3% -109.1 1889 Stockfish 6 64-bit 3224 +14 -14 67.5% -110.6 1577 Stockfish 5 64-bit 3189 +15 -15 64.4% -88.5 1406 Stockfish DD 64-bit 3162 +15 -15 61.8% -71.6 1373 Stockfish 4 64-bit 3119 +17 -17 58.0% -48.2 1017 Stockfish 3 64-bit 3093 +16 -16 56.5% -38.6 1186 Stockfish 2.3.1 64-bit 3075 +17 -17 57.1% -43.5 1065 Stockfish 2.2.2 64-bit 3070 +20 -19 54.5% -32.3 792 Stockfish 2.2 64-bit 3054 +35 -34 55.2% -31.0 242 Stockfish 2.2.2 32-bit 3051 +13 -13 59.8% -64.6 1759 Stockfish 2.1.1 64-bit 3050 +15 -15 51.2% -7.0 1258 Stockfish 2.3.1 32-bit 3038 +17 -17 57.2% -44.3 1095 Stockfish 1.9.1 64-bit 3032 +21 -20 60.0% -68.3 763 Stockfish 2.0.1 64-bit 3027 +23 -22 57.7% -48.6 611 Stockfish 1.8 64-bit 3026 +21 -21 61.2% -77.5 761 Stockfish 2.1.1 32-bit 3023 +15 -15 61.9% -79.5 1465 Stockfish 2.0.1 32-bit 3012 +16 -16 62.8% -83.9 1302 Stockfish 1.7.1 64-bit 3007 +22 -22 61.3% -74.9 669 Stockfish 1.9.1 32-bit 3002 +15 -15 63.1% -84.8 1382 Stockfish 1.8 32-bit 2990 +16 -16 63.6% -86.4 1268 Stockfish 1.7.1 32-bit 2978 +15 -15 63.7% -90.3 1402 Stockfish 1.6.3 64-bit 2963 +20 -20 58.6% -54.4 801 Stockfish 1.6s 64-bit 2950 +29 -29 52.0% -14.5 346 Stockfish 1.6.3 32-bit 2934 +16 -16 60.0% -63.3 1212 Stockfish 1.5.1 64-bit 2886 +27 -27 54.4% -27.5 428 Stockfish 1.5.1 32-bit 2862 +14 -14 54.5% -29.1 1562 Stockfish 1.4 64-bit 2848 +19 -19 53.1% -17.6 844 Stockfish 1.4 32-bit 2830 +16 -16 53.6% -24.1 1184 Stockfish 1.3.1 32-bit 2793 +25 -25 46.1% +25.0 512 Stockfish 1.2 Default 2780 +21 -22 46.1% +23.8 679 Stockfish 1.01 2746 +31 -31 50.5% -5.5 322Code: Select all
CCRL 40/15 Rating List - Custom engine selection 1394344 games played by 3092 programs, run by 25 testers Ponder off, General books (up to 12 moves), 3-4-5 piece EGTB Time control: Equivalent to 40 moves in 15 minutes on an Intel i7-4770k. Computed on May 21, 2022 with Bayeselo based on 1'394'344 games Tested by CCRL team, 2005-2022, http://ccrl.chessdom.com/ccrl/4040/ Rank Engine Elo + - Score AvOp Games 1 Stockfish 15 64-bit 4CPU 3540 +17 -17 68.3% -109.7 994 Stockfish 14 64-bit 4CPU 3539 +18 -18 66.9% -101.5 874 Stockfish 13 64-bit 4CPU 3537 +17 -17 75.3% -164.9 1192 Stockfish 2021-01-11 64-bit 4CPU 3537 +18 -17 74.5% -157.2 1088 Stockfish 14.1 64-bit 4CPU 3524 +16 -16 64.1% -81.4 1054 Stockfish 12 64-bit 4CPU 3510 +19 -19 71.8% -139.9 841 Stockfish 2019-10-09 64-bit 4CPU 3485 +23 -23 74.7% -155.8 616 Stockfish 11 64-bit 4CPU 3473 +19 -19 73.0% -143.1 840 Stockfish 10 64-bit 4CPU 3457 +17 -16 73.4% -155.2 1227 Stockfish 270918 64-bit 4CPU 3447 +28 -27 77.9% -188.8 457 Stockfish 9 64-bit 4CPU 3426 +15 -15 72.3% -152.9 1423 Stockfish 8 64-bit 4CPU 3373 +13 -13 64.8% -94.7 1665 Stockfish 7 64-bit 4CPU 3323 +15 -15 66.2% -100.6 1364 Stockfish 6 64-bit 4CPU 3289 +13 -13 63.3% -82.0 1704 Stockfish 5 64-bit 4CPU 3269 +14 -14 64.3% -88.6 1663 Stockfish DD 64-bit 4CPU 3230 +16 -16 61.5% -69.9 1106 Stockfish 4 64-bit 4CPU 3202 +16 -16 59.1% -53.6 1172 Stockfish 3 64-bit 4CPU 3163 +21 -21 56.0% -35.3 663 Stockfish 2.2.2 64-bit 4CPU 3145 +16 -16 58.3% -50.2 1205 Stockfish 2.3.1 64-bit 4CPU 3139 +19 -19 55.4% -32.9 846 Stockfish 2.0.1 64-bit 4CPU 3122 +24 -24 56.4% -40.1 514 Stockfish 1.9.1 64-bit 4CPU 3115 +21 -20 57.3% -50.9 750 Stockfish 2.1.1 64-bit 4CPU 3109 +27 -27 54.7% -29.4 406 Stockfish 1.8 64-bit 4CPU 3107 +17 -17 57.4% -52.0 1058 Stockfish 1.7.1 64-bit 4CPU 3101 +18 -18 58.9% -58.6 972 Stockfish 1.6.3 64-bit 4CPU 3034 +22 -22 57.0% -45.9 667 Stockfish 1.5.1 64-bit 4CPU 2968 +24 -25 53.7% -23.2 495 Stockfish 1.4 64-bit 4CPU 2946 +17 -17 54.2% -28.4 1131
Stockfish does not have a contempt.
I guess it is possible to get 60 elo improvement in the future in normal chess but making stockfish faster is not enough and you need to set traps for the opponents for this purpose.
Komodo rules!
-
Frank Quisinsky
- Posts: 6966
- Joined: Wed Nov 18, 2009 7:16 pm
- Location: Gutweiler, Germany
- Full name: Frank Quisinsky
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
Hi Larry,
at first:
Draw quote is to high.
In my list Stockfish from end of the year is around 10 Elo stronger as Stockfish 15.
Same for versions 01.04, 12.04 ...
My 80min games on 1 core with 4.4Ghz on much more modern hardware is around the same as CCRL on 4 cores.
at second:
Most testers have an other problem in testing engines?
Example:
SF 14 test against the programs at this time are available ...
SF 15 test against the programs at this time are available ...
A big problem in NN times!!
Better is to test vs. the same group of engines in Neural Network times because the Elo difference is not clear.
If not ... Elo's for best programs goes higher and higher in a list.
The reasons that it make no sense to hold a rating list a long time on life.
My own is now after a half year completly outdated.
Better as better is ...
Tourneys between TOP-50, or TOP-40 each one against each other.
More excactly ratings are not possible.
Best
Frank
Next week ...
SF last version without NN
Komodo last version without NN
as Gauntlet ...
Both around vs. the same group of opponents!
Very easy to see that the different for Stockfish 15 (before NN to current version) is not higher as around 180 Elo.
Same for Dragon by Komodo 3 and a proof that it make no sense to hold all the old versions of engines in one rating list.
at first:
Draw quote is to high.
In my list Stockfish from end of the year is around 10 Elo stronger as Stockfish 15.
Same for versions 01.04, 12.04 ...
My 80min games on 1 core with 4.4Ghz on much more modern hardware is around the same as CCRL on 4 cores.
at second:
Most testers have an other problem in testing engines?
Example:
SF 14 test against the programs at this time are available ...
SF 15 test against the programs at this time are available ...
A big problem in NN times!!
Better is to test vs. the same group of engines in Neural Network times because the Elo difference is not clear.
If not ... Elo's for best programs goes higher and higher in a list.
The reasons that it make no sense to hold a rating list a long time on life.
My own is now after a half year completly outdated.
Better as better is ...
Tourneys between TOP-50, or TOP-40 each one against each other.
More excactly ratings are not possible.
Best
Frank
Next week ...
SF last version without NN
Komodo last version without NN
as Gauntlet ...
Both around vs. the same group of opponents!
Very easy to see that the different for Stockfish 15 (before NN to current version) is not higher as around 180 Elo.
Same for Dragon by Komodo 3 and a proof that it make no sense to hold all the old versions of engines in one rating list.
-
Frank Quisinsky
- Posts: 6966
- Joined: Wed Nov 18, 2009 7:16 pm
- Location: Gutweiler, Germany
- Full name: Frank Quisinsky
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
Very interesting, or?
With all the new NN files from currently available NN engines an optimaztion vs. current others engines.
More or less the main problem for a rating list.
But after all ...
A good chance for all the others in comparing to Stockfish or Komodo.
Wasp to Stockfish for 2 years ... the different is around 400 Elo, today a bit lesser as 250 Elo.
The same for a all the others ...
In reality the "weaker engines" in comparing to Stockfish have a much higher performance with NN files and the big lead melts away.
With all the new NN files from currently available NN engines an optimaztion vs. current others engines.
More or less the main problem for a rating list.
But after all ...
A good chance for all the others in comparing to Stockfish or Komodo.
Wasp to Stockfish for 2 years ... the different is around 400 Elo, today a bit lesser as 250 Elo.
The same for a all the others ...
In reality the "weaker engines" in comparing to Stockfish have a much higher performance with NN files and the big lead melts away.
-
Frank Quisinsky
- Posts: 6966
- Joined: Wed Nov 18, 2009 7:16 pm
- Location: Gutweiler, Germany
- Full name: Frank Quisinsky
Re: Stockfish 15 64-bit 1CPU and 4CPU Gauntlets for CCRL 40/15
Like that a lot because the style of the engine goes more and more in foreground.
Elo of engine is more and more not longer interesting!!
I can't build the opinion that for Stockfish people it's a long time project and we can not speaking about it how much Stockfish will be better in 5 years. Because it is not very interesting. In 5 years we have more as 20 programs which are very in the near of Stockfish and the mythos Stockfish disintegrates.
Much more important is to try to optimaze length of games and to work on the style of engine!
Programs like Komodo, Berserk, Koivisto and many others have a much to high move average for draws. To fight for each point is very boring if the game situation is very clear. Others programs play that in perfection, a good example is Wasp. If Wasp can see the game is 100% draw Wasp try to finish the game a.s.a.p..
And all the long games over 150 moves will be killed with a very fine work in programmings!
Elo of engine is more and more not longer interesting!!
I can't build the opinion that for Stockfish people it's a long time project and we can not speaking about it how much Stockfish will be better in 5 years. Because it is not very interesting. In 5 years we have more as 20 programs which are very in the near of Stockfish and the mythos Stockfish disintegrates.
Much more important is to try to optimaze length of games and to work on the style of engine!
Programs like Komodo, Berserk, Koivisto and many others have a much to high move average for draws. To fight for each point is very boring if the game situation is very clear. Others programs play that in perfection, a good example is Wasp. If Wasp can see the game is 100% draw Wasp try to finish the game a.s.a.p..
And all the long games over 150 moves will be killed with a very fine work in programmings!