SF 17 coming soon

[Jouni]

What to expect? This is 8 core test vs SF 16 with 8 moves book

Code: Select all

Elo: 8.16 ± 0.7 (95%) LOS: 100.0%
Total: 35004 W: 1380 L: 558 D: 33066
Ptnml(0-2): 2, 215, 16249, 1031, 5
nElo: 43.43 ± 3.1 (95%) PairsRatio: 4.77 

So +8 Elo in 14 months.

[Antihelion]

Code: Select all

Elo: 15.10 ± 0.8 (95%) LOS: 100.0%
Total: 57036 W: 4142 L: 1664 D: 51230
Ptnml(0-2): 12, 990, 24075, 3390, 51
nElo: 54.24 ± 2.7 (95%) PairsRatio: 3.43

Code: Select all

Elo: 46.30 ± 1.5 (95%) LOS: 100.0%
Total: 51218 W: 16664 L: 9878 D: 24676
Ptnml(0-2): 35, 3020, 12855, 9522, 177
nElo: 96.76 ± 3.2 (95%) PairsRatio: 3.17

Code: Select all

Elo: 42.46 ± 1.8 (95%) LOS: 100.0%
Total: 29390 W: 9403 L: 5829 D: 14158
Ptnml(0-2): 5, 1409, 8316, 4937, 28
nElo: 96.72 ± 4.2 (95%) PairsRatio: 3.51

[Luigi335]

There is no more religion!

Only with insane openings can one hope to measure some improvement ...

[BrendanJNorman]

There is no more religion!

Only with insane openings can one hope to measure some improvement ...

There are many ways to measure improvement if you're creative enough imo.

Example:

1. Use a constant opponent who is strong, but measurably weaker (let's say Komodo 7).
2. Play a constant number of game vs constant opponent, let's say 1000 games.
3. Take the average number of moves in each win and get a "Average win length " number.
4. Goal is to make the wins shorter and shorter.

Could call this hypothetical example the "K7 Test".

Test all new engines vs K7 and report the average length of wins after 1000 games.

This approach is just off the top of my head and can be tweaked, but you can see it isn't that difficult to dream up ways to measure strength increases besides constant "bull headbutting equal strength bull" matches.

[Viz]

Winning in a shorter way wouldn't make you stronger in a common sense.
In fact when stockfish had contempt that actually brought elo high values of it did exactly the opposite - sf was just refusing to trade pieces in hope of opponent missteps and because of this it was winning very very grindy games with, making closed positions from like all of the openings.
But it indeed was extremely effective, one simple setting brought up to 50 elo vs engines that were 200 elo weaker.
So trying to win faster vs a weaker engine usually backfires in terms of elo.

[BrendanJNorman]

Winning in a shorter way wouldn't make you stronger in a common sense.
In fact when stockfish had contempt that actually brought elo high values of it did exactly the opposite - sf was just refusing to trade pieces in hope of opponent missteps and because of this it was winning very very grindy games with, making closed positions from like all of the openings.
But it indeed was extremely effective, one simple setting brought up to 30-40 elo vs engines that were 150 elo weaker.

All chess players (some non-chessplaying engine tinkerers might not know) know that if you beat your opponent in 30 moves, and your friend beats his opponent after a long 66 moves endgame - that you are probably more superior to your opponent in strength than your friend is.

There are anomalies, but the rule is reliable.

More than some anecdotes about contempt and so on...

Put Stockfish 10 vs Komodo 7 and Compare win lengths vs Stockfish 16.1 vs Komodo 7.

Another option could be "average of ten shortest wins" which could even out stylistic issues (aggressive players have more short wins).

SF16's shortest average 10 wins will be shorter than SF10's and a reliable indication of strength increase.

Just a guess, but its an interesting test don't you think?

[pohl4711]

There is no more religion!

Only with insane openings can one hope to measure some improvement ...

These openings are UHO. Which means Unbalanced Human Openings. Each opening line was played by humans. So, calling these openings insane means, calling these humans insane. And that is really insane and an insult to these chessplayers.

[Graham Banks]

There is no more religion!

Only with insane openings can one hope to measure some improvement ...

These openings are UHO. Which means Unbalanced Human Openings. Each opening line was played by humans. So, calling these openings insane means, calling these humans insane. And that is really insane and an insult to these chessplayers.

To be fair, how many were played by GM's?

[AndrewGrant]

What to expect? This is 8 core test vs SF 16 with 8 moves book

Code: Select all

Elo: 8.16 ± 0.7 (95%) LOS: 100.0%
Total: 35004 W: 1380 L: 558 D: 33066
Ptnml(0-2): 2, 215, 16249, 1031, 5
nElo: 43.43 ± 3.1 (95%) PairsRatio: 4.77 

So +8 Elo in 14 months.

As everyone competent in the space knows, these dead drawn books cannot be used to gauge the strength of engines anymore.

[Graham Banks]

What to expect? This is 8 core test vs SF 16 with 8 moves book

Code: Select all

Elo: 8.16 ± 0.7 (95%) LOS: 100.0%
Total: 35004 W: 1380 L: 558 D: 33066
Ptnml(0-2): 2, 215, 16249, 1031, 5
nElo: 43.43 ± 3.1 (95%) PairsRatio: 4.77 

So +8 Elo in 14 months.

As everyone competent in the space knows, these dead drawn books cannot be used to gauge the strength of engines anymore.

Here are the top humans. From number 2 down to number 20 is a 70 Elo range. More compressed than the range of the top 20 engines.
We don't ask them to play highly unfavourable opening lines.