Super Tournament XXXVII

[Rebel]

Why was this game declared a draw? They were shuffling, but the "winning" side was going to make a pawn move and change everything sooner or later.

Hi Chris,

I've probably mentioned this before, but I've always used the default ChessGUI adjudication settings:



98% of the time, they're pretty much right.

I don't even fully understand the ChessGUI picture, but lemme give it a try.

1. Looks like you rewrote the 50 move rule to 30 moves with an abs(score<80) for last 10 moves.

Haven't changed anything yet.

2. To declare a win when the last 4 moves are within 4.75 is strange with nowadays high scores, I use 9.99 and never decide a game when the losing engine still has a queen.

3. No idea what Max Game Length = 50 means. Probably don't do any adjudication before move 50? Please enlighten me.

For testing to avoid long silly draws I use a game length of at least 160 moves when the mutual scores have remained under one pawn, for serious rating games 160 is a bit too low, I would suggest 200.

I don't set a move limit.

The picture is about Game adjudication, please describe the meaning of the 8 listed settings. I get the feeling you don't understand them and neither do I.

[Modern Times]

Very clear to me, and will be to Graham too.

[Graham Banks]

Very clear to me, and will be to Graham too.

Having run well over a million games at this time control (most using ChessGUI), I find it quite disrespectful to have it insinuated that I don't understand the settings I use.

[lkaufman]

If my memory serves me correctly, TCEC aim for a draw rate below 70%.

I think that is reasonable.

70% draw rate for top engines at long time controls on big hardware is hard to achieve, they only can do it by very unbalanced openings. But if you start with a database of blitz games on 1 or a few cores by a wide range of engines, it is just a normal result with normal openings. That's roughly the draw rate in the Hiarcs book, which is overwhelmingly good openings. Such openings will produce 99% or more draws with TCEC conditions, at least in the final stages. My point is that you have to set the limit far below 70% in a database such as Hiarcs opening book or CCRL blitz games, in order to get anywhere near a 70% draw rate among the top few engines on 4 or more threads at Rapid TC.

[Krzysztof Grzelak]

Hi Graham,
please enter 0.15 and it should be fine.

[Modern Times]

Back in the day when I used ChessGUI, I had the following:

Game > 59 moves and last 10 moves less then 5 centipawns = adjudicate draw
Adjudicate a tablebase result (draw or win) but not before the distance got to less than 10 moves away
Adjudicate a win if eval was > 950 centipawns for 5 moves

The latter two meant that the games were almost played out to their conclusion anyway, so maybe I shouldn't have bothered. When I did watch games I liked to see how the engines played them out though.

These days I only use CuteChess, and there I don't have any evaluation based adjudication at all, I only have tablebase adjudication turned on. That is what SPCC does and I decided to do the same.

[Rebel]

Very clear to me, and will be to Graham too.

Having run well over a million games at this time control (most using ChessGUI), I find it quite disrespectful to have it insinuated that I don't understand the settings I use.

Maybe so and I am happy to withdraw that, but you did not answer my question and explain the 8 settings you use for game adjudication.

[Krzysztof Grzelak]

Back in the day when I used ChessGUI, I had the following:

Game > 59 moves and last 10 moves less then 5 centipawns = adjudicate draw
Adjudicate a tablebase result (draw or win) but not before the distance got to less than 10 moves away
Adjudicate a win if eval was > 950 centipawns for 5 moves

The latter two meant that the games were almost played out to their conclusion anyway, so maybe I shouldn't have bothered. When I did watch games I liked to see how the engines played them out though.

These days I only use CuteChess, and there I don't have any evaluation based adjudication at all, I only have tablebase adjudication turned on. That is what SPCC does and I decided to do the same.

SPCC can't be taken seriously just humorously. Because of a bad debut book.

[Modern Times]

Very clear to me, and will be to Graham too.

Having run well over a million games at this time control (most using ChessGUI), I find it quite disrespectful to have it insinuated that I don't understand the settings I use.

Maybe so and I am happy to withdraw that, but you did not answer my question and explain the 8 settings you use for game adjudication.

It is 3 settings not 8, look at my post and you'll understand what his screenshot is saying.

[chrisw]

I have no inclination or desire to start, or to be involved with, a new rating list.

If/when I stop my CCRL testing, I'll happily continue to run my Amateur Series tournaments for those still interested, although probably with an incremental time control.

I run engine v engine testing because I enjoy it - the tournaments in particular.
I like watching some of the games, which is why bullet or blitz hold no interest for me whatsoever.

No one was suggesting blitz games to replace Rapid. My suggestion was to retain the current pace of your 40/15 games for the first 60 moves or so, then use increment to speed up the long endgames (which are usually drawn) a bit, especially if the adjudication rules are tightened. Maybe it wouldn't even need a new list, perhaps the change would be deemed minor enough to combine them. I would think that would make them more enjoyable to watch with no loss in average quality.

Do you think that 30 minutes with 10 second increments would be on a par with 40/15 games, which on average take 50 minutes to complete (if using 40/16)?
On my 5950x, I use 40/11 repeating, with the average games taking around 35 minutes, so I'm guessing that would be about 20 minutes with 7 second increments?

I could also drop the draw adjudication from 10 consecutive moves past move 60 with less than 80, to less than 30.

I do draw the line at using what I perceive to be unfair opening lines though.

I think you meant to say 15 minutes (per player) plus 10 second increment would be equal to 40/16, since 15 min plus 10 sec games would take fifty minutes for sixty moves with all time used. For your machine, you mean to say 10 minutes per player plus 7 seconds, which would take 34 minutes for sixty moves with all time used. Probably without adjudication (or with minimal adjudication) the average would be somewhat more than sixty moves, so I would recommend 12 minutes plus 8 seconds on your 5950x as the most equivalent to your current time control. I can assure you that the quality of play will in general be higher with 12 minutes plus 8 seconds, though of course it will be slightly lower when playing out very long endgames.

Adjudication at 30 cp is of course much better than at 80 cp, but as far as I know no one else uses adjudications at any double-digit centipawn value. I agree with Chris than zero would be best (with increment to make it practical), but any single-digit value is okay from my perspective.

Regarding "unfair" opening lines, I suppose you mean lines that are likely winning, or perhaps very close to winning, for one side (presumably White). I think that we all agree that winning lines should be excluded; it's just a question of where to draw the line for likely or nearly winning. I would specify a maximum White score, definitely below 75% since above that indicates a likely won position; 70% seems a good value to me. Setting it too low would mean almost all draws when the top engines meet.

I would argue that 70% is a bit low, since it would leave out a bunch of King's Indians and Benonis, which are too big a part of the common chess heritage. While White gets a sizable advantage in theory, I would not say they're automatic losses for Black, and nowhere near "grotesquely unbalanced lines", as Chris put it.

Checking the HIarcs database, which I believe is mostly strong engine games now, the mar del plata king's indian after 8...Ne7, presumably the main line, is only showing 55.5% for White. The main line Benoni with 8.h3 and 9.Bd3 shows 61.3% for White, still far below 70%. Of course you can find some minor sidelines that exceed 70%, but no need for such lines in a book, the lines actually played by grandmasters with any frequency are almost always below 70% score for White. Probably it would be best to use stats from a book such as Hiarcs book, which has 1.8+ million games and an average Elo showing over 2900. With samples and quality like this, no need to even consider engine evals, just set a maximum White score like 70% and a maximum total draw percentage, perhaps 55 to 60% (the draw % in the Hiarcs book is in the 68-69% range), and a minimum number of total games from the position, whatever value produces the desired number of total games.

Here you are talking about making unbalanced human openings based on game results, something which has already been done.

The first book I have to Graham for this tourney was same idea, only using comp-comp games and with a high eval cull (0.70). This did no better than a random book for decreasing draw rate.

To lower the draw rate, one MUST not cull high evals, one must cull low evals (out of book).
Alternatively, no culling of high nondrawrate from results, culling of lownondrawrate.