Completely different ELO results Gauntlet vs Duel

Discussion of chess software programming and technical issues.

Moderator: Ras

OliverBr
Posts: 793
Joined: Tue Dec 18, 2007 9:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch

Completely different ELO results Gauntlet vs Duel

Post by OliverBr »

Hello dear chess community,
I have here an unexplained phenomena:

1: In a duell between OliThink 5.11.4b and Minic 2.16, OliThink is about 31 ELO stronger.

Code: Select all

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)     W     D     L  D(%)  CFS(%)
   1 OliThink 5.11.4b    :       0   ----  5443.5   10000  54.4  4324  2239  3437  22.4     100
   2 Minic 2.16          :     -31      6  4556.5   10000  45.6  3437  2239  4324  22.4     ---

White advantage = 31.78 +/- 3.21
Draw rate (equal opponents) = 22.62 % +/- 0.45

2. In a gauntlet tournament it's only 1 ELO stronger:

Code: Select all

   # PLAYER              :  RATING  ERROR  POINTS  PLAYED   (%)     W     D     L  D(%)  CFS(%)
   1 OliThink 5.11.4a    :       5     10   3547.0    6863  51.7   2772  1550   2541  22.6      54
   2 OliThink 5.11.4d    :       4     10   3542.0    6863  51.6   2753  1578   2532  23.0      57
   3 OliThink 5.11.3     :       3     10   3532.5    6863  51.5   2746  1573   2544  22.9      73
   4 OliThink 5.11.4f    :       0   ----   3500.5    6862  51.0   2729  1543   2590  22.5      88
   5 OliThink 5.11.4b    :      -6     10   3442.0    6864  50.1   2692  1500   2672  21.9      61
   6 Minic 2.16          :      -7      7  16751.0   34315  48.8  12879  7744  13692  22.6     ---

White advantage = 25.96 +/- 1.67
Draw rate (equal opponents) = 22.66 % +/- 0.22

White advantage = 31.78 +/- 3.21
Draw rate (equal opponents) = 22.62 % +/- 0.45
First I suspected some "memory" of Minic, but even with option "restart=on": the same result.

Who has an explanation for this?

Software Used:

Code: Select all

cutechess-cli 1.4.0-beta3
ordo 1.2.6
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink
OliverBr
Posts: 793
Joined: Tue Dec 18, 2007 9:38 pm
Location: Munich, Germany
Full name: Dr. Oliver Brausch

Re: Completely different ELO results Gauntlet vs Duel

Post by OliverBr »

It is reproducible. The more opponents in the gauntlet, the better Minic 2.16 performs. And it's not an error in the ELO calculation as clearly shown by the win/draw ratio of Minic.

Does anybody else start gauntlet tournaments with different versions of its engine?
Chess Engine OliThink: http://brausch.org/home/chess
OliThink GitHub:https://github.com/olithink