ArenaSwiss 0.3a

Discussion of computer chess matches and engine tournaments.

Moderator: Ras

Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: ArenaSwiss 0.3c

Post by Ferdy »

GONeill wrote:I'm pleased you enjoy using the program.

The term 'Fine' was in the original version from Eberhard and I left it there in case it meant something. But 'Tie Break' is better, so I'll change it.

It's the same with the odd number of players. Eberhard wrote it to drop the last player (it shows this in the ArenaSwiss.log file). I'll change it to include a '[Bye]' player. I assume 1 point for a bye is standard practise?
I think it to be so, 1 point for a bye for players with lowest score or for first round the lowest rated, or perhaps seeded.

But I just learned also that score for unplayed games like this cannot be rated. Though it gets a score of say 1, there is no actual rating of opponent that we can get from, hence it is unrated :) . I will not play odd number of players in a tournament then as it only complicates calculation of rating.

Some further comments.
Regarding the table generated by ArenaSwiss, I would suggest to place the Elo column right after the engine column. As human would look at the engine, it usually asks what is the rating of this? Also these are the ratings of engines before the tournament.

Code: Select all

Rank Engine      1    2    3    4    5    6   Score    Fine    ELO   White
--------------------------------------------------------------------------
  1: Ozwald    11W+  6B+  7W+  2B+  4B+  3W+    6.0     20.0   1280     3
  2: Amyan      4B+  5W+  3B+  1W-  7B+  6W=    4.5     23.0   1200     3
  3: Kace      12B+ 14B+  2W-  5W+ 10B+  1B-    4.0     18.5    929     2
  4: Rybka      2W- 10W=  9B+  6B+  1W-  7B+    3.5     22.0   1200     3
  5: Tornado   14W+  2B-  6W=  3B-  8B+ 10W+    3.5     17.0   1200     3
  6: Cheng4    10B+  1W-  5B=  4W- 11W+  2B=    3.0     22.0   1200     3
  7: Houdini    8B+ 13W+  1B- 11B+  2W-  4W-    3.0     21.0   1200     3
  8: Blitzter   7W-  9W- 12B+ 13B+  5W- 11B+    3.0     15.5   1034     3
  9: Hiarcs    13W-  8B+  4W- 10W- 12B+ 14B+    3.0     13.0   1200     3
 10: Ufim       6W-  4B= 14W+  9B+  3W-  5B-    2.5     17.0   1200     3
 11: Rodent     1B- 12W+ 13B+  7W-  6B-  8W-    2.0     19.0   1200     3
 12: MadChess   3W- 11B-  8W- 14B+  9W- 13B+    2.0     14.0   1200     3
 13: Jchecs     9B+  7B- 11W-  8W- 14B+ 12W-    2.0     13.0   1107     3
 14: Arasan     5B-  3W- 10B- 12W- 13W-  9W-    0.0     17.0   1200     4
I have a request, if this does not take too much time from you :).
Add a new column to calculate the rating change (RC) or gain. For example,
Kace 929 defeated MadChess 1200 in the first round for example.
RC of Kace = K * (S-E)
where:
K = 10, a constant, but with some different rules.
S = 1.0, this is the score against MadChess
E = Expected score of Kace vs MadChess, with the formula below, a function of ratings of the two players
E = 1 / (1 + 10^( (1200-929)/400) )

In this case,
RC of Kace = 10 * (1.0-E) = 10 * (1.0 - 0.174) = +8.3
This is how FIDE calculates its rating change, but with different K value depending on some situations.

Then doing it every opponent of Kace, and Sum those, to get the RC for this tournament. Currently I relies on scid vs pc and CB12 calculation.

Here is the sample output of scid vs pc. That Chg column is the total rating change/gain.

Code: Select all

Swiss5b
Home, 2014.07.18 - 2014.07.19
Average Rating: 1167
             Rtng    Score    (Tie)   1    2    3    4    5    6    Perf Chg
-----------------------------------------------------------------------------
 1: Ozwald   1280   6.0 /  6  20.0  11w+  6b+  7w+  2b+  4b+  3w+   2154 +20
 2: Amyan    1200   4.5 /  6  23.0   4b+  5w+  3b+  1w-  7b+  6w=   1361 +12
 3: Kace      929   4.0 /  6  18.5  12b+ 14b+  2w-  5w+ 10b+  1b-   1338 +31
 4: Rybka    1200   3.5 /  6  22.0   2w- 10w=  9b+  6b+  1w-  7b+   1270  +6
 5: Tornado  1200   3.5 /  6  17.0  14w+  2b-  6w=  3b-  8b+ 10w+   1184  -2
 6: Cheng4   1200   3.0 /  6  22.0  10b+  1w-  5b=  4w- 11w+  2b=   1213  +1
 7: Houdini  1200   3.0 /  6  21.0   8b+ 13w+  1b- 11b+  2w-  4w-   1170  -3
 8: Blitzter 1034   3.0 /  6  15.5   7w-  9w- 12b+ 13b+  5w- 11b+   1184 +13
 9: Hiarcs   1200   3.0 /  6  13.0  13w-  8b+  4w- 10w- 12b+ 14b+   1156  -4
10: Ufim     1200   2.5 /  6  17.0   6w-  4b= 14w+  9b+  3w-  5b-   1097  -9
11: Rodent   1200   2.0 /  6  19.0   1b- 12w+ 13b+  7w-  6b-  8w-   1045 -13
12: MadChess 1200   2.0 /  6  14.0   3w- 11b-  8w- 14b+  9w- 13b+    986 -18
13: Jchecs   1107   2.0 /  6  13.0   9b+  7b- 11w-  8w- 14b+ 12w-   1047  -5
14: Arasan   1200   0.0 /  6  17.0   5b-  3w- 10b- 12w- 13w-  9w-    140 -35
-----------------------------------------------------------------------------
42 games: +11 -28 =3
User avatar
GONeill
Posts: 87
Joined: Sun Jun 15, 2014 6:40 am
Location: New Zealand
Full name: Graham O'Neill

Re: ArenaSwiss 0.3c

Post by GONeill »

Hi Ferdy,

Both good ideas, and I'm now testing version 0.3d. This adds a 'Bye' player if the number of engines is odd and gives a 1-0 win to any opponent paired with it.

But I have done it so that Bye games are not included in the rating change calculation, which I am also including. Rather than use your formula for E, I am using the table from FIDE (section 8.1b):

http://www.fide.com/fide/handbook.html? ... ew=article

Hopefully I won't have to change it in a couple of weeks, as this document ends in July 2014!

I saw your message in the other forum about SCIDvsPC calculating ratings and now I see why it thinks Ozwald is +20 rather than +21 in your example. Don't worry, my calculation is based on the actual game scores against each opponent (e.g. 4.5-3.5 in an 8 game pairing rather than just the 1-0 result). I've used K=10, but since it is a constant applied to all games, if you wanted to use 15 (for Blitz) you just divide the rating change by 10 and multiply by 15. I haven't taken into account the unrated player rules though.

And I have also reformatted the results table as suggested.

I should have 0.3d ready for you in the next couple of days.

Regards,
Graham
Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: ArenaSwiss 0.3c

Post by Ferdy »

I haven't taken into account the unrated player rules though.
There is no need for this as for engine letting it play to get more games is easier than humans. In my rating list for engines using fide elo calculation, all engines are considered rated players and got their initial rating before the tournament.

Thanks a lot for implementing the rating change and other stuff.
User avatar
GONeill
Posts: 87
Joined: Sun Jun 15, 2014 6:40 am
Location: New Zealand
Full name: Graham O'Neill

ArenaSwiss 0.3d

Post by GONeill »

ArenaSwiss 0.3d

All references to 'Fine' have been changed to 'Tie Break'. Also the program allows for Byes when there are an odd number of players and it calculates the rating changes based on the game results.

The download is still at:

http://goneill.co.nz/arenaswiss

Regards,
Graham

PS. Don't interrupt a tournament running in 0.3c to load this new version as the results file layout has been changed.
Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: ArenaSwiss 0.3d

Post by Ferdy »

Thanks again Graham, looks very good with a new feature on K=10 fide elo rating change automatic calculation.

Code: Select all

================================= ArenaSwiss Ver. 0.3d =================================

Tournament name:   Swiss100-UCI_Elo_1200
Tournament start:  Tue, 22 Jul 2014, 10:14:56

Pairing method:    Fide
Tie Break method:  Buchholz
Time control:      Tournament 40 in 3 min repeated
Conditions:        Hash: 128MB, Tablebases: Off, Ponder Off
No. of rounds:     7
Games per pairing: 1

[...]

 PARTICIPANTS 
==============

  1: Arasan v17.1 elo1200             1200
  2: Cheng4 v0.36c elo1200            1200
  3: Deuterium v14.2.33.276 elo1200   1200
  4: Hiarcs v14 elo1200               1200
  5: Houdini v4 elo1200               1200
  6: MadChess v1.4 elo1200            1200
  7: Rodent v1.3 elo1200              1200
  8: Rybka v2.3.2a elo1200            1200
  9: Tornado v4.4 elo1200             1200
 10: Ufim v8.02 elo1200               1200
 11: Amyan v1.72 elo1200              1200
 12: <Bye>                               1


 ROUND 1
=========

 <Bye>                           0.0 -- 1.0  MadChess v1.4 elo1200         
 Houdini v4 elo1200              0.0 -- 1.0  Amyan v1.72 elo1200           
 Hiarcs v14 elo1200              0.0 -- 1.0  Ufim v8.02 elo1200            
 Tornado v4.4 elo1200            1.0 -- 0.0  Deuterium v14.2.33.276 elo1200
 Cheng4 v0.36c elo1200           0.5 -- 0.5  Rybka v2.3.2a elo1200         
 Rodent v1.3 elo1200             0.0 -- 1.0  Arasan v17.1 elo1200

Rank Engine                           ELO    1   Score      Tie  White   ELO
----------------------------------------------------------------------------
  1: Amyan v1.72 elo1200             1200   9B+    1.0      0.0      0    +5
  2: Tornado v4.4 elo1200            1200  11W+    1.0      0.0      1    +5
  3: Ufim v8.02 elo1200              1200  10B+    1.0      0.0      0    +5
  4: Arasan v17.1 elo1200            1200   8B+    1.0      0.0      0    +5
  5: MadChess v1.4 elo1200           1200  12B+    1.0      0.0      0     0
  6: Cheng4 v0.36c elo1200           1200   7W=    0.5      0.5      1     0
  7: Rybka v2.3.2a elo1200           1200   6B=    0.5      0.5      0     0
  8: Rodent v1.3 elo1200             1200   4W-    0.0      1.0      1    -5
  9: Houdini v4 elo1200              1200   1W-    0.0      1.0      1    -5
 10: Hiarcs v14 elo1200              1200   3W-    0.0      1.0      1    -5
 11: Deuterium v14.2.33.276 elo1200  1200   2B-    0.0      1.0      0    -5
 12: <Bye>                              1   5W-    0.0      1.0      1     0

[...]

 ROUND 3
=========

 Deuterium v14.2.33.276 elo1200  1.0 -- 0.0  <Bye>                         
 Hiarcs v14 elo1200              1.0 -- 0.0  Rodent v1.3 elo1200           
 MadChess v1.4 elo1200           0.0 -- 1.0  Tornado v4.4 elo1200          
 Arasan v17.1 elo1200            0.0 -- 1.0  Houdini v4 elo1200            
 Ufim v8.02 elo1200              0.0 -- 1.0  Rybka v2.3.2a elo1200         
 Cheng4 v0.36c elo1200           0.0 -- 1.0  Amyan v1.72 elo1200           


Rank Engine                           ELO    1    2    3   Score      Tie  White   ELO
--------------------------------------------------------------------------------------
  1: Amyan v1.72 elo1200             1200   5B+  3W+  6B+    3.0      5.5      1   +15
  2: Rybka v2.3.2a elo1200           1200   6B=  7W+  4B+    2.5      4.5      1   +10
  3: Tornado v4.4 elo1200            1200  11W+  1B-  9B+    2.0      5.0      1    +5
  4: Ufim v8.02 elo1200              1200   7B+  8B+  2W-    2.0      4.5      1    +5
  5: Houdini v4 elo1200              1200   1W- 12W+  8B+    2.0      4.0      2     0
  6: Cheng4 v0.36c elo1200           1200   2W=  9B+  1W-    1.5      6.5      2     0
  7: Hiarcs v14 elo1200              1200   4W-  2B- 10W+    1.0      5.5      2    -5
  8: Arasan v17.1 elo1200            1200  10B+  4W-  5W-    1.0      5.0      2    -5
  9: MadChess v1.4 elo1200           1200  12B+  6W-  3W-    1.0      3.5      2   -10
 10: Rodent v1.3 elo1200             1200   8W- 11B+  7B-    1.0      3.0      1    -5
 11: Deuterium v14.2.33.276 elo1200  1200   3B- 10W- 12W+    1.0      3.0      2   -10
 12: <Bye>                              1   9W-  5B- 11B-    0.0      4.0      1     0

[...]
Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: ArenaSwiss 0.3d

Post by Ferdy »

Done some test on odd number of participants and have a comment, just a minor issue. After 3 rounds the table is showing <bye> is higher than the player, I thought that a <bye> should always be the last. This table is just an update, but I am concerned if this will happen in the last round table :) .

Code: Select all

Rank Engine                          ELO    1    2    3   Score      Tie  White   ELO
-------------------------------------------------------------------------------------
  1: BlackMamba v1.2c elo1200       1200  11B+  9W+  4W+    3.0      4.0      2   +15
  2: Horizon v4.4 elo1200           1200  15B+  3W+  5W+    3.0      4.0      2   +10
  3: Cheng4 v0.36c elo1200          1200   6W+  2B- 14B+    2.0      6.0      1    +5
  4: Ufim v8.02 elo1200             1200  13W+ 10B+  1B-    2.0      5.0      1    +5
  5: Amyan v1.72 elo1200            1200  14B+ 12W+  2B-    2.0      5.0      1    +5
  6: Rybka v2.3.2a elo1200          1200   3B- 11W+ 12B+    2.0      4.0      1    +5
  7: Deuterium v14.3.34.21 elo1200  1200   9B- 13B+ 10W+    2.0      3.0      1    +5
  8: SlowChess 2.960e elo1200       1200  10W- 16B+  9W+    2.0      2.0      2    +5
  9: Tornado v4.4 elo1200           1200   7W+  1B-  8B-    1.0      7.0      1    -5
 10: Hiarcs v14 elo1200             1200   8B+  4W-  7B-    1.0      6.0      1    -5
 11: Rodent v1.3 elo1200            1200   1W-  6B- 16B+    1.0      5.0      1    -5
 12: MadChess v1.4 elo1200          1200  16W+  5B-  6W-    1.0      4.0      2    -5
 13: DanaSahZ 0.4 elo1200           1200   4B-  7W- 15W+    1.0      4.0      2   -10
 14: Houdini v4 elo1200             1200   5W- 15B+  3W-    1.0      4.0      2   -10
 15: <Bye>                             1   2W- 14W- 13B-    0.0      5.0      2     0
 16: Arasan v17.1 elo1200           1200  12B-  8W- 11W-    0.0      4.0      2   -15
User avatar
GONeill
Posts: 87
Joined: Sun Jun 15, 2014 6:40 am
Location: New Zealand
Full name: Graham O'Neill

Re: ArenaSwiss 0.3d

Post by GONeill »

Bye is above Arasan because they both have zero points and its Tie score is higher.

Because the ELO for the participants are all the same, it is essentially random who got paired with Bye in the first round, but it looks like it was one of the stronger programs (Horizon). So Bye will always benefit from 'losing' that game by having a higher Tie score. As Horizon wins more games, Bye's Tie score will get even higher.

However, Bye will only avoid staying bottom if another player remains on zero points too. But the way Swiss works, in each round one of those players will be paired against Bye so will immediately get one point. So it won't take too many rounds before all players with zero points have been given a Bye (or played another player with zero points and got 1 or half a point).

In your tournament it is very likely that round 4 had Arasan paired with Bye, putting Bye back in bottom position.

I'm sure there is a mathematical formula for the required number of rounds to ensure that only Bye ends up with zero points, but it's too early in the morning for me and I haven't had my coffee yet!
Ferdy
Posts: 4851
Joined: Sun Aug 10, 2008 3:15 pm
Location: Philippines

Re: ArenaSwiss 0.3d

Post by Ferdy »

GONeill wrote:Bye is above Arasan because they both have zero points and its Tie score is higher.
I perfectly understand that. But during presentation of partial result table, the player has to be ahead compared to <Bye> because <Bye> is not a player :) .

Code: Select all

[...]
 13: DanaSahZ 0.4 elo1200           1200   4B-  7W- 15W+    1.0      4.0      2   -10 
 14: Houdini v4 elo1200             1200   5W- 15B+  3W-    1.0      4.0      2   -10 
 15: <Bye>                             1   2W- 14W- 13B-    0.0      5.0      2     0 
 16: Arasan v17.1 elo1200           1200  12B-  8W- 11W-    0.0      4.0      2   -15
User avatar
GONeill
Posts: 87
Joined: Sun Jun 15, 2014 6:40 am
Location: New Zealand
Full name: Graham O'Neill

Re: ArenaSwiss 0.3d

Post by GONeill »

OK! I'm not sure most people will be concerned by this as it's only an interim result, but it's a very small change to the program so I have re-released version 0.3d. The new version forces Bye to have a Tie Break score of zero.
User avatar
GONeill
Posts: 87
Joined: Sun Jun 15, 2014 6:40 am
Location: New Zealand
Full name: Graham O'Neill

ArenaSwiss 0.3e

Post by GONeill »

ArenaSwiss 0.3e

One thing led to another and I ended up making quite a lot of changes for this version!

If there are multiple games per pairing it wasn't possible to interrupt the tournament until the end of each pairing. But if you were playing 8 games per pairing and each one was 2 hours long, then you couldn't stop the tournament until 16 hours later! If you cancelled Arena after the first 7 games ArenaSwiss would start the pairing all over again.

I changed it so that results of each completed game are now retained so that the tournament can be interrupted at any point. Any game that is actually playing will be lost, but all previous results in the pairing will be saved.

That meant the work files in the Arena\Tournaments directory need to be kept separate from any work files from other tournaments, so I simplified how files are named. Now the output files (PGN, TXT, HTML and LOG) are named using the Arena tournament 'Files' screen. All the other files are named using the WorkFiles setting in the INI file.

With it now easy to run different tournaments with different INI files I had to add the ability to run ArenaSwiss with a parameter to specify the INI file name. Without the parameter the default is still Arenaswiss.ini

Having set it to use the 'Files' tab in Arena it made sense to also use the 'Table' tab for the output options for the TXT and HTML outputs. I also changed the HTML output to be an HTML5 page with CSS classes.

There are some other changes such as tie break by ELO change, pause before each pairing and a pause timeout option.

The download is still at:

http://goneill.co.nz/arenaswiss

Regards,
Graham

PS. Don't interrupt a tournament running in 0.3d to load this new version as the results file layout has been changed.