Discussion of computer chess matches and engine tournaments.
Moderator: Ras
Sedat Canbaz
Posts: 3018Joined: Thu Mar 09, 2006 11:58 amLocation: Antalya/Turkey
Post by Sedat Canbaz Wed Aug 29, 2012 7:01 pm
Daniel Shawul wrote:
If you have both collection of games before and after the fruit games were added, I would be happy to do comparisons for you.
SCCT games:
http://www.sedatcanbaz.com/chess/games/scct_3m2s.rar
Note that the current online database includes 29250 games
Where Rybka 4.1 NO-SSE version is played 1000 games per player
Fruit 090705 is played 1150 games per player
And very soon i plan to upload all games (including Fruit's new 50 games,plus Rybka NO-SSE new 500 games and Hiarcs 14's games too)
Best,
Sedat
Laskos
Posts: 10948Joined: Wed Jul 26, 2006 10:21 pmFull name: Kai Laskos
Post by Laskos Wed Aug 29, 2012 8:23 pm
Daniel Shawul wrote:
As a side note Elostat and Ordo agree because they both use simplistic methods to calculate elo. Bayeselo is far advanced than both for realistic predictions of elo. This has been researched a lot (bayeselo vs elostat) so I urge you to look at that yourself if you are into it.
Here Sedat is right. Ordo and EloStat are simpler than Bayeselo, EloStat is even wrong, but Bayeselo is a bit broken in the case presented by Sedat. Ordo and EloStat give more or less correct results for Fruit to _decrease_ its rating after playing with Rybka. Fruit was expected to perform _less_ than 400 Elos weaker than Rybka, but it performed _more_ than 400 Elos weaker. Therefore, in this case, Fruit performed worse than expected, and its rating decreased very slightly (2-3 points as shown by Ordo and EloStat). +16 Elos increase in Fruit strength given by Bayeselo is ridiculous. Also, could you explain the error margins shown by Bayeselo before and after 50 games pretty ordinary match (pretty expected outcome, in line with their respective rating) between Fruit and Rybka?
That all if Sedat presented correctly things.
Kai
Sedat Canbaz
Posts: 3018Joined: Thu Mar 09, 2006 11:58 amLocation: Antalya/Turkey
Post by Sedat Canbaz Wed Aug 29, 2012 9:07 pm
Laskos wrote: Daniel Shawul wrote:
As a side note Elostat and Ordo agree because they both use simplistic methods to calculate elo. Bayeselo is far advanced than both for realistic predictions of elo. This has been researched a lot (bayeselo vs elostat) so I urge you to look at that yourself if you are into it.
Here Sedat is right. Ordo and EloStat are simpler than Bayeselo, EloStat is even wrong, but Bayeselo is a bit broken in the case presented by Sedat. Ordo and EloStat give more or less correct results for Fruit to _decrease_ its rating after playing with Rybka. Fruit was expected to perform _less_ than 400 Elos weaker than Rybka, but it performed _more_ than 400 Elos weaker. Therefore, in this case, Fruit performed worse than expected, and its rating decreased very slightly (2-3 points as shown by Ordo and EloStat). +16 Elos increase in Fruit strength given by Bayeselo is ridiculous. Also, could you explain the error margins shown by Bayeselo before and after 50 games pretty ordinary match (pretty expected outcome, in line with their respective rating) between Fruit and Rybka?
That all if Sedat presented correctly things.
Kai
Thanks a lot for your useful comments dear Kai
I thought that i am only one who believes in that way...
Btw,
Actually i keep all BayesElo calculations,so here are:
1st.Calculation:Rybka 4.1 NO-SSE x64 6c + 1000 games player Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3363 12 12 1700 70% 3211 39%
2 Houdini 2.0t3* Pro x64 6c 3362 15 15 1000 75% 3177 37%
3 Houdini 2.0z Pro x64 6c 3358 12 12 1550 71% 3193 36%
4 Houdini 2.0s2 Pro x64 6c 3356 15 15 1000 74% 3170 34%
5 Houdini 1.5a x64 6c 3345 14 14 1100 68% 3212 41%
6 Houdini 2.0Bar2 x64 6c 3343 15 15 1000 73% 3177 43%
7 Houdini 2.0c Pro x64 6c 3343 13 13 1450 71% 3183 39%
8 Houdini 2.0Higgs Pro x64 6c 3339 15 15 1000 71% 3186 42%
9 Houdini2Bar1 Pro x64 6c 3329 14 14 1100 69% 3193 46%
10 Critter 1.6 x64 6c 3300 10 10 1900 63% 3209 53%
11 Critter 1.4 x64 6c 3288 14 13 1150 67% 3164 47%
12 Rybka 4.1 79DT v1 x64 6c 3287 14 14 1100 66% 3168 38%
13 Stockfish 120430P x64 6c 3282 11 11 1850 60% 3208 50%
14 Deep Rybka 4.1 x64 6c 3274 11 11 1750 60% 3204 48%
15 Stockfish 2.2.2 JA x64 6c 3274 13 13 1200 62% 3184 47%
16 Ivanhoe B46fE.02 x64 6c 3273 10 10 1900 59% 3210 53%
17 Rybka 4.1 NO-SSE x64 6c 3273 14 14 1000 63% 3181 49%
18 Ivanhoe B46fC x64 6c 3273 13 13 1200 64% 3173 47%
19 Stockfish VE09 x64 6c 3263 14 14 1000 63% 3170 48%
20 Fire 2.2 xTreme x64 6c 3260 10 10 1900 57% 3210 52%
21 Vitruvius 1.11C x64 6c 3257 10 10 1900 56% 3210 51%
22 Gull II beta2 x64 6c 3209 12 12 1400 50% 3204 51%
23 Strelka 5.5 x64 1c 3189 11 11 1650 45% 3224 48%
24 Bouquet 1.4 x64 6c 3177 13 13 1250 46% 3201 47%
25 Naum 4.2 x64 6c 3168 10 10 1900 44% 3213 44%
26 Komodo 4.0 x64 1c 3149 11 11 1900 41% 3213 42%
27 Equinox 1.35 x64 6c 3117 12 12 1550 40% 3186 40%
28 Deep Fritz 13 w32 6c 3116 11 11 1900 36% 3214 43%
29 Spike 1.4 Leiden w32 6c 3097 11 11 1900 34% 3214 38%
30 Chiron 1.1a x64 6c 3095 11 11 1900 34% 3214 39%
31 Deep Fritz 12 w32 6c 3080 14 14 1150 37% 3173 42%
32 Deep Junior 13.3 x64 6c 3077 12 12 1700 31% 3222 36%
33 Protector 1.4.0 x64 6c 3073 11 11 1900 31% 3215 36%
34 Spark 1.0 x64 6c 3070 11 11 1850 31% 3212 39%
35 Deep Junior 13 x64 6c 3068 13 13 1300 35% 3181 36%
36 Deep Shredder 12 x64 6c 3067 11 11 1900 30% 3215 37%
37 Hiarcs 13.2 w32 6c 3051 11 11 1900 29% 3216 32%
38 Zappa Mexico II x64 6c 3035 12 12 1550 29% 3197 34%
39 Fruit 090705 x64 6c 2965 15 15 1150 23% 3178 29%
2nd.Calculation:Rybka 4.1 NO-SSE x64 6c + 1261 games player Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3363 12 12 1700 70% 3212 39%
2 Houdini 2.0t3* Pro x64 6c 3362 15 15 1000 75% 3177 37%
3 Houdini 2.0z Pro x64 6c 3358 12 12 1574 71% 3194 36%
4 Houdini 2.0s2 Pro x64 6c 3356 15 15 1000 74% 3171 34%
5 Houdini 2.0Bar2 x64 6c 3345 15 14 1030 73% 3180 44%
6 Houdini 1.5a x64 6c 3345 14 14 1100 68% 3212 41%
7 Houdini 2.0c Pro x64 6c 3342 13 12 1473 71% 3184 39%
8 Houdini 2.0Higgs Pro x64 6c 3341 15 15 1030 71% 3189 41%
9 Houdini2Bar1 Pro x64 6c 3330 14 14 1100 69% 3193 46%
10 Critter 1.6 x64 6c 3300 11 11 1900 63% 3209 53%
11 Critter 1.4 x64 6c 3288 13 13 1173 67% 3166 47%
12 Rybka 4.1 79DT v1 x64 6c 3287 14 14 1100 66% 3168 38%
13 Stockfish 120430P x64 6c 3282 11 11 1850 60% 3208 50%
14 Rybka 4.1 SSE42 x64 6c 3275 11 11 1781 60% 3206 48%
15 Stockfish 2.2.2 JA x64 6c 3274 13 13 1200 62% 3185 47%
16 Ivanhoe B46fE.02 x64 6c 3273 10 10 1900 59% 3210 53%
17 Ivanhoe B46fC x64 6c 3273 13 13 1230 63% 3176 48%
18 Rybka 4.1 NO-SSE x64 6c 3271 13 13 1261 61% 3193 49%
19 Stockfish VE09 x64 6c 3263 14 14 1000 63% 3170 48%
20 Fire 2.2 xTreme x64 6c 3260 10 10 1900 57% 3210 52%
21 Vitruvius 1.11C x64 6c 3258 10 10 1900 56% 3210 51%
22 Gull II beta2 x64 6c 3209 12 12 1400 50% 3204 51%
23 Strelka 5.5 x64 1c 3189 11 11 1650 45% 3224 48%
24 Bouquet 1.4 x64 6c 3177 13 13 1250 46% 3201 47%
25 Naum 4.2 x64 6c 3169 11 11 1900 44% 3213 44%
26 Komodo 4.0 x64 1c 3149 11 11 1900 41% 3213 42%
27 Equinox 1.35 x64 6c 3117 12 12 1550 40% 3187 40%
28 Deep Fritz 13 w32 6c 3117 11 11 1900 36% 3214 43%
29 Spike 1.4 Leiden w32 6c 3098 11 11 1900 34% 3215 38%
30 Chiron 1.1a x64 6c 3095 11 11 1900 34% 3215 39%
31 Deep Fritz 12 w32 6c 3079 14 14 1173 37% 3175 42%
32 Deep Junior 13.3 x64 6c 3078 12 12 1700 31% 3223 36%
33 Protector 1.4.0 x64 6c 3073 11 11 1900 31% 3215 36%
34 Spark 1.0 x64 6c 3070 11 11 1850 31% 3212 39%
35 Deep Junior 13 x64 6c 3068 13 13 1300 35% 3181 36%
36 Deep Shredder 12 x64 6c 3067 11 11 1900 30% 3215 37%
37 Hiarcs 13.2 w32 6c 3051 11 11 1900 29% 3216 32%
38 Zappa Mexico II x64 6c 3038 12 12 1573 29% 3198 34%
39 Fruit 090705 x64 6c 2966 15 15 1174 23% 3180 29% 3rd.Calculation:Rybka 4.1 NO-SSE x64 6c + 1500 games player
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3359 14 14 1700 70% 3217 39%
2 Houdini 2.0t3* Pro x64 6c 3359 19 19 1000 75% 3185 37%
3 Houdini 2.0z Pro x64 6c 3356 15 15 1600 71% 3202 36%
4 Houdini 2.0s2 Pro x64 6c 3355 19 19 1000 74% 3179 34%
5 Houdini 1.5a x64 6c 3342 17 17 1100 68% 3218 41%
6 Houdini 2.0Bar2 x64 6c 3342 18 18 1050 72% 3190 44%
7 Houdini 2.0c Pro x64 6c 3341 15 15 1500 71% 3193 39%
8 Houdini 2.0Higgs Pro x64 6c 3338 18 18 1050 70% 3198 42%
9 Houdini2Bar1 Pro x64 6c 3328 17 17 1100 69% 3200 46%
10 Critter 1.6 x64 6c 3300 13 13 1900 63% 3215 53%
11 Critter 1.4 x64 6c 3290 16 16 1200 66% 3177 47%
12 Rybka 4.1 79DT v1 x64 6c 3287 17 17 1100 66% 3176 38%
13 Stockfish 120430P x64 6c 3284 13 13 1850 60% 3214 50%
14 Rybka 4.1 SSE42 x64 6c 3276 13 13 1800 59% 3212 49%
15 Ivanhoe B46fC x64 6c 3276 16 16 1250 63% 3185 48%
16 Ivanhoe B46fE.02 x64 6c 3276 13 13 1900 59% 3216 53%
17 Stockfish 2.2.2 JA x64 6c 3275 16 16 1200 62% 3192 47%
18 Rybka 4.1 NO-SSE x64 6c 3275 14 14 1500 60% 3204 49%
19 Fire 2.2 xTreme x64 6c 3263 12 12 1900 57% 3216 52%
20 Stockfish VE09 x64 6c 3263 17 17 1000 63% 3178 48%
21 Vitruvius 1.11C x64 6c 3261 13 13 1900 56% 3216 51%
22 Gull II beta2 x64 6c 3215 15 14 1400 50% 3211 51%
23 Strelka 5.5 x64 1c 3198 14 14 1650 45% 3229 48%
24 Bouquet 1.4 x64 6c 3185 15 15 1250 46% 3207 47%
25 Naum 4.2 x64 6c 3178 13 13 1900 44% 3218 44%
26 Komodo 4.0 x64 1c 3160 13 13 1900 41% 3219 42%
27 Equinox 1.35 x64 6c 3129 14 14 1550 40% 3194 40%
28 Deep Fritz 13 w32 6c 3129 13 13 1900 36% 3220 43%
29 Spike 1.4 Leiden w32 6c 3110 13 14 1900 34% 3220 38%
30 Chiron 1.1a x64 6c 3108 13 13 1900 34% 3220 39%
31 Deep Fritz 12 w32 6c 3093 16 17 1200 36% 3185 42%
32 Deep Junior 13.3 x64 6c 3092 14 15 1700 31% 3228 36%
33 Protector 1.4.0 x64 6c 3087 14 14 1900 31% 3221 36%
34 Spark 1.0 x64 6c 3084 14 14 1850 31% 3218 39%
35 Deep Junior 13 x64 6c 3082 16 16 1300 35% 3189 36%
36 Deep Shredder 12 x64 6c 3080 14 14 1900 30% 3221 37%
37 Hiarcs 13.2 w32 6c 3064 14 14 1900 29% 3221 32%
38 Zappa Mexico II x64 6c 3053 15 15 1600 29% 3206 34%
39 Fruit 090705 x64 6c 2981 18 18 1200 23% 3190 29% Games:
http://www.sedatcanbaz.com/chess/games/scct_3m2s_2.rar
*Note:the current online database includes all games (total:30408 games, up to 20:56 29.08.2012)
The previous database is still available (29250 games,where Fruit 090705 is played 1150 games per player):
http://www.sedatcanbaz.com/chess/games/scct_3m2s.rar
Best Regards,
Sedat
Sedat Canbaz
Posts: 3018Joined: Thu Mar 09, 2006 11:58 amLocation: Antalya/Turkey
Post by Sedat Canbaz Wed Aug 29, 2012 9:34 pm
Sedat Canbaz wrote:
EDIT:
-There was
4 Elo difference between Houdini calculations:
1st.Calculation:Rybka 4.1 NO-SSE x64 6c + 1000 games player Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3363 12 12 1700 70% 3211 39% 3rd.Calculation:Rybka 4.1 NO-SSE x64 6c + 1500 games player
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3359 14 14 1700 70% 3217 39%
Best,
Sedat
Laskos
Posts: 10948Joined: Wed Jul 26, 2006 10:21 pmFull name: Kai Laskos
Post by Laskos Wed Aug 29, 2012 9:37 pm
Sedat Canbaz wrote: EDIT:
-There was
4 Elo difference between Houdini calculations:
1st.Calculation:Rybka 4.1 NO-SSE x64 6c + 1000 games player Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3363 12 12 1700 70% 3211 39% 3rd.Calculation:Rybka 4.1 NO-SSE x64 6c + 1500 games player
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3359 14 14 1700 70% 3217 39%
Best,
Sedat
I thought you played only those 50 games between Fruit and Rybka, but it turns out you played 500 games with Rybka against various opposition. In this case, I cannot be sure of what happens, but it seems that Bayeselo does show some strange behaviour.
Kai
Daniel Shawul
Posts: 4186Joined: Tue Mar 14, 2006 11:34 amLocation: Ethiopia
Post by Daniel Shawul Wed Aug 29, 2012 9:46 pm
This is a gigantic waste of time. I redid his calculation with his data and I get exactly 1 elo difference between fruit and rybka
First 29250 games
Code: Select all
version 0056, Copyright (C) 1997-2007 Remi Coulom.
compiled Jan 30 2007 20:30:07.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under the terms and conditions of the GNU General Public License.
See http://www.gnu.org/copyleft/gpl.html for details.
ResultSet>read scct1.pgn
Unknown command: read
type '?' for help
ResultSet>readpgn scct1.pgn
29250 game(s) loaded, 0 game(s) with unknown result ignored.
ResultSet>elo
ResultSet-EloRating>mm 1 1
00:00:00,01
ResultSet-EloRating>ratings
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 151 12 12 1700 70% 0 39%
2 Houdini 2.0t3* Pro x64 6c 150 15 15 1000 75% -34 37%
3 Houdini 2.0z Pro x64 6c 147 12 12 1550 71% -19 36%
4 Houdini 2.0s2 Pro x64 6c 145 16 16 1000 74% -41 34%
5 Houdini 1.5a x64 6c 133 14 14 1100 68% 1 41%
6 Houdini 2.0Bar2 x64 6c 132 15 15 1000 73% -34 43%
7 Houdini 2.0c Pro x64 6c 131 13 13 1450 71% -29 39%
8 Houdini 2.0Higgs Pro x64 6c 128 15 15 1000 71% -25 42%
9 Houdini2Bar1 Pro x64 6c 118 14 14 1100 69% -19 46%
10 Critter 1.6 x64 6c 89 10 10 1900 63% -2 53%
11 Critter 1.4 x64 6c 77 14 14 1150 67% -47 47%
12 Rybka 4.1 79DT v1 x64 6c 76 14 14 1100 66% -44 38%
13 Stockfish 120430P x64 6c 71 11 11 1850 60% -3 50%
14 Deep Rybka 4.1 x64 6c 63 11 11 1750 60% -7 48%
15 Stockfish 2.2.2 JA x64 6c 62 13 13 1200 62% -27 47%
16 Ivanhoe B46fE.02 x64 6c 62 10 10 1900 59% -2 53%
17 Rybka 4.1 NO-SSE x64 6c 62 14 14 1000 63% -31 49%
18 Ivanhoe B46fC x64 6c 61 13 13 1200 64% -39 47%
19 Stockfish VE09 x64 6c 52 14 14 1000 63% -42 48%
20 Fire 2.2 xTreme x64 6c 48 10 10 1900 57% -1 52%
21 Vitruvius 1.11C x64 6c 46 10 10 1900 56% -1 51%
22 Gull II beta2 x64 6c -3 12 12 1400 50% -7 51%
23 Strelka 5.5 x64 1c -22 11 11 1650 45% 12 48%
24 Bouquet 1.4 x64 6c -35 13 13 1250 46% -11 47%
25 Naum 4.2 x64 6c -43 10 10 1900 44% 1 44%
26 Komodo 4.0 x64 1c -63 11 11 1900 41% 2 42%
27 Equinox 1.35 x64 6c -95 12 12 1550 40% -25 40%
28 Deep Fritz 13 w32 6c -95 11 11 1900 36% 2 43%
29 Spike 1.4 Leiden w32 6c -114 11 11 1900 34% 3 38%
30 Chiron 1.1a x64 6c -117 11 11 1900 34% 3 39%
31 Deep Fritz 12 w32 6c -132 14 14 1150 37% -38 42%
32 Deep Junior 13.3 x64 6c -134 12 12 1700 31% 11 36%
33 Protector 1.4.0 x64 6c -138 11 11 1900 31% 4 36%
34 Spark 1.0 x64 6c -141 11 11 1850 31% 0 39%
35 Deep Junior 13 x64 6c -144 13 13 1300 35% -30 36%
36 Deep Shredder 12 x64 6c -145 11 11 1900 30% 4 37%
37 Hiarcs 13.2 w32 6c -161 11 11 1900 29% 4 32%
38 Zappa Mexico II x64 6c -176 13 13 1550 29% -14 34%
39 Fruit 090705 x64 6c -246 15 15 1150 23% -33 29%
ResultSet-EloRating>
Then 30408 games
Code: Select all
version 0056, Copyright (C) 1997-2007 Remi Coulom.
compiled Jan 30 2007 20:30:07.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under the terms and conditions of the GNU General Public License.
See http://www.gnu.org/copyleft/gpl.html for details.
ResultSet>readpgn scct2.pgn
30408 game(s) loaded, 0 game(s) with unknown result ignored.
ResultSet>elo
ResultSet-EloRating>mm 1 1
00:00:00,01
ResultSet-EloRating>ratings
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 153 12 12 1735 70% 0 39%
2 Houdini 2.0t3* Pro x64 6c 152 15 15 1000 75% -33 37%
3 Houdini 2.0z Pro x64 6c 147 12 12 1600 71% -14 36%
4 Houdini 2.0s2 Pro x64 6c 146 16 16 1000 74% -39 34%
5 Houdini 2.0Bar2 x64 6c 135 15 15 1050 72% -28 44%
6 Houdini 1.5a x64 6c 135 14 14 1100 68% 2 41%
7 Houdini 2.0c Pro x64 6c 131 12 12 1500 71% -24 39%
8 Houdini 2.0Higgs Pro x64 6c 131 15 15 1050 70% -19 42%
9 Houdini2Bar1 Pro x64 6c 120 14 14 1100 69% -17 46%
10 Critter 1.6 x64 6c 91 10 10 1935 63% -2 53%
11 Critter 1.4 x64 6c 79 13 13 1200 66% -41 47%
12 Rybka 4.1 79DT v1 x64 6c 79 14 14 1134 66% -43 38%
13 Stockfish 120430P x64 6c 71 11 11 1884 60% -3 50%
14 Rybka 4.1 SSE42 x64 6c 65 11 11 1800 59% -4 49%
15 Ivanhoe B46fE.02 x64 6c 64 10 10 1935 59% -1 52%
16 Stockfish 2.2.2 JA x64 6c 64 13 13 1200 62% -25 47%
17 Ivanhoe B46fC x64 6c 63 13 13 1250 63% -33 48%
18 Rybka 4.1 NO-SSE x64 6c 63 12 12 1500 60% -13 49%
19 Stockfish VE09 x64 6c 53 14 14 1000 63% -40 48%
20 Fire 2.2 xTreme x64 6c 51 10 10 1935 57% -1 52%
21 Vitruvius 1.11C x64 6c 48 10 10 1934 57% -1 51%
22 Gull II beta2 x64 6c -1 12 12 1435 51% -7 51%
23 Strelka 5.5 x64 1c -21 11 11 1684 45% 12 48%
24 Bouquet 1.4 x64 6c -34 13 13 1285 46% -11 47%
25 Naum 4.2 x64 6c -42 10 10 1935 44% 2 44%
26 Komodo 4.0 x64 1c -61 11 11 1935 41% 2 42%
27 Deep Hiarcs 14 WCSC w32 6c -64 18 18 658 45% -25 44%
28 Equinox 1.35 x64 6c -93 12 12 1550 40% -23 40%
29 Deep Fritz 13 w32 6c -93 11 11 1935 37% 3 44%
30 Spike 1.4 Leiden w32 6c -113 11 11 1934 34% 3 38%
31 Chiron 1.1a x64 6c -115 11 11 1935 34% 3 39%
32 Deep Fritz 12 w32 6c -131 14 14 1200 36% -33 42%
33 Deep Junior 13.3 x64 6c -132 12 12 1735 31% 11 36%
34 Protector 1.4.0 x64 6c -137 11 11 1934 31% 4 37%
35 Spark 1.0 x64 6c -140 11 11 1884 31% 1 39%
36 Deep Junior 13 x64 6c -142 13 13 1300 35% -29 36%
37 Deep Shredder 12 x64 6c -143 11 11 1935 30% 4 37%
38 Hiarcs 13.2 w32 6c -159 11 11 1900 29% 6 32%
39 Zappa Mexico II x64 6c -172 12 12 1600 29% -10 34%
40 Fruit 090705 x64 6c -246 15 15 1200 23% -28 29%
ResultSet-EloRating>scale
0.692166
ResultSet-EloRating>
Difference b/n Rybka 4.1 NO-SSE x64 6c and Fruit 090705 x64 6c
Diff1 = 62 - (-246) = 308
Diff 2 = 63 - (-246) = 309
Increment = 309 - 308 = 1 elo
I will do elostat calculation later. Maybe I will even use what is embedded in bayeselo.
Last edited by Daniel Shawul on Wed Aug 29, 2012 9:49 pm, edited 1 time in total.
Sedat Canbaz
Posts: 3018Joined: Thu Mar 09, 2006 11:58 amLocation: Antalya/Turkey
Post by Sedat Canbaz Wed Aug 29, 2012 9:47 pm
Laskos wrote: Sedat Canbaz wrote: EDIT:
-There was
4 Elo difference between Houdini calculations:
1st.Calculation:Rybka 4.1 NO-SSE x64 6c + 1000 games player Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3363 12 12 1700 70% 3211 39% 3rd.Calculation:Rybka 4.1 NO-SSE x64 6c + 1500 games player
Code: Select all
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3 Pro x64 6c 3359 14 14 1700 70% 3217 39%
Best,
Sedat
I thought you played only those 50 games between Fruit and Rybka, but it turns out you played 500 games with Rybka against various opposition. In this case, I cannot be sure of what happens, but it seems that Bayeselo does show some strange behaviour.
Kai
Strange indeed...i have also no any idea about what is going on with BayesElo
For example,
Rybka 4.1 NO-SSE is played 500 games more,where Fruit is played only 50 games
In other words,(about adding the latest Fruit's 50 games):
-We can't say: it's a true/accurate measuring by BayesElo !
And the most important:
-How can we trust to BayesElo 0056 in the next calculations ?
Best,
Sedat
Daniel Shawul
Posts: 4186Joined: Tue Mar 14, 2006 11:34 amLocation: Ethiopia
Post by Daniel Shawul Wed Aug 29, 2012 9:58 pm
And here is elostat's output i.e using tool inside bayeselo, guess what difference I got? Yes it is a 1 elo increment which is exactly sameas bayeselo's. Enough said...
Before
Code: Select all
version 0056, Copyright (C) 1997-2007 Remi Coulom.
compiled Jan 30 2007 20:30:07.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under the terms and conditions of the GNU General Public License.
See http://www.gnu.org/copyleft/gpl.html for details.
ResultSet>readpgn scct1.pgn
29250 game(s) loaded, 0 game(s) with unknown result ignored.
ResultSet>elostat
Unknown command: elostat
type '?' for help
ResultSet>elo
ResultSet-EloRating>elostat
16 iterations
00:00:00,00
ResultSet-EloRating>ratings
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3* Pro x64 6c 164 18 17 1000 75% -24 37%
2 Houdini 2.0t3 Pro x64 6c 158 13 13 1700 70% 11 39%
3 Houdini 2.0s2 Pro x64 6c 154 19 18 1000 74% -30 34%
4 Houdini 2.0z Pro x64 6c 151 15 14 1550 71% -8 36%
5 Houdini 2.0Bar2 x64 6c 149 17 16 1000 73% -23 43%
6 Houdini 2.0Higgs Pro x64 6c 140 17 16 1000 71% -14 42%
7 Houdini 2.0c Pro x64 6c 138 15 14 1450 71% -18 39%
8 Houdini 1.5a x64 6c 138 16 16 1100 68% 11 41%
9 Houdini2Bar1 Pro x64 6c 128 15 15 1100 69% -8 46%
10 Critter 1.6 x64 6c 98 11 11 1900 63% 9 53%
11 Critter 1.4 x64 6c 86 15 14 1150 67% -36 47%
12 Rybka 4.1 79DT v1 x64 6c 82 17 16 1100 66% -33 38%
13 Stockfish 120430P x64 6c 79 11 11 1850 60% 8 50%
14 Rybka 4.1 NO-SSE x64 6c 72 16 15 1000 63% -20 49%
15 Stockfish 2.2.2 JA x64 6c 72 15 14 1200 62% -16 47%
16 Deep Rybka 4.1 x64 6c 72 12 12 1750 60% 4 48%
17 Ivanhoe B46fE.02 x64 6c 71 11 11 1900 59% 9 53%
18 Ivanhoe B46fC x64 6c 69 14 14 1200 64% -28 47%
19 Stockfish VE09 x64 6c 65 16 15 1000 63% -31 48%
20 Fire 2.2 xTreme x64 6c 56 11 11 1900 57% 10 52%
21 Vitruvius 1.11C x64 6c 55 11 11 1900 56% 10 51%
22 Gull II beta2 x64 6c 7 13 13 1400 50% 4 51%
23 Strelka 5.5 x64 1c -13 12 12 1650 45% 23 48%
24 Bouquet 1.4 x64 6c -25 14 14 1250 46% 0 47%
25 Naum 4.2 x64 6c -32 12 12 1900 44% 12 44%
26 Komodo 4.0 x64 1c -52 12 12 1900 41% 12 42%
27 Equinox 1.35 x64 6c -82 13 14 1550 40% -14 40%
28 Deep Fritz 13 w32 6c -83 12 12 1900 36% 13 43%
29 Spike 1.4 Leiden w32 6c -102 12 13 1900 34% 14 38%
30 Chiron 1.1a x64 6c -104 12 13 1900 34% 14 39%
31 Deep Fritz 12 w32 6c -119 15 16 1150 37% -27 42%
32 Deep Junior 13.3 x64 6c -120 13 14 1700 31% 22 36%
33 Protector 1.4.0 x64 6c -126 13 13 1900 31% 14 36%
34 Deep Junior 13 x64 6c -127 15 16 1300 35% -20 36%
35 Spark 1.0 x64 6c -128 12 13 1850 31% 11 39%
36 Deep Shredder 12 x64 6c -132 13 13 1900 30% 15 37%
37 Hiarcs 13.2 w32 6c -144 13 14 1900 29% 15 32%
38 Zappa Mexico II x64 6c -161 14 15 1550 29% -4 34%
39 Fruit 090705 x64 6c -231 18 19 1150 23% -23 29%
ResultSet-EloRating>
After
Code: Select all
version 0056, Copyright (C) 1997-2007 Remi Coulom.
compiled Jan 30 2007 20:30:07.
This program comes with ABSOLUTELY NO WARRANTY.
This is free software, and you are welcome to redistribute it
under the terms and conditions of the GNU General Public License.
See http://www.gnu.org/copyleft/gpl.html for details.
ResultSet>read scct2.pgn
Unknown command: read
type '?' for help
ResultSet>readpgn scct2.pgn
30408 game(s) loaded, 0 game(s) with unknown result ignored.
ResultSet>elo
ResultSet-EloRating>elostat
16 iterations
00:00:00,00
ResultSet-EloRating>ratings
Rank Name Elo + - games score oppo. draws
1 Houdini 2.0t3* Pro x64 6c 164 18 17 1000 75% -24 37%
2 Houdini 2.0t3 Pro x64 6c 159 13 13 1735 70% 9 39%
3 Houdini 2.0s2 Pro x64 6c 154 19 18 1000 74% -30 34%
4 Houdini 2.0z Pro x64 6c 150 14 14 1600 71% -5 36%
5 Houdini 2.0Bar2 x64 6c 150 16 15 1050 72% -19 44%
6 Houdini 2.0Higgs Pro x64 6c 140 17 16 1050 70% -10 42%
7 Houdini 1.5a x64 6c 138 16 16 1100 68% 11 41%
8 Houdini 2.0c Pro x64 6c 137 14 14 1500 71% -15 39%
9 Houdini2Bar1 Pro x64 6c 128 15 15 1100 69% -8 46%
10 Critter 1.6 x64 6c 99 11 10 1935 63% 7 53%
11 Critter 1.4 x64 6c 86 14 14 1200 66% -32 47%
12 Rybka 4.1 79DT v1 x64 6c 84 16 16 1134 66% -33 38%
13 Stockfish 120430P x64 6c 78 11 11 1884 60% 6 50%
14 Stockfish 2.2.2 JA x64 6c 72 15 14 1200 62% -16 47%
15 Rybka 4.1 SSE42 x64 6c 72 12 11 1800 59% 5 49%
16 Ivanhoe B46fE.02 x64 6c 71 11 11 1935 59% 8 52%
17 Rybka 4.1 NO-SSE x64 6c 70 13 13 1500 60% -4 49%
18 Ivanhoe B46fC x64 6c 69 14 14 1250 63% -24 48%
19 Stockfish VE09 x64 6c 65 16 15 1000 63% -31 48%
20 Fire 2.2 xTreme x64 6c 56 11 11 1935 57% 8 52%
21 Vitruvius 1.11C x64 6c 55 11 11 1934 57% 8 51%
22 Gull II beta2 x64 6c 8 13 13 1435 51% 3 51%
23 Strelka 5.5 x64 1c -13 12 12 1684 45% 21 48%
24 Bouquet 1.4 x64 6c -25 14 14 1285 46% -1 47%
25 Naum 4.2 x64 6c -32 12 12 1935 44% 11 44%
26 Komodo 4.0 x64 1c -52 12 12 1935 41% 11 42%
27 Deep Hiarcs 14 WCSC w32 6c -53 20 20 658 45% -16 44%
28 Equinox 1.35 x64 6c -82 13 14 1550 40% -14 40%
29 Deep Fritz 13 w32 6c -83 12 12 1935 37% 12 44%
30 Spike 1.4 Leiden w32 6c -102 12 13 1934 34% 13 38%
31 Chiron 1.1a x64 6c -104 12 12 1935 34% 13 39%
32 Deep Junior 13.3 x64 6c -120 13 14 1735 31% 20 36%
33 Deep Fritz 12 w32 6c -121 15 15 1200 36% -24 42%
34 Protector 1.4.0 x64 6c -127 13 13 1934 31% 13 37%
35 Deep Junior 13 x64 6c -127 15 16 1300 35% -20 36%
36 Spark 1.0 x64 6c -129 12 13 1884 31% 10 39%
37 Deep Shredder 12 x64 6c -132 12 13 1935 30% 13 37%
38 Hiarcs 13.2 w32 6c -144 13 14 1900 29% 15 32%
39 Zappa Mexico II x64 6c -160 14 15 1600 29% -2 34%
40 Fruit 090705 x64 6c -234 18 19 1200 23% -19 29%
ResultSet-EloRating>
Difference:
Diff1 = 72 - (-231) = 303
Diff2 = 70 - (-234) = 304
Increment = 1 elo!
Bye
Daniel
Laskos
Posts: 10948Joined: Wed Jul 26, 2006 10:21 pmFull name: Kai Laskos
Post by Laskos Wed Aug 29, 2012 10:04 pm
Daniel Shawul wrote: This is a gigantic waste of time. I redid his calculation with his data and I get exactly 1 elo difference between fruit and rybka
Good, something of order 1-2-3 Elos increment in difference is what was expected.
Daniel Shawul
Posts: 4186Joined: Tue Mar 14, 2006 11:34 amLocation: Ethiopia
Post by Daniel Shawul Wed Aug 29, 2012 10:20 pm
Laskos wrote: Daniel Shawul wrote: This is a gigantic waste of time. I redid his calculation with his data and I get exactly 1 elo difference between fruit and rybka
Good, something of order 1-2-3 Elos increment in difference is what was expected.
Why exactly ? You even said it should decrease ,which it didn't. I see so many ridiculous claims it is not funny anymore... Like I said so many times it is not a popularity contest.
