Houdini 2.0c x64 Pro - Komodo64 4 SSE Match Results

[ATOMICC]

http://atomicc-testing.blogspot.com/201 ... 4-sse.html

Also, the new ratings (for what they're worth right now) are out. Click on the ratings page.

[Adam Hair]

http://atomicc-testing.blogspot.com/201 ... 4-sse.html

Also, the new ratings (for what they're worth right now) are out. Click on the ratings page.

Great!

I have one suggestion if you are not already doing it. When running Bayeselo, use "mm 0 1" instead of "mm". That way white advantage is removed (playing from both sides should remove that advantage) and the drawelo value will be computed from your results rather than using the default value. The percentage of draws for your testing is much higher than WBEC results, which is the database that was used to compute the default.

By the way, I like that you did not use an offset for your ratings. That way the focus is on the ratings difference, not an arbitrary Elo value.

Thanks,

Adam

[ATOMICC]

...When running Bayeselo, use "mm 0 1" instead of "mm". That way white advantage is removed (playing from both sides should remove that advantage) and the drawelo value will be computed from your results rather than using the default value. The percentage of draws for your testing is much higher than WBEC results, which is the database that was used to compute the default.

Thanks! There is not much documentation with the Bayeselo program, so I spent some time making sure I had things right. I originally did not think I had to adjust the advantage of playing White since each player is playing both sides of the same opening, but I think your approach is more correct. I will recalculate.

By the way, I like that you did not use an offset for your ratings. That way the focus is on the ratings difference, not an arbitrary Elo value.

Yes, I feel this is more to the point, and who knows which engine has which Elo value at this speed (of hardware) and time control? Glad you like my 0 offset value.

[ATOMICC]

All right, Adam. I re-ran the ratings with "mm 0 1" instead of "mm", and afterwards Komodo dropped 4 Elo, Critter dropped 6, and Stockfish dropped 9. The +/- also evened out. Nice.

[michiguel]

All right, Adam. I re-ran the ratings with "mm 0 1" instead of "mm", and afterwards Komodo dropped 4 Elo, Critter dropped 6, and Stockfish dropped 9. The +/- also evened out. Nice.

Just for fun, ratings with Ordo.

Code: Select all

                        
                        ENGINE:  RATING  ERROR  POINTS  PLAYED    (%)
     Houdini 2.0c Pro x64 1CPU:  2331.7   19.7   110.0     200   55.0%
   Komodo64 SSE Version 4 1CPU:  2303.1   19.3    99.0     200   49.5%
 Critter 1.4a 64-bit SSE4 1CPU:  2294.3   19.7   104.0     200   52.0%
 Stockfish 2.2.2 JA SSE42 1CPU:  2270.9   19.7    87.0     200   43.5%

Above is the general error (compared to the average of the pool).
More interesting is the the error matrix, head to head (1000 ratings simulated). For instance, the error Houdini vs SF is 35.5

Code: Select all

"N","NAME"                        ,   0,     1,     2,    3
0,"Houdini 2.0c Pro x64 1CPU"      ----
1,"Komodo64 SSE Version 4 1CPU"   ,29.7   ----
2,"Critter 1.4a 64-bit SSE4 1CPU" ,30.8,  35.1   ----
3,"Stockfish 2.2.2 JA SSE42 1CPU" ,35.5,  30.3,  30.2  ----

Miguel

[ATOMICC]

Hey, Miguel. Interesting. Thanks for sharing.

I will be testing Gaviota 6CPU next week for the CCRL, and am looking forward to it. Have a good day.

[Adam Hair]

Brent would be better served by not listening to me. The values 0 and 1 in the command "mm 0 1" appears to tell Bayeselo to use the default value for advantage and use the computed value for drawelo. Since he is using reversed colors, advantage should be set to 0. So, the proper commands should be (I think. I have not had a good run lately on being right.):

readpgn
elo
advantage 0
mm 0 1
exactdist

[ATOMICC]

Brent would be better served by not listening to me. The values 0 and 1 in the command "mm 0 1" appears to tell Bayeselo to use the default value for advantage and use the computed value for drawelo. Since he is using reversed colors, advantage should be set to 0. So, the proper commands should be (I think. I have not had a good run lately on being right.):

readpgn
elo
advantage 0
mm 0 1
exactdist

I agree with this method, Adam. Here are the results after re-running the Elo script with advantage 0: http://atomicc-testing.blogspot.com/p/rating-list.html