VERY INTERESTING Stockfish vs. Houdini Match Has Begun!

[geots]

Hello again:

I followed that, but your extrapolation of George's result to 100 and 1,000 games is incorrect. If LOS of SF is 87%, then the probability of winning matches of 100 or 1,000 games are less than 87%. You are assuming a fixed gap of 37 or whatever points, and extrapolate this to number of games, which is correct procedure for fixed gap. But it's incorrect to extrapolate from George's match the probabilities for 100 or 1,000 games matches such way.

Please note that I am not extrapolating George's results. I did the calculations under Don's assumptions: Houdini 3 being 37 Elo stronger than current SF development version at LTC (personally I think that the gap is bigger) and 60% to 70% of expected draw ratio. Basically he said that it was not infrequent to see people reporting SF wins in short matches (even in 100-game matches). I quantified those possibilities between 2% and 6% (not being too estrict). Don said that he was sure that SF will lose a 1000-game match against Houdini with the same TC. Again I quantified this possibility with near 100% of chances in favour of Houdini.

Summarizing, I take a fixed Elo gap as input data, not George's results or LOS of his match. In other words, George's results fall in the 2% to 6% I mentioned above, which is not so rare or infrequent. The strange, outstanding fact will be seeing current SF win a 1000-game match against Houdini 3. If match conditions are fair (equal number of cores, same hash table, etc.) then this result is an outlier undoubtedly and VERY uncommon.

------------

@George: thank you very much for the effort. Sorry for hijacking your thread with the realm of probabilities.

Regards from Spain.

Ajedrecista.

Ok, got it Jesus. You assume SF is 37 points weaker, and extrapolate that to 100 and 1,000 games. Extrapolate better to 50 games, George is playing that amount. Should be in in some ~10% margins that SF wins, so it's indeed not very surprising that it happens.

And Kai, I certainly always appreciate your feedback as well. Thank you. You and Jesus both certainly know your business- and I respect both of you.


Best to you,

gts

[Ajedrecista]

Hello:

Ok, got it Jesus. You assume SF is 37 points weaker, and extrapolate that to 100 and 1,000 games. Extrapolate better to 50 games, George is playing that amount. Should be in in some ~10% margins that SF wins, so it's indeed not very surprising that it happens.

Ups! I wrongly thought that George's match consisted in 100 games instead of 50. Repeating the calculations with the same supposed Elo gap of other posts (expected draw ratios of 50%, 60% and 70%):

Code: Select all

Probabilities for a match of    50 games (rounded up to 0.0001 %):
 
Rating difference (rounded up to 0.01 Elo):   37.00 Elo.
 
Probability of a win  = W ~ 30.3047 %
Probability of a draw = D ~ 50.0000 %
Probability of a lose = L ~ 19.6953 %

[...]

                           SUMMARY:
 
 Probability that the first player wins the match ~  83.4515 %
                      Probability of a tied match ~   4.5159 %
Probability that the second player wins the match ~  12.0327 %
 
--------------------------------------------------------------
 
 Prob.(first player wins) + 0.5*Prob.(tied match) ~  85.7094 %
Prob.(second player wins) + 0.5*Prob.(tied match) ~  14.2906 %

Code: Select all

Probabilities for a match of    50 games (rounded up to 0.0001 %):
 
Rating difference (rounded up to 0.01 Elo):   37.00 Elo.
 
Probability of a win  = W ~ 25.3047 %
Probability of a draw = D ~ 60.0000 %
Probability of a lose = L ~ 14.6953 %

[...]

                           SUMMARY:
 
 Probability that the first player wins the match ~  86.2371 %
                      Probability of a tied match ~   4.3789 %
Probability that the second player wins the match ~   9.3840 %
 
--------------------------------------------------------------
 
 Prob.(first player wins) + 0.5*Prob.(tied match) ~  88.4266 %
Prob.(second player wins) + 0.5*Prob.(tied match) ~  11.5734 %

Code: Select all

Probabilities for a match of    50 games (rounded up to 0.0001 %):
 
Rating difference (rounded up to 0.01 Elo):   37.00 Elo.
 
Probability of a win  = W ~ 20.3047 %
Probability of a draw = D ~ 70.0000 %
Probability of a lose = L ~  9.6953 %

[...]

                           SUMMARY:
 
 Probability that the first player wins the match ~  89.7965 %
                      Probability of a tied match ~   3.9683 %
Probability that the second player wins the match ~   6.2352 %
 
--------------------------------------------------------------
 
 Prob.(first player wins) + 0.5*Prob.(tied match) ~  91.7806 %
Prob.(second player wins) + 0.5*Prob.(tied match) ~   8.2194 %

The possible outcomes vary a lot depending on the expected draw ratio, but in this case (and under those assumptions) I think it is safe to say that SF had between 6% and 12% of chances of win the 50-game match, and around 4% to 5% of chances of tie it. Of course I am not taking a zillion of decimals.

Thanks for your comprehension, George: it is always appreciated.

Regards from Spain.

Ajedrecista.

[geots]

Well, maybe it was a bit longer than momentarily. But I am here with the final results. There is nothing I can say that I haven't already. I consider it a great match between two of the best out there. At my testing facilities, Stockfish 290613 is now the World Champion. (Which must be on a minimum of 4 Cores and a time control no faster than 40/40 repeating. If longer FIDE controls are used, the number of games will be lowered.) Trust me- if "290613" is not worthy of this position, he will not be here long. I am hoping that by the first of the week there will be a match involving Komodo MP, for the right to challenge 290613 for the top spot.

The only caveat is that in the next few days I plan on ordering another Alienware AURORA R4 Intel i7 6-Core system EXACTLY like this one, which will give me 2, 4-Core i5s networked together as well as then 2 Alienware 6-Core i7s as well. At that time, I might consider moving the 40/40 controls to 100 game matches, and the longer controls to 50 games.

(And for sure- congratulations to Marco and the Stockfish team!)





Alienware AURORA_R4
Intel i7 w/6 True Cores
Fritz 11 gui
6 Cores each/64bit
256MB hash
Bases=NONE
Ponder_Learning=OFF
Perfect 2012b.ctg w/12-move limit
40/19 Repeating- benchmarked to adapt to 40/40
Match=50 games

Code: Select all

SP8-i7, 19'/40+19'/40+19'/40  7/7/2013  

                                
Stockfish 290613 64 SSE4.2   +28    +12/-8/=30   54.00%   27.0/50
Houdini 3 x64                -28    +8/-12/=30   46.00%   23.0/50 

Enjoy-

[sekos]

Where can we find games?
Did U put zp file with PGN collection ?

Thnx for tournament.

[geots]

Where can we find games?
Did U put zp file with PGN collection ?

Thnx for tournament.

I appreciate your interest in the tournament. I will be glad to upload them and give you a link- and anyone else interested. If you would please give me a couple hours- as I am going to have to lie down and rest my back for a bit. Damn thing has been giving me more trouble than usual lately. But it will be here- that I promise.


All the best-

george

[sekos]

Thank you George for your kind words.
Your tournament is great . I will follow it further.

[geots]

Thank you George for your kind words.
Your tournament is great . I will follow it further.

Pawel,here is the link for the complete set of PGNs- Stockfish v Houdini 3. Please note that each time I took a break and restarted, in the PGNs it started back at Round 1. Where it stayed 1 thru 50 in the database. But all 50 games are in the PGN set for download. Hope you enjoy and thanks again for your interest.

The following link will remain here for 7 days:


https://dl.dropboxusercontent.com/u/115 ... %20x64.rar




All the best-