Sunday Update- 40x(2) v Houdini 2.0c!

[geots]

Engine 40x(2) v Houdini 2.0c x64

(Update after 404 games)


Intel i5 w/4TCs
Fritz 11 gui
1CPU/64bit
128MB hash
Bases=NONE
Ponder_Learning=OFF
Perfect 12.32 book w/12-move limit
10'+10"
Match=1000 games

Code: Select all

1   Houdini 2.0c x64    +121/-96/=187   53.00%   214.5/404
2   Engine 40x(2)       +96/-121/=187   47.00%   189.5/404

I guess a lot could happen when you have 596 games remaining. What will be will be. See you soon.

george

[ernest]

Code: Select all

1   Houdini 2.0c x64    +121/-96/=187   53.00%   214.5/404
2   Engine 40x(2)       +96/-121/=187   47.00%   189.5/404

I guess a lot could happen when you have 596 games remaining.

Here you can see the first effect of statistics.

All your previous runs gave a Houdini result of around 56%.
Now, since on Friday you had:

Code: Select all

1   Houdini 2.0c x64    +81/-54/=118   56.00%   140.0/253
2   Engine 40x(2)       +54/-81/=118   44.00%   113.0/253

the latest run of 251 games (Friday to Sunday) is, by difference:

Code: Select all

1   Houdini 2.0c x64    +40/-42/=169   49.60%   124.5/251

49.6% ! Nothing is wrong, just statistics!

[Ajedrecista]

Hello!

Code: Select all

1   Houdini 2.0c x64    +121/-96/=187   53.00%   214.5/404
2   Engine 40x(2)       +96/-121/=187   47.00%   189.5/404

I guess a lot could happen when you have 596 games remaining.

Here you can see the first effect of statistics.

All your previous runs gave a Houdini result of around 56%.
Now, since on Friday you had:

Code: Select all

1   Houdini 2.0c x64    +81/-54/=118   56.00%   140.0/253
2   Engine 40x(2)       +54/-81/=118   44.00%   113.0/253

the latest run of 251 games (Friday to Sunday) is, by difference:

Code: Select all

1   Houdini 2.0c x64    +40/-42/=169   49.60%   124.5/251

49.6% ! Nothing is wrong, just statistics!

I agree with Ernest regarding statistics and I also agree with George, regarding that everything can happen in the remaining 596 games. By the way, I am very interested in this match.

I refined a little my tiny programme Minimum_score_for_no_regression and the ugly parameter k is no longer required but internally calculated. Since Fortran 95 does not have erf function (or at least I am not aware of it), I had to approximate the definite integral of the probability density function of the normal distribution by the composite Simpson's rule, and then solve k by regula falsi... a difficult trick just due to the lack of erf function in Fortran 95! At least this trick works fine. I also added the calculated standard deviation to the output for getting more info.

Taking the example of George's match:

Code: Select all

Minimum_score_for_no_regression, ® 2012.

Calculation of the minimum score for no regression in a match between two engines:

 Write down the number of games of the match (it must be a positive integer, up to 1073741823):

404

Write down the draw ratio (in percentage):

46.287128712871

Write down the confidence level (in percentage) between 75% and 99.9%:

95

Calculating...

Theoretical minimum score for no regression: 53.5564 %
Theoretical standard deviation in this case:  1.8145 %

Minimum number of won points for the engine in this match:       216.5 points.

Minimum Elo advantage, which is also the negative part of the error bar:
 24.9827 Elo

End of the calculations.

Thanks for using Minimum_score_for_no_regression. Press Enter to exit.

With those 404 games, Houdini can not claim (with 95% confidence) that is better than Engine 40x(2) using my imperfect model. Although it is better very likely because Houdini score is very close to 216.5 out of 404. LOS tables are very useful here and, of course, I am not smart enough to even try to calculate them!

A question for George: are you generating those code boxes with scores using Fritz 11 GUI? I see that scores are like 56.00% - 44.00% or 53.00% - 47.00%, but those .00 are not correct (i.e. in the last update it should be more less 53.09% - 46.91%), and make me think that Fritz 11 GUI (if you are using it for generate those code boxes) does not round up to 0.01% but up to 1%... just a guess. Please keep up the good work.

Regards from Spain.

Ajedrecista.

[geots]

Hello!

Code: Select all

1   Houdini 2.0c x64    +121/-96/=187   53.00%   214.5/404
2   Engine 40x(2)       +96/-121/=187   47.00%   189.5/404

I guess a lot could happen when you have 596 games remaining.

Here you can see the first effect of statistics.

All your previous runs gave a Houdini result of around 56%.
Now, since on Friday you had:

Code: Select all

1   Houdini 2.0c x64    +81/-54/=118   56.00%   140.0/253
2   Engine 40x(2)       +54/-81/=118   44.00%   113.0/253

the latest run of 251 games (Friday to Sunday) is, by difference:

Code: Select all

1   Houdini 2.0c x64    +40/-42/=169   49.60%   124.5/251

49.6% ! Nothing is wrong, just statistics!

I agree with Ernest regarding statistics and I also agree with George, regarding that everything can happen in the remaining 596 games. By the way, I am very interested in this match.

I refined a little my tiny programme Minimum_score_for_no_regression and the ugly parameter k is no longer required but internally calculated. Since Fortran 95 does not have erf function (or at least I am not aware of it), I had to approximate the definite integral of the probability density function of the normal distribution by the composite Simpson's rule, and then solve k by regula falsi... a difficult trick just due to the lack of erf function in Fortran 95! At least this trick works fine. I also added the calculated standard deviation to the output for getting more info.

Taking the example of George's match:

Code: Select all

Minimum_score_for_no_regression, ® 2012.

Calculation of the minimum score for no regression in a match between two engines:

 Write down the number of games of the match (it must be a positive integer, up to 1073741823):

404

Write down the draw ratio (in percentage):

46.287128712871

Write down the confidence level (in percentage) between 75% and 99.9%:

95

Calculating...

Theoretical minimum score for no regression: 53.5564 %
Theoretical standard deviation in this case:  1.8145 %

Minimum number of won points for the engine in this match:       216.5 points.

Minimum Elo advantage, which is also the negative part of the error bar:
 24.9827 Elo

End of the calculations.

Thanks for using Minimum_score_for_no_regression. Press Enter to exit.

With those 404 games, Houdini can not claim (with 95% confidence) that is better than Engine 40x(2) using my imperfect model. Although it is better very likely because Houdini score is very close to 216.5 out of 404. LOS tables are very useful here and, of course, I am not smart enough to even try to calculate them!

A question for George: are you generating those code boxes with scores using Fritz 11 GUI? I see that scores are like 56.00% - 44.00% or 53.00% - 47.00%, but those .00 are not correct (i.e. in the last update it should be more less 53.09% - 46.91%), and make me think that Fritz 11 GUI (if you are using it for generate those code boxes) does not round up to 0.01% but up to 1%... just a guess. Please keep up the good work.

Regards from Spain.

Ajedrecista.

Good day Jesus. Thank you very much for your interest. I am computing the percentages myself, as I am running this match in 3 Fritz 11 guis at once. And I combine the 3 to get one crosstable- and the % is the only thing I did not worry too much about rounding off to perfection. In general terms- for my purposes it is very close, I would think.

I apologize to you for not putting the 2 (what I call markings) above your name in 2 places. Not quite sure how to do it- but if it bothers you I will make sure they get there somehow. I certainly would never want to offend you like that.


Best

george

[Ajedrecista]

Hi George:

Good day Jesus. Thank you very much for your interest. I am computing the percentages myself, as I am running this match in 3 Fritz 11 guis at once. And I combine the 3 to get one crosstable- and the % is the only thing I did not worry too much about rounding off to perfection. In general terms- for my purposes it is very close, I would think.

I apologize to you for not putting the 2 (what I call markings) above your name in 2 places. Not quite sure how to do it- but if it bothers you I will make sure they get there somehow. I certainly would never want to offend you like that.


Best

george

I said that because I faced the same problem in Fortran a week ago and I was curious. It was due to an incorrect round in my programme, that printed things like 12.3400% (this is a random number) instead of 12.3456% because I choosed less numbers to round. So you are running three Fritz 11 GUIs and then you merge the results. Perfect. I thought it was a flaw in Fritz 11 GUI, nothing else.

Of course you do not offend me in any way: those marks are typical Spanish (they are also seen in more countries) but I know that anglo-saxon keyboards do not have them, or at least I think so. I am very comprehensive with this issue, so please do not worry about it.

More than 40% of the whole match is finished... I expect a relatively close result after all!

Regards from Spain.

Ajedrecista.

[Adam Hair]

Hi George and Jesús,

In case you ever want to use those characters (in Jesús' case, if you are in a English-speaking country), here is what you do:

Make sure the Number keypad is locked on
Press and hold the Alt key
Type 163 in the Number keypad for ú
Or type 164 for ñ
Then release the Alt key. You should then see the proper character (at least you will in this forum).

For a table of extended ascii characters, look here: http://www.asciitable.com/

If you want more unasked for and irrelevant information, I'll be around.


Peace out,
Adam

[geots]

Hi George and Jesús,

In case you ever want to use those characters (in Jesús' case, if you are in a English-speaking country), here is what you do:

Make sure the Number keypad is locked on
Press and hold the Alt key
Type 163 in the Number keypad for ú
Or type 164 for ñ
Then release the Alt key. You should then see the proper character (at least you will in this forum).

For a table of extended ascii characters, look here: http://www.asciitable.com/

If you want more unasked for and irrelevant information, I'll be around.


Peace out,
Adam

That is very interesting. But I would have to print a copy of your thread to follow what you say. My mind can't hold all that at once. (I am glad it is fine with him if I skip it.) My mind is probably twice the age of yours, and doesn't hold 1/1000th of the knowledge. Where do you learn all this stuff?


Best,

george

[Adam Hair]

Hi George and Jesús,

In case you ever want to use those characters (in Jesús' case, if you are in a English-speaking country), here is what you do:

Make sure the Number keypad is locked on
Press and hold the Alt key
Type 163 in the Number keypad for ú
Or type 164 for ñ
Then release the Alt key. You should then see the proper character (at least you will in this forum).

For a table of extended ascii characters, look here: http://www.asciitable.com/

If you want more unasked for and irrelevant information, I'll be around.


Peace out,
Adam

That is very interesting. But I would have to print a copy of your thread to follow what you say. My mind can't hold all that at once. (I am glad it is fine with him if I skip it.) My mind is probably twice the age of yours, and doesn't hold 1/1000th of the knowledge. Where do you learn all this stuff?


Best,

george

You are less than 1½ times as old as I am, if I remember correctly.

Anyway, I wanted to know how to type different characters. I knew it had to be possible. So I googled a bit and found out.

If you are speaking about Jesús, I am sure he knew some or all of it (the statistics and Fortran) already. And what he did not know, he went looking for the answers. Don't let his humbleness fool you, he is sharp.

Anyway, it is the same for every one of us. If there is something you want to learn bad enough, you find a way. You managed to get your 64-bit computer going, even though it was unfamiliar to you, because you wanted to.

Anyway, I am sorry for spamming your thread. Many people like what you are doing. So keep up the good work, George.

Adam

[geots]

You are not spamming anything, Adam. I welcome your comments and your help any time.


Thanks,

george

[Dan Honeycutt]

Hi George and Jesús,

In case you ever want to use those characters (in Jesús' case, if you are in a English-speaking country), here is what you do:

Make sure the Number keypad is locked on
Press and hold the Alt key
Type 163 in the Number keypad for ú
Or type 164 for ñ
Then release the Alt key. You should then see the proper character (at least you will in this forum).

For a table of extended ascii characters, look here: http://www.asciitable.com/

If you want more unasked for and irrelevant information, I'll be around.


Peace out,
Adam

Or highlight the name with your cursor, press Ctrl-C, put the cursor where you want the name to go, press Ctrl-V and voila . . . Jesús.

Best
Dan H.