STS - List the Order of Importance

[noctiferus]

Great .
Give me the weekend for a preliminary shallow standard analysis.
Later (don't hold your breath), I'll try some more sophisticated (? ) techniques...hoping they work...

[swami]

Great .
Give me the weekend for a preliminary shallow standard analysis.
Later (don't hold your breath), I'll try some more sophisticated (? ) techniques...hoping they work...

Finished up entering datas, and calculated the total scores. Have sent it to your id. Thanks for the work, will be looking forward to the interesting piece of research. If there's not enough rating information besides WBEC, CEGT or CCRL, you may use some other ratings list as well

(UEL, Chesswar... but these have rating measurement that is 200 to 300 elo apart from the likes of CCRL/CEGT, for example, Slowchess would be rated 2450 to 2550 in UEL/Chess war but it's 2700 in CCRL)

In meantime, have fun experimenting with it!

[swami]

This is the excel file with results of nearly 50+ engines in case if anyone else is interested:

http://sites.google.com/site/strategict ... STS1-8.xls

PS: To Carlos, can you sort it in total scores order and make a Gif image of it and post it here? Thanks!

[noctiferus]

I think that the ratings you give are more than enough.
The only trouble are the misssing values.

At first analysis, missing engines sould be ignored: filling up missings needs at least a 2-step procedure.. it is a bit time consuming, so better analyses will follow... later...

Anyway, it is a fun job (unless results will be so bad that I'll have to scratch my head, and this is much less funny: I spent three months, some time ago, trying to get DM models from a DB, in which there was no significant info..-:( ).

Sure this is not the case.

[swami]

I think that the ratings you give are more than enough.
The only trouble are the misssing values.
At first analysis, missing engines sould be ignored: filling up missings needs at least a 2-step procedure.. it is a bit time consuming, so better analyses will follow... later...

Yes, ratings of some engines are missing because those engines haven't been tested by the respective testing team, or they have only tested it in multi-cpu, or they had tested a completely different version.

I could also do STS tests on few more engines in lower order in the range of 2100-2300 over the next weekend if that satisfies the requirement needed to get more precise data.

[noctiferus]

Relax, Swami, let's see what comes out...

[swami]

Relax, Swami, let's see what comes out...

Sure, Have fun with it!

Next up is STS v9.0 Advancement of a/b/c pawns which will be released in week's time! I have already submitted 200 positions to Dann. This is certainly more interesting test IMO. The queenside fights.

[Edmund]

Thats what I am getting after a linear regression:
r² = 0.6577
sqr(mse) = 68.912

sts10-80 are the coefficients
cons is the constant
the result is the elo in the CEGT scale

Code: Select all

------------------------------------------------------------------------------
         elo |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       sts10 |   2.272232    2.96261     0.77   0.448    -3.736218    8.280683
       sts20 |    3.20341    3.01614     1.06   0.295    -2.913606    9.320425
       sts30 |   .4575349   2.331547     0.20   0.846    -4.271063    5.186132
       sts40 |   2.358127   2.152502     1.10   0.281     -2.00735    6.723604
       sts50 |   7.583491   3.162585     2.40   0.022      1.16947    13.99751
       sts60 |  -2.137339   2.555934    -0.84   0.409    -7.321013    3.046336
       sts70 |   .8115352   2.652622     0.31   0.761    -4.568232    6.191302
       sts80 |   4.939367    2.18094     2.26   0.030      .516216    9.362519
       _cons |   1342.862   209.7266     6.40   0.000     917.5169    1768.207
------------------------------------------------------------------------------

[swami]

Thats what I am getting after a linear regression:
r² = 0.6577
sqr(mse) = 68.912

sts10-80 are the coefficients
cons is the constant
the result is the elo in the CEGT scale

Code: Select all

------------------------------------------------------------------------------
         elo |      Coef.   Std. Err.      t    P>|t|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
       sts10 |   2.272232    2.96261     0.77   0.448    -3.736218    8.280683
       sts20 |    3.20341    3.01614     1.06   0.295    -2.913606    9.320425
       sts30 |   .4575349   2.331547     0.20   0.846    -4.271063    5.186132
       sts40 |   2.358127   2.152502     1.10   0.281     -2.00735    6.723604
       sts50 |   7.583491   3.162585     2.40   0.022      1.16947    13.99751
       sts60 |  -2.137339   2.555934    -0.84   0.409    -7.321013    3.046336
       sts70 |   .8115352   2.652622     0.31   0.761    -4.568232    6.191302
       sts80 |   4.939367    2.18094     2.26   0.030      .516216    9.362519
       _cons |   1342.862   209.7266     6.40   0.000     917.5169    1768.207
------------------------------------------------------------------------------

So, according to this piece of information, one could conclude that STS 6.0 is the least important of all test suites which even has a negative co-efficient and that it is better not to do well in it? Looks little confusing and probably not true.

Also, with highest co-efficient value for STS 5.0 followed by STS 8.0 (?!) indicates they are the two most important according to this regression.

So this data gives the ranks for order of importance, it's something like:

STS 5
STS 8
STS 2
STS 4
STS 1
STS 7
STS 3
STS 6

Not a bad try at all. Since I expect all the middle ranks to be in right place except STS 6 and 8.

[mcostalba]

Code: Select all

       sts60 |  -2.137339   2.555934    -0.84   0.409    -7.321013    3.046336

I think the coefficients should be constrained to be positive, otherwise it means that the higher score get an engine in STS 6 the weaker it is