MYG

[Nordlandia]

Kai Laskos: Is new Houdini on par or superior to SF9 Dev?

[Laskos]

Kai Laskos: Is new Houdini on par or superior to SF9 Dev?

At this time control on one core Houdini seems stronger than the best SF dev. But TCEC is a bit different, hard to say.

[stavros]

Hello all,

most of you guessed right - but as this is a Houdini pre release version it will not be included officially in my list.
Nonetheless here is how it would look like:

Code: Select all

   # PLAYER              : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 NEW                 :   3343     10   81.7%    30.3    3059     100        2694.5    2195     999     106    3300
   2 Komodo 11.2.2       :   3313     10   78.9%    34.5    3061      98        2604.0    2034    1140     126    3300
   3 Stockfish 8         :   3298     10   77.4%    39.1    3062     100        2555.5    1910    1291      99    3300
   4 Shredder 13         :   3119      8   56.3%    50.7    3074     100        1859.5    1023    1673     604    3300
   5 Fizbo 1.9           :   3069      8   49.7%    42.9    3078      96        1640.0     932    1416     952    3300
   6 Ginkgo 2.0          :   3059      8   48.3%    49.8    3078      70        1593.0     772    1642     886    3300
   7 Gull 3              :   3056      8   47.8%    47.9    3078     100        1579.0     788    1582     930    3300
   8 Booot 6.2           :   3025      8   43.7%    50.6    3080      57        1442.5     608    1669    1023    3300
   9 Jonny 8.00          :   3024      7   43.6%    46.4    3081      65        1438.0     672    1532    1096    3300
  10 Andscacs 0.90       :   3022      8   43.3%    45.3    3081     100        1428.5     681    1495    1124    3300
  11 Equinox 3.30        :   3004      8   40.8%    47.9    3082      97        1348.0     558    1580    1162    3300
  12 Critter 1.6a        :   2993      8   39.4%    47.2    3083      50        1300.0     522    1556    1222    3300
  13 Chiron 4            :   2993      9   39.4%    45.3    3083      51        1300.0     553    1494    1253    3300
  14 Fritz 15            :   2993      8   39.4%    47.2    3083     100        1299.5     520    1559    1221    3300
  15 Nirvanachess 2.4    :   2964      8   35.6%    44.9    3085      90        1175.5     434    1483    1383    3300
  16 Hannibal 1.7        :   2956      8   34.6%    44.2    3085     ---        1142.5     413    1459    1428    3300

and as a comparision this is the current list:

Code: Select all

   # PLAYER              : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 Komodo 11.2.2       :   3315     10   79.5%    34.7    3059      99        2625.0    2053    1144     103    3300
   2 Stockfish 8         :   3299     10   78.0%    39.7    3060      99        2573.0    1918    1310      72    3300
   3 Houdini 5.01        :   3281     10   76.2%    39.2    3061     100        2514.0    1868    1292     140    3300
   4 Shredder 13         :   3120      8   56.8%    51.8    3072     100        1875.0    1021    1708     571    3300
   5 Fizbo 1.9           :   3070      8   50.0%    43.4    3075      94        1651.0     935    1432     933    3300
   6 Ginkgo 2.0          :   3062      8   48.8%    50.8    3075      81        1611.0     772    1678     850    3300
   7 Gull 3              :   3056      8   48.1%    48.3    3076     100        1587.0     790    1594     916    3300
   8 Booot 6.2           :   3028      8   44.2%    51.5    3078      68        1458.5     608    1701     991    3300
   9 Jonny 8.00          :   3025      8   43.8%    46.9    3078      66        1446.5     672    1549    1079    3300
  10 Andscacs 0.90       :   3023      8   43.5%    45.6    3078     100        1436.0     684    1504    1112    3300
  11 Equinox 3.30        :   3006      8   41.2%    48.4    3079      96        1358.0     560    1596    1144    3300
  12 Fritz 15            :   2995      8   39.7%    47.9    3080      55        1311.0     520    1582    1198    3300
  13 Chiron 4            :   2994      8   39.6%    45.8    3080      58        1307.5     551    1513    1236    3300
  14 Critter 1.6a        :   2993      8   39.5%    47.4    3080     100        1302.5     520    1565    1215    3300
  15 Nirvanachess 2.4    :   2967      8   36.0%    45.6    3082      87        1187.0     434    1506    1360    3300
  16 Hannibal 1.7        :   2960      8   35.1%    44.9    3082     ---        1157.0     416    1482    1402    3300

I am impressed, I did not expect that result!

I hope you enjoyed the run as much as I did

Ingo

PS: FYI: a 1 month old SF dev was below that in my setup

Thank you very much Ingo. Impressive results, and my ELO predictions were pretty accurate (aside the last, where I missed a couple of ELO points). Congratulations to Robert![/quote

give some congratulations to open source sf too. i call this as houdifish 6

[Laskos]

give some congratulations to open source sf too. i call this as houdifish 6

Well, I will not enter these issues, as I might wrongly say that all of the very top engines must have "fish" termination.

[JJJ]

So, it is not the final Houdini here. I m glad, because it still need some elo to beat Stockfish directly.

And I was right since the begining, that's good to me. Where is my free copy ?:D

[JJJ]

Kai Laskos: Is new Houdini on par or superior to SF9 Dev?

At this time control on one core Houdini seems stronger than the best SF dev. But TCEC is a bit different, hard to say.

On TCEC Houdini 5 was beaten by 54% against Stockfish 8( short sample )
On fast.gm Houdini was beaten 51% on 1 core and 1 hour time control by Stockfish 8

Maybe we could guess Stockfish 8 is slighty better than Houdini 5 in TCEC condition.

So, this Houdini win barely Stockfish 8, probably loose against Stockfish dev and has to play in TCEC condition where he might be slighty below as well.

So the goal to be the absolute best is not totally achevied here. Still a good update, because it seems closer than Stockfish than last time.

[Laskos]

give some congratulations to open source sf too. i call this as houdifish 6

Well, I will not enter these issues, as I might wrongly say that all of the very top engines must have "fish" termination.

A year ago I posted this as a joke:

http://www.talkchess.com/forum/viewtopi ... 5&start=53

The performance of Shredder is really wonderful

How many wins against SF and Komodo, in these two tournaments + the last one from December?

Shredder 160625 isn't as strong as Stockfish and Komodo but Shredder 160625 is +300 elo stronger than Shredder 12.

+300 elo progress at 18 months is incredible. Shredder can reach the level of Komodo and Houdini next year.

If Stockfish continues it this current manner, it is very possible the improvement of many very strong engines will look like that:



The improvement would hardly be in this case linear on entire span, and overcoming Stockfish is a bit in another paradigm.

[JJJ]

Lyudmil, Houdini doesn't use contempt anymore. I know you still think it does in his way, but I disagree.

Also, it is better to have Houdini 6 close to Stockfish than Houdini not updated. Also, Komodo is cloosing the gap and that's better than having Stockfish alone better than the others two engines by ~50-80 elo like it was for a while, until Houdini 5 and until his progress slowed down.

So of course, you wanted a much better engine or a total new one, like we all, but it is to me far better than nothing, than not having a better Houdini.

[Houdini]

Thank you, Ingo, for running the test! It was fun to watch.

it already starts getting annoying: having 3 top engines for 5 years at fully or almost fully the same strength: for how long could that continue?

A top engine got improved by 50 to 60 Elo in 10 months, and your contribution is... a rant?

Thank you very much Ingo. Impressive results, and my ELO predictions were pretty accurate (aside the last, where I missed a couple of ELO points). Congratulations to Robert!

You made some pretty astute predictions.
But maybe not sufficiently taking into account the error margins on the individual results - with 200 games it must be like ±30 Elo. With only 2 or 3 data points on the right side of the diagram, the slope of your line is pretty uncertain.

give some congratulations to open source sf too. i call this as houdifish 6

Undoubtedly without SF all engines (including Houdini and Komodo) would be weaker, maybe by 100 points.
But it's not nearly as straightforward as you think. While engines have many high-level similarities, the low level differences usually are too big for any of the SF improvement ideas or "patches" to be applicable or useful for other engines.

[JJJ]

That's a good progress Robert. Will you try to get more elo before an official release ?