Houdini 6 has been released

[IWB]

I am not at home and can't finish the two missing games (GUI crashes probably) and make the final calculation. Back home late tomorrow or Friday morning.

It seems to be a little bit stronger than the new version, why that is is something Robert has to explain.

Ingo

[Laskos]

Well, still a little unclear to me if this is N°1....

ASMFishW is about 30% faster than Stockfish so I am not sure how 30% faster translates into ELO.

Would that cover the about 30 ELO advantage Houdini 6 may have over latest Stockfish developments?

rgds

Run from Ingo on Inwobe.de practically complete:

H6

Houdini 6 - Komodo 11.2.2 (3315) 123.5 - 96.5 56.14% Perf=3357
Houdini 6 - Stockfish 8 (3299) 120.0 - 100.0 54.55% Perf=3330
Houdini 6 - Shredder 13 (3120) 171.5 - 48.5 77.95% Perf=3339
Houdini 6 - Fizbo 1.9 (3070) 179.5 - 40.5 81.59% Perf=3328
Houdini 6 - Ginkgo 2.0 (3062) 185.0 - 35.0 84.09% Perf=3351
Houdini 6 - Gull 3 (3056) 188.0 - 32.0 85.45% Perf=3363
Houdini 6 - Booot 6.2 (3028) 192.5 - 27.5 87.50% Perf=3366
Houdini 6 - Jonny 8.00 (3025) 188.0 - 32.0 85.45% Perf=3332
Houdini 6 - Andscacs 0.90 (3023) 197.0 - 23.0 89.55% Perf=3396
Houdini 6 - Equinox 3.30 (3006) 194.5 - 24.5 88.81% Perf=3365
Houdini 6 - Fritz 15 (2995) 198.0 - 22.0 90.00% Perf=3376
Houdini 6 - Chiron 4 (2994) 197.5 - 22.5 89.77% Perf=3371
Houdini 6 - Critter 1.6a (2993) 197.5 - 22.5 89.77% Perf=3370
Houdini 6 - Nirvanachess 2.4 (2967) 196.0 - 23.0 89.50% Perf=3339
Houdini 6 - Hannibal 1.7 (2960) 188.5 - 31.5 85.68% Perf=3270
2717.0 - 581.0 82.38% Perf=3327


3298 out of 3300 games played

H6 scores 54.55% against Stockfish 8. Multiply 4.55 * 7 Elopoints That is probly close to what Stockfish Dev would score (barring statistical uncertainty margins of course. Which are huge!). The difference is in the contempt, so H6 overall result could be better. No offense, and of course there is no Stockfish Dev in the list so that is moot. A match with contempt reduced would be interesting for sure.

How do you know a significant Contempt was used by Houdini 6? With roughly weighted data, I got the following plot of performance, which shows that Contempt, if it was used, was very mild. It may account only for 2-3 ELO points in the final rating, no more, if Contempt was in use at all.

The final rating in Ordo Ingo will present tomorrow I get as (error bars are errors of my estimation, not statistical errors of rating):

Code: Select all

Houdini 6:     3351 +/- 2
Komodo 11.2.2: 3313 +/- 2
Stockfish 8:   2998 +/- 2

We will see tomorrow if I was right.

The plot to roughly see Contempt. Statistic here is weak, but enough to see that no large Contempt was used.

[Houdini]

That Hannibal result is quite the outlier .
It's funny how all Houdini versions over the years have always struggled against the same engines - Hannibal and Stockfish.

[Eelco de Groot]

How do you know a significant Contempt was used by Houdini 6?

Because Houdini has a much more eloborate implementation than the original, very simple, value of the draw ≠ 0. It also means more attractive games, against all but the strongest opposition. It probably works for most Elo lists, that have more weaker opponents than stronger ones. I would be surprised all that changed in H6.

With roughly weighted data, I got the following plot of performance, which shows that Contempt, if it was used, was very mild. It may account only for 2-3 ELO points in the final rating, no more, if Contempt was in use at all.

The slope over the whole range of your graph seems to be about ten points? So that could easily mean 5 points better against Stockfish, and 5 points worse at the lower end, if no contempt was used. And if you leave out Hannibal, what, maybe ten? And I'm assuming it is timecontrol related, aggressive play does not work so well when the surprise effect is tempered because you can out-calculate unsound moves.

The final rating in Ordo Ingo will present tomorrow I get as (error bars are errors of my estimation, not statistical errors of rating):

Code: Select all

Houdini 6:     3351 +/- 2
Komodo 11.2.2: 3313 +/- 2
Stockfish 8:   2998 +/- 2

We will see tomorrow if I was right.

The plot to roughly see Contempt. Statistic here is weak, but enough to see that no large Contempt was used.

[JJJ]

In the meantime, why Houdini and Komodo are both stronger than Stockfish against weaker opponent ?

[Laskos]

How do you know a significant Contempt was used by Houdini 6?

Because Houdini has a much more eloborate implementation than the original, very simple, value of the draw ≠ 0. It also means more attractive games, against all but the strongest opposition. It probably works for most Elo lists, that have more weaker opponents than stronger ones. I would be surprised all that changed in H6.

With roughly weighted data, I got the following plot of performance, which shows that Contempt, if it was used, was very mild. It may account only for 2-3 ELO points in the final rating, no more, if Contempt was in use at all.

The slope over the whole range of your graph seems to be about ten points? So that could easily mean 5 points better against Stockfish, and 5 points worse at the lower end, if no contempt was used. And if you leave out Hannibal, what, maybe ten? And I'm assuming it is timecontrol related, aggressive play does not work so well when the surprise effect is tempered because you can out-calculate unsound moves.

The total difference on the fitted line between the weakest point and the strongest is something like 5 ELO points, not 10. Divide that roughly by 2, and you get my 2-3 ELO points gain. Besides that, the statistic here is such, that the total difference is more like 5 +/- 8 ELO points, might be no slope at all. The total slope of Komodo IIRC was in excess of 20 ELO points, and statistically significant. It did gain some 10 ELO points. So, if Contempt in present in Houdini 6 default, it has a significantly smaller impact than the default Contempt=10 of Komodo.

The final rating in Ordo Ingo will present tomorrow I get as (error bars are errors of my estimation, not statistical errors of rating):

Code: Select all

Houdini 6:     3351 +/- 2
Komodo 11.2.2: 3313 +/- 2
Stockfish 8:   3298 +/- 2

We will see tomorrow if I was right.

[Lyudmil Tsvetkov]

This means Stockfish dev would be ~ 66 Elo stronger than Stockfish 8...

Surely you follow the SF development and probably know that the current SF dev is at best 30 Elo better than SF 8 (see the regression tests at the SF development site).

http://www.cruxis.com/chess/houdini.htm

the strongest engine that money can buy

I believe Robert chose his words very carefully...

The words were chosen carefully not to be repetitive.
The web page also says two lines above the quote you made "Houdini 6: The strongest engine in the world", which I believe Houdini 6 is (and IPON agrees with me).

you are the only person in the world who knows, based on a statistically large sample of games, which one is really better: latest SF dev or latest Houdini, when both face each other with contempt 0, so the word believe is not a very strong indication for me Houdini wins the challenge.

I would suppose it is either <51% for Houdini, <51% for SF, or dead equal.

anyway, a difference smaller than 1% is really too negligeable to call any engine stronger.

for me, we have perfect equality.

you have the right, of course, to advertise your product.

[JJJ]

They might be equal in direct encounter, not against others opponent. Also, the first test are with brainfish or faster compile of Stockfish at blitz time control. Nothing yet like 10 minutes or more, wich would probably decrease a little the advantage of a fast compile.

[APassionForCriminalJustic]

This means Stockfish dev would be ~ 66 Elo stronger than Stockfish 8...

Surely you follow the SF development and probably know that the current SF dev is at best 30 Elo better than SF 8 (see the regression tests at the SF development site).

http://www.cruxis.com/chess/houdini.htm

the strongest engine that money can buy

I believe Robert chose his words very carefully...

The words were chosen carefully not to be repetitive.
The web page also says two lines above the quote you made "Houdini 6: The strongest engine in the world", which I believe Houdini 6 is (and IPON agrees with me).

The issue with what you are doing is that you release one version every year approximately which is pointless. Three months down the road Houdini 6 will be old news like every other compile that gets created. Stockfish 8 is already dead in the grass. If you went to a subscription -based model then it would give people, like myself, more incentive to invest money into your product. Annual releases just do not mean much.

Secondly, until Stockfish is dethroned it's still number one. You're fighting a community with vast resources. Development might be slow right now, but the other way will surely come. The community will always win. It's not an engine reliant on one person or two fellows - but a group of individuals dedicated to the cause of creating the world's strongest engine.

So forget about IPON. Let's wait and see what Houdini can do face to face versus the development version, and if Houdart can continue with his surge.

[h1a8]

Houdini 6 has been released today.
More information on our web page http://www.cruxis.com/chess/houdini.htm .
Also in the updated User's Manual: http://www.cruxis.com/chess/manual/index.html .

Make sure to refresh the browser cache using F5 or Ctrl-F5 to get the most recent page.

Thank you for reading,
Robert

Any plans on releasing Houdini 6 for ios? We desperately need a top engine for analysis for such power Apple has on its devices.
Hopefully it will be part of an app that supports multiple arrow analysis and drag and drop moving.