Jonny 8.0 tested by Ingo

[APassionForCriminalJustic]

When I asked Johannes how much Jonny gained from a 24 core version on one machine to the full cluster of 2400 cores, he said about 350 elo. I do not know how much data he had (the cluster takes a lot of power), but that is what he said.

So elo gain from 1-24 core plus +350 would be the overall gain.

350 ELO sounds insane...

[Uri Blass]

In 1 year, SF and Komodo will be around 3300.
That still +300 Elo to do for Jonny.
May be 2 years is more reasonable to get in top 2.

Code: Select all

   # PLAYER              : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 Komodo 10           :   3249     10   80.0%    31.0    2988     100        2640.0    2129    1022     149    3300
   2 Stockfish 7         :   3229     10   78.1%    38.4    2990     100        2577.0    1944    1266      90    3300
   3 Houdini 4           :   3114      9   65.0%    36.9    2997     100        2146.0    1537    1218     545    3300
   4 Gull 3              :   3064      8   58.5%    45.8    3001     100        1931.0    1175    1512     613    3300
   5 Jonny 8.00          :   3025      8   53.2%    44.5    3003      84        1756.5    1022    1469     809    3300

It is not like Komodo and Stockfish development are going to slowdown. Unless someone with serious skill(s) and programming talent comes around it will be hard-pressed to imagine seeing another engine effectively match the two present giants. There are definitely better engines out there to fight for the top five as opposed to Jonny. For instance, Houdart is making a comeback, Fire looks great, Andscacs is up and coming, etc,.

I'm not buying any of this Jonny stuff. Let's see.

I think that being weaker than houdini and Gull is clearly worse than what I expected from Jonny considering TCEC results.

Jonny7.30 has 13 out of 19 in TCEC
It is better than what other engines have except stockfish and komodo.


http://tcec.chessdom.com/live.php

[Dann Corbit]

It is ten more doublings of core count compared to 24.
If 40 Elo per doubling held, it would be 400 Elo.
If it is 350 Elo, then it is very efficient.
It would be interesting to know what SMP scheme is used.

[lkaufman]

It is ten more doublings of core count compared to 24.
If 40 Elo per doubling held, it would be 400 Elo.
If it is 350 Elo, then it is very efficient.
It would be interesting to know what SMP scheme is used.

The core count was a hundred times 24, not a thousand, so it's only about 6 2/3 doublings, not ten. If Jonny really gained 350 elo, that's more than fifty per doubling. Fifty elo for twice the speed is more or less normal, but to get that much per doubling of core count would be amazing. If it's true, it was a miracle that Komodo won the world championship and beat Jonny both in the tournament proper and in the playoff without losing a game. I don't think Komodo got much benefit from going from 24 cores to 48.

[mjlef]

It is ten more doublings of core count compared to 24.
If 40 Elo per doubling held, it would be 400 Elo.
If it is 350 Elo, then it is very efficient.
It would be interesting to know what SMP scheme is used.

The core count was a hundred times 24, not a thousand, so it's only about 6 2/3 doublings, not ten. If Jonny really gained 350 elo, that's more than fifty per doubling. Fifty elo for twice the speed is more or less normal, but to get that much per doubling of core count would be amazing. If it's true, it was a miracle that Komodo won the world championship and beat Jonny both in the tournament proper and in the playoff without losing a game. I don't think Komodo got much benefit from going from 24 cores to 48.

There is a clear diminishing of returns with core doublings. Some good data here:

http://www.fastgm.de/threads2.html

This data include Komodo 9.3 which is pretty recent, and shows elo gains from doublings up to 16 threads. The doubling from 8 to 16 threads was only worth 49 elo for Komodo 9.3. It shows a clear drop off (the 1 to 2 thread result was 88 elo).

Fitting the data to a simple linear equation we get:

y = -13.9 x + 108

where x is the doubling (1 is doubling from 1 to 2 threads, 2 is doubling 2 to 4 and so on) and y is the elo gain. So somewhere between the 7 and 8th doubling the formula would predict an elo drop. Of course reality would have to be tested.

There are other ways to use core than the scheme now in Komodo. You can ponder multiple nodes, and preponder another move ahead. These techniques could give the program up to a doubling in search time.

All of these results are at short time controls. Programs will have more draws at longer time controls, meaning the benefit of another doubling will be less. This will depend on how strong a program is on one core.

[Laskos]

When I asked Johannes how much Jonny gained from a 24 core version on one machine to the full cluster of 2400 cores, he said about 350 elo. I do not know how much data he had (the cluster takes a lot of power), but that is what he said.

So elo gain from 1-24 core plus +350 would be the overall gain.

24 -> 2400 is 350 ELO points or 1 -> 24 is 350 ELO points? Anyway, 350 points is exaggerated for both cases. Effective speed-up 1 -> 24 is no larger than 12, or 3.6 doublings, or at that time control, about 200 ELO points. The effective speed-up 24 -> 2400 cores is no larger than 10, or 3.2 doublings, 160 ELO points at that time control. So, maybe the total spedd-up 1 -> 2400 cores is about 350 ELO points at LTC, and the advantage over Komodo hardware in WCCC of 150 ELO points. So on 2400 cores Jonny is still a bit weaker than Komodo on 48 cores.

[JJJ]

In 1 year, SF and Komodo will be around 3300.
That still +300 Elo to do for Jonny.
May be 2 years is more reasonable to get in top 2.

Code: Select all

   # PLAYER              : RATING  ERROR     (%)    D(%)  OppAvg   CFS(next)    POINTS       W       D       L  PLAYED
   1 Komodo 10           :   3249     10   80.0%    31.0    2988     100        2640.0    2129    1022     149    3300
   2 Stockfish 7         :   3229     10   78.1%    38.4    2990     100        2577.0    1944    1266      90    3300
   3 Houdini 4           :   3114      9   65.0%    36.9    2997     100        2146.0    1537    1218     545    3300
   4 Gull 3              :   3064      8   58.5%    45.8    3001     100        1931.0    1175    1512     613    3300
   5 Jonny 8.00          :   3025      8   53.2%    44.5    3003      84        1756.5    1022    1469     809    3300

It is not like Komodo and Stockfish development are going to slowdown. Unless someone with serious skill(s) and programming talent comes around it will be hard-pressed to imagine seeing another engine effectively match the two present giants. There are definitely better engines out there to fight for the top five as opposed to Jonny. For instance, Houdart is making a comeback, Fire looks great, Andscacs is up and coming, etc,.

I'm not buying any of this Jonny stuff. Let's see.

I think that being weaker than houdini and Gull is clearly worse than what I expected from Jonny considering TCEC results.

Jonny7.30 has 13 out of 19 in TCEC
It is better than what other engines have except stockfish and komodo.


http://tcec.chessdom.com/live.php

Jonny has won a lost game because chiron crashed. Jonny is doing less good than Fire 5 or maybe equal. Fire 5 is probably now better than Houdini 4, all it needed was +50 elo ( and Fire won 50 elo each years with update, now it's been 1,5 years since last one ) and to have a working version for TCEC hardware.

[mjlef]

When I asked Johannes how much Jonny gained from a 24 core version on one machine to the full cluster of 2400 cores, he said about 350 elo. I do not know how much data he had (the cluster takes a lot of power), but that is what he said.

So elo gain from 1-24 core plus +350 would be the overall gain.

24 -> 2400 is 350 ELO points or 1 -> 24 is 350 ELO points? Anyway, 350 points is exaggerated for both cases. Effective speed-up 1 -> 24 is no larger than 12, or 3.6 doublings, or at that time control, about 200 ELO points. The effective speed-up 24 -> 2400 cores is no larger than 10, or 3.2 doublings, 160 ELO points at that time control. So, maybe the total spedd-up 1 -> 2400 cores is about 350 ELO points at LTC, and the advantage over Komodo hardware in WCCC of 150 ELO points. So on 2400 cores Jonny is still a bit weaker than Komodo on 48 cores.

The Jonny programmer claimed going from a 24 core version to a 2400 core version was worth 350 elo. I have no idea if this was a mere estimate or something he tested. I do not think much cluster work was done on Jonny for the last year, so it might have been tested with an earlier version of Jonny. Weaker programs gain more with more processors.

Komodo 9.3 at a 60+0.05 seconds per game time control on the fastge.de page I referenced gains 293 elo. That would naturally be lower at a much longer time controls, but note the final tie break games were at 3 min + 5 secs, which is nnot so far off the fastgm.de results.

[Werewolf]

When I asked Johannes how much Jonny gained from a 24 core version on one machine to the full cluster of 2400 cores, he said about 350 elo. I do not know how much data he had (the cluster takes a lot of power), but that is what he said.

So elo gain from 1-24 core plus +350 would be the overall gain.

350 ELO sounds insane...

I tried telling you...Jonny on that hardware is very highly rated and I think at least equal to the 48 core Komodo.

But we have tiny game samples, the final game was at 3 mins and it was manually entered into a board making the result not much to go on.

[peter]

the final game was at 3 mins and it was manually entered into a board making the result not much to go on.

+1