pedantfish vs SF8

[Eelco de Groot]

Well. I think if asmFish is consistently about 15 points over the Stockfish development version of the same time, not exactly same time because they are not usually tested at the same time, but I think that you will get more than 5 Elo even at say game in 2 hours per player. That is still testable. Above that we just don't have enough information. Self play not considering, as Stefan's list is against other engines. So PedantFish might be much weaker, but I think you could get 15 Elo with AsmFish up to a very decent timecontrol.

Well a 15 ELO increase , with everything else being equal, implies a 30 % in speed - with random oppenents it would be slightly lower at this level. Its more or less accepted doubling of speed at this level is worth perhaps a 35 point ELO gain. Naturally self play with exact same engines would inflate the ELO as the faster engine sees everything the slower identical engine sees plus a little bit more. Totally different scenario when you play against random opponents. But to tout a 25 ELO gain with a 10% increase in speed with what the original poster referenced would be incorrect.

If we take 15 Elo as a given, and 15% increase in speed for both asmFish and pedantFish, you could get there with 70 Elo per doubling (instead of your 35..). It depends on the engine, but Vas Rajlich once, long time ago, quoted 70 Elo per doubling for Rybka, although that was probably at bullet speed (the speed with which Rybka was tested then). I think a part of the 15 Elo can be explained by the openings that Stefan Pohl uses so then it is less than 70. I don't know by how much.

I would think it is more than 35 though for Stefan's tests certainly, and as an estimate for Stockfish against other engines. Is that 35 from your own testing Mike? I would rather go with 50 here (for just a speed doubling, not the opening book), as that was Larry Kaufman's estimate: Elo value of doubling

I decided to approach this question from two different angles, both using CCRL rating lists. First, I ran a match between latest Komodo dev (which is close to ten elo above K10) and the latest version of Arasan, 19.0.1, both on one core. Komodo gave 60 to 1 time odds (almost 6 doublings) with ponder off, specifically 60' + 30" vs 1' + 0.5". Result after 105 games was almost one hundred elo plus for Komodo (99.4 to be precise). This Arasan is quite a strong program, rated 2916 on CCRL 40/40 list. This seems too low by human FIDE standards, since Arasan is rated above versions of Fritz, Junior, and Shredder that are many generations and hundreds of elo points advanced from the versions that fought Kasparov and Kramnik on even terms about 15 years ago. Anyway, Komodo's performance of 3015 compared to its presumed CCRL rating of 3295 (3286 for K10 adding nine for improvement since) means that almost six doublings were worth 280 elo, so each doubling was worth about 48 elo. My previously reported match of this Komodo with Robbolito .084 with Komodo giving 30 to 1 time odds was won by Robbo by just six elo, which would imply that a bit under five doublings were worth sbout 225 elo (using 2070 based on ratings of very similar versions). So maybe 46 elo per doubling. But the Arasan test extended further into the blitz region, so these two results are totally consistent.

My second method was to compare 4-cpu ratings to 1-cpu ratings for recent versions (starting with K8 and SF6) of Komodo and SF. For Komodo the average gain was 81 on 40/40 list, 84 on 40/4 list. For SF (with only 2 versions to average vs. 7 for Komodo) it was 76.5 for 40/40, 71 for 40/4.
The rule of thumb is that four cpu should give about the same performance as 1 cpu with triple the time. It's not exact, but probably close enough for this purpose. So the elo gap should be divided by the ratio of log 3 to log 2, which is 1.59. That gives an average of 52 elo per doubling for Komodo, and 46 for SF (with much more uncertainty).
So it appears that all methods used indicate an average elo value based on CCRL lists of about fifty points per doubling in the range between 40/4 and 40/40. This is less than I expected.

[MikeB]

The examples you are referencing are many hundreds of points weaker than Stockfish and it s been proven that doubling of speed provides greater benefit with weaker engines and a lessor benefit with stronger engines. Secondly , my initial comment was with respect to an engine that was 10% faster than the standard Stockfish. So whether you accept doubling of speed is worth 35 points or 50 points is irrelevant -a 10 % increase in speed for the exact same engine would max out at 5 ELO as I initially stated and there is no room for debate - ask anybody who understands this , ask Larry Kaufman, ask Bob Hyatt, ask Raj - how much ELO would you gain from a 10% increase and I can guarantee you that at the most they would say 5 ELO.

[syzygy]

15% speedup is about 1/5th of a doubling in speed (1.15^5 = 2.01).

So if a doubling is 50 Elo, then 15% speedup is 10 Elo.
If a doubling is 70 Elo, then 15% speedup is 14 Elo.

[Eelco de Groot]

The examples you are referencing are many hundreds of points weaker than Stockfish and it s been proven that doubling of speed provides greater benefit with weaker engines and a lessor benefit with stronger engines. Secondly , my initial comment was with respect to an engine that was 10% faster than the standard Stockfish. So whether you accept doubling of speed is worth 35 points or 50 points is irrelevant -a 10 % increase in speed for the exact same engine would max out at 5 ELO as I initially stated and there is no room for debate - ask anybody who understands this , ask Larry Kaufman, ask Bob Hyatt, ask Raj - how much ELO would you gain from a 10% increase and I can guarantee you that at the most they would say 5 ELO.

You said against random opponents it would be less than 5 but I think, as you say above, it should be more than 5 (Edit: sorry, my mistake I mean against much weaker opponents, not random opponents. As per your statement above) If you take 5 Elo, that is consistent with your 35 Elo per doubling. Log 2/log 1.1 ≈ 0,301 / 0.041 ≈ 7.27,

7.27 * 5 ≈ 35 But I think the value of 35 Elo per doubling would be reached only at very, very long timecontrols. So for correspondence analysis you might very well be correct. I have not seen anyone else posting as low as 35. No offense to you Michael, you are being consistent in your calculations with your 35 Elo per doubling figure.

[MikeB]

15% speedup is about 1/5th of a doubling in speed (1.15^5 = 2.01).

So if a doubling is 50 Elo, then 15% speedup is 10 Elo.
If a doubling is 70 Elo, then 15% speedup is 14 Elo.

Agree - so a 10.5 % increase is worth about 7 ELO if you believe doubling is worth 50 ELO (1.105^7 = 2.01 : 50/7 or 5 ELO if doubling is worth 35 ELO (35/7). So you and I agree on the mathematics - it's just a question of what is doubling is worth. I'm more conservative - but maybe it's 50. I was just responding to a post where someone that my obersavation of 5 ELO gain is debateable for a 10% increase and I guess it is , maybe it's worth 7 ELO - but certainly not an ELO difference that would one make a comment "that's debateable "

[Eelco de Groot]

I am sorry for not making clear what I meant with my remark. I did not mean to question the method or calculation, "This is debatable" is an unlucky phrase and I should have known better than that because it had the same effect in a very different discussion long ago. "Debatable" is not the right word if you just want to argue that a number could be higher.

The main point that I wanted to stress in my post was a very different one: I was trying to get some attention to the apparent plight of the programmer of asmFish, although I must admit I have only very limited information on that, and, if true, he probably would not want the attention anyway. But I don't think Trump should just get away with the way he set up everybody against each other in the election -Hillary just was goaded into this, it is true, not discussing issues but only the man-, setting people up against minorities, strangers, foreigners, illegal immigrants, women...

I fear much of this will happen again here in The Netherlands, coming March when we are having parliamentary elections and we have in Geert Wilders somebody whose methods are very much like those of Donald Trump. Preparing for Nexit++11

[MikeB]

I understand. I wasn't looking for an apology , but if you are offering one, it is accepted. For what it is worth , I did some short tests and it does appear that doubling speed is worth at 50, maybe even slightly more , so yes I now agree a 10% speed pick up is worth at least 7 ELO and the 5 EL0 I suggested was too low.

[ernest]

So for my intended use - IDeA analysis in Aquarium,...

For analysis, don't forget to use Large Pages !

Compared to SF8, you will then be in the 25%-30% range of increase in speed !