lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
lkaufman wrote: it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
That simply doesn't follow.
But your argument is the reverse. You are saying say 100 Elo for 100% speedup, so therefore 10% speedup must be 10% Elo. I very much doubt that is the case. 10% speedup could just be 1 Elo for example. To me it is probably a stepped line, the Elo increase corresponding with the point at which the engine starts to reach higher depths as a result.
There is zero evidence that Elo scales in a perfectly linear fashion with speedup increases. I've never seen any proper tests of it.
Over the decades, there has been quite a bit of evidence to that effect. It is super easy to test. Take two engines A and B and test them for 100 games at 40/1. Now, test them again with A using 40/1 and B using 40/2. Now, test again with A at 40/1 and B using 40/4. Cutechess has the ability to do these matches. Also, pick A and B such that Elo(A) ~= Elo(B).
Also, the line may be different if you pick two engines at 2000 Elo vs two engines at 3000 Elo.
That would be a useful test yes, but 100 games doesn't give any confidence to me. Maybe 1,000 or much more and then we might have something to talk about. If I knew how to use cutechess I'd run such a test, but I've never used it.
Also, the line may be different if you pick two engines at 2000 Elo vs two engines at 3000 Elo.
The you are disagreeing with Larry then. The way he talks, it is the same straight line connecting the two end points for every engine.
Modern Times wrote:That would be a useful test yes, but 100 games doesn't give any confidence to me. Maybe 1,000 or much more and then we might have something to talk about. If I knew how to use cutechess I'd run such a test, but I've never used it.
Also, the line may be different if you pick two engines at 2000 Elo vs two engines at 3000 Elo.
The you are disagreeing with Larry then. The way he talks, it is the same straight line connecting the two end points for every engine.
No. I am not disagreeing with him. I am just giving you a way to test the idea.
lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
according to your logic then, it should be 340 ELO stronger?!
I said to use the log (natural) of the speed ratio, not the ratio itself. See Miguel's respons. Also it doesn't work like this for MP, since many nodes are redundantly searched. Speedup must be one that doesn't change the search, like faster hardware or better coding, for this to apply.
lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
Which is valid as smaller dRI are used
if a = 100, then 1% of speed (dRI = 0.01) equals 1 extra elo, and that is proportional for small values.
Miguel
ln (1 + x) ~ x, as long as x is much smaller than 1
using extremely small values (for ex: < 1) to approach results that approximate equivalence does not mean the relationship is directly proportional
that's cherry picking
lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
according to your logic then, it should be 340 ELO stronger?!
I said to use the log (natural) of the speed ratio, not the ratio itself. See Miguel's respons. Also it doesn't work like this for MP, since many nodes are redundantly searched. Speedup must be one that doesn't change the search, like faster hardware or better coding, for this to apply.
how do you arrive at 10% = 10 ELO, as per in your quotes above
the natural logarithm of 10 is 2.30
lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
according to your logic then, it should be 340 ELO stronger?!
I said to use the log (natural) of the speed ratio, not the ratio itself. See Miguel's respons. Also it doesn't work like this for MP, since many nodes are redundantly searched. Speedup must be one that doesn't change the search, like faster hardware or better coding, for this to apply.
how do you arrive at 10% = 10 ELO, as per in your quotes above
the natural logarithm of 10 is 2.30
Please read more carefully. I said the natural log of the ratio, which in this case is 1.1. Then multiply by 100, since the assumption is that 1% = one elo. What do you get? Then do it for a ratio of 0.9. Take the average. This is the value of a ten percent speedup averaging the two possible meanings of a 10% speedup. For example, if we run twice as fast, is that a 100% speedup or a 50% reduction in time?
lkaufman wrote:
In the past, when I've noticed that someone running tests on AMD machines (for example Ingo/Ipon) gets worse ratings for Komodo relative to Houdini than on either my Intel tests or others (CEGT at same time limit as IPON or myself), we've checked relative nodes per second and invariably the ratio in favor of Houdini is significantly higher (ballpark of ten percent) on AMD. So roughly running on AMD costs Komodo about ten elo vs. Houdini compared to running on Intel.
lkaufman wrote:
I can't prove that it is unwise to punch a gorilla, but it is a pretty safe bet. I have never seen any test results that make me doubt this, and I've seen quite a lot. it's really just common sense. If a one percent speedup is worth one elo point, then ten of them should be worth ten elo. The assumption is that elo gains are additive, which seems to be universally true for engines. Do you doubt this?
Larry-
yes of course I doubt it...NPS and ELO are not directly proportional
according to your logic then, it should be 340 ELO stronger?!
I said to use the log (natural) of the speed ratio, not the ratio itself. See Miguel's respons. Also it doesn't work like this for MP, since many nodes are redundantly searched. Speedup must be one that doesn't change the search, like faster hardware or better coding, for this to apply.
how do you arrive at 10% = 10 ELO, as per in your quotes above
the natural logarithm of 10 is 2.30
Please read more carefully. I said the natural log of the ratio, which in this case is 1.1. Then multiply by 100, since the assumption is that 1% = one elo. What do you get? Then do it for a ratio of 0.9. Take the average. This is the value of a ten percent speedup averaging the two possible meanings of a 10% speedup. For example, if we run twice as fast, is that a 100% speedup or a 50% reduction in time?
ok now I understand how you arrived at 10 ELO...
of course any assumptions in your method may add error
for ex: Ray says he measured it here and got only 2%
Modern Times wrote:Well, I have done the nps comparison here on an Intel Sandybridge, and an AMD Phenom II X6.
The 10% difference doesn't exist here. It is 2%. So that dispels the theory about Komodo being disadvantaged on AMD anyway.
But in any event, thx much for explaining it
my apologies to Andrev if our discussion took his thread off-topic
