It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
harware vs software advances
Moderators: hgm, Rebel, chrisw
-
- Posts: 13447
- Joined: Wed Mar 08, 2006 9:02 pm
- Location: Dallas, Texas
- Full name: Matthew Hull
Re: harware vs software advances
Matthew Hull
-
- Posts: 6401
- Joined: Thu Mar 09, 2006 8:30 pm
- Location: Chicago, Illinois, USA
Re: harware vs software advances
That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
-
- Posts: 13447
- Joined: Wed Mar 08, 2006 9:02 pm
- Location: Dallas, Texas
- Full name: Matthew Hull
Re: harware vs software advances
None of that matters when comparing the same program on old and new hardware. All that matters is what was the HW speedup and what was the ELO gain for the 1995 program at that speedup.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
Matthew Hull
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: harware vs software advances
Thank you Miguel. You said it better than I could and this is really clear.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
I was getting frustrated trying to make Bob understand this concept and it was just because I am not very good at explaining myself.
-
- Posts: 13447
- Joined: Wed Mar 08, 2006 9:02 pm
- Location: Dallas, Texas
- Full name: Matthew Hull
Re: harware vs software advances
It's not relevant to the question being asked. All you need to know is how much ELO to attribute to HW speedup. How something was or was not optimized to HW has nothing to do with the answer. Only the speedup numbers matter. Whether a particular project sped up 10x or 1000x is the only answer you need. Then figure how how much ELO that translates to. That gives you the HW ELO increase (for that chess project). Any increase above that in a newer version (of that project) would be a candidate for software ELO improvement. This seems to be the general idea.Don wrote:Thank you Miguel. You said it better than I could and this is really clear.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
I was getting frustrated trying to make Bob understand this concept and it was just because I am not very good at explaining myself.
I mean, can the HW speedup for 1995 Crafty be determined or not? If it can, where is the problem?
Matthew Hull
-
- Posts: 5106
- Joined: Tue Apr 29, 2008 4:27 pm
Re: harware vs software advances
Ok, if we go along with your premise, then why not use the Rebel numbers? In your words, "only the speedup numbers matter." And the numbers are 100 to 1 for Rebel. And I'm actually using an IDENTICAL program, you are not.mhull wrote:It's not relevant to the question being asked. All you need to know is how much ELO to attribute to HW speedup. How something was or was not optimized to HW has nothing to do with the answer. Only the speedup numbers matter. Whether a particular project sped up 10x or 1000x is the only answer you need. Then figure how how much ELO that translates to. That gives you the HW ELO increase (for that chess project). Any increase above that in a newer version (of that project) would be a candidate for software ELO improvement. This seems to be the general idea.Don wrote:Thank you Miguel. You said it better than I could and this is really clear.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
I was getting frustrated trying to make Bob understand this concept and it was just because I am not very good at explaining myself.
I mean, can the HW speedup for 1995 Crafty be determined or not? If it can, where is the problem?
-
- Posts: 13447
- Joined: Wed Mar 08, 2006 9:02 pm
- Location: Dallas, Texas
- Full name: Matthew Hull
Re: harware vs software advances
Ok, Rebel it is. 100 to 1. No problem. What's the ELO increase for Rebel with 100x speedup? I can go with that for Rebel.Don wrote:Ok, if we go along with your premise, then why not use the Rebel numbers? In your words, "only the speedup numbers matter." And the numbers are 100 to 1 for Rebel. And I'm actually using an IDENTICAL program, you are not.
For 1995 crafty, the speedup will be more than 100x, for the well-understood reason that it's more capable of it by design. But that's not important. What matters is what is 1995 Crafty's speedup, and what does that particular speedup translate to in ELO terms for 1995 crafty?
It's the same measurement for both programs, differing only by degree of speedup. But the function curve (in theory) should be similar, it's just that the two examples will touch the curve at different points along the speedup-ELO function.
Matthew Hull
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: harware vs software advances
And your point would be? I did the same for parallel search if you recall. But I _designed_ Crafty to work on 32 bit machines. The original rotated stuff had a ton of 32 bit optimizations, with the idea that one day they would not be needed.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
Somehow you seem to imply that planning for the future, having the foresight to predict that 64 bit processors would become standard, and such, is somehow a bad idea? I give raises to people that thing progressively, not fire 'em. That's a good characteristic to have, not a bad one. And for me, it worked. I had a competitive program in 1995. It gained more from 64 bit hardware than those that chose to stick with status quo. That is a problem?
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: harware vs software advances
There is no "concept" to explain here. I chose a data structure that was forward-thinking, based on a lot of hardware experience, and somehow that makes this an invalid discussion? Shoot, let's all use regressive software design techniques, never ask "what might we have next year so that we can factor that into our design" and, as a result, never see the advantages the new things offer. makes sense to me.Don wrote:Thank you Miguel. You said it better than I could and this is really clear.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
I was getting frustrated trying to make Bob understand this concept and it was just because I am not very good at explaining myself.
I'd reverse this. Any program that only gets a 100x speedup from P5/90 to i7 is a piece of crap. That's the right way to think. Not to make it the gold standard just because it is the worst example of progress you can find. This sounds more like a debate technique where you first decide on the answer you want, and then start searching for data to support that, whether the data is sensible or not is irrelevant, it just needs to exist.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: harware vs software advances
First, he _is_ using the identical program also. Unless you want to argue that Crafty 10.18 is not the same as Crafty 10.18? If you want to use the Rebel numbers, that's fine. And you can report what Rebel gained. I'm going to report gaining more from the hardware, because I did. I designed Crafty for the future at the time, not for status quo. Some consider that a good approach. But apparently not everyone. Microsoft needs you guys...Don wrote:Ok, if we go along with your premise, then why not use the Rebel numbers? In your words, "only the speedup numbers matter." And the numbers are 100 to 1 for Rebel. And I'm actually using an IDENTICAL program, you are not.mhull wrote:It's not relevant to the question being asked. All you need to know is how much ELO to attribute to HW speedup. How something was or was not optimized to HW has nothing to do with the answer. Only the speedup numbers matter. Whether a particular project sped up 10x or 1000x is the only answer you need. Then figure how how much ELO that translates to. That gives you the HW ELO increase (for that chess project). Any increase above that in a newer version (of that project) would be a candidate for software ELO improvement. This seems to be the general idea.Don wrote:Thank you Miguel. You said it better than I could and this is really clear.michiguel wrote:That is why Crafty is a bad example. Bob has been saying that programs in the past were not optimized for current hardware. Well, Crafty _was_ being optimized for hardware of the future, in detriment to the current hardware of the time. He was saying this all the time in the 90's and there were countless threads about it. "64 bits will become standard" etc. etc.mhull wrote:It's still going to be crafty 1995 vs crafty 1995 (new hardware), just like your Rebel test. The only difference is that crafty 1995 has more speedup potential than Reble because it (1995 version) can also benefit from 64-bit. Rebel can't.Don wrote:Rebel vs Rebel - apples to apples.mhull wrote:I don't think it persuasive to introduce an orange in a comparison of apples. Crafty is apples to apples.Don wrote:Says you.bob wrote:I do not quite see the point for all the tangents. It would seem to me we have a pretty good idea of what/how to test.
Rebel is only 100 times faster on modern hardware. You add a zero to this and consider it a reasonable test.
Crafty 1995 vs Crafty 2010 - apples vs oranges.
Miguel
I was getting frustrated trying to make Bob understand this concept and it was just because I am not very good at explaining myself.
I mean, can the HW speedup for 1995 Crafty be determined or not? If it can, where is the problem?