Vladimir Xern wrote:The titled-player commentators are calling out computer engine programmers saying that they have taken too much credit in their creations' strength over humans when it's mostly been the inexorable progress of hardware speed.
That's simply false, but what do those guys know about chess...
I suspect that there is more right about their statement than wrong. That is, I believe that well beyond 50% of the improvements over the past 20 years has been hardware related. I REALLY want to take my old Cray Blitz source, circa 1989, and run it on today's hardware and test it on my cluster to see where it comes in. It had no reductions or forward pruning other than null-move R=1. Only work I need to do is add basic xboard protocol support as it was not xboard compatible. But it used the same "seaboard" command as Crafty so that is done, and it inputs and outputs SAN moves. All I really need is time/otim and it should probably work on my cluster.
I remember running Cray Blitz (fortran-only code of course) on a Pentium 133mhz box and hit about 100 nodes per second. I got this old version to compile cleanly a couple of years ago but don't remember the speed on more modern hardware. 64 bit hardware should be even better...
Be an interesting test.
I have to agree with you on this. The past 23 years the hardware speed increased 1000 fold, this accounts for something like 700 ELO.
I still remember running on one of the 'super computers' from our national Dutch computer center 'Sara' in 1991 where my program did about 15knps.
Nowadays it does 15mnps on a fast home computer.
I also believe that the increased hardware speed made some software tricks possible that were not feasible on slow hardware.
It is my estimation that the improvement due to software is something like 350 ELO which is not bad either.
Your estimation for the improvement due to software is clearly wrong.
Software improved clearly more than 350 elo and you do not get results of 99:1 by 350 elo improvement espacially when the old programs get more time on the same hardware in ponder off games.
We also talk about the last 20 years and not about the last 23 years.
I believe that there were some years when most of the improvement were due to hardware but it stopped in 2004 when fabien released fruit and if you look at the progress in software in the last 10 years then you will find that it is clearly bigger than the improvement in hardware in the last 10 years.
It is possible that the last 10 years the improvement in software is bigger then the one in hardware but I doubt it.
Anyway the total progress made in software the last 23 years is about 350 ELO so for the last 10 years it will always be equal or less.
And it has not only to do with speed but also with the memory size and the word size of the hardware.
Most of the old programs are 8 or 16 bit, what do you think will happen when you e.g. modify Stockfish to run on such an architecture, it will lose much more playing strength than you would expect.
You are clearly wrong. Here is the thread "10 years of Computer Chess" http://talkchess.com/forum/viewtopic.ph ... =0&t=45902
The difference in software is at least 600 Elo points in 10 years. In typical hardware it's 4 fold due to number of cores and 3 fold due to speed per core, so a total of 12, therefore 3.5 doublings for a total of 3.5*70 ~ 250 Elo points at typical time control of hardware improvement (not even counting sub-linear parallelization)..
So, in the last 10 years, it is about 600 Elo points from software and 250 Elo points from hardware. Bob, in his typical fashion, probably refers to his Crafty or who knows what, to make bold and systematically wrong statements.
I don't understand where your 600 Elo software increase is based on.
Ten years ago the ratings of the best programs were around 2800.
According to your calculations they should be now at 2800+250+600= 3650 Elo. I agree on the 250 Elo for hardware though.
Look at the equal hardware lists, for example CCRL 40/4' http://www.computerchess.org.uk/ccrl/40 ... t_all.html
The difference between Houdini 3 and Fritz 8, a 9 years span, is about 550 Elo points. Fritz 8 of 2004 AFAIK was even stronger than Shredder 7.04 of year 2003 (10 years span) by 50 Elo points, so a total of at least 600 Elo points from Shredder 7.04 to Houdini 3 on EQUAL hardware. And this is according to BayseElo, which compresses the ratings by ~10%, so it might be that software improvement in the last 10 years is close to 700 logistic Elo points. Say 600 at LEAST.
Joost Buijs wrote:I have to agree with you on this. The past 23 years the hardware speed increased 1000 fold, this accounts for something like 700 ELO.
I still remember running on one of the 'super computers' from our national Dutch computer center 'Sara' in 1991 where my program did about 15knps.
Nowadays it does 15mnps on a fast home computer.
I also believe that the increased hardware speed made some software tricks possible that were not feasible on slow hardware.
It is my estimation that the improvement due to software is something like 350 ELO which is not bad either.
Hardware improved nowhere near 1000 fold. There are threads here where this was discussed in detail, and different ppl did a lot of tests.
Basically, in last 20 years hardware sped up 150-200x which is around 500Elo, while in software you got 600-800Elo (depending on which same hardware you compare old and new software - on old one, new one, or something intermediate).
Software improvements are still more important than hardware advance.
Last edited by Milos on Tue Aug 26, 2014 11:57 am, edited 2 times in total.
I think you can't compare it like this. The modern programs are clearly taking much more advantage of the modern hardware than you would expect if you only compare it speed wise.
Anyway I think 600 to 700 Elo is bollocks. And everybody is entitled to his own opinion.
Joost Buijs wrote:I think you can't compare it like this. The modern programs are clearly taking much more advantage of the modern hardware than you would expect if you only compare it speed wise.
Anyway I think 600 to 700 Elo is bollocks. And everybody is entitled to his own opinion.
You can compare software only improvements in 3 ways:
1) You take old hardware as a reference point, so you compile
(and optimize) new software for 20 years old hardware and test it versus old software
2) You take some intermediate hardware (i.e. 10 years old) and compile and optimize both new and old software.
3) You take newest hardware as a reference point, and compile and optimize old software for it and test versus new software.
The result will ofc depend on the comparison method, but in first case you'd still get at least 500Elo difference, while in last case you might get even 800Elo or more difference.
Joost Buijs wrote:I think you can't compare it like this. The modern programs are clearly taking much more advantage of the modern hardware than you would expect if you only compare it speed wise.
Anyway I think 600 to 700 Elo is bollocks. And everybody is entitled to his own opinion.
Sure, even to completely unfounded opinions. All lists, all year-by-year improvements show at least 600 Elo points SOFTWARE improvement in last 10 years, and 250 points hardware. During Rybka era, the improvement in software was above 60 Elo points per year, and SF improves faster than 60 Elo points per year purely software-wise at 40/4' TC. That modern programs are using modern hardware better, tell me, how much better? A factor of 1.5 in speed? That is about 40 Elo points at 40/4'. be it, still around 550-650 points purely SOFTWARE.
Last edited by Laskos on Tue Aug 26, 2014 12:01 pm, edited 1 time in total.
Joost Buijs wrote:I have to agree with you on this. The past 23 years the hardware speed increased 1000 fold, this accounts for something like 700 ELO.
I still remember running on one of the 'super computers' from our national Dutch computer center 'Sara' in 1991 where my program did about 15knps.
Nowadays it does 15mnps on a fast home computer.
I also believe that the increased hardware speed made some software tricks possible that were not feasible on slow hardware.
It is my estimation that the improvement due to software is something like 350 ELO which is not bad either.
Hardware improved nowhere near 1000 fold. There are threads here where this was discussed in detail, and different ppl did a lot of tests.
Basically, in last 20 years hardware sped up 150-200x which is around 500Elo, while in software you got 700-1000Elo (depending on which same hardware you compare old and new software - on old one, new one, or something intermediate).
Software improvements are still more important than hardware advance.
I,ve been in computer chess for 38 years and I have used and owned almost every hardware in existence, I remember very clearly that my program was running at 7 knps on a fast PC in 1991. Now it is running at 2000 times that speed. Maybe this is partly due to better compilers but 1000x is certainly possible.
Joost Buijs wrote:I have to agree with you on this. The past 23 years the hardware speed increased 1000 fold, this accounts for something like 700 ELO.
I still remember running on one of the 'super computers' from our national Dutch computer center 'Sara' in 1991 where my program did about 15knps.
Nowadays it does 15mnps on a fast home computer.
I also believe that the increased hardware speed made some software tricks possible that were not feasible on slow hardware.
It is my estimation that the improvement due to software is something like 350 ELO which is not bad either.
Hardware improved nowhere near 1000 fold. There are threads here where this was discussed in detail, and different ppl did a lot of tests.
Basically, in last 20 years hardware sped up 150-200x which is around 500Elo, while in software you got 700-1000Elo (depending on which same hardware you compare old and new software - on old one, new one, or something intermediate).
Software improvements are still more important than hardware advance.
I,ve been in computer chess for 38 years and I have used and owned almost every hardware in existence, I remember very clearly that my program was running at 7 knps on a fast PC in 1991. Now it is running at 2000 times that speed. Maybe this is partly due to better compilers but 1000x is certainly possible.
And you used the same eval function (same pawn hash, same bitboards, same asm code for bsf/bsr), same move generator in 1991. and today? Yea right
Joost Buijs wrote:I think you can't compare it like this. The modern programs are clearly taking much more advantage of the modern hardware than you would expect if you only compare it speed wise.
Anyway I think 600 to 700 Elo is bollocks. And everybody is entitled to his own opinion.
Sure, even to completely unfounded opinions. All lists, all year-by-year improvements show at least 600 Elo points SOFTWARE improvement in last 10 years, and 250 points hardware. During Rybka era, the improvement in software was above 60 Elo points per year, and SF improves faster than 60 Elo points per year purely software-wise at 40/4' TC. That modern programs are using modern hardware better, tell me, how much better? A factor of 1.5 in speed? That is about 40 Elo points at 40/4'. be it, still around 550-650 points purely SOFTWARE.
And I think that a lot of so called software improvement is basically better tuning of various parameters by trial and error.
You can of course argue if this is really software improvement.
Joost Buijs wrote:I have to agree with you on this. The past 23 years the hardware speed increased 1000 fold, this accounts for something like 700 ELO.
I still remember running on one of the 'super computers' from our national Dutch computer center 'Sara' in 1991 where my program did about 15knps.
Nowadays it does 15mnps on a fast home computer.
I also believe that the increased hardware speed made some software tricks possible that were not feasible on slow hardware.
It is my estimation that the improvement due to software is something like 350 ELO which is not bad either.
Hardware improved nowhere near 1000 fold. There are threads here where this was discussed in detail, and different ppl did a lot of tests.
Basically, in last 20 years hardware sped up 150-200x which is around 500Elo, while in software you got 700-1000Elo (depending on which same hardware you compare old and new software - on old one, new one, or something intermediate).
Software improvements are still more important than hardware advance.
I,ve been in computer chess for 38 years and I have used and owned almost every hardware in existence, I remember very clearly that my program was running at 7 knps on a fast PC in 1991. Now it is running at 2000 times that speed. Maybe this is partly due to better compilers but 1000x is certainly possible.
And you used the same eval function (same pawn hash, same bitboards, same asm code for bsf/bsr), same move generator in 1991. and today? Yea right
Of course not, nowadays you can take advantage of the much better hardware like I said.
Joost Buijs wrote:I think you can't compare it like this. The modern programs are clearly taking much more advantage of the modern hardware than you would expect if you only compare it speed wise.
Anyway I think 600 to 700 Elo is bollocks. And everybody is entitled to his own opinion.
Sure, even to completely unfounded opinions. All lists, all year-by-year improvements show at least 600 Elo points SOFTWARE improvement in last 10 years, and 250 points hardware. During Rybka era, the improvement in software was above 60 Elo points per year, and SF improves faster than 60 Elo points per year purely software-wise at 40/4' TC. That modern programs are using modern hardware better, tell me, how much better? A factor of 1.5 in speed? That is about 40 Elo points at 40/4'. be it, still around 550-650 points purely SOFTWARE.
And I think that a lot of so called software improvement is basically better tuning of various parameters by trial and error.
You can of course argue if this is really software improvement.
Sure, what Rybka started is to tune on many, many games, not some wild guesses tested in 40 games or some test-suites. It is still software improvement.
