New Methods On Old Hardware?

[mhull]

Could a new program with the latest search enhancements and evaluation terms compete with say an old Spracklen program, both running on the old hardware, say a 68000 or 68030? A memory upgrade might be in order for cache and data but would it be too slow to compete with the more primitive but compact program? Has this been done already?

[bob]

Could a new program with the latest search enhancements and evaluation terms compete with say an old Spracklen program, both running on the old hardware, say a 68000 or 68030? A memory upgrade might be in order for cache and data but would it be too slow to compete with the more primitive but compact program? Has this been done already?

Probably not. The problem is limited processor power. Since I started this in 1968, it has always been a balance of what I want to do, and what I can afford to do. The mobility calculations done today (not to mention the 64 bit stuff in general) would perform terribly on the old generations of processors. If you slow down too far, the searches are so shallow that the cute eval ideas don't help and you get swamped in tactics.

the programs have really grown up synergistically with the hardware advances. I used to run Cray Blitz on a machine that would do 10 ply searches. On a Vax, it could do 4-5 max. And it played terribly tactically although not bad positionally. Earlier versions (prior to the cray) had lacked lots of the knowledge and search enhancements in Cray Blitz, because there was simply not enough computing power to support them.

Old programs on modern hardware would certainly scream, speed wise, but they would certainly be inferior to today's programs. But probably not as inferior as todays programs on old hardware.

[mhull]

Could a new program with the latest search enhancements and evaluation terms compete with say an old Spracklen program, both running on the old hardware, say a 68000 or 68030? A memory upgrade might be in order for cache and data but would it be too slow to compete with the more primitive but compact program? Has this been done already?

Probably not. The problem is limited processor power. Since I started this in 1968, it has always been a balance of what I want to do, and what I can afford to do. The mobility calculations done today (not to mention the 64 bit stuff in general) would perform terribly on the old generations of processors. If you slow down too far, the searches are so shallow that the cute eval ideas don't help and you get swamped in tactics.

the programs have really grown up synergistically with the hardware advances. I used to run Cray Blitz on a machine that would do 10 ply searches. On a Vax, it could do 4-5 max. And it played terribly tactically although not bad positionally. Earlier versions (prior to the cray) had lacked lots of the knowledge and search enhancements in Cray Blitz, because there was simply not enough computing power to support them.

Old programs on modern hardware would certainly scream, speed wise, but they would certainly be inferior to today's programs. But probably not as inferior as todays programs on old hardware.

Can you estimate how slow in Mhz (on say i686) Crafty would have to run (uni-processor) with best play to top-out at say 2400 Elo (USCF)? And how much memory could be beneficially utilized at that speed?

[mhull]

I guess my real question is: If someone wanted to build a custom chess playing machine (table or table-top) and wanted the strength to be no greater than master or senior master level, what would be the target hardware? It would be something cheap and off-the-shelf but effective for the purpose.

[bob]

Could a new program with the latest search enhancements and evaluation terms compete with say an old Spracklen program, both running on the old hardware, say a 68000 or 68030? A memory upgrade might be in order for cache and data but would it be too slow to compete with the more primitive but compact program? Has this been done already?

Probably not. The problem is limited processor power. Since I started this in 1968, it has always been a balance of what I want to do, and what I can afford to do. The mobility calculations done today (not to mention the 64 bit stuff in general) would perform terribly on the old generations of processors. If you slow down too far, the searches are so shallow that the cute eval ideas don't help and you get swamped in tactics.

the programs have really grown up synergistically with the hardware advances. I used to run Cray Blitz on a machine that would do 10 ply searches. On a Vax, it could do 4-5 max. And it played terribly tactically although not bad positionally. Earlier versions (prior to the cray) had lacked lots of the knowledge and search enhancements in Cray Blitz, because there was simply not enough computing power to support them.

Old programs on modern hardware would certainly scream, speed wise, but they would certainly be inferior to today's programs. But probably not as inferior as todays programs on old hardware.

Can you estimate how slow in Mhz (on say i686) Crafty would have to run (uni-processor) with best play to top-out at say 2400 Elo (USCF)? And how much memory could be beneficially utilized at that speed?

I can't right off the top since I have no valid USCF number for Crafty. If you divide the speed by 16, you would drop off about 400 Elo if you figure about 100 elo per doubling of speed. Rather than today's 24M + nps, you'd want more like 1M-1.5M. And something tells me that would be stronger than the target based on seeing tons of GMs fall to Crafty searching at 1M nps and less a few years ago.

Pretty difficult question, in fact. For crafty, my "optimal" memory estimate is usually

NPS * time per move in seconds * 16 bytes per hash entry / 10 (about 90% of nodes are in q-search which don't get hashed in Crafty.

at 100 secs / move on good hardware, that is 25,000,000 * 100 / 10 for the number of entries, or 250,000,000 entries. at 16 bytes per entry something in the range of 4gb for hash would be about right. If you cut the hardware speed 10x, then 400mb would work. at 10 secs per move on that slower hardware, 40mb would be fine.

[Uri Blass]

Could a new program with the latest search enhancements and evaluation terms compete with say an old Spracklen program, both running on the old hardware, say a 68000 or 68030? A memory upgrade might be in order for cache and data but would it be too slow to compete with the more primitive but compact program? Has this been done already?

Probably not. The problem is limited processor power. Since I started this in 1968, it has always been a balance of what I want to do, and what I can afford to do. The mobility calculations done today (not to mention the 64 bit stuff in general) would perform terribly on the old generations of processors. If you slow down too far, the searches are so shallow that the cute eval ideas don't help and you get swamped in tactics.

the programs have really grown up synergistically with the hardware advances. I used to run Cray Blitz on a machine that would do 10 ply searches. On a Vax, it could do 4-5 max. And it played terribly tactically although not bad positionally. Earlier versions (prior to the cray) had lacked lots of the knowledge and search enhancements in Cray Blitz, because there was simply not enough computing power to support them.

Old programs on modern hardware would certainly scream, speed wise, but they would certainly be inferior to today's programs. But probably not as inferior as todays programs on old hardware.

I disagree with the opinion that new programs are not going to beat old programs on old hardware because there are many improvements that are not dependent on hardware(the only case that they cannot beat old programs is if they cannot run on old hardware but if the problem is only that they are not optimized to run fast on old hardware then this problem is not significant enough)

I also do not agree that old programs on modern hardware are better than today programs on old hardware.

All the tests by testing chess organizations show huge difference between old programs and new programs on the same hardware.

For example from the CEGT 40/4 list

8 Rybka 4.0 x64 1CPU 3181 11 11 2670 68.4% 3047 38.2%
534 Fritz 5.32 2629 13 13 1786 49.7% 2631 30.2%

I believe that the Spracklen program is clearly weaker than Fritz5.32 so
you are going to find a clearly higher difference.

The difference with old hardware may be smaller but it certainly going to be a big difference even if the programs are not optimized for the old hardware and based on my knowledge old rybka and old fruit on old hardware performed well inspite of not being optimized for that hardware.

From the ssdf list:

Resurrection Rybka 2.2 StrongARM 203 MH 2484 43 -43 260 51%
Resurrection Fruit '05 StrongARM 203 MH 2395 67 -63 120 60% 2320

For comparison Genius5 on better hardare got the same rating as fruit(P200 MMX is better hardware than StrongArm 203 MH) and Genius is clearly stronger than the old Spracklen program that the poster talk about

Genius 5.0 DOS 46MB P200 MMX 2395 20 -20 1197 49% 2400

[bob]

I guess my real question is: If someone wanted to build a custom chess playing machine (table or table-top) and wanted the strength to be no greater than master or senior master level, what would be the target hardware? It would be something cheap and off-the-shelf but effective for the purpose.

I'd probably go for an atom today. Low enough power requirements, still fast. Certainly the price is right.

[bob]

Could a new program with the latest search enhancements and evaluation terms compete with say an old Spracklen program, both running on the old hardware, say a 68000 or 68030? A memory upgrade might be in order for cache and data but would it be too slow to compete with the more primitive but compact program? Has this been done already?

Probably not. The problem is limited processor power. Since I started this in 1968, it has always been a balance of what I want to do, and what I can afford to do. The mobility calculations done today (not to mention the 64 bit stuff in general) would perform terribly on the old generations of processors. If you slow down too far, the searches are so shallow that the cute eval ideas don't help and you get swamped in tactics.

the programs have really grown up synergistically with the hardware advances. I used to run Cray Blitz on a machine that would do 10 ply searches. On a Vax, it could do 4-5 max. And it played terribly tactically although not bad positionally. Earlier versions (prior to the cray) had lacked lots of the knowledge and search enhancements in Cray Blitz, because there was simply not enough computing power to support them.

Old programs on modern hardware would certainly scream, speed wise, but they would certainly be inferior to today's programs. But probably not as inferior as todays programs on old hardware.

I disagree with the opinion that new programs are not going to beat old programs on old hardware because there are many improvements that are not dependent on hardware(the only case that they cannot beat old programs is if they cannot run on old hardware but if the problem is only that they are not optimized to run fast on old hardware then this problem is not significant enough)

I also do not agree that old programs on modern hardware are better than today programs on old hardware.

All the tests by testing chess organizations show huge difference between old programs and new programs on the same hardware.

For example from the CEGT 40/4 list

8 Rybka 4.0 x64 1CPU 3181 11 11 2670 68.4% 3047 38.2%
534 Fritz 5.32 2629 13 13 1786 49.7% 2631 30.2%

I believe that the Spracklen program is clearly weaker than Fritz5.32 so
you are going to find a clearly higher difference.

The difference with old hardware may be smaller but it certainly going to be a big difference even if the programs are not optimized for the old hardware and based on my knowledge old rybka and old fruit on old hardware performed well inspite of not being optimized for that hardware.

From the ssdf list:

Resurrection Rybka 2.2 StrongARM 203 MH 2484 43 -43 260 51%
Resurrection Fruit '05 StrongARM 203 MH 2395 67 -63 120 60% 2320

For comparison Genius5 on better hardare got the same rating as fruit(P200 MMX is better hardware than StrongArm 203 MH) and Genius is clearly stronger than the old Spracklen program that the poster talk about

Genius 5.0 DOS 46MB P200 MMX 2395 20 -20 1197 49% 2400

You can believe what you want. I believe what I have _learned_. I actually worked on such hardware back in the 80's, where we developed for a Cray, but tested on a Vax. Every improvement on the Cray usually hurt us on the Vax, because of the hardware differences (mainly speed but there are other issues).

Mobility (as done today) could be done for almost nothing on the Cray. It was _hugely_ expensive on PCs and such.

I know of no significant example in chess programming that is not somehow related to hardware performance. On slow hardware depth is critical. On fast hardware depth is less of an issue and knowledge or better search becomes more important. On old hardware null-move R=2 is horrible. At today's depths it works well. The list goes on and on.

The above results mean nothing whatsoever. We are talking about two options.

(1) taking an old program and putting it on modern hardware. It will speed up significantly and play much stronger.

(2) taking a new program and putting it on old hardware. It will slow down drastically and play much weaker.

I happen to believe that (2) would end up a worse dropoff. Speeding up a program has a finite effect on it, and no matter what search and evaluation assumptions you have made, going faster always helps. But when going the other way, you suddenly notice "wish I was not doing the mobility stuff, it is killing my search depth." or "the aggressive forward pruning works well at depth 20, but it is terrible at depth 5..." and such.

Very few have had to deal with both ends of the performance spectrum at one time. In the case of Cray Blitz, where we ran on a Cray but developed on a Vax, we had issues no one had thought about before we reported them.

[Uri Blass]

about your examples:

1)I find it hard to believe that mobility does not help on old hardware

Do you find in your tests that mobility is counter productive at very fast time control(like 1 seconds for 40 moves) or maybe your claim is that the price in speed for mobility is significantly bigger on old hardware?

2)I find it hard to believe that aggresive pruning does not help with old hardware.

Do you find in your test that your LMR does not help at very fast time control?

The claim that aggresive pruning is not good for old hardware does not seem logical.
It may be the opposite and
I suspect that aggresive pruning can be more productive for very old
hardware in terms of elo points.

The only reason that you do pruning is that there are positions when you cannot search deep enough to find the right move without pruning.

Based on common sense when you search deeper you have more cases when you can search deep enough without pruning so the advantage of pruning is smaller.

It may be interesting to know if you tested the advantage of LMR and null move pruning at super fast time control to know if the advantage is not bigger (in term of elo points)

Uri

[bob]

about your examples:

1)I find it hard to believe that mobility does not help on old hardware

Can't help much with what you believe or don't believe. All I can do is report on what I have discovered. You write a program that searches 5 plies deep. You add mobility and it drops to 4 plies. Good deal or not? Not quite the same question as today where a program might drop from 25 plies back to 24. Back in the early days, _every_ idea that had significant computational cost had to be examined carefully, because the depths back then were so shallow. Cray Blitz, in 1993-1994, running on PC hardware, could barely reach 100 nodes per second, even though we were breaking a million on Cray hardware. 100 nodes per second _really_ limits what you can do.

Do you find in your tests that mobility is counter productive at very fast time control(like 1 seconds for 40 moves) or maybe your claim is that the price in speed for mobility is significantly bigger on old hardware?

The old hardware is the main point. The old dedicated boxes had no cache, for example.

2)I find it hard to believe that aggresive pruning does not help with old hardware.

Do you find in your test that your LMR does not help at very fast time control?

Very fast time controls does not quite relate to old hardware. Hardware speeds have been pretty static for several years now. In 1994, running on a sun sparc that was far faster than any PC being made, Crafty was hitting around 5K nodes per second. Back then I found that even null-move R=2 was too dangerous (you can find comments in main.c about some of this.) As speeds improved, and along with that, depth, some things started working better.

One thing I have found is that the reduction/pruning stsuff I do does get better with longer time controls. A recent test at very fast time controls showed +40 for the changes made since 23.2 was released. I then ran a longer test that took about 2 days per run, and it showed over +50 improvement, a gain over over 10.

The claim that aggresive pruning is not good for old hardware does not seem logical.
It may be the opposite and
I suspect that aggresive pruning can be more productive for very old
hardware in terms of elo points.

The only reason that you do pruning is that there are positions when you cannot search deep enough to find the right move without pruning.

The point is that we are able to do aggressive pruning in the last 3-4-5 plies only. Going back further toward the root starts to kill playing skill. Going back to old hardware would disable that completely when early programs (on micros) were not even getting 4-5 plies full-width.

Based on common sense when you search deeper you have more cases when you can search deep enough without pruning so the advantage of pruning is smaller.

It may be interesting to know if you tested the advantage of LMR and null move pruning at super fast time control to know if the advantage is not bigger (in term of elo points)

Uri