Computer chess progress over say the last 20 years?

[bob]

Recently I have tried running Rodent II at 32768 nodes per second. It was enough to get a blitz score of 52% against good old Comet B68, an engine from pre-LMR times, running 10-11 times as fast. What is interesting, Comet was much better in endgame, but struggled in the middlegame.

I would bet some experimentation with tuning Rodent would improve those results. What one can get away with at 3M nodes per second is quite a bit different from what you can get away with 100x (or 1000x) slower...

Most engines are already being tuned at ultra-bullet time controls.

10s + 0.1s is not quite the same as 30K nodes per second...

Crafty is doing about 6M NPS for testing. at 10s + 0.1s, that turns into maybe about 10s / 30 + 0.1s per move, or almost 1/2 second. 3M nodes searched. A program searching 30K NPS would search only 15K nodes (or a little less)...

With 30K nodes per second you can get 3M nodes per move if you use 100 seconds per move.

I do not think that this type of time control was irrelevant at the time programs searched 30K nodes per second and the average time that you had in computer world championship is slower than 100 seconds per move.

The point is simple. Faster hardware allowed us to do things that did not work with much slower hardware. IE faster hardware allowed improvements in software that were not feasible with very slow hardware.

[syzygy]

The point is simple. Faster hardware allowed us to do things that did not work with much slower hardware. IE faster hardware allowed improvements in software that were not feasible with very slow hardware.

Faster hardware allowed us to tune chess engines at ultra-bullet time controls, which means they have now been perfectly tuned for playing at regular time control on hardware of 20 years ago.

But you have already left your original position, which was proved to be untenable:

What one can get away with at 3M nodes per second is quite a bit different from what you can get away with 100x (or 1000x) slower...

SF's search of today works just fine at regular time control on hardware that is 100x as slow, because regular time control on such hardware corresponds to the conditions under which SF is being tested and tuned.

[bob]

The point is simple. Faster hardware allowed us to do things that did not work with much slower hardware. IE faster hardware allowed improvements in software that were not feasible with very slow hardware.

Faster hardware allowed us to tune chess engines at ultra-bullet time controls, which means they have now been perfectly tuned for playing at regular time control on hardware of 20 years ago.

But you have already left your original position, which was proved to be untenable:

What one can get away with at 3M nodes per second is quite a bit different from what you can get away with 100x (or 1000x) slower...

SF's search of today works just fine at regular time control on hardware that is 100x as slow, because regular time control on such hardware corresponds to the conditions under which SF is being tested and tuned.

I haven't left my original position at all. Point still stands from my perspective. I can do things today I considered too expensive in 1995. I could do things in 1995 that I considered to be unbearably expensive in 1968.

[syzygy]

The point is simple. Faster hardware allowed us to do things that did not work with much slower hardware. IE faster hardware allowed improvements in software that were not feasible with very slow hardware.

Faster hardware allowed us to tune chess engines at ultra-bullet time controls, which means they have now been perfectly tuned for playing at regular time control on hardware of 20 years ago.

But you have already left your original position, which was proved to be untenable:

What one can get away with at 3M nodes per second is quite a bit different from what you can get away with 100x (or 1000x) slower...

SF's search of today works just fine at regular time control on hardware that is 100x as slow, because regular time control on such hardware corresponds to the conditions under which SF is being tested and tuned.

I haven't left my original position at all. Point still stands from my perspective. I can do things today I considered too expensive in 1995. I could do things in 1995 that I considered to be unbearably expensive in 1968.

But the undeniable mathematical reality is that what today tests fine at ultra-bullet would have worked, and still works, just as fine on hardware that is 100x slower at time controls that are 100x longer.

So again, it is rather unlikely that Pawel's Rodent could be tuned to do better at 30Knps, as Rodent most likely is better tuned for regular games at 30Knps than for regular games at 3Mnps or whatever it reaches on modern hardware. Because most likely it has already been tuned at time controls on modern hardware that correspond to regular time controls at those 30Knps.

[bob]

The point is simple. Faster hardware allowed us to do things that did not work with much slower hardware. IE faster hardware allowed improvements in software that were not feasible with very slow hardware.

Faster hardware allowed us to tune chess engines at ultra-bullet time controls, which means they have now been perfectly tuned for playing at regular time control on hardware of 20 years ago.

But you have already left your original position, which was proved to be untenable:

What one can get away with at 3M nodes per second is quite a bit different from what you can get away with 100x (or 1000x) slower...

SF's search of today works just fine at regular time control on hardware that is 100x as slow, because regular time control on such hardware corresponds to the conditions under which SF is being tested and tuned.

I haven't left my original position at all. Point still stands from my perspective. I can do things today I considered too expensive in 1995. I could do things in 1995 that I considered to be unbearably expensive in 1968.

But the undeniable mathematical reality is that what today tests fine at ultra-bullet would have worked, and still works, just as fine on hardware that is 100x slower at time controls that are 100x longer.

So again, it is rather unlikely that Pawel's Rodent could be tuned to do better at 30Knps, as Rodent most likely is better tuned for regular games at 30Knps than for regular games at 3Mnps or whatever it reaches on modern hardware. Because most likely it has already been tuned at time controls on modern hardware that correspond to regular time controls at those 30Knps.

Not necessarily. While they might do "just fine" who is testing to see what new ideas they can tone down or throw away at bullet? Nobody. Because playing bullet games is not the goal. For example, singular extensions. Not so good at bullet.

[bnemias]

I've been running SF on very slow hardware over on FICS for years. Currently, slowbox is running hardware roughly equal to a 90 Mhz pentium with Stockfish 7. Feel free to test-- the bot gladly accepts unrated challenges. It gets about 12 knps from the start pos and benches around 15 knps. The memory footprint of the router it runs on is also similar to the hardware of the 90 Mhz pentium era.

In addition, I have SF6 running on hardware about 1/3 as fast (an actiontec modem with 16 meg of ram). It's been dormant since I upgraded slowbox to SF7, but all I have to do is turn it on. I wasn't able to get SF7 running on this hardware and gave up after a few days trying.

The slowest actual desktop I still retain is a Pentium II-350 with 448 Megs of RAM. I'd be happy to set that up for a week or so running a guest acct if anyone wants to play around that way. It has Win95, Win98, or Win2000 installed. Win2000 would be the fastest choice since it is a real 32 bit kernel, but 9x is closer to what people actually ran back then. I can build SF7 for it, I think. But I have no idea if any modern commercial engines would run on it because modern Visual Studio has stubs that link to versions of kernel32 that win2000 doesn't have. Ofc, it lacks sse2 and sse though it does support mmx instructions.

[bnemias]

I also ran some personal tests against mchess & mchess pro against the actiontec running stockfish 6. Mchess was run in a virtual machine running winxp 32 bit achieving about 300,000 nps at the start pos and the actiontec running Stockfish 6 was getting about 4,200 nps.

14 games played with a 40/2 time control. mchess scored 1.5 points despite a 71x speed advantage.

Granted the nps between the engines probably isn't 1:1. Of note was that I got far far better results running mchess in an XP virtual machine than directly using dosbox. In dosbox, the most it ever got was about 27,000 nps.

[fierz]

Dear Eric,

I don't think it's necessary to actually go back to old hardware to test old engines against new ones. It should be sufficient to run both engines on standard hardware at fast speed, that should simulate old hardware pretty well.

I'm right now running a match of stockfish 7 against ruffian 1.0.5 - one of the earliest top (or rather near-top) programs I could get my hands on. It's 14 years old and in the CCRL 40/40 list, stockfish 7 64 bit is on 3246, ruffian on 2608, a 638 point rating difference. SF 2.2.2 is on the list with both 32- and 64 bit versions, and has a 26 point rating difference there, so I could estimate a 612 point rating difference between SF7 and ruffian 1.0.5 according to the CCRL 40/40 list.

I'm running a match on a pentium M 1.5GHz with 1s/move, which should be around 100x slower than what CCRL 40/40 is. If there is anything to the claim that you can now do things that you couldn't in the past due to low NPS or low total nodes searched per move, the rating difference should decrease at such fast matches.

Ruffian is getting a bit over 1MN/s on this hardware, Stockfish is slower, about half the speed. If Ruffian was running at 3min/move = ~200s, that corresponds to a machine which could do 5kN/s, and that must be really really long ago.

You can all try to make a prediction on the match

cheers
Martin

[Uri Blass]

The point is simple. Faster hardware allowed us to do things that did not work with much slower hardware. IE faster hardware allowed improvements in software that were not feasible with very slow hardware.

Faster hardware allowed us to tune chess engines at ultra-bullet time controls, which means they have now been perfectly tuned for playing at regular time control on hardware of 20 years ago.

But you have already left your original position, which was proved to be untenable:

What one can get away with at 3M nodes per second is quite a bit different from what you can get away with 100x (or 1000x) slower...

SF's search of today works just fine at regular time control on hardware that is 100x as slow, because regular time control on such hardware corresponds to the conditions under which SF is being tested and tuned.

I haven't left my original position at all. Point still stands from my perspective. I can do things today I considered too expensive in 1995. I could do things in 1995 that I considered to be unbearably expensive in 1968.

But the undeniable mathematical reality is that what today tests fine at ultra-bullet would have worked, and still works, just as fine on hardware that is 100x slower at time controls that are 100x longer.

So again, it is rather unlikely that Pawel's Rodent could be tuned to do better at 30Knps, as Rodent most likely is better tuned for regular games at 30Knps than for regular games at 3Mnps or whatever it reaches on modern hardware. Because most likely it has already been tuned at time controls on modern hardware that correspond to regular time controls at those 30Knps.

Not necessarily. While they might do "just fine" who is testing to see what new ideas they can tone down or throw away at bullet? Nobody. Because playing bullet games is not the goal. For example, singular extensions. Not so good at bullet.

singular extensions are good enough for bullet with the hardware of today(at least for stockfish) and they only accept changes that work at bullet.

[jdart]

It is difficult to build apps that run under Windows versions before Windows XP because the current dev tools don't support them. If you want to try out current generation programs you might need an old compiler to go with your old OS.

--Jon