If you run crafty on a 64 core system, and Rybka on one, the truth will be far different than what you imagine.Eizenhammer wrote:I dont see how all this guessing could change the fact that rybka is about as strong as Uri suggests. Everybody who has both programs can see it, i ran a match on equal hardware, old hardware, ok, and in 50 games crafty got a draw, or maybe two, nothing more: Now, by using long books and effective learning one can obfuscate this difference, of course, but it does hardly change the basic facts. I would really be surprised if Crafty could win a long match, using the Silver suite or similar neutral suites, on all its 64 cores against a decent 64Bit single version of rybka ...bob wrote:My point was a bit subtle for most. A cluster is _not_ the best platform to run on. I have run on machines with 64 cores and shared memory. That is _far_ faster than a cluster with 64 cores and message passing, even if using something like Infiniband. Testers don't have a "good book" for Crafty. Too many variables to take a rating list and use that to estimate actual Elo differences in a real tournament.BubbaTough wrote:Hehe, cannot argue with that. Trying to translate speed into ELO is a pretty funny business.Or not...
-Sam
sorry, but sometimes one has to say the truth, even if it hurts.
Peter
Crafty on Cray Blitz vs. Rybka3
Moderator: Ras
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Crafty on Cray Blitz vs. Rybka3
-
- Posts: 388
- Joined: Wed Mar 08, 2006 10:08 pm
Re: Crafty on Cray Blitz vs. Rybka3
I don't have data on Rybka3 vs Crafty22.1 with 1:30 H/W difference.BubbaTough wrote:Hehe, cannot argue with that. Trying to translate speed into ELO is a pretty funny business.Or not...
-Sam
What I do have now is Rybka2.2 (free version) vs Crafty22.1 with 1:5 H/W difference, and the result so far is that Rybka is way ahead. So, I'm speculating that Rybka3 (being much better than Rybka2.2) can afford to be still better than Crafty22.1 even with a 10:1 H/W difference (or 20:1, 30:1, if I can find the machine pair with that much gap to test on).
My test rig involves an old 1ghz P3M Vaio notebook, running Rybka2.2, vs a 2.4ghz q6600 HP Optiplex, running Crafty22.1. Crafty bench shows 0.45M nps on the Vaio, and 2.35M nps (2 threads) on the HP. That translates to more than 5 to 1 HW difference. Games are for the 20 Nunn positions, and time control is 0 10 2, with pondering enabled.
Tournament UI is Arena, running on a third machine. There is no chess UI running with Rybka or Crafty - just RemoteThinker.
After 8 games, it is 6-0-2, in favor or Rybka2.2. The test is still running...
-
- Posts: 10900
- Joined: Thu Mar 09, 2006 12:37 am
- Location: Tel-Aviv Israel
Re: Crafty on Cray Blitz vs. Rybka3
I am not surprised.CThinker wrote:I don't have data on Rybka3 vs Crafty22.1 with 1:30 H/W difference.BubbaTough wrote:Hehe, cannot argue with that. Trying to translate speed into ELO is a pretty funny business.Or not...
-Sam
What I do have now is Rybka2.2 (free version) vs Crafty22.1 with 1:5 H/W difference, and the result so far is that Rybka is way ahead. So, I'm speculating that Rybka3 (being much better than Rybka2.2) can afford to be still better than Crafty22.1 even with a 10:1 H/W difference (or 20:1, 30:1, if I can find the machine pair with that much gap to test on).
My test rig involves an old 1ghz P3M Vaio notebook, running Rybka2.2, vs a 2.4ghz q6600 HP Optiplex, running Crafty22.1. Crafty bench shows 0.45M nps on the Vaio, and 2.35M nps (2 threads) on the HP. That translates to more than 5 to 1 HW difference. Games are for the 20 Nunn positions, and time control is 0 10 2, with pondering enabled.
Tournament UI is Arena, running on a third machine. There is no chess UI running with Rybka or Crafty - just RemoteThinker.
After 8 games, it is 6-0-2, in favor or Rybka2.2. The test is still running...
My guess is that Crafty may need 100:1 hardware advantage to
make a match against rybka3 interesting when slower time control is going to help rybka.
The difference for rybka2.2 may be something like 20:1 or 25:1 because I guess that rybka3 is about 4-5 times better than free rybka.
Uri
-
- Posts: 388
- Joined: Wed Mar 08, 2006 10:08 pm
Re: Crafty on Cray Blitz vs. Rybka3
I stoped the test after 18 games (from 9 Nunn positions). Crafty crashed at game 11. At game 13, Rybka is up be a pawn, but it was a drawn endgame, so, I adjudicated it as such.Uri Blass wrote:I am not surprised.CThinker wrote:I don't have data on Rybka3 vs Crafty22.1 with 1:30 H/W difference.BubbaTough wrote:Hehe, cannot argue with that. Trying to translate speed into ELO is a pretty funny business.Or not...
-Sam
What I do have now is Rybka2.2 (free version) vs Crafty22.1 with 1:5 H/W difference, and the result so far is that Rybka is way ahead. So, I'm speculating that Rybka3 (being much better than Rybka2.2) can afford to be still better than Crafty22.1 even with a 10:1 H/W difference (or 20:1, 30:1, if I can find the machine pair with that much gap to test on).
My test rig involves an old 1ghz P3M Vaio notebook, running Rybka2.2, vs a 2.4ghz q6600 HP Optiplex, running Crafty22.1. Crafty bench shows 0.45M nps on the Vaio, and 2.35M nps (2 threads) on the HP. That translates to more than 5 to 1 HW difference. Games are for the 20 Nunn positions, and time control is 0 10 2, with pondering enabled.
Tournament UI is Arena, running on a third machine. There is no chess UI running with Rybka or Crafty - just RemoteThinker.
After 8 games, it is 6-0-2, in favor or Rybka2.2. The test is still running...
My guess is that Crafty may need 100:1 hardware advantage to
make a match against rybka3 interesting when slower time control is going to help rybka.
The difference for rybka2.2 may be something like 20:1 or 25:1 because I guess that rybka3 is about 4-5 times better than free rybka.
Uri
The result looks like this:
Code: Select all
Score 123456789012345678
----------------------------------------------
1: Rybka2.2 13.5 / 18 11=1=1111011=10110
2: Crafty22.1 4.5 / 18 00=0=0000100=01001
----------------------------------------------
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Crafty on Cray Blitz vs. Rybka3
What do you mean "crashed"???CThinker wrote:I stoped the test after 18 games (from 9 Nunn positions). Crafty crashed at game 11. At game 13, Rybka is up be a pawn, but it was a drawn endgame, so, I adjudicated it as such.Uri Blass wrote:I am not surprised.CThinker wrote:I don't have data on Rybka3 vs Crafty22.1 with 1:30 H/W difference.BubbaTough wrote:Hehe, cannot argue with that. Trying to translate speed into ELO is a pretty funny business.Or not...
-Sam
What I do have now is Rybka2.2 (free version) vs Crafty22.1 with 1:5 H/W difference, and the result so far is that Rybka is way ahead. So, I'm speculating that Rybka3 (being much better than Rybka2.2) can afford to be still better than Crafty22.1 even with a 10:1 H/W difference (or 20:1, 30:1, if I can find the machine pair with that much gap to test on).
My test rig involves an old 1ghz P3M Vaio notebook, running Rybka2.2, vs a 2.4ghz q6600 HP Optiplex, running Crafty22.1. Crafty bench shows 0.45M nps on the Vaio, and 2.35M nps (2 threads) on the HP. That translates to more than 5 to 1 HW difference. Games are for the 20 Nunn positions, and time control is 0 10 2, with pondering enabled.
Tournament UI is Arena, running on a third machine. There is no chess UI running with Rybka or Crafty - just RemoteThinker.
After 8 games, it is 6-0-2, in favor or Rybka2.2. The test is still running...
My guess is that Crafty may need 100:1 hardware advantage to
make a match against rybka3 interesting when slower time control is going to help rybka.
The difference for rybka2.2 may be something like 20:1 or 25:1 because I guess that rybka3 is about 4-5 times better than free rybka.
Uri
The result looks like this:If I find the time to dust off an even older notebook (500mhz) and get Crafty to use 4 threads withouth crashing, then we might see a 1:20 test.Code: Select all
Score 123456789012345678 ---------------------------------------------- 1: Rybka2.2 13.5 / 18 11=1=1111011=10110 2: Crafty22.1 4.5 / 18 00=0=0000100=01001 ----------------------------------------------
Crafty doesn't normally crash, _ever_. Last weekend I played over 1/2 million games on our cluster with no crashes whatsoever. Is this windows or linux???
-
- Posts: 388
- Joined: Wed Mar 08, 2006 10:08 pm
Re: Crafty on Cray Blitz vs. Rybka3
The Watson dialog came up with the usual options of "terminate" or "debug". This is on a Windows 2008 Server. The UI is Arena. I should have saved the crash dump.bob wrote: What do you mean "crashed"???
Crafty doesn't normally crash, _ever_. Last weekend I played over 1/2 million games on our cluster with no crashes whatsoever. Is this windows or linux???
The next time I try out Crafty, I will build it myself. That way, I can attach a debugger to it (with symbols and all) if it crashes.
Re: Crafty on Cray Blitz vs. Rybka3
I have also had a problem with Crafty. Crafty used to be stable for me, but 22.1 occasionally crashes when playing, and it cannot communicate properly with XBoard in analysis mode after I take back a move.bob wrote:What do you mean "crashed"???
Crafty doesn't normally crash, _ever_. Last weekend I played over 1/2 million games on our cluster with no crashes whatsoever. Is this windows or linux???
I have sent you and Tracy a detailed e-mail on Sept 27, but perhaps it got lost in a spam filter? If you want I can send it again.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Crafty on Cray Blitz vs. Rybka3
By all means. There is a known issue with smpnice=1, ponder=off, with mt set to something > 1. I think the new version will work just fine since I have gone back to posix threads since the development of the NPTL stuff for linux eliminated my old complaints...glorfindel wrote:I have also had a problem with Crafty. Crafty used to be stable for me, but 22.1 occasionally crashes when playing, and it cannot communicate properly with XBoard in analysis mode after I take back a move.bob wrote:What do you mean "crashed"???
Crafty doesn't normally crash, _ever_. Last weekend I played over 1/2 million games on our cluster with no crashes whatsoever. Is this windows or linux???
I have sent you and Tracy a detailed e-mail on Sept 27, but perhaps it got lost in a spam filter? If you want I can send it again.
Re: Crafty on Cray Blitz vs. Rybka3
I have resent the e-mail, although it was exactly the kind of problem you described. I didn't know it was a known issue.bob wrote:By all means. There is a known issue with smpnice=1, ponder=off, with mt set to something > 1. I think the new version will work just fine since I have gone back to posix threads since the development of the NPTL stuff for linux eliminated my old complaints...
If I set smpnice=0 the problem goes away, but this is not the ideal solution, because e.g. if I am analyzing a tablebase position on a quad, one core is almost idle (the one that looks up the tablebase I guess) but the other three are at 100% usage not doing anything useful.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Crafty on Cray Blitz vs. Rybka3
Someone actually fixed the problem (it was simple to fix) and released a patched version of 22.1 that did not do that. We should have 22.2 ready for release right after the ACCA event in November. It will play significantly better and also not have this problem at all...glorfindel wrote:I have resent the e-mail, although it was exactly the kind of problem you described. I didn't know it was a known issue.bob wrote:By all means. There is a known issue with smpnice=1, ponder=off, with mt set to something > 1. I think the new version will work just fine since I have gone back to posix threads since the development of the NPTL stuff for linux eliminated my old complaints...
If I set smpnice=0 the problem goes away, but this is not the ideal solution, because e.g. if I am analyzing a tablebase position on a quad, one core is almost idle (the one that looks up the tablebase I guess) but the other three are at 100% usage not doing anything useful.