New engine _Crab_ based on StockFish

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Look

Re: New engine _Crab_ based on StockFish

Post by Look »

Hi Marco Costalba

Regarding my testing, since I am clearly biased, you can not rely on my results. I hope to get independent testing results after someone experienced in compiling manages to compile and post it.
svchbe

Re: New engine _Crab_ based on StockFish

Post by svchbe »

Code: Select all

[Event "[19] stockfish-171-64-ja vs crab (10/06/2010)"]
[Date "2010.06.10"]
[Round "2"]
[White "crab"]
[Black "stockfish-171-64-ja"]
[Result "1/2-1/2"]
[ECO "C49"]
[TimeControl "1 min/game + 1 sec/move"]

 1. e4 {[%clk 0:01:00][%clko 0:01:00][%emt
0:00:00]}{[%eval 11,1]} 1... e5 {[%clk 0:01:00][%clko 0:01:00][%emt 0:00:00]}
{[%eval 11,1]} 2. Nf3 {[%clk 0:01:00][%clko 0:01:00][%emt 0:00:00]}{[%eval
11,1]} 2... Nc6 {[%clk 0:01:00][%clko 0:01:00][%emt 0:00:00]}{[%eval 11,1]} 3.
Nc3 {[%clk 0:01:02][%clko 0:01:00][%emt 0:00:00]}{[%eval 4,1]} 3... Nf6 {[%clk
0:01:02][%clko 0:01:02][%emt 0:00:00]}{[%eval 7,1]} 4. Bb5 {[%clk
0:01:03][%clko 0:01:02][%emt 0:00:00]}{[%eval 7,1]} 4... Bb4 {[%clk
0:01:03][%clko 0:01:03][%emt 0:00:00]}{[%eval 7,1]} 5. O-O {[%clk
0:01:04][%clko 0:01:03][%emt 0:00:00]}{[%eval 7,1]} 5... O-O {[%clk
0:01:04][%clko 0:01:04][%emt 0:00:00]}{[%eval 7,1]} 6. d3 {[%clk 0:01:05][%clko
0:01:04][%emt 0:00:00]}{[%eval 4,1]} 6... d6 {[%clk 0:01:05][%clko
0:01:05][%emt 0:00:00]}{[%eval 22,1]} 7. Bg5 {[%clk 0:01:06][%clko
0:01:05][%emt 0:00:00]}{[%eval 22,1]} 7... Bxc3 {[%clk 0:01:06][%clko
0:01:06][%emt 0:00:00]}{[%eval 26,1]}{White out of book} 8. bxc3 {[%clk
0:00:57][%clko 0:01:00][%emt 0:00:02]}{[%eval 12,16]}{Black out of book} 8...
Qe7 {[%clk 0:01:00][%clko 0:00:57][%emt 0:00:00]}{[%eval 0]} 9. Rb1 {[%clk
0:00:55][%clko 0:01:00][%emt 0:00:04]}{[%eval 44,17]} 9... h6 {[%clk
0:01:00][%clko 0:00:55][%emt 0:00:00]}{[%eval 0]} 10. Bxf6 {[%clk
0:00:51][%clko 0:01:00][%emt 0:00:05]}{[%eval 12,18]} 10... Qxf6 {[%clk
0:00:59][%clko 0:00:51][%emt 0:00:03]}{[%eval 28,17]} 11. Bxc6 {[%clk
0:00:50][%clko 0:00:59][%emt 0:00:03]}{[%eval 16,18]} 11... bxc6 {[%clk
0:01:00][%clko 0:00:50][%emt 0:00:00]}{[%eval 24,15]} 12. Qc1 {[%clk
0:00:47][%clko 0:01:00][%emt 0:00:03]}{[%eval 20,19]} 12... c5 {[%clk
0:00:57][%clko 0:00:47][%emt 0:00:04]}{[%eval 16,18]} 13. Qe3 {[%clk
0:00:45][%clko 0:00:57][%emt 0:00:03]}{[%eval 8,17]} 13... Be6 {[%clk
0:00:54][%clko 0:00:45][%emt 0:00:04]}{[%eval -8,21]} 14. Rb7 {[%clk
0:00:43][%clko 0:00:54][%emt 0:00:02]}{[%eval 0,20]} 14... Rfb8 {[%clk
0:00:51][%clko 0:00:43][%emt 0:00:03]}{[%eval 0,22]} 15. Rfb1 {[%clk
0:00:42][%clko 0:00:51][%emt 0:00:02]}{[%eval 0,20]} 15... Qd8 {[%clk
0:00:49][%clko 0:00:42][%emt 0:00:03]}{[%eval -8,21]} 16. c4 {[%clk
0:00:41][%clko 0:00:49][%emt 0:00:02]}{[%eval 0,20]} 16... Bd7 {[%clk
0:00:47][%clko 0:00:41][%emt 0:00:03]}{[%eval -8,19]} 17. h3 {[%clk
0:00:38][%clko 0:00:47][%emt 0:00:03]}{[%eval -8,19]} 17... Rxb7 {[%clk
0:00:45][%clko 0:00:38][%emt 0:00:03]}{[%eval -12,19]} 18. Rxb7 {[%clk
0:00:39][%clko 0:00:45][%emt 0:00:00]}{[%eval -4,17]} 18... Rb8 {[%clk
0:00:40][%clko 0:00:39][%emt 0:00:06]}{[%eval -8,21]} 19. Rb3 {[%clk
0:00:37][%clko 0:00:40][%emt 0:00:03]}{[%eval -8,18]} 19... Ba4 {[%clk
0:00:39][%clko 0:00:37][%emt 0:00:02]}{[%eval -4,19]} 20. Rxb8 {[%clk
0:00:36][%clko 0:00:39][%emt 0:00:02]}{[%eval -8,17]} 20... Qxb8 {[%clk
0:00:40][%clko 0:00:36][%emt 0:00:00]}{[%eval -4,14]} 21. Qc1 {[%clk
0:00:34][%clko 0:00:40][%emt 0:00:03]}{[%eval -8,20]} 21... Qb4 {[%clk
0:00:39][%clko 0:00:34][%emt 0:00:02]}{[%eval 0,20]} 22. c3 {[%clk
0:00:33][%clko 0:00:39][%emt 0:00:02]}{[%eval 0,20]} 22... Qb6 {[%clk
0:00:36][%clko 0:00:33][%emt 0:00:04]}{[%eval 0,21]} 23. g4 {[%clk
0:00:32][%clko 0:00:36][%emt 0:00:02]}{[%eval 0,21]} 23... Bd7 {[%clk
0:00:34][%clko 0:00:32][%emt 0:00:02]}{[%eval 0,20]} 24. Kg2 {[%clk
0:00:31][%clko 0:00:34][%emt 0:00:03]}{[%eval 0,20]} 24... Kh7 {[%clk
0:00:33][%clko 0:00:31][%emt 0:00:02]}{[%eval 0,18]} 25. Kg3 {[%clk
0:00:27][%clko 0:00:33][%emt 0:00:05]}{[%eval 16,18]} 25... Qa6 {[%clk
0:00:32][%clko 0:00:27][%emt 0:00:02]}{[%eval 0,20]} 26. Qd2 {[%clk
0:00:26][%clko 0:00:32][%emt 0:00:02]}{[%eval 0,18]} 26... Qb6 {[%clk
0:00:31][%clko 0:00:26][%emt 0:00:02]}{[%eval 0,21]} 27. Ne1 {[%clk
0:00:24][%clko 0:00:31][%emt 0:00:02]}{[%eval 0,19]} 27... Qb1 {[%clk
0:00:28][%clko 0:00:24][%emt 0:00:02]}{[%eval 0,20]} 28. h4 {[%clk
0:00:24][%clko 0:00:28][%emt 0:00:02]}{[%eval 0,20]} 28... Kg6 {[%clk
0:00:28][%clko 0:00:24][%emt 0:00:02]}{[%eval -8,19]} 29. f4 {[%clk
0:00:23][%clko 0:00:28][%emt 0:00:01]}{[%eval 0,20]} 29... exf4+ {[%clk
0:00:27][%clko 0:00:23][%emt 0:00:02]}{[%eval -8,21]} 30. Kxf4 {[%clk
0:00:21][%clko 0:00:27][%emt 0:00:02]}{[%eval 0,24]} 30... Kf6 {[%clk
0:00:25][%clko 0:00:21][%emt 0:00:02]}{[%eval -8,21]} 31. Kg3 {[%clk
0:00:21][%clko 0:00:25][%emt 0:00:02]}{[%eval 0,19]} 31... Ke7 {[%clk
0:00:25][%clko 0:00:21][%emt 0:00:02]}{[%eval -8,20]} 32. Ng2 {[%clk
0:00:20][%clko 0:00:25][%emt 0:00:02]}{[%eval 0,20]} 32... c6 {[%clk
0:00:23][%clko 0:00:20][%emt 0:00:02]}{[%eval -16,19]} 33. h5 {[%clk
0:00:19][%clko 0:00:23][%emt 0:00:02]}{[%eval -10,17]} 33... Qf1 {[%clk
0:00:23][%clko 0:00:19][%emt 0:00:02]}{[%eval -28,18]} 34. Ne3 {[%clk
0:00:18][%clko 0:00:23][%emt 0:00:02]}{[%eval -8,18]} 34... Qg1+ {[%clk
0:00:21][%clko 0:00:18][%emt 0:00:01]}{[%eval -8,18]} 35. Ng2 {[%clk
0:00:18][%clko 0:00:21][%emt 0:00:01]}{[%eval -12,19]} 35... Be6 {[%clk
0:00:21][%clko 0:00:18][%emt 0:00:02]}{[%eval -16,21]} 36. Qe1 {[%clk
0:00:17][%clko 0:00:21][%emt 0:00:01]}{[%eval 0,19]} 36... Qxe1+ {[%clk
0:00:21][%clko 0:00:17][%emt 0:00:00]}{[%eval 0,14]} 37. Nxe1 {[%clk
0:00:18][%clko 0:00:21][%emt 0:00:00]}{[%eval 0,17]} 37... g6 {[%clk
0:00:21][%clko 0:00:18][%emt 0:00:01]}{[%eval -8,21]} 38. Nf3 {[%clk
0:00:18][%clko 0:00:21][%emt 0:00:01]}{[%eval -20,21]} 38... g5 {[%clk
0:00:21][%clko 0:00:18][%emt 0:00:02]}{[%eval 0,22]} 39. Nd2 {[%clk
0:00:17][%clko 0:00:21][%emt 0:00:02]}{[%eval -12,20]} 39... d5 {[%clk
0:00:21][%clko 0:00:17][%emt 0:00:02]}{[%eval 0,27]} 40. Kf3 {[%clk
0:00:17][%clko 0:00:21][%emt 0:00:01]}{[%eval -24,20]} 40... Kd6 {[%clk
0:00:21][%clko 0:00:17][%emt 0:00:01]}{[%eval 0,23]} 41. Kg3 {[%clk
0:00:16][%clko 0:00:21][%emt 0:00:01]}{[%eval -20,19]} 41... dxc4 {[%clk
0:00:20][%clko 0:00:16][%emt 0:00:01]}{[%eval 0,26]} 42. Nxc4+ {[%clk
0:00:16][%clko 0:00:20][%emt 0:00:01]}{[%eval 0,31]} 42... Bxc4 {[%clk
0:00:20][%clko 0:00:16][%emt 0:00:02]}{[%eval 0,35]} 43. dxc4 {[%clk
0:00:16][%clko 0:00:20][%emt 0:00:01]}{[%eval 0,33]} 43... Ke5 {[%clk
0:00:19][%clko 0:00:16][%emt 0:00:02]}{[%eval 0,37]} 44. Kf3 {[%clk
0:00:15][%clko 0:00:19][%emt 0:00:02]}{[%eval 0,38]} 44... a6 {[%clk
0:00:18][%clko 0:00:15][%emt 0:00:01]}{[%eval 0,39]} 45. a3 {[%clk
0:00:15][%clko 0:00:18][%emt 0:00:02]}{[%eval 0,35]} 45... a5 {[%clk
0:00:17][%clko 0:00:15][%emt 0:00:02]}{[%eval 0,39]} 46. a4 {[%clk
0:00:14][%clko 0:00:17][%emt 0:00:01]}{[%eval 0,39]} 46... f6 {[%clk
0:00:17][%clko 0:00:14][%emt 0:00:02]}{[%eval 0,44]} 47. Ke3 {[%clk
0:00:14][%clko 0:00:17][%emt 0:00:01]}{[%eval 0,39]} 47... Ke6 {[%clk
0:00:16][%clko 0:00:14][%emt 0:00:02]}{[%eval 0,46]} 48. Kd3 {[%clk
0:00:13][%clko 0:00:16][%emt 0:00:02]}{[%eval 0,42]} 48... Kd6 {[%clk
0:00:16][%clko 0:00:13][%emt 0:00:01]}{[%eval 0,47]} 49. Ke2 {[%clk
0:00:13][%clko 0:00:16][%emt 0:00:02]}{[%eval 0,40]} 49... Ke6 {[%clk
0:00:15][%clko 0:00:13][%emt 0:00:02]}{[%eval 0,49]} 50. Kd3 {[%clk
0:00:12][%clko 0:00:15][%emt 0:00:01]}{[%eval 0,44]} 50... Ke5 {[%clk
0:00:14][%clko 0:00:12][%emt 0:00:01]}{[%eval 0,48]} 51. Ke3 {[%clk
0:00:12][%clko 0:00:14][%emt 0:00:01]}{[%eval 0,47]} 51... Ke6 {[%clk
0:00:13][%clko 0:00:12][%emt 0:00:02]}{[%eval 0,50]} 52. Kd3 {[%clk
0:00:10][%clko 0:00:13][%emt 0:00:02]}{[%eval 0,48]} 1/2-1/2
http://rapidshare.com/files/397457369/crab_PGO.7z

quick 1+1' tournament temporary result: stockfish171-crab 2,5-1,5
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: New engine _Crab_ based on StockFish

Post by mcostalba »

Look wrote:Hi Marco Costalba

Regarding my testing, since I am clearly biased, you can not rely on my results. I hope to get independent testing results after someone experienced in compiling manages to compile and post it.
Hi Adam,

a test is a test, it is not made by you but by your computer. :-)

I think that reports test conditions and results cannot be biased...

Actually, to be honset,I asked for the test, not to know the test result, but I am mainly interested in test conditions: how did you test, from there I can get an idea of what can be expected.
Albert Silver
Posts: 3019
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: New engine _Crab_ based on StockFish

Post by Albert Silver »

"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
Look

Re: New engine _Crab_ based on StockFish

Post by Look »

Thanks Sven Berger for the compile. Though note that if you want to compare two engines IMO it is best to have exactly same conditions for both engines other than the applied changes. Your PGO compile runs in my 32 bit system( faster than my own compile as expected :) ) so I assume its a 32bit build. OTOH your are playing a match against a 64bit ja compile of SF if I underestand correctly. So in my opinion this would not be an ideal test. Either use a 32bit PGO compile from SF team or compile SF 1.7.1 same as you did with crab. Note than their nps and depth should be _about_ the same,Then test. For a quick result you may want to use a faster time control like 10sec+100ms. 1min+1sec is expected to be much more exact, but you need many games (like 1000+) which takes time, I think if Crab is sound other users might try longer TCs anyway.
Albert Silver
Posts: 3019
Joined: Wed Mar 08, 2006 9:57 pm
Location: Rio de Janeiro, Brazil

Re: New engine _Crab_ based on StockFish

Post by Albert Silver »

Look wrote:Thanks Sven Berger for the compile. Though note that if you want to compare two engines IMO it is best to have exactly same conditions for both engines other than the applied changes. Your PGO compile runs in my 32 bit system( faster than my own compile as expected :) ) so I assume its a 32bit build.
Ah... good to know. I have a 64-bit system, and don't even have SF 32-bit on it. I'll have to check to compare.
"Tactics are the bricks and sticks that make up a game, but positional play is the architectural blueprint."
Look

Re: New engine _Crab_ based on StockFish

Post by Look »

Sorry Marco, I think someone who makes changes in a program is by definition biased towards the changes he/she has made. That bias might affect the decisions to be made for the test. So eventually there would be a situation where changes appear to be good even superior to previous version while they are not. Having said this, I hope I have sent you the correct version I intended to. In my own version system that is 0.1.1.199 where 199 represents the number of compiles including those faced with errors.
svchbe

Re: New engine _Crab_ based on StockFish

Post by svchbe »

yes i know the 64bit JA build is faster

1800kN/s (32bit JA build)
2100kN/s (my 32bit crab build)
2700kN/s (64bit JA build)

anyway i need to get back to my CFD simulations ;)

and the final results:
13,5-6,5
Look

Re: New engine _Crab_ based on StockFish

Post by Look »

Again thanks for your build Sven Berger. I hope you do a better job on your simulations though ... :)
svchbe

Re: New engine _Crab_ based on StockFish

Post by svchbe »

http://rapidshare.com/files/397493205/crab-x64_PGO.7z

gets around 2700kN/s so it's as fast as the JA build ;)

:roll: no i am debugging this code for hours seems that i messed up my equations some where....