New engine _Crab_ based on StockFish

[Albert Silver]

it's the same story with the stockfish builds...
so it's probably some kind of optimization...

Ok, I wasn't aware of this, not having tested the 32-bit builds before. FWIW, the speeds say it is crunching nodes about 15-20%+ faster (closer to 15%).

[Albert Silver]

So Silver, are you suggesting that, Berger has added some of his simulation code to Crab to revenge ?

Yes, it is a plot.

Seriously though, I get solutions at different depths, almost randomly, on every position, the plycount varies, etc. I don't know if this is normal, but it makes comparing them, without extensive testing, difficult.

Maybe it's nearly an original program and not entirely based on Stockfish.

It could be categorized as engines akin to Protector 1.3.5 which I think was considered original rather than derivative.

What I meant was that the 32-bit build of Crab was producing different analysis than the 64-bit build of Crab.

[David Dahlem]

[Look]

That is a great icon David Dahlem, I did not know that Crab is that scary though

edit: Hmm, H1-A8 diagonal is expected to be light squares.

[michiguel]

So Silver, are you suggesting that, Berger has added some of his simulation code to Crab to revenge ?

Yes, it is a plot.

Seriously though, I get solutions at different depths, almost randomly, on every position, the plycount varies, etc. I don't know if this is normal, but it makes comparing them, without extensive testing, difficult.

Maybe it's nearly an original program and not entirely based on Stockfish.

It could be categorized as engines akin to Protector 1.3.5 which I think was considered original rather than derivative.

What I meant was that the 32-bit build of Crab was producing different analysis than the 64-bit build of Crab.

I had that problem before with Gaviota.

Maybe some variables that are 32 bits become 64, changing their size and as a consequence, the number or entries allowed in certain tables. That will cause discrepancy in the node counting. This may or may not be the symptom of bugs, but it is certainly a behavior that is not desirable, because it may hyde them.

I would recommend to try more than one compiler and also try to compile it with no optimizations at all and compare the node counting. It may be a compiler issue too (I had that problem before too).

Sometime these are great opportunities to catch dormant bugs.

Miguel

[michiguel]

That is a great icon David Dahlem, I did not know that Crab is that scary though

edit: Hmm, H1-A8 diagonal is expected to be light squares.

They are, crabs walk sideways.

Miguel

[David Dahlem]

Hmm, H1-A8 diagonal is expected to be light squares.

True, but that's not a full board. No way to tell which section of board it represents.

[UncombedCoconut]

Hi Adam,

a test is a test, it is not made by you but by your computer.

I think that reports test conditions and results cannot be biased...

Actually, to be honset,I asked for the test, not to know the test result, but I am mainly interested in test conditions: how did you test, from there I can get an idea of what can be expected.

Dear Marco,
I think it would be very helpful if the Stockfish team created a "patch submission guidelines" page on stockfishchess.com. It could perhaps explain all of these things in one place: why testing is required for even an "intuitive" patch; why contributors have to do initial tests before you spend your CPU cycles on them; what form of test is needed to confirm optimizations, eval, and search/time management changes; and finally a word on coding standards or simplicity.

This is a "feature request" but could also save you a lot of time. You've had to repeat these explanations a lot as Stockfish's strength and fame have been attracted more contributors than ever.

[mcostalba]

Dear Marco,
I think it would be very helpful if the Stockfish team created a "patch submission guidelines" page on stockfishchess.com. It could perhaps explain all of these things in one place: why testing is required for even an "intuitive" patch; why contributors have to do initial tests before you spend your CPU cycles on them; what form of test is needed to confirm optimizations, eval, and search/time management changes; and finally a word on coding standards or simplicity.

Well, anybody is free to submit patches or fork Stockfish as long as he wants and I am very glad of this (as I think also my teammates), so I am not in a role to force people to follow a guidance. What I can say is my opinion on this subject, but I _don't_ assume my opinion is the rule and people should follow it: everybody is free to do whaterver he prefers.

But becasue among everybody there is also me I am free too !

Regarding your points, I will try to answer with my opinon, that I don't state is the truth or the correct one, it is just my opinion.

1) Why contributors have to do initial tests before you spend your CPU cycles on them ?

Because if a patch author dosen't have the motivation to test his works I don't see why I should. Testing, apart from the value itself, is a clear and concrete fact that shows that someone is motivated and willing to spend time on what he propose. If he doesn't care enough then it is just a waste of time also for me and because my time is very limited I try to avoid.

2 )what form of test is needed to confirm optimizations, eval, and search/time management changes?

Well, here there is a vast literature on talkchess and everybody can add his two cents: we go from 20.000 games on a NASA like cluster deemed as "necessary and unavoidable" to a quick test on some sample tactic positions to check if the engine finds the best move. Between these two extreme points there is an whole world !

What I can say is that below a 1000 games at 1' per game test is not reliable enough for me. I have never seriously considered 10.000 games at super fast TC, but I can be wrong on this and perhaps (especially after the serious regression we are facing in these days) has arrived the time to reconsider this matter and give super fast TC a chance, of course not for any kind of patch and not for any part of the engine, but we have accumulated a bit of experience to know where this kind of tests could be reliable enough.


3) and finally a word on coding standards or simplicity.

Coding standard and coding quality are important for us becasue SF is intended as a teaching tool, so we put a lot of effort in making the code self-explanatory, easy to understand and uniform in all the different parts of the engine. We don't like small optimzations and optimization tricks when they badly affect code readibility. We gladly accept to miss some ELO point to keep the code clean. This policy has been rewarded. Keeping code simple and clean allow easier modifications and easier changes and in the long term it turn outs that avoiding small and ugly tricks that may cost you few ELO make you gain big ELO form new features that find simple and natural implementation due to code simplicity and linearity.

[Dr.Wael Deeb]

Hmm, H1-A8 diagonal is expected to be light squares.

True, but that's not a full board. No way to tell which section of board it represents.

You're a big devil Dave,always have a ready answer to shoot