Testing of Asmfish in CCRL?

[Isaac]

I wonder if there's any project to test either Asmfish in the CCRL.
I fail to understand why Sugar was tested and not Asmfish. Asmfish is not equivalent to a faster Stockfish, it has a few differences impacting on the strength (around 1 or 2 elo?) and is faster. If one is looking for a faster Stockfish, there's Pedantfish.
So the bench is not the same for Asmfish than for Stockfish.
Hence the questions :
1)Why was Sugar tested but not Asmfish?
2)Is there a project to test Asmfish on CCRL?

I understand that the CCRL is made of a team of volonteers and that they don't have the time and resources to test all engines, but still, the above 2 questions remain valid to me.

Thanks Graham and all the others involved!

[tpoppins]

Asmfish is not equivalent to a faster Stockfish, it has a few differences impacting on the strength (around 1 or 2 elo?) and is faster. If one is looking for a faster Stockfish, there's Pedantfish.
So the bench is not the same for Asmfish than for Stockfish.

Your info is out of date by five months. Asmfish has had the same bench as SF master since mid-May.

[Isaac]

Asmfish is not equivalent to a faster Stockfish, it has a few differences impacting on the strength (around 1 or 2 elo?) and is faster. If one is looking for a faster Stockfish, there's Pedantfish.
So the bench is not the same for Asmfish than for Stockfish.

Your info is out of date by five months. Asmfish has had the same bench as SF master since mid-May.

My bad! I didn't know this. Thanks for the update
Then only my question 2) remains.

[Graham Banks]

I won't be testing it.
I already have 17 engines on my waiting list to run gauntlets for, ranging from Devel down to Embla..

Also, I think that one SF derivative is enough for now. You can find plenty of other lists that include the countless number of SF derivatives or knock-offs that are around.

However, that's just my stance and I can't speak for other testers.

[Modern Times]

I don't want to test ASMFish. I'd much rather test the latest Stockfish development version. In fact we do that on the blitz list from time to time - Stockfish 071017 64-bit was tested recently, although only on 1 CPU.

[Isaac]

I don't want to test ASMFish. I'd much rather test the latest Stockfish development version. In fact we do that on the blitz list from time to time - Stockfish 071017 64-bit was tested recently, although only on 1 CPU.

Thanks Grahams for your honest replies.

My guess is that even though Asmfish doesn't seem to be up to date with the latest dev version of SF, it's still ahead of it in terms of elo, at all time controls. It may be worth testing and apparently it will be updated from time to time.

[Werewolf]

I don't want to test ASMFish. I'd much rather test the latest Stockfish development version. In fact we do that on the blitz list from time to time - Stockfish 071017 64-bit was tested recently, although only on 1 CPU.

Thanks Grahams for your honest replies.

My guess is that even though Asmfish doesn't seem to be up to date with the latest dev version of SF, it's still ahead of it in terms of elo, at all time controls. It may be worth testing and apparently it will be updated from time to time.

+1

ASMFish is very important, more so than sugar.

[tpoppins]

Team CCRL has been kind enough to accept my offer of help as a tester, so expect asmFish to show up on their rating list in the nearest future.

I completed yesterday a 40/4 4CPU gauntlet against 12 other engines (including SF8 and SugaR 1.2, for comparison's sake). The asmFish exec is a POPCNT build compiled from source with their new fasmg compiler. Oddly, no new execs have been posted on the asmFish Github page since Oct 4, although the page states "Latest version: 19.10.2017". The bench (5620312), incidentally, is identical to the SF dev build from Oct 2; thus this version doesn't include the three functional commits since then. OTOH, no time losses for asmFish have been observed in this gauntlet.

Expect some surprising results, although the individual sample size is rather small at 50 games vs. each opponent. A new thread will go up once (/if?) the test results are approved by the CCRL admins.