ShashChess

[amchess]

If ShashChess really is as strong as Stockfish at game playing and really is as strong as Houdini at tactical solving, then that is having your cake and eating it too.

It would be very soothing to see the results of a 1000 game contest, as far as confidence goes.

One other thing, the strongest tactical program is Bluefish XI-LP FD (Tactical=2; defensive=off)

I know there are statistical fluctuations, but I don't agree with a testing methodology based on a lot of games, at least at long time controls.
Very often, for example, Stockfish patches are validated and then removed.
For example, if you play even 50000 games on the Sicilian poisoned pawn variation, you'll get all draw, because a top engine plays this variation "perfectly".
The starting positions to test must be carefully choosen and not random. As every true chess player knows, the characteristics concept is very poweful not only for practical chess player, but also for testers to have sample positions.
Given a position, its characteristics is defined by its pawn structure, kings position and bishop pair. The idea is if two quiescent positions have the same characteristics, the plans/manoeuvres are the same.
For this, I choosen the 10 most frequent characteristics and for them the most sharp variations.
This is my humble opinion.
Finally, about Bluefish XI-LP FD, I imagine you tested this engine on the same positions battery to came to this conclusion.
I will do it because I never used this engine.
Thanks anyway for your observations.
Andrea

[amchess]

amchess

• 319 hard positions test suite

According to your TestSuite08302020.ods file I only get 318 positions:

198 Tal
. 78 Capablanca
. 42 All / Petrosian
------
318

Yes, I refined the test and I have yet to refine...

[amchess]

The explanations:
https://github.com/amchess/ShashChess/wiki/Tests

https://github.com/amchess/ShashChess/wiki/Match
https://github.com/amchess/ShashChess/wiki/Battery

You might want to remove this one from your tests:
1R6/8/8/2p5/p1k5/6r1/1K6/8 w - - c0 "Nalimov EGTB files prove that Ra8 Rc8 Re8 Rf8 Rh8 all draw";

Thanks!
Actually, I have also to update the suite battery because the engines are stronger and stronger...

[amchess]

One other thing, the strongest tactical program is Bluefish XI-LP FD (Tactical=2; defensive=off)

We tested this engine on the same suite and with your suggested configuration: 179 solved positions.
Not bad compared to Stockfish but a lot less than Houdini tactical and ShashChess.

Andrea

[Cornfed]

One other thing, the strongest tactical program is Bluefish XI-LP FD (Tactical=2; defensive=off)

We tested this engine on the same suite and with your suggested configuration: 179 solved positions.
Not bad compared to Stockfish but a lot less than Houdini tactical and ShashChess.

Andrea

<mike drop>

[Dann Corbit]

Not surprising.
His test set has a lot of quiet positions.
I guess that Bluefish solved very few of those.
It's only really good on tactical stuff.
That's why it terrorizes test suites and gets terrorized by stronger programs in contests.
I think there are a few bugs in his test suite also.

[amchess]

What proofs do you have we were wrong? I can only judge based on numbers and code.
If not I can't do more and this discussion will end here.
All positions are well know in chess forums and even their solutions.
The suite has HARD positions, not only tactical:
1.198 Tal
2. 78 Capablanca
3. 42 Petrosian
We use them justly to improve and test Shashin theory.
The tactical positions are 1 (attacking) and 3 (defensive), so the majority.
If there are "bugs", I'm not responsible of them, they are very small and potentially there isn't a "perfect" suite.
The proof is they're continuously improved in chess forums like this one.
I can also show you the result in all of those 3 types of positions...
Anyway, all this doesn't justify the great difference in score.
Moreover, I'm a computer science engineer and I'm improving my skills even at work.
So, I've taken a look at BlueFish code:
except multipv (a very old idea),it's a small subset of ShashChess code (essentially, Crystal's kingDanger, a very good function by John Ellis, imho a very talented programmer).
After all, everyone, even not so skilled, can see it.
So, I'm not surprised.
In general, even if our code is free, we never release without testing and show the tests results, because it's not professional/scientific to pass sentence without proofs. We can surely make some errors because nobody is perfect, but like this we hope to reduce them.
Surely, we made a very hard and long work (even more than one year) to overtake the Houdini tactical version, imho a very good and original work by Robert Houdart, even now at more than two years of distance!
Indeed, in our previous release, we simply told we were near this engine, not we were the best, always based on the results and not on generic sensations".
We, and a lot of people, are happy about this release, but, just because free, clearly you can use/like it or not.
Very humbly, we always speak based on facts, very clearly and not to toot our own horn:
the only goal is to show Shashin theory is flexible enough to ensure this tactical strength without loosing in strength: this isn't true for BlueFish, Crystal or other existing engines. We are convinced that this power is very useful.
I hope I was clear.
Andrea

[peter]

All positions are well know in chess forums and even their solutions.

All well said, Andrea, thanks for the suite!

I guess I would find most of the positions in my own problem- database, looked up to nr.10 of Sh326.cbh only till now, as I called the into chessbase- format imported version, but it's even with cb much work to see two (or three or four) databases through for 326 positions to be found in more than one .epd- or .chb-collections.

So, if you could give some more sources for your collection from already known suites too, it would be very helpful for me to have lines of solutions already stored at my disk.

As I wrote, I found about half of the first 10 postions in HTC20 too, so I guess this one hard suite will have some more overlapping. I have many other suites stored too, so if you'd have another one as for a hint, I'd be thankful. I saw 1 position in Eret too among first ten.
How about that (Eigenmann Rapid Engine Test), or Walter Eigenmann's Nightmare- suites?
"Alternative Chess Test"-suite?
Any other?

If I had my own lines of solutions together with all the positions of your suite, I could more easily compare results like yours, which I don't doubt, with the ones I have from "my" suites already.

Always interested in good test positions and new suites regards

[amchess]

All positions are well know in chess forums and even their solutions.

All well said, Andrea, thanks for the suite!

I guess I would find most of the positions in my own problem- database, looked up to nr.10 of Sh326.cbh only till now, as I called the into chessbase- format imported version, but it's even with cb much work to see two (or three or four) databases through for 326 positions to be found in more than one .epd- or .chb-collections.

So, if you could give some more sources for your collection from already known suites too, it would be very helpful for me to have lines of solutions already stored at my disk.

As I wrote, I found about half of the first 10 postions in HTC20 too, so I guess this one hard suite will have some more overlapping. I have many other suites stored too, so if you'd have another one as for a hint, I'd be thankful. I saw 1 position in Eret too among first ten.
How about that (Eigenmann Rapid Engine Test), or Walter Eigenmann's Nightmare- suites?
"Alternative Chess Test"-suite?
Any other?

If I had my own lines of solutions together with all the positions of your suite, I could more easily compare results like yours, which I don't doubt, with the ones I have from "my" suites already.

Always interested in good test positions and new suites regards

I found them mostly in this forum by the search engine and in the rybka chess forum (there is a related section).
You're right: I eliminated the reference in the positions, but you can find them directly putting the fen on a search engine.
If you're able to collect an updated suite, this is welcome: always we search for collaborators.
We need the three types of positions:
-quiescent or Capablanca
-attacking tactics or Tal
-defensive tactics or Petrosian.
To classify them I created a special tool.
Best regards,
Andrea

[peter]

If you're able to collect an updated suite, this is welcome: always we search for collaborators.
We need the three types of positions:
-quiescent or Capablanca
-attacking tactics or Tal
-defensive tactics or Petrosian.
To classify them I created a special tool.

As I wrote, I have so many test suites, I mainly do let run one after the other one to get more discrimination, yet with different hardware- TCs according to the grade of difficulty of the positions on average.

That's the main problem with test- suites, to have an average- difficulty in common of most of the positions, so a certain TC/position gives selective but not too selective results. Therefore different TCs for different suites run one after the other gives better overall results to be compared to each other than putting all positions of interest togehter in one big suite, Such a single run yet would have only a single one result too, still not to be compared to other results from other suites.

For Eret I have normally 30"/pos., which is more than Walter Eigenmann uses, for HTC I give 100" (Vincent Lejeune tests with 30 minutes/pos. but single core only because of better reproduceability without SMP), my hardware is normally the old 12x3GHz Xeon- X5670 CPU.

So having seen the tactical positions I already had a look at in your suite seem to be of an average grading of about HTC20- positions, I'll give ShashChess 13.2 with it's SSE4.1-popcnt-compile 100"/pos. at 24 threads with 8G hash.
NNUE- eval, Tal, Capablanca and Petrosjan checked, if you'd say that to be ok for you.
Net nn-82215d0fd0df.nnue by Sergio Vieri, downloaded from SF- dev. site, at the moment embedded in SF12.
If any of these hardware- TC- setting- parameters of mine were bad for you, let me know, please.

Looking forward regards