Stockfish and Tactics

[Frank Quisinsky]

I forgot ..

Important is also to change the time-control for testing engines!

If the endgame is near the perfection (for many "super-strong-engines") the priority must have the midgame.
Can I do with the time control.

3x more time for the midgame as for the endgame!
Should be logical, not a brain-burner I think.

And unfortunately ...
All the engines produced the same style of chess, or engines produced a to high average of moves are't interesting for a test with longer time-controls. To many computing time for testing will lost with nonsensical endgames in clearly draw position.

Maybe I am wrong but the ideas I have written before should be interesting enough for a test in detail.

I start such a tournament, December 24th, 2023.

Best
Frank

[Ciekce]

to those who think they know better than engine devs (particularly sf devs) how to test engines:

why? where does your arrogance stem from? I recommend looking up the dunning-kruger effect

also "real chess" is a completely meaningless term - moving pieces (virtually or not) on a chessboard (virtual or not) according to the rules of chess is "real chess", whether you start from positions with an advantage or not

edit: to clarify, I'm talking about testing for development

[smatovic]

to those who think they know better than engine devs (particularly sf devs) how to test engines:

why? where does your arrogance stem from? I recommend looking up the dunning-kruger effect
....

viewtopic.php?f=2&t=82938#p955507

--
Srdja

[Frank Quisinsky]

I think with UHU you are not training ... how I can get an advantage in the early stages of the game, you are training more or less "how I can hold a draw with a bad starting position". This helps more in the endgames than at the beginning of the games and explains Stockfish's stagnation. Stockfish lost a lot of tactical power in the first phase of the game to the big group of the others.

Here we have a clear number one:
Uralochka ... that is a tactical monster.

A great engine for the grandmaster and his opening training, or for any player to analyse openings.

Real chess:
I never hear from a club player ... let us play Uhu!

You wrote:
I'm talking about testing for development.

If you are interested in a passive engine with positional skills and strengths in endgames, UHU openings can help a bit for training, I think. But most chess players I know like to see "fire on the board". And most have no interest in watching games that never end. All the boring endgames and all the fights for nothing in clear draw positions ... that the main reason why people lost here interest on computer chess.

[Frank Quisinsky]

Hi there,

how can I make a processor faster (a small problem Intel have since a while).
So developers are right when they look for other ways ...

- lower power consumption for the same performance
- more cores for the same performance and lower power consumption

Can be compared to computer chess ...

- lower move average for the same performance in strength.

To me it is more logical than fighting for 1-3 Elo in ultra-long games.
Makes no sense!

Many programmers are on the right way ...
Look in Pawn 1.0 to Pawn 2.0 ... in Alexandria 5.0.0 to 5.1.0 (move-average seems to improve a lot ... but I don't have enough games). Can be seen with the latest Uralochka version or engines like CSTal and so many others. What Chess.cpp does is really great.

The younger programmers are probably more interested in neural networks and less interested in important detailed optimisations. Sooner or later neural-network is boring and they looking more in details.

Best
Frank

[Ciekce]

I think with UHU you are not training ... how I can get an advantage in the early stages of the game, you are training more or less "how I can hold a draw with a bad starting position".

training? since when does anyone train on an opening book? people use Unbalanced Human "Upenings" books for testing

computer chess does not work by *getting* an advantage, but by *not losing it* - advantages come from bad moves, not good ones

in Alexandria 5.0.0 to 5.1.0 (move-average seems to improve a lot ... but I don't have enough games) [...] What Chess.cpp does is really great.

I can with 100% certainty guarantee that neither of these engines' authors care about their move average

I think you are the only person even tangentially related to computer chess who I've seen voice any opinion on it at all

The younger programmers are probably more interested in neural networks and less interested in important detailed optimisations. Sooner or later neural-network is boring and they looking more in details.

this is bordering on insulting

you have no reference for the amount of work any of us put into our engines, nor where we focus it

[Frank Quisinsky]

[pgn][Event "40 moves in 20 minutes repeatedly"] [Site "fcp-tourney-2024, Gutweiler"] [Date "2023.12.02"] [Round "1.14"] [White "Stormphrax 3.0.0 NN"] [Black "Alexandria 5.1.0 NN"] [Result "0-1"] [EventDate "????.??.??"] [ECO "B51"] [PlyCount "50"] 1.e4 {book 0s} 1...c5 {book 0s} 2.Nf3 {book 0s} 2...Nc6 {book 0s} 3.Bb5 {book 0s} 3...d6 {book 0s} 4.Bxc6+ {book 0s} 4...bxc6 {book 0s} 5.O-O {book 0s} 5...e5 {book 0s} 6.c3 {book 0s} 6...Bg4 {book 0s} 7.h3 {book 0s} 7...Bxf3 {book 0s} 8.Qxf3 {+0.50/31 17s} 8...Ne7 {+0.73/29 47s} 9.Qd1 {+0.61/33 1:06m} 9...Ng6 {+0.71/30 38s} 10.d4 {+0.62/34 38s} 10...cxd4 {+0.88/30 29s} 11.cxd4 {+0.48/31 33s} 11...Be7 {+0.79/30 32s} 12.dxe5 {+0.52/31 16s} 12...Nxe5 {+0.75/30 40s} 13.Qc2 {+0.51/32 34s} 13...O-O {+0.66/28 58s} 14.Nc3 {+0.53/30 57s} 14...Qd7 {+0.60/26 26s} 15.Be3 {+0.51/32 23s} 15...f5 {+0.75/27 26s} 16.Rad1 {+0.56/30 21s} 16...f4 {0.00/34 26s} 17.Bd4 {+0.77/27 20s} 17...Nf3+ {0.00/38 47s} 18.gxf3 {+0.29/22 15s} 18...Qxh3 {0.00/40 41s} 19.Rfe1 {+0.01/28 1:07m} 19...Kh8 {-3.34/28 26s} 20.Ne2 {-0.20/28 38s} 20...Rf6 {-10.80/27 26s} 21.Bxf6 {-4.85/26 2:26m} 21...gxf6 {-13.16/26 27s} 22.Ng3 {-5.81/25 2:17m} 22...Rg8 {-14.95/25 27s} 23.Rc1 {-6.17/21 27s} 23...fxg3 {-16.66/27 26s} 24.a3 {-6.50/21 47s} 24...g2 {-M2/128 0s} 25.Qxc6 {-M1/80 35s} 25...Qh1# {-M1/128 0s} 0-1[/pgn]

The fastest won game after around 17.300 games in my still-running tournament.

Enough room for working more in details.
For you and for each things I do since many years (if I am looking on all of my blunders every day).

Have a nice day Conor!
My intension isn't to insulting someone, such things are never in my brain.
Can do the others!

Best
Frank

[Eelco de Groot]

A little moan:
I know it's a marvel that SF is able to search so deeply, prune so thoroughly, and still be tactically strong.

However, I am noticing tactical blind spots more frequently and wondered if others did too?

I tested the latest Dev (16th Nov 2023) build on a 64 core threadripper running a (sorry private) tactical testsuite.
On the threadripper it was searching at about 40 MN/s.

Giving 2 minutes per position, Stockfish scored 89/100.
Increasing the time to 10 minutes per position it only scored 91/100.

On several positions, the old Rybka Cluster from 10 years ago (140 cores) was significantly faster. In one position the Hydra FPGA supercomputer from 2005 was much faster.

As a reference, I tested SF Cluster via Chessify on 160 cores and then 768 cores and they scored 98/100 and 99/100 respectively (the bigger cluster achieved a significantly faster solve time). They were only allowed 2 mins per position. It made me wonder if SF Cluster has a broader search because the smaller cluster searching for 2 minutes should search fewer nodes than the 64 core threadripper at 10 minutes.

I'm now going to go back and test Rybka and Houdini on regular hardware. But I am coming to this conclusion: Stockfish's incredible depth comes at a cost...

I think that almost any cluster version of a program, that is reasonably sophisticated will be better on tactics than the 'normal' version. Reasoning is simple. Any 'second' computer will not be able to access the hash table of the first computer. If it just copies the first computer's search, everything is double work. If it does the same search, but one or two plies deeper, like you would do with Lazy SMP over several threads, you still can not profit from the hash of the first computer so that is still much the same thing computer 1 does. Another strategy for lazy SMP would be for another thread to start at depth one again to better order everything that is already in the first hashtable, again this does not work because it can't access the first hash table.

So in order to still be useful, computer two has to do something different from computer one. There are probably many choices but as we have seen, what they have in common is that you can not copy computer one. It is just intuitive that you will probably do something about all the moves that have not been searched thoroughly by computer one, so it helps if you know what is the best move so far for computer one. It will have been searched to death, relatively speaking, because if you make a mistake in the principal variation, in chess this is fatal. You can not make mistakes there, unless it is for instance a small undervaluation that does not change the ordering of moves at root.

Computer two most likely will search something that takes move number two (second best move if it knows that) and sees if it still is number two compared to all other moves, except move number one. You see this is pretty close already to Multi PV.

Ergo: the search is wider, in anything resembling the Rybka cluster and probably even all the clusters that came before it, that were not totally testing something other than regular alpha beta and all its variants. QED

[pohl4711]

I forgot ..

Important is also to change the time-control for testing engines!

If the endgame is near the perfection (for many "super-strong-engines") the priority must have the midgame.
Can I do with the time control.

3x more time for the midgame as for the endgame!
Should be logical, not a brain-burner I think.

This is the reason, why my timecontrol is 3min+1sec (ratio is: 180:1 (basetime/increment). When you look into my database, you see, in the middlegame the thinking-time of the engines is around 3-8 secs per move. And in the endgame, the speed increases until 1sec per move (because 1sec is the increment).
So, with a base thinking-time around 150-200x bigger than the increment you get exactly, what makes sense.

So, if you want longer thinking-times, use just 15min+5secs, instead of 3min+1sec or something like this.

[Frank Quisinsky]

Hi Stefan,

yes, you also did a lot of organising for this and that, in my opinion.
To different things in organising I am bit old school. But I am working on better conditions for my next tournaments (fun is more or less in the foreground).

To the time control I have interest to use for my next "mp" tournament, start December 24th, 2024 ... hope so:
Nostalgia ... for the first winboard tournaments and first rating list with winboard engines (around 25-26 years ago) I used one minute per move (40 in 40). On my Notebook I used most of time 3 minutes +2 seconds/move ... 3Ghz, 11th generation i7.

1st tournament (TOP engines on start, currently still-running).
2nd tournament (move average and different other things are in foreground ... with longer time controls).
3rd tournament (a ponder=on tourney with engines around 2800-3000 Elo I have in my brain. Some interesting developments are available I will give them a look and will mix for me new engine with good known engines from the past).

BTW to Opsidian 8.0 (second *.exe available on the github site).
No crashes after around 215 games.

Have a nice week!

Best
Frank

PS: If the programmer from Stormphrax read that ...
My last message was not a war declaration. Stormphrax is a very strong engine and in TOP-30 in the WORLD. Reason enough to be proud of the own performance!