test the intelligence of your engine

[Uri Blass]

How many iteration does your program do in the following position:
Only few smart programs do not more than 1 iteration
Another question is how many different moves chess programs are going to suggest(all moves have the same stalemate value so no move is a mistake)

[d]k2N4/P7/K7/3PPP2/8/8/R7/R5B1 w - - 35 1

I tested a few programs:
Stockfish developement version stop only at iteration 245 and suggest 1.d6

Houdini3 suggest 1.f6 and get to iteration 99

Senpai1.0 suggest 1.Rb2 and get to iteration 99

Gull3.0 suggest 1.Nf7 and get to iteration 63
Sos5.1 suggest 1.e6 and get to iteration 99
Anmon5.75 suggest 1.e6 and get to iteration 64

Hermann2.8 goes only to iteration 1 and no more than it.
Boot6 write f5-f6 as best move with no stupid iterations

[Terje]

Weiss: 127 (max) iterations in 3ms - Rh2

[zullil]

How many iteration does your program do in the following position:
Only few smart programs do not more than 1 iteration
Another question is how many different moves chess programs are going to suggest(all moves have the same stalemate value so no move is a mistake)

[d]k2N4/P7/K7/3PPP2/8/8/R7/R5B1 w - - 35 1

I tested a few programs:
Stockfish developement version stop only at iteration 245 and suggest 1.d6

Houdini3 suggest 1.f6 and get to iteration 99

Senpai1.0 suggest 1.Rb2 and get to iteration 99

Gull3.0 suggest 1.Nf7 and get to iteration 63
Sos5.1 suggest 1.e6 and get to iteration 99
Anmon5.75 suggest 1.e6 and get to iteration 64

Hermann2.8 goes only to iteration 1 and no more than it.
Boot6 write f5-f6 as best move with no stupid iterations

What move did Black just make?

[abulmo2]

How many iteration does your program do in the following position:
Only few smart programs do not more than 1 iteration
Another question is how many different moves chess programs are going to suggest(all moves have the same stalemate value so no move is a mistake)

[d]k2N4/P7/K7/3PPP2/8/8/R7/R5B1 w - - 35 1
What move did Black just make?

b8a8
and previous white move was b6xa7

[zullil]

[d]k2N4/P7/K7/3PPP2/8/8/R7/R5B1 w - - 35 1
What move did Black just make?

b8a8
and previous white move was b6xa7

Yup, I forgot to consider the pawn capturing on a7.

[Alayan]

The test is absolutely stupid.

Stockfish OF COURSE sees immediately that all moves lead to immediate stalemate, as would any decent engine. But it has no wasteful stop condition and just goes to max depth before exiting. Wasteful, because on positions with play left, making the test wastes CPU time, while on the position where it saves something the gain is negligible and the win/draw trivial.

Stockfish also goes to max depth on mate in 1, that doesn't make it stupid. Max depth is reached almost instantly anyway.

Judge engines by moves and eval after a given hardware time, not by nodes or depth.

[Uri Blass]

The test is absolutely stupid.

Stockfish OF COURSE sees immediately that all moves lead to immediate stalemate, as would any decent engine. But it has no wasteful stop condition and just goes to max depth before exiting. Wasteful, because on positions with play left, making the test wastes CPU time, while on the position where it saves something the gain is negligible and the win/draw trivial.

Stockfish also goes to max depth on mate in 1, that doesn't make it stupid. Max depth is reached almost instantly anyway.

Judge engines by moves and eval after a given hardware time, not by nodes or depth.

I understand your point of view.
I look at it from human point of view and
I look at a chess engine as some human that the programmer teach to play chess with the only difference that the chess engine is faster and have a bigger memory.

I understood that it is not correct after understanding the reason that people removed code to force stockfish to stop the search in obviously drawn position like KB vs K

I understood that the code probably make stockfish stronger because the speed advantage of not having if command gives bigger advantage then saving the little time that stockfish practically search KB vs K positions but it still seems stupid from my point of view because more elo for engines is not the only target that I would like programmers to work on.

I do not like the idea of avoiding if commands to make the engine faster when the same if commands seem logical to me and do not influence the speed of humans.

[Joerg Oster]

The test is absolutely stupid.

Stockfish OF COURSE sees immediately that all moves lead to immediate stalemate, as would any decent engine. But it has no wasteful stop condition and just goes to max depth before exiting. Wasteful, because on positions with play left, making the test wastes CPU time, while on the position where it saves something the gain is negligible and the win/draw trivial.

Stockfish also goes to max depth on mate in 1, that doesn't make it stupid. Max depth is reached almost instantly anyway.

Judge engines by moves and eval after a given hardware time, not by nodes or depth.

I understand your point of view.
I look at it from human point of view and
I look at a chess engine as some human that the programmer teach to play chess with the only difference that the chess engine is faster and have a bigger memory.

I understood that it is not correct after understanding the reason that people removed code to force stockfish to stop the search in obviously drawn position like KB vs K

I understood that the code probably make stockfish stronger because the speed advantage of not having if command gives bigger advantage then saving the little time that stockfish practically search KB vs K positions but it still seems stupid from my point of view because more elo for engines is not the only target that I would like programmers to work on.

I do not like the idea of avoiding if commands to make the engine faster when the same if commands seem logical to me and do not influence the speed of humans.

You will probably never get over this, will you?

Why don't you simply re-add the code and run a test in fishtest?
Or open a PR directly and await the maintainer's decision!

[RubiChess]

Hermann2.8 goes only to iteration 1 and no more than it.
Boot6 write f5-f6 as best move with no stupid iterations

Hermann and probably Boot (haven't tested) violate the UCI protocol:

...
infinite
search until the "stop" command. Do not exit the search without being told so in this mode!
...

Edit: And so does RubiChess after iteration 255

Regards, Andreas

[bob]

The test is absolutely stupid.

Stockfish OF COURSE sees immediately that all moves lead to immediate stalemate, as would any decent engine. But it has no wasteful stop condition and just goes to max depth before exiting. Wasteful, because on positions with play left, making the test wastes CPU time, while on the position where it saves something the gain is negligible and the win/draw trivial.

Stockfish also goes to max depth on mate in 1, that doesn't make it stupid. Max depth is reached almost instantly anyway.

Judge engines by moves and eval after a given hardware time, not by nodes or depth.

I understand your point of view.
I look at it from human point of view and
I look at a chess engine as some human that the programmer teach to play chess with the only difference that the chess engine is faster and have a bigger memory.

I understood that it is not correct after understanding the reason that people removed code to force stockfish to stop the search in obviously drawn position like KB vs K

I understood that the code probably make stockfish stronger because the speed advantage of not having if command gives bigger advantage then saving the little time that stockfish practically search KB vs K positions but it still seems stupid from my point of view because more elo for engines is not the only target that I would like programmers to work on.

I do not like the idea of avoiding if commands to make the engine faster when the same if commands seem logical to me and do not influence the speed of humans.

What is so important about depth? What about time? IE Crafty goes to depth 128 requiring 10 milliseconds total (on my MacBook, no parallel threads). If you don't "noise 0"" you see nothing but the last PV showing a draw and a claim of stalemate. Pretty much instant. Does the 128 on the line rather than 1 make any real difference?

If there is a mate, Crafty won't go to max depth. It will simply go deep enough to prove that it has found the shortest mate (since mates with quiet moves take more iterations than a mate that is all checks but not necessarily shortest.