Rybka bug also found in Houdini??

[pawnslinger]

Has anyone noticed that Houdini slows down quite a bit when evaluating "won" positions?

Normally, on my setup, Houdini can reach a ply depth of 20 in its search in about 15 seconds or less. This seems to be true for most positions, but give it a won position, say with an eval of 7.00 in its favor, and it is lucky to get a ply dept of 18 in 30 seconds.

I don't know what causes this behaviour, but it is annoying when running a constant depth analysis thru the IDEA in Aquarium. In fact, almost the exact opposite would be nice. No need to waste a lot of time on getting the evaluation exactly right, when it is sooooo much in your favor. Instead, it would be nice to get the "close" positions deeper analysis to try to find the edge, but just quickly discard obviously won positions.

This behaviour of both Rybka and Houdini (and perhaps others) can be especially annoying when trying to do a deep analysis on endgame positions with large numbers of men on the board.

I don't know if this can be easily fixed. I know it is a known problem with Rybka for a long time, and it doesn't appear to have been solved yet. I wish I had a position to demonatrate this, but the only one I have, at the moment, is one of my live games on ICCF and it would not be good to share it yet.

[bob]

Has anyone noticed that Houdini slows down quite a bit when evaluating "won" positions?

Normally, on my setup, Houdini can reach a ply depth of 20 in its search in about 15 seconds or less. This seems to be true for most positions, but give it a won position, say with an eval of 7.00 in its favor, and it is lucky to get a ply dept of 18 in 30 seconds.

I don't know what causes this behaviour, but it is annoying when running a constant depth analysis thru the IDEA in Aquarium. In fact, almost the exact opposite would be nice. No need to waste a lot of time on getting the evaluation exactly right, when it is sooooo much in your favor. Instead, it would be nice to get the "close" positions deeper analysis to try to find the edge, but just quickly discard obviously won positions.

This behaviour of both Rybka and Houdini (and perhaps others) can be especially annoying when trying to do a deep analysis on endgame positions with large numbers of men on the board.

I don't know if this can be easily fixed. I know it is a known problem with Rybka for a long time, and it doesn't appear to have been solved yet. I wish I had a position to demonatrate this, but the only one I have, at the moment, is one of my live games on ICCF and it would not be good to share it yet.

Usually this is just an alpha/beta window issue. If you are at +7, there are probably a ton of +8, +9 and such positions around as well. Depending on whether you use a reasonable aspiration window or not, you might find it difficult to track that forced +7 down, through all the +8 and +9 positions that you have to traverse with a bad window. Also, at +7 you are probably seeing lots of tactical stuff due to checks and captures that can cause the tree to explode a bit.

That's the primary reason that fixed depth is a bad idea for anything except debugging..

[pawnslinger]

I think you are correct. I notice when this happens that IDEA shows many positions with an eval > 7 (in absolute value). The normal method in the past, used to control this problem, is to cut back the time given to each position. I do that myself, but if I cut back the time too much, then the needed time for the critical positions is cut back also. I am using 30 seconds as the maximum per position now, and it usually lets the close positions go thru under 15 and the out of balance positions get chopped at 30 seconds, usually with about 18 plies. Usually this is not too great of a problem, but some positions have a great number of these out of balance positions, and it can slow down the process considerably.

In my current analysis, the endgame has 12 pieces on the board and there are many out of balance positions, eval > 7, for both sides. The correct way to solve the position is to look for the positions that are not so completely out of balance (because they are blunders, and not likely to actually occur), and spend more time looking at the quieter positions...

In IDEA under Aquarium, using a fixed depth ply search is usually the best bet. Maybe with this type of position a fixed time search would be better. But I'd really like to give more time to the quiet positions and less to the "blunder" positions.

[Osipov Jury]

In the case of Rybka, the reason for the slowdown and the growth of a tree in the won positions lies in the method of evaluation hashing (10 bit).
Houdini uses a different method of hashing.

[zullil]

Houdini uses a different method of hashing.

How do you know this?

[Matthias Gemuh]

Houdini uses a different method of hashing.

How do you know this?

Ossipov speaks Russian and Assembler.

[Houdini]

Has anyone noticed that Houdini slows down quite a bit when evaluating "won" positions?

Normally, on my setup, Houdini can reach a ply depth of 20 in its search in about 15 seconds or less. This seems to be true for most positions, but give it a won position, say with an eval of 7.00 in its favor, and it is lucky to get a ply dept of 18 in 30 seconds.

Edwin,

Can you give an example position in which this behaviour occurs?

Robert

[pawnslinger]

Has anyone noticed that Houdini slows down quite a bit when evaluating "won" positions?

Normally, on my setup, Houdini can reach a ply depth of 20 in its search in about 15 seconds or less. This seems to be true for most positions, but give it a won position, say with an eval of 7.00 in its favor, and it is lucky to get a ply dept of 18 in 30 seconds.

Edwin,

Can you give an example position in which this behaviour occurs?

Robert

Hi Robert:

I will look thru some old games and see if I can spot this situation. The position I am analyzing right now is a live game and I don't feel right about sharing it until the game is over.

But let me be clear, I am using Houdini under Aquarium, in their IDEA mode. This is where the problem really gets annoying. When using Houdini under another GUI in regular "Kibitzer" mode, one can hardly notice the difference at all (but I guess it probably exists there too... just not so clear cut).

Are you familiar with Aquarium's IDEA mode? It is a ply search mode outside of the engine, in the GUI. The GUI uses the results of many calls to the engine to build a mini-database (tree) of the combined analysis from the engine (or actually multiple engines can be used simultaneously... or even parallel instances of the same engine).

Anyway, it is a great way to actually dig deep into a position... really deep. But there are some flaws, and the time spent analyzing individual positions is not always well controlled. When I am analyzing a given position, I usually get several thousand leafs in my IDEA tree. And if it takes a long time to generate some of the leaf nodes, it can slow the process considerably. One way that IDEA tries to control this is thru allowing the user to configure such things as: tree width, ply depth or time for each evaluation, etc, etc. Kind of hard to explain, if you haven't used IDEA.

I will try to find a position that I can share that displays this behaviour.

Sincerely,
Ed

[Houdini]

Ed,

I've taken a look at the position you sent me by PM.
There's really nothing special going on, what you're seeing is that the normal pruning/reduction/razoring search strategy becomes less efficient when the evals are 3 to 5 pawns below the expected value. The engine has to consider a lot more "bullshit" variations to make sure that none actually improves upon the bad score. In even other words, the branching factor increases a lot in these positions.

This problem can only be solved by the GUI. The GUI should have an option for immediately stopping the analysis on any position that falls below a certain threshold, say, -3.50 for Houdini. (Could be different for other engines).
I recommend that you write the Aquarium support and suggest this enhancement, it should be a valuable feature for every IDeA user.

Robert

[Uri Blass]

The problem is not only problem of houdini but also of stockfish.

This is a game at fixed depth between houdini and stockfish on my slow computer.

watch the time of the engines at move 24.

I think that maybe some reduction based on evaluation can help and the reduction that I suggest is simply to reduce one ply from the remaining depth if the static evaluation is more than 5 pawns advantage for the side to move and the opponent has no positional advantage like king attack.


I think that these positions are not interesting to search deeper and they should cause reductions.

[Event "URI-AMD, 20Ply / 19Ply"]
[Site "URI-AMD"]
[Date "2010.12.29"]
[Round "4"]
[White "Houdini 1.5 w32"]
[Black "Stockfish 1.9 JA"]
[Result "1-0"]
[ECO "D17"]
[Annotator "0.12;-0.24"]
[PlyCount "81"]
[TimeControl "40/300:40/300:40/300"]

{W=19.9 ply; 916kN/s B=21.0 ply; 770kN/s} 1. d4 d5 2. c4 c6 3. Nf3 Nf6 4. Nc3
dxc4 5. a4 Bf5 6. Ne5 e6 7. f3 Bb4 8. e4 Bxe4 9. fxe4 Nxe4 10. Bd2 Qxd4 11.
Nxe4 Qxe4+ 12. Qe2 Bxd2+ 13. Kxd2 Qd5+ 14. Kc2 Na6 15. Nxc4 {
Both last book move} O-O {-0.24/20 66} 16. Qe5 {0.12/19 46} Rfc8 {
(Ra8-d8) 0.00/20 37} 17. Be2 {(Ra1-d1) 0.40/19 32} f6 {(Rc8-d8) 0.56/20 91} 18.
Qe3 {0.56/19 63} Re8 {(Qd5-d7) 0.56/20 45} 19. Rad1 {(g2-g3) 0.91/19 33} Qxg2 {
(Qd5-f5+) 0.84/20 70} 20. Rhg1 {(Kc2-c1) 1.11/19 61} Qxh2 {1.77/20 94} 21. Rd7
{(Rg1-h1) 1.69/19 392} g6 {0.88/20 139} 22. Qe4 {(Kc2-b1) 1.79/19 54} Nb4+ {
(Na6-c5) 1.81/20 139} 23. Kb3 {1.86/19 301} f5 {(Nb4-a6) 1.69/20 66} 24. Qd4 {
(Rg1-h1) 11.69/19 1677} e5 {(Qh2-h3+) 9.01/20 2287} 25. Qd6 {13.85/19 264} Qh6
{9.17/20 300} 26. Qf6 {(Kb3xb4) #30/19 447} Red8 {13.41/20 606} 27. Re7 {
(Qf6-e6+) #16/19 19} Re8 {#16/20 6937} 28. Qf7+ {#15/19 6} Kh8 {91.65/20 10}
29. Rxe8+ {#14/19 1} Rxe8 {95.92/20 3} 30. Qxe8+ {#13/19 2} Kg7 {#13/20 585}
31. Qxe5+ {#12/19 3} Kg8 {#11/20 235} 32. Nd6 {#11/19 3} Qf8 {#10/20 113} 33.
Bc4+ {#10/19 7} Nd5 {#9/20 32} 34. Bxd5+ {#9/19 3} cxd5 {#7/20 11} 35. Nxf5 {
#8/19 2} b6 {(b7-b5) #6/20 3} 36. Ne7+ {#6/19 1} Kf7 {#5/20 0} 37. Rf1+ {
#5/19 0} Ke8 {#4/20 0} 38. Nxg6+ {(Ne7xd5+) #4/19 0} Kd7 {#3/20 0} 39. Nxf8+ {
#3/19 0} Kc8 {(Kd7-d8) #2/20 0} 40. Rf7 {#2/19 0} d4 {(a7-a5) #1/20 0} 41. Qe8#
{#1/1 0} 1-0