Test Suite for evaluating qSearch ?

[plattyaj]

I don't even know what you mean by 296/300.

296 positions with the best moved matched out of the total of 300 positions in the test suite.

Andy.

[JVMerlino]

OK, I have been using the WAC suite for testing my alphaBeta search, and they are useful for when qSearch is turned off, cause you can see with your own eyes exactly how many ply is required and what the final eval should be.

I also tweak my engine based on the testing with WAC suite. It manages about 180/300 at ply 4, 200/300 at 5 ply and 240/300 at 6 ply with a still buggy qsearch on and with it off gives 111, 119 and 158 respectively. It is a bit depressing that state of the art amateur engines now score 296/300!!!

Agreed. Myrddin is approaching 2000 ELO, but still only gets 277/300 at 10s per position.

jm

[Dann Corbit]

OK, I have been using the WAC suite for testing my alphaBeta search, and they are useful for when qSearch is turned off, cause you can see with your own eyes exactly how many ply is required and what the final eval should be.

I also tweak my engine based on the testing with WAC suite. It manages about 180/300 at ply 4, 200/300 at 5 ply and 240/300 at 6 ply with a still buggy qsearch on and with it off gives 111, 119 and 158 respectively. It is a bit depressing that state of the art amateur engines now score 296/300!!!

Agreed. Myrddin is approaching 2000 ELO, but still only gets 277/300 at 10s per position.

jm

I recall when some engines were solving 300/300 on WAC, Shredder was solving 285/300, and creaming the 300 solvers in game play.

[JVMerlino]

A few interesting positions. Myrddin doesn't crash in any of them, and solves all but the first and the fifth (out of six) instantly -- including the contrived position. The other two it doesn't solve in under two minutes.

jm

[JVMerlino]

Agreed. Myrddin is approaching 2000 ELO, but still only gets 277/300 at 10s per position.

jm

I recall when some engines were solving 300/300 on WAC, Shredder was solving 285/300, and creaming the 300 solvers in game play.

I thought that solving 300 on WAC was suspect, because (at least) one of the problems had an incorrect solution -- 230, I think. Is that not the case?

jm

[Dann Corbit]

Agreed. Myrddin is approaching 2000 ELO, but still only gets 277/300 at 10s per position.

jm

I recall when some engines were solving 300/300 on WAC, Shredder was solving 285/300, and creaming the 300 solvers in game play.

I thought that solving 300 on WAC was suspect, because (at least) one of the problems had an incorrect solution -- 230, I think. Is that not the case?

jm

There is an even more difficult to find refutation of WAC.230, which does lead to a draw. However, the given move is clearly the only move with winning chances and it is not worse than any alternative. It appears that most moves really lead to a draw for WAC.230.

BTW, I have never seen *any* chess engine resolve the alternative draw, though some do choose Alex Szabo's key move after a very, very long think.

In addition, I would vastly prefer an engine which chooses the key move and understands the {apparent} winning nature of the passed pawn over one which does not see the wall and wanders around aimlessly, thinking it is one full piece ahead. This is an indication of chess understanding, to me. If an engine should see the key move and Alex Szabo's refutation and in addition resolve the draw score, then I will pronounce it "grand pubah of all chess engines."

So something to work on for Myriddin.

[JVMerlino]

I thought that solving 300 on WAC was suspect, because (at least) one of the problems had an incorrect solution -- 230, I think. Is that not the case?

jm

There is an even more difficult to find refutation of WAC.230, which does lead to a draw. However, the given move is clearly the only move with winning chances and it is not worse than any alternative. It appears that most moves really lead to a draw for WAC.230.

BTW, I have never seen *any* chess engine resolve the alternative draw, though some do choose Alex Szabo's key move after a very, very long think.

In addition, I would vastly prefer an engine which chooses the key move and understands the {apparent} winning nature of the passed pawn over one which does not see the wall and wanders around aimlessly, thinking it is one full piece ahead. This is an indication of chess understanding, to me. If an engine should see the key move and Alex Szabo's refutation and in addition resolve the draw score, then I will pronounce it "grand pubah of all chess engines."

So something to work on for Myriddin.

Something else to work on are the five missed WAC positions (111, 190, 221, 235 and 299) in which Myrddin wants to play a silly check (or two checks resulting in the original position) before playing the correct move. But I'm not sure if that should be high on my priority list or not.

jm