Hard Test Suite 2008 (Vincent Lejeune) CEGT results

[Uri Blass]

I found that position number 5 is not correct
Qg4 seems to be clearly winning and there is no reason to consider Qg3 as the only solution(analysis by rybka with 64 mbytes hash)

Pos - King's attack
[d]2bk3r/q5b1/p2pp2p/1p6/1P6/5Q2/P1P2RPP/5R1K w - - 0 1

Analysis by Rybka 2.3.2a 32-bit :

1.Qg4 Be5
= (0.00) Depth: 5 00:00:00
1.Qg4 Be5 2.Qh4+ Ke8
= (0.00) Depth: 6 00:00:00 2kN
1.Qg4 Qd4 2.Rf4 Qe5
µ (-0.88) Depth: 7 00:00:00 11kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7
µ (-0.84) Depth: 8 00:00:00 19kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8
³ (-0.39) Depth: 9 00:00:00 32kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c4
³ (-0.66) Depth: 10 00:00:00 43kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c4 d5
µ (-0.76) Depth: 11 00:00:01 76kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c4 d5 5.cxd5
³ (-0.35) Depth: 12 00:00:04 254kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3
= (0.00) Depth: 12 00:00:05 274kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3
= (0.00) Depth: 13 00:00:06 315kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3 Qxb4 7.c3
= (0.00) Depth: 14 00:00:08 413kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3 Qxb4 7.c3 Qxc3
= (0.00) Depth: 15 00:00:11 610kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3 Qxb4 7.c3 Qxc3
= (0.00) Depth: 16 00:00:17 975kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3 Qxb4 7.c3 Qxc3
= (0.00) Depth: 17 00:00:28 1642kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3 Qxb4 7.c3 Qxc3
= (0.00) Depth: 18 00:00:48 3103kN
1.Qc6 Qd7 2.Qe4 Qe7 3.Rf7 Rf8 4.Qg6 Rxf7 5.Rxf7 Qh4 6.g3 Qxb4 7.c3 Qxc3
= (0.00) Depth: 19 00:01:24 5664kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 Kd8 6.Qh4+ Qg5 7.Qe4 d5
+- (1.42) Depth: 19 00:01:52 6987kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 Kd8 6.Qh4+ Qg5 7.Qe4 d5
+- (1.63) Depth: 20 00:02:41 9842kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 Kd8 6.Qh4+ Qg5 7.Qe4 d5
+- (1.67) Depth: 21 00:04:03 14607kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qh4 Bf6 7.Qxf6 Qxf6
+- (1.72) Depth: 22 00:06:36 23722kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qh4 Bf6 7.Qxf6 Qxf6
+- (1.72) Depth: 23 00:11:08 40454kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qh4 Bf6 7.Qxf6 Qxf6
+- (1.72) Depth: 24 00:20:14 74417kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qf3 d5 7.cxb5 axb5
+- (1.88) Depth: 25 00:44:16 157510kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qf3 d5 7.cxb5 axb5
+- (1.88) Depth: 26 01:14:44 268762kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qf3 d5 7.cxb5 axb5
+- (1.94) Depth: 27 02:23:23 512018kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qf3 d5 7.cxb5 axb5
+- (1.98) Depth: 28 04:54:50 1062546kN
1.Qg4 Qd4 2.Rf4 Qe5 3.Rf7 Rg8 4.c3 Ke8 5.c4 h5 6.Qf3 d5 7.cxb5 axb5
+- (2.01) Depth: 29 11:10:34 2475867kN

(Uri, MyTown 04.08.2008)

[Heinz Van Kempen]

Hi Jon and Uri ,

did you test others if they are "valid"? Then I could concentrate on testing a few more and remove those offering more than one solution.

[Vinvin]

I found that position number 5 is not correct
Qg4 seems to be clearly winning and there is no reason to consider Qg3 as the only solution(analysis by rybka with 64 mbytes hash)

Thanks, Uri. I looked closer, comparing Qg3 and Qg4. In fact, Qg4 seems as strong as Qg3. No definitive evaluation yet, but almost certain. I suppose there will be about 5-6 positions to correct in the suite.

Thanks !

[Vinvin]

Hi Jon and Uri ,

did you test others if they are "valid"? Then I could concentrate on testing a few more and remove those offering more than one solution.

Note that position 75 and 76 (Fortress NN vs Q) is a duplicate one.
I suppose there will be some corrections (about 5) again.
Thanks for you help to improve the suite, 10 min on a quad with 10 engines is very valuable !

It's sill the problem with engine who finds the good solution for wrong reasons
Example position 72; Fortress; 1. Rd1+ is impossible to find for right reason with current engines because it's too deep.

I hope to produce the latest version of the suite in some weeks.

My best,
Vincent

[Heinz Van Kempen]

Hi Vincent ,

thanks for your work. The suite is quite entertaining with a lot of variety.

I am curious what Rybka 3 human will get here.

Maybe with more positions we would have results closer to the rating lists. Somehow it seems that strength does not mean to find always the best moves, but more often the second best moves compared to the opponents.

But correspondence players would like to see always the best move if possible and maybe in most cases will get it with much longer times they give to top engines or by going deeper into the main lines.

[M ANSARI]

A quick question ... how do you test all the positions automatically and get the data that you have. How does it go to the next position ... is there a set ply depth or set time?

[Heinz Van Kempen]

Hi ,

I did this automatically under Fritz 11 GUI and Zap!Chess 64-bit GUI. Just choose Tools-Analyses-Process Test Set and give the path to a .cbh file with the positions to solve.

I chose 600 seconds as maximum for each position. You can give less or more. It follows with the next position as soon as the one in progress is solved or the time limit exceeded. Giving ply depth is not possible, but maybe in other GUI´s. Did not check this.

[Uri Blass]

Hi Jon and Uri ,

did you test others if they are "valid"? Then I could concentrate on testing a few more and remove those offering more than one solution.

I did not make long analysis of other positions but I suspect that there are more cases with more than one winning move.

Uri

[Vinvin]

Hi ,

I did this automatically under Fritz 11 GUI and Zap!Chess 64-bit GUI. Just choose Tools-Analyses-Process Test Set and give the path to a .cbh file with the positions to solve.

I chose 600 seconds as maximum for each position. You can give less or more. It follows with the next position as soon as the one in progress is solved or the time limit exceeded. Giving ply depth is not possible, but maybe in other GUI´s. Did not check this.

Note that Arena is better than chessbase interface, because CB stop when the solution move is found (sometimes randomly at shallow depth) but Arena allways uses the whole time and report the first time since engine didn't change its mind (menu engine -> automatic analysis -> source -> EPD and open the file; select the engine and seconds to analyse and don't forget to write to file in the output tab)

[Heinz Van Kempen]

Hi Vincent ,

yes, but to be fair I also tell that you can set to 1 or more extra plies in CB GUI´s in order to make sure that the solution was not found accidentally.

You can always copy all results to a text file or word program.