positional test suite sought

Discussion of chess software programming and technical issues.

Moderator: Ras

smcracraft
Posts: 737
Joined: Wed Mar 08, 2006 8:08 pm
Location: Orange County California
Full name: Stuart Cracraft

positional test suite sought

Post by smcracraft »

Hi,

Looking for a good positional test suite
in EPD format.

Thanks,

Stuart
Dann Corbit
Posts: 12781
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: positional test suite sought

Post by Dann Corbit »

I guess that this is the most famous one:
http://home.interact.se/~w100107/fentest.htm

You can also get a copy here:
http://cap.connx.com/EPD/tony-pos.epd.bz2
smcracraft
Posts: 737
Joined: Wed Mar 08, 2006 8:08 pm
Location: Orange County California
Full name: Stuart Cracraft

Re: positional test suite sought

Post by smcracraft »

Thanks/appreciated.
smcracraft
Posts: 737
Joined: Wed Mar 08, 2006 8:08 pm
Location: Orange County California
Full name: Stuart Cracraft

Re: positional test suite sought

Post by smcracraft »

Dann Corbit wrote:I guess that this is the most famous one:
http://home.interact.se/~w100107/fentest.htm

You can also get a copy here:
http://cap.connx.com/EPD/tony-pos.epd.bz2
Score=127 with a bare-bones PVS searcher, Fruit's
pc/sq tables and material as sole terms in static
evaluation, and 5 seconds per position on a MacBook
(2.4ghz Intel Core 2 Duo) with a small hash table,
null move R=3, late move reduction, futility,
and the only major extension being if in check...
quiescence searches winning captures and all
promotions only. Maximum depth of search is
limited to 60 ply.

Not sure if the above positional test results table was
on super-slow hardware (Athlon?) or what but I don't
see why such a simple engine could get 127 otherwise.

Too simple a test positionally?

Stuart
Dann Corbit
Posts: 12781
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: positional test suite sought

Post by Dann Corbit »

smcracraft wrote:
Dann Corbit wrote:I guess that this is the most famous one:
http://home.interact.se/~w100107/fentest.htm

You can also get a copy here:
http://cap.connx.com/EPD/tony-pos.epd.bz2
Score=127 with a bare-bones PVS searcher, Fruit's
pc/sq tables and material as sole terms in static
evaluation, and 5 seconds per position on a MacBook
(2.4ghz Intel Core 2 Duo) with a small hash table,
null move R=3, late move reduction, futility,
and the only major extension being if in check...
quiescence searches winning captures and all
promotions only. Maximum depth of search is
limited to 60 ply.

Not sure if the above positional test results table was
on super-slow hardware (Athlon?) or what but I don't
see why such a simple engine could get 127 otherwise.

Too simple a test positionally?

Stuart
You must be doing something right. Here is a table of results from other engines:

Code: Select all

Program/Hardware 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Tot 
Rybka 1.2 Athlon 1200                   NEW 10 10 10 7 7 10 10 10 8 10 7 10 8 10 10 6 143 
Shredder 9 UCI Athlon 1200          10 9 10 7 10 10 10 10 10 - 7 8 6 10 9 7 133 
Hiarcs 8 Athlon 1200                                  10 9 10 7 7 10 10 5 10 8 5 7 8 5 10 7 128 
Junior 7 Athlon 1200                                  9 6 10 8 7 10 8 10 10 - 7 10 8 8 9 7 127 
Chess Tiger 14 CB Athlon 1200                 10 9 10 7 10 10 7 5 8 7 10 7 10 - 9 7 126 
Hiarcs 9 Athlon 1200                                - 9 10 7 7 9 10 10 10 8 10 7 6 5 10 7 125 
AnMon 5.07 AMD K6-2 450        10 9 10 8 7 10 7 10 9 10 10 8 - 10 - 7 125 
Junior 9 Athlon 1200                        10 10 10 7 7 10 10 10 - - 7 7 8 8 10 7 121 
Deep Fritz 8 Athlon 1200                          - 9 10 7 7 10 10 10 10 - 7 7 8 10 10 6 121 
Chessmaster 8000 Athlon 1200                  10 7 10 7 7 10 7 10 10 10 - 7 8 - 10 7 120 
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: positional test suite sought

Post by mcostalba »

smcracraft wrote:
Too simple a test positionally?

Stuart
With these positions Stockfish achieves a better score with lower depth!

Below are two extreme examples when the _suggested_ best move is found almost immediately and then is lost going deep with thinking to the effect that the best move after more then 5 minutes thinking and 30M nodes searched is even not scored among the possible ones!

Code: Select all


POSITION 1:

Positional scores are: b5=10, Ngxe5=9, Ncxe5=9, Qc7=8, Na5=7, a6=6, h6=5, f6=3"; id "tony.pos.09";

Searching: 1r1q1rk1/pp1bbppp/2n1p1n1/4P3/2BpN3/3P1N2/PP2QPPP/R1B1R1K1 b - -
infinite: 1 ponder: 0 time: 0 increment: 0 moves to go: 0
 2     +1.08   00:00      184 Ngxe5 Bf4 
 2     +1.20   00:00      557 b5 Bb3 
 3     +1.20   00:00      864 b5 Bb3 Ngxe5 
 4     +0.98   00:00     1624 b5 Bb3 Ngxe5 Bf4 
 5     +0.67   00:00     5956 b5 Bb3 Ngxe5 Bf4 Nxf3+ Qxf3 
 5     +0.76   00:00    13380 Ngxe5 Bf4 Qa5 Bxe5 Nxe5 Nxd4 
 6     +0.76   00:00    18654 Ngxe5 Bf4 Qa5 Bxe5 Nxe5 Nxd4 Rfc8 
 7     +1.08   00:00    30842 Ngxe5 Bf4 Qa5 Rac1 Nxf3+ Qxf3 Ne5 Qg3 
 8     +0.73   00:00    54149 Ngxe5 Bf4 Qa5 Nxe5 Nxe5 Ng3 f6 Nf5 Nxc4 
 9     +0.76   00:00   123727 Ngxe5 Nxe5 Nxe5 Bf4 Qa5 Ng3 Nxc4 Bxb8 Rxb8 dxc4 
                              Bb4 Red1 
10     +0.67   00:00   319060 Ngxe5 Nxe5 Nxe5 Bf4 Qa5 Ng3 Nxc4 Bxb8 Rxb8 dxc4 
                              Bb4 Qe5 d3 
10     +0.76   00:00   559085 Qc7 Bg5 Ngxe5 Bxe7 Nxf3+ Qxf3 Nxe7 Qh5 Rbc8 Rac1 
                              Nd5 Ng5 
11     +0.82   00:00   756804 Qc7 Bg5 Ngxe5 Rac1 Rbc8 Nxe5 Qxe5 Qh5 Qf5 Qh4 Bb4 
                              Re2 
12     +0.92   00:01    1574k Qc7 Bg5 Bb4 Rec1 h6 Bd2 Bxd2 Qxd2 Ngxe5 Nxe5 Qxe5 
                              Rab1 Rbc8 Bb3 
13     +0.90   00:02    2396k Qc7 Bg5 Bb4 Rec1 h6 Bd2 Bxd2 Qxd2 Ngxe5 Nxe5 Qxe5 
                              Rab1 Ne7 Bb3 Nf5 
14     +0.57   00:07    7241k Qc7 Ng3 Rfc8 Bd2 Na5 Rac1 Nxc4 Rxc4 Qb6 b3 Rxc4 
                              bxc4 Bc6 Ne4 Qb2 Bg5 
15     +0.45   00:18   18488k Qc7 Ng3 Rfc8 Bd2 Na5 Rac1 Nxc4 Rxc4 Qb6 b3 Rxc4 
                              bxc4 Bc6 Ne4 Qb2 Bg5 Qxe2 Rxe2 
16     +0.39   00:49   48925k Qc7 Ng3 Rfc8 Bd2 Na5 Rac1 Nxc4 Rxc4 Qb6 b3 Rxc4 
                              bxc4 Bc6 Ne4 Rd8 Bg5 Bxg5 Nexg5 Nf4 
16     +0.57   01:26   86094k Qb6 Bg5 Bxg5 Nexg5 h6 Nh3 Rfc8 Bb3 Nce7 Qe4 Nf5 
                              Qg4 Bb5 Rad1 Rc5 Nf4 Nxf4 
17     +0.51   01:43  102836k Qb6 Bg5 Bxg5 Nexg5 h6 Nh3 Nce7 a4 Bc6 a5 Qd8 Nd2 
                              Qc7 Qh5 Nd5 Nf3 b5 Bxd5 
18     +0.53   02:51  171468k Qb6 Bg5 Bxg5 Nexg5 h6 Nh3 Nce7 Qe4 Nf5 Rab1 Rbc8 
                              Qg4 Bc6 Nf4 Nxf4 Qxf4 Rfd8 Rec1 Qa5 Bb3 
Nodes: 298372057
Nodes/second: 994573
Best move: Qb6
Ponder move: Bg5



POSITION 2:

Positional scores are: d5=10, Bh6=8, Rae8=7, fxe4=5"; id "tony.pos.11";

Searching: r4rk1/p2qn1bp/1pnp2p1/2p2p2/4PP1N/2PPB3/PP2QN1P/R4RK1 b - -
infinite: 1 ponder: 0 time: 0 increment: 0 moves to go: 0
 2     -0.04   00:00      298 d5 e5 
 2     -0.02   00:00      344 fxe4 dxe4 
 2     +0.00   00:00      446 Rae8 Nf3 
 2     +0.02   00:00      554 a6 Rae1 
 2     +0.18   00:00      702 Bf6 Nf3 
 3     -0.06   00:00     1079 Bf6 Nf3 d5 
 3     +0.22   00:00     1254 a6 Rae1 fxe4 
 4     -0.04   00:00     2803 a6 Nf3 Bh6 Rab1 
 4     +0.12   00:00     3367 fxe4 dxe4 d5 Rad1 
 5     +0.06   00:00     4827 fxe4 dxe4 d5 Rad1 Rae8 
 5     +0.14   00:00     7187 a6 Nf3 Bf6 Rae1 fxe4 
 6     +0.08   00:00    14660 a6 Nf3 Bf6 Qc2 Qe6 Rae1 
 6     +0.12   00:00    24551 Bf6 Nf3 Rae8 Rad1 Qe6 Rde1 
 6     +0.14   00:00    30564 Rae8 Nf3 fxe4 dxe4 d5 Rad1 
 7     +0.18   00:00    49427 Rae8 Nf3 Bf6 Rad1 a6 Rfe1 fxe4 
 8     +0.20   00:00    61432 Rae8 Nf3 Bf6 Rad1 a6 Rfe1 Qe6 a3 
 9     +0.16   00:00   102821 Rae8 Nf3 Bf6 Rad1 Qe6 c4 fxe4 dxe4 Nf5 
10     +0.24   00:00   489090 Rae8 Nf3 Bf6 a3 fxe4 dxe4 d5 Rad1 Qe6 e5 
11     +0.35   00:01    1003k Rae8 Nf3 Bh6 Nh4 Qe6 Qf3 fxe4 dxe4 Bg5 Qg3 Nf5 
12     +0.45   00:02    2043k Rae8 Qc2 Na5 Qe2 Qe6 Nf3 Bh6 b4 fxe4 Nxe4 Nc4 
                              dxc4 
13     +0.59   00:04    4208k Rae8 Nf3 Bh6 Nh4 Nd8 Qf3 Ne6 e5 Nc6 Qd5 Rd8 exd6 
                              Qxd6 
14     +0.59   00:13   12415k Rae8 Rfd1 Nd8 Qc2 Bh6 Qb3+ Ne6 Nh3 Kh8 Ng2 Nc6 
                              Re1 Rf7 Rad1 fxe4 
15     +0.63   00:17   17362k Rae8 Rfd1 Nd8 Qc2 Ne6 Nh3 d5 Qb3 Rf7 Rac1 d4 cxd4 
                              Nxd4 Bxd4 Qxd4+ Kg2 
16     +0.55   00:42   40911k Rae8 Qf3 fxe4 dxe4 d5 Rad1 Qe6 exd5 Nxd5 Bxc5 
                              Rxf4 Qxd5 bxc5 Rfe1 Be5 Qxe6+ Rxe6 Re4 Re8 
16     +0.73   01:30   89168k Qe6 Rfe1 fxe4 dxe4 d5 Bd2 dxe4 Nxe4 Nd5 Qg2 Qd7 
                              Qg5 Rae8 Rad1 a6 Be3 Rxe4 Rxd5 
17     +0.78   02:21  141064k Qe6 Rfe1 fxe4 dxe4 d5 Bd2 Rad8 e5 d4 Ne4 d3 Qg2 
                              Nd5 Qg3 c4 Rad1 Rxf4 Bxf4 
18     +0.76   03:04  183865k Qe6 Rfe1 fxe4 dxe4 d5 Bd2 Rad8 e5 d4 Ne4 d3 Qg2 
                              Nd5 Qg3 c4 Rad1 Rde8 Be3 Nxe3 
Nodes: 300517336
Nodes/second: 1001674
Best move: Qe6
Ponder move: Rfe1

I am not an expert of evaluation tuning but my first impression is that his kind of tests hardly substitute real games to validate a change.

Marco
User avatar
michiguel
Posts: 6401
Joined: Thu Mar 09, 2006 8:30 pm
Location: Chicago, Illinois, USA

Re: positional test suite sought

Post by michiguel »

mcostalba wrote:
smcracraft wrote:
Too simple a test positionally?

Stuart
With these positions Stockfish achieves a better score with lower depth!

Below are two extreme examples when the _suggested_ best move is found almost immediately and then is lost going deep with thinking to the effect that the best move after more then 5 minutes thinking and 30M nodes searched is even not scored among the possible ones!

Code: Select all


POSITION 1:

Positional scores are: b5=10, Ngxe5=9, Ncxe5=9, Qc7=8, Na5=7, a6=6, h6=5, f6=3"; id "tony.pos.09";

Searching: 1r1q1rk1/pp1bbppp/2n1p1n1/4P3/2BpN3/3P1N2/PP2QPPP/R1B1R1K1 b - -
infinite: 1 ponder: 0 time: 0 increment: 0 moves to go: 0
 2     +1.08   00:00      184 Ngxe5 Bf4 
 2     +1.20   00:00      557 b5 Bb3 
 3     +1.20   00:00      864 b5 Bb3 Ngxe5 
 4     +0.98   00:00     1624 b5 Bb3 Ngxe5 Bf4 
 5     +0.67   00:00     5956 b5 Bb3 Ngxe5 Bf4 Nxf3+ Qxf3 
 5     +0.76   00:00    13380 Ngxe5 Bf4 Qa5 Bxe5 Nxe5 Nxd4 
 6     +0.76   00:00    18654 Ngxe5 Bf4 Qa5 Bxe5 Nxe5 Nxd4 Rfc8 
 7     +1.08   00:00    30842 Ngxe5 Bf4 Qa5 Rac1 Nxf3+ Qxf3 Ne5 Qg3 
 8     +0.73   00:00    54149 Ngxe5 Bf4 Qa5 Nxe5 Nxe5 Ng3 f6 Nf5 Nxc4 
 9     +0.76   00:00   123727 Ngxe5 Nxe5 Nxe5 Bf4 Qa5 Ng3 Nxc4 Bxb8 Rxb8 dxc4 
                              Bb4 Red1 
10     +0.67   00:00   319060 Ngxe5 Nxe5 Nxe5 Bf4 Qa5 Ng3 Nxc4 Bxb8 Rxb8 dxc4 
                              Bb4 Qe5 d3 
10     +0.76   00:00   559085 Qc7 Bg5 Ngxe5 Bxe7 Nxf3+ Qxf3 Nxe7 Qh5 Rbc8 Rac1 
                              Nd5 Ng5 
11     +0.82   00:00   756804 Qc7 Bg5 Ngxe5 Rac1 Rbc8 Nxe5 Qxe5 Qh5 Qf5 Qh4 Bb4 
                              Re2 
12     +0.92   00:01    1574k Qc7 Bg5 Bb4 Rec1 h6 Bd2 Bxd2 Qxd2 Ngxe5 Nxe5 Qxe5 
                              Rab1 Rbc8 Bb3 
13     +0.90   00:02    2396k Qc7 Bg5 Bb4 Rec1 h6 Bd2 Bxd2 Qxd2 Ngxe5 Nxe5 Qxe5 
                              Rab1 Ne7 Bb3 Nf5 
14     +0.57   00:07    7241k Qc7 Ng3 Rfc8 Bd2 Na5 Rac1 Nxc4 Rxc4 Qb6 b3 Rxc4 
                              bxc4 Bc6 Ne4 Qb2 Bg5 
15     +0.45   00:18   18488k Qc7 Ng3 Rfc8 Bd2 Na5 Rac1 Nxc4 Rxc4 Qb6 b3 Rxc4 
                              bxc4 Bc6 Ne4 Qb2 Bg5 Qxe2 Rxe2 
16     +0.39   00:49   48925k Qc7 Ng3 Rfc8 Bd2 Na5 Rac1 Nxc4 Rxc4 Qb6 b3 Rxc4 
                              bxc4 Bc6 Ne4 Rd8 Bg5 Bxg5 Nexg5 Nf4 
16     +0.57   01:26   86094k Qb6 Bg5 Bxg5 Nexg5 h6 Nh3 Rfc8 Bb3 Nce7 Qe4 Nf5 
                              Qg4 Bb5 Rad1 Rc5 Nf4 Nxf4 
17     +0.51   01:43  102836k Qb6 Bg5 Bxg5 Nexg5 h6 Nh3 Nce7 a4 Bc6 a5 Qd8 Nd2 
                              Qc7 Qh5 Nd5 Nf3 b5 Bxd5 
18     +0.53   02:51  171468k Qb6 Bg5 Bxg5 Nexg5 h6 Nh3 Nce7 Qe4 Nf5 Rab1 Rbc8 
                              Qg4 Bc6 Nf4 Nxf4 Qxf4 Rfd8 Rec1 Qa5 Bb3 
Nodes: 298372057
Nodes/second: 994573
Best move: Qb6
Ponder move: Bg5



POSITION 2:

Positional scores are: d5=10, Bh6=8, Rae8=7, fxe4=5"; id "tony.pos.11";

Searching: r4rk1/p2qn1bp/1pnp2p1/2p2p2/4PP1N/2PPB3/PP2QN1P/R4RK1 b - -
infinite: 1 ponder: 0 time: 0 increment: 0 moves to go: 0
 2     -0.04   00:00      298 d5 e5 
 2     -0.02   00:00      344 fxe4 dxe4 
 2     +0.00   00:00      446 Rae8 Nf3 
 2     +0.02   00:00      554 a6 Rae1 
 2     +0.18   00:00      702 Bf6 Nf3 
 3     -0.06   00:00     1079 Bf6 Nf3 d5 
 3     +0.22   00:00     1254 a6 Rae1 fxe4 
 4     -0.04   00:00     2803 a6 Nf3 Bh6 Rab1 
 4     +0.12   00:00     3367 fxe4 dxe4 d5 Rad1 
 5     +0.06   00:00     4827 fxe4 dxe4 d5 Rad1 Rae8 
 5     +0.14   00:00     7187 a6 Nf3 Bf6 Rae1 fxe4 
 6     +0.08   00:00    14660 a6 Nf3 Bf6 Qc2 Qe6 Rae1 
 6     +0.12   00:00    24551 Bf6 Nf3 Rae8 Rad1 Qe6 Rde1 
 6     +0.14   00:00    30564 Rae8 Nf3 fxe4 dxe4 d5 Rad1 
 7     +0.18   00:00    49427 Rae8 Nf3 Bf6 Rad1 a6 Rfe1 fxe4 
 8     +0.20   00:00    61432 Rae8 Nf3 Bf6 Rad1 a6 Rfe1 Qe6 a3 
 9     +0.16   00:00   102821 Rae8 Nf3 Bf6 Rad1 Qe6 c4 fxe4 dxe4 Nf5 
10     +0.24   00:00   489090 Rae8 Nf3 Bf6 a3 fxe4 dxe4 d5 Rad1 Qe6 e5 
11     +0.35   00:01    1003k Rae8 Nf3 Bh6 Nh4 Qe6 Qf3 fxe4 dxe4 Bg5 Qg3 Nf5 
12     +0.45   00:02    2043k Rae8 Qc2 Na5 Qe2 Qe6 Nf3 Bh6 b4 fxe4 Nxe4 Nc4 
                              dxc4 
13     +0.59   00:04    4208k Rae8 Nf3 Bh6 Nh4 Nd8 Qf3 Ne6 e5 Nc6 Qd5 Rd8 exd6 
                              Qxd6 
14     +0.59   00:13   12415k Rae8 Rfd1 Nd8 Qc2 Bh6 Qb3+ Ne6 Nh3 Kh8 Ng2 Nc6 
                              Re1 Rf7 Rad1 fxe4 
15     +0.63   00:17   17362k Rae8 Rfd1 Nd8 Qc2 Ne6 Nh3 d5 Qb3 Rf7 Rac1 d4 cxd4 
                              Nxd4 Bxd4 Qxd4+ Kg2 
16     +0.55   00:42   40911k Rae8 Qf3 fxe4 dxe4 d5 Rad1 Qe6 exd5 Nxd5 Bxc5 
                              Rxf4 Qxd5 bxc5 Rfe1 Be5 Qxe6+ Rxe6 Re4 Re8 
16     +0.73   01:30   89168k Qe6 Rfe1 fxe4 dxe4 d5 Bd2 dxe4 Nxe4 Nd5 Qg2 Qd7 
                              Qg5 Rae8 Rad1 a6 Be3 Rxe4 Rxd5 
17     +0.78   02:21  141064k Qe6 Rfe1 fxe4 dxe4 d5 Bd2 Rad8 e5 d4 Ne4 d3 Qg2 
                              Nd5 Qg3 c4 Rad1 Rxf4 Bxf4 
18     +0.76   03:04  183865k Qe6 Rfe1 fxe4 dxe4 d5 Bd2 Rad8 e5 d4 Ne4 d3 Qg2 
                              Nd5 Qg3 c4 Rad1 Rde8 Be3 Nxe3 
Nodes: 300517336
Nodes/second: 1001674
Best move: Qe6
Ponder move: Rfe1

I am not an expert of evaluation tuning but my first impression is that his kind of tests hardly substitute real games to validate a change.

Marco
No, but they may tell you what an engine is lacking (or good at).

Miguel
swami
Posts: 6659
Joined: Thu Mar 09, 2006 4:21 am

Re: positional test suite sought

Post by swami »

smcracraft wrote:Hi,

Looking for a good positional test suite
in EPD format.

Thanks,

Stuart
Dann and I have been working on to build some strategical positional test suite, Chapter 1 Undermining/Pawn weakening is nearly getting finished. It consists of 100 set of problems in an EPD test suite - agreed on with by the Top 2 engines after longer analysis. It may get released in coming weeks, and then we will move on to creating the chapter 2.
smcracraft
Posts: 737
Joined: Wed Mar 08, 2006 8:08 pm
Location: Orange County California
Full name: Stuart Cracraft

Re: positional test suite sought

Post by smcracraft »

swami wrote:
smcracraft wrote:Hi,

Looking for a good positional test suite
in EPD format.

Thanks,

Stuart
Dann and I have been working on to build some strategical positional test suite, Chapter 1 Undermining/Pawn weakening is nearly getting finished. It consists of 100 set of problems in an EPD test suite - agreed on with by the Top 2 engines after longer analysis. It may get released in coming weeks, and then we will move on to creating the chapter 2.
Sounds great. Count me in as a beta-tester if you need any.
Steelman

Re: positional test suite sought

Post by Steelman »

Dann Corbit wrote:I guess that this is the most famous one:
http://home.interact.se/~w100107/fentest.htm

You can also get a copy here:
http://cap.connx.com/EPD/tony-pos.epd.bz2
I liked this test
At the 10 minutes per position my program scored 98.

After going back to the positions I was able to determine that my bishops did not want to go where they needed to. After looking at the middle game bishop square table I realized most of it was still a copy of the knights table!
So I corrected it for the bishops.

And I noticed my rooks did not work together very well. That was solved by changing some values in the rook eval routine. Again a few errors on my part.

At the 10 minutes per position my program now scored scored 123.
Chess Moron 10 7 - 7 7 10 10 8 10 8 7 10 10 10 9 - 123

I score nothing on the 3rd game only because I have no transposition tables (yet). I do not do well on endgames.
And the last game well after ply 4 it was q-d2 all the way with r-c1 (worth 10 points) as whites second move. Oh well.

Very helpful this test! Thanks!