weaker winboard engines that support setboard?

[MattieShoes]

That looks like 10 seconds/position. Otherwise, the "Found in:" line would read differently...

[brianr]

That looks like 10 seconds/position. Otherwise, the "Found in:" line would read differently...

Will re-run...

[MattieShoes]

Gradualtest feeds positions to engines using setboard, sets a very high searchtime number, issues go. When time is up, interrupt it with the "?" command (move now). No special coding is necessary for EPDs if setboard, st, go, and ? are all supported.

If it's in the 1500 range, the results probably won't be very useful. My engine is weak relative to the big boys but it should get about a perfect score vs 1500. I'd be happy to test it out though!

[pedrox]

Thanks,

results for danasah:

Code: Select all

underminegt_120.epd
Number of tests:                       100
Max score:                             1000
Engines score:                         846
Number with full score:                80
Found in: 10 sec = 80,
Time used for tests (wrong answer=10s): 1000s

Positions with error (id):
Undermine.1, Undermine.2, Undermine.5, Undermine.13, Undermine.16,
Undermine.23, Undermine.24, Undermine.34, Undermine.35, Undermine.37, Undermine.
44,
Undermine.45, Undermine.57, Undermine.61, Undermine.73, Undermine.74, Undermine.
86,
Undermine.87, Undermine.95, Undermine.100

I'm sure that each position was analyzed 120 seconds, perhaps a bug in the output of gradualtest in found in 10 s?

I use:

GradualTest /i underminegt.epd /o dana397b_underminegt_120.epd /s "xboard\nst 12000\nnew" /t 120 danasah397b.exe

Pedro

[JVMerlino]

Actually, I'm running the undermining suite on Myrddin right now at 2 minutes per move, just for comparison's sake. Looks like it will be getting about 30-40 correct (so far has 11 out of 30 correct).

I hope to be releasing the next version at the end of this month. I would hope that it might provide some OTB amusement for the average patzer on this board.

jm

[MattieShoes]

Thanks for the tip! Gaviota is significantly better than mine at present, scoring 16/20, ouch! Probably a good engine to test against though -- weak enough that improvements might appear, strong enough to punish weaknesses I might accidentally introduce

[brianr]

Tinker 7.18 at 2 min re-run

Code: Select all

Number of tests:                       100
Max score:                             1000
Engines score:                         742
Number with full score:                64
Found in: 1 sec = 30, 2 sec = 7, 5 sec = 4, 10 sec = 5, 30 sec = 10, 1 min = 5,
2 min = 3,

[MattieShoes]

Hmm, did you actually watch it and see the searches go past 10 seconds? That's just bizarre. Mine shows the right times, and apparently so do others. The only non-trivial difference in command lines used is the /s part. (I've never used /s) Unless it's parsing command line arguments incorrectly, I don't know why it'd be doing that. :-/

[brianr]

I don't use the /s option. Here is some log file info.
Note the long search time and the ? interruption at right about 2 minutes.
The 86400 seconds is 24 hours (I think) and Tinker adjusts its search time based on the presumed game move number and moves left (meaningless here, but there are some defaults), hence the 19 hours.

Code: Select all

20:54:17.344  Got1: setboard r4rk1/pp1b2b1/n2p1nq1/2pP1p1p/2P1pP2/PP4PP/1BQ1N1B1/R3RNK1 b - - 0 1
20:54:17.344  clearing hash tables....
20:54:17.473  
20:54:17.474         +---+---+---+---+---+---+---+---+
20:54:17.474      8  | *R|   |   |   |   | *R| *K|   |
20:54:17.474         +---+---+---+---+---+---+---+---+
20:54:17.474      7  | *P| *P|   | *B|   |   | *B|   |
20:54:17.474         +---+---+---+---+---+---+---+---+
20:54:17.474      6  | *N|   |   | *P|   | *N| *Q|   |
20:54:17.475         +---+---+---+---+---+---+---+---+
20:54:17.475      5  |   |   | *P| P |   | *P|   | *P|
20:54:17.475         +---+---+---+---+---+---+---+---+
20:54:17.475      4  |   |   | P |   | *P| P |   |   |
20:54:17.475         +---+---+---+---+---+---+---+---+
20:54:17.475      3  | P | P |   |   |   |   | P | P |
20:54:17.475         +---+---+---+---+---+---+---+---+
20:54:17.475      2  |   | B | Q |   | N |   | B |   |
20:54:17.476         +---+---+---+---+---+---+---+---+
20:54:17.476      1  | R |   |   |   | R | N | K |   |
20:54:17.476         +---+---+---+---+---+---+---+---+
20:54:17.476           a   b   c   d   e   f   g   h

20:54:17.476  Got1: 
20:54:17.476  Got1: time 8640000
20:54:17.476  Got1: otim 8640000
20:54:17.476  Got1: go
20:54:17.476  tellics tell Tinker DxRxAxW Score Go 0(20)
20:54:17.476  tellics tell Tinker DxRxAxW Score think() 10(20)
20:54:17.477  tellics tell Tinker Initial search time:  19:12:00 (4:48:00)
20:54:17.477  ply    score   time     nodes  pv
20:54:17.477    1     -303      0        21  1.  ...   f6d5  2.  b2g7  g6g7  3.  c4d5
20:54:17.487  Root researching
20:54:17.487    1     -302      0        37  1.  ...   a6c7
20:54:17.497    1      -86      2        55  1.  ...   a6c7
20:54:17.507  Root researching
20:54:17.507    1      -85      3       137  1.  ...   g8f7
20:54:17.517    1      -72      3       155  1.  ...   g8f7
20:54:17.527  Root researching
20:54:17.527    1      -71      5       185  1.  ...   h5h4
20:54:17.537    1      -71      5       206  1.  ...   h5h4
20:54:17.547    1->    -71      7       208  1.  ...   h5h4
20:54:17.557  Search beta1 restarting after x -41
20:54:17.557    2        8      8       541  1.  ...   h5h4  2.  b2f6  g7f6
20:54:17.567    2->      8      8       793  1.  ...   h5h4  2.  b2f6  g7f6
20:54:17.578  Search alpha1 restarting after x -22
20:54:17.578    3      -40     10      3171  1.  ...   h5h4  2.  g3h4  a6c7
20:54:17.589    3->    -40     11      4972  1.  ...   h5h4  2.  g3h4  a6c7
20:54:17.599  Search beta1 restarting after x -10
20:54:17.600    4        5     11      6292  1.  ...   h5h4  2.  g3h4  a6c7  3.  b2f6  g7f6
20:54:17.616    4->      5     14     15839  1.  ...   h5h4  2.  g3h4  a6c7  3.  b2f6  g7f6
20:54:17.629    5       -5     14     20344  1.  ...   h5h4  2.  g3h4  g6h5  3.  b2f6  g7f6
20:54:17.645    5->     -5     16     33756  1.  ...   h5h4  2.  g3h4  g6h5  3.  b2f6  g7f6
20:54:17.665  Search beta1 restarting after x 25
20:54:17.694    6        8     22    102165  1.  ...   h5h4  2.  g1f2  g6h5  3.  g3h4  h5h4
20:54:17.726    6->      8     25    144121  1.  ...   h5h4  2.  g1f2  g6h5  3.  g3h4  h5h4
20:54:17.758    7       -1     28    182356  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:17.849    7->     -1     36    333580  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:17.903  Search beta1 restarting after x 29
20:54:17.909    8       27     42    413752  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:17.975    8->     27     50    542166  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:18.355  Root researching
20:54:18.355    9       -2     88   1229937  1.  ...   g6h7
20:54:18.610    9        6    113   1639878  1.  ...   g6h7  2.  h3h4  f6g4  3.  b2g7  h7g7
20:54:18.855    9->      6    138   2073089  1.  ...   g6h7  2.  h3h4  f6g4  3.  b2g7  h7g7
20:54:19.040   10        6    156   2382014  1.  ...   g6h7  2.  h3h4  f6g4  3.  b2g7  h7g7
20:54:19.377  Root researching
20:54:19.377   10        7    189   3071311  1.  ...   h5h4
20:54:20.000   10       29    252   4277821  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:20.023   10->     29    255   4315056  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:20.482   11       16    300   5124406  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:22.238   11->     16    476   8635411  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:26.748   12       36    927  17747804  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:27.758   12->     36   1028  20005386  1.  ...   h5h4  2.  g3h4  g6h5  3.  f1g3  h5h4
20:54:43.059   13       37   2559  50957454  1.  ...   h5h4  2.  g1f2  g6h5  3.  c2d2  a6c7
20:54:45.333   13->     37   2785  55860297  1.  ...   h5h4  2.  g1f2  g6h5  3.  c2d2  a6c7
20:55:40.030   14       39   8256 166709621  1.  ...   h5h4  2.  g1f2  g6h5  3.  f2e3  g8f7
20:55:45.923   14->     39   8846 178939809  1.  ...   h5h4  2.  g1f2  g6h5  3.  f2e3  g8f7
20:56:17.197  Got2: ?
20:56:17.197  
move20:56:17.197   h5h4
20:56:17.197  tellics whisper +0.39 at 14(31/46) ply 119.72 secs 2040 Knps pv: 1.  ...   h5h4  2.  g1f2  g6h5
20:56:17.207  
20:56:17.210  Got1: new

[michiguel]

Thanks for the tip! Gaviota is significantly better than mine at present, scoring 16/20, ouch! Probably a good engine to test against though -- weak enough that improvements might appear, strong enough to punish weaknesses I might accidentally introduce

Good! that will give you a nice range. One of the few nice things about Gaviota 0.33 is that it is very stable. It has been running on WBEC since 2003 without causing Leo any trouble. In addition, by this time, Gaviota 0.33 played many games so its rating is reliable.

However, it may not be the best if you test it at super fast time controls because it has a fairly large time buffer not to lose on time. This is ok if you are not planning to scale up your testing. In other words, you may think that your engine is doing poorer at 40 moves/5 min compared to 40 moves/20 seconds and the reason is that gaviota does not use all its time at that superfast control.

My suggestion is to test against as many engines as possible. Go to WBEC website, find Gaviota in the ranking, and check if any engine with an ELO equal to gaviota or 100 points lower satisfies you. You do not want to specialize your engine against a couple of opponents. This is very important!!

Miguel