weaker winboard engines that support setboard?

[MattieShoes]

So now that I've got a relatively stable engine, I'd like to do some quick and dirty testing against other engines, perhaps using the nunn positions.

There's a bushel of strong engines that support setboard, but I'm having a bit of trouble locating weak ones. (well, weak ones other than my own, anyway...)

Do all y'all have any suggestions? I appreciate it!

Edit: Also, I have gradualtest.exe. Is there anything significantly better for running an EPD suite?

[Robert Pope]

Beaches supports setboard. It's in the same ballpark as TSCP.

[MattieShoes]

Excellent, thanks for the tip!

Well, olithink and beaches certainly bracket my engine, though probably too widely to be terribly useful...

Code: Select all

Rank Name                  Elo    +    - games score oppo. draws
   1 OliThink 5.1.8alpha   374  135   98    20   88%    69    5%
   2 Moneypenny 1.5         69   90   97    40   56%   -35    3%
   3 Beaches 2.26         -444  133  283    20    0%    69    0%

[swami]

So now that I've got a relatively stable engine, I'd like to do some quick and dirty testing against other engines, perhaps using the nunn positions.

There's a bushel of strong engines that support setboard, but I'm having a bit of trouble locating weak ones. (well, weak ones other than my own, anyway...)

Do all y'all have any suggestions? I appreciate it!

Edit: Also, I have gradualtest.exe. Is there anything significantly better for running an EPD suite?

Hi Matt, I'd like to see how your engine performs in this Undermining test suite, If possible, can you please run this test on perhaps 2 minutes per position and let me know of the results?
http://computerchessblogger.googlepages.com/sts

I have sample test scores from wide range of engines, so perhaps I might have an idea of how your engine ranks up in positional understanding.

Next test suite on Open Files and Diagonals is getting finished, Dann Corbit will finalise the select 100 set before release.

PS: Arena 1.99 beta 5 or the latest one has an automatic test feature where you can use it to test your engine on EPDs on given time, and several other conditions.

[michiguel]

So now that I've got a relatively stable engine, I'd like to do some quick and dirty testing against other engines, perhaps using the nunn positions.

There's a bushel of strong engines that support setboard, but I'm having a bit of trouble locating weak ones. (well, weak ones other than my own, anyway...)

Do all y'all have any suggestions? I appreciate it!

Edit: Also, I have gradualtest.exe. Is there anything significantly better for running an EPD suite?

You do not need that. Winboard can load positions in pgn and nunn positions are available in pgn too. Besides, pgn is preferable because it avoids weird situations with time management (in some engines). BTW, I do not recommend to use the nunn positions. Use them in combination with many others such as silver and noomen. It is not a good idea to use a very small set of positions.

I have try them (I will), but maybe you would be interested in this set (EasyWay)
http://wbec-ridderkerk.nl/html/details1/RomiChess.html

I am collecting my own, handpicking them from ECO, but it will take a while (a long while)

Miguel
Edit: Gaviota supports setboard.

[MattieShoes]

I'd be happy to
I noticed the EPD has only one best move listed, but it has several listed in a comment. As far as I can tell, Arena or GradualTest will both only give credit if it's listed in the bm tag. Gradualtest has its own odd system for weighted best moves like Tonypos.epd that looks like this:

Code: Select all

1r2r1k1/1pqbbppp/p2p1n2/4p3/P3PP2/2N1BB2/1PP2QPP/R4R1K b - - bm Rbc8 exf4 Bc6 Rec8 b6 b5 Be6 h6; c0 "Karpov-Kasparov, Moscow 1985"; c1 "10 9 7 6 6 5 4 3"; id "T01";

c1 contains the "scores" for each move.
Were you looking for just the # of best moves or some sort of scored result like in tonypos?

Matt

Edit: I ran it 10 seconds/position just to see if it works okay with Arena and it got 50/100. I'll run a longer test in a bit.

[Laurens Winkelhagen]

Hi,

my engine frank-walter should be in that ball park, and I even have several versions available (probably in the range between beeches and olithink^^)

http://chess.winkelhagen.com


The only thing it that it's a java engine, so some additional set-up might be required.

[swami]

I'd be happy to
I noticed the EPD has only one best move listed, but it has several listed in a comment. As far as I can tell, Arena or GradualTest will both only give credit if it's listed in the bm tag. Gradualtest has its own odd system for weighted best moves like Tonypos.epd that looks like this:

Code: Select all

1r2r1k1/1pqbbppp/p2p1n2/4p3/P3PP2/2N1BB2/1PP2QPP/R4R1K b - - bm Rbc8 exf4 Bc6 Rec8 b6 b5 Be6 h6; c0 "Karpov-Kasparov, Moscow 1985"; c1 "10 9 7 6 6 5 4 3"; id "T01";

c1 contains the "scores" for each move.
Were you looking for just the # of best moves or some sort of scored result like in tonypos?

Matt

Edit: I ran it 10 seconds/position just to see if it works okay with Arena and it got 50/100. I'll run a longer test in a bit.

Hi Matt, I was mainly looking for just the # of best moves. Scores given for other moves as in Tony's test evaluation was implemented by Dann Corbit. Perhaps he knows how to run those kind of test and get the scores based on guesses order for each engines, I haven't tried that yet. I'd be happy if someone else explains to me how to do just that in Arena.

50/100 in 10 seconds per position is not bad, Rotor scored 70 something in 7 minutes per position. So I guess your engine could get past 60's when given more time.

[hgm]

The various versions of micro-Max all support setting up of a position, although they don't do it through the setboard command but through the WB protocol v1 edit command. And they span Elo ranges from ~1500 (uMax 1.6) to 2050 (uMax 4.8_PII).

Beware that many of the weaker engines also behave weird w.r.t. time control after loading a position as PGN (not counting the moves that were forced). Such engines would only work well with incremental / sudden-death time controls when starting from a loaded game.

[MattieShoes]

I don't know that it's possible in Arena. I wrote an ugly script in a couple minutes to convert the epd file to what Gradualtest.exe uses for scored moves. The results at 10 sec/position are slightly different (!?) Maybe caused by it bouncing between two PV's and a slight timing difference.

Code: Select all

Number of tests:                       100
Max score:                             1000
Engines score:                         621
Number with full score:                52
Found in: 2 sec = 1, 5 sec = 15, 10 sec = 36,
Time used for tests (wrong answer=10s): 802s

I'll run it at 2 minutes/position now but I suspect it won't do any better. I wouldn't be terribly surprised if it did worse