Interview with Richard Vida (english) online ...

[Frank Quisinsky]

Hi there,

a new interview is online.

Richard Vida (programmer of Critter)
Interview is in English language

SCHACHWELT
http://www.schach-welt.de/index.php?opt ... Itemid=135

Best
Frank

[swami]

Hi there,

a new interview is online.

Richard Vida (programmer of Critter)
Interview is in English language

SCHACHWELT
http://www.schach-welt.de/index.php?opt ... Itemid=135

Best
Frank

Thanks, Frank and Richard. Awesome interview. Delighted to hear this:

Frank Quisinsky

Code: Select all

How do you test new versions of Critter? With which eng-eng matches, positions or other things? Which engines of other programmers do you prefer in testing your own?

Richard Vida

Code: Select all

The only objective way to test the strength is to play games. This is extremely time consuming since confirming a small ELO increase requires a _lot_ of games to get low enough error margins. To not waste time on obviously weak or flawed versions, first I usually do a quick "sanity" test, running through some easy tactical positions. Most WAC positions should be solved in under a second. If something is seriously broken it shows up quickly. Then I proceed to positional tests (Swaminathan's Strategic Test Suite). If the new version does not perform significantly worse, then engine-engine matches follow. I test against 10 engines from range 100 ELO better to 100 ELO weaker than the last public Critter release, 500-1000 games each. I wish I had more computers at my disposal.

Best wishes,
Swami

[mcostalba]

Thanks, Frank and Richard. Awesome interview. Delighted to hear this:

Frank Quisinsky

Code: Select all

How do you test new versions of Critter? With which eng-eng matches, positions or other things? Which engines of other programmers do you prefer in testing your own?

Richard Vida

Code: Select all

The only objective way to test the strength is to play games. This is extremely time consuming since confirming a small ELO increase requires a _lot_ of games to get low enough error margins. To not waste time on obviously weak or flawed versions, first I usually do a quick "sanity" test, running through some easy tactical positions. Most WAC positions should be solved in under a second. If something is seriously broken it shows up quickly. Then I proceed to positional tests (Swaminathan's Strategic Test Suite). If the new version does not perform significantly worse, then engine-engine matches follow. I test against 10 engines from range 100 ELO better to 100 ELO weaker than the last public Critter release, 500-1000 games each. I wish I had more computers at my disposal.

Best wishes,
Swami

Congratulations to Richard, his testing method is very serious, if he does 10 matches against different engines at 500-1000 games each he does way _more_ then we do to validate a patch. Perhaps time control is very very small to accomplish that in a realistic time.....

Prefiltering with WAC and STS is also good, unfortunatly this works well for big changes, but for small tuning it is almost a waste of time because, for instance, a small evaluation tweak hardly will result in much lower WAC and STS scores, so that at the end real games should be done anyway.

[Frank Quisinsky]

Hi Swami,



The programmer of Critter seems to be a very cool guy. I like his answers

Very interesting in Critter.
I start with Critter on Wednesday in SWCR.
You can be sure that I replay each of the games.

Nice test you are working, YES !!
The reason I create a News-Ticker message for SCHACHWELT in the past.
You can send me each time news about your test. I will set your text in our News-Ticker.

Best
Frank

[swami]

Thanks for your kind words, Frank.

Will let you know about updates!

[alpha123]

I wish I had more computers at my disposal.

If you want, I have a Core 2 Quad Q8300 @ 2.50gHz with 6GB that has a lot of free time....

I don't know if you need help beta testing, but I will help if you do.

Peter