Interview with Richard Vida (english) online ...

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

Frank Quisinsky
Posts: 6958
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Interview with Richard Vida (english) online ...

Post by Frank Quisinsky »

Hi there,

a new interview is online.

Richard Vida (programmer of Critter)
Interview is in English language

SCHACHWELT
http://www.schach-welt.de/index.php?opt ... Itemid=135

Best
Frank
swami
Posts: 6659
Joined: Thu Mar 09, 2006 4:21 am

Re: Interview with Richard Vida (english) online ...

Post by swami »

Frank Quisinsky wrote:Hi there,

a new interview is online.

Richard Vida (programmer of Critter)
Interview is in English language

SCHACHWELT
http://www.schach-welt.de/index.php?opt ... Itemid=135

Best
Frank
Thanks, Frank and Richard. Awesome interview. Delighted to hear this:

Frank Quisinsky

Code: Select all

How do you test new versions of Critter? With which eng-eng matches, positions or other things? Which engines of other programmers do you prefer in testing your own?
Richard Vida

Code: Select all

The only objective way to test the strength is to play games. This is extremely time consuming since confirming a small ELO increase requires a _lot_ of games to get low enough error margins. To not waste time on obviously weak or flawed versions, first I usually do a quick "sanity" test, running through some easy tactical positions. Most WAC positions should be solved in under a second. If something is seriously broken it shows up quickly. Then I proceed to positional tests (Swaminathan's Strategic Test Suite). If the new version does not perform significantly worse, then engine-engine matches follow. I test against 10 engines from range 100 ELO better to 100 ELO weaker than the last public Critter release, 500-1000 games each. I wish I had more computers at my disposal.
Best wishes,
Swami
mcostalba
Posts: 2684
Joined: Sat Jun 14, 2008 9:17 pm

Re: Interview with Richard Vida (english) online ...

Post by mcostalba »

swami wrote: Thanks, Frank and Richard. Awesome interview. Delighted to hear this:

Frank Quisinsky

Code: Select all

How do you test new versions of Critter? With which eng-eng matches, positions or other things? Which engines of other programmers do you prefer in testing your own?
Richard Vida

Code: Select all

The only objective way to test the strength is to play games. This is extremely time consuming since confirming a small ELO increase requires a _lot_ of games to get low enough error margins. To not waste time on obviously weak or flawed versions, first I usually do a quick "sanity" test, running through some easy tactical positions. Most WAC positions should be solved in under a second. If something is seriously broken it shows up quickly. Then I proceed to positional tests (Swaminathan's Strategic Test Suite). If the new version does not perform significantly worse, then engine-engine matches follow. I test against 10 engines from range 100 ELO better to 100 ELO weaker than the last public Critter release, 500-1000 games each. I wish I had more computers at my disposal.
Best wishes,
Swami
Congratulations to Richard, his testing method is very serious, if he does 10 matches against different engines at 500-1000 games each he does way _more_ then we do to validate a patch. Perhaps time control is very very small to accomplish that in a realistic time.....

Prefiltering with WAC and STS is also good, unfortunatly this works well for big changes, but for small tuning it is almost a waste of time because, for instance, a small evaluation tweak hardly will result in much lower WAC and STS scores, so that at the end real games should be done anyway.
Frank Quisinsky
Posts: 6958
Joined: Wed Nov 18, 2009 7:16 pm
Location: Gutweiler, Germany
Full name: Frank Quisinsky

Re: Interview with Richard Vida (english) online ...

Post by Frank Quisinsky »

Hi Swami,

:-)

The programmer of Critter seems to be a very cool guy. I like his answers :-)

Very interesting in Critter.
I start with Critter on Wednesday in SWCR.
You can be sure that I replay each of the games.

Nice test you are working, YES !!
The reason I create a News-Ticker message for SCHACHWELT in the past.
You can send me each time news about your test. I will set your text in our News-Ticker.

Best
Frank
swami
Posts: 6659
Joined: Thu Mar 09, 2006 4:21 am

Re: Interview with Richard Vida (english) online ...

Post by swami »

Thanks for your kind words, Frank. :)

Will let you know about updates!
alpha123
Posts: 660
Joined: Sat Dec 05, 2009 5:13 am
Location: Colorado, USA

Re: Interview with Richard Vida (english) online ...

Post by alpha123 »

I wish I had more computers at my disposal.
If you want, I have a Core 2 Quad Q8300 @ 2.50gHz with 6GB that has a lot of free time....

I don't know if you need help beta testing, but I will help if you do.

Peter