Need beta testers for Twisted Logic

[Edsel Apostol]

Hi Everyone,

I need beta testers for development versions of Twisted Logic. Anyone interested, just send me an e-mail or reply to this post.

I currently don't have my own PC, so I do my programming on internet cafes. My time is very limited so improvement of my engine is very slow. Testing is out of the question as I could only let it play for a couple of blitz games and that wouldn't say anything about any improvements I do on the code.

By the way, to those who are interested, here are my conditions:

Fix set of opening position (Noomen Testsuite 2006 or Silver or MLmfl)
4 to 8 opponents
The engine would play each position as black and white
Time control is 1' + 1'' or 2' + 2''

I choose Noomen as the set of opening positions as the creator of this test suite is the book author of Rybka, so I think that it is good.

The choice of opponents should vary in playing style and should not be too far from the engines playing strenght. I would suggest Fruit2.1, SlowChessWV2.1, Booot4.13.1, Ufim8.02.

Directions:
1. Play all the opponents in a round robin match playing each position on the test suite as black and white. This means that when there is 8 opponents and the number of positions in the test suite is 30, the total number of games is 30 x 2 x 8 x 7 / 2 = 1680 games. In a 1' + 1" game, the average duration is around 4 mins. so the total number of hours this would cost is 1680 x 4 / 60 = 112 hours.

30 positions:
Time Control = 1' + 1'' and 8 opponents => 112 hours
Time Control = 1' + 1'' and 4 opponents => 56 hours
Time Control = 2' + 2'' and 8 opponents => 224 hours
Time Control = 2' + 2'' and 4 opponents => 112 hours

2. Play the engine in a gauntlet match against the opponents. Engine must play black and white in each position.

30 positions:
Time Control = 1' + 1'' and 8 opponents => 32 hours
Time Control = 1' + 1'' and 4 opponents => 16 hours
Time Control = 2' + 2'' and 8 opponents => 64 hours
Time Control = 2' + 2'' and 4 opponents => 32 hours

3. Send me the results and I will send back a new version to be tested again using direction number 2.

This is somewhat tedious and I have nothing to pay for the testers except that I could put their names on the acknowledgement on the readme file or I could put their names in the display of the engine as it boots. Example:

Twisted Logic 20070915
by Edsel Apostol
Use UCI commands
Testers:
Name1
Name2
etc...

Edsel Apostol

[Rubinus]

Not like short time. But try match 90+30 .

[Alessandro Scotti]

Why wasting games in a round robin tournament?

[Edsel Apostol]

Hi Alessandro,

I'm including the round robin for the opponents so that when I run Elostat or Bayeselo on the PGN games, I would get a more accurate rating. Sometimes other engines are better against my program while against other engines they doesn't do well. Same with engines that my engine is better against, some of them are better against other opponents, but not against my engine.

It seems Hamsters is a little ahead of my new engine, I used it as its sparring partner.

Do you have any suggestion about a better testing methodology?

Edsel Apostol

[Edsel Apostol]

Hi Pavel,

I don't like short time too but there is no viable solution to the testing problem. You must have a lot of games to decide if the engine is better against previous versions. If you are using long time controls, you wouldn't have enough games in a short time.

I don't have the patience to wait for months before I could decide that one version is better than the other. When I am active in my program development, I am having 5 versions in a day to test.

Long time control testing is only viable if the programmer is a good chess player that can analyze the individual games well.

Edsel Apostol

[mariaclara]

.tagal na ako nag volunteer, kaya lang d mo pinansin
.
..nakakahiya naman kung ipilit ko pa sarili ko.

oh well..........................

[Edsel Apostol]

Hi Claire,

I seem to forget. I'm too busy with other things. It's a good thing you remind me.

Can you start with the latest release Twisted Logic? As it is what I'm going to make as a basis for improvement of other versions.

Edsel Apostol