nondeterministic testing

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: nondeterministic testing

Post by bob »

hgm wrote:
Uri Blass wrote:... (it does something equivalent to clearing the hash by increasing generation number in the hash and not using information from hash of previous search when the only difference is that it does not waste time on clearing hash).
Perhaps a bit of topic: I used an elegant solution for exactly that in uMax 4.0. In stead of clearing the hash (which uMax 3.2 did) I just incremented the hash key in the root by 1 before every search. This means it can no longer recognize any of the old entries, as their hash keys were all off by 1. (If you probe 4 consecutive entries, and do not store the full key, you would of course have to increment by at least 4.) This solves the problem with a single instruction per search, no extra space in the hash entry required to store a generation number, no overhead to update it and (relevant for uMax) only two '+' characters required.
Makes it impossible to transpose back into the opening book once you leave it however. :)
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: nondeterministic testing

Post by hgm »

Like I wrote, the concept of a book is alien to micro-Max. Eden seems to have sort of a book, but I am not sure how many moves came out of book. uMax is usually quite adept getting opponents out of book... :lol: :lol: :lol:
User avatar
hgm
Posts: 27796
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: nondeterministic testing

Post by hgm »

bob wrote:Makes it impossible to transpose back into the opening book once you leave it however. :)
Well, that depends on your book probing code, of course. It is not that difficult to subtract the move number from the hash key before probing. (Or for those that XOR the key components, in stead of adding them as uMax does, XOR with the move number.)
nczempin

Re: nondeterministic testing

Post by nczempin »

hgm wrote:Like I wrote, the concept of a book is alien to micro-Max. Eden seems to have sort of a book, but I am not sure how many moves came out of book. uMax is usually quite adept getting opponents out of book... :lol: :lol: :lol:
Well, it's quite easy: If the time spent is very close to zero and no value is given, it was a book move. And I even have lines in the book such as (for white)
1. e4 h6 2. d4

And once I get back to uMax' level (if ever) each game against it will be a good reason to reduce uMax' capacity to take Eden out of book.
nczempin

Re: nondeterministic testing

Post by nczempin »

bob wrote:Are you playing without any opening book? Otherwise you can land in a lost position or won position before the engine itself has to think. This then is not a test of the engine, but more a test of the book.

That's why many of us factor the book out when evaluating changes, and play matches from known starting positions that are pretty equal, and then we play both sides to be sure there is no advantage.
Yes, such as the Nunn positions that hgm mentioned. I am sure I will have to use them at a later stage when Eden is stronger, but for now I'm fine with my approach.
nczempin

Re: nondeterministic testing

Post by nczempin »

nczempin wrote:
hgm wrote:Like I wrote, the concept of a book is alien to micro-Max. Eden seems to have sort of a book, but I am not sure how many moves came out of book. uMax is usually quite adept getting opponents out of book... :lol: :lol: :lol:
Well, it's quite easy: If the time spent is very close to zero and no value is given, it was a book move. And I even have lines in the book such as (for white)
1. e4 h6 2. d4

And once I get back to uMax' level (if ever) each game against it will be a good reason to reduce uMax' capacity to take Eden out of book.
Or you can just go through Eden's book manually, it is all in clear text; purposefully human-readable. You can be sure that Eden won't get into that Bb5 position ever again :-)
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: nondeterministic testing

Post by bob »

Pretty easily. If all you are doing is making the thing faster, version after version, then there is little doubt that A' (a faster version of A) will be better. No matter what testing shows over a small number of games...