Search found 17077 matches

by bob
Thu Sep 06, 2007 6:36 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

A little perspective on weak engines; I think the situation is different once the strength exceeds about 2000 (in any semi-realistic scale). My engine is pretty weak, and at this level the variability depends on a few factors, I'll list the ones that come immediately to mind. 1. Engines that have n...
by bob
Thu Sep 06, 2007 6:32 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

This is wrong. I can produce thousands of games played that start from Albert Silvers 40 starting positions. I can play the same position over and over and not get the same results every time. In fact, I can show you games played from the same starting position where I win all 4 and lose 2 of them ...
by bob
Thu Sep 06, 2007 6:29 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

This is wrong. I can produce thousands of games played that start from Albert Silvers 40 starting positions. I can play the same position over and over and not get the same results every time. In fact, I can show you games played from the same starting position where I win all 4 and lose 2 of them ...
by bob
Thu Sep 06, 2007 6:21 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

Does anyone have any theories/thoughts why a change would be more/less effective at different time controls? Apart from the odd exception I would think that most changes would be equally valid at any time control? The main problem case is where you depend on a combination of things, like search + e...
by bob
Thu Sep 06, 2007 6:18 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

The main question is if tests at super fast time control(1 second per game) are good indicator for longer time control(at least for evaluation changes). I wonder what is Bob's experience about this because if tests at super fast time control are good indicator then you can practically try a lot of ...
by bob
Thu Sep 06, 2007 6:13 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

The bottom line is that the only practical solution is a _bunch_ of games. Anything else leads to reduced (significantly reduced) accuracy. I'm working on a paper for the JICGA that really illustrates how this variability is created, and the effect it has on games. It will convince anyone testing t...
by bob
Thu Sep 06, 2007 3:55 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

For testing very small modifications, that lead to a different move choice only once every 10 moves or so, the tree-comparison method I proposed might solve the problem. The problem is in the timing. A small change on this move might not make much different, but it loads the hash table, history cou...
by bob
Thu Sep 06, 2007 3:49 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

A little perspective on weak engines; I think the situation is different once the strength exceeds about 2000 (in any semi-realistic scale). My engine is pretty weak, and at this level the variability depends on a few factors, I'll list the ones that come immediately to mind. 1. Engines that have n...
by bob
Wed Sep 05, 2007 10:21 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: nondeterministic testing
Replies: 86
Views: 34772

Re: nondeterministic testing

You are having far better luck than I am. I am using 40 positions, 2 games each, with 4 standard opponents. A single run has a wild variance when compared with a second run or a third run...
by bob
Wed Sep 05, 2007 7:25 pm
Forum: Computer Chess Club: Programming and Technical Discussions
Topic: How do I test the quality of the Zobrist keys?
Replies: 41
Views: 18143

Re: How do I test the quality of the Zobrist keys?

I don'ttest any positions at all. I am just determining the dependence of the keys by exhaustively searching all combinations of keys. The smallest number of dependent keys tells you how many differences two positions need to have before they can collide. For 64-bit keys in Chess this number cannot...