Search found 17077 matches
- Thu Sep 06, 2007 6:36 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
A little perspective on weak engines; I think the situation is different once the strength exceeds about 2000 (in any semi-realistic scale). My engine is pretty weak, and at this level the variability depends on a few factors, I'll list the ones that come immediately to mind. 1. Engines that have n...
- Thu Sep 06, 2007 6:32 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
This is wrong. I can produce thousands of games played that start from Albert Silvers 40 starting positions. I can play the same position over and over and not get the same results every time. In fact, I can show you games played from the same starting position where I win all 4 and lose 2 of them ...
- Thu Sep 06, 2007 6:29 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
This is wrong. I can produce thousands of games played that start from Albert Silvers 40 starting positions. I can play the same position over and over and not get the same results every time. In fact, I can show you games played from the same starting position where I win all 4 and lose 2 of them ...
- Thu Sep 06, 2007 6:21 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
Does anyone have any theories/thoughts why a change would be more/less effective at different time controls? Apart from the odd exception I would think that most changes would be equally valid at any time control? The main problem case is where you depend on a combination of things, like search + e...
- Thu Sep 06, 2007 6:18 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
The main question is if tests at super fast time control(1 second per game) are good indicator for longer time control(at least for evaluation changes). I wonder what is Bob's experience about this because if tests at super fast time control are good indicator then you can practically try a lot of ...
- Thu Sep 06, 2007 6:13 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
The bottom line is that the only practical solution is a _bunch_ of games. Anything else leads to reduced (significantly reduced) accuracy. I'm working on a paper for the JICGA that really illustrates how this variability is created, and the effect it has on games. It will convince anyone testing t...
- Thu Sep 06, 2007 3:55 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
For testing very small modifications, that lead to a different move choice only once every 10 moves or so, the tree-comparison method I proposed might solve the problem. The problem is in the timing. A small change on this move might not make much different, but it loads the hash table, history cou...
- Thu Sep 06, 2007 3:49 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
A little perspective on weak engines; I think the situation is different once the strength exceeds about 2000 (in any semi-realistic scale). My engine is pretty weak, and at this level the variability depends on a few factors, I'll list the ones that come immediately to mind. 1. Engines that have n...
- Wed Sep 05, 2007 10:21 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: nondeterministic testing
- Replies: 86
- Views: 34772
Re: nondeterministic testing
You are having far better luck than I am. I am using 40 positions, 2 games each, with 4 standard opponents. A single run has a wild variance when compared with a second run or a third run...
- Wed Sep 05, 2007 7:25 pm
- Forum: Computer Chess Club: Programming and Technical Discussions
- Topic: How do I test the quality of the Zobrist keys?
- Replies: 41
- Views: 18143
Re: How do I test the quality of the Zobrist keys?
I don'ttest any positions at all. I am just determining the dependence of the keys by exhaustively searching all combinations of keys. The smallest number of dependent keys tells you how many differences two positions need to have before they can collide. For 64-bit keys in Chess this number cannot...