| View previous topic :: View next topic |
| Author |
Message |
Robert Hyatt
Joined: 27 Feb 2006
Posts: 15816
Location: Birmingham, AL
|
 Post subject: Re: Arena question Posted: Tue May 15, 2012 10:40 pm |
  |
|
| Rebel wrote: |
| bob wrote: |
| Rebel wrote: |
I tried various logical options including yours, to no avail. Then looked into the executable and unfortunately a command-line option for fixed depth does not exist. Will have a look at Winboard then  |
Obvious question: "Why fixed depth?" Are you SURE that the two programs have equivalent "plies"? Identical extensions and reductions and such, so that this is not a handicap match where the handicap is not obvious??? |
I am using cutechess-cli in the meantime. To test my eval I run fixed depth self-play matches controlled by a depth based flexible parameter. For instance when I set the following parameter [PLY = 8] as a base then:
1. midgame positions will be done 8 plies
2. midgame (minus queens) 10 plies
3. normal endgame 11 plies
4. rook endings 12 plies
5. minor endings 13 plies
6. pawn endings 14 plies
7. queen endings 10 plies
When I raise the parameter to 9 all depths increase with one and vice versa. When I have a handful of improvements I try them with regular time control.
I am indirectly using the system also to measure the impact of search changes. Say you add a static reduction which gives you a 10% speed-up but loses 20 elo points, it's likely a bad idea. Getting insight in the mysterious ways of the search is the keyword here. |
I don't see how you can conclude the latter. If you make the search 10% faster, at fixed depth you see absolutely no benefit. Take a simple example. Search to 10 plies and stop. You now decide to try more aggressive pruning, and you notice that the search is now 2x faster. But at a fixed depth of 10, it may well lose a little due to the pruning. But the extra ply (which you will never see in a fixed depth test) might gain more than the pruning loss. IF you had been using a fixed time limit instead so that the 2x faster would turn into one extra ply.
In some cases, fixed nodes or fixed depth might be useful, but I would never use 'em to decide whether a change is good or bad if it introduces more error AND more depth at the same time... You miss the "more depth" and only see the "more error" and reach the wrong conclusion...
YMMV of course... |
|
| Back to top |
|
 |
|
|
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum
|
|
FAQ
Search
Memberlist
Usergroups
Register
Profile
Log in to check your private messages
Log in
Arena question