Advice for Testing
Moderator: Ras
-
outAtime
- Posts: 226
- Joined: Sun Mar 08, 2009 3:08 pm
- Location: Canada
Advice for Testing
Hi. I was wondering about testing engines in matches with ponder on. It seems the engine I would like to test performs very differently when the other engine is allowed to ponder as well (on the same computer) so I was wondering how much difference it makes if I turn the ponder off on the other engines Im testing against, or maybe I should test without ponder on at all on any engine? Thanks.
outAtime
-
bob
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: Advice for Testing
This just introduces more randomness into the results, unless you test on a dual-core (or better) box so that each engine gets its own core 100% of the time. And even then, randomness will go up.outAtime wrote:Hi. I was wondering about testing engines in matches with ponder on. It seems the engine I would like to test performs very differently when the other engine is allowed to ponder as well (on the same computer) so I was wondering how much difference it makes if I turn the ponder off on the other engines Im testing against, or maybe I should test without ponder on at all on any engine? Thanks.
-
wgarvin
- Posts: 838
- Joined: Thu Jul 05, 2007 5:03 pm
- Location: British Columbia, Canada
Re: Advice for Testing
The only fair way to test with pondering on is to get two identical machines and run each engine on its own machine. If you try to run them both on the same machine, the pondering engine can interfere with the performance of the non-pondering one in a lot of different ways, making the test unfair (it can monopolize CPU time, thread scheduling, cache or TLB thrashing, memory paging to disk, system calls, and probably other ways too).outAtime wrote:Hi. I was wondering about testing engines in matches with ponder on. It seems the engine I would like to test performs very differently when the other engine is allowed to ponder as well (on the same computer) so I was wondering how much difference it makes if I turn the ponder off on the other engines Im testing against, or maybe I should test without ponder on at all on any engine? Thanks.
-
outAtime
- Posts: 226
- Joined: Sun Mar 08, 2009 3:08 pm
- Location: Canada
Re: Advice for Testing
I guess ponder off for both engines is the only way to go on a single cpu then. Thanks!
outAtime
-
jwes
- Posts: 778
- Joined: Sat Jul 01, 2006 7:11 am
Re: Advice for Testing
You probably would want ponder on for your engine if you are testing time management. 