Rebel wrote:It seems "Node" testing is beneficial for
me.
From:
http://www.top-5000.nl/tuning2.htm
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
After collecting a number of improvements via the Nodes testing method we pitched the new (1.87) version against the old (1.86) version and the scaling at increasing time controls is excellent.
Code: Select all
1.87 vs 1.86
Level Games Score Run Time
Nodes=100.000 16.000 54.0% 8 hours
40 moves in 15 seconds 8.555 53.3% 28 hours
40 moves in 30 seconds 3.018 53.1% 21 hours
40 moves in 60 seconds 2.034 53.8% 27 hours
As it seems we have created ourselves a more reliable testing method as with 16.000 games (or 32.000 for that matter) we can now measure 1-2 elo improvements in a much faster setting.
How does that demonstrate that node testing is better for you? Really the question you should ask yourself is this: for a given testing throughput (nb of games per minute) which method produces the highest quality games? This is not such a trivial question to answer, because you need to calibrate time, nodes, and plies in a way that they are comparable as precisely as possible. And then you can compare the draw rate of self play (higher is better as indicates better quality games).
I remember Don Dailey explained that he did such an experiment and found that node testing was significantly worse than depth testing.
Anyway, I think all methods have advantages, and disadvantages, depending on what you are testing. A few examples:
* you can generally test eval changes at fixed depth or fixed nodes
* it's out of question to test a change affecting search extensions or reductions at fixed depth (obviously depth=4 with no LMR beats the crap out of depth=4 with LMR)
* nodes puts more emphasis on the endgame, compared to depth (you reach high depths in the endgame but your NPS should be in the same ballpark).
I generally use time, with a fixed ratio time=120*inc, like 6"+0.05" or 2.4"+0.02". I sometimes use depth and/or nodes testing to do some quick pre-filtering, but I never use depth only. I prefer to combine depth+nodes, example depth=8 and nodes=64000, meaning the search stops when any of the two conditions is met (max depth or max nodes). You never know, even at depth=8 you could fall into a search explosion position and get stuck on it for a while...
Theory and practice sometimes clash. And when that happens, theory loses. Every single time.