I would compare with and without SEE using a fixed depth search, not fixed time as you did. Otherwise you can't tell which version uses less time for the same number of iterations, which would be crucial in my opinion to judge whether SEE helps in your case (for that single position - see below) by returning the same result faster, and therefore being able to search somewhat deeper.
Second point is that doing a comparison based on only one test position is usually not sufficient to draw any conclusion from it. You should use a larger test set at least, but even better in my opinion would be to play a statistically significant number of games.
Also it is difficult for the reader to judge about your results since it is unknown at which point in time your program prints a new PV line:
a) immediately if a new root move improves the old PV, or
b) only at the end of each iteration.
I would assume you do a), which implies that the times that were needed for completing a whole iteration are not visible directly.
Sven
You´re right, I do a).
The time used (in centiseconds) is in the 3rd column.
I tought that results were comparable because both setups produced the same scores and PVs (except in the first iteration). I will try tomorrow with a wider collection of positions.
I would compare with and without SEE using a fixed depth search, not fixed time as you did. Otherwise you can't tell which version uses less time for the same number of iterations, which would be crucial in my opinion to judge whether SEE helps in your case (for that single position - see below) by returning the same result faster, and therefore being able to search somewhat deeper.
Second point is that doing a comparison based on only one test position is usually not sufficient to draw any conclusion from it. You should use a larger test set at least, but even better in my opinion would be to play a statistically significant number of games.
Also it is difficult for the reader to judge about your results since it is unknown at which point in time your program prints a new PV line:
a) immediately if a new root move improves the old PV, or
b) only at the end of each iteration.
I would assume you do a), which implies that the times that were needed for completing a whole iteration are not visible directly.
Sven
You´re right, I do a).
The time used (in centiseconds) is in the 3rd column.
I tought that results were comparable because both setups produced the same scores and PVs (except in the first iteration). I will try tomorrow with a wider collection of positions.
The point is that in case of a) each printed PV line does only tell us the time needed until searching that move was finished but not the whole iteration. So if you see one PV line for iteration 8 and one for iteration 9 then you simply do not know when the next iteration was started.
If you compare two methods X and Y which do both deliver essentially the same result (i.e., the same PV after searching to the same depth) but differ in speed then the question is of course "which method allows deeper search within the same amount of time", as this is relevant for playing games. But to find this out you need to know first which method finishes the same number of iterations earlier while returning the same PV. This is not always possible when searching with a fixed amount of time unless you also print a PV line at the very end of each iteration, therefore I proposed to use fixed depth instead.