Ralph Stoesser wrote:You removed knowledge from eval and also lowered search selectivity. Is there a relation between these two changes? In other words, was it neccessary(in terms of playing strength) to lower search selectivity after the removal of eval knowledge or vice versa?
No there is no connection. Eval knowledge removal is simply following the rule of simplicity: "If otherwise equal, choose the simplest version".
The selectivity is following the rule: "If otherwise equal, choose the version with least selectivity". The reasoning behind this is the observation that increasing selectivity favours short time controls. If short time controls are equal, there is a risk that increased selectivity hurts at longer time controls.
Do you have any solid evidence of the second rule about selectivity? To me this is still a question I don't have a firm answer for.
My "hunch" is that you want to be as selective as you can get away with and it will scale better - but I have not proved that conclusively.
I think LMR is big today because it probably was not clearly superior in the old days when computers were slow. In computer GO, a similar thing with Monte Carlo Tree Search - it was just not feasible when computers were slow.
We do not have evidence, an indirect test is 1.9 itself because apart from what described very little has changed.
Your guess on why LMR was useful only recently is very possible. On the other hand, SF 1.8 has probably one of the most aggressive LMR/ pruning setup, it seems even more then Rybka 4 (there are more then the expected 3-4 plies of difference in search depth), so has a sense to try to scale it down a bit, but LMR and pruning require a test coverage that is not possible to properly get for a single team: you need to test against many engines at very different TC, and this is suitable only for the public rating lists.
The bottom line is to understand if it is better to go for the extra ply at depth 20 or to find the extra tactic say at depth 16.
In the old days the compromise could have been to go for an extra ply at depth 14 or to find an extra tactic move at depth 8 and in _this_ case the result could have been that the extra tactic was better. But with increasing average search depth the balance could be different today. I have read something about diminishing return of search depth somewhere...
I think it's very important to know the answer to this as it would point to future directions for us all. It seems like one of the improvements of Rybka 3 over previous versions was that they were somewhat less selective but I'm not sure I am remembering this correctly. In my own testing it seems like every version that is a little safer plays worse at longer time controls but I have never tried to measure this with low enough error margins. As you point out this requires a LOT of CPU power to do well.
With some patience I could probably get the answer given a few weeks time. I'm just not sure I can afford to invest the CPU time to answer this one question.
I could do this with Komodo, as there is a range of settings for LMR that seem to work well, some are more conservative than others.
I agree
It seems that there is a small improvement in stockfish based on the IPON test
Perf=2907 after 1375 out of 1900 games played that is 11 elo better than stockfish1.8
Assuming that the improvement at 4+2 is going to be near 10 elo then
unfortuantely it is hard to know what caused the improvement when there are some changes.
1)evaluation change
2)Less pruning change
3)time management change.
My guess is that 2 and 3 are productive at long time control when
maybe 1 is counter productive at long time control but it is only a guess.
please can someone explain what the time parameters mean? they make no intuitive sense to me. obviously i'm not a programmer. but stockfish seems to take too long to make moves at the beginning and gets in time trouble in my blitz games. i would like it to play faster early on.
emergency move horizon - have no idea what this means.
emergency base time - minimum time stockfish keeps on his clock? so increasing this will make him move faster right?
emergency move time - again have no idea, if my emergency base time guess is correct.
min thinking time - is this in seconds or milliseconds or something else?
emergency move horizon - have no idea what this means.
emergency base time - minimum time stockfish keeps on his clock? so increasing this will make him move faster right?
emergency move time - again have no idea, if my emergency base time guess is correct.
min thinking time - is this in seconds or milliseconds or something else?
thank you very much!
Example (1+0):
emergency move horizon: 40
emergency base time: 200
emergency move time: 10
min thinking time: 30
Stockfish will use emergency time buffer of: 200 + 40 * 10 = 600ms. This means that program will pretend that it has 600ms less time than it actually has and calculate optimum thinking time using this "fake time".
Min thinking time means that SF will _always_ use at least 30ms thinking time time for each time (even if this could cause losing directly by time).
Uri Blass wrote:
I agree
It seems that there is a small improvement in stockfish based on the IPON test
Perf=2907 after 1375 out of 1900 games played that is 11 elo better than stockfish1.8
The problem is that, the Shredder GUI is calculating with an internal algorithm which seems to be exactly the simple Elo calculation algorithm from WIKIPEDIA with a static average Bayes elo of all participants.
The later real list is Bayeselo. Unfortunately the top enignes are always a little bit lower than the Shredder-classic calculation.
But I agree that Stockfish 1.9 seems to be a bit better than the 1.8.
emergency move horizon - have no idea what this means.
emergency base time - minimum time stockfish keeps on his clock? so increasing this will make him move faster right?
emergency move time - again have no idea, if my emergency base time guess is correct.
min thinking time - is this in seconds or milliseconds or something else?
thank you very much!
Example (1+0):
emergency move horizon: 40
emergency base time: 200
emergency move time: 10
min thinking time: 30
Stockfish will use emergency time buffer of: 200 + 40 * 10 = 600ms. This means that program will pretend that it has 600ms less time than it actually has and calculate optimum thinking time using this "fake time".
Min thinking time means that SF will _always_ use at least 30ms thinking time time for each time (even if this could cause losing directly by time).
Hope this makes some sense!
It does not make sense.
Why do you allow the program to use more than the remaining time.
I see no situation when it is logical(regardless of the time control).
You should have some variable that is called max_thinking_time and always play if you see that you use more than max_thinking_time.
max_thinking_time should always be slightly smaller than the real remaining time to have no big risk of losing on time in the next move(except cases when the real time is very small that mean real remaining time<10 milliseconds
and in this case it may be better if you play at fixed small depth of 1 or 2 plies)
You can still have a min thinking time parameter but
I suggest to change the meaning of this parameter to
min thinking time except special circumstances.
Thinking about it again I can see only one situation when minimal time that is bigger than the remaining time make sense and it is event like WCCC(world computer chess championshop) because you play on a different clock than the clock of the program
In these events you can get cases when the remaining time on the internal clock is 0 or close to 0 but you need to play a move on the real board.
If the program calculates for 30 milliseconds it make no difference relative to the case that it calculates for 1 millisecond except the quality of the move and the time that the human is going to play the move in the board is nearly the same but I think that a free source code does not need to be something that is tuned for stupid events like WCCC and espacially the blitz events that are simply not exactly computer chess but competitions of teams of programs and operators.
Uri
Last edited by Uri Blass on Sun Oct 03, 2010 9:41 pm, edited 1 time in total.
Uri Blass wrote:
It seems that there is a small improvement in stockfish based on the IPON test
Perf=2907 after 1375 out of 1900 games played that is 11 elo better than stockfish1.8
Assuming that the improvement at 4+2 is going to be near 10 elo .
looking again at the IPON rating list I see that the time control is 5+3 and for some reason I remembered 4+2
perf=2910 after 1486 games so considering the fact that the real rating is slightly lower than perf it seems that the difference is going to be near 10 elo.
Why do you allow the program to use more than the remaining time.
I see no situation when it is logical(regardless of the time control).
Just to keep things simple. It's never wise to made a bad move, but if this happens in one of 10000 games, it's not worth of complicating code to avoid this.
I used much time and energy to design the new system (which should give 5-10 elo boost), so before making hasty suggestions, I'd warmly suggest to read the source and understand the logic behind the system. Then...
