Stockfish 1.5.1

[mcostalba]

One stress test that should always be run is to decrease the value of the arrrays that hold PVs etc (PVSIZE) to 8 or something like that, and ran a match. The engine should never crash even in those conditions. A golden rule to catch bugs is to increase the chances for them to show up during testing.

Miguel

You are right !

I made your suggested test only to discover Stockfish crashes when it reach maximum allowed ply in a position where he is in check.

The wrong line is here, in search functions:

Code: Select all

     if (ply >= PLY_MAX - 1)
        return evaluate(pos, ei, threadID);

If we are in check we should not be allowed to call evaluate() that must be called from a non-in-check position.

Thanks a lot for your suggestion !


BTW this bug has always been there, since Glaurung ages.

[Gian-Carlo Pascutto]

I'm somewhat curious how the material coefficients were calculated, and what improvement you measured from them.

[mcostalba]

I'm somewhat curious how the material coefficients were calculated, and what improvement you measured from them.

Could you please explain better ? i don't think I have understood the question (well, perhaps I have but i would like to hear from you)

Thanks

[mcostalba]

I'm somewhat curious how the material coefficients were calculated, and what improvement you measured from them.

Could you please explain better ? i don't think I have understood the question (well, perhaps I have but i would like to hear from you)

Thanks

Ok,

from Stockfish 1.3 one of us (Joona) developed a way to fine tune evaluation parameters. The fact that SF 1.4 was so succesful is due to this.

In these months we have refined and further developed this system and we used a variation of this technique, designed ad-hoc, to fine tune material coefficents.

I am not going to reveal how this technique work, because it is the secret recipe of Stockfish and is something new, none has posted something similar up to now.

I can only tell you that with traditional tuning you will never get to material coeficents that work as expected and also with our very powerful system we needed _a lot_ of time and effort to make the idea work.

Material coefficents are one of the main improvments of SF 1.5

Hope this is enough for you please don't ask me more.

[George Tsavdaris]

from Stockfish 1.3 one of us (Joona) developed a way to fine tune evaluation parameters. The fact that SF 1.4 was so succesful is due to this.

In these months we have refined and further developed this system and we used a variation of this technique, designed ad-hoc, to fine tune material coefficents.

I am not going to reveal how this technique work, because it is the secret recipe of Stockfish and is something new, none has posted something similar up to now.

I can only tell you that with traditional tuning you will never get to material coeficents that work as expected and also with our very powerful system we needed _a lot_ of time and effort to make the idea work.

Material coefficents are one of the main improvments of SF 1.5

Hope this is enough for you please don't ask me more.

I will.
Since Stockfish is not commercial why you don't reveal this method? Or do you plan do make it commercial one time in the future?

Also your powerful system of tuning material coefficients indeed sounds very powerful and extremely interesting! I hope that you will reveal it in the future.

[Dann Corbit]

from Stockfish 1.3 one of us (Joona) developed a way to fine tune evaluation parameters. The fact that SF 1.4 was so succesful is due to this.

In these months we have refined and further developed this system and we used a variation of this technique, designed ad-hoc, to fine tune material coefficents.

I am not going to reveal how this technique work, because it is the secret recipe of Stockfish and is something new, none has posted something similar up to now.

I can only tell you that with traditional tuning you will never get to material coeficents that work as expected and also with our very powerful system we needed _a lot_ of time and effort to make the idea work.

Material coefficents are one of the main improvments of SF 1.5

Hope this is enough for you please don't ask me more.

I will.
Since Stockfish is not commercial why you don't reveal this method? Or do you plan do make it commercial one time in the future?

I guess that they want to retain a competitive advantage for a while.
They have met the full requirements for their open source license by disclosure of the code. There is no requirement for them to explain what went into creation of the code (though it would be nice if someday they decide to explain it).

Also your powerful system of tuning material coefficients indeed sounds very powerful and extremely interesting! I hope that you will reveal it in the future.

[Tord Romstad]

Since Stockfish is not commercial why you don't reveal this method? Or do you plan do make it commercial one time in the future?

Also your powerful system of tuning material coefficients indeed sounds very powerful and extremely interesting! I hope that you will reveal it in the future.

It is quite possible that it will be revealed some time in the future, although it is currently not planned. This will probably be up to Joona to decide, since he invented the method and wrote the program to do the tuning.

Some additional information and clarification:

This is not a system for tuning material coefficients, but for tuning evaluation parameters in general. In fact, the material coefficient tuning was largely a failure. Although it did seem to give us some improvement, it wasn't as much as we hoped, and a little disappointing considering the very long time we tried to tune it. We believe that we need some very different form of automatic tuning to get anywhere close to optimal values for the material parameters.

Although there are some clever ideas, the technique is far from magic. It still involves playing thousands of automated games and changing parameter values based on the results, just like everybody does. However, the parameters are tuned by a program rather than by a human programmer. Like with many other optimization methods, the parameters will generally end up close to a local optimum rather than a global optimum.

If there is any interest, I'll try to explain the idea behind the material balance evaluation in a different post. Such a post probably should probably be posted on the programming sub-forum, and not here.

[Tord Romstad]

Or do you plan do make it commercial one time in the future?

I just realized I forgot to reply to this question. There are no plans of a commercial version of Stockfish.

[Gian-Carlo Pascutto]

Although there are some clever ideas, the technique is far from magic. It still involves playing thousands of automated games and changing parameter values based on the results, just like everybody does. However, the parameters are tuned by a program rather than by a human programmer. Like with many other optimization methods, the parameters will generally end up close to a local optimum rather than a global optimum.

Thanks. My interest was if this was a genetic optimization/simulated annealing/etc method or a direct calculation kind of system. I guess it's sort of the former kind. I would expect rather huge computing time demands to get good results out of something like that (although it scales rather perfectly in parallel), which is why I pursued the other route.

Due to the nature of the parameters (linear and quadratic) a direct calculation seemed likely, but I guess it also helps to keep the degrees of freedom low.

I'm not sure what to make of the contradicting statements whether this helps or not, but I guess it's easy enough to disable it and see the effect.

[Gian-Carlo Pascutto]

I guess that they want to retain a competitive advantage for a while.

Can't you just rip out the table? You probably don't need to bother with the difficult optimization step if you already got the results.

I'm just not much into the "ripping out other people's code" business.