Question for Bob Hyatt

[lkaufman]

I can't duplicate the experiment precisely without involving Don, whom I'd rather not distract from improving Doch with this, but I may be able to roughly simulate it myself when I have time. But for me the big question is this: If this vastly superior search has been know for a couple of years now due to Strelka, and if many of the programmers share your view that it was the main reason for Rybka's superiority (I believe many do), then why is it that not a single strong program (other than those that are Rybka clones or derivatives) has implemented some form of this type of search (i.e. safety-first, focus on the PV)? At least I don't know of any that show any signs of doing this. Is the reason:
A. No one understand the Strelka search
B. Every programmer was too lazy to do this?
C. Every programmer refused to take Rybka ideas on principle?
D. Programmers tried it, but it did not help their own programs.
E. Some other reason/reasons.

[Uri Blass]

I can't duplicate the experiment precisely without involving Don, whom I'd rather not distract from improving Doch with this, but I may be able to roughly simulate it myself when I have time. But for me the big question is this: If this vastly superior search has been know for a couple of years now due to Strelka, and if many of the programmers share your view that it was the main reason for Rybka's superiority (I believe many do), then why is it that not a single strong program (other than those that are Rybka clones or derivatives) has implemented some form of this type of search (i.e. safety-first, focus on the PV)? At least I don't know of any that show any signs of doing this. Is the reason:
A. No one understand the Strelka search
B. Every programmer was too lazy to do this?
C. Every programmer refused to take Rybka ideas on principle?
D. Programmers tried it, but it did not help their own programs.
E. Some other reason/reasons.

I think the following:

1)It is possible that there is some strong program that implemented that type of search.

2)Strelka is not rybka
Strelka is based on rybka1
and I believe rybka3 has even a better search than rybka1.

For the reasons that you give my guess is the following.

A and B is correct for many programmers
Another possible problem is that the data structure of some programmers is too slow and they were lazy to rewrite their program.

Uri

[mcostalba]

A. No one understand the Strelka search
B. Every programmer was too lazy to do this?
C. Every programmer refused to take Rybka ideas on principle?
D. Programmers tried it, but it did not help their own programs.
E. Some other reason/reasons.

In my case is A (becasue I didn't looked at Strelka), I have possibly seen Rybka like search the first time in October 2009 with Ippolit sources, and D becasue all the features are strictly correlated and you just can't cherry pick something (not trivial) and hope it works also in you engine. It is first mandatory to understand _why_ it works, and is not easy and then eventually to adopt the underlying _idea_ , the source code itself is useless as is and not reusable.

[bob]

One note. I believe the inflated piece values were a direct response to programs trading knight for 3 pawns and ending up in hopeless positions, and such. I did the "bad trade" idea in Crafty to avoid this, since the bad trade idea directly addresses the issue rather than indirectly thru modifying piece values.

I wonder to what extent it is that programs do not understand how to play with material differences, e.g. with 3 pawns vs. a piece, you need to use the pawns aggressively.

That is one thing that makes this tuning stuff so difficult. I remember many years ago that we simply could not come up with a scheme to handle some of the openings where the program would play g3/g6 and then Bg2/Bg7. The bishop is often critical, and trading it for a knight is generally not a good idea unless the knight is causing lots of problems where it stands. So we simply tuned the opening book to avoid such lines and did just fine (this was a Cray Blitz issue, by the way). Very early Crafty versions used the old CB book, but as I worked on king safety, slowly this problem went away. Yet the book avoided the Bg2 type positions and would instead go into something that became even more problematic.

Bottom line is that as the evaluation is modified, all terms suddenly become suspect. Sort of like optimizing for speed. As one peak gets driven down by optimizations you apply, others rise to take its place, and the process is actually never completed, just continually improved/refined...

It would be an interesting (and tedious) experiment to collect a few thousand relatively even positions with unbalanced material, e.g. N v PPP,
and tune a version of crafty specifically for those positions to see how much better it would play in those positions than regular crafty.

I feel you've been missing what happened to crafty past dozens of months.

With just 'a few positions' you aren't going to be able to approximate the millions of 'monte carlo type' datapoints crafty has already been tuned to by means of millions of games.

Assuming you don't fix the chessknowledge, but just tune parameters, you can already estimate that most likely the first few months in your experiment you will manage to lose an elo of 200 or so, not win anything.

Vincent

1)If you tune parameters in your program only for specific small set and
are stupid enough to use these value everywhere then
you may lose elo but I think that you are unable to lose 200 elo or even only 80 elo by tuning your program to play well in many N v PPP positions.

2)If you tune your program for N vs PPP positions then you do not need to use the values that you get everywhere.

You can have a preprocessor and use a different evaluation when the position in the board is of the type of N vs PPP.

Uri

That idea absolutely is fraught with problems. You do not want to turn things on and off at the drop of a hat. Around that discontinuity, the search will create strange positions that take advantage of crossing that "boundary" when it is convenient, or introducing horizon-effect type moves to avoid crossing the boundary when it is bad.

That's the very issue that led to the various ways of interpolating the scores between MG and EG, currently in vogue.

[Osipov Jury]

Larry, You can find my investigations of evaluation and search of Rybka 3 here:

http://strelkachess.narod.ru/rybka3.html

Sorry, unfortunately on Russian.

[Jan Brouwer]

Larry, You can find my investigations of evaluation and search of Rybka 3 here:

http://strelkachess.narod.ru/rybka3.html

Sorry, unfortunately on Russian.

I tried to read the Google translation into English, confusing but amusing!
A lot about elephants, arrows and spit

Jan

[Milos]

Larry, You can find my investigations of evaluation and search of Rybka 3 here:

http://strelkachess.narod.ru/rybka3.html

Sorry, unfortunately on Russian.

So what you inspected fully contradicts what Vincent's claims. You concluded that just taking R3 eval brings nothing at all, and speculate that the nice blend of search and evaluation in R3 has some (unmeasurable) contribution to it's strength.
On the other hand, Vincent claims Larry had not much influence in real eval, and that eval is automatically tuned to perfection in some CIA/NSA kind of environment and that it can bring 200 ELO...

[Osipov Jury]

Translation from Google into English:

Arrow, Spit = Strelka
Fish = Rybka
Elephant = Bishop
Parties = Games

[lkaufman]

"Cunning Fish proved that belopolny
and chernopolny elephants have a different assessment of the material!"

Could you perhaps translate that into English? If it means that the king and queen bishop have different values, that is true, but the difference is tiny and insignificant. If it refers to good and bad bishops, that concept was entirely missing from Rybka 2.xx and older versions, and is one of the clear improvements in Rybka 3.

[lkaufman]

Programs with the Rybka search are readily identifiable; they don't change their mind very often, and they are "too fast" in getting thru the plies, unless the count is non-standard as in Rybka herself. Surely if any of the existing programs had the Rybka-like search (excluding the recent R3 derivatives) someone would have commented on this by now. I'm recently certain that none of the leading programs that I have (Stockfish, Naum, Deep Shredder 12, Fritz 12, Hiarcs 12, Zappa Mexico, and of course Doch) do anything like the Rybka search.

It seems that very few who have looked at the Strelka or Ippolit code have understood the search, and those who have have not commented much about it. Nowhere have I read "The idea of Rybka's search is XXX" and apparently it's not because those who know the idea are using it in their own engines. All I've read is the general observation that Rybka devotes more resources to the main line and less to finding new moves.