Question for Bob Hyatt

[Osipov Jury]

"Cunning Fish proved that belopolny
and chernopolny elephants have a different assessment of the material!"

Could you perhaps translate that into English? If it means that the king and queen bishop have different values, that is true, but the difference is tiny and insignificant. If it refers to good and bad bishops, that concept was entirely missing from Rybka 2.xx and older versions, and is one of the clear improvements in Rybka 3.

Rybka 3 has different indexes in material imbalance table for white-square and black-square bishops. I don't know what is "good" or "bad" bishop. I can't find this items in Rybka 3.

[Osipov Jury]

Larry, You can find my investigations of evaluation and search of Rybka 3 here:

http://strelkachess.narod.ru/rybka3.html

Sorry, unfortunately on Russian.

So what you inspected fully contradicts what Vincent's claims. You concluded that just taking R3 eval brings nothing at all, and speculate that the nice blend of search and evaluation in R3 has some (unmeasurable) contribution to it's strength.
On the other hand, Vincent claims Larry had not much influence in real eval, and that eval is automatically tuned to perfection in some CIA/NSA kind of environment and that it can bring 200 ELO...

I think, Vincent does not have the sources of Rybka 3.

[lkaufman]

Good and bad bishops refer to bishops of the same color as friendly pawns or of the opposite color. Rybka 3 considers not only the number of such pawns, but what file they are on and whether or not they are blockaded. This is pretty much the same thing that textbooks on chess teach human players. Basically my job in Rybka 3 was to propose ideas known in human chess that were not already in the program, propose values for those new terms, and then try to optimize those values by hyperspeed testing, while periodically reviewing the weights of old terms since they are all interrelated.

[Osipov Jury]

Good and bad bishops refer to bishops of the same color as friendly pawns or of the opposite color. Rybka 3 considers not only the number of such pawns, but what file they are on and whether or not they are blockaded.

Perhaps this will be implemented in the next version of Rybka.

[lkaufman]

It most certainly is implemented in Rybka 3. Maybe it's disguised in such a way that you cannot recognize it. I think it only applies when one side has just a single bishop, since with two bishops the terms would cancel out.

[diep]

I can't duplicate the experiment precisely without involving Don, whom I'd rather not distract from improving Doch with this, but I may be able to roughly simulate it myself when I have time. But for me the big question is this: If this vastly superior search has been know for a couple of years now due to Strelka, and if many of the programmers share your view that it was the main reason for Rybka's superiority (I believe many do), then why is it that not a single strong program (other than those that are Rybka clones or derivatives) has implemented some form of this type of search (i.e. safety-first, focus on the PV)? At least I don't know of any that show any signs of doing this. Is the reason:
A. No one understand the Strelka search
B. Every programmer was too lazy to do this?
C. Every programmer refused to take Rybka ideas on principle?
D. Programmers tried it, but it did not help their own programs.
E. Some other reason/reasons.

Unlike what you might guess most of the top programmers in general are not big copycats.

but if i had to pick one then: E) ippolit has a weaker search and weaker evaluation function than rybka3.

it seems rybka has mating extensions similar to zappa 2005 (and before) had them and obviously ippolit doesn't have them.

rybka has more knowledge somewhere that's not in ippolit. This wins game after game against the 'clones'.

in itself such knowledge is not new, but it seems rather well tuned. in contradiction there the huge knowledge i have there in diep seems rather buggy and somehow i lose game after game to just this feature with diep to rybka, despite that on paper diep has a lot more knowledge there. So that just sees how important debugging it is and tuning it right.

ippolit obviously is the same source code base as rybka but has been deliberately weakened therefore. the proof that it derives from the same source code base has been delivered by a few English chaps already. maybe they want to post it publicly.

Knowing it is the same source code base but that 2 features that win bigtime elo, especially at longer time controls, have been left out we can further study it and see that it has different parts.

I have some ideas on how it is possible the source code leaked out but i'm not willing to share confidential information that someone privately told me. I tend to believe it was a rather innocent action after which it got in the hands of a non-programmer who is not so innocent, whom will be really tough to figure out, if we ever would want to. That chap, being a compiler guy possibly (and no not Nalimov) most likely comes from Russia (as most compiler guys come from there) therefore speaks Russian a tad and then it ended up in the hands of one of the team dudes who speaks at his work a tad Russian and that's how it got done. So no this never happened in Russia i suspect. Yet that'll explain ippolit and obviously also why it was weakened first.

I support the opinion of some russian programmer posted here (was it yuri usipov?) that it is different programmers who obviously made different parts of rybka. In ippolit this is very clearly visible. Whether that has been done over time i don't know. The parameter tuning of material seems a neural network to me. the parameter tuning of the other parameters seems more thoroughly tested. how to achieve that is another good question.

It scales better at long time controls elowise than the 'clones'. What causes this i have an idea there that has to do with some theoretic game tree search theory; but then i would reveal an old algorithm i invented one day (somewhere mid 90s) which i want to revive again as nowadays i feel it is possible to do more with your nps than in the 90s where nothing worked.

Yet it indicates that the testing method applied is rather pragmatic and pretty much based upon long time controls.

Knowing most 'testers' test short time controls (if not then because of limited core counts) they each time get lured a little bit that something 'might' be equal strong, some new clone, but each time that gets disproven easily by throwing more serious testing time against it of course.

Costalba noticed he saw nothing interesting in the evaluation function, yet he's missing what brings elo there and additional to that he's looking in ippolit rather than rybka 2.3.2 or 3.0. Speakig of someone who WOULD cut'n paste possibly.

Furthermore Costalba is missing the parallel search that Tord has built and ippolit doesn't have it. How can you miss that?

As has been proven before it is relative easy to tune an engine close to the elostrength to another engine, provided you don't go test at a bunch of cores with newer hardware, which is what happens each time; therefore the clones can't keep up of course. It is like cloning DNA. It nearly looks like it, but each time goes wrong.

The question is whether free source code from stockfish floating around, if i'd add 10 things in its evaluation function which work very well in diep and retune it a tad for stockfish, it's like +200 elo.

So releasing that code is rather dangerous IMHO.

SMK still is right of course with his 2004 statement.

"If i have an evaluation feature in Shredder that others do not have, you win every game based upon it".

It's right under the nose of everyone yet they're missing it.

[diep]

Programs with the Rybka search are readily identifiable; they don't change their mind very often, and they are "too fast" in getting thru the plies, unless the count is non-standard as in Rybka herself. Surely if any of the existing programs had the Rybka-like search (excluding the recent R3 derivatives) someone would have commented on this by now. I'm recently certain that none of the leading programs that I have (Stockfish, Naum, Deep Shredder 12, Fritz 12, Hiarcs 12, Zappa Mexico, and of course Doch) do anything like the Rybka search.

If you prune everything except the mainline you hardly ever change your mind quickly of course once hashtables are filled during a game.
That's logics everyone should understand.

It seems that very few who have looked at the Strelka or Ippolit code have understood the search, and those who have have not commented much about it. Nowhere have I read "The idea of Rybka's search is XXX" and apparently it's not because those who know the idea are using it in their own engines.

Pardon me?

You really guess people would post that here on CCC?

All I've read is the general observation that Rybka devotes more resources to the main line and less to finding new moves.

Which explains exactly what you described before.

Vincent

[diep]

Larry, You can find my investigations of evaluation and search of Rybka 3 here:

http://strelkachess.narod.ru/rybka3.html

Sorry, unfortunately on Russian.

So what you inspected fully contradicts what Vincent's claims. You concluded that just taking R3 eval brings nothing at all, and speculate that the nice blend of search and evaluation in R3 has some (unmeasurable) contribution to it's strength.
On the other hand, Vincent claims Larry had not much influence in real eval, and that eval is automatically tuned to perfection in some CIA/NSA kind of environment and that it can bring 200 ELO...

I think, Vincent does not have the sources of Rybka 3.

Look it's obvious what i try to accomplish with Diep, that's putting in a huge amount of chessknowledge.

In such untuned evaluation what is really difficult to do is something like:

if( depthleft <= 3.5 ply
&& eval >= beta + 1.2 pawn ) then return eval;

// trying nullmove
R=3;
if( eval >= beta + 1.5 pawn ) R++;

So if its evaluation is 1 pawn off from reality it's going to miss really a lot, besides the reductions also that are in between it directly skips 4.5 ply that most engines do not skip searching.

That means basically that the way to beat rybka is to have MORE chessknowledge. It's entire mainline checking is going to miss everything then.

You can prove easily that its success depends upon having slightly better knowledge than other 'beancounter' engines.

Just 1 tiny thing is enough, according to SMK's lemma.

Mainline checking is walking on a very thing layer of ice. So the release of ippolit's source code is, looking backwards, a big blessing, as now others didn't accurately compare rybka3 with ippolit; missing some evaluation features that scores point after point.

Vincent

[bob]

Programs with the Rybka search are readily identifiable; they don't change their mind very often, and they are "too fast" in getting thru the plies, unless the count is non-standard as in Rybka herself. Surely if any of the existing programs had the Rybka-like search (excluding the recent R3 derivatives) someone would have commented on this by now. I'm recently certain that none of the leading programs that I have (Stockfish, Naum, Deep Shredder 12, Fritz 12, Hiarcs 12, Zappa Mexico, and of course Doch) do anything like the Rybka search.

It seems that very few who have looked at the Strelka or Ippolit code have understood the search, and those who have have not commented much about it. Nowhere have I read "The idea of Rybka's search is XXX" and apparently it's not because those who know the idea are using it in their own engines. All I've read is the general observation that Rybka devotes more resources to the main line and less to finding new moves.

Some have looked pretty carefully, as they discussed the TT table search extension idea among others. But I do not know who has tried that idea or how it worked.

There are two angles to a search. (1) you can try to replace the PV by doing lots of work on non-PV branches, in an effort to find something that fails high. (2) you can try to search the PV more carefully, and force it to fail low so that you will automatically switch to a different move. Which is better is a guess. But there is something about (2) that leaves me feeling it is a bad idea, as you keep an OK pv and miss a better move, which is (IMHO) worse than finding a better move without over-searching the PV space. It just doesn't "feel right" to me. May be a great idea. And things like LMR and forward pruning tend to behave like that since later moves get reduced/pruned more. But still, there are probably limits to this approach.

[BubbaTough]

If you prune everything except the mainline you hardly ever change your mind quickly.

A lot of programs...probably most of the top programs...do a lot of LMR, null move, razoring/pruning stuff in the search and not in the PV. This is not unique to Rybka, nor did it start with Rybka. It does have the effect you describe. I don't think Rybka 1 did this anymore aggressively than many others. When I looked at strelka, its search did not look particularly innovative to me, just very efficiently coded. In my conversations with Anthony Cozzie his conclusions matched mine. Rybka 3, on the other hand, is a different beast, and might do things more aggressively, I am not sure not having looked at it or the programs it "inspired" that closely.

-Sam