book building for A using B

[Edsel Apostol]

Theoretically, using Rybka to tune your book would be best, reason being that it is the best available engine out there.

I say wrong. Tuning for Rybka could easily hurt and engine's performance assuming that the engine has a different personality/style
from Rybka. There are many positions that Rybka will play well and other programs will not assuming they are not Rybka clones.
Thus, it does an engine no good to end up in a position that it can not play well regardless of how well Rybka plays that position.

As far as finding bad positions goes, yes, Rybka losing to weak opponents says its possibly a bad position. Then you also end up with
positions that only Rybka can play well which is not what you should want.

Lines that Rybka favored have more chance to win than lines favored by let's say an engine around 2500 elo. Claiming that there will be a conflict of playing style is just an assumption. If you end up in a winning position favored by the best engine out there and your engine loses that game, then it is not the book to blame but the engine, because it was playing crap on a winning position.

Our goal is to have the best book and not to have the book at the level of our engine. Our engine is not that strong yet so we are not that confident in tuning with it for now.

My theory is that when you tune your book to your engine, the quality of the book will only reach the quality of your engine. If your engine is only 2500, the book will only reach that level. We are not trying to have the book at our engine's level. We want a book that is something better than that.

It has worked in our engine, it might not work for you. My another theory is that at the top, I mean engines within 250 elo of the best engine, most of those engines would agree that a winning line is a winning line. Therefore tuning with Rybka will only be feasible if you are somewhere within 250 elo of Rybka. If your engine is way out of that league, then tuning for your own engine's quirkiness might give you better results.

This is simply a mistake and it is easy to see why. Suppose Rybka does better in certain types of openings, such as QGD, or black against the Ruy Lopze, because its evaluation is better than yours. If you play the openings it plays best, you get into lines where you play something like g6/Bg7, but don't realize how critical that Bg7 is. You trade it away, get attacked, and lose. Rybka, knowing that if you play Bg7 you had better hang on to it, doesn't make that mistake. Why would you possibly want to play opening systems that your program mishandles? That's not the way Kure, Noomen, et. al. prepare their books. They specifically search for lines where the program they are supporting does well. It is the only way that makes sense.

I am not an expert on this book building so I'm just saying what is logical for me. I will stick to my argument that when you tune the book to a specific engine, the quality of that book will only be equal to the quality of the engine. If one has a 2200 elo engine and you tune the book with it, most probably the quality of that book will only be 2200. At least if one is to use a book that was tuned for the best engine, even if it is only 2200 elo engine, it will be using a 3000 elo book and it might have a bigger chance of winning or might at least get a draw.

Let's say a 2200 elo engine with a self-tuned 2200 elo book played against a 3000 elo engine. The 2200 elo engine has a position out of the book that it thinks it is winning but for the 3000 elo engine it is seeing that it is a win for itself (or maybe even if it is only an equal position it sees for itself). The 3000 elo engine would certainly crush the 2200 elo engine immediately.

Let's say the 2200 elo engine uses a book tuned for the 3000 elo engine and it has a position out of the book that is winning, do you think that the 3000 elo engine would crush the 2200 elo engine immediately as the position out of the book is losing for the 3000 elo engine? No.

What if we are using a 2800 elo engine instead of a 2200 elo engine and it has a position out of the 3000 elo book that is winning? Do you think that the 3000 elo engine could easily beat the 2800 engine? No. The 2800 elo engine might not win but it would have at least a chance to draw.

That's basically the idea on using books that are tuned for the top engines. You have a good point by the way, and I'm not sure which is the best method, maybe combining both would give a much better benefit (I think this is what Audy, my operator and tester, is doing as he tuned the book using top engines and he is adjusting manually the positions where our own engine doesn't play well). There is no certainly a black and white solution, it's most probably grey.

[BubbaTough]

I'd be willing to bet that 99% of book building is done with "own engine". Otherwise you choose lines that are good for the other engine, but not for yours. This is how it has been done since early computer chess days...

Well, I am quite curious if this is still true. It always seemed to me that if a position in your book was good for you but that your engine played it poorly, you should fix the engine not the book. But that's just me. When I started this thread I already knew the Team TwistedLogic and Team Crafty perspective. Now team Telepath has thrown in its 2 cents. Are there any other book builders that would like to share their perspective? Is Bob right, do 99% of book builders just use their own engine, just like in the good old days?

-Sam

[BubbaTough]

I think most book builders use Rybka to build or tune their books. But in my case, if I have a game where TwistedLogic defeats a stronger opponent such as Naum or Rybka, then I also use that game to tune the book since those moves are for Twisted's playing style. And those are the games that are more important to me for the book.

Theoretically, using Rybka to tune your book would be best, reason being that it is the best available engine out there. Thats why when I do play my newly constructed book with Rybka and it loses to a weaker opponent, then that means that something is wrong with that line...

How do you know it was a weaker opponent? it could have been an account that was using Rybka without telling you.

Miguel

Hehe, nice one Miguel . Are there any other book builders out there that want to reveal their technique? I know humans use a lot of Rybka in their "book" preparation, I am curious if computer book builders are different.

-Sam

This is a naive statement. Humans use Rybka to tactically analyze specific variations. I don't think a human just says "OK, Rybka plays this series of moves, so I am going to mimic that exactly.

If you really get into building a sophisticated book, you had _better_ use the engine that is going to use the book. Otherwise, disasters are right around the corner.

Hmmm....regarding what humans do and don't do, I have some knowledge on this subject having chatted with GMs about it, so don't really need to rely too much on logic. There is nothing naive about my perspective...its pretty data driven. I think your opinions are reasonable, I just wish you would show others the same respect I try to show you. Wasting time arguing about how dumb or naive I am is annoying and a waste of time for me, and makes me want to go back into passive mode where I just lift ideas from others and contribute nothing. I don't think thats good for anyone.

Regarding how computers can/do/should use resources like Rybka, well you have made your perspective quite clear.

-Sam

[CRoberson]

The problem here is that you are thinking at a "high" level and not considering any of the "corner cases" which are quite an issue. From
my experience, I can tell you that there are only two ways in which your ancient idea works.

1) You clone the engine you are using to tune the book.
2) Your book is very deep, so "won position" means that you are up material in a strong position.

Point number 1 works until you get caught. Point number 2 only works some of the time. The refutation of 2 is very simple.
I'll leave it to you to figure out or experience.

I'll give you a hint. Your argument is just the very next pedantic step from the old and bad book statements from the Romi team years ago.

[BubbaTough]

The problem here is that you are thinking at a "high" level and not considering any of the "corner cases" which are quite an issue. From
my experience, I can tell you that there are only two ways in which your ancient idea works.

1) You clone the engine you are using to tune the book.
2) Your book is very deep, so "won position" means that you are up material in a strong position.

Point number 1 works until you get caught. Point number 2 only works some of the time. The refutation of 2 is very simple.
I'll leave it to you to figure out or experience.

I'll give you a hint. Your argument is just the very next pedantic step from the old and bad book statements from the Romi team years ago.

I don't quite understand what you are saying, could you elaborate? It seems like your thesis is that chess computers are extremely unique, that each one plays some positions great and others terrible, and thus engine A can only play positions that are good for engine B if it is a clone. Is that a correct interpretation?

-Sam

[CRoberson]

The problem here is that you are thinking at a "high" level and not considering any of the "corner cases" which are quite an issue. From
my experience, I can tell you that there are only two ways in which your ancient idea works.

1) You clone the engine you are using to tune the book.
2) Your book is very deep, so "won position" means that you are up material in a strong position.

Point number 1 works until you get caught. Point number 2 only works some of the time. The refutation of 2 is very simple.
I'll leave it to you to figure out or experience.

I'll give you a hint. Your argument is just the very next pedantic step from the old and bad book statements from the Romi team years ago.

I don't quite understand what you are saying, could you elaborate? It seems like your thesis is that chess computers are extremely unique, that each one plays some positions great and others terrible, and thus engine A can only play positions that are good for engine B if it is a clone. Is that a correct interpretation?

-Sam

There is more to it than that. The refutation to point 2 is the one you aren't fully understanding.

[bob]

I think most book builders use Rybka to build or tune their books. But in my case, if I have a game where TwistedLogic defeats a stronger opponent such as Naum or Rybka, then I also use that game to tune the book since those moves are for Twisted's playing style. And those are the games that are more important to me for the book.

Theoretically, using Rybka to tune your book would be best, reason being that it is the best available engine out there. Thats why when I do play my newly constructed book with Rybka and it loses to a weaker opponent, then that means that something is wrong with that line...

How do you know it was a weaker opponent? it could have been an account that was using Rybka without telling you.

Miguel

Hehe, nice one Miguel . Are there any other book builders out there that want to reveal their technique? I know humans use a lot of Rybka in their "book" preparation, I am curious if computer book builders are different.

-Sam

This is a naive statement. Humans use Rybka to tactically analyze specific variations. I don't think a human just says "OK, Rybka plays this series of moves, so I am going to mimic that exactly.

If you really get into building a sophisticated book, you had _better_ use the engine that is going to use the book. Otherwise, disasters are right around the corner.

Hmmm....regarding what humans do and don't do, I have some knowledge on this subject having chatted with GMs about it, so don't really need to rely too much on logic. There is nothing naive about my perspective...its pretty data driven. I think your opinions are reasonable, I just wish you would show others the same respect I try to show you. Wasting time arguing about how dumb or naive I am is annoying and a waste of time for me, and makes me want to go back into passive mode where I just lift ideas from others and contribute nothing. I don't think thats good for anyone.

Regarding how computers can/do/should use resources like Rybka, well you have made your perspective quite clear.

-Sam

If you had only been around long enough, this discussion would not be going on. Alex Kure used to post here. Jeroen Noomen as well. Discussing book preparation has been a regular topic in the past, both here and in r.g.c.c....

Books go with a program, they are not separate projects with no connection. What is good for Rybka is not necessarily good for X. The inverse is also true. As a human, I don't play an opening system just because a human says it is good. I can only imagine what Korchnoi might have said when someone suggested "You have got to try 1. e4, Kasparov does great with this..." He would probably respond, "I will play e4 when you pry my cold, dead fingers from around my trusty d-pawn."

There are some things in computer chess that are (or should be) intuitively obvious to the casual observer. And this _really_ is one of those. If you follow the previous line of reasoning and prepare a 3000-level book and use it with a 2500-level program, the book is not going to pull the program closer to 3000. It is most likely going to pull the program down below 2500. A book really does need to be tuned to the program that is using it, with the sole exception of "cooking" where you tune a book to favor your program and to attack the weaknesses of a specific opponent or lay a trap for a specific opponent, where the move choice selection is not based on "best play" but rather is based on "best play against a specific opponent." And this is a dangerous practice, for the same reason as using a different program to tune the book than what will be actually using the book. A trap that works on X might lose you a game if you try it on Y.

But as I said, this is all well-known. It isn't new. It isn't original. It isn't mysterious. And it most certainly is common knowledge against people that have been dealing with computer chess for any length of time. So I am not sure why this discussion was even started, because the answer to the original question was known 40 years ago and it still hasn't changed today. The book and the program that uses the book create a symbiotic entity that are mutually complementary...

You need to pose your question to experienced program authors, not the "general population" here. Unfortunately, there are not many of us left posting here, for various reasons. Take my advice as it was intended, or ignore it. That's your choice. But my advice is _not_ wrong here.

[bob]

Theoretically, using Rybka to tune your book would be best, reason being that it is the best available engine out there.

I say wrong. Tuning for Rybka could easily hurt and engine's performance assuming that the engine has a different personality/style
from Rybka. There are many positions that Rybka will play well and other programs will not assuming they are not Rybka clones.
Thus, it does an engine no good to end up in a position that it can not play well regardless of how well Rybka plays that position.

As far as finding bad positions goes, yes, Rybka losing to weak opponents says its possibly a bad position. Then you also end up with
positions that only Rybka can play well which is not what you should want.

Lines that Rybka favored have more chance to win than lines favored by let's say an engine around 2500 elo. Claiming that there will be a conflict of playing style is just an assumption. If you end up in a winning position favored by the best engine out there and your engine loses that game, then it is not the book to blame but the engine, because it was playing crap on a winning position.

That is theory. But reality says you have to steer the game toward positions your program can play well, and away from positions that it can't. Not the other way around. Unless your goal is to create a book that causes your overall rating to _drop_, which is where this is going to lead you as you explain what you are doing.

Our goal is to have the best book and not to have the book at the level of our engine. Our engine is not that strong yet so we are not that confident in tuning with it for now.

Unfortunately, inexperience is leading you down a faulty path. A book _complements_ a chess program. If the book takes the engine into positions where it plays poorly, that is _not_ going to make the program look better. It is going to cause it to loose games and look foolish in doing so.

My theory is that when you tune your book to your engine, the quality of the book will only reach the quality of your engine. If your engine is only 2500, the book will only reach that level. We are not trying to have the book at our engine's level. We want a book that is something better than that.

That is plain and simple wrong thinking. The book either complements the style of play the engine is capable of, or it is opposed to that style of play. A GM might like gambits in some cases. But he would not tell a 1400-level player to play those gambits, since he is neither tactically capable, nor positionally capable, of taking advantage of the tempo advantage the loss of material offers. And all that would do is drag the 1400 level player even lower rating-wise.

It has worked in our engine, it might not work for you. My another theory is that at the top, I mean engines within 250 elo of the best engine, most of those engines would agree that a winning line is a winning line. Therefore tuning with Rybka will only be feasible if you are somewhere within 250 elo of Rybka. If your engine is way out of that league, then tuning for your own engine's quirkiness might give you better results.

Most of us would _not_ agree with that idea. Some engines handle kingside attacks better, some excel in the endgame, some excel with passed pawns, some excel with defending against attacks, etc. Each program has strengths and weaknesses. The book has to complement the strengths, and avoid the weaknesses, or it is a bad book.

In theory, what you say might be right. In reality, it is a very bad idea at present. Take that with whatever quantity of salt you want, but you ought to ask real book authors (Kure, Noomen, etc). This is not exactly a new topic, and these discussions have been popping up for 30 years now.

This is simply a mistake and it is easy to see why. Suppose Rybka does better in certain types of openings, such as QGD, or black against the Ruy Lopze, because its evaluation is better than yours. If you play the openings it plays best, you get into lines where you play something like g6/Bg7, but don't realize how critical that Bg7 is. You trade it away, get attacked, and lose. Rybka, knowing that if you play Bg7 you had better hang on to it, doesn't make that mistake. Why would you possibly want to play opening systems that your program mishandles? That's not the way Kure, Noomen, et. al. prepare their books. They specifically search for lines where the program they are supporting does well. It is the only way that makes sense.

I am not an expert on this book building so I'm just saying what is logical for me. I will stick to my argument that when you tune the book to a specific engine, the quality of that book will only be equal to the quality of the engine. If one has a 2200 elo engine and you tune the book with it, most probably the quality of that book will only be 2200. At least if one is to use a book that was tuned for the best engine, even if it is only 2200 elo engine, it will be using a 3000 elo book and it might have a bigger chance of winning or might at least get a draw.

Let's say a 2200 elo engine with a self-tuned 2200 elo book played against a 3000 elo engine. The 2200 elo engine has a position out of the book that it thinks it is winning but for the 3000 elo engine it is seeing that it is a win for itself (or maybe even if it is only an equal position it sees for itself). The 3000 elo engine would certainly crush the 2200 elo engine immediately.

You miss the key though, that if you use the 3000 engine's book on that 2200 engine, it will likely lose even quicker, because it plays right into the strength of that opponent, while it is not capable of handling the resulting positions reasonably.

Let's say the 2200 elo engine uses a book tuned for the 3000 elo engine and it has a position out of the book that is winning, do you think that the 3000 elo engine would crush the 2200 elo engine immediately as the position out of the book is losing for the 3000 elo engine? No.

Absolutely yes. The instant you leave book, you are once again a 2200 player, most likely playing in positions you do not understand, and therefore do not like. A recipe for a disaster.

What if we are using a 2800 elo engine instead of a 2200 elo engine and it has a position out of the 3000 elo book that is winning? Do you think that the 3000 elo engine could easily beat the 2800 engine? No. The 2800 elo engine might not win but it would have at least a chance to draw.

The closer they are, the more important the book lines become. Again, ask Kure or Noomen. They have done this for a long time (nowhere near as long as I have, but long enough). You seem to be missing the point that you can't drop a 2200 engine into a position it naturally plays poorly, and expect anything but a train wreck. The book has to complement the engine, otherwise it is going to do more harm than good.

That's basically the idea on using books that are tuned for the top engines. You have a good point by the way, and I'm not sure which is the best method, maybe combining both would give a much better benefit (I think this is what Audy, my operator and tester, is doing as he tuned the book using top engines and he is adjusting manually the positions where our own engine doesn't play well). There is no certainly a black and white solution, it's most probably grey.

[bob]

The problem here is that you are thinking at a "high" level and not considering any of the "corner cases" which are quite an issue. From
my experience, I can tell you that there are only two ways in which your ancient idea works.

1) You clone the engine you are using to tune the book.
2) Your book is very deep, so "won position" means that you are up material in a strong position.

Point number 1 works until you get caught. Point number 2 only works some of the time. The refutation of 2 is very simple.
I'll leave it to you to figure out or experience.

I'll give you a hint. Your argument is just the very next pedantic step from the old and bad book statements from the Romi team years ago.

I don't quite understand what you are saying, could you elaborate? It seems like your thesis is that chess computers are extremely unique, that each one plays some positions great and others terrible, and thus engine A can only play positions that are good for engine B if it is a clone. Is that a correct interpretation?

-Sam

dead on.

[bob]

I'd be willing to bet that 99% of book building is done with "own engine". Otherwise you choose lines that are good for the other engine, but not for yours. This is how it has been done since early computer chess days...

Well, I am quite curious if this is still true. It always seemed to me that if a position in your book was good for you but that your engine played it poorly, you should fix the engine not the book. But that's just me. When I started this thread I already knew the Team TwistedLogic and Team Crafty perspective. Now team Telepath has thrown in its 2 cents. Are there any other book builders that would like to share their perspective? Is Bob right, do 99% of book builders just use their own engine, just like in the good old days?

-Sam

It is not nearly as easy to "fix the engine" (try solving the king-safety issue, where your side has played g3 or g6, as an example) as it is to tune the book to avoid the positions where you tend to screw up, and instead head for the positions where you tend to play well.

If you take this to the max, why have a book at all? Can't you just fix the engine so it can play all possible openings equally well? Food for thought. And the answer reveals a lot about this overall issue, once you think about it for a minute. Then the light comes on, and you begin to understand the actual purpose of the opening book and how it is prepared.