book building for A using B

[Michael Sherwin]

I'll give you a hint. Your argument is just the very next pedantic step from the old and bad book statements from the Romi team years ago.

Romi's learning system was not intended for computer vs. computer. It was designed for human vs. computer sparring. The engine will play the human's winning lines back against the human as a psychological weapon and as a way to improve ones understanding of the line played. The other part of the learning system fishes around for ways to win just on its own without being shown.

I can not help it if everyone ignored my original intent to explore how well this system worked against other computers. And it does do quite well against other computers. Even if it is not all good.

At WBEC Romi P3k did not do well with the opening book that was given her to use. However, Romi P3k has now tuned that book well enough to barley gain a promotion against an ever strengthening pool of engines. Two of which are skipping the next higher section altogether.

And on top of that there are bugs in Romi's learning code that render a full third of her learn file as useless and worse, limiting. It would perform much more impressively if those bugs were fixed.

[BubbaTough]

You need to pose your question to experienced program authors, not the "general population" here. Unfortunately, there are not many of us left posting here, for various reasons. Take my advice as it was intended, or ignore it. That's your choice. But my advice is _not_ wrong here.

That is good advice...and what I am trying to do. I have some idea what Erdo does, since he asked for and received a private copy of LearningLemming to play with in his book experiments. I was curious about others. From listening to chats on ICC during the last tournament, there was some implications that there was a lot of leverage of Rybka or at least Rybka-Rybka Playchess games. I was hoping to hear from book pro's, but apparently you are right, not many book folks posting here.

-Sam

[BubbaTough]

It is not nearly as easy to "fix the engine" (try solving the king-safety issue, where your side has played g3 or g6, as an example) as it is to tune the book to avoid the positions where you tend to screw up, and instead head for the positions where you tend to play well.

If you take this to the max, why have a book at all? Can't you just fix the engine so it can play all possible openings equally well? Food for thought. And the answer reveals a lot about this overall issue, once you think about it for a minute. Then the light comes on, and you begin to understand the actual purpose of the opening book and how it is prepared.

Regarding how easy it is to "fix your engine", you are speaking to the choir. A huge portion of my eval was built trying to get it to understand opening positions that the learning function liked, but I knew it shouldn't. At first my program didn't understand fianchetto's (because of mobility when you stick your knight in front of the bishop). Then its pawn vs. development tuning was poor (it liked an early ...Qh4 in the scotch). Then there were a couple lines where white saqs a couple pawns to trap black's king in the center. Getting it to understand those issues was interesting. When 2 bishops are good, and when knights are good is always challenge to explain to the computer in terms it understands. The list goes on and on. Anyway, for me its more fun to tune the engine to understand chess, than to tune the book to appreciate the limits of my engine. Of course, that is a bit off topic.

Regarding why to have an opening book, well my perspective is a bit warped of course, since my "book" has always been learned, and in my opinion my "book" is probably inferior to most top books. But it seems to me there are two main reasons to have a book. Neither is to create positions that suite your program's "style", though I understand this perspective and grant it has merit. The first reason is to save time. It is a massive advantage to get to do 10 instant moves that your opponent has to think about. Without a book you are often granting your opponent this advantage. The second is a book gives you the benefit of having the results of deep, careful analysis stored that you can play instantly. Thus, in a five minute game you are playing moves of the quality you would only get with hours per move of time to analyze.

Anyway, that is the way it seems to me. Its quite possible you and Charles are right, that computer programs are still quite limited, not just at the Telepath or Crafty level but also at the top, and that top performance can only be achieved by crafting books that are tailored to the particular limitations of each program. I'm sure my own program's performance would benefit from a top book (which is not an interest of mine since tournament results is not my top priority). But at some point, I believe that will no longer be true. Top programs will at some point be well rounded enough that the best correlation in good results will be the objective quality of the position, not how well it fits its style. My question was posed really to ascertain whether book developers feel we are there yet.

-Sam

[bob]

You need to pose your question to experienced program authors, not the "general population" here. Unfortunately, there are not many of us left posting here, for various reasons. Take my advice as it was intended, or ignore it. That's your choice. But my advice is _not_ wrong here.

That is good advice...and what I am trying to do. I have some idea what Erdo does, since he asked for and received a private copy of LearningLemming to play with in his book experiments. I was curious about others. From listening to chats on ICC during the last tournament, there was some implications that there was a lot of leverage of Rybka or at least Rybka-Rybka Playchess games. I was hoping to hear from book pro's, but apparently you are right, not many book folks posting here.

-Sam

Some consider this a "trade secret" on a par with engine source code and internal algorithms being used. My goal is to make it the exact opposite, where everybody understands what is going on.

[bob]

It is not nearly as easy to "fix the engine" (try solving the king-safety issue, where your side has played g3 or g6, as an example) as it is to tune the book to avoid the positions where you tend to screw up, and instead head for the positions where you tend to play well.

If you take this to the max, why have a book at all? Can't you just fix the engine so it can play all possible openings equally well? Food for thought. And the answer reveals a lot about this overall issue, once you think about it for a minute. Then the light comes on, and you begin to understand the actual purpose of the opening book and how it is prepared.

Regarding how easy it is to "fix your engine", you are speaking to the choir. A huge portion of my eval was built trying to get it to understand opening positions that the learning function liked, but I knew it shouldn't. At first my program didn't understand fianchetto's (because of mobility when you stick your knight in front of the bishop). Then its pawn vs. development tuning was poor (it liked an early ...Qh4 in the scotch). Then there were a couple lines where white saqs a couple pawns to trap black's king in the center. Getting it to understand those issues was interesting. When 2 bishops are good, and when knights are good is always challenge to explain to the computer in terms it understands. The list goes on and on. Anyway, for me its more fun to tune the engine to understand chess, than to tune the book to appreciate the limits of my engine. Of course, that is a bit off topic.

Regarding why to have an opening book, well my perspective is a bit warped of course, since my "book" has always been learned, and in my opinion my "book" is probably inferior to most top books. But it seems to me there are two main reasons to have a book. Neither is to create positions that suite your program's "style", though I understand this perspective and grant it has merit. The first reason is to save time. It is a massive advantage to get to do 10 instant moves that your opponent has to think about. Without a book you are often granting your opponent this advantage. The second is a book gives you the benefit of having the results of deep, careful analysis stored that you can play instantly. Thus, in a five minute game you are playing moves of the quality you would only get with hours per move of time to analyze.

Anyway, that is the way it seems to me. Its quite possible you and Charles are right, that computer programs are still quite limited, not just at the Telepath or Crafty level but also at the top, and that top performance can only be achieved by crafting books that are tailored to the particular limitations of each program. I'm sure my own program's performance would benefit from a top book (which is not an interest of mine since tournament results is not my top priority). But at some point, I believe that will no longer be true. Top programs will at some point be well rounded enough that the best correlation in good results will be the objective quality of the position, not how well it fits its style. My question was posed really to ascertain whether book developers feel we are there yet.

-Sam

Playing a move instantly is important. But it _must_ be a move that leads to something the program understands, or it is no good.

As far as having "deep analysis" that doesn't work very well. The branching factor is so large, following a book line very deep is only likely when both programs / opponents use the same opening systems and are trying to out-book each other. Generally, in CC tournaments, we see long book lines, where one side is always very unhappy with the position that they have when they leave book, because of opponent preparation.

A rely on the book for a completely different reason. I want to be able to steer the games away from potential "cooking" in important events, so that the programs play the game, not the book authors.

[BubbaTough]

Playing a move instantly is important. But it _must_ be a move that leads to something the program understands, or it is no good.

As far as having "deep analysis" that doesn't work very well. The branching factor is so large, following a book line very deep is only likely when both programs / opponents use the same opening systems and are trying to out-book each other. Generally, in CC tournaments, we see long book lines, where one side is always very unhappy with the position that they have when they leave book, because of opponent preparation.

A rely on the book for a completely different reason. I want to be able to steer the games away from potential "cooking" in important events, so that the programs play the game, not the book authors.

The out-booking issue is a good point. It is one of the reasons I started suspecting people were using Rybka games to make their books. They wouldn't be getting these lost positions first move out of book deep into the game if they were using their own engines to tune their book instead of copying some Rybka game would they? Anyway I agree with you its much more fun when everyone is out of book early. One of my favorite tournament memories was a LearningLemming - Crafty game in some ACCA event where we were out of book on move 2 or something, and Lemming came up with some weird gambit idea around move 4 or something. Those games really let you see what the engines think about chess, instead of letting the books hide most of an engine's opinions by getting their kings castled and their pieces in decent spots before they start thinking.

-Sam

[bob]

Playing a move instantly is important. But it _must_ be a move that leads to something the program understands, or it is no good.

As far as having "deep analysis" that doesn't work very well. The branching factor is so large, following a book line very deep is only likely when both programs / opponents use the same opening systems and are trying to out-book each other. Generally, in CC tournaments, we see long book lines, where one side is always very unhappy with the position that they have when they leave book, because of opponent preparation.

A rely on the book for a completely different reason. I want to be able to steer the games away from potential "cooking" in important events, so that the programs play the game, not the book authors.

The out-booking issue is a good point. It is one of the reasons I started suspecting people were using Rybka games to make their books. They wouldn't be getting these lost positions first move out of book deep into the game if they were using their own engines to tune their book instead of copying some Rybka game would they? Anyway I agree with you its much more fun when everyone is out of book early. One of my favorite tournament memories was a LearningLemming - Crafty game in some ACCA event where we were out of book on move 2 or something, and Lemming came up with some weird gambit idea around move 4 or something. Those games really let you see what the engines think about chess, instead of letting the books hide most of an engine's opinions by getting their kings castled and their pieces in decent spots before they start thinking.

-Sam

Here's the problem you are overlooking. I prepare a book line and play it against you. If I have your program handy, I can even do this in total privacy. I look for places where there is a tactical opportunity to (say) offer you a pawn that is nearly impossible to resist. And I work on that line manually, using any program I want because all I am hunting for is a tactical line that you can't see. And I verify that you can't see it because I have your program handy or play it on ICC. And I know exactly what you are going to play because it is too tempting to be able to refuse unless you can pull off a 30+ ply search which is not going to happen. The problem is, once you are "in for a nickle, you are in for a dime" and are committed. And dead lost if my preparation was solid. We have seen WCCC games end in a mate where one side was still in book.

On the other hand, if I am simply trying to choose opening lines to play, I use my program only, because if it is not happy with a position, I don't want it to play that opening. I try to be creative with my tournament books so that the previously mentioned cooked preparation doesn't work because I don't play an opening I have previously played to prevent this.