Crafty Book Creation

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

jhaglund
Posts: 173
Joined: Sun May 11, 2008 7:43 am

Crafty Book Creation

Post by jhaglund »

Can you add to the book creating of Crafty, so it also includes sub-variations, such as annotated pv lines in a PGN?

Example:

Code: Select all

[Event "?"]
[Site "A00"]
[Date "????.??.??"]
[Round "?"]
[White "Grob"]
[WhiteElo ""]
[Black "Fritz gambit"]
[BlackElo ""]
[Result "*"]
[Annotator "Crafty v23.2 JA"]
{annotating both black and white moves.}
{using a scoring margin of +0.01 pawns.}
{search time limit is 2.00}

  1.      g4
                ({14:-0.53}  1. g4 d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3 $15)
                ({14:+0.13}  1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 $10)
                ({suggested 14:-0.38} g4 d5 g5 e5 d3 Bb4+ Bd2 Nc6 Nf3 Bxd2+ Qxd2 Nge7 $15)
                ({suggested 14:+0.13} Nf3 Nc6 e4 e6 Nc3 Nf6 Bb5 Bb4 O-O O-O $10)

  1.     ...      d5
                ({suggested 12:-0.51} d5 g5 h6 d4 hxg5 Bxg5 f6 Bd2 Nc6 Nf3 e5 e3 $15)
                ({suggested 12:-0.48} Nc6 Nf3 e5 d4 exd4 Nxd4 Nf6 Nxc6 dxc6 Qxd8+ Kxd8 g5 Ne4 $15)
  2.     Bg2
*

Normally, it would only include: 1. g4 d5 2. Bg2
I'd like it to grab all the evaluations made.

Maybe by...
string b[2] = "({";
if(b)
then

So it would actually include:

Code: Select all

1. g4 d5 2. Bg2

1. d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3

1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 

1. g4 d5 2. g5 h6 3. d4 hxg5 4. Bxg5 f6 5. Bd2 Nc6 6. Nf3 e5 7. e3 

1. g4 d5 2. g5 e5 3. d3 Bb4+ 4. Bd2 Nc6 5. Nf3 Bxd2+ 6. Qxd2 Nge7 

1. Nf3 Nc6 2. e4 e6 3. Nc3 Nf6 4. Bb5 Bb4 5. O-O O-O 

1. Nc6 Nf3 2. e5 d4 3. exd4 Nxd4 4. Nf6 Nxc6 5. dxc6 Qxd8+ 6. Kxd8 g5 7. Ne4 
What do you think Bob? Any chance you'd consider?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Crafty Book Creation

Post by bob »

jhaglund wrote:Can you add to the book creating of Crafty, so it also includes sub-variations, such as annotated pv lines in a PGN?

Example:

Code: Select all

[Event "?"]
[Site "A00"]
[Date "????.??.??"]
[Round "?"]
[White "Grob"]
[WhiteElo ""]
[Black "Fritz gambit"]
[BlackElo ""]
[Result "*"]
[Annotator "Crafty v23.2 JA"]
{annotating both black and white moves.}
{using a scoring margin of +0.01 pawns.}
{search time limit is 2.00}

  1.      g4
                ({14:-0.53}  1. g4 d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3 $15)
                ({14:+0.13}  1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 $10)
                ({suggested 14:-0.38} g4 d5 g5 e5 d3 Bb4+ Bd2 Nc6 Nf3 Bxd2+ Qxd2 Nge7 $15)
                ({suggested 14:+0.13} Nf3 Nc6 e4 e6 Nc3 Nf6 Bb5 Bb4 O-O O-O $10)

  1.     ...      d5
                ({suggested 12:-0.51} d5 g5 h6 d4 hxg5 Bxg5 f6 Bd2 Nc6 Nf3 e5 e3 $15)
                ({suggested 12:-0.48} Nc6 Nf3 e5 d4 exd4 Nxd4 Nf6 Nxc6 dxc6 Qxd8+ Kxd8 g5 Ne4 $15)
  2.     Bg2
*

Normally, it would only include: 1. g4 d5 2. Bg2
I'd like it to grab all the evaluations made.

Maybe by...
string b[2] = "({";
if(b)
then

So it would actually include:

Code: Select all

1. g4 d5 2. Bg2

1. d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3

1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 

1. g4 d5 2. g5 h6 3. d4 hxg5 4. Bxg5 f6 5. Bd2 Nc6 6. Nf3 e5 7. e3 

1. g4 d5 2. g5 e5 3. d3 Bb4+ 4. Bd2 Nc6 5. Nf3 Bxd2+ 6. Qxd2 Nge7 

1. Nf3 Nc6 2. e4 e6 3. Nc3 Nf6 4. Bb5 Bb4 5. O-O O-O 

1. Nc6 Nf3 2. e5 d4 3. exd4 Nxd4 4. Nf6 Nxc6 5. dxc6 Qxd8+ 6. Kxd8 g5 7. Ne4 
What do you think Bob? Any chance you'd consider?
That's doable, but there are issues. For example, one parameter is the "min played" limit. For such comments it would likely have to be ignored since it is unlikely that such analysis would be included in many games. But if that is done, then many PGN games have crappy analysis included as well. Putting that into a book is also problematic...

Give some thought to those issues and then suggest something you think would work OK...
jhaglund
Posts: 173
Joined: Sun May 11, 2008 7:43 am

Re: Crafty Book Creation

Post by jhaglund »

bob wrote:
jhaglund wrote:Can you add to the book creating of Crafty, so it also includes sub-variations, such as annotated pv lines in a PGN?

Example:

Code: Select all

[Event "?"]
[Site "A00"]
[Date "????.??.??"]
[Round "?"]
[White "Grob"]
[WhiteElo ""]
[Black "Fritz gambit"]
[BlackElo ""]
[Result "*"]
[Annotator "Crafty v23.2 JA"]
{annotating both black and white moves.}
{using a scoring margin of +0.01 pawns.}
{search time limit is 2.00}

  1.      g4
                ({14:-0.53}  1. g4 d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3 $15)
                ({14:+0.13}  1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 $10)
                ({suggested 14:-0.38} g4 d5 g5 e5 d3 Bb4+ Bd2 Nc6 Nf3 Bxd2+ Qxd2 Nge7 $15)
                ({suggested 14:+0.13} Nf3 Nc6 e4 e6 Nc3 Nf6 Bb5 Bb4 O-O O-O $10)

  1.     ...      d5
                ({suggested 12:-0.51} d5 g5 h6 d4 hxg5 Bxg5 f6 Bd2 Nc6 Nf3 e5 e3 $15)
                ({suggested 12:-0.48} Nc6 Nf3 e5 d4 exd4 Nxd4 Nf6 Nxc6 dxc6 Qxd8+ Kxd8 g5 Ne4 $15)
  2.     Bg2
*

Normally, it would only include: 1. g4 d5 2. Bg2
I'd like it to grab all the evaluations made.

Maybe by...
string b[2] = "({";
if(b)
then

So it would actually include:

Code: Select all

1. g4 d5 2. Bg2

1. d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3

1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 

1. g4 d5 2. g5 h6 3. d4 hxg5 4. Bxg5 f6 5. Bd2 Nc6 6. Nf3 e5 7. e3 

1. g4 d5 2. g5 e5 3. d3 Bb4+ 4. Bd2 Nc6 5. Nf3 Bxd2+ 6. Qxd2 Nge7 

1. Nf3 Nc6 2. e4 e6 3. Nc3 Nf6 4. Bb5 Bb4 5. O-O O-O 

1. Nc6 Nf3 2. e5 d4 3. exd4 Nxd4 4. Nf6 Nxc6 5. dxc6 Qxd8+ 6. Kxd8 g5 7. Ne4 
What do you think Bob? Any chance you'd consider?
That's doable, but there are issues. For example, one parameter is the "min played" limit. For such comments it would likely have to be ignored since it is unlikely that such analysis would be included in many games. But if that is done, then many PGN games have crappy analysis included as well. Putting that into a book is also problematic...

Give some thought to those issues and then suggest something you think would work OK...
That should work. Grabbing comments can be ignored if the "minplayed" is >1.
If you don't use if("minplayed" <=1) it should grab any pv lines upto "maxply".

It can copy from "({ .... } to the )" & strip what isn't needed for the book creation,...

Else...

Add another variable at the end of book create 60 3... 1 for comments included, 0; default not included.

Is there any other problems?
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Crafty Book Creation

Post by bob »

jhaglund wrote:
bob wrote:
jhaglund wrote:Can you add to the book creating of Crafty, so it also includes sub-variations, such as annotated pv lines in a PGN?

Example:

Code: Select all

&#91;Event "?"&#93;
&#91;Site "A00"&#93;
&#91;Date "????.??.??"&#93;
&#91;Round "?"&#93;
&#91;White "Grob"&#93;
&#91;WhiteElo ""&#93;
&#91;Black "Fritz gambit"&#93;
&#91;BlackElo ""&#93;
&#91;Result "*"&#93;
&#91;Annotator "Crafty v23.2 JA"&#93;
&#123;annotating both black and white moves.&#125;
&#123;using a scoring margin of +0.01 pawns.&#125;
&#123;search time limit is 2.00&#125;

  1.      g4
                (&#123;14&#58;-0.53&#125;  1. g4 d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3 $15&#41;
                (&#123;14&#58;+0.13&#125;  1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 $10&#41;
                (&#123;suggested 14&#58;-0.38&#125; g4 d5 g5 e5 d3 Bb4+ Bd2 Nc6 Nf3 Bxd2+ Qxd2 Nge7 $15&#41;
                (&#123;suggested 14&#58;+0.13&#125; Nf3 Nc6 e4 e6 Nc3 Nf6 Bb5 Bb4 O-O O-O $10&#41;

  1.     ...      d5
                (&#123;suggested 12&#58;-0.51&#125; d5 g5 h6 d4 hxg5 Bxg5 f6 Bd2 Nc6 Nf3 e5 e3 $15&#41;
                (&#123;suggested 12&#58;-0.48&#125; Nc6 Nf3 e5 d4 exd4 Nxd4 Nf6 Nxc6 dxc6 Qxd8+ Kxd8 g5 Ne4 $15&#41;
  2.     Bg2
*

Normally, it would only include: 1. g4 d5 2. Bg2
I'd like it to grab all the evaluations made.

Maybe by...
string b[2] = "({";
if(b)
then

So it would actually include:

Code: Select all

1. g4 d5 2. Bg2

1. d5 2. g5 Nc6 3. d3 e5 4. Nc3 d4 5. Ne4 f5 6. gxf6 Nxf6 7. Nf3 Bb4+ 8. c3 dxc3 9. Nxc3

1. Nf3 Nc6 2. Nc3 Nf6 3. e4 e6 4. Bb5 Bb4 5. O-O O-O 6. d3 d5 7. Bd2 Bxc3 8. Bxc3 dxe4 9. Bxc6 bxc6 

1. g4 d5 2. g5 h6 3. d4 hxg5 4. Bxg5 f6 5. Bd2 Nc6 6. Nf3 e5 7. e3 

1. g4 d5 2. g5 e5 3. d3 Bb4+ 4. Bd2 Nc6 5. Nf3 Bxd2+ 6. Qxd2 Nge7 

1. Nf3 Nc6 2. e4 e6 3. Nc3 Nf6 4. Bb5 Bb4 5. O-O O-O 

1. Nc6 Nf3 2. e5 d4 3. exd4 Nxd4 4. Nf6 Nxc6 5. dxc6 Qxd8+ 6. Kxd8 g5 7. Ne4 
What do you think Bob? Any chance you'd consider?
That's doable, but there are issues. For example, one parameter is the "min played" limit. For such comments it would likely have to be ignored since it is unlikely that such analysis would be included in many games. But if that is done, then many PGN games have crappy analysis included as well. Putting that into a book is also problematic...

Give some thought to those issues and then suggest something you think would work OK...
That should work. Grabbing comments can be ignored if the "minplayed" is >1.
If you don't use if("minplayed" <=1) it should grab any pv lines upto "maxply".

It can copy from "({ .... } to the )" & strip what isn't needed for the book creation,...

Else...

Add another variable at the end of book create 60 3... 1 for comments included, 0; default not included.

Is there any other problems?
Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
jhaglund
Posts: 173
Joined: Sun May 11, 2008 7:43 am

Re: Crafty Book Creation

Post by jhaglund »

Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
Use the current header values or even the defaults, don't reset until you finish stepping through the comments and find no more comment identifiers such as "{(" and then the result_tag '*' at the end of PGN.

Count the steps and inc each time one is found and repeat until none are found.

count = 0;
while(!=result_tag){
If(comment) // find begin chars
grab(); //do your magic

count = count + 1; //line in pgn
step(); // your location

if(result_tag)
reset();

}

next_pgn();
bob
Posts: 20943
Joined: Mon Feb 27, 2006 7:30 pm
Location: Birmingham, AL

Re: Crafty Book Creation

Post by bob »

jhaglund wrote:
Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
Use the current header values or even the defaults, don't reset until you finish stepping through the comments and find no more comment identifiers such as "{(" and then the result_tag '*' at the end of PGN.

Count the steps and inc each time one is found and repeat until none are found.

count = 0;
while(!=result_tag){
If(comment) // find begin chars
grab(); //do your magic

count = count + 1; //line in pgn
step(); // your location

if(result_tag)
reset();

}

next_pgn();
That's not my point. When I find a line starting with "[" I simply reset the game to the starting position and then search for the result tag followed by the first move. This requires more complex code, as I have to remember the point where a "branch" (alternate analyis) starts and when I get to the end, back up to that point. But it has to be "stacked" because nested analysis is perfectly allowable in PGN. Makes the parsing a lot more complicated. Might be easier to write something to eat a PGN file and expand a single game into many games by unnesting the analysis comments...
Dann Corbit
Posts: 12542
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Crafty Book Creation

Post by Dann Corbit »

bob wrote:
jhaglund wrote:
Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
Use the current header values or even the defaults, don't reset until you finish stepping through the comments and find no more comment identifiers such as "{(" and then the result_tag '*' at the end of PGN.

Count the steps and inc each time one is found and repeat until none are found.

count = 0;
while(!=result_tag){
If(comment) // find begin chars
grab(); //do your magic

count = count + 1; //line in pgn
step(); // your location

if(result_tag)
reset();

}

next_pgn();
That's not my point. When I find a line starting with "[" I simply reset the game to the starting position and then search for the result tag followed by the first move. This requires more complex code, as I have to remember the point where a "branch" (alternate analyis) starts and when I get to the end, back up to that point. But it has to be "stacked" because nested analysis is perfectly allowable in PGN. Makes the parsing a lot more complicated. Might be easier to write something to eat a PGN file and expand a single game into many games by unnesting the analysis comments...
If I remember right, the Fritz4 book was a single, nested PGN game.
jwes
Posts: 778
Joined: Sat Jul 01, 2006 7:11 am

Re: Crafty Book Creation

Post by jwes »

bob wrote:
jhaglund wrote:
Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
Use the current header values or even the defaults, don't reset until you finish stepping through the comments and find no more comment identifiers such as "{(" and then the result_tag '*' at the end of PGN.

Count the steps and inc each time one is found and repeat until none are found.

count = 0;
while(!=result_tag){
If(comment) // find begin chars
grab(); //do your magic

count = count + 1; //line in pgn
step(); // your location

if(result_tag)
reset();

}

next_pgn();
That's not my point. When I find a line starting with "[" I simply reset the game to the starting position and then search for the result tag followed by the first move. This requires more complex code, as I have to remember the point where a "branch" (alternate analyis) starts and when I get to the end, back up to that point. But it has to be "stacked" because nested analysis is perfectly allowable in PGN. Makes the parsing a lot more complicated. Might be easier to write something to eat a PGN file and expand a single game into many games by unnesting the analysis comments...
Also you wouldn't introduce a bug into your regular code trying to handle some special case. :wink:
jhaglund
Posts: 173
Joined: Sun May 11, 2008 7:43 am

Re: Crafty Book Creation

Post by jhaglund »

bob wrote:
jhaglund wrote:
Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
Use the current header values or even the defaults, don't reset until you finish stepping through the comments and find no more comment identifiers such as "{(" and then the result_tag '*' at the end of PGN.

Count the steps and inc each time one is found and repeat until none are found.

count = 0;
while(!=result_tag){
If(comment) // find begin chars
grab(); //do your magic

count = count + 1; //line in pgn
step(); // your location

if(result_tag)
reset();

}

next_pgn();
That's not my point. When I find a line starting with "[" I simply reset the game to the starting position and then search for the result tag followed by the first move. This requires more complex code, as I have to remember the point where a "branch" (alternate analyis) starts and when I get to the end, back up to that point. But it has to be "stacked" because nested analysis is perfectly allowable in PGN. Makes the parsing a lot more complicated. Might be easier to write something to eat a PGN file and expand a single game into many games by unnesting the analysis comments...
Yes, of course...
When you start... "["
Then the first move....
(By result_tag, I was thinking the end of the pgn, not the actual tag header...)

use_pgn_tags();
move_list[counter][move][comment];

counter - keeps track where you are. Count until you cannot count anymore = no more comments detected.

move - is the pv move(s).
comment - is the ananlyzed move(s).

if('*') // no more comments. done.
next_pgn();

if comments are detected, the previous tags found/default empties could be used. This way you wouldn't have to generate anything "physical"...
jhaglund
Posts: 173
Joined: Sun May 11, 2008 7:43 am

Re: Crafty Book Creation

Post by jhaglund »

Dann Corbit wrote:
bob wrote:
jhaglund wrote:
Only other issue I can think of is the actual parsing, since comments can be nested. I have seen one that would choke a large mule with comments 14 levels deep. It is a pretty significant change, since encountering a PGN header resets the game to the initial position at present. This would need to back up to the previous position where analysis was found...
Use the current header values or even the defaults, don't reset until you finish stepping through the comments and find no more comment identifiers such as "{(" and then the result_tag '*' at the end of PGN.

Count the steps and inc each time one is found and repeat until none are found.

count = 0;
while(!=result_tag){
If(comment) // find begin chars
grab(); //do your magic

count = count + 1; //line in pgn
step(); // your location

if(result_tag)
reset();

}

next_pgn();
That's not my point. When I find a line starting with "[" I simply reset the game to the starting position and then search for the result tag followed by the first move. This requires more complex code, as I have to remember the point where a "branch" (alternate analyis) starts and when I get to the end, back up to that point. But it has to be "stacked" because nested analysis is perfectly allowable in PGN. Makes the parsing a lot more complicated. Might be easier to write something to eat a PGN file and expand a single game into many games by unnesting the analysis comments...
If I remember right, the Fritz4 book was a single, nested PGN game.
That's probably because, in the Chessbase GUI, you're able to analyze a game, and all the comments are imported into the book, from the PGN. Which is what this would do...