Annotating a game using a database of annotated pgn games

[giovanni]

Hi, Ferdinand. I tried, but I didn't figure out how to pass the name of the ref.pgn database to the program, because I see from the batch command that you can pass only one input file.

[giovanni]

Hi, Ferdinand. I tried, but I didn't figure out how to pass the name of the ref.pgn database to the program, because I see from the batch command that you can pass only one input file.

Oops! Sorry, I din't see from the instructions that the "ref.pgn" file is predefined.

I tried again on ly linux box and, after converting the "addcomment.py" with the dos2unix tool, I obtained the following output:

giovanni@giovanni-Lenovo-G505s:~/Downloads$ ./addcomment.py --inputfile "raw.pgn" --outputfile "raw_out.pgn"
./addcomment.py: line 40:
A. Program name
Add Comment

B. Program description
Add comment to a game based from a ref annotated games.

C. License notice

This program is free software, you can redistribute it and/or modify
it under the terms of the GPLv3 License as published by the
Free Software Foundation.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY. See the GNU General Public License
for more details.

You should have received a copy of the GNU General Public License (LICENSE)
along with this program, if not visit https://www.gnu.org/licenses/gpl.html

D. Program development log
version 1
1. Annotate a pgn file based on annotated ref pgn database
2. Added GNU GPLv3 license

E. Dependent modules
python-chess v0.14.1
https://pypi.python.org/pypi/python-chess

F. Programming language
Python v2.7.11
https://www.python.org/

G. Program other info
1. filename: addcomment.py
2. version: 1
3. author:
4. created: July 26, 2016

: No such file or directory
from: too many arguments
./addcomment.py: line 49: syntax error near unexpected token `('
./addcomment.py: line 49: `def usage():'

[Ferdy]

Hi, Ferdinand. I tried, but I didn't figure out how to pass the name of the ref.pgn database to the program, because I see from the batch command that you can pass only one input file.

There is nothing to pass. Ref.pgn should be the filename of the ref. annotated games.

[giovanni]

Hi Ferdinand. I got a little bit confused. If I read you correctly, the filename "ref.pgn" is predefined, whereas the "raw.pgn" can be input on the command line. So, the following command should work with your zipped files:

./addcomment.py --inputfile "raw.pgn" --outputfile "raw_out.pgn"

Is that correct?

[Ferdy]

Hi Ferdinand. I got a little bit confused. If I read you correctly, the filename "ref.pgn" is predefined, whereas the "raw.pgn" can be input on the command line. So, the following command should work with your zipped files:

./addcomment.py --inputfile "raw.pgn" --outputfile "raw_out.pgn"

Is that correct?

Yes.

[jdart]

I think what you want is something like the "Novelty Annotation" feature that is in Chessbase. It will identify the point where a novelty was played and will also indicate key alternative lines from the ChessBase online database.

--Jon

[giovanni]

Hi Ferdinand. I got a little bit confused. If I read you correctly, the filename "ref.pgn" is predefined, whereas the "raw.pgn" can be input on the command line. So, the following command should work with your zipped files:

./addcomment.py --inputfile "raw.pgn" --outputfile "raw_out.pgn"

Is that correct?

Yes.

Thanks, Ferdinand. I tried the script with the files you supplied and it worked very well:

However, when I fed it with a ten game file and a 5 MBytes "ref.pgn" database it choked on it:

giovanni@giovanni-Lenovo-G505s:~/Downloads$ python addcomment.py --inputfile "head.pgn" --outputfile "head_out.pgn"
No handlers could be found for logger "chess.pgn"

It stayed for half an hour, without producin any output, besides the header of the first game. Is there any workaround?

Thanks again.

Giovanni

[Ferdy]

Hi Ferdinand. I got a little bit confused. If I read you correctly, the filename "ref.pgn" is predefined, whereas the "raw.pgn" can be input on the command line. So, the following command should work with your zipped files:

./addcomment.py --inputfile "raw.pgn" --outputfile "raw_out.pgn"

Is that correct?

Yes.

Thanks, Ferdinand. I tried the script with the files you supplied and it worked very well:

However, when I fed it with a ten game file and a 5 MBytes "ref.pgn" database it choked on it:

giovanni@giovanni-Lenovo-G505s:~/Downloads$ python addcomment.py --inputfile "head.pgn" --outputfile "head_out.pgn"
No handlers could be found for logger "chess.pgn"

It stayed for half an hour, without producin any output, besides the header of the first game. Is there any workaround?

Thanks again.

Giovanni

How many games are in 5MB ref.pgn?
Perhaps when it find 5 or so comments it should already stopped searching for comments in other games in ref.pgn to make it faster .

[Ferdy]

Is there any workaround?i

Another way of making it faster is to select games by ECO. If the game to be annotated is Ruy Lopez, perhaps select only some ECO (ruy lopez or close to it) in the ref.pgn. There is no need to check the position if the game in the ref.pgn is a sicilian or KID, or nimzo, testing the script, this is what I have seen now of where the script was spending more on its time. Finding related ECO is much faster than searching a position from ruy lopez in all the positions from the Nimzo-indian game, not even sure if we can hit one.

[Ferdy]

Here is a good site to see which openings are related.

https://en.wikibooks.org/wiki/Chess_Ope ... O_volume_A
https://en.wikibooks.org/wiki/Chess_Ope ... O_volume_B
https://en.wikibooks.org/wiki/Chess_Ope ... O_volume_C
https://en.wikibooks.org/wiki/Chess_Ope ... O_volume_D
https://en.wikibooks.org/wiki/Chess_Ope ... O_volume_E

Later when I get the time I will use this to detect related openings, so this can be used for optimizing position matching.

Perhaps also pgn-extract has option for position matching, which could be faster, I have not checked this yet.

Another option is to use CQL, this is also fast in searching position matching, I tried this in past.

Granting we can search the position faster and CQL returns 100 games where the position we are searching is found (though we still don't know if that position has a comment because we are also interested on the comment not only the position matching). Now the script is going to check these 100 games one at a time position by position to find that certain position and comment, extract the comments and save it to be appended later to the game to be annotated. This still takes a lot of time for the script.

But I am not sure I will do this right now. This takes more work too. Perhaps at a later time.