I am looking for a utility to process a PGN file.
Specifically, I want to provide a list of EPD positions and remove all the games in the file that have that position.
Equally good would be a utility that can create a new file with all of the games that contain a position from the EPD list.
I am not asking for someone to write this. I can do the job with (for instance) Scid-vs-Pc by setting the EPD positions one at a time and deleting the games. But perhaps someone familiar with what is already available know of such a utility.
Looking for a PGN/EPD utility
Moderators: hgm, Rebel, chrisw
-
- Posts: 12541
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
-
- Posts: 1535
- Joined: Sun Oct 25, 2009 2:30 am
Re: Looking for a PGN/EPD utility
I'm going to be needing something like that, a few months down the road. Let's see if someone can spot which one, of the myriad of PGN tools out there, can perform such a task.
-
- Posts: 1056
- Joined: Thu Mar 09, 2006 4:15 pm
- Location: Long Island, NY, USA
Re: Looking for a PGN/EPD utility
If you search for an epd using all 4 fields of the epd, you have to be careful about false "en passant" notations. They could cause the search to miss if one epd record has a false ep and the other just has "-", or if they have different false ep.
-
- Posts: 1535
- Joined: Sun Oct 25, 2009 2:30 am
Re: Looking for a PGN/EPD utility
Could an epd record of the final position, as produced by epdFin, include false "en passant" information? Assuming the answer is "no", how would one proceed to find games, with that position, in another PGN file?Norm Pollock wrote:If you search for an epd using all 4 fields of the epd, you have to be careful about false "en passant" notations. They could cause the search to miss if one epd record has a false ep and the other just has "-", or if they have different false ep.
-
- Posts: 4833
- Joined: Sun Aug 10, 2008 3:15 pm
- Location: Philippines
Re: Looking for a PGN/EPD utility
Try pgn-extract this way.Dann Corbit wrote:I am looking for a utility to process a PGN file.
Specifically, I want to provide a list of EPD positions and remove all the games in the file that have that position.
Equally good would be a utility that can create a new file with all of the games that contain a position from the EPD list.
I am not asking for someone to write this. I can do the job with (for instance) Scid-vs-Pc by setting the EPD positions one at a time and deleting the games. But perhaps someone familiar with what is already available know of such a utility.
1. Create tag.txt where it contains fen lines based from your epd file
Code: Select all
FEN "rnbqkb1r/pp2pppp/3p1n2/8/3NP3/8/PPP2PPP/RNBQKB1R w KQkq - 0 1"
FEN "rnbqkbnr/ppp1pppp/8/3p4/2PP4/8/PP2PPPP/RNBQKBNR b KQkq - 0 1"
[...]
From batch file e.bat
Code: Select all
:: (1) extract games when all pos in a game is not found in all pos from epd file
pgn-extract-17-21 -s -ttag.txt -nsrc-out-notfound-in-epd.pgn src.pgn
:: (2) extract games when at least a pos in a game is found in any pos found in epd file
pgn-extract-17-21 -s -ttag.txt -osrc-out-found-in-epd.pgn src.pgn
-ttag.txt, option -t to read tag.txt
-nsrc-out-notfound-in-epd.pgn, using -n option will output games if the pos in this game is not in the epd file. The output file is src-out-notfound-in-epd.pgn.
-osrc-out-found-in-epd.pgn, using -o option the normal way of outputting what is in the epd file (overwrite mode). The output filename is src-out-found-in-epd.pgn.
Sample script in python 2.7... to create tags, just in case you need it.
Even if your epd has more than 4 fields with op codes, this script will only consider the first 4 fields.
Then it will add 0 and 1 for hmvc and fmvn since pgn-extract will give warning when using FEN tag without hmvc and fmvn. Use the latest pgn-extract-17-21 as it has some bug fixes and improvements.
Sample output in tag.txt.
Code: Select all
FEN "rnbqkb1r/pp2pppp/3p1n2/8/3NP3/8/PPP2PPP/RNBQKB1R w KQkq - 0 1"
Code: Select all
import sys
import os
def delete_file(fn):
""" Delete file fn """
if os.path.isfile(fn):
os.remove(fn)
def creat_tag(in_epd_fn, out_tag):
""" read epd file and create out_tag """
# Delete existing tag file
delete_file(out_tag)
with open(in_epd_fn, 'r') as f:
for lines in f:
line = lines.strip()
splitted_epd = line.split(' ')
# Take the first 4 fields only
epd = ' '.join(splitted_epd[:4])
# Save the epd line with different format
with open(out_tag, 'a') as fo:
fo.write('FEN "%s 0 1"\n' % epd)
def main(argv):
""" starts at main """
# Vars
epd_fn = '1.epd'
tag_fn = 'tag.txt'
# check epd file 1.epd is existing
if not os.path.isfile(epd_fn):
print 'process stopped, %s file is missing' % epd_fn
raw_input('Press enter key to exit')
sys.exit(1)
# Create tag.txt
creat_tag(epd_fn, tag_fn)
print 'done!!'
raw_input('Press enter key to exit')
if __name__ == "__main__":
main(sys.argv[1:])
-
- Posts: 12541
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Looking for a PGN/EPD utility
I always strip off false e.p. fields. It saves me a lot of analysis time. However, if the tool that is looking for the PGN wants the field to be there, then it is a problem.Norm Pollock wrote:If you search for an epd using all 4 fields of the epd, you have to be careful about false "en passant" notations. They could cause the search to miss if one epd record has a false ep and the other just has "-", or if they have different false ep.
Supposedly, this is 9 different game positions (if we toss in all 8 irrelevant e.p. fields):
[d]rnbqkbnr/8/8/pppppppp/PPPPPPPP/8/8/RNBQKBNR w KQkq -
Same thing, if we change the side to move to black.
Clearly a defect in the original specification.
-
- Posts: 1535
- Joined: Sun Oct 25, 2009 2:30 am
Re: Looking for a PGN/EPD utility
That should work, thx.
-
- Posts: 12541
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Looking for a PGN/EPD utility
This looks like the solution.
That you ever so much.
Saved me from having to write something to do the same job.
That you ever so much.
Saved me from having to write something to do the same job.
-
- Posts: 1056
- Joined: Thu Mar 09, 2006 4:15 pm
- Location: Long Island, NY, USA
Re: Looking for a PGN/EPD utility
I suggest that whenever anyone uses PGN-Extract for creating an epd version of a pgn description, to use the command "--nofauxep". This will ensure that no false en passant notation is output to the epd file. This command is new to the latest version, 17-21.
-
- Posts: 1535
- Joined: Sun Oct 25, 2009 2:30 am
Re: Looking for a PGN/EPD utility
From pgn-extract's change history: "after a suggestion by Norm Pollock". So, that answers my previous question.Norm Pollock wrote:This command is new to the latest version, 17-21.