how to create a labeled epd from pgn?

[brtzsnr]

Given a set of PGNs I use the following command (works on my Debian)

Code: Select all

cat ok/pgn-split.* | grep "FEN\|Result\|ECO" | tac | paste - - -d' ' | grep -v ECO | sed 's/\[FEN "//' | sed 's/0 1"\] \[Result/c9/' | sed 's/\]/;/' > quiet-labeled.epd

Let's split:

Code: Select all

cat ok/pgn-split.*

just concatenates everything

Code: Select all

grep "FEN\|Result\|ECO"

keeps only the lines containing FEN, ECO or RESULT

Code: Select all

tac

reverses the lines

Code: Select all

paste - - -d' ' 

merges the lines two by two

Code: Select all

grep -v ECO

removes ECO lines. Some cutechess versions output ECO instead of FEN

Code: Select all

sed 's/\[FEN "//' | sed 's/0 1"\] \[Result/c9/' | sed 's/\]/;/'

removes extra characters and adds c9.

[jdart]

I have two tools for this in my source tree (https://github.com/jdart1/arasan-chess):

1. A program called pgnselect that takes a PGN file and produces a sampled set of EPD positions (unlabeled). If you run it with the -q switch, it will select quiet positions. This is built from my Makefile using "make utils".

2. A python3 script named "label_positions.py" in the tools directory that takes the unlabeled EPD positions and add labels, using cutechess-cli. Cutechess-cli should be set up with an engines.json file containing the engine configuration. This script will probably require a few edits to adapt to your environment.

--Jon

[zenpawn]

Thanks, everyone. These ought to do it.

[AlvaroBegue]

Code: Select all

cat something.pgn | perl -ne '$result = $1 if /Result (\".*\")/; print "$1 $result\n" if /FEN \"(.*)\"/'

The invocation `perl -ane' means "for each line of input, execute the following piece of Perl code". What the code does is remember the result if it finds a line that matches some pattern, and print out a line if it matches a different pattern. `$1' is the part of the pattern captured between the parentheses.

[zenpawn]

Good stuff. Thanks.