Page 1 of 1
how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 3:33 am
by zenpawn
I'd like to create my own set of positions with game result labels for Texel tuning, ala the quiet-labeled.epd file shared on this forum, which contains lines such as:
rn2kb1r/ppp1pp1p/2q3p1/3nN3/3P4/2N1P3/PPb2PPP/R1B1KB1R b KQkq - c9 "0-1";
The first two steps in the process are easy enough: 1) generate a file of quiet position FENs, 2) run fast games starting from these positions to get results. Is there a script that will take the PGN from #2 and create a labeled epd file combining the starting positions with their results (either in a comment or just appended to the end of the line)?
Thanks,
-Erin
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 7:49 am
by AlvaroBegue
zenpawn wrote:I'd like to create my own set of positions with game result labels for Texel tuning, ala the quiet-labeled.epd file shared on this forum, which contains lines such as:
rn2kb1r/ppp1pp1p/2q3p1/3nN3/3P4/2N1P3/PPb2PPP/R1B1KB1R b KQkq - c9 "0-1";
The first two steps in the process are easy enough: 1) generate a file of quiet position FENs, 2) run fast games starting from these positions to get results. Is there a script that will take the PGN from #2 and create a labeled epd file combining the starting positions with their results (either in a comment or just appended to the end of the line)?
Thanks,
-Erin
Since the FEN and the result are both described in the header of the PGN file, it should be very easy to parse. I would use Perl to do it. If you post an example game from your database, I'll give you a tiny piece of Perl that will do what you want.
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 11:32 am
by zenpawn
That would be great. Here's an example:
Code: Select all
[Event "SF8-SF8-STS-10sec"]
[Site "ROOK"]
[Date "2017.12.02"]
[Round "21"]
[White "Stockfish_8_x64"]
[Black "Stockfish_8_x64 [001]"]
[Result "1/2-1/2"]
[BlackElo "3456"]
[Time "05:21:30"]
[WhiteElo "3456"]
[TimeControl "10+0"]
[SetUp "1"]
[FEN "1rb3k1/2pn2pp/p2p4/4p3/1pP4q/1P1PBP1P/1PQ2P2/R3R1K1 w - - 0 1"]
[Termination "normal"]
[PlyCount "14"]
[WhiteType "program"]
[BlackType "program"]
1. c5 {(c4-c5 d6-d5 Be3-c1 c7-c6 d3-d4 Qh4xh3 d4xe5 Qh3xf3 e5-e6 Qf3-g4+
Kg1-f1 Qg4-h3+ Kf1-g1 Qh3-g4+) 0.00/14 0} d5 {(d6-d5 Be3-c1 c7-c6 d3-d4
Qh4xh3 Qc2-e2 Qh3-h5 Qe2-e3 h7-h6 Bc1-d2 e5xd4 Qe3xd4 Nd7xc5 Qd4xc5 Qh5-g6+
Kg1-f1 Qg6-d3+ Kf1-g1 Qd3xd2 Re1-e8+ Kg8-h7 Qc5-f8 Qd2-g5+ Kg1-h1 Qg5-h5+
Kh1-g1 Qh5-g5+) 0.00/16 0} 2. Bc1 {(Be3-c1 c7-c6 d3-d4 Qh4xd4 Bc1-g5 Qd4xc5
Qc2xc5 Nd7xc5 Re1xe5 h7-h6 Bg5-e7 Nc5xb3 Ra1-d1 Kg8-f7 Be7-d6 Rb8-b7 Kg1-h2
Nb3-a5 Rd1-g1 b4-b3 Rg1-g3) +0.22/15 0} Qf6 {(Qh4-f6 Qc2-e2 Nd7xc5 Qe2xe5
Qf6xe5 Re1xe5 Nc5xb3 Ra1-b1 Bc8xh3 Kg1-h2 Bh3-d7 Re5-e7 Bd7-b5 Bc1-e3 d5-d4
Rb1-g1 g7-g6 Be3-f4 Rb8-f8 Bf4xc7 Bb5xd3) +0.12/17 1} 3. f4 {(f3-f4 Qf6-g6+
Kg1-h2 Qg6-f5 Kh2-g1 Qf5-g6+) 0.00/17 1} Qg6+ {(Qf6-g6+ Kg1-h2 Qg6-f5
Kh2-g1 Qf5-g6+) 0.00/18 0} 4. Kh2 {(Kg1-h2 Qg6-f5 Kh2-g1 Qf5-g6+) 0.00/19
0} Qf5 {(Qg6-f5 Kh2-g1 Qf5-g6+) 0.00/19 0} 5. Kg1 {(Kh2-g1 Qf5-g6+) 0.00/19
0} Qg6+ {(Qf5-g6+) 0.00/19 0} 6. Kh2 {(Kg1-h2) 0.00/26 0} Qf5 {(Qg6-f5)
0.00/23 0} 7. Kg1 {(Kh2-g1) 0.00/23 0} Qg6+ {(Qf5-g6+) 0.00/19 0 3-fold
repetition} 1/2-1/2
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 12:12 pm
by brtzsnr
Given a set of PGNs I use the following command (works on my Debian)
Code: Select all
cat ok/pgn-split.* | grep "FEN\|Result\|ECO" | tac | paste - - -d' ' | grep -v ECO | sed 's/\[FEN "//' | sed 's/0 1"\] \[Result/c9/' | sed 's/\]/;/' > quiet-labeled.epd
Let's split:
just concatenates everything
keeps only the lines containing FEN, ECO or RESULT
reverses the lines
merges the lines two by two
removes ECO lines. Some cutechess versions output ECO instead of FEN
Code: Select all
sed 's/\[FEN "//' | sed 's/0 1"\] \[Result/c9/' | sed 's/\]/;/'
removes extra characters and adds c9.
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 3:20 pm
by jdart
I have two tools for this in my source tree (
https://github.com/jdart1/arasan-chess):
1. A program called pgnselect that takes a PGN file and produces a sampled set of EPD positions (unlabeled). If you run it with the -q switch, it will select quiet positions. This is built from my Makefile using "make utils".
2. A python3 script named "label_positions.py" in the tools directory that takes the unlabeled EPD positions and add labels, using cutechess-cli. Cutechess-cli should be set up with an engines.json file containing the engine configuration. This script will probably require a few edits to adapt to your environment.
--Jon
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 3:37 pm
by zenpawn
Thanks, everyone. These ought to do it.
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 4:08 pm
by AlvaroBegue
Code: Select all
cat something.pgn | perl -ne '$result = $1 if /Result (\".*\")/; print "$1 $result\n" if /FEN \"(.*)\"/'
The invocation `perl -ane' means "for each line of input, execute the following piece of Perl code". What the code does is remember the result if it finds a line that matches some pattern, and print out a line if it matches a different pattern. `$1' is the part of the pattern captured between the parentheses.
Re: how to create a labeled epd from pgn?
Posted: Sat Dec 02, 2017 4:18 pm
by zenpawn
Good stuff. Thanks.