I'd like to create my own set of positions with game result labels for Texel tuning, ala the quiet-labeled.epd file shared on this forum, which contains lines such as:
rn2kb1r/ppp1pp1p/2q3p1/3nN3/3P4/2N1P3/PPb2PPP/R1B1KB1R b KQkq - c9 "0-1";
The first two steps in the process are easy enough: 1) generate a file of quiet position FENs, 2) run fast games starting from these positions to get results. Is there a script that will take the PGN from #2 and create a labeled epd file combining the starting positions with their results (either in a comment or just appended to the end of the line)?
Thanks,
-Erin
how to create a labeled epd from pgn?
Moderators: hgm, Rebel, chrisw
-
- Posts: 349
- Joined: Sat Aug 06, 2016 8:31 pm
- Location: United States
-
- Posts: 931
- Joined: Tue Mar 09, 2010 3:46 pm
- Location: New York
- Full name: Álvaro Begué (RuyDos)
Re: how to create a labeled epd from pgn?
Since the FEN and the result are both described in the header of the PGN file, it should be very easy to parse. I would use Perl to do it. If you post an example game from your database, I'll give you a tiny piece of Perl that will do what you want.zenpawn wrote:I'd like to create my own set of positions with game result labels for Texel tuning, ala the quiet-labeled.epd file shared on this forum, which contains lines such as:
rn2kb1r/ppp1pp1p/2q3p1/3nN3/3P4/2N1P3/PPb2PPP/R1B1KB1R b KQkq - c9 "0-1";
The first two steps in the process are easy enough: 1) generate a file of quiet position FENs, 2) run fast games starting from these positions to get results. Is there a script that will take the PGN from #2 and create a labeled epd file combining the starting positions with their results (either in a comment or just appended to the end of the line)?
Thanks,
-Erin
-
- Posts: 349
- Joined: Sat Aug 06, 2016 8:31 pm
- Location: United States
Re: how to create a labeled epd from pgn?
That would be great. Here's an example:
Code: Select all
[Event "SF8-SF8-STS-10sec"]
[Site "ROOK"]
[Date "2017.12.02"]
[Round "21"]
[White "Stockfish_8_x64"]
[Black "Stockfish_8_x64 [001]"]
[Result "1/2-1/2"]
[BlackElo "3456"]
[Time "05:21:30"]
[WhiteElo "3456"]
[TimeControl "10+0"]
[SetUp "1"]
[FEN "1rb3k1/2pn2pp/p2p4/4p3/1pP4q/1P1PBP1P/1PQ2P2/R3R1K1 w - - 0 1"]
[Termination "normal"]
[PlyCount "14"]
[WhiteType "program"]
[BlackType "program"]
1. c5 {(c4-c5 d6-d5 Be3-c1 c7-c6 d3-d4 Qh4xh3 d4xe5 Qh3xf3 e5-e6 Qf3-g4+
Kg1-f1 Qg4-h3+ Kf1-g1 Qh3-g4+) 0.00/14 0} d5 {(d6-d5 Be3-c1 c7-c6 d3-d4
Qh4xh3 Qc2-e2 Qh3-h5 Qe2-e3 h7-h6 Bc1-d2 e5xd4 Qe3xd4 Nd7xc5 Qd4xc5 Qh5-g6+
Kg1-f1 Qg6-d3+ Kf1-g1 Qd3xd2 Re1-e8+ Kg8-h7 Qc5-f8 Qd2-g5+ Kg1-h1 Qg5-h5+
Kh1-g1 Qh5-g5+) 0.00/16 0} 2. Bc1 {(Be3-c1 c7-c6 d3-d4 Qh4xd4 Bc1-g5 Qd4xc5
Qc2xc5 Nd7xc5 Re1xe5 h7-h6 Bg5-e7 Nc5xb3 Ra1-d1 Kg8-f7 Be7-d6 Rb8-b7 Kg1-h2
Nb3-a5 Rd1-g1 b4-b3 Rg1-g3) +0.22/15 0} Qf6 {(Qh4-f6 Qc2-e2 Nd7xc5 Qe2xe5
Qf6xe5 Re1xe5 Nc5xb3 Ra1-b1 Bc8xh3 Kg1-h2 Bh3-d7 Re5-e7 Bd7-b5 Bc1-e3 d5-d4
Rb1-g1 g7-g6 Be3-f4 Rb8-f8 Bf4xc7 Bb5xd3) +0.12/17 1} 3. f4 {(f3-f4 Qf6-g6+
Kg1-h2 Qg6-f5 Kh2-g1 Qf5-g6+) 0.00/17 1} Qg6+ {(Qf6-g6+ Kg1-h2 Qg6-f5
Kh2-g1 Qf5-g6+) 0.00/18 0} 4. Kh2 {(Kg1-h2 Qg6-f5 Kh2-g1 Qf5-g6+) 0.00/19
0} Qf5 {(Qg6-f5 Kh2-g1 Qf5-g6+) 0.00/19 0} 5. Kg1 {(Kh2-g1 Qf5-g6+) 0.00/19
0} Qg6+ {(Qf5-g6+) 0.00/19 0} 6. Kh2 {(Kg1-h2) 0.00/26 0} Qf5 {(Qg6-f5)
0.00/23 0} 7. Kg1 {(Kh2-g1) 0.00/23 0} Qg6+ {(Qf5-g6+) 0.00/19 0 3-fold
repetition} 1/2-1/2
-
- Posts: 433
- Joined: Fri Jan 16, 2015 4:02 pm
Re: how to create a labeled epd from pgn?
Given a set of PGNs I use the following command (works on my Debian)
Let's split:
just concatenates everything
keeps only the lines containing FEN, ECO or RESULT
reverses the lines
merges the lines two by two
removes ECO lines. Some cutechess versions output ECO instead of FEN
removes extra characters and adds c9.
Code: Select all
cat ok/pgn-split.* | grep "FEN\|Result\|ECO" | tac | paste - - -d' ' | grep -v ECO | sed 's/\[FEN "//' | sed 's/0 1"\] \[Result/c9/' | sed 's/\]/;/' > quiet-labeled.epd
Let's split:
Code: Select all
cat ok/pgn-split.*
Code: Select all
grep "FEN\|Result\|ECO"
Code: Select all
tac
Code: Select all
paste - - -d' '
Code: Select all
grep -v ECO
Code: Select all
sed 's/\[FEN "//' | sed 's/0 1"\] \[Result/c9/' | sed 's/\]/;/'
zurichess - http://www.zurichess.xyz
-
- Posts: 4367
- Joined: Fri Mar 10, 2006 5:23 am
- Location: http://www.arasanchess.org
Re: how to create a labeled epd from pgn?
I have two tools for this in my source tree (https://github.com/jdart1/arasan-chess):
1. A program called pgnselect that takes a PGN file and produces a sampled set of EPD positions (unlabeled). If you run it with the -q switch, it will select quiet positions. This is built from my Makefile using "make utils".
2. A python3 script named "label_positions.py" in the tools directory that takes the unlabeled EPD positions and add labels, using cutechess-cli. Cutechess-cli should be set up with an engines.json file containing the engine configuration. This script will probably require a few edits to adapt to your environment.
--Jon
1. A program called pgnselect that takes a PGN file and produces a sampled set of EPD positions (unlabeled). If you run it with the -q switch, it will select quiet positions. This is built from my Makefile using "make utils".
2. A python3 script named "label_positions.py" in the tools directory that takes the unlabeled EPD positions and add labels, using cutechess-cli. Cutechess-cli should be set up with an engines.json file containing the engine configuration. This script will probably require a few edits to adapt to your environment.
--Jon
-
- Posts: 349
- Joined: Sat Aug 06, 2016 8:31 pm
- Location: United States
Re: how to create a labeled epd from pgn?
Thanks, everyone. These ought to do it.
-
- Posts: 931
- Joined: Tue Mar 09, 2010 3:46 pm
- Location: New York
- Full name: Álvaro Begué (RuyDos)
Re: how to create a labeled epd from pgn?
Code: Select all
cat something.pgn | perl -ne '$result = $1 if /Result (\".*\")/; print "$1 $result\n" if /FEN \"(.*)\"/'
-
- Posts: 349
- Joined: Sat Aug 06, 2016 8:31 pm
- Location: United States
Re: how to create a labeled epd from pgn?
Good stuff. Thanks.