Utility to remove duplicate positions from a PGN?
Moderators: hgm, Rebel, chrisw
-
- Posts: 47
- Joined: Fri Feb 15, 2019 11:41 pm
- Full name: Brent M
Utility to remove duplicate positions from a PGN?
Is there a utility to remove duplicate positions from a PGN? Thanks
-
- Posts: 47
- Joined: Fri Feb 15, 2019 11:41 pm
- Full name: Brent M
Re: Utility to remove duplicate positions from a PGN?
Looks like pgnscanner is the answer. Is it common for it to keep finding absolute or positional double positions, even after they've been removed? The first round, it found 9 doubles. I removed them. Then it found 4. I removed them. Then it found 5. I removed them. Now it found 4 more. The total number of positions in the file keeps going down by the number of positions I remove, so I know it's reading current and correct information.
-
- Posts: 12542
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Utility to remove duplicate positions from a PGN?
Duplicate positions, or duplicate games?
pgn-extract removes duplicate games with the -d flag.
I do not understand why you would want to remove duplicate positions unless you are analyzing EPD.
And in that case it is trivial to pipe the sorted EPD to uniq first and get a distinct list
pgn-extract removes duplicate games with the -d flag.
I do not understand why you would want to remove duplicate positions unless you are analyzing EPD.
And in that case it is trivial to pipe the sorted EPD to uniq first and get a distinct list
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 47
- Joined: Fri Feb 15, 2019 11:41 pm
- Full name: Brent M
Re: Utility to remove duplicate positions from a PGN?
I was creating my own opening pgn for engine testing (both Silver suites, some Noomen, some Balsa, some Pohl, some Lc0, some TCEC, some of my own...). I found the duplicates.
-
- Posts: 6808
- Joined: Wed Nov 18, 2009 7:16 pm
- Location: Gutweiler, Germany
- Full name: Frank Quisinsky
Re: Utility to remove duplicate positions from a PGN?
Hi,
or looking in the FEOBOS documentation for such things (included in download file _basics.zip).
Here I added a file: readme-to-the-important-parameters.txt
http://www.amateurschach.de (menue FEOBOS).
Here you can find the information about tools, parmeters and also the files from FEOBOS without move transpositions and doubles. And of course sorted files with a ranking system for opening books in *.epd, *.pgn and all the other material.
Best
Frank
or looking in the FEOBOS documentation for such things (included in download file _basics.zip).
Here I added a file: readme-to-the-important-parameters.txt
http://www.amateurschach.de (menue FEOBOS).
Here you can find the information about tools, parmeters and also the files from FEOBOS without move transpositions and doubles. And of course sorted files with a ranking system for opening books in *.epd, *.pgn and all the other material.
Best
Frank