Page 3 of 4
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 4:06 pm
by Norm Pollock
Ozymandias wrote:Norm Pollock wrote:http://www.mediafire.com/download/7xkuc ... sition.exe
I set "size" to 100000 from 100
100000 is going to take awhile to process.
I suggest putting a number comment in each game of the pgn file if you have not already done so. One such program that does this is "gameNum". It is in the 40H-PGN suite on my home page (www below).
I finally tried epdPostion and I still get the same message (Array Index Out of Bounds error!), only at 100001 lines, of course. The search.epd file is 730 MB, and the temp.epdn is 1.11 GB.
Can we amp up the steroids again (3 more zeros)?
Juan--
Changed 100000 to 100000000
http://www.mediafire.com/file/n76acc3sw ... teroid.exe
this is the executable, no need to unpack it
use it the same as "epdPosition" except the name is "epdPositionSteroid"
if it works, please let me know the capacities of your machine.
-Norm
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 4:35 pm
by Ozymandias
Norm Pollock wrote:if it works, please let me know the capacities of your machine.
-Norm
Thx for the quick reply, it didn't work, I get a window saying: Too many heap sections (Fatal error in gc).
5820K, Win8 64, 16 GB
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 4:44 pm
by Norm Pollock
took out one "0", and added output line to show line limit for search.epd
line limit now 10000000
same link
http://www.mediafire.com/file/n76acc3sw ... teroid.exe
good luck
-Norm
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 5:17 pm
by Ozymandias
Norm Pollock wrote:took out one "0"
I'm making the search.epd smaller, but still the same gc error. Maybe the original epdPosition was already at the limit? At least with that one I simply got an out of array error.
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 5:31 pm
by Norm Pollock
search limit now down to 1000000
If that does not work you could split search.epd using "txtSplit" with 40H-TXT
same link as above
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 9:51 pm
by Ozymandias
Norm Pollock wrote:search limit now down to 1000000
If that does not work you could split search.epd using "txtSplit" with 40H-TXT
same link as above
That's probably a sensible number, and a reasonable limit. I've had it running for hours now, with a search.epd of less than a million lines, but I'm still waiting to see if the result is ok. I'm using the second version of epdPositionSteroid you posted, but it'd probably make no practical difference.
Thx for the heads up on txtSplit, the first of the suite I'm putting to good use.
Re: Looking for a PGN/EPD utility
Posted: Wed Feb 15, 2017 10:02 pm
by Norm Pollock
Good to hear it is at least running. Hope the output is ok.
I will put a clone of "txtSplit" ("epdSplit") into 40H-EPD next go-around. Likewise for "txtSort".
Re: Looking for a PGN/EPD utility
Posted: Thu Feb 16, 2017 7:32 am
by Ozymandias
Norm Pollock wrote:Hope the output is ok.
Doesn't look that way, from what I could sample (still running):
Code: Select all
3rr1k1/1bqn1ppp/p2b4/2pP4/Pp6/4BN1P/1PQ1BPP1/2RR2K1 w - -
game = 1 after ply 503215
r4rk1/1q1n1ppp/p4n2/4pN2/1PP1p3/4bP2/3QB1PP/R4RK1 w - -
game = 1 after ply 503635
r4k2/3b1pp1/p3p2p/b1np4/8/2P5/PP2BPPP/2K2R1R w - -
game = 1 after ply 503659
2rq1rk1/1b1nbppp/p3p3/8/PpN1P3/3BB3/1P2QPPP/2R2RK1 w - -
game = 1 after ply 503805
r4rk1/2p2ppp/p7/1pB5/2p2P2/2nq1QPB/P7/R4RK1 w - -
game = 1 after ply 504294
1q3rk1/3n1ppp/4pn2/1N6/1p1P4/1Q3B2/1P1B1PPb/R6K w - -
game = 1 after ply 504418
4r1k1/1r2bppp/pqNp4/1p1Pp1Pn/7P/P4P2/1PPQ3R/2KR1B2 w - -
game = 1 after ply 504766
r2q1rk1/pp2nppp/8/2n1b1PP/2BN2b1/P1N1P3/2QB1P2/R3K2R w KQ -
game = 1 after ply 504928
r3k2r/2pb1pp1/pq5p/1p2P3/2p2P2/2P3P1/P3Q1BP/3R1RK1 w kq -
game = 1 after ply 505753
It goes on like that. For one, I used gameNum in the PGN that I want to finally process, and there's no way that more than 3 positions are found in the same game. And the ply counts are ridiculous, games are truncated at ply 40, both in the PGN that produced the search.epd as well as the final one.
Re: Looking for a PGN/EPD utility
Posted: Thu Feb 16, 2017 8:39 am
by Ozymandias
Actually, I tried with a much smaller search.epd and the original epdPosition. I see the same pattern. This is the bat file, up to that point, is there something wrong?
Code: Select all
pgn-extract -Wepd --nofauxep -s -otemp.epd set1a20.pgn
epdFin temp.epd
del temp.epd
rename outF.epd search.epd
gameNum pyramid_ply40.pgn
pgn-extract -Wepd --nofauxep -s -otemp.epd out4.pgn
epdPosition search.epd temp.epd
Re: Looking for a PGN/EPD utility
Posted: Thu Feb 16, 2017 2:24 pm
by Norm Pollock
Ozymandias wrote:Norm Pollock wrote:Hope the output is ok.
Doesn't look that way, from what I could sample (still running):
Code: Select all
3rr1k1/1bqn1ppp/p2b4/2pP4/Pp6/4BN1P/1PQ1BPP1/2RR2K1 w - -
game = 1 after ply 503215
r4rk1/1q1n1ppp/p4n2/4pN2/1PP1p3/4bP2/3QB1PP/R4RK1 w - -
game = 1 after ply 503635
r4k2/3b1pp1/p3p2p/b1np4/8/2P5/PP2BPPP/2K2R1R w - -
game = 1 after ply 503659
2rq1rk1/1b1nbppp/p3p3/8/PpN1P3/3BB3/1P2QPPP/2R2RK1 w - -
game = 1 after ply 503805
r4rk1/2p2ppp/p7/1pB5/2p2P2/2nq1QPB/P7/R4RK1 w - -
game = 1 after ply 504294
1q3rk1/3n1ppp/4pn2/1N6/1p1P4/1Q3B2/1P1B1PPb/R6K w - -
game = 1 after ply 504418
4r1k1/1r2bppp/pqNp4/1p1Pp1Pn/7P/P4P2/1PPQ3R/2KR1B2 w - -
game = 1 after ply 504766
r2q1rk1/pp2nppp/8/2n1b1PP/2BN2b1/P1N1P3/2QB1P2/R3K2R w KQ -
game = 1 after ply 504928
r3k2r/2pb1pp1/pq5p/1p2P3/2p2P2/2P3P1/P3Q1BP/3R1RK1 w kq -
game = 1 after ply 505753
It goes on like that. For one, I used gameNum in the PGN that I want to finally process, and there's no way that more than 3 positions are found in the same game. And the ply counts are ridiculous, games are truncated at ply 40, both in the PGN that produced the search.epd as well as the final one.
Wow! I ran a similar project to yours with 15K games in the file for search.epd, and 47K games in the input pgn file. All ran fine.
Based on the output of "manifest-ps" that you sent me, my first suspicion is that the "pgn" files you used were not up to PGN Standards. Perhaps they did not have a blank line separating games.
Please use "trim" on those files and then rename.
trim alpha.pgn
outr.pgn alpha-t.pgn
I am surprised that you were able to get output from manifest-ps during execution. It must have been due to an overflow of the buffer because during execution of my test the files "manifest-ps" and "numbers", the two output files of epdPosition, had 0 bytes.
Based on the size of your test, it will take a very long time for it to finish. Perhaps a month. In which time there could be a power blip that wipes everything out. I would do the project in small parts.
Also I would rename "out4.pgn" before re-using it.
gameNum alpha.pgn
ren out4.pgn alpha-n.pgn
And finally, are you using a recent version of pgn-extract? Use "17-21" or later. I noticed D. Barnes' new version "17-30" outputs "17-26" when using the "--version" command.
-Norm