Automated tool to append "bm" PV to EPD file.

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Automated tool to append "bm" PV to EPD file.

Post by Ferdy » Tue Feb 20, 2018 12:57 pm

MikeGL wrote:I wonder if there is already a tool that can parse an EPD file and
append SF9 PV analysis as 'bm' on each position in EPD file.
I don't know what you are up to, but chess artist can analyze and output like below.

Code: Select all

r1bk1b1r/ppp2ppp/2p5/4Pn2/8/5N2/PPP2PPP/RNB2RK1 w - - acd 17; acs 10; bm Nc3; ce 30; Ae "Brainfish 280816 64 POPCNT";
Do you really need the uci pv?

chess artist:
https://github.com/fsmosca/chess-artist

You may try the released exe file.

Code: Select all

a. Command line
chess-artist -infile a.epd -outfile out_a.epd -eng Branfish.exe -engoptions "Hash value 128, Threads value 1" -movetime 10000 -eval search

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Automated tool to append "bm" PV to EPD file.

Post by Ferdy » Tue Feb 20, 2018 1:05 pm

Jesse Gersenson wrote:This is just speculation, as I am can not able to tell the difference between EPD and FEN...
FEN:

Code: Select all

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - - 0 1
EPD:

Code: Select all

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - -

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - - fmvn 1; hmvc 0;

1B2N3/8/3P1P2/5r1p/5p2/2b2k2/8/7K w - - bm d7;

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Automated tool to append "bm" PV to EPD file.

Post by Dann Corbit » Tue Feb 20, 2018 9:39 pm

Beware about Arena:
It's output can become decoupled from the EPD file you are analyzing and (for instance) three outputs in a row could be for the same problem instead of for 3 different problems. Or it can skip a problem altogether and all sorts of horrible things like that. The faster you set the time control, the worse the problem becomes.

For that reason, I modify open source programs to write analysis to a log file with the position together with the analysis.

Sometimes, though, you don't have the source (e.g. Houdini) so you have to use an algorithm to tie the log file to the EPD records (I have programs I wrote to do this). But in such a case, I always validate the legality of the selected move as a cross check to ensure that we did not become decoupled.

Other GUIs have similar problems or only write out non-standard looking analysis
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

MikeGL
Posts: 901
Joined: Thu Sep 01, 2011 12:49 pm

Re: Automated tool to append "bm" PV to EPD file.

Post by MikeGL » Wed Feb 21, 2018 8:21 am

Ferdy wrote:
MikeGL wrote:I wonder if there is already a tool that can parse an EPD file and
append SF9 PV analysis as 'bm' on each position in EPD file.
I don't know what you are up to, but chess artist can analyze and output like below.

Code: Select all

r1bk1b1r/ppp2ppp/2p5/4Pn2/8/5N2/PPP2PPP/RNB2RK1 w - - acd 17; acs 10; bm Nc3; ce 30; Ae "Brainfish 280816 64 POPCNT";
Do you really need the uci pv?

chess artist:
https://github.com/fsmosca/chess-artist

You may try the released exe file.

Code: Select all

a. Command line
chess-artist -infile a.epd -outfile out_a.epd -eng Branfish.exe -engoptions "Hash value 128, Threads value 1" -movetime 10000 -eval search
Noted. Already downloaded your tool and included it on my tool box for
my future analysis of my personal PGN and EPD games.
Thanks for the link.

Dann Corbit wrote:Beware about Arena:
It's output can become decoupled from the EPD file you are analyzing and (for instance) three outputs in a row could be for the same problem instead of for 3 different problems. Or it can skip a problem altogether and all sorts of horrible things like that. The faster you set the time control, the worse the problem becomes.

For that reason, I modify open source programs to write analysis to a log file with the position together with the analysis.

Sometimes, though, you don't have the source (e.g. Houdini) so you have to use an algorithm to tie the log file to the EPD records (I have programs I wrote to do this). But in such a case, I always validate the legality of the selected move as a cross check to ensure that we did not become decoupled.

Other GUIs have similar problems or only write out non-standard looking analysis
Thanks for the warning. Would analyze using longer time controls, probably 15 to 20 seconds.
I told my wife that a husband is like a fine wine; he gets better with age. The next day, she locked me in the cellar.

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Automated tool to append "bm" PV to EPD file.

Post by Dann Corbit » Wed Feb 21, 2018 11:38 am

Thanks for the warning. Would analyze using longer time controls, probably 15 to 20 seconds.
At 12 minutes, failures are rare.
At 10-20 seconds you will see lots of failures on overnight runs.
Not every run will fail, but many of them will.
I find that 3.0 fails less than 3.51 but both of them fail.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Jesse Gersenson
Posts: 575
Joined: Sat Aug 20, 2011 7:43 am
Contact:

Re: Automated tool to append "bm" PV to EPD file.

Post by Jesse Gersenson » Wed Feb 21, 2018 6:21 pm

Dann Corbit wrote:... become decoupled.
I wrote another version of my above script which only launches the engine once, and then feeds it positions. For whatever reason it always outputs the total number of positions - 1. 100 positions in gives 99 PV's out. But I do get 100 bestmove lines. Also, the PV appear out of order!? The stockfish benchmark code must have a better solution.

The script I posted above, on my machine, will break if time per move is < 70ms.
The work around I came up with, available here, http://termbin.com/eabt , was to put the PV output in a variable.

Would users typically use one core per position, or many cores per position, on this type of problem set?

Dann Corbit
Posts: 10207
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Automated tool to append "bm" PV to EPD file.

Post by Dann Corbit » Wed Feb 21, 2018 7:44 pm

Jesse Gersenson wrote:
Dann Corbit wrote:... become decoupled.
I wrote another version of my above script which only launches the engine once, and then feeds it positions. For whatever reason it always outputs the total number of positions - 1. 100 positions in gives 99 PV's out. But I do get 100 bestmove lines. Also, the PV appear out of order!? The stockfish benchmark code must have a better solution.

The script I posted above, on my machine, will break if time per move is < 70ms.
The work around I came up with, available here, http://termbin.com/eabt , was to put the PV output in a variable.

Would users typically use one core per position, or many cores per position, on this type of problem set?
I generally use a large number of cores at long time control, but sometimes people will use small core counts at extremely high speed. I have done that also, but never with Arena compiled data.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

Post Reply