Chess Tools

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
User avatar
Nordlandia
Posts: 2475
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Chess Tools

Post by Nordlandia » Sun Jul 08, 2018 7:43 pm

tpoppins wrote:
Sun Jul 08, 2018 5:49 pm
Ferdy, would it be possible to make a util to calculate the total, average and (optionally) median times of a PGN in hh:mm:ss?

E.g.,

Code: Select all

test.pgn
xxx games found
Total length: xxxh xxm xxs
Average game length: xh xxm xxs
Median game length: xh xxm xxs
Excellent request!

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Chess Tools

Post by Ferdy » Sun Jul 08, 2018 8:58 pm

tpoppins wrote:
Sun Jul 08, 2018 5:49 pm
Ferdy, would it be possible to make a util to calculate the total, average and (optionally) median times of a PGN in hh:mm:ss?

E.g.,

Code: Select all

test.pgn
xxx games found
Total length: xxxh xxm xxs
Average game length: xh xxm xxs
Median game length: xh xxm xxs
Try GTS (Game Time Stats) at,

https://sites.google.com/view/deuterium ... /tools/pgn

User avatar
Nordlandia
Posts: 2475
Joined: Fri Sep 25, 2015 7:38 pm
Location: Sortland, Norway

Re: Chess Tools

Post by Nordlandia » Sun Jul 08, 2018 11:33 pm

How do i use the tool (GTS) clicked on the .exe but cmd opened for a second then closed.

Please show example output from engine game.

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Chess Tools

Post by Ferdy » Mon Jul 09, 2018 2:50 am

Nordlandia wrote:
Sun Jul 08, 2018 11:33 pm
How do i use the tool (GTS) clicked on the .exe but cmd opened for a second then closed.

Please show example output from engine game.
1. Create a chess folder in C drive
2. Copy gts.exe in chess folder
3. Copy too your_games.pgn in chess folder
4. In windows OS open command prompt
https://www.digitalcitizen.life/7-ways- ... -windows-8
5. Navigate to the location of gts.exe and open it
https://www.digitalcitizen.life/command ... c-commands
In cmd box type
cd\
cd chess
gts.exe -i your_games.pgn

View the chess folder and you will see gts_output.txt, open it with notepad or any word processor.
It would look like the following.

Code: Select all

your_games.pgn
138 games found
Total length: 017h 51m 23s
Average game length: 0h 07m 45s
Median game length: 0h 08m 01s
To verify correctness of gts.exe, enable the logging by typing,

Code: Select all

gts.exe -i your_games.pgn --log 1
And examine the duration of every game in gts_log.txt. With cutechess pgn output, it is easy to verify since it has game duration tag

Code: Select all

[GameDuration "00:05:19"]
To show help type,

Code: Select all

gts.exe -h
You will see,

Code: Select all

usage: gts.exe [-h] [-i INPUT] [-o OUTPUT] [-l {0,1}]

optional arguments:
  -h, --help            show this help message and exit
  -i INPUT, --input INPUT
                        input pgn filename default is games.pgn
  -o OUTPUT, --output OUTPUT
                        output text filename default is gts_output.txt
  -l {0,1}, --log {0,1}
                        output some info to gts_log.txt, by default log is
                        disabled or 0

tpoppins
Posts: 919
Joined: Tue Nov 24, 2015 8:11 pm
Location: upstate

Re: Chess Tools

Post by tpoppins » Mon Jul 09, 2018 5:00 am

Thank you very much, Ferdy; this is exactly what I needed!
Tirsa Poppins
CCRL

User avatar
George Tsavdaris
Posts: 1614
Joined: Thu Mar 09, 2006 11:35 am

Re: Chess Tools

Post by George Tsavdaris » Tue Jul 10, 2018 7:43 am

Ferdy wrote:
Sun Jul 08, 2018 3:15 pm
George Tsavdaris wrote:
Thu Jun 14, 2018 9:49 am
Can anyone make a tool to extract time PER MOVE, PER GAME from a PGN?
Assuming the PGN has time tags of course. :D

I mean if the PGN had 3 games then use the tool to extract 3 text files(or one with having some separators to separate the games) with the time per move (in e.g seconds) for each side e.g for one game:
12 15
22 19
25 18
23 19
30 35
31 20
8 18
etc...........
Try GTE (Game Time Extractor) at,

https://sites.google.com/view/deuterium ... /tools/pgn
Wow many many thanks! :D
After his son's birth they've asked him:
"Is it a boy or girl?"
YES! He replied.....

User avatar
PeterO
Posts: 121
Joined: Sun Jul 31, 2016 4:35 pm

Re: Polyglot book creator

Post by PeterO » Tue Jul 10, 2018 4:32 pm

Hi,
Can you add „merge books“ to the Polyglot book creator?

Peter :D

Spill_The_Tea
Posts: 22
Joined: Mon Dec 17, 2018 2:33 am
Full name: Jase de Lace

Re: Chess Tools

Post by Spill_The_Tea » Wed Apr 10, 2019 10:53 pm

Hi Ferdy,
Big Fan of your work and chess tools.
If this thread is still active, I would like to make a request.
Recently, I have been collecting epd testsuites; however, testsuites are notoriously incestuous, meaning they almost always include identical positions often with different ids.

In contrast, managing pgn databases is a bit easier to remove duplicate games using tools such as pgn-extract, but I have not found any similar tool to manage epd databases (specifically removing duplicate fen positions from epd files).

Right now, the easiest way to accomplish this is to strip all op codes from a database (using scidvspc tools) and then use a simple awk expression in terminal to remove duplicate lines: awk '!seen[$0]++' Input.epd > Output.epd
But this simply perpetuates the same problem I am describing, because I end up prescribing a different ID to positions already accounted for.

Could you create a tool to manage epd databases, that parses for the FEN of each epd position, and remove the lines of any subsequent duplicates?

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Chess Tools

Post by Ferdy » Thu Apr 11, 2019 12:39 am

Spill_The_Tea wrote:
Wed Apr 10, 2019 10:53 pm
Hi Ferdy,
Big Fan of your work and chess tools.
If this thread is still active, I would like to make a request.
Recently, I have been collecting epd testsuites; however, testsuites are notoriously incestuous, meaning they almost always include identical positions often with different ids.

In contrast, managing pgn databases is a bit easier to remove duplicate games using tools such as pgn-extract, but I have not found any similar tool to manage epd databases (specifically removing duplicate fen positions from epd files).

Right now, the easiest way to accomplish this is to strip all op codes from a database (using scidvspc tools) and then use a simple awk expression in terminal to remove duplicate lines: awk '!seen[$0]++' Input.epd > Output.epd
But this simply perpetuates the same problem I am describing, because I end up prescribing a different ID to positions already accounted for.

Could you create a tool to manage epd databases, that parses for the FEN of each epd position, and remove the lines of any subsequent duplicates?
Given epd's
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e4; id "my test 1";
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d4; id "my test 2";

Are the two epd's above identical? Perhaps not.

But this one has, and 2nd epd can be removed.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e4; id "my test 1";
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e4; id "my test 2";

What if there are other opcodes? But it seems like [piece locations] [stm] [castle right] [ep square] bm [move] can be considered as independent and can be used to identify identical, depends on your criteria too.

I think you need to give me some examples which epd's are considered identical.

Spill_The_Tea
Posts: 22
Joined: Mon Dec 17, 2018 2:33 am
Full name: Jase de Lace

Re: Chess Tools

Post by Spill_The_Tea » Thu Apr 11, 2019 2:16 am

Ferdy wrote:
Thu Apr 11, 2019 12:39 am
Spill_The_Tea wrote:
Wed Apr 10, 2019 10:53 pm
Hi Ferdy,
Big Fan of your work and chess tools.
If this thread is still active, I would like to make a request.
Recently, I have been collecting epd testsuites; however, testsuites are notoriously incestuous, meaning they almost always include identical positions often with different ids.

In contrast, managing pgn databases is a bit easier to remove duplicate games using tools such as pgn-extract, but I have not found any similar tool to manage epd databases (specifically removing duplicate fen positions from epd files).

Right now, the easiest way to accomplish this is to strip all op codes from a database (using scidvspc tools) and then use a simple awk expression in terminal to remove duplicate lines: awk '!seen[$0]++' Input.epd > Output.epd
But this simply perpetuates the same problem I am describing, because I end up prescribing a different ID to positions already accounted for.

Could you create a tool to manage epd databases, that parses for the FEN of each epd position, and remove the lines of any subsequent duplicates?
Given epd's
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e4; id "my test 1";
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm d4; id "my test 2";

Are the two epd's above identical? Perhaps not.

But this one has, and 2nd epd can be removed.
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e4; id "my test 1";
rnbqkbnr/pppppppp/8/8/8/8/PPPPPPPP/RNBQKBNR w KQkq - bm e4; id "my test 2";

What if there are other opcodes? But it seems like [piece locations] [stm] [castle right] [ep square] bm [move] can be considered as independent and can be used to identify identical, depends on your criteria too.

I think you need to give me some examples which epd's are considered identical.
Hi Ferdy,

Right, I have previously found situations like this before in my database, where identical positions had different best moves. So I am asking for an imperfect solution, and ignore the bm/am/pm op code. Here is an example I found in various sources I pulled together:

Example Pooled EPD File (i.e. Input):

line 1: 8/2N4r/1p3pkp/8/5K1p/2P4N/P3Bn2/8 w - - bm Bh5+; id "W.Eigenmann: Brillant 002";
line 2 : 8/2N4r/1p3pkp/8/5K1p/2P4N/P3Bn2/8 w - - id ?; bm Bh5;
line 3 : 8/2p1k3/3p3p/2PP1pp1/1P1K1P2/6P1/8/8 w - - bm g4; id "arasan19.45"; c0 "Camacho Martinez-An. C. Hernandez, Cuba 1995";
line 4 : 8/2p1k3/3p3p/2PP1pp1/1P1K1P2/6P1/8/8 w - - bm g4; id peg124;

Desired Output:

line 1: 8/2N4r/1p3pkp/8/5K1p/2P4N/P3Bn2/8 w - - bm Bh5+; id "W.Eigenmann: Brillant 002";
line 2 : 8/2p1k3/3p3p/2PP1pp1/1P1K1P2/6P1/8/8 w - - bm g4; id "arasan19.45"; c0 "Camacho Martinez-An. C. Hernandez, Cuba 1995";

And If the lines in the example input file were reversed, then lines 4 and 2 would be printed instead:
line 1 : 8/2N4r/1p3pkp/8/5K1p/2P4N/P3Bn2/8 w - - id ?; bm Bh5;
line 2 : 8/2p1k3/3p3p/2PP1pp1/1P1K1P2/6P1/8/8 w - - bm g4; id peg124;

Please note that there are also situations with more than two replicates, for example:
8/2p1p1pp/2P1P2k/6pP/p7/P4Bp1/1P4P1/n5K1 w - - bm Bd1; id "MES.256";
8/2p1p1pp/2P1P2k/6pP/p7/P4Bp1/1P4P1/n5K1 w - - bm Bd1; id "Holmes Endgame Pos. 1362"; c0 "white pieces=8 black pieces=9"; c1 "material balance: -1,0";
8/2p1p1pp/2P1P2k/6pP/p7/P4Bp1/1P4P1/n5K1 w - - bm Bd1; c0 "level: med-3"; id "EG 0866";

Post Reply