program to generare epd file of tactical exercises from pgn

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

Uri Blass
Posts: 10280
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

program to generare epd file of tactical exercises from pgn

Post by Uri Blass »

I would like to generate epd file of tactical exercises from a pgn and I wonder if there is a program that can do it.

I would like to have the following epd files based on a pgn game:

1)good moves in the pgn that are significantly stronger than the second best move
2)good moves from positions in the pgn that are not in the pgn that are significantly stronger than the second best move

For the definition of good moves
possible parameters that you can define manually:
1)engine to analyze.
2)minimal time of search or minimal depth of search or minimal nodes to search to analyze every move.
3)minimal difference between best move and second best move in centi-pawns.
4)maximal time to analyze candidate moves that I suspect after I found a significant difference after the minimal time.
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: program to generare epd file of tactical exercises from pgn

Post by Dann Corbit »

You can easily turn a chess game into EPD records with
pgn2fen <file.pgn> -e -l > <file.epd>
Now you can take a chess engine, or series of chess engines and analyze the whole game a ply at a time on multi-pv.

Something to watch for is a jump in eval from one ply to the next. That always means either a very good move was played or a very bad one. So a learning lesson or pat on the back.

With Arena, for instance, I can schedule 5 strong engines to analyze the file and let it run for a day.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Uri Blass
Posts: 10280
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: program to generare epd file of tactical exercises from pgn

Post by Uri Blass »

Dann Corbit wrote: Fri Jan 18, 2019 8:20 pm You can easily turn a chess game into EPD records with
pgn2fen <file.pgn> -e -l > <file.epd>
Now you can take a chess engine, or series of chess engines and analyze the whole game a ply at a time on multi-pv.

Something to watch for is a jump in eval from one ply to the next. That always means either a very good move was played or a very bad one. So a learning lesson or pat on the back.

With Arena, for instance, I can schedule 5 strong engines to analyze the file and let it run for a day.
I am interested in tactical exercices for humans when the target is to find a good move that is simply better than other moves
You do not need to have a mistake for this purpose(or a very good move that cause jump in the evaluation based on evaluation of the engine).

Here is an example that is easy for computers but not for weak humans(the woman with white solved it correctly).

[d]2b2rk1/1p4b1/3p3p/3Pnpq1/1P1Np3/1Q2P1P1/3N1nB1/B1R3K1 w - - 0 23

find the best move for white.

last move of black was correct and it is also a tactical exercise for humans that you can generate from the pgn

[d]2b2rk1/1p4b1/3p3p/3Pnpq1/1P1Np1n1/1Q2P1P1/3N1PB1/B1R3K1 b - - 1 22

find the best move of black.

This is the relevant game
[pgn][Event "World Blitz Women 2018"] [Site "St Petersburg RUS"] [Date "2018.12.29"] [Round "1.2"] [White "Sukhareva, Evgeniya"] [Black "Lagno, Kateryna"] [Result "0-1"] [BlackElo "2560"] [ECO "A41"] [Opening "Neo-Old Indian"] [Variation "2.c4 e5 3.d5"] [WhiteElo "2126"] [TimeControl "120+6"] [Termination "normal"] [PlyCount "108"] [WhiteType "human"] [BlackType "human"] 1. d4 d6 2. c4 e5 3. d5 f5 4. g3 Nf6 5. Bg2 g6 6. b3 Bg7 7. Bb2 O-O 8. Nf3 h6 9. h4 Nbd7 10. b4 a5 11. a3 axb4 12. axb4 Rxa1 13. Bxa1 e4 14. Nd4 Ne5 15. Qb3 Qe7 16. Nd2 Neg4 17. e3 Ne5 18. O-O Nfg4 19. c5 g5 20. cxd6 cxd6 21. hxg5 Qxg5 22. Rc1 Nxf2 23. Nf1 Nfd3 24. Rc7 Qd8 25. Rc2 Kh7 26. Ne6 Bxe6 27. dxe6 Ne1 28. Rc1 N1f3+ 29. Bxf3 exf3 30. Bd4 Qe7 31. b5 Re8 32. b6 Nc6 33. Bxg7 Kxg7 34. Qc3+ Qf6 35. Qd3 Qxe6 36. Rd1 Rd8 37. Qc3+ Qf6 38. Qd3 Ne5 39. Qd5 Rd7 40. Rc1 Qf7 41. Qd4 Kh7 42. Rc8 Qg7 43. Qh4 Ng4 44. Qh5 Rf7 45. Rd8 f2+ 46. Kg2 Qg6 47. Qxg6+ Kxg6 48. Rxd6+ Kg5 49. Rd8 Rf6 50. Nd2 Rxb6 51. Nf3+ Kf6 52. Rf8+ Kg6 53. Nh4+ Kg7 54. Rxf5 Nxe3+ 0-1 [/pgn]
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: program to generare epd file of tactical exercises from pgn

Post by Dann Corbit »

Uri Blass wrote: Fri Jan 18, 2019 9:15 pm Here is an example that is easy for computers but not for weak humans(the woman with white solved it correctly).

[d]2b2rk1/1p4b1/3p3p/3Pnpq1/1P1Np3/1Q2P1P1/3N1nB1/B1R3K1 w - - 0 23

find the best move for white.
Is there a best move if all move choices lose?
I think the game is already lost at that point.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
Uri Blass
Posts: 10280
Joined: Thu Mar 09, 2006 12:37 am
Location: Tel-Aviv Israel

Re: program to generare epd file of tactical exercises from pgn

Post by Uri Blass »

Dann Corbit wrote: Sat Jan 19, 2019 12:36 am
Uri Blass wrote: Fri Jan 18, 2019 9:15 pm Here is an example that is easy for computers but not for weak humans(the woman with white solved it correctly).

[d]2b2rk1/1p4b1/3p3p/3Pnpq1/1P1Np3/1Q2P1P1/3N1nB1/B1R3K1 w - - 0 23

find the best move for white.
Is there a best move if all move choices lose?
I think the game is already lost at that point.
From human point of view there are cases that yes and there are cases that no.

There is a difference between -2.45 and -6.80 by the computer unless the -6.80 is a very complex win that you are sure that there are good chances that even a strong human opponent is not going to find so maybe the -6.80 is practically better.

-2.45 by stockfish at depth 30 is not something that I usually expect humans to convert easily.

I cannot use only the evaluation of the computer to decide if the position is a good or bad test for humans and I need to use common sense.
Dann Corbit
Posts: 12538
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: program to generare epd file of tactical exercises from pgn

Post by Dann Corbit »

Uri Blass wrote: Sat Jan 19, 2019 5:59 am
Dann Corbit wrote: Sat Jan 19, 2019 12:36 am
Uri Blass wrote: Fri Jan 18, 2019 9:15 pm Here is an example that is easy for computers but not for weak humans(the woman with white solved it correctly).

[d]2b2rk1/1p4b1/3p3p/3Pnpq1/1P1Np3/1Q2P1P1/3N1nB1/B1R3K1 w - - 0 23

find the best move for white.
Is there a best move if all move choices lose?
I think the game is already lost at that point.
From human point of view there are cases that yes and there are cases that no.

There is a difference between -2.45 and -6.80 by the computer unless the -6.80 is a very complex win that you are sure that there are good chances that even a strong human opponent is not going to find so maybe the -6.80 is practically better.

-2.45 by stockfish at depth 30 is not something that I usually expect humans to convert easily.

I cannot use only the evaluation of the computer to decide if the position is a good or bad test for humans and I need to use common sense.
At this point, I might not resign, even though it is clear to me that the game is lost (maybe a computer could draw it, but I do not think I could) if I played sensibly.

I might try this...

From here:
[d]2b2rk1/1p4b1/3p3p/3Pnpq1/1P1Np3/1Q2P1P1/3N1nB1/B1R3K1 w - -
I might try to fork the rook and queen with my knight.
[d]2b2rk1/1p4b1/3pN2p/3Pnpq1/1P2p3/1Q2P1P1/3N1nB1/B1R3K1 b - -
If the opponent did not think hard about it, they might recapture with the bishop, giving me a draw:
[d]5rk1/1p4b1/3pb2p/3Pnpq1/1P2p3/1Q2P1P1/3N1nB1/B1R3K1 w - -

But chances are good I would just take a beating.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
USGroup1
Posts: 33
Joined: Sun Oct 14, 2018 7:01 pm
Full name: Sina Vaziri

Re: program to generare epd file of tactical exercises from pgn

Post by USGroup1 »

Uri Blass wrote: Fri Jan 18, 2019 9:51 am
1)good moves in the pgn that are significantly stronger than the second best move
Every sequence of trading pieces fit that description, you need more filters. ChessTricks uses similar filters, you might find it useful for this.
https://chesstricksapp.wordpress.com
User avatar
Guenther
Posts: 4605
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: program to generare epd file of tactical exercises from pgn

Post by Guenther »

USGroup1 wrote: Sat Jan 19, 2019 11:02 am
Uri Blass wrote: Fri Jan 18, 2019 9:51 am
1)good moves in the pgn that are significantly stronger than the second best move
Every sequence of trading pieces fit that description, you need more filters. ChessTricks uses similar filters, you might find it useful for this.
https://chesstricksapp.wordpress.com
I have tried the latest version 4.9. The error also appears with 4.8 and 4.7, but here it appears first, when starting an analyse.
With 4.9 the error appears already before the GUI starts.

my system : old Quadcore Win7-64 with .Net 4.7
(this is just the first and most important part of the error message)

Code: Select all

************** Ausnahmetext **************
System.IO.FileNotFoundException: Die Datei oder Assembly "System.ValueTuple, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51" oder eine Abhängigkeit davon wurde nicht gefunden. Das System kann die angegebene Datei nicht finden.
Dateiname: "System.ValueTuple, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51"
...
https://rwbc-chess.de

trollwatch:
Chessqueen + chessica + AlexChess + Eduard + Sylwy
USGroup1
Posts: 33
Joined: Sun Oct 14, 2018 7:01 pm
Full name: Sina Vaziri

Re: program to generare epd file of tactical exercises from pgn

Post by USGroup1 »

Guenther wrote: Sat Jan 19, 2019 12:00 pm
USGroup1 wrote: Sat Jan 19, 2019 11:02 am
Uri Blass wrote: Fri Jan 18, 2019 9:51 am
1)good moves in the pgn that are significantly stronger than the second best move
Every sequence of trading pieces fit that description, you need more filters. ChessTricks uses similar filters, you might find it useful for this.
https://chesstricksapp.wordpress.com
I have tried the latest version 4.9. The error also appears with 4.8 and 4.7, but here it appears first, when starting an analyse.
With 4.9 the error appears already before the GUI starts.

my system : old Quadcore Win7-64 with .Net 4.7
(this is just the first and most important part of the error message)

Code: Select all

************** Ausnahmetext **************
System.IO.FileNotFoundException: Die Datei oder Assembly "System.ValueTuple, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51" oder eine Abhängigkeit davon wurde nicht gefunden. Das System kann die angegebene Datei nicht finden.
Dateiname: "System.ValueTuple, Version=4.0.2.0, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51"
...
Make sure that these files are in the folder:

Code: Select all

ChessTricks.exe
stockfish_19011016_x64.exe
System.Reactive.dll
System.Reactive.Linq.dll
It seems like it can't find stockfish engine file at startup. Also try "Run as administrator", see if that solves it.
User avatar
Guenther
Posts: 4605
Joined: Wed Oct 01, 2008 6:33 am
Location: Regensburg, Germany
Full name: Guenther Simon

Re: program to generare epd file of tactical exercises from pgn

Post by Guenther »

USGroup1 wrote: Sat Jan 19, 2019 12:53 pm
Make sure that these files are in the folder:

Code: Select all

ChessTricks.exe
stockfish_19011016_x64.exe
System.Reactive.dll
System.Reactive.Linq.dll
It seems like it can't find stockfish engine file at startup. Also try "Run as administrator", see if that solves it.
Of course I did all of this.

After making an offline install of .Net 4.72 all works here. Seems sth was missing in my 4.7 installation?
https://rwbc-chess.de

trollwatch:
Chessqueen + chessica + AlexChess + Eduard + Sylwy