Converting PGN to CSV file

[Jon12345]

I need to convert a PGN file into a CSV file. Any utilities out there that do this?

Thanks,

Jon

[Don]

I need to convert a PGN file into a CSV file. Any utilities out there that do this?

Thanks,

Jon

Yes, sed, awk, m4, etc.

When you say CSV, what are the fields that you want to have the in CSV file?

[Jon12345]

Hey Don, you have given me hope! Spent ages on this so far.

Well, I want to take any PGN file and get it into an Access database. Or maybe just a CSV file. Fields can be any or all of them.

The moves are the most important.

Those tools you mentioned, are they parsing commands in UNIX or something? Or tools I can download?

I want to be able to do the conversions on a regular basis.

[Don]

Hey Don, you have given me hope! Spent ages on this so far.

Well, I want to take any PGN file and get it into an Access database. Or maybe just a CSV file. Fields can be any or all of them.

The moves are the most important.

Those tools you mentioned, are they parsing commands in UNIX or something? Or tools I can download?

I want to be able to do the conversions on a regular basis.

You can make a little converter using a good scripting language such as ruby, perl, python, etc. It would be trivial.

With linux you could probably do it easily with these unix tools I mentioned in a quick and dirty script.

Are you on windows?

I assume you would want to see the 7 tag roster, then have each move in a new field?

I can make this script in 10 minutes.

[Jon12345]

Well, not quite. I want to still use it in PGN format. But I have to convert it to CSV format so that I can use a tool to import all the data. The import tool requires CSV format.

So, I was hoping that I could somehow take the PGN data and put commas after each block of data. I assume the moves (which seem to cover several lines) would have a comma at the end of the line.

[Don]

Well, not quite. I want to still use it in PGN format. But I have to convert it to CSV format so that I can use a tool to import all the data. The import tool requires CSV format.

So, I was hoping that I could somehow take the PGN data and put commas after each block of data. I assume the moves (which seem to cover several lines) would have a comma at the end of the line.

What is a block of data? A single pgn game?

I think if you are trying to export this game to a sql style database you should skip the CSV part, define your database schema, and then make a utility that will export a pgn file to the appropriate sql insert statements. I have done this type of thing many times, though not with PGN files.

[Jon12345]

I'm struggling to explain myself very well. Forgive me.

Here is some of my sample PGN data. I've stripped out some of the header stuff.

1. e4 c5 2. f4 e6 3. Nf3 Nc6 4. c3 d5 5. e5 f5 6. d4 Nh6 7. a3 Nf7 8. Be3 Qb6
9. Qd2 Bd7 10. Qf2 c4 11. Bxc4 dxc4 12. d5 Qc7 13. dxc6 Bxc6 14. Bxa7 Bxf3
15. gxf3 g5 16. Be3 gxf4 17. Bxf4 Nxe5 18. Bxe5 Qxe5+ 19. Qe2 Qxe2+ 20. Kxe2
h5 21. Nd2 Rc8 22. Rhg1 Kf7 23. Rg2 Be7 24. Rag1 Bf6 25. Nf1 e5 26. Ne3 Ke6
27. Rd1 Rhg8 28. Rxg8 Rxg8 29. Nxc4 Rg2+ 30. Kd3 Rxh2 31. Rd2 Rh3 32. Ke2 b5
33. Ne3 Rh2+ 34. Ke1 Rxd2 35. Kxd2 Bg5 36. Ke2 Bxe3 37. Kxe3 h4 38. Kf2 e4
39. Kg2 e3 40. Kh3 e2 0-1

1. e4 c5 2. f4 e6 3. Nf3 Nc6 4. c3 d5 5. e5 f5 6. d4 Nh6 7. a3 Nf7 8. Be3 Qb6
9. Qd2 Bd7 10. Qf2 c4 11. Bxc4 dxc4 12. d5 Qc7 13. dxc6 Bxc6 14. Bxa7 Bxf3
15. gxf3 g5 16. Be3 gxf4 17. Bxf4 Nxe5 18. Bxe5 Qxe5+ 19. Qe2 Qxe2+ 20. Kxe2
h5 21. Nd2 Rc8 22. Rhg1 Kf7 23. Rg2 Be7 24. Rag1 Bf6 25. Nf1 e5 26. Ne3 Ke6
27. Rd1 Rhg8 28. Rxg8 Rxg8 29. Nxc4 Rg2+ 30. Kd3 Rxh2 31. Rd2 Rh3 32. Ke2 b5
33. Ne3 Rh2+ 34. Ke1 Rxd2 35. Kxd2 Bg5 36. Ke2 Bxe3 37. Kxe3 h4 38. Kf2 e4
39. Kg2 e3 0-1

I would like to turn this into a CSV file so I can use this import program. Not sure how doing SQL inserts will help if it needs CSV?

I'm kinda thinking I need to have a comma after each game block.

[Don]

Earlier you mentioned getting it into access, which I assume is Microsoft access which is a SQL database.

So is that what you are trying to do? You want it in CSV so that you can "import" it to something, and if that something is Microsoft Access then CSV is a round about way to do it.

If you convert the PGN to SQL statements (in a text file) then it's a much more natural way to export.

Don

I'm struggling to explain myself very well. Forgive me.

Here is some of my sample PGN data. I've stripped out some of the header stuff.

1. e4 c5 2. f4 e6 3. Nf3 Nc6 4. c3 d5 5. e5 f5 6. d4 Nh6 7. a3 Nf7 8. Be3 Qb6
9. Qd2 Bd7 10. Qf2 c4 11. Bxc4 dxc4 12. d5 Qc7 13. dxc6 Bxc6 14. Bxa7 Bxf3
15. gxf3 g5 16. Be3 gxf4 17. Bxf4 Nxe5 18. Bxe5 Qxe5+ 19. Qe2 Qxe2+ 20. Kxe2
h5 21. Nd2 Rc8 22. Rhg1 Kf7 23. Rg2 Be7 24. Rag1 Bf6 25. Nf1 e5 26. Ne3 Ke6
27. Rd1 Rhg8 28. Rxg8 Rxg8 29. Nxc4 Rg2+ 30. Kd3 Rxh2 31. Rd2 Rh3 32. Ke2 b5
33. Ne3 Rh2+ 34. Ke1 Rxd2 35. Kxd2 Bg5 36. Ke2 Bxe3 37. Kxe3 h4 38. Kf2 e4
39. Kg2 e3 40. Kh3 e2 0-1

1. e4 c5 2. f4 e6 3. Nf3 Nc6 4. c3 d5 5. e5 f5 6. d4 Nh6 7. a3 Nf7 8. Be3 Qb6
9. Qd2 Bd7 10. Qf2 c4 11. Bxc4 dxc4 12. d5 Qc7 13. dxc6 Bxc6 14. Bxa7 Bxf3
15. gxf3 g5 16. Be3 gxf4 17. Bxf4 Nxe5 18. Bxe5 Qxe5+ 19. Qe2 Qxe2+ 20. Kxe2
h5 21. Nd2 Rc8 22. Rhg1 Kf7 23. Rg2 Be7 24. Rag1 Bf6 25. Nf1 e5 26. Ne3 Ke6
27. Rd1 Rhg8 28. Rxg8 Rxg8 29. Nxc4 Rg2+ 30. Kd3 Rxh2 31. Rd2 Rh3 32. Ke2 b5
33. Ne3 Rh2+ 34. Ke1 Rxd2 35. Kxd2 Bg5 36. Ke2 Bxe3 37. Kxe3 h4 38. Kf2 e4
39. Kg2 e3 0-1

I would like to turn this into a CSV file so I can use this import program. Not sure how doing SQL inserts will help if it needs CSV?

I'm kinda thinking I need to have a comma after each game block.

[Jon12345]

The tool I want to use is not Access, although I mentioned Access as I could manipulate it there and export it as CSV.

I have found the PGN format very tricky to manipulate. Wondering if you have to use RegEx to do it.

[Dann Corbit]

Earlier you mentioned getting it into access, which I assume is Microsoft access which is a SQL database.

So is that what you are trying to do? You want it in CSV so that you can "import" it to something, and if that something is Microsoft Access then CSV is a round about way to do it.

If you convert the PGN to SQL statements (in a text file) then it's a much more natural way to export.

Don

I'm struggling to explain myself very well. Forgive me.

Here is some of my sample PGN data. I've stripped out some of the header stuff.

1. e4 c5 2. f4 e6 3. Nf3 Nc6 4. c3 d5 5. e5 f5 6. d4 Nh6 7. a3 Nf7 8. Be3 Qb6
9. Qd2 Bd7 10. Qf2 c4 11. Bxc4 dxc4 12. d5 Qc7 13. dxc6 Bxc6 14. Bxa7 Bxf3
15. gxf3 g5 16. Be3 gxf4 17. Bxf4 Nxe5 18. Bxe5 Qxe5+ 19. Qe2 Qxe2+ 20. Kxe2
h5 21. Nd2 Rc8 22. Rhg1 Kf7 23. Rg2 Be7 24. Rag1 Bf6 25. Nf1 e5 26. Ne3 Ke6
27. Rd1 Rhg8 28. Rxg8 Rxg8 29. Nxc4 Rg2+ 30. Kd3 Rxh2 31. Rd2 Rh3 32. Ke2 b5
33. Ne3 Rh2+ 34. Ke1 Rxd2 35. Kxd2 Bg5 36. Ke2 Bxe3 37. Kxe3 h4 38. Kf2 e4
39. Kg2 e3 40. Kh3 e2 0-1

1. e4 c5 2. f4 e6 3. Nf3 Nc6 4. c3 d5 5. e5 f5 6. d4 Nh6 7. a3 Nf7 8. Be3 Qb6
9. Qd2 Bd7 10. Qf2 c4 11. Bxc4 dxc4 12. d5 Qc7 13. dxc6 Bxc6 14. Bxa7 Bxf3
15. gxf3 g5 16. Be3 gxf4 17. Bxf4 Nxe5 18. Bxe5 Qxe5+ 19. Qe2 Qxe2+ 20. Kxe2
h5 21. Nd2 Rc8 22. Rhg1 Kf7 23. Rg2 Be7 24. Rag1 Bf6 25. Nf1 e5 26. Ne3 Ke6
27. Rd1 Rhg8 28. Rxg8 Rxg8 29. Nxc4 Rg2+ 30. Kd3 Rxh2 31. Rd2 Rh3 32. Ke2 b5
33. Ne3 Rh2+ 34. Ke1 Rxd2 35. Kxd2 Bg5 36. Ke2 Bxe3 37. Kxe3 h4 38. Kf2 e4
39. Kg2 e3 0-1

I would like to turn this into a CSV file so I can use this import program. Not sure how doing SQL inserts will help if it needs CSV?

I'm kinda thinking I need to have a comma after each game block.

Here is what you need:
A PGN parser.
The PGN Parser reads the PGN file and turns it into a parse tree.
Then you walk the parse tree and form SQL statements which insert the data into your SQL database.

Perl PGN parser:
http://search.cpan.org/~gmax/Chess-PGN- ... 9/Parse.pm

F# PGN Parser:
http://blogs.msdn.com/b/chrsmith/archiv ... -in-f.aspx

Java PGN Parser:
http://www.deligiannakis.com/code/chess ... .java.html

Java PGN Parser with formal Antlr grammar:
http://www.java-chess.de/start.htm

More Java Antlr stuff (see bottom of page):
http://csc.colstate.edu/woolbright/CPSC5135.xml

Another Java PGN parser:
http://www.radebatz.net/mano/javacc/

Another Java PGN parser:
http://sourceforge.net/projects/pgnparser/


PHP PGN Parser:
http://greg.chiaraquartet.net/archives/ ... nally.html

C# Projects with PGN parser:
http://www.codeproject.com/KB/cs/gpchesscontrol.aspx
http://www.codeproject.com/KB/game/SrcChess.aspx

Python PGN Parser:
http://pyparsing.wikispaces.com/file/view/pgn.py

C++ PGN Parser (sample use is demonstrated in PGN2EPD program):
http://remi.coulom.free.fr/

Spirit C++ chess parser:
http://boost-spirit.com/repository/appl ... _chess.cpp

THere are lots more besides these, of course. I guess that 1/2 of all open source chess programs have a PGN parser