How to cut these pgn games.

[Jonathan003]

Sorry I mean thanks Ferdy lol. It was to late to change it when I was seeing the mistake.

[Jonathan003]

I find a way to make it work. I just took the convert-cerebellum.py file included in pollyglot_tollerant and changed the name to pgn_modifier.py and replaced the text within with the code you send.

You can find pollyglot_tollerant here by the way: https://chess.massimilianogoi.com/downl ... ttolerant/

It's great for making bin books with really large pgn files. It also has an option to convert a bin book to pgn.

I have tried the pgn_modifier.py on larger pgn files.

It doesn't always put the eco code and opening name as annotation in the games.
Here is the pgn file I used and the result.

https://drive.google.com/open?id=1dJ0GF ... zyDaTtV-2-

Let me know if the download link doesn't work. It's the first time a share a file this way.

Would it in addition be possible to make a script that delete all other comments and variations. But keep the eco code and opening annotation. Without cutting the games? I could use this to for my repertoires.

[Jonathan003]

I have tried it with the games of a black repertoire to.
Here is the link: https://drive.google.com/open?id=1g_PRQ ... B8Qhq3omxq

Most of the time it works correct. But sometimes there is no annotation with the eco code and the openings name. Also sometime there is another annotation right in front of the eco code.

I'm trying to figer out how you did it. But it seams really complicated.

[Ferdy]

Most of the time it works correct. But sometimes there is no annotation with the eco code and the openings name. Also sometime there is another annotation right in front of the eco code.

I'm trying to figer out how you did it. But it seams really complicated.

Could you post a sample input game where the output does not work?

[Jonathan003]

You need a trailing asterisk in the PGN body text after trimming.
Seems like you are simply looking for a named opening db.

Maybe you want something like this:
https://drive.google.com/open?id=1LZsVP ... U_-jZvxINR

No I first want to study my repertoire to eco related lengt. My plan is to study a wide repertoire, only the initial moves. And a more narrow repertoire many ply's deep.

[Jonathan003]

Most of the time it works correct. But sometimes there is no annotation with the eco code and the openings name. Also sometime there is another annotation right in front of the eco code.

I'm trying to figer out how you did it. But it seams really complicated.

Could you post a sample input game where the output does not work?

Hi Ferdy I will send you some examples later. It's almost 3 am here in Belgium. Thank you for willing to help me. I appreciate it very much.

[Jonathan003]

I think your script doesn't always work because chessbase 15 generates errors in the notation when doing tactical analysis.
I have send an email to chessbase. I hope they can fix it.
Here are some exemples. I have many more exemples.

[Ferdy]

I see why the script failed. ECO D4 is missing in the generated ECO in the script, the script only has D04. I will revise the script to handle that D4.

The ECO here https://www.365chess.com/eco.php is using Letter + 2 digits.

[Ferdy]

Try this code if it will work.

Changes:

Code: Select all

Dev log:
    v0.2
    * Read ECO letter + single digit, that is A1, B2 etc.
    * Prompt user of pgn file and the output is 'output_[user pgn filename].

Code: Select all

"""
pgn_modifier.py

Read pgn and remove some comments and cut the move length at specific move.

Requirements:
    python 3
    python-chess v0.26.0 and up
    
Dev log:
    v0.2
    * Read ECO letter + single digit, that is A1, B2 etc.
    * Prompt user of pgn file and the output is 'output_[user pgn filename].

"""


import os
import chess.pgn

    
def main(): 
    pgnfn = input('enter pgn filename? ')
    output_pgnfn = 'output_' + pgnfn
    
    # Delete existing output_mygames.pgn
    if os.path.isfile(output_pgnfn):
        os.remove(output_pgnfn)
    
    # Build ECO, used to cut the game length
    ECO = []
    for i in range(100):
        # A00 to E99, A0, A1 ... A9 ... E9
        if i < 10:
            aecos = 'A{}'.format(i)
            becos = 'B{}'.format(i)
            cecos = 'C{}'.format(i)
            decos = 'D{}'.format(i)
            eecos = 'E{}'.format(i)
        
        aeco = 'A{:02d}'.format(i)
        beco = 'B{:02d}'.format(i)
        ceco = 'C{:02d}'.format(i)
        deco = 'D{:02d}'.format(i)
        eeco = 'E{:02d}'.format(i)
        
        ECO.append(aecos)
        ECO.append(becos)
        ECO.append(cecos)
        ECO.append(decos)
        ECO.append(eecos)
        
        ECO.append(aeco)
        ECO.append(beco)
        ECO.append(ceco)
        ECO.append(deco)
        ECO.append(eeco)
    
    gcnt = 0
    
    # Parse games in pgn file
    with open(pgnfn, 'r') as pgn:
        game = chess.pgn.read_game(pgn)        
        while game:
            gcnt += 1
            
            # Console progress
            print('game: {}'.format(gcnt))
            
            mygame = chess.pgn.Game()
            mynode = mygame
            
            # Copy orig game header to our saved game header
            for k, v in game.headers.items():
                mygame.headers[k] = v
            
            for node in game.mainline():
                game_move = node.move                 
                move_comment = node.comment
                
                # Modify move comment, remove text before ECO
                # [%emt 0:00:05] A87: Dutch Defence: Leningrad System: Nf3
                if '[' in move_comment and ']' in move_comment:                    
                    move_comment = move_comment.split(']')[1].strip()
                
                mynode = mynode.add_main_variation(game_move)
                
                # Check the move comment where we can cut the game length
                is_foundvalidcomment = False
                for c in ECO:
                    if c in move_comment:
                        is_foundvalidcomment = True
                        break
                if is_foundvalidcomment:
                    mynode.comment = move_comment  
                
                # Don't copy other moves we stop here
                if is_foundvalidcomment:
                    break
            
            # Print to output file
            with open(output_pgnfn, 'a') as w:
                w.write('{}\n\n'.format(mygame))
            
            # Print to console
            print(mygame)
            print()
            
            # Read another game
            game = chess.pgn.read_game(pgn)


if __name__ == '__main__':
    main()

Example.

enter pgn filename? error1.pgn
game: 1
[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "White Opening"]
[Black "My Black Opening"]
[Result "*"]
[PlyCount "38"]

1. c4 e6 2. d4 Nf6 3. Nc3 Bb4 4. Bg5 b6 5. Nf3 Bb7 6. e3 h6 7. Bh4 Bxc3+ 8. bxc3 d6 9. Bd3 Nbd7 10. O-O g5 11. Bg3 Ne4 12. Nd2 { E3: Queen's Indian/Nimzo-Indian hybrid: 4 Nc3 Bb7 5 Bg5 h6 6 Bh4
Bb4 } *

The output filename is
output_error1.pgn

If your input pgn file is big, remove this in the code, so that it will complete faster.

Code: Select all

# Print to console
print(mygame)                        
print()

[Jonathan003]

Here are some exemples where it goes wrong.

pgn: my games.pgn

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[ECO "A58"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[PlyCount "39"]

{[%evp 18,27,45,34,34,34,40,34,48,48,54,54]} 1. d4 Nf6 2. c4 c5 3. d5 b5 4.
cxb5 a6 5. bxa6 Bxa6 6. Nc3 d6 7. g3 g6 8. Bg2 Bg7 9. Nf3 Nbd7 10. Rb1 {
White is slightly better.} Nb6 {[%emt 0:00:05] A8: Benko/Volga Gambit: Lines
with 5 bxa6} 11. a4 O-O 12. O-O Ra7 13. b3 Qa8 {[%cal Rf6d5]} 14. Nh4 {
Predecessor:} Rb8 $146 15. Bb2 Nbd7 16. Ba1 Ne5 17. Qc2 Rab7 18. Rfd1 Qa7 19.
Bf1 Ne8 20. Ne4 {1-0 (44) Piket,J (2609)-Topalov,V (2700) Wijk aan Zee 1999} *

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[ECO "A67"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[PlyCount "34"]

{[%evp 17,27,6,32,32,42,42,46,46,46,46,58,22]} 1. d4 Nf6 2. c4 c5 3. d5 e6 4.
Nc3 exd5 5. cxd5 d6 6. e4 g6 7. f4 Bg7 8. Bb5+ Nfd7 9. a4 a6 10. Bd3 {White is
slightly better. A67: Modern Benoni: Taimanov Variation} Qh4+ 11. g3 Qd8 12.
Nf3 O-O 13. O-O Nf6 14. f5 {Predecessor:} Nbd7 $146 15. Bg5 c4 16. Bxc4 Qb6+
17. Kh1 Qxb2 {1-0 (27) Bosiocic,M (2566)-Jacob,V (2247) Germany 2015} *

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[ECO "A70"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[PlyCount "27"]

{[%evp 13,27,45,45,16,26,26,31,31,31,31,42,42,46,46,46,46]} 1. d4 Nf6 2. c4 c5
3. d5 e6 4. Nc3 d6 5. Nf3 exd5 6. cxd5 g6 7. e4 a6 8. Qe2 {[%emt 0:00:07]
White wants to play e5. A0: Modern Benoni: 6 e4 g6 7 Nf3} Bg4 9. e5 dxe5 10.
Qxe5+ {White is slightly better.} Qe7 11. Qxe7+ Bxe7 12. Ne5 Nbd7 13. Nxg4 Nxg4
14. h3 $146 ({Predecessor:} 14. Bf4 Nge5 15. O-O-O Bd6 {1-0 (61) Carlsen,M
(2770)-Ivanchuk,V (2746) Leon 2009}) *

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[ECO "A87"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[PlyCount "37"]

{[%evp 19,27,39,42,41,53,53,58,58,58,58]} 1. d4 f5 2. Nf3 Nf6 3. g3 g6 4. Bg2
Bg7 5. O-O d6 6. c4 O-O 7. Nc3 Qe8 8. Re1 Qf7 9. b3 Ne4 10. Bb2 Nc6 {White is
slightly better.} 11. Rc1 {[%emt 0:00:05] A87: Dutch Defence: Leningrad System:
Nf3} h6 (11... e5 12. d5 Nb4 13. a3 Nxc3 14. Rxc3 Na6 15. b4 e4 16. Nd4 Bd7 17.
f3 c5 18. dxc6 bxc6 19. fxe4 fxe4 {0-1 (40) Eljanov,P (2663)-Gurevich,M (2652)
Khanty-Mansiysk 2005}) 12. d5 Ne5 13. Nxe5 Bxe5 14. f4 {Predecessor:} Bg7 $146
15. Qc2 Nxc3 16. Bxc3 Bd7 17. e4 Bxc3 18. Qxc3 fxe4 19. Rxe4 {1-0 (50) Carlsen,
M (2450)-Potapov,P (2267) Chalkidiki 2003} *


Here is the output. pgn output_mygames.pgn


[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]

*

[Event "?"]
[Site "?"]
[Date "????.??.??"]
[Round "?"]
[White "?"]
[Black "?"]
[Result "*"]

1. d4 Nf6 2. c4 c5 3. d5 b5 4. cxb5 a6 5. bxa6 Bxa6 6. Nc3 d6 7. g3 g6 8. Bg2 Bg7 9. Nf3 Nbd7 10. Rb1 Nb6 11. a4 O-O 12. O-O Ra7 13. b3 Qa8 14. Nh4 Rb8 15. Bb2 Nbd7 16. Ba1 Ne5 17. Qc2 Rab7 18. Rfd1 Qa7 19. Bf1 Ne8 20. Ne4 *

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[ECO "A67"]
[PlyCount "34"]

1. d4 Nf6 2. c4 c5 3. d5 e6 4. Nc3 exd5 5. cxd5 d6 6. e4 g6 7. f4 Bg7 8. Bb5+ Nfd7 9. a4 a6 10. Bd3 { White is
slightly better. A67: Modern Benoni: Taimanov Variation } *

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[ECO "A70"]
[PlyCount "27"]

1. d4 Nf6 2. c4 c5 3. d5 e6 4. Nc3 d6 5. Nf3 exd5 6. cxd5 g6 7. e4 a6 8. Qe2 Bg4 9. e5 dxe5 10. Qxe5+ Qe7 11. Qxe7+ Bxe7 12. Ne5 Nbd7 13. Nxg4 Nxg4 14. h3 *

[Event "Chess Position Trainer"]
[Site "Chess Position Trainer"]
[Date "????.??.??"]
[Round "?"]
[White "My White Opening"]
[Black "Black Opening"]
[Result "*"]
[Annotator "Tactical Analysis 2.10 (6s)"]
[ECO "A87"]
[PlyCount "37"]

1. d4 f5 2. Nf3 Nf6 3. g3 g6 4. Bg2 Bg7 5. O-O d6 6. c4 O-O 7. Nc3 Qe8 8. Re1 Qf7 9. b3 Ne4 10. Bb2 Nc6 11. Rc1 { A87: Dutch Defence: Leningrad System:
Nf3 } *

I especially selected some games where chessbase 15 displayed these strange numbers before the eco code. I think it's strange that the numbers before the eco code are not to find in the text of the pgn files. But it is related I think.

I didn't tried the new script yet. I will try it.