games with material balance

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Robert Pope
Posts: 522
Joined: Sat Mar 25, 2006 7:27 pm

games with material balance

Post by Robert Pope » Tue Jul 07, 2020 12:14 am

I'm trying to filter pgn files for certain endings, except I want the PGN to be truncated once that ending is reached (e.g. KQRKB).

pgn-extract -z mat_kqrkb.txt -D -o kqrkb.pgn 1.pgn 2.pgn 3.pgn

will find the KQRKB games, but it gives the full pgn of the game, rather than stopping once KQRKB is reached. Is there another flag or tool I can try?

User avatar
Ajedrecista
Posts: 1455
Joined: Wed Jul 13, 2011 7:04 pm
Location: Madrid, Spain.
Contact:

Re: Games with material balance.

Post by Ajedrecista » Wed Jul 15, 2020 12:08 pm

Hello Robert:

Probably combining pgn-extract and Norm's EPD utilities could do the trick:

Norm's utilities.

Readme for EPD Tools & Utilities

Code: Select all

[...]

Comments:
     
     1, To output games from a "pgn" file whose ending position 
        has a user-specified number range of pieces:
        
          PGN-Extract -Wepd --nofauxep -s -otemp.epd alpha.pgn

          epdFin temp.epd
          
          epdPieces outF.epd 10 12
          
          numExtract numbers alpha.pgn  
          
        The above commands will output all games in alpha.pgn that 
        have 10, 11 or 12 pieces in the ending position.        
                 
     2. To output "pgn" games having a user-specified number range 
        of pieces in the position after a user-specified ply:
        
          PGN-Extract -Wepd --nofauxep -s -otemp.epd alpha.pgn

          epdPly temp.epd 30
          
          epdPieces outY.epd 14 15
          
          numExtract numbers alpha.pgn   
          
        The above commands will output all games in alpha.pgn that 
        have 14 or 15 pieces in the position after ply #30.
        
[...]
Using epdPieces utility and setting the parameter to 5 in the case of KQRkb. I have not tried myself because I am unable to extract the games with pgn-extract and its -z flag (I would ask for help), although it might not work because you would need to know at what ply the 5-man endgame is reached (counting 32 - 5 = 27 'x' capture characters in each game), which will be different between games.

You can also take a look at other EPD utilities such as epdNumPiece and epdMaterial in the same readme file, just in case. Good luck!

Regards from Spain.

Ajedrecista.

Norm Pollock
Posts: 1029
Joined: Thu Mar 09, 2006 3:15 pm
Location: Long Island, NY, USA
Contact:

Re: games with material balance

Post by Norm Pollock » Thu Jul 16, 2020 12:32 am

Hi,

I think the way to do this is to use "epdMaterial" to extract the epd records with the precise material, then follow up with "pgnPosition" which is a script utilizing "epdPosition".

Summary: Use the output file from "epdMaterial", which is "outV.epd", and rename it to "search.epd". Then use "search.epd" together with the included script "pgnPosition" on the original "pgn" file. It will produce "outZ.pgn" with the games having those positions as well as "manifest-ps" which tells the game number for each position. All tools, including "pgn-Extract.exe", "epdMaterial.exe", "epdPosition.exe", "numExtract.exe" (from 40H-PGN) and "pgnPosition.cmd" must be in the same folder with the data or on the system path.

"pgnPosition" script is described in the "epdPosition" section of the readme and is in the "40H-EPD" download..

All needed instructions are in the "readme-EPD.txt" file on my page (given above).

-Norm
It is not the strongest of the species that survive, nor the most intelligent, but the one most responsive to change. -- Charles Darwin

Robert Pope
Posts: 522
Joined: Sat Mar 25, 2006 7:27 pm

Re: games with material balance

Post by Robert Pope » Thu Jul 30, 2020 3:39 am

Following up on this, I ended up adding code to Abbess to pull the selection of pgn files I wanted - pgn games truncated to where a certain number of pieces are left.

However, I actually only want games that are still in progress, i.e. no games ending in checkmate or draw. pgn-extract has --checkmate which sort of the opposite of what I want (by collecting only checkmate games). Is there a handy tool that can filter on partial/incomplete games?

Zenmastur
Posts: 898
Joined: Sat May 31, 2014 6:28 am

Re: games with material balance

Post by Zenmastur » Thu Jul 30, 2020 5:46 am

Robert Pope wrote:
Thu Jul 30, 2020 3:39 am
Following up on this, I ended up adding code to Abbess to pull the selection of pgn files I wanted - pgn games truncated to where a certain number of pieces are left.

However, I actually only want games that are still in progress, i.e. no games ending in checkmate or draw. pgn-extract has --checkmate which sort of the opposite of what I want (by collecting only checkmate games). Is there a handy tool that can filter on partial/incomplete games?
First KQRvsKB isn't a common endgame. I searched a database of 7.5 million games and found only 67.

This is so few of them you can do what you want by hand unless you are handling hundreds of millions or billions of games.

Because the endgame is so lopsided the end result will almost always be a checkmate if play is continued. Of the 67 games only 9 of them end in a drawn game.
Only 2 defining forces have ever offered to die for you.....Jesus Christ and the American Soldier. One died for your soul, the other for your freedom.

Ferdy
Posts: 4270
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: games with material balance

Post by Ferdy » Thu Jul 30, 2020 9:40 am

Robert Pope wrote:
Thu Jul 30, 2020 3:39 am
Following up on this, I ended up adding code to Abbess to pull the selection of pgn files I wanted - pgn games truncated to where a certain number of pieces are left.

However, I actually only want games that are still in progress, i.e. no games ending in checkmate or draw. pgn-extract has --checkmate which sort of the opposite of what I want (by collecting only checkmate games). Is there a handy tool that can filter on partial/incomplete games?
Try this python-chess script. Install python-chess, modify pgnfn and define your material balance. It will parse the game from move 1 to end but will stop and print the truncated game once it found your material balance.

Code: Select all

"""
material_balance.py

Needed:
    Python

Requirements:
    Python-chess
"""


import chess.pgn


def material_balance(game):
    sgame = None        
    
    for node in game.mainline():
        board = node.board()
        
        Q = len(board.pieces(chess.QUEEN, chess.WHITE))                
        q = len(board.pieces(chess.QUEEN, chess.BLACK))
        R = len(board.pieces(chess.ROOK, chess.WHITE))
        r = len(board.pieces(chess.ROOK, chess.BLACK))
        B = len(board.pieces(chess.BISHOP, chess.WHITE))
        b = len(board.pieces(chess.BISHOP, chess.BLACK))
        N = len(board.pieces(chess.KNIGHT, chess.WHITE))
        n = len(board.pieces(chess.KNIGHT, chess.BLACK))
        P = len(board.pieces(chess.PAWN, chess.WHITE))
        p = len(board.pieces(chess.PAWN, chess.BLACK))
        
        wmi, bmi = B + N, b + n
        wma, bma = Q + R, q + r

        # Create your material balance.
        
        # RvBN, minors is 1 pawn ahead
        if (Q == 0 and R == 1 and wmi == 0
                and bma == 0 and bmi == 2 and b == 1
                and p - 1 == P):
            sgame = chess.pgn.Game().from_board(board)
            break
        
        # QRvB
        # if (Q == 1 and R == 1 and wmi == 0 and P == 0
        #     and bma == 0 and bmi == 1 and b == 1 and p == 0):
        #     sgame = chess.pgn.Game().from_board(board)
        #     break
    
    return sgame


def main():
    pgnfn = 'top10-60min.pgn'
    
    with open(pgnfn) as pgn:
        while True:
            game = chess.pgn.read_game(pgn)
                   
            if game is None:
                break
            
            sgame = material_balance(game)
            
            if sgame is not None:
                print(f'{sgame}\n')


if __name__ == '__main__':
    main()

Robert Pope
Posts: 522
Joined: Sat Mar 25, 2006 7:27 pm

Re: games with material balance

Post by Robert Pope » Fri Jul 31, 2020 10:56 pm

This is so much cleaner than what I had been trying to do - Thank you!
Ferdy wrote:
Thu Jul 30, 2020 9:40 am
Robert Pope wrote:
Thu Jul 30, 2020 3:39 am
Following up on this, I ended up adding code to Abbess to pull the selection of pgn files I wanted - pgn games truncated to where a certain number of pieces are left.

However, I actually only want games that are still in progress, i.e. no games ending in checkmate or draw. pgn-extract has --checkmate which sort of the opposite of what I want (by collecting only checkmate games). Is there a handy tool that can filter on partial/incomplete games?
Try this python-chess script. Install python-chess, modify pgnfn and define your material balance. It will parse the game from move 1 to end but will stop and print the truncated game once it found your material balance.

Code: Select all

"""
material_balance.py

Needed:
    Python

Requirements:
    Python-chess
"""


import chess.pgn


def material_balance(game):
    sgame = None        
    
    for node in game.mainline():
        board = node.board()
        
        Q = len(board.pieces(chess.QUEEN, chess.WHITE))                
        q = len(board.pieces(chess.QUEEN, chess.BLACK))
        R = len(board.pieces(chess.ROOK, chess.WHITE))
        r = len(board.pieces(chess.ROOK, chess.BLACK))
        B = len(board.pieces(chess.BISHOP, chess.WHITE))
        b = len(board.pieces(chess.BISHOP, chess.BLACK))
        N = len(board.pieces(chess.KNIGHT, chess.WHITE))
        n = len(board.pieces(chess.KNIGHT, chess.BLACK))
        P = len(board.pieces(chess.PAWN, chess.WHITE))
        p = len(board.pieces(chess.PAWN, chess.BLACK))
        
        wmi, bmi = B + N, b + n
        wma, bma = Q + R, q + r

        # Create your material balance.
        
        # RvBN, minors is 1 pawn ahead
        if (Q == 0 and R == 1 and wmi == 0
                and bma == 0 and bmi == 2 and b == 1
                and p - 1 == P):
            sgame = chess.pgn.Game().from_board(board)
            break
        
        # QRvB
        # if (Q == 1 and R == 1 and wmi == 0 and P == 0
        #     and bma == 0 and bmi == 1 and b == 1 and p == 0):
        #     sgame = chess.pgn.Game().from_board(board)
        #     break
    
    return sgame


def main():
    pgnfn = 'top10-60min.pgn'
    
    with open(pgnfn) as pgn:
        while True:
            game = chess.pgn.read_game(pgn)
                   
            if game is None:
                break
            
            sgame = material_balance(game)
            
            if sgame is not None:
                print(f'{sgame}\n')


if __name__ == '__main__':
    main()

Post Reply