tool to find tactical mistakes in games?

[brtzsnr]

Is there a tool to find tactical mistakes in games?

Ideally I would give a set of games (.pgn), a player name (e.g. zurichess), the tool will run a stronger engine (e.g. stockfish or crafty) on each position and it would test if the move was a tactical mistakes (e.g. difference between actual move and best move >1 pawn).

[Ajedrecista]

Hello Alexandru:

Is there a tool to find tactical mistakes in games?

Ideally I would give a set of games (.pgn), a player name (e.g. zurichess), the tool will run a stronger engine (e.g. stockfish or crafty) on each position and it would test if the move was a tactical mistakes (e.g. difference between actual move and best move >1 pawn).

I do not know if this tool is what you are looking for:

Re: Looking for a tool that find ups and downs in engine's eval

I recommend you to read all the thread.

Regards from Spain.

Ajedrecista.

[bob]

Is there a tool to find tactical mistakes in games?

Ideally I would give a set of games (.pgn), a player name (e.g. zurichess), the tool will run a stronger engine (e.g. stockfish or crafty) on each position and it would test if the move was a tactical mistakes (e.g. difference between actual move and best move >1 pawn).

Crafty will do this trivially.

Create a PGN file. Only gotcha is to make sure that the player you want to do this test for has a consistent name, IE "Robert Hyatt" and "Bob Hyatt" won't cut it.

Then enter this:

annotate <filename> <side> <move-range> <margin> <time> <N>

filename is the name of the PGN file.

side is either the specific name of a player, and it will annotate the game for that player, or you can enter a color (b w or bw for both).

move-range is the range for the moves to annotate. 1-999 will get 'em all, or you can do 1-40 if you want to cut it off at move 40.

margin is the "trigger". If the best move it can find is > score for the move played in the game + margin, it will insert the score/PV info, if it is less than that nothing is added. You can use -99 to get EVERY move's PV inserted. This is a floating point value, 1 or 1.0 is one pawn. If you just want to pick up moves that lose material, you might use 1.0...

time is the time per move. Note that #games times #moves per game times time is the lower bound on the time this will take. Any move where the best move (according to Crafty) was not played will take at least 2x the normal time since it has to get a score/PV for the move played and the best move.

N is the number of moves to display. IE 2 says show the two best moves rather than just the best.

A good "blunder check" might be

annotate file.pgn name 1-99 1.0 30

to use a 30 second search to look only for moves that lost more than a pawn compared to the best move. (note the N is optional and defaults to "1" if omitted).

I use this regularly. If you want to do it overnight, you can stretch the time limit out to 3 minutes or whatever. If you reduce the margin, more moves will generate comments. That -99 I mentioned will produce a comment for EVERY move showing the score/PV for the best move if this move was played in the game, or the score/PV for the move played AND the best move if they are different by 99 pawns... in either direction (better or worse). This can begin to burn CPU time, obviously...

edit: forgot this: once you complete the above, look for a filename matching the file you used, but with a .can (crafty annotation) extension. That will contain the complete games from the original file + lines inserted by Crafty.

[brtzsnr]

Thanks. The game below was generated by Arena.

I tried

Code: Select all

White(1): annotate all.pgn Zurichess 1-99 2.0 5
Player name doesn't match any PGN name tag, retry
White(1): annotate all.pgn Black 1-99 2.0 5
Player name doesn't match any PGN name tag, retry
White(1): annotate all.pgn "Zurichess" 1-99 2.0 5
Player name doesn't match any PGN name tag, retry

I'm obviously missing something.



[pgn]
[Event "YAT-general-repetition"]
[Site "EDDY-PC"]
[Date "2015.12.03"]
[Round "1"]
[White "Zurichess"]
[Black "Arasan 1.81x"]
[Result "0-1"]
[BlackElo "2200"]
[ECO "E20"]
[Opening "Nimzo-Indian"]
[Time "10:40:50"]
[Variation "4.f3 c5 5.d5"]
[WhiteElo "2200"]
[TimeControl "40/240:40/240:40/240"]
[Termination "adjudication"]
[PlyCount "83"]
[WhiteType "program"]
[BlackType "program"]

1. d4 {0.00/1} e6 2. c4 {0.00/1} Bb4+ 3. Bd2 {0.00/1} Qe7 4. e4 {0.00/1}
Nf6 5. Bd3 {+0.48/13 2} Bxd2+ {-0.41/19 6} 6. Qxd2 {+0.41/15 12} d6
{-0.42/20 6} 7. Ne2 {+0.31/15 5} O-O {-0.49/20 11} 8. O-O {+0.29/16 16} Nc6
{-0.51/19 6} 9. Bc2 {+0.42/13 2} b6 {-0.46/18 6} 10. Nbc3 {+0.49/12 2} Ba6
{-0.44/17 5} 11. Ba4 {+0.54/15 10} Na5 {-0.54/19 0} 12. b3 {+0.53/13 3} Bb7
{-0.53/17 4} 13. d5 {+0.55/14 8} c5 {-0.49/17 0} 14. Qd3 {+0.57/13 7} Rac8
{-0.45/17 7} 15. Ng3 {+0.63/11 2} Rfd8 {-0.53/16 7} 16. Rad1 {+0.59/10 3}
h5 {-0.56/18 7} 17. Rfe1 {+0.70/11 1} Ng4 {-0.49/19 10} 18. dxe6 {+0.90/11
2} fxe6 {-0.52/18 6} 19. Nb5 {+0.59/14 17} Qf6 {-0.33/17 6} 20. f3
{+1.25/11 2} Ne5 {-1.04/20 20} 21. Qc3 {+1.01/15 2} h4 {-0.67/20 4} 22. Ne2
{+1.06/15 12} Nac6 {-0.55/20 0} 23. Nxd6 {+1.37/13 3} Nd4 {-0.16/22 6} 24.
Nxd4 {+1.11/15 8} Rxd6 {-0.07/21 0} 25. Nb5 {+0.97/14 3} Rxd1 {-0.04/22 4}
26. Rxd1 {+0.27/15 16} Rf8 {0.00/23 0} 27. h3 {+1.14/12 3} Qg5 {+0.41/22 7}
28. Rf1 {+0.69/12 1} Qg3 {+0.72/21 7} 29. Qe3 {+0.81/13 2} Rd8 {+0.86/23 7}
30. Nxa7 {+0.53/12 2} Ng6 {+2.55/24 7} 31. Qe1 {-0.31/17 4} Nf4 {+3.39/28
5} 32. Qxg3 {-0.50/19 30} hxg3 {+5.83/30} 33. Kh1 {-0.50/17 27} Rd2
{+6.20/31 0} 34. Rg1 {-0.62/17 3} Nxh3 {+6.24/30 9} 35. Ra1 {-0.62/17 11}
Nf4 {+6.63/28 10} 36. Rg1 {-2.70/14 0} Kf7 {+6.68/27 12} 37. b4 {-3.64/17
1} Rxa2 {+7.71/23 7} 38. bxc5 {-3.64/15 2} Ne2 {+10.46/28 10} 39. Bb3
{+0.02/2 0} Rxa7 {+M1/11 0} 40. Re1 {-M4/17 0} Ra8 {+M0/6 0} 41. Rxe2
{-M3/63 0} Ra1+ {+M0/4 0} 42. Bd1 {-M3/63 Arena Adjudication} 0-1
[/pgn]

[jordanbray]

IMHO, it sounds like you're running into search bugs, so I would think a list of mates would solve that. HGM posted a large database of mates in EPD format, and I'm actually running through them now to debug some search issues.

You can then extract all the positions where your engine is missing a mate in x, which should make it easier to debug.

I noticed at the end your engine was incorrectly reporting mate scores, which could be an indication of some search issues somewhere.

[bob]

Thanks. The game below was generated by Arena.

I tried

Code: Select all

White(1): annotate all.pgn Zurichess 1-99 2.0 5
Player name doesn't match any PGN name tag, retry
White(1): annotate all.pgn Black 1-99 2.0 5
Player name doesn't match any PGN name tag, retry
White(1): annotate all.pgn "Zurichess" 1-99 2.0 5
Player name doesn't match any PGN name tag, retry

I'm obviously missing something.



[pgn]
[Event "YAT-general-repetition"]
[Site "EDDY-PC"]
[Date "2015.12.03"]
[Round "1"]
[White "Zurichess"]
[Black "Arasan 1.81x"]
[Result "0-1"]
[BlackElo "2200"]
[ECO "E20"]
[Opening "Nimzo-Indian"]
[Time "10:40:50"]
[Variation "4.f3 c5 5.d5"]
[WhiteElo "2200"]
[TimeControl "40/240:40/240:40/240"]
[Termination "adjudication"]
[PlyCount "83"]
[WhiteType "program"]
[BlackType "program"]

1. d4 {0.00/1} e6 2. c4 {0.00/1} Bb4+ 3. Bd2 {0.00/1} Qe7 4. e4 {0.00/1}
Nf6 5. Bd3 {+0.48/13 2} Bxd2+ {-0.41/19 6} 6. Qxd2 {+0.41/15 12} d6
{-0.42/20 6} 7. Ne2 {+0.31/15 5} O-O {-0.49/20 11} 8. O-O {+0.29/16 16} Nc6
{-0.51/19 6} 9. Bc2 {+0.42/13 2} b6 {-0.46/18 6} 10. Nbc3 {+0.49/12 2} Ba6
{-0.44/17 5} 11. Ba4 {+0.54/15 10} Na5 {-0.54/19 0} 12. b3 {+0.53/13 3} Bb7
{-0.53/17 4} 13. d5 {+0.55/14 8} c5 {-0.49/17 0} 14. Qd3 {+0.57/13 7} Rac8
{-0.45/17 7} 15. Ng3 {+0.63/11 2} Rfd8 {-0.53/16 7} 16. Rad1 {+0.59/10 3}
h5 {-0.56/18 7} 17. Rfe1 {+0.70/11 1} Ng4 {-0.49/19 10} 18. dxe6 {+0.90/11
2} fxe6 {-0.52/18 6} 19. Nb5 {+0.59/14 17} Qf6 {-0.33/17 6} 20. f3
{+1.25/11 2} Ne5 {-1.04/20 20} 21. Qc3 {+1.01/15 2} h4 {-0.67/20 4} 22. Ne2
{+1.06/15 12} Nac6 {-0.55/20 0} 23. Nxd6 {+1.37/13 3} Nd4 {-0.16/22 6} 24.
Nxd4 {+1.11/15 8} Rxd6 {-0.07/21 0} 25. Nb5 {+0.97/14 3} Rxd1 {-0.04/22 4}
26. Rxd1 {+0.27/15 16} Rf8 {0.00/23 0} 27. h3 {+1.14/12 3} Qg5 {+0.41/22 7}
28. Rf1 {+0.69/12 1} Qg3 {+0.72/21 7} 29. Qe3 {+0.81/13 2} Rd8 {+0.86/23 7}
30. Nxa7 {+0.53/12 2} Ng6 {+2.55/24 7} 31. Qe1 {-0.31/17 4} Nf4 {+3.39/28
5} 32. Qxg3 {-0.50/19 30} hxg3 {+5.83/30} 33. Kh1 {-0.50/17 27} Rd2
{+6.20/31 0} 34. Rg1 {-0.62/17 3} Nxh3 {+6.24/30 9} 35. Ra1 {-0.62/17 11}
Nf4 {+6.63/28 10} 36. Rg1 {-2.70/14 0} Kf7 {+6.68/27 12} 37. b4 {-3.64/17
1} Rxa2 {+7.71/23 7} 38. bxc5 {-3.64/15 2} Ne2 {+10.46/28 10} 39. Bb3
{+0.02/2 0} Rxa7 {+M1/11 0} 40. Re1 {-M4/17 0} Ra8 {+M0/6 0} 41. Rxe2
{-M3/63 0} Ra1+ {+M0/4 0} 42. Bd1 {-M3/63 Arena Adjudication} 0-1
[/pgn]

I just tested, using a match.pgn produced by xboard. It worked just fine. Are you certain that "Zurichess" is spelled exactly correct? IE it has to match exactly, no extra version info or anything... case sensitive.. whole nine yards...

The PGN should look like this (tags):

[Event "Computer Chess Game"]
[Site "scrappy.attlocal.net"]
[Date "2015.12.01"]
[Round "1"]
[White "Crafty-25.0"]
[Black "Scorpio_2.7.7"]
[Result "0-1"]
[TimeControl "60+1"]
[FEN "r1bqk2r/1pppbppp/p7/4P3/B2nn3/8/PPP2PPP/RNBQ1RK1 w kq - 0 1"]
[SetUp "1"]

{--------------
r . b q k . . r
. p p p b p p p
p . . . . . . .
. . . . P . . .
B . . n n . . .
. . . . . . . .
P P P . . P P P
R N B Q . R K .
white to play
--------------}
1. Qxd4 {-0.09/20} Nc5 {-0.31/19 4} 2. Nc3 O-O {-0.32/18 7} 3. Rd1 Re8
{-0.38/17 2.3} 4. Bf4 {+0.06/18 1.1} b6 {-0.21/16 2.1} 5. Bb3 {+0.19/17 5}
Bb7 6. Bd5 {+0.25/18 2.4} Bxd5 {-0.26/18 0.3} 7. Nxd5 {+0.21/19 2.1} c6
{-0.06/16 2.2} 8. Nxe7+ Qxe7 {-0.02/20 2.1} 9. Qc4 {+0.11/20 15} Qh4
{+0.21/17 2.0} 10. Rd4 {+0.00/19 2.4} Ne6 {+0.23/20 1.1} 11. g3
{+0.00/21 0.7} Qe7 {+0.10/20 1.9} 12. Rd6 {+0.27/17 2.1} Nxf4

It works with odd starting positions and everything.

To annotate the above file, I did this:

annotate match.pgn Crafty-25.0 1-99 1 5

Just to test for sanity, and it worked as expected.

[Dann Corbit]

I'm surprised Crafty 25.0 lost that game.
It's absolutely kicking tail on my machine.
Looks a little stronger than the current Nirvana, but not enough games to be sure.
It whitewashed the current Arasan in a test match.

BTW, there is something wrong with the current Nirvana. There are rare crashes and also bizarre mate scores that are not mates from time to time. Still, it clearly plays well.

[brtzsnr]

Everything is spelled correctly. This is the interaction.

Code: Select all

$ head all.pgn
[Event "YAT-general-repetition"]
[Site "EDDY-PC"]
[Date "2015.12.03"]
[Round "1"]
[White "Zurichess"]
[Black "Arasan 1.81x"]
[Result "0-1"]
[BlackElo "2200"]
[ECO "E20"]
[Opening "Nimzo-Indian"]

$ crafty

Machine is NUMA, 1 nodes (16 cpus/node)

unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].

Crafty v23.4 (1 cpus)

White(1): annotate all.pgn Zurichess 1-99 2.0 5
Player name doesn't match any PGN name tag, retry
White(1): 

[Ferdy]

I have just drafted such tool for windows.

Command line is something like this,

Code: Select all

blunder_checker2 --analyzer sf6.exe --hash 128 --threads 1 --inputfile g1.pgn --outputfile out_g1.pgn --blundermargincp 100 --analysistimems 1000 --player "Zurichess"

Output is below, there is NAG (?) if there is blunder, based on margin, also added the move and eval of the best move found by engine. By default I don't analyze the move of the player if the score of analyzer engine is already -300 cp. The eval of actual move is also shown based from the analyzer engine. This is capable of analyzing more than 1 game in a file. If you are interested I will prepare the exe file. If you have other request in the output, mention it. Test epd can be easily generated, I think I will add it so that future version of the program can just be tested with it quickly.

Code: Select all

[Event "YAT-general-repetition"]
[Site "EDDY-PC"]
[Date "2015.12.03"]
[Round "1"]
[White "Zurichess"]
[Black "Arasan 1.81x"]
[Result "0-1"]
[WhiteElo "2200"]
[BlackElo "2200"]
[WhiteType "program"]
[BlackType "program"]
[ECO "E20"]
[Opening "Nimzo-Indian"]
[Variation "4.f3 c5 5.d5"]
[TimeControl "40/240:40/240:40/240"]
[Time "10:40:50"]
[Termination "adjudication"]
[PlyCount "83"]
[Annotator "sf6 @1.0s/pos"]

{blunder_margin = 1.00, hash = 128, threads = 1}
1.d4 e6 2.c4 Bb4+ 3.Bd2 Qe7 4.e4 
Nf6 5.Bd3 Bxd2+ 6.Qxd2 d6 7.Ne2 O-O 
8.O-O Nc6 9.Bc2 b6 10.Nbc3 Ba6 11.Ba4 
Na5 12.b3 Bb7 13.d5 c5 14.Qd3 Rac8 
15.Ng3 Rfd8 16.Rad1 h5 17.Rfe1 Ng4 18.dxe6 
fxe6 19.Nb5 Qf6 20.f3 Ne5 21.Qc3 h4 
22.Ne2 Nac6 23.Nxd6 Nd4 24.Nxd4 Rxd6 25.Nb5 
Rxd1 26.Rxd1 Rf8 27.h3 Qg5 28.Rf1 Qg3 
29.Qe3 Rd8 30.Nxa7 $2 {-3.38/22} (30.Qf2 {-0.15/17}) Ng6 31.Qe1 Nf4 32.Qxg3 
hxg3 33.Kh1 Rd2 34.Rg1 Nxh3 35.Ra1 Nf4 
36.Rg1 Kf7 37.b4 Rxa2 38.bxc5 Ne2 39.Bb3 
Rxa7 40.Re1 Ra8 41.Rxe2 Ra1+ 42.Bd1 0-1

[pgn][Event "YAT-general-repetition"]
[Site "EDDY-PC"]
[Date "2015.12.03"]
[Round "1"]
[White "Zurichess"]
[Black "Arasan 1.81x"]
[Result "0-1"]
[WhiteElo "2200"]
[BlackElo "2200"]
[WhiteType "program"]
[BlackType "program"]
[ECO "E20"]
[Opening "Nimzo-Indian"]
[Variation "4.f3 c5 5.d5"]
[TimeControl "40/240:40/240:40/240"]
[Time "10:40:50"]
[Termination "adjudication"]
[PlyCount "83"]
[Annotator "sf6 @1.0s/pos"]

{blunder_margin = 1.00, hash = 128, threads = 1}
1.d4 e6 2.c4 Bb4+ 3.Bd2 Qe7 4.e4
Nf6 5.Bd3 Bxd2+ 6.Qxd2 d6 7.Ne2 O-O
8.O-O Nc6 9.Bc2 b6 10.Nbc3 Ba6 11.Ba4
Na5 12.b3 Bb7 13.d5 c5 14.Qd3 Rac8
15.Ng3 Rfd8 16.Rad1 h5 17.Rfe1 Ng4 18.dxe6
fxe6 19.Nb5 Qf6 20.f3 Ne5 21.Qc3 h4
22.Ne2 Nac6 23.Nxd6 Nd4 24.Nxd4 Rxd6 25.Nb5
Rxd1 26.Rxd1 Rf8 27.h3 Qg5 28.Rf1 Qg3
29.Qe3 Rd8 30.Nxa7 $2 {-3.38/22} (30.Qf2 {-0.15/17}) Ng6 31.Qe1 Nf4 32.Qxg3
hxg3 33.Kh1 Rd2 34.Rg1 Nxh3 35.Ra1 Nf4
36.Rg1 Kf7 37.b4 Rxa2 38.bxc5 Ne2 39.Bb3
Rxa7 40.Re1 Ra8 41.Rxe2 Ra1+ 42.Bd1 0-1[/pgn]

[Wijnand]

Dear Ferdinand,
Your program blunder_checker seems a great idea. At the moment, I am using your wonderful program (thank you!) Game analyzer (ga.exe) for that purpose.
I create a result.txt file with all FENs and evaluations by:

Type params.txt | ga.exe > result.txt

where params.txt contains all the answers , each one on a line, to the questions ga asks, with three returns at the end of the file, forcing ga.exe to return to Ms-Dos when finished.
Then I read that result.txt file into Excel , separate the evaluations from the FEN strings. I use a lot of formulas and macros to do that. Combining all that work into one exe file would be great.

I am particularly interested in big drops/rises of the evaluation, with the most interesting a sequence like +0.6 +1.2 +0.7 that signals: mistake but neither of the players has seen it. So I would welcome an output with, for every move on a separate line:
FEN *sepatator* move *separator* evaluation after the move
and, if you could manage that,
*separator* change in evaluation since last move.

Best regards, Wijnand.