jja: convert CTG books to PolyGlot format (and more!)

[alpltl]

Show better step-by-step actions in the video tutorial and at the same time how to convert from CTG to BIN — this will be a real and visual help for everyone.

I am sorry but I do not have access to a Windows machine. I have already shared with you the demo which you may watch for visual guidance. Converting a CTG to a BIN is as easy as "jja.exe edit -o <path-to-bin> <path-to-ctg>", see "jja.exe edit --help" for more information.

[Engine]

Show better step-by-step actions in the video tutorial and at the same time how to convert from CTG to BIN — this will be a real and visual help for everyone.

I am sorry but I do not have access to a Windows machine. I have already shared with you the demo which you may watch for visual guidance. Converting a CTG to a BIN is as easy as "jja.exe edit -o <path-to-bin> <path-to-ctg>", see "jja.exe edit --help" for more information.

Only "jja.exe" for some reason does not start for me.

[alpltl]

Only "jja.exe" for some reason does not start for me.

I'm sorry to hear that you're still having trouble starting jja.exe on your Windows machine. Let's try to walk you through the process step by step to help you run the application and convert a CTG file to a BIN file.

1. First, open the Command Prompt on your Windows machine. You can do this by pressing the Windows key and typing cmd or Command Prompt. Click on the Command Prompt app when it appears in the search results.
2. Next, you need to navigate to the directory where the jja.exe file is located. You can do this using the cd command followed by the path to the folder containing the jja.exe file. For example, if the jja.exe file is located in the Downloads folder, you can type:

Code: Select all

cd C:\Users\YourUsername\Downloads

Make sure to replace "YourUsername" with your actual Windows username.

3. Now that you're in the directory containing the jja.exe file, you can run the application by typing jja.exe followed by the desired command. In your case, you want to convert a CTG file to a BIN file. To do this, use the following command:

Code: Select all

jja.exe edit -o output_file.bin input_file.ctg

Replace "output_file.bin" with the name you want for your converted BIN file and "input_file.ctg" with the path to the CTG file you want to convert.

4. Press Enter to execute the command. The application should now run, convert the CTG file to a BIN file, and save it in the specified location.

Please follow these steps and let me know if you are still experiencing any issues or if you have any further questions.

[chesskobra]

I have now made a test of the make option. I have a game collection with about 695000 games (started with caissabase, filtered games after 1980, both players rated >= 2400, each game with at least 50 plies). I made a book with --min-games 256. The resulting book is about 28k bytes, and pgn dump from jja gives only 7 lines. On the other hand with the same option -min-game 256 with polyglot, I get a book that is 80k bytes and has more than 1100 lines according to the pgn dump from jja (while polyglot info-book says more than 1200 white lines and more than 1200 black lines). Why is there so much difference between the books built with jja and polyglot for the same min-games value?

[alpltl]

I have now made a test of the make option. I have a game collection with about 695000 games (started with caissabase, filtered games after 1980, both players rated >= 2400, each game with at least 50 plies). I made a book with --min-games 256. The resulting book is about 28k bytes, and pgn dump from jja gives only 7 lines. On the other hand with the same option -min-game 256 with polyglot, I get a book that is 80k bytes and has more than 1100 lines according to the pgn dump from jja (while polyglot info-book says more than 1200 white lines and more than 1200 black lines). Why is there so much difference between the books built with jja and polyglot for the same min-games value?

The issue you reported pertained to the discrepancy between the book outputs of jja and polyglot when using the same --min-games value. Specifically, the jja output had less variations and a smaller size compared to polyglot​​.

In response to your issue, a commit has been made to the jja project. The commit, titled "polyglot: fix encoding of castle", corrected the way that castling moves are encoded. Previously, they were encoded as e1g1, e1c1, e8g8, and e8c8. After the correction, they are now properly encoded as e1h1, e1a1, e8h8, and e8a8​​.

While this fix might help address some of the discrepancy between jja and polyglot, it is worth noting that it does not necessarily resolve all the differences between the two. This commit is a significant step towards improving the accuracy of jja's output, but further work might still be required to completely align the outputs of jja and polyglot.

[Dann Corbit]

I gave it a try on Windows:

K:\chess\ctg>jja info tactics.ctg
# jja Chess File Information
Path: tactics.ctg
Size: 491.52 kB (491520 bytes)
Type: ctg
## jja CTG File Information
Total pages: 65

K:\chess\ctg>jja edit tactics.ctg -o tactics.bin
Creating output PolyGlot opening book.
Success creating output PolyGlot opening book.
Searching for positions in the CTG input file. This may take a while.
Found 144 chess positions in the CTG input file.
Saving 144 entries from input book to the output book.
Success copying entries from the input book to the output book.
Saving [=================================================>] 144/144

[Dann Corbit]

Output as PGN or EPD would be nice.

[alpltl]

I gave it a try on Windows:

Thank you very much.

Output as PGN or EPD would be nice.

PGN output support is already implemented, see my quote below for details. Note this is not in a released version yet, I have a few more items in my TODO list before I release a new version. Till then, I recommend you to use a git build. At the time of writing, the latest build for Windows is 991820. To download simply download "jja.exe" in the list of Artifacts on the left, or use this direct link to jja.exe. Note these links will expire in 90 days, hopefully by then I will have released jja-0.5.0.

I am happy to inform you that as of 1fffa5e, jja can export opening books as PGN. Again this is supported for all opening book formats jja supports (abk, ctg, bin, and obk). To use the functionality, simple specify a file with .pgn extension as --output argument to "jja edit". You can customize the PGN generation further with the arguments:

Code: Select all

      --event <pgn-event>
          Specify the Event tag for PGN output, defaults to book name
      --site <pgn-site>
          Specify the Site tag for PGN output, defaults to host name
      --date <pgn-date>
          Specify the Date tag for PGN output, defaults to last modification date
      --black <pgn-black>
          Specify the Black tag for PGN output [default: ?]
      --white <pgn-white>
          Specify the White tag for PGN output [default: ?]
      --result <pgn-result>
          Specify the Result tag for PGN output [default: *] [possible values: *, 1/2-1/2, 0-1, 1-0]

jja keeps track of the "Round" PGN tag automatically. Note, this change is not in a released version yet so to test it you need a git build. Browse to the respective operating system build, windows, linux-glibc, or linux-musl, and download the artifact on the left.

[alpltl]

We're excited to announce that JJA has reached another milestone with the release of version 0.5.0! Following our previous version 0.4.0, this latest update brings substantial enhancements, critical fixes, and notable new features that are sure to make your chess analysis even more powerful. Read more about it in the blog post here.

Download

- Windows: jja-0.5.0.exe, sha512sum, signature
- Linux-Glibc: jja-0.5.0-glibc.bin, sha512sum, signature
- Linux-Musl: jja-0.5.0-musl.bin, sha512sum, signature

To build from source, use

Code: Select all

cargo install jja

ChangeLog

0.5.0

important fix, for encoding of castling in polyglot books. Previously, we encoded castling as e1g1, e1c1, e8g8, and e8c8, whereas the correct encoding is e1h1, e1a1, e8h8, and e8a8.
when printing version with --version, prefer git version over package version for git builds
upgrade clap crate from 4.2 to 4.3
upgrade ctrlc crate from 3.2 to 3.3
upgrade rocksdb crate to 0.21.0 which bundles RocksDB-8.1.1
document all the public code and enable the lint #[deny(missing_docs)]
edit learned to export opening books in PGN format, use an output file with pgn extension to export an opening book to a PGN
add back the build dependency upon the built crate
find learned -l <max-ply>, --line=<max-ply> to display lines from the opening book as a table of opening variations reverse-sorted by cumulative weight
make --min-score now accepts floating point values as argument rather than an unsigned 64-bit integer
make learned --win-factor, --draw-factor, and --loss-factor to specify respective factors during score calculation, the defaults, 2, 1, and 0 respectively, resembles the original polyglot tool
find --tree=<max-ply> no longer panics on broken pipe, so it's more convenient to use with a pager
make learned -p, --min-pieces to specify the minimum number of pieces on the board for a position to be included in the book, defaults to 8
merge learned -c, --weight-cutoff to specify the minimum weight of entries to be included in the book
merge learned about merge strategies avg and wavg to merge using average weight or weighted average weight respectively; the weighted averages should be specified for wavg using -w, --weight1, and -W, --weight2
in-place editing of polyglot files allows editing empty books which makes it practical to create polyglot book from scratch
merge learned about merge strategies max, min, ours, and sum, default is sum which adds together move weights, max picks the one with the maximum weight, min picks up the one with the minimum weight, and ours always picks the entries from the first book
translate the README to German language
turn i18n support into a feature which defaults to on. For static linking this must be disabled as embed makes use of proc macros.
drop depedency on unused libc crate

0.4.1

fix docs.rs build
update shakmaty, and pgn-reader creates
add initial translation to Turkish language
add initial translation to German language
use gettext for i18n

[Dann Corbit]

Thanks for your efforts.
This is a very useful tool.