jja: convert CTG books to PolyGlot format (and more!)

[alpltl]

jja-0.9.1 has been released.

## Download

* Windows: jja-0.9.1.exe, sha512sum, signature
* Linux GLibc: jja-0.9.1-glibc.bin, sha512sum, signature
* Linux Musl: jja-0.9.1-musl.bin, sha512sum, signature

To install from source, use "cargo install jja".

## ChangeLog

- upgrade nix crate from 0.27 to 0.28.
- upgrade tempfile crate from 3.8 to 3.10.
- upgrade rust-embed crate from 8.1 to 8.4.
- upgrade rusqlite crate from 0.30 to 0.31.
- upgrade rayon crate from 1.8 to 1.10.
- upgrade hostname crate from 0.3 to 0.4.
- Add support for restoring BrainLearn experience files from Lichess
evaluations export JSON.
- TotalPlyCount PGN tag is now supported in make filters.
- fix --win,draw,loss-factor calculation for make
- Add 50-moves detection and 3-position repetition detection to the play
subcommand.
- upgrade once_cell crate from 1.18 to 1.19.
- upgrade rayon crate from 1.7 to 1.8.
- upgrade zstd crate from 0.12 to 0.13.
- upgrade rusqlite crate from 0.29 to 0.30.
- upgrade dialoguer crate from 0.10 to 0.11.
- upgrade i18n-embed crate from 0.13 to 0.14.
- upgrade rust-embed crate from 6.8 to 8.0.
- upgrade built crate from 0.6 to 0.7.
- fix quote matching at word boundaries when no wildcard characters are given.
- merge learned -O<u16>, --outlier-threshold=<u16> to specify an outlier threshold
where moves with weights higher than the given threshold are filtered out from the
merged output book. This stands from the fact that if the weight differences of
the same move in two different books have a high difference, calculating any
average may not yield meaningful results. The default is 0, which disables this
feature.
- merge learned --strategy dynmid which stands for
jja::merge::MergeStrategy::DynamicMidpoint.
- merge learned --strategy wdist which stands for
jja::merge::MergeStrategy::WeightedDistance.
- upgrade nix crate from 0.26 to 0.27 and add the signal feature.
- merge learned --strategy wmedian which stands for
jja::merge::MergeStrategy::WeightedMedian.
- merge learned --strategy entropy which stands for
jja::merge::MergeStrategy::Entropy.
- merge learned --strategy quadratic which stands for
jja::merge::MergeStrategy::QuadraticMean. This strategy merges using the Quadratic
Mean (Root Mean Square) approach. The Quadratic Mean (also known as the Root Mean
Square) is a statistical measure of the magnitude of a set of numbers. It offers a
more balanced view, especially when dealing with numbers of varying magnitudes.
- merge learned --strategy lavg which stands for
jja::merge::MergeStrategy::LogarithmicAverage. This strategy merges using a
logarithmic averaging approach. Given that logarithmic functions compress large
values and expand small values, we can use them to get a merge strategy that's
sensitive to differences in smaller weights while being more resistant to
disparities in larger weights.

[alpltl]

I tried the ctg->abk conversion with the following command:

Code: Select all

jja edit Book.ctg -o Book.abk -C -N --author=amchess --comment=amchessBook --probability-priority=2 --probability-games=15 --probability-win-percent=0

The book is created, but the probabilities in Arena are totally wrong (decreasing order) and overall the book is unusable with an engine (demo mode, for example), contrary to ohter abk books.
In what I'm wrong?

Interesting. First of all thank you very much for testing and your valuable feedback. To help me diagnose the issue better can you send me the output of "jja find" for a few positions from the CTG book and corresponding entries in the ABK book where you spotted that the probabilities are wrong? You can use "jja find" with a FEN or a PGN line, e.g: "jja find -p'e4 c6' book.ctg" or jja find -f<FEN> book.abk".