New engine releases 2020

[Damir]

It is nice to see new strong engines on the scene, but what engine I was particularly was looking forward to was Xiphos. It has been months since any new release... I guess Milos no longer has time to work on his engine.

[xr_a_y]

It is nice to see new strong engines on the scene, but what engine I was particularly was looking forward to was Xiphos. It has been months since any new release... I guess Milos no longer has time to work on his engine.

Not much to improve in fact ! Being 6th on the CCRL 40/15 with a so well written 5500 lines program is quite a huge achievement for one human being.
What more to expect without massive testing facilities ?

[jorose]

Jonathan,

Winter 0.7 was positionally excellent already, and I liked its evaluation. I use it to analyze my opening repertoire and whole games in these openings, and its output makes sense. I guess it is because it doesn't reach absurd depths too fast, so I can understand it. On the other hand, I do not use it for matches, so I am not influenced by the rating things etc.

The evaluation is at times a bit optimistic, but I thought it was mainly at lower depths. With enough time, I thought it was quite reasonable. It missed (misevaluated) tactical shots from time to time, but the alternatives were OK. It reminds me of Komodo 7 and 8. I missed the fact that it was not user friendly enough (no multi-pv, no learning, no real options), but it was all right nonetheless.

I followed the 2. league of the TCEC and I liked Winter's play. It was impressive in the last round win against Booot.

Winter opens a new path for chess engines development, and I hope you will continue to work on it.

I am happy to hear you enjoy analyzing with it, though I am a bit surprised people are doing this.

I will likely add multi-pv and search-moves in Winter v0.9 release. A big feature that probably isn't supported by GUIs too well yet is WDL output. Winter for quite some time has already evaluated WDL probabilities at the leaves, however those get translated to win percentages for the sake of the actual tree search. Current plan is to use actual WDL in the tree search, which will propagate all the way to the root evaluation. Imo this will be a nice feature for analysis if GUIs support such score output.

Tablebases is frequently suggested and requested, though I am honestly not overly motivated to implement support for it. Chess960 will likely get supported before version 1.0, perhaps already for version 0.9

What kind of other features are you referring to or would like to see?

Thank you for your feedback,
Jonathan

[matejst]

When analyzing, I try to find and use engines whose lines make sense to me, and which I can use for playing OTB. Winter, Benjamin, Komodo TCEC, The Baron 3.29, Frenzee are engines I most often use. I cannot grasp plans and ideas of engines achieving depth 40 after one mn. Then, I have my test positions: when an engine choses e3 in a position where g3 is played and the fianchetto prepared, I am reticent to use it. Nowadays most engines are tuned with the Texel method, and the results are often ugly (although revealing the depth of chess).

I tested the new Winter 0.8 briefly and compared his move choice with Winter 0.7 (I ran them side by side): indeed, in relatively quiet positions (g3 or Be2 Najdorf, e.g.), Winter 0.7 will opt for a plan with f4, g4 when Winter 0.8 will chose a sequence like a4, Kh1, a maneuver like Nb3-d2-f1-e3. So, it could be that it plays more positionally, but I will know more in a few weeks. I also analyzed yesterday an ending from a Karpov-Tukmakov game and Winter's suggestions were on the point. It's endgame play is good, even without TBs, which are useful, I guess, mostly in eng-eng matches.

I did not notice the speed boost you mentioned (both engines run at roughly 200 kn on my old laptop under 1 thread, and I timed the depths after 1 and 2 mn, but while Winter 0.8 seems a bit faster achieving depth 20, Winter 0.7 reached depths 14-18 faster on my old system; I guess it is in relation with the peculiarities of my old laptop), but at 1s a move, the 0.8 version won a 40 games match easily, so I presume the evaluation is improved.

About features: the most featured free engine (and probably the best) was Sugar NN until the 26012020 version, with personalities, analysis features and learning. But I do think that the authors are kind enough to provide us with very strong engines as is, so it is a question of their courtesy. I would like you to pursue your project the way you see it fit, although I would greatly appreciate a more comfortable engine. I am especially interested in your work on integrating alpha/beta and NNs, perhaps using external weight files, and I feel you will find a way to implement continuous learning/training of the NNs.

Finally, a question: I had the impression (and it could be just an impression) that, while the eval is outstanding in positions of the French, the Ruy Lopez and Sicilian, Winter has difficulties with some strategic positions of the Slav. Does the way the hybrid NN was trained impact its evaluation?

[jorose]

It seems I might not have expressed myself clearly enough. While clang gives a big speed boost, Winter 0.8 should still be noticeably slower in terms of N/s when compared to its predecessor. The reason for this is a heavier eval function which relies on a CNN for pawn structure feature extraction. In the starting position I get 2/3 the N/s I did with v0.7

Thank you for the tip regarding Sugar, I will take a look into that.

The NN was trained in a purely supervised fashion. While I would like to go for some reinforcement learning approach again in the future, I think that is difficult to justify with my current resources and priorities. It is possible and likely that my dataset has an effect on which openings and pawn structures Winter plays well. Thank you for your insight regarding the slav, I will take a look into some of its games in the slav. Either way improving and increasing the size of my training dataset is a priority, as I feel it is currently a bottleneck in training.

[tmokonen]

Bagatur 2.2:
https://github.com/bagaturchess/Bagatur ... p?raw=true

MagiC 1.0:
https://github.com/zzhangb4/chess-engin ... 0/releases

[Gabor Szots]

MagiC 1.0:
https://github.com/zzhangb4/chess-engin ... 0/releases

Thanks Tony for making this known. Unfortunately I could not determine name and nationality of the author although it seems he is called Z. Zhang and might be from China.

[Raphexon]

Stockfish NNUE

TC=20s+0.2s threads=4
Score of stockfish.nnue vs PeSTO_bmi2: 18 - 5 - 7 [0.717]
Elo difference: 161.2 +/- 124.2, LOS: 99.7 %, DrawRatio: 23.3 %

https://github.com/nodchip/Stockfish/re ... 2020-05-30

All credit goes to Nodchip, I'm not the author. (And credit to SF team for the search too, obviously)
Zero idea how much ELO can be gained from extra training.

+instructions how to train a new net, or reinforcing an existing net.
Instructions are cleaned up a little.

Code: Select all

Nodchip released a new binary set "stockfish-nnue-2020-05-30" for training data generation and training.
https://github.com/nodchip/Stockfish/releases/tag/stockfish-nnue-2020-05-30
Please get it before trying the below.

Training in Stockfish+NNUE consists of two phases, "training data generation phase" and "training phase".

In the training data generation phase, we will create training data with the "gensfen" command.
In the first iteration, we will create training data with the original Stockfish evaluation function.
This can be done with "stockfish.nnue-gen-sfen-from-original-eval.exe" in "stockfish-nnue-2020-05-30".
The command will be like:

uci
setoption name Hash value 32768  <- This value must be lower than the total memory size of your PC.
setoption name Threads value 8  <- This value must be equal to or lower than the number of the logical CPU cores of your PC.
isready
gensfen depth 8 loop 10000000 output_file_name trainingdata\generated_kifu.bin
quit

Before creating the training data, please make a folder for the training data. 
In the command above, the name of the folder is "trainingdata".
The traning data generation takes a long time.  Please be patient.
For detail options of the "gensfen" command, please refer learn/learner.cpp.<- In the source code (src\learn\learner.cpp)

We also need validation data so that we measure if the training goes well. 
The command will be like:

uci
setoption name Hash value 32768
setoption name Threads value 8
isready
gensfen depth 8 loop 1000000 output_file_name validationdata\generated_kifu.bin
quit

Before creating the validation data, please make a folder for the validation data.  
In the command above, the name of the folder is "validationdata".

In the training phase, we will train the NN evalution function with the "learn" command.  Please use "stockfish.nnue-learn-use-blas.k-p_256x2-32-32.exe" for the "learn" command.
In the first iteration, we need to initialize the NN parameters with random values, and learn from learning data.
Setting the SkipLoadingEval option will initialize the NN with random parameters.  The command will be like:

uci
setoption name SkipLoadingEval value true
setoption name Threads value 8
isready
learn targetdir trainingdata loop 100 batchsize 1000000 eta 1.0 lambda 0.5 eval_limit 32000 nn_batch_size 1000 newbob_decay 0.5 eval_save_interval 10000000 loss_output_interval 1000000 mirror_percentage 50 validation_set_file_name validationdata\generated_kifu.bin
quit

Please make sure that the "test_cross_entropy" in the progress messages will be decreased.
If it is not decreased, the training will fail.  In that case, please adjust "eta", "nn_batch_size", or other parameters.
If test_cross_entropy is decreased enough, the traning will befinished.
Congrats!

We could repeat the "training data generation phase" and "training phase" again and again with the output NN evaluation functions in the previous iteration.
This is a kind of reinforcement learning.
After the first iteration, please use "stockfish.nnue-learn-use-blas.k-p_256x2-32-32.exe" to generate training data so that we use the output NN parameters in the previous iteration.
Also, please set "SkipLoadingEval" to false in the training phase so that the trainer loads the NN parameters in the previous iteration.

We also could change the network architecture.
The network architecuture in "stockfish-nnue-2020-05-30" is "k-p_256x2-32-32".
"k-p" means the input feature.
"k" means "king", the one-shot encoded position of a king.
"p" means "peace", the one-shot encoded position and type of a piece other than king.
"256x2-32-32" means the number of the channels in each hidden layer.
The number of the channels in the first hidden layer is "256x2".
The number of the channels in the second and the third is "32".

The standard network architecture in computer shogi is "halfkp_256x2-32-32".
"halfkp" means the direct product of "k" and "p" for each color.
If we use "halfkp_256x2-32-32", we could need more training data because the number of the network paramters is much larger than "k-p_256x2-32-32".
We could need 300,000,000 traning data for each iteration.

[Norbert Raimund Leisner]

FoxSEE 3.3.4 https://github.com/redsalmon91/FoxSEE/releases
KnightX 2.0 http://technochess.free.fr/
Minic 2.33 (unofficial edition) https://github.com/tryingsomestuff/Mini ... ter/Minic2 -
v2.31 https://github.com/tryingsomestuff/Minic/releases
Sapeli 1.90 https://github.com/SamuraiDangyo/Sapeli/releases
Stash 16.0 https://gitlab.com/mhouppin/stash-bot/-/releases

genecook – C++ - Xboard
https://github.com/genecook/chess/releases (v0.2 alpha Ubuntu)

robertomil81mil – UCI
https://github.com/robertomil81mil/PayFleens/releases (v1.0.1 Linux)

[silentshark]

Francesca 0.29 is released today, June 1st. You can download from www.silentshark.co.uk

From the readme file:

"Lots of changes. Improved endgame recognition (helped by new way of tracking material). Small
performance optimisations. Better concept of threat in the eval (pawn and minor piece threats).
Change to king safety to better consider threats and pieces in proximity to king. Bigger eval for
passed pawnscoring. Minor bugfixes.

Testing against other versions of Francesca and other engines shows a +45 ELO improvement at fast
timescales."