Official Release of Ethereal 10.00 (Two-years Anniversary)

[AndrewGrant]

Hello all,

This is the official release of Ethereal 10.00

Ethereal was initially released in June of 2016

This (nearly) marks the two-year anniversary of official development

nopopcnt, popcnt, pext, and android-armv8 compiles are here

There are a handful of people I must thank ...

Firstly to Lucas Braesch. He has been taking a look at some of the core code behind Ethereal. He has cleaned up a great deal of it, made the workings more understandable, and often times faster. He, and his experiences in chess programming, are very valuable.

Secondly, to Can Cetin, author of Defenchess (formerly SCTR). We have been talking at great length while watching the TCEC games. He has given me a handful of elo gaining ideas. Hopefully I have done the same for him. Having someone to bounce ideas off of is both refreshing and fruitful. I look forward to watching Defenchess' return to TCEC in Season 13.

Thirdly, to the team behind TCEC. The community surrounding TCEC is absolutely wonderful. The competition has inspired me to improve Ethereal as fast as possible. They have provided a great deal of exposure for Ethereal. I am very grateful for the invitation.

Finally, to the testers. The CCRL guys have tested 18 versions of Ethereal for their 40/4 list, and 15 versions for 40/40. The guys over at CEGT have tested nearly a dozen. Ipmanchess has recently added Ethereal to his lists, and FGRL has been playing with each official release since 9.00. Thank you for your interest in this project. Two years ago it was me and one instructor. Now so many more.

Here is my testing data, for those concerned with elo increases

All games played on a Ryzen 1950x, using 8moves_v3 for openings

Code: Select all

    |================|=================|========|=======|======|======|======|======|========|=======|
    |      Test      |       Base      |  Time  |  Hash |   W  |   L  |   D  |   N  |   Elo  | Error |
    |================|=================|========|=======|======|======|======|======|========|=======|
    | Ethereal  9.92 |   Ethereal 9.65 |  5+.05 |   8MB |  848 |  431 | 1221 | 2500 |  58.50 |  9.78 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.94 |   Ethereal 9.65 |  5+.05 |   8MB |  874 |  411 | 1215 | 2500 |  65.10 |  9.81 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.97 |   Ethereal 9.65 |  5+.05 |   8MB |  911 |  358 | 1231 | 2500 |  78.14 |  9.75 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal 10.00 |   Ethereal 9.65 |  5+.05 |   8MB |  913 |  434 | 1153 | 2500 |  67.40 | 10.06 |
    |================|=================|========|=======|======|======|======|======|========|=======|

    |================|=================|========|=======|======|======|======|======|========|=======|
    |      Test      |       Base      |  Time  |  Hash |   W  |   L  |   D  |   N  |   Elo  | Error |
    |================|=================|========|=======|======|======|======|======|========|=======|
    | Ethereal  9.86 |  Laser 18052018 |  5+.05 |   8MB | 1765 | 1474 | 1761 | 5000 |  20.24 |  7.76 |
    | Ethereal  9.86 |  Laser 18052018 | 20+.20 |  32MB | 1488 | 1473 | 2039 | 5000 |   1.04 |  7.40 |
    | Ethereal  9.86 |  Laser 18052018 | 60+.60 | 128MB | 1226 | 1434 | 2340 | 5000 | -14.46 |  7.02 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.86 |    Xiphos 0.2.6 |  5+0.5 |   8MB |  954 |  861 |  685 | 2500 |  12.93 | 11.61 |
    | Ethereal  9.86 |    Xiphos 0.2.6 | 20+.20 |  32MB |  878 |  791 |  831 | 2500 |  12.10 | 11.13 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.86 | Dchess 18052018 |  5+.05 |   8MB | 1101 |  618 |  781 | 2500 |  67.98 | 11.47 |
    | Ethereal  9.86 | Dchess 18052018 | 20+.20 |  32MB |  871 |  651 |  978 | 2500 |  30.65 | 10.66 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.97 |  Laser 27052018 |  5+.05 |   8MB |  971 |  656 |  873 | 2500 |  44.01 | 11.06 |
    | Ethereal  9.97 |  Laser 27052018 | 20+.20 |  32MB |  788 |  678 | 1034 | 2500 |  15.30 | 10.43 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.97 |    Xiphos 0.2.8 |  5+.05 |   8MB | 1015 |  767 |  718 | 2500 |  34.58 | 11.56 |
    | Ethereal  9.97 |    Xiphos 0.2.8 | 20+.20 |  32MB |  915 |  743 |  842 | 2500 |  23.94 | 11.12 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal  9.97 | Dchess 27052018 |  5+.05 |   8MB | 1078 |  571 |  851 | 2500 |  71.45 | 11.23 |
    | Ethereal  9.97 | Dchess 27052018 | 20+.20 |  32MB |  907 |  557 | 1036 | 2500 |  48.96 | 10.48 |
    |----------------|-----------------|--------|-------|------|------|------|------|--------|-------|
    | Ethereal 10.00 |   Laser30052018 | 20+.20 |  32MB |  351 |  248 |  401 | 1000 |  35.91 | 16.77 |
    |================|=================|========|=======|======|======|======|======|========|=======|

As always, a change log for those interested

Patches with a * are done by Lucas Braesch

Code: Select all

     9.66   : Cleanup some issues in Fathom's code base
            : Use Ethereal's movegen and bit functions
     9.67   : Simplify the way in which helper thread are stopped
            : Move some time management code to its own file
            : Added the UCI option MoveOverhead
     9.68   : There was an eval term for Minor threads against
            : Majors. Changed this to be against just Queens
     9.69   : Remove about 40 Lines from moveIsPsuedoLegal()
     9.70   : Implement SEE(). Use to prune in QSearch
     9.71   : Use SEE() to prune captures and quiets in Search
     9.72   : Penalize weak squares in the King area
     9.73   : Use different SEE() pruning values for improving nodes
     9.74   : Use SEE() for ProbCut, define it for special moves
     9.75   : Change to the definition for Check Extension
     9.76   : Only Check Extend for captures in the search
     9.77 * : Cleanup some code dealing with Bitboards
     9.78 * : Cleanup some code using bit functions (+Speedup)
     9.79 * : Rewrite Knight and King Attack generation
     9.80 * : Rewrite Magic Bitboard generation (-300 Lines)
     9.81   : Use the aging replacement scheme for the TT from Stockfish
     9.82 * : Precompute pawnAttacks(square) (+Speedup)
     9.83 * : Enable PEXT compiles using (+Speedup)
     9.84   : Add nopopcnt, popcnt, and pext targets to the makefile
     9.85   : Store a static eval in the TT
     9.86   : Adjust version number and fix makefile typos
     9.87 * : Adjust the Bitboard masks to be functions instead of arrays
     9.88 * : Retire square.c and square.h, merge with other units
     9.89 * : Add code for printBitboard(), simplify printBoard()
     9.90 * : Rewrite FEN parser, add FEN output to board printing
     9.91   : Simplify some conditions for doing less Late Move Reductions
     9.92   : Remove the eval term QueenAttackedByMinor
     9.93   : Retune threat values after the last change
     9.94   : Do less Move Count Pruning when in an improving node
     9.95   : Condense the Tuner, remove 500+ Lines. More useable.
     9.96   : Evaluate overloaded minors and majors as per Stockfish
     9.97   : Full King Safety rewrite
     9.98 * : Simplify some conversion of moves to strings
     9.99   : Evaluate Safe Checks in King Safety
    10.00   : Redefine Safe Checks, and retune

Thanks,
Andrew Grant

[Graham Banks]

Well done Andrew! Ethereal keeps getting better and better.

[nabildanial]

Congrats on another milestone! Ethereal has an interesting playstyle, very enjoyable to watch on TCEC. I wasn't aware of Lucas' contributions in Ethereal's codebase, he's certainly a very valuable contributor in the computer chess scene as a whole although I still bear a slight grudge against him for removing the aggressiveness and cowardice parameters in SF .

[AndrewGrant]

Thanks Andrew. Have I got a false impression that 9.97 was better than 10.0?

Yes -- I think that would be a false impression. That table is just a set 2,500 game pairing between versions. You can see that the error bars are pretty high, and the time control is fairly short.

Additionally, here are the stats confirming patch 9.99:

Code: Select all

SPRT @ 7.5+0.075s Threads=1 Hash=8MB
LLR: 2.24 (-2.20, 2.20) [0.00, 5.00]
Games: 3050 W: 772 L: 655 D: 1623

SPRT @ 30.0+0.3s Threads=1 Hash=32MB
LLR: 2.24 (-2.20, 2.20) [0.00, 5.00]
Games: 2675 W: 566 L: 458 D: 1651

And for patch 10.00 (9.98 was non functional, so no data there):

Code: Select all

SPRT @ 7.5+0.075s Threads=1 Hash=8MB
LLR: 2.21 (-2.20, 2.20) [0.00, 5.00]
Games: 3550 W: 879 L: 761 D: 1910

SPRT @ 30.0+0.3s Threads=1 Hash=32MB
LLR: 2.20 (-2.20, 2.20) [0.00, 5.00]
Games: 2450 W: 518 L: 413 D: 1519

[CMCanavessi]

Awesome job!!! Ethereal has to be the fastest growing chess engine (strenght-wise) of the recent times.

Ponder next?

[Rubinus]

Who is *pext.exe? Modern from *popcnt.exe?

[Dann Corbit]

Thank you for your efforts.
I can't wait to put my nose in your code.

[tmokonen]

Exactly 1000 commits for version 10.00.

Thank you for the new version.

[AndrewGrant]

Exactly 1000 commits for version 10.00.

Looks like you discovered my versioning secret

[AndrewGrant]

Testing not far away...

I look forward to it. I think I'll slow down on releases to justify doing 4cpu testing