Pyrrhic, Fathom for Humanoids.

[AndrewGrant]

I took master of jdart/fathom as a starting point. So at some point Ronald got dropped from those headers. I will add him back.

In regards to the includes/headers/defines issue: Its moot now, since I've cleaned it up already. Maybe with some focus you can avoid conflicting, but in my case I could not.

Also, I'm pretty sure the angle bracket "trick" is not a thing. Every time I have ever forked Fathom, I've had to go and update those to be quotes, because Fathom is not a system header.

As an example of the sort of things I removed, here is some original code from Fathom:

Code: Select all

static inline int type_of_piece_moved(Pos *pos, TbMove move) {
  for (int i = PAWN; i <= KING; i++) {
    if ((pieces_by_type(pos,(Color)(pos->turn == WHITE),(PieceType)i) & board(move_from(move))) != 0) {
      return i;
    }
  }
  assert(0);
  return 0;
}

Here are the ONLY two times this function is ever called:

Code: Select all

if (type_of_piece_moved(pos,move) != PAWN || is_capture(pos, move))
    continue;

and

Code: Select all

    if (is_capture(pos, move) || type_of_piece_moved(pos, move) == PAWN)
    continue;

So we delete that function, with loops and branching, and are able to replace it with:

Code: Select all

bool pyrrhic_is_pawn_move(const PyrrhicPosition *pos, PyrrhicMove move) {
    uint64_t us = pos->turn ? pos->white : pos->black;
    return pyrrhic_test_bit(us & pos->pawns, pyrrhic_move_from(move));
}

That is one of many examples.

[jdart]

Also, I'm pretty sure the angle bracket "trick" is not a thing. Every time I have ever forked Fathom, I've had to go and update those to be quotes, because Fathom is not a system header.

No, it does work. The Arasan Github sources include Fathom as a submodule, so it is imported "as is." The Makefile sets -I . (line 45) so the main source directory is ahead of the Fathom directory in the include path.

--Jon

[jdart]

Also, I'm pretty sure the angle bracket "trick" is not a thing. Every time I have ever forked Fathom, I've had to go and update those to be quotes, because Fathom is not a system header.

No, it does work. The Arasan Github sources include Fathom as a submodule, so it is imported "as is." The Makefile sets -I . (line 45) so the main source directory is ahead of the Fathom directory in the include path.

To be clear I have no objection in principle to anything you've done in your fork, but I don't think restructuring Fathom was necessary.

--Jon

[lucasart]

TL;DR: Cleaned up the Fathom code base and put it up on Github under the name Pyrrhic. Code is faster, easier to look at, and far less likely to have any conflicts with your code names. It also has all the "safety nets" removed.

https://github.com/AndyGrant/Pyrrhic/

So I was looking at compiling Ethereal under C++, as a potential segway to NNUE as it seems I will inevitably be forced to go that route if I want to stay in my current position. So it turns out that the version of Fathom I was using (some time in 2018) was real rough for compiling in C++, so I went to JDart's Fathom and pulled the latest in hopes of grabbing 7man TB support as well.

That was also as mess. As great as Fathom is, it takes up some names in the namespace that it really has no right to. A few examples: WHITE_KING , ... BLACK_PAWN, BEST_NONE, SCORE_ILLEGAL, BOARD_RANK_1, BOARD_FILE_A, BLACK, WHITE, W_PAWN, ... B_KING, and much more. Fathom, or Syzygy, has no right to claim those things. I expected to be able to simply update my Fathom files and have things work -- wrong. There were all sorts of namespace collisions that went undetected, leading to crashes.

So I took Fathom and basically renamed everything. If it appears in tbchess.c, Fathom's "mini" chess implementation to support probing, then it starts with PYRRHIC_ or pyrrhic_. I tried not to touch the actual Syzygy probing code too much, but I also renamed plenty of things there, like #define max, #define min, #define Swap, #define PAWN, #define PIECE.

I also deleted a lot of safety nets that are in Fathom that I see no purpose for. Fathom would check to see if the board was legal -- that's your job not Fathom's. Fathom would check if castling rights were null and that the previous move was zeroing when probing WDL -- that's your job not Fathom's. I also removed some of the DTM probing code, as I currently don't make use if it and therefore cannot test it, and therefore would not feel comfortable releasing something with it in.

Over the course of the cleanup I probably played around 200,000 games at various stages to confirm that nothing was functionally different. The final test, based on the initial commit of the Pyrrhic repo, returned the following: Score of Pyrrhic vs Fathom: 6554 - 6427 - 21019 [0.502] 34000. So at the end of the day, Pyrrhic performs the same as Fathom, with slightly more speed (this has been consistent in the other 166,000 games), and no introduced crashes or segfaults that I could detect.

As of now, I would expect a user to modify about a dozen lines in tbconfig.h, and nothing more. Terje, the author of Weiss, is going to try it as well shortly I believe. You now MUST provide Fathom with your move generation code. No more free lunches for people who cannot encapsulate their attack generation.

I will be replacing Ethereal's Fathom with Pyrrhic today.

Nice. Any attempt at making this complicated codebase understandable and easier to use is welcome.

But Ronald's code was GPL (with Ronald as copyright owner). Pyrrhic is now MIT, and Ronald has disappeared from the list of copyright holders, who are now only basil, Jon Dart and yourself. I understand that there has been some work on top of Ronald's code, but this is merely refactoring, and compatibility fixes. The order of magnitude of work involved does not compare to Ronald's original work IMO. Shouldn't Ronald have a say in this license change ?

Edit: Sorry, I didn't realize, but this license change, and removal of Ronald from copyright holders came from basil, not Pyrrhic
https://github.com/basil00/Fathom/blob/master/LICENSE

[jdart]

The probing code (Ronalds's) never was GPL. It was released for unrestricted use. The chess-related code that supports it was from Basil. I think this was MIT licensed in the beginning.

The latest 7-man TB probing code was part of Cfish, which is GPL, but tbprobe.c from that codebase still says it is for "unrestricted use." See https://github.com/syzygy1/Cfish/blob/m ... /tbprobe.c.

The current Fathom has removed all the CFish dependencies out of tbprobe.c and re-instated Basil's chess functions, with necessary modifications. When I released it I put the whole thing under the MIT license. IIRC one reason this was done is that I moved code around and now there was code from two license sources together so I put it under the more restrictive (but still liberal) one.

--Jon

[AndrewGrant]

Nice. Any attempt at making this complicated codebase understandable and easier to use is welcome.

The main effort was cleaning up tbchess.c, the "mini" chess implementation. I am still afraid to touch Syzygy too much though.

Edit: Sorry, I didn't realize, but this license change, and removal of Ronald from copyright holders came from basil, not Pyrrhic
https://github.com/basil00/Fathom/blob/master/LICENSE

I've added Ronald back to the header of the tbprobe.c and tbprobe.h files.
Although some of the code in tbchess.c is likely in his credit as well.

[elcabesa]

Sorry for being out of topic,
Some time ago I totally refactor the syzygy code using Stockfish code as reference. in the end I have a codebase with unit-test and very much faster that Fathom.

The main speed problem of fathom is that it does not use the chess engine position representation, but it require the pass to it and it transform then the position to his internal format. This is due to the fact that fathom is more or less standalone.

Unfortunately this code is heavily integrated in Vajolet codebase and it cannot be used as an external library

if you like the code it's inside the Vajolet repository:
https://github.com/elcabesa/vajolet/tre ... src/syzygy
tests:
https://github.com/elcabesa/vajolet/tre ... yzygyTests

regarding copyright, some files retain the full list of authors, some not... I have to fix it

Code: Select all

	Copyright (c) 2013 Ronald de Man
	Copyright (c) 2015 basil00
	Copyright (C) 2016-2019 Marco Costalba, Lucas Braesch
	Modifications Copyright (c) 2016-2019 by Jon Dart
	Modifications Copyright (c) 2019 by Marco Belli

[Terje]

Replaced Fathom with Pyrrhic in Weiss. More readable, less cluttered code, not using var names like WHITE/BLACK - overall a nice upgrade.

Hope the copyright/crediting stuff gets completely fixed (if it isn't already now)

Btw @jdart, there's a bug in fathom, if you init once with a proper path it sets TB_LARGEST, then you init again with empty string to turn them off TB_LARGEST isn't set to 0. The line that sets it to 0 should be higher up in the init function. Allie author gonzo made me aware of it so credit to him

[xr_a_y]

Also, I'm pretty sure the angle bracket "trick" is not a thing. Every time I have ever forked Fathom, I've had to go and update those to be quotes, because Fathom is not a system header.

No, it does work. The Arasan Github sources include Fathom as a submodule, so it is imported "as is." The Makefile sets -I . (line 45) so the main source directory is ahead of the Fathom directory in the include path.

To be clear I have no objection in principle to anything you've done in your fork, but I don't think restructuring Fathom was necessary.

--Jon

Works like a charm in Minic. My Makefile is downloading Fathom master from github and compile as it.

[jdart]

Hope the copyright/crediting stuff gets completely fixed (if it isn't already now)

Btw @jdart, there's a bug in fathom, if you init once with a proper path it sets TB_LARGEST, then you init again with empty string to turn them off TB_LARGEST isn't set to 0. The line that sets it to 0 should be higher up in the init function. Allie author gonzo made me aware of it so credit to him

I have updated Fathom to fix this bug, restore the copyright from Ronald de Man in tbprobe.c, and add some more documentation about overriding tbconfig.h (https://github.com/jdart1/Fathom).

--Jon