Progress on Loki

[niel5946]

Loki 4.0 is finally nearing it's release
As I have mentioned earlier in this thread, I have been contemplating whether or not to postpone the neural network evaluation function, and I have made my decision: It will, sadly, not be available until version 5.0.. This is primarily because of the massive amounts of time it takes to develop, which doesn't seem feasible since I want to release Loki 4.0 rather soon. The neural network library, I have written, will however be available in the source code of Loki 4.0, but it won't be used anywhere in the engine.

My final list of changes between Loki 3.5 and Loki 4.0 will be the following:

1. Add staged move generation (gains ~20-30 elo)
2. Make the transposition table thread-safe for 32-bit builds (has been done, but no elo change is expected for single-threaded use).
3. Fix a bug where Loki freezes after deep searches with multiple threads running (has also been fixed now).
4. Add late move reductions. These have been added, but I need to run a final test between this and master.
5. Add late move pruning.
6. Add an evaluation hash table. This has been added, but there are some memory-corruption bugs that need to be sorted out before merging.
7. Re-tune the evaluation function on a bigger dataset.
8. Merge the neural branch into master (what I described above).
9. Test - and potentially re-add - disabled search features. These include verified null move pruning, IID, Delta pruning and futility pruning in quiescence.
10. A little bit of code cleanup.

This, in comparison to the old list, seems more feasible to accomplish in the near future. My biggest problem with Loki 4.0 is the disappointment of not getting the neural network evaluation... But hey, at least it's ready for Loki 5
And speaking of Loki 5, this is my current ideas:

1. Incorporate the neural network evaluation and train a network.
2. Add MultiPV for analysis.
3. Better time management.
4. Test new pruning methods or other search features.

All in all, I am happy with the new goal for Loki 4.0, even though an NN would've been nice

[CMCanavessi]

Does Loki play FRC?

[niel5946]

Does Loki play FRC?

No, not at the moment, but now that you mention it, Chess960 would be a nice addition

[niel5946]

The great refactor
Hello everyone
I know it has been some time since I was active on this forum, but I thought that I would give you all a little update on how Loki is doing. I have been very preoccupied with my new job recently, and have therefore not worked much on chess programming, but now I feel a renewed motivation to give Loki a little makeover before making it stronger.
I have decided to do a complete refactoring of Loki due to the following reasons:

• I have learned a lot (C#, C++ and general programming knowledge) from my job, and I therefore think I am a better programmer now than before. Therefore, a refactoring of Loki would be a way of "proving" this to myself.
• Even though I would not call Loki a derivative of other engines, I feel that it's too close to be one. For one, the structure of Loki's code is not really my preferred way of writing/designing C++ applications - it has some times felt like I was coding C and not C++. Secondly, some algorithms has been implemented with great inspiration from other engines (magic bitboards is the most obvious example of this), and I would like to understand these better and write my own implementations.
• There are probably a lot of subtle bugs and bottlenecks, which I would like to remove. I just saw in another thread, that someone suspects Loki of playing illegal moves, which is of course a pretty significant bug, that I should fix.

This means that my goal of the refactoring is the following:

• Write more object-oriented code and remove all global variables that are not constexpr. Additionally, more safety should be added by strongly preferring smart pointers instead of raw pointers.
• Fix bugs that I find along the way.
• Make the code more manageable by writing it in my own style.

I haven't given the potential elo gain/loss of this refactoring any thought, but I would like to think that any losses will pay off in the long term by making Loki more bug-free and easier to work on.
Lastly, as I mentioned in the start, I spend a lot of time working, so this project might take some time. But I will not quit development of Loki in the near future!

[niel5946]

Magic bitboards
I have just finished re-implementing magic bitboards, and this time I have done it my way
I decided to use plain magic bitboards with the main reason being, that memory isn't really a problem on most computers nowadays. It is made by combining my own coding preferences and lots of different implementations I have looked at to get inspiration.
The code can be found here under movegen/magics.
I like that it is written with templates instead of the previous "dependency" the two tables had to each other, and the code reusage.
I have only really tested it for compilation and some very simple occupancy bitboards and squares, so there may be errors, but I'll get to that when I have a working perft function. I also haven't optimized it yet because of that reason.

[tcusr]

Magic bitboards
I have just finished re-implementing magic bitboards, and this time I have done it my way
I decided to use plain magic bitboards with the main reason being, that memory isn't really a problem on most computers nowadays. It is made by combining my own coding preferences and lots of different implementations I have looked at to get inspiration.
The code can be found here under movegen/magics.
I like that it is written with templates instead of the previous "dependency" the two tables had to each other, and the code reusage.
I have only really tested it for compilation and some very simple occupancy bitboards and squares, so there may be errors, but I'll get to that when I have a working perft function. I also haven't optimized it yet because of that reason.

just a suggestion, in C++20 you can use the <bit> header instead of compiler intrinsics for bit manipulation functions. thanks to it i have zero lines of OS/compiler specific code in my engine.

also i think there's an error here, to check if a bit is set there are two ways

Code: Select all

x & (1 << shift)
(x >> shift) & 1

you mixed them up

[niel5946]

Magic bitboards
I have just finished re-implementing magic bitboards, and this time I have done it my way
I decided to use plain magic bitboards with the main reason being, that memory isn't really a problem on most computers nowadays. It is made by combining my own coding preferences and lots of different implementations I have looked at to get inspiration.
The code can be found here under movegen/magics.
I like that it is written with templates instead of the previous "dependency" the two tables had to each other, and the code reusage.
I have only really tested it for compilation and some very simple occupancy bitboards and squares, so there may be errors, but I'll get to that when I have a working perft function. I also haven't optimized it yet because of that reason.

just a suggestion, in C++20 you can use the <bit> header instead of compiler intrinsics for bit manipulation functions. thanks to it i have zero lines of OS/compiler specific code in my engine.

also i think there's an error here, to check if a bit is set there are two ways

Code: Select all

x & (1 << shift)
(x >> shift) & 1

you mixed them up

Thanks for your suggestion! I'll look into the bit header after writing this.

And you're right, I mixed them up. I just noticed it was like that in the entire castle_rights class! It'll be fixed by my next commit

[j.t.]

Personally, I prefer putting stuff like that in a function. E.g.:

Code: Select all

#include <cstdint>

using Bitboard = uint64_t;

enum Square {
    a1, b1, c1, d1, e1, f1, g1, h1,
    a2, b2, c2, d2, e2, f2, g2, h2,
    a3, b3, c3, d3, e3, f3, g3, h3,
    a4, b4, c4, d4, e4, f4, g4, h4,
    a5, b5, c5, d5, e5, f5, g5, h5,
    a6, b6, c6, d6, e6, f6, g6, h6,
    a7, b7, c7, d7, e7, f7, g7, h7,
    a8, b8, c8, d8, e8, f8, g8, h8,
    noSquare
};

inline Bitboard toBitboard(const Square square){
	return 1 << square;
}

inline bool isSet(const Bitboard bb, const Square square){
	return bb && toBitboard(square);
}

I find that this makes code much easier to read and understand.

[niel5946]

Personally, I prefer putting stuff like that in a function. E.g.:

Code: Select all

#include <cstdint>

using Bitboard = uint64_t;

enum Square {
    a1, b1, c1, d1, e1, f1, g1, h1,
    a2, b2, c2, d2, e2, f2, g2, h2,
    a3, b3, c3, d3, e3, f3, g3, h3,
    a4, b4, c4, d4, e4, f4, g4, h4,
    a5, b5, c5, d5, e5, f5, g5, h5,
    a6, b6, c6, d6, e6, f6, g6, h6,
    a7, b7, c7, d7, e7, f7, g7, h7,
    a8, b8, c8, d8, e8, f8, g8, h8,
    noSquare
};

inline Bitboard toBitboard(const Square square){
	return 1 << square;
}

inline bool isSet(const Bitboard bb, const Square square){
	return bb && toBitboard(square);
}

I find that this makes code much easier to read and understand.

Additionally, using such functions would reduce the risk for mistakes like the one i made above

[niel5946]

Little update
I have just finished writing my FEN parser/generator, and I must say: Generally I am, as of the time of writing this, very excited about this refactor! I beleive that the code is much better, modular (OOP) and modern. I still don't know about any speed differences from the old code, but that will come when I have written a perft function.
Currently, I am writing my make_move and undo_move functions, which is the last code I need to write before I can perft-test the new Loki.