Progress on Loki

[amanjpro]

UCI rework update: I have finished re-writing the UCI implementation to C++, but the CheckUp function, called in search, is still from Vice. I don't really know how to implement that optimally. I will try using a thread to listen for input from the GUI, but I fear it will be slow..

All in all, I would say the development of Loki is going great

Good I have a function that checks the board and game state, but it only runs in debug mode. It degrades engine performance by 75%.

My suggestion would (still) be to not try and implement more than one function at the time. Implement LMR, and try to get the most ELO-gain possible. Then implement LMP, and do the same there. Then implement staged move generation. In the end, when you know all three functions are working perfectly, you can fiddle with the values of LMR and LMP to try and increase playing strength even more.

Problem, the R value that is good for LMR alone, might be terrible when you add LMP on top of it. That can easily explode.

What do you mean by that? If a good LMR reduction value is chosen, can't LMP just be adjusted to match that if they're implemented one after another?

Sure you can do that. But that doesn't mean it gives the optimal values, probably you will be better of modifying both LMP and LMR for the optimal results. I understand this is a rather hard task, especially if you have many features in your engine. That is why I'm looking into an automation for tweaking search paramus as well as evaluation params

[niel5946]

UCI rework update: I have finished re-writing the UCI implementation to C++, but the CheckUp function, called in search, is still from Vice. I don't really know how to implement that optimally. I will try using a thread to listen for input from the GUI, but I fear it will be slow..

All in all, I would say the development of Loki is going great

Good I have a function that checks the board and game state, but it only runs in debug mode. It degrades engine performance by 75%.

My suggestion would (still) be to not try and implement more than one function at the time. Implement LMR, and try to get the most ELO-gain possible. Then implement LMP, and do the same there. Then implement staged move generation. In the end, when you know all three functions are working perfectly, you can fiddle with the values of LMR and LMP to try and increase playing strength even more.

Problem, the R value that is good for LMR alone, might be terrible when you add LMP on top of it. That can easily explode.

What do you mean by that? If a good LMR reduction value is chosen, can't LMP just be adjusted to match that if they're implemented one after another?

Sure you can do that. But that doesn't mean it gives the optimal values, probably you will be better of modifying both LMP and LMR for the optimal results. I understand this is a rather hard task, especially if you have many features in your engine. That is why I'm looking into an automation for tweaking search paramus as well as evaluation params

That is right. What I was thinking was to first off optimize LMR as much as possible, and only then try to optimize LMP to fit the LMR implementation. After this you can run an automatic tuner to get the optimal values for both features in conjunction with each other.

Regarding the evalation parameter optimization, you can have a look at my two files, texel.h and texel.cpp. This uses a texel tuner with an SPSA gradient descent algorithm. It doesn't tune search ATM since I am still looking into the optimal way of finding the objective function using self-play. I hope it can be of help

[niel5946]

[...] which Bluefever got from another place himself.

Edit: I just noticed that the link no longer works. Must be because the Vice development videos are rather old now. Anyways, I can't find the original link...

[Guenther]

[...] which Bluefever got from another place himself.

Edit: I just noticed that the link no longer works. Must be because the Vice development videos are rather old now. Anyways, I can't find the original link...

Hi Niels,

You can always try the biggest resource for archiving older vanished sites - the wayback machine.
https://web.archive.org/
(just drop the old not working original link in the start window and it shows you if and how often and when it was saved)
https://web.archive.org/web/20160608190 ... ike/chess/

OTH Olithink has since long a new homepage, but I haven't looked, if the same resource is still there
http://brausch.org/home/chess/

[niel5946]

[...] which Bluefever got from another place himself.

Edit: I just noticed that the link no longer works. Must be because the Vice development videos are rather old now. Anyways, I can't find the original link...

Hi Niels,

You can always try the biggest resource for archiving older vanished sites - the wayback machine.
https://web.archive.org/
(just drop the old not working original link in the start window and it shows you if and how often and when it was saved)
https://web.archive.org/web/20160608190 ... ike/chess/

OTH Olithink has since long a new homepage, but I haven't looked, if the same resource is still there
http://brausch.org/home/chess/

Thank you for the help

I haven't been able to find a link to the original post though.. I will just refer to Vice which is where I got it from.

[amanjpro]

Hello Niels,

I am working on Texel tuning for Zahak, currently playing a match between two versions of the engine with very short time control (inf/1+0.08), which means almost all the moves are 5-6 plies deep only. Which makes me worried that their eval might not reflect the actual eval of the position (given the shallow search).

What did you do for the tuning positions? did you use someone else's games, or your engine's with longer time controls?

[niel5946]

Hello Niels,

I am working on Texel tuning for Zahak, currently playing a match between two versions of the engine with very short time control (inf/1+0.08), which means almost all the moves are 5-6 plies deep only. Which makes me worried that their eval might not reflect the actual eval of the position (given the shallow search).

What did you do for the tuning positions? did you use someone else's games, or your engine's with longer time controls?

I used another set of FEN's with game results appended. That seemed like the best idea for two reasons:
1) The positions and their results would probably be more reliable considering that my eval hadn't been tuned beforehand and had rather inaccurate values.
2) It saved time while getting the tuner to work. Before beginning to create my own training sets, I was more focused on just getting a working gradient descent.
Both of these however, aren't needed any more. A goal for version 4.0 is that I create my own framework for generating tuning positions. These positions will be generated with very short time controls too (something like the 1+0.08 you're referring to).

Regarding your question in particular, I am not sure what you are using the self-play games for. Are you using it to generate test positions or to tune your eval?

• In the first case, I would go for someone else's tuning positions until I was sure the tuner was working correctly. The depth isn't really important here if you're using the same version of the engine. This is because each side would be equally strong and thus equally impaired by the low depths. As long as you make sure to filter out positions with very large scores (like > 600cp or < -600cp) and resolve the captures, I think it'll be fine. This is because large scores are usually caused by tactics while captures lead to many tactics.
• In the second case, you don't need that. You just need to fit the evaluation function to a kind of winning probability function and then take the squared errors of that compared to each game result. Then you can do gradient descent with that.

[niel5946]

The revised list of improvements before Loki 4.0:

1. Reworked UCI-implementation. This is hopefully completed soon, and I am running some stability tests now.
2. Join OpenBench. As I have been informed, OpenBench is imperative for effective development. For this point, I need to do two things:
   1. Finish the UCI rework. This will probably be accompanied with a version 3.5 release since line 32 in uci.cpp is unstable on Linux.
   2. Implement a "bench" command. This should be easy enough and probably be finished in a day.
3. Implement LMR. I am working on this right now, with promising results (92 elo). I would like the elo gain from LMR alone to be >100, so it is not fully finished yet.
4. Implement LMP. As stated before, I have been fiddling with this a couple of times, but it hasn't really worked. This will be done after LMR, and in the same branch.
5. Make a working piece-activity evaluation. I already have a term for this, but it doesn't gain anything yet, so it is disabled ATM.
6. New eval term: I would like to call it "coordination". It should be responsible for scoring - as the name suggests - piece coordination. This will reward pieces that target (x-ray and direct attacks) the same part of the board or the same square.
7. Staged move generation. I have already worked a lot on this feature, but it hasn't worked yet. I hope to make it work some day soon though
8. Test all search features that are disabled at the moment. I will test things like aspiration windows, delta pruning, IID, etc.. Hopefully I can make them work and release version 4.0 with them enabled.
9. Tuning-set generator. This should generate tuning positions for the SPSA tuner. It will be the last thing, I'll work on before version 4.0.

Ragarding point number 6, positions like this:
[d]r2q1rk1/ppp1bppp/2n1pn2/3p4/2PP4/1P1BPP2/PB3P1P/RN1QK1R1 b Q - 4 9
A "coordination" term will score this positively. When the position opens up, the bishops on b2 and d3, and the rook on g1, will all be placed excellently. This way, "coordination" will be a good predictor of future threats and attacks.
I haven't quite thought of an implementation yet, and I think it'll be relatively complicated. Hopefully it'll pay off

This seems like a good list of things to do before version 4.0!

[niel5946]

Update on earlier:
I have now completed two points from the above post. Namely:

1. Reworked UCI-implementation. The UCI-implementation has been merged into master and seems to work fine. It loses in very short time controls (i.e. 1+0.1s), but seems to be good and stable otherwise.
2. Join OpenBench. Thanks to Jay Honnold, who has generously let me join his OpenBench server, I am now doing proper testing.

On top of this, I am now using this OpenBench testing to test my LMR implementation properly. After these tests, I will try to make some improvements and then move on to LMP.

All in all, good progress in one day

[mvanthoor]

Update on earlier:
I have now completed two points from the above post. Namely:

1. Reworked UCI-implementation. The UCI-implementation has been merged into master and seems to work fine. It loses in very short time controls (i.e. 1+0.1s), but seems to be good and stable otherwise.
2. Join OpenBench. Thanks to Jay Honnold, who has generously let me join his OpenBench server, I am now doing proper testing.

On top of this, I am now using this OpenBench testing to test my LMR implementation properly. After these tests, I will try to make some improvements and then move on to LMP.

All in all, good progress in one day

Certainly good progress

I've been re-testing my current two versions of the engine (didn't save the games in the previous tests). Everything seems fine, so somewhere in the coming two weeks I'll implement the next feature.

Couldn't you join the server on http://chess.grantnet.us/, or run your own instance?