Building NN-powered engine

[maelic]

@jpqy thank you for the sources!

@brianr I am aware of the hardware requirements but I only have 1050 Ti at my disposal. If things go well though I will get my hands on some Titan our institute has - I will have to fight for getting it once I am reasonably sure the project works and it will not be just wasted time.
I will take a look at the Zeta repos, any sources are very welcome.

Right now I only have Hiarcs Chess Explorer and it does not allow me to use .py script as engine - do other GUIs let you do it? I will have to check with my old Fritz GUI later but I don't have it installed right now. It would be very nice not having to compile cause any transformation can cause more bugs as I have learned in the past..

Superwised training is a very good idea - especially considering the time requirements it would take to start from zero. I am not exactly sure how I will manage the learning process though - I am still concentrating on building basic structure. My referend told me he would ask for the python-chess to be allowed but I should not hope too much - it is Master's thesis after all and it has to be difficult. My idea would be to start learning process from a rather large database of chosen high quality games (that's where my ICCF experience and database access can be useful) and then set the engine to continue learning via self-play onwards. That should hopefully get me some good rating to start with and even with not enough calculation power to really inmprove via self-play all that much I will be able to reach the strength criteria.

I will start looking into linux only if I encounter a problem not workable under Windows - I do plan to get to learning how to properly use linux but I lack the motivation for that right now..

[brianr]

Does Hiarcs GUI specifically exlude "python.exe" as an engine command file? If not, it should work.
Otherwise, Winboard/Xboard and Arena work.

Perhaps I am not clear on the specific goals of your thesis. Is the focus on writing a chess engine or the NN learning? Either one is a rather large topic space on it's own. There a many papers on chess engines and quite a few on chess with NN learning. Some focus only on the search, some on the evaluation, opening lines (books), endgame tablebases, some on the NN arch, and some non-chess specific ones on just the hyper-parameters, learning rate change strategies, etc. Whatever topic space you pick, why not just use the common existing tools for as much of the things outside that space as possible.

PS There is clearly a lot of work that could be done. Avoiding the unnecessary "busy work" compiling something from Python seems like a good idea.

[maelic]

The point of the thesis should be using neural networks for playing chess - but whether or not I have to write myself all the other things is up to our head of institute - a very hyperactive guy, expert in the field of information technology and automation, author of some very advanced genetic algorithms.. I am afraid in his eyes using any of the already done work would not be quite enough.. I will discuss this further with him though.

While working with chess engines for a long time, I have not ever tried to use .py script as one - how do you connect it? I am afraid I do not fully understand. Most of the GUIs let you connect an UCI protocol based engine (which is what I would like to build) but those tend to be .exe files. For example HCE lets you only select as engines files .exe or .bat. Do you mean I should select an python itself as executable? I would need to get different parameters than simple command 'uci' to start the engine in the first place I reckon - and I do not see any way how to give this command under HCE. Maybe under Arena it is different, I will have to check it out.
Another option might be to build some kind of adapter - an exe file which would load .py script and communicate with both GUI and the script - a man in the middle.

[elcabesa]

I am afraid in his eyes using any of the already done work would not be quite enough..

maybe you can try to argument that without reinventing the weel ( write from scratch a chess engine ) you can concentrate on the topic of the thesis which is develop and evaluate NN.
In the end a good modern programmer should not reimplement the wheel every time, this is my point of view.

[hgm]

Another option might be to build some kind of adapter - an exe file which would load .py script and communicate with both GUI and the script - a man in the middle.

That is exactly what python.exe is, right? An .exe file that takes a .py script and executes it. It 'communicates' with the script by loading the file that is in, and then it communicates through stdin / stdout with the GUI, as per script specification.

So yes, you have to use python.exe as the engine. By starting it with the proper command-line arguments you can cause it to load the desired .py script at startup, and start executing it. A Python user should know what arguments to use on python.exe to achieve that.

[maelic]

@hgm

You're right, silly me. The only problem is passing the parameters on load then. I see some options in Arena but my idea of the adapter was mainly for the GUIs that do not have this option. Adapter could take care of it without the need for passing additional options from GUI's side.
I admit I am not adamant in python just yet but I am working on it. I still have a better portion of a year to make this happen. It will be super-hard work but I believe it to be possible.

Thank you all for your comments, I appreciate it very much!!

[saikrishnagv]

This is interesting: https://github.com/saikrishna-1996/deep_pepper_chess

[Henk]

A simple but we'll programmed engine will be able to defeat a 1800 elo player very easily.

processing power is not so important, unless you want to defeat very strong engines.

Good luck with you peoject

My first alpha beta engine had ELO of about 1300. My neural network engine has ELO 600?
So 1800 is a good initial goal.

If my computer has to learn 200000 training examples it will take at least 200 hours I guess. But I don't have GPU and my processor Is relatively slow.

Usually after three days of training I get other ideas and start with a new network. So I never make it to 200 hours. Also terribly boring if you have to wait two or three months.

First step in engine development is creating a move generator. En passant and promotion moves gave tough bugs in my code (disappearing pawns)

[Henk]

Maybe more fun than creating a chess engine is perhaps implementing a GAN (Generative adversarial network). With a GAN you can generate all kinds of fake images (examples) that are difficult to distinguish from real. But perhaps collecting training examples can be difficult/tedious.

[Uri Blass]

A simple but we'll programmed engine will be able to defeat a 1800 elo player very easily.

processing power is not so important, unless you want to defeat very strong engines.

Good luck with you peoject

My first alpha beta engine had ELO of about 1300. My neural network engine has ELO 600?
So 1800 is a good initial goal.

If my computer has to learn 200000 training examples it will take at least 200 hours I guess. But I don't have GPU and my processor Is relatively slow.

Usually after three days of training I get other ideas and start with a new network. So I never make it to 200 hours. Also terribly boring if you have to wait two or three months.

First step in engine development is creating a move generator. En passant and promotion moves gave tough bugs in my code (disappearing pawns)

I agree that a move generator without bugs is the first step.

If you have some pgn file in the folder that include the exe file of movei(let call it xxx.pgn) then you can run movei in console mode and print perftpgn xxx.pgn 3 and get the sum of perft 3 for all positions in the pgn.

If you have a move generator and want to test it for bugs then I suggest that you implement the same function for a pgn and check if you get the same result.
If you get the same result for a big pgn then it means that you probably have a move generator that has no bugs.

Edit:Movei is an old engine and I have no homepage for it but I searched and found that you can download movei in the following site
http://www.computer-chess.org/doku.php? ... nload_list