Before things become more messy than they already are.

Discussion of anything and everything relating to chess playing software and machines.

Moderator: Ras

AndrewGrant
Posts: 1976
Joined: Tue Apr 19, 2016 6:08 am
Location: U.S.A
Full name: Andrew Grant

Re: Before things become more messy than they already are.

Post by AndrewGrant »

xr_a_y wrote: Sat Jul 03, 2021 9:48 am
connor_mcmonigle wrote: Sat Jul 03, 2021 4:47 am I think people really overestimate the difficulty associated with achieving option 3. Generating ~100 million labeled positions from fixed low depth games needed to basically saturate a 768->512->1 network and even train a reasonable halfkp net takes maybe two days on modern hardware. Worst case, if your hardware is ancient, game generation is going to take a week of your time.
On my 8 cores i7 and GTX 1060, here are the figures :
- data generation for 1B "good enough" positions (d5 selfplay, qsearch leaf of pv move, d8 to d12 search based on game phase) : 3 weeks
- training (around 300 epoch of 100M fens, current Minic net topology) : a week
- testing (all nets saved at each epoch is tested versus previous master net) : a week (this generates this kind of picture : https://github.com/tryingsomestuff/NNUE ... losity.png)

So basically, 1 net == 1 month for Minic

Fortunately I'm able to rent some more hardware on the cloud that is generating data from last version of Minic while my own computer is training and testing.
If you are going to use d8->d12 searches on positions, you might as well just play the games using d8, and then you already have that information ready to collect. Post processing a list of fens to get evals is your biggest time sink. I'd also suggest thinking about very minor things: Disabling most UCI output, spawning multiple cutechess copies since its input reading is single threaded. Perhaps use linux and abandon cutechess, and use Lucas' c-chess-cli.
User avatar
xr_a_y
Posts: 1872
Joined: Sat Nov 25, 2017 2:28 pm
Location: France

Re: Before things become more messy than they already are.

Post by xr_a_y »

AndrewGrant wrote: Sat Jul 03, 2021 9:56 am
xr_a_y wrote: Sat Jul 03, 2021 9:48 am
connor_mcmonigle wrote: Sat Jul 03, 2021 4:47 am I think people really overestimate the difficulty associated with achieving option 3. Generating ~100 million labeled positions from fixed low depth games needed to basically saturate a 768->512->1 network and even train a reasonable halfkp net takes maybe two days on modern hardware. Worst case, if your hardware is ancient, game generation is going to take a week of your time.
On my 8 cores i7 and GTX 1060, here are the figures :
- data generation for 1B "good enough" positions (d5 selfplay, qsearch leaf of pv move, d8 to d12 search based on game phase) : 3 weeks
- training (around 300 epoch of 100M fens, current Minic net topology) : a week
- testing (all nets saved at each epoch is tested versus previous master net) : a week (this generates this kind of picture : https://github.com/tryingsomestuff/NNUE ... losity.png)

So basically, 1 net == 1 month for Minic

Fortunately I'm able to rent some more hardware on the cloud that is generating data from last version of Minic while my own computer is training and testing.
If you are going to use d8->d12 searches on positions, you might as well just play the games using d8, and then you already have that information ready to collect. Post processing a list of fens to get evals is your biggest time sink. I'd also suggest thinking about very minor things: Disabling most UCI output, spawning multiple cutechess copies since its input reading is single threaded. Perhaps use linux and abandon cutechess, and use Lucas' c-chess-cli.
Thanks for the input Andrew.
I indeed used to play game, collect fen in pgn and eventually re-analyse them. But this is not the case anymore for a while.
In fact, there is no cutechess or c-chess-cli here, I'm using an inner selfplay function of Minic now. Using a game manager was indeed too slow.

Playing all the game at d8 will involve having searched d8 for a lot of capture that will later be discarded or pushed to qsearch and then analyzed again. So I play the game d5, then look for a quiet position from the current one, then analyze this one at d8-d12. It probably can be better clearly...
smatovic
Posts: 3600
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: Before things become more messy than they already are.

Post by smatovic »

hgm wrote: Fri Jul 02, 2021 9:24 pm Whatever anyone's preference, it should be clear that programmers are allowed to do whatever they want, as long as they don't violate copyright licences. And that as long as they honestly specify what they have been doing, there is never anything unethical about that.
hgm wrote: Fri Jul 02, 2021 11:14 pm
Rebel wrote: Fri Jul 02, 2021 9:53 pmWhat you say is self understood but the context of this thread is about fair competition.
Well, deciding what is 'fair competition' is pretty much the prerogative of those organizing that competition. And diversity is usually a good thing. So my preference is that everything would be 'allowed', and that everyone interested in our fantastic hobby can have fun with it, without being pestered by others because they are not doing exactly the same thing as they do. Be it people that are only interested in training networks, writing an engine with HCE from scratch, or marginally improving some existing engine.

There seems to be room for all.
I go with HGM here. This poll is posted on TalkChess, not on a discussion list of rating-list or tournament organizers. Thus I view all this as neural network Nazi stuff ;) who owns this, who owns that. Like HGM said, just stick to the licenses and everything is fine. Like others said, where the breaking line of a clone and original work is, was/is/ and always will be a matter of discussion. I vote for computer chess being inclusive not exclusive, or alike.

***edit***
I read the OP in a generalized way, I see no indication that is was in strict context of rating-lists and tournaments only.

--
Srdja