Data steal

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Henk
Posts: 7217
Joined: Mon May 27, 2013 10:31 am

Data steal

Post by Henk »

When using MCTS and a neural network maybe it is best to store the training examples that are gathered during search. So if you later find out that the configuration of neural network is not that great and you chose to use another one then you can still use these training examples for tuning your new network.
smatovic
Posts: 2645
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: Data steal - games

Post by smatovic »

I suggest to use games for supervised learning first,
to make sure the NN behaves like expected,
then start the Zero approach with selfplay.

There are some games on CCRL,

http://www.computerchess.org.uk/ccrl/404/games.html

With pgn extract

https://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/

and the option -Wuci
you can extract pgn games to moves in can notation and final result,
so there is no need to implement an own pgn parser.

--
Srdja
AlvaroBegue
Posts: 931
Joined: Tue Mar 09, 2010 3:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: Data steal - games

Post by AlvaroBegue »

smatovic wrote:I suggest to use games for supervised learning first,
to make sure the NN behaves like expected,
then start the Zero approach with selfplay.

There are some games on CCRL,

http://www.computerchess.org.uk/ccrl/404/games.html

With pgn extract

https://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/

and the option -Wuci
you can extract pgn games to moves in can notation and final result,
so there is no need to implement an own pgn parser.

--
Srdja
I am generating a database of games precisely for this purpose. I don't want to deal with differences in strength of the participants, so I am using Stockfish-vs-Stockfish at 1 second + 0.1 seconds/move. So far I have about 2.5 million games, and I intend to use 3 million.

If anyone is interested, I'll upload them somewhere so we don't all have to spend a couple of months generating games.
smatovic
Posts: 2645
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: Data steal - games

Post by smatovic »

If anyone is interested, I'll upload them somewhere so we don't all have to spend a couple of months generating games.
I would appreciate it, thanks.

--
Srdja
AlvaroBegue
Posts: 931
Joined: Tue Mar 09, 2010 3:46 pm
Location: New York
Full name: Álvaro Begué (RuyDos)

Re: Data steal - games

Post by AlvaroBegue »

smatovic wrote:
If anyone is interested, I'll upload them somewhere so we don't all have to spend a couple of months generating games.
I would appreciate it, thanks.

--
Srdja
I was planning on publishing 3 million games in about two weeks. But I can publish 2 million right away if you are interested.
smatovic
Posts: 2645
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: Data steal - games

Post by smatovic »

I was planning on publishing 3 million games in about two weeks. But I can publish 2 million right away if you are interested.
I am in no hurry,
two weeks are fine,
still assembling my gpu machine for training
and doing some research on NN.

--
Srdja