Data steal

Discussion of chess software programming and technical issues.

Moderators: hgm, Harvey Williamson, bob

Post Reply
Henk
Posts: 5077
Joined: Mon May 27, 2013 8:31 am

Data steal

Post by Henk » Fri Jan 26, 2018 2:42 pm

When using MCTS and a neural network maybe it is best to store the training examples that are gathered during search. So if you later find out that the configuration of neural network is not that great and you chose to use another one then you can still use these training examples for tuning your new network.

smatovic
Posts: 479
Joined: Wed Mar 10, 2010 9:18 pm
Location: Germany
Contact:

Re: Data steal - games

Post by smatovic » Thu Feb 01, 2018 11:38 am

I suggest to use games for supervised learning first,
to make sure the NN behaves like expected,
then start the Zero approach with selfplay.

There are some games on CCRL,

http://www.computerchess.org.uk/ccrl/404/games.html

With pgn extract

https://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/

and the option -Wuci
you can extract pgn games to moves in can notation and final result,
so there is no need to implement an own pgn parser.

--
Srdja

AlvaroBegue
Posts: 880
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York

Re: Data steal - games

Post by AlvaroBegue » Thu Feb 01, 2018 4:07 pm

smatovic wrote:I suggest to use games for supervised learning first,
to make sure the NN behaves like expected,
then start the Zero approach with selfplay.

There are some games on CCRL,

http://www.computerchess.org.uk/ccrl/404/games.html

With pgn extract

https://www.cs.kent.ac.uk/people/staff/djb/pgn-extract/

and the option -Wuci
you can extract pgn games to moves in can notation and final result,
so there is no need to implement an own pgn parser.

--
Srdja
I am generating a database of games precisely for this purpose. I don't want to deal with differences in strength of the participants, so I am using Stockfish-vs-Stockfish at 1 second + 0.1 seconds/move. So far I have about 2.5 million games, and I intend to use 3 million.

If anyone is interested, I'll upload them somewhere so we don't all have to spend a couple of months generating games.

smatovic
Posts: 479
Joined: Wed Mar 10, 2010 9:18 pm
Location: Germany
Contact:

Re: Data steal - games

Post by smatovic » Thu Feb 01, 2018 5:21 pm

If anyone is interested, I'll upload them somewhere so we don't all have to spend a couple of months generating games.
I would appreciate it, thanks.

--
Srdja

AlvaroBegue
Posts: 880
Joined: Tue Mar 09, 2010 2:46 pm
Location: New York

Re: Data steal - games

Post by AlvaroBegue » Thu Feb 01, 2018 6:20 pm

smatovic wrote:
If anyone is interested, I'll upload them somewhere so we don't all have to spend a couple of months generating games.
I would appreciate it, thanks.

--
Srdja
I was planning on publishing 3 million games in about two weeks. But I can publish 2 million right away if you are interested.

smatovic
Posts: 479
Joined: Wed Mar 10, 2010 9:18 pm
Location: Germany
Contact:

Re: Data steal - games

Post by smatovic » Thu Feb 01, 2018 6:48 pm

I was planning on publishing 3 million games in about two weeks. But I can publish 2 million right away if you are interested.
I am in no hurry,
two weeks are fine,
still assembling my gpu machine for training
and doing some research on NN.

--
Srdja

Post Reply