Best Nets Made from Human Games Data

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

BrendanJNorman
Posts: 2564
Joined: Mon Feb 08, 2016 12:43 am
Full name: Brendan J Norman

Best Nets Made from Human Games Data

Post by BrendanJNorman »

Hi Guys,

Just wondering what the best nets available are which were trained on human games (liChess, Kingbase, whatever).

These tend to have an interesting style and I'd love to check a few out.

Thanks for any info in this regard,

Brendan
Dann Corbit
Posts: 12615
Joined: Wed Mar 08, 2006 8:57 pm
Location: Redmond, WA USA

Re: Best Nets Made from Human Games Data

Post by Dann Corbit »

I guess that there are very few (if any) nets made exclusively from human games. I guess you could build the nets based on the win/loss/draw alone.
You would need a lot of statistics to create such a net.

The Lichess games would be a good candidate for that, because the volume is so high.
I use a formula like this to estimate centipawns, based on wins/losses/draws:
UPDATE Epd SET coef = (white_wins - black_wins)* 1.0/(1e-20+white_wins+black_wins+draws)
UPDATE Epd SET coef = -coef WHERE Epd like '% b %'
and then:
cce = round(coef * 444.0,0)
It's only a good fit for a score of +/- one full piece. At 444 or -444 it is all wins or losses.

I would not want to use the data if there are less than 32 decisive games for the position.
Also, I would use only games where both players were at or above 2500.
Because there are so many Lichess games, that will still leave you more than ten million games from which to draw analysis.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
smatovic
Posts: 2797
Joined: Wed Mar 10, 2010 10:18 pm
Location: Hamburg, Germany
Full name: Srdja Matovic

Re: Best Nets Made from Human Games Data

Post by smatovic »

dkappe
Posts: 1632
Joined: Tue Aug 21, 2018 7:52 pm
Full name: Dietrich Kappe

Re: Best Nets Made from Human Games Data

Post by dkappe »

Dann Corbit wrote: Thu Jun 08, 2023 4:44 am I guess that there are very few (if any) nets made exclusively from human games. I guess you could build the nets based on the win/loss/draw alone.
You would need a lot of statistics to create such a net.

The Lichess games would be a good candidate for that, because the volume is so high.
I use a formula like this to estimate centipawns, based on wins/losses/draws:
UPDATE Epd SET coef = (white_wins - black_wins)* 1.0/(1e-20+white_wins+black_wins+draws)
UPDATE Epd SET coef = -coef WHERE Epd like '% b %'
and then:
cce = round(coef * 444.0,0)
It's only a good fit for a score of +/- one full piece. At 444 or -444 it is all wins or losses.

I would not want to use the data if there are less than 32 decisive games for the position.
Also, I would use only games where both players were at or above 2500.
Because there are so many Lichess games, that will still leave you more than ten million games from which to draw analysis.
From Kingbase with material qsearch mixed in. Uses the original SF nnue format. Could use some better, deeper material search.

https://www.patreon.com/posts/harmon-nnue-44549655
Fat Titz by Stockfish, the engine with the bodaciously big net. Remember: size matters. If you want to learn more about this engine just google for "Fat Titz".
jdart
Posts: 4368
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Best Nets Made from Human Games Data

Post by jdart »

There are two issues with human games for training: first, humans blunder a lot. Even strong players. Secondly, you won't get enough games that way. You need billions of positions to get a well-trained net.
chrisw
Posts: 4360
Joined: Tue Apr 03, 2012 4:28 pm

Re: Best Nets Made from Human Games Data

Post by chrisw »

jdart wrote: Sat Jun 10, 2023 4:35 pm There are two issues with human games for training: first, humans blunder a lot. Even strong players. Secondly, you won't get enough games that way. You need billions of positions to get a well-trained net.

Could be possible. A bit of crowd sourced help would be useful (supply of cores).
I would guess there are enough human-human games on LiChess of "reasonable" strength players.
Positions could be blunderchecked by SF or whatever, that's the time consuming part.
A 50 core machine can blunder check about 5000 games per second at d9 (which would be enough, probably) = 400M positions a day. 10B positions would take 25 days per 50 core machine. Any core volunteers? I'll organise it.
chesskobra
Posts: 215
Joined: Thu Jul 21, 2022 12:30 am
Full name: Chesskobra

Re: Best Nets Made from Human Games Data

Post by chesskobra »

Dann Corbit wrote: Thu Jun 08, 2023 4:44 am I guess that there are very few (if any) nets made exclusively from human games. I guess you could build the nets based on the win/loss/draw alone.
You would need a lot of statistics to create such a net.
I have a naive question. Suppose we have a certain amount of data, say 10M games. Can we make many copies of the games, shuffle them together, and use it for training?
jdart
Posts: 4368
Joined: Fri Mar 10, 2006 5:23 am
Location: http://www.arasanchess.org

Re: Best Nets Made from Human Games Data

Post by jdart »

I have a naive question. Suppose we have a certain amount of data, say 10M games. Can we make many copies of the games, shuffle them together, and use it for training?
Well, you can do that, but you are not going to get improved results that way. Training already visits the same positions over and over, because it is iterative and each iteration goes through the positions, or a subset of them. If you put multiple copies of a position set into the training set, that is pretty much the same process: it is like bunching some of those iterations together in one batch.

Also note there is no technical barrier to training on a small position set. You just won't get good results, for multiple reasons, one of them being that you might not visit positions with all the features that you are trying to train for.
chesskobra
Posts: 215
Joined: Thu Jul 21, 2022 12:30 am
Full name: Chesskobra

Re: Best Nets Made from Human Games Data

Post by chesskobra »

Let us say I play a game and analyse it, and I learn some things. In a few months, after I have improved a little, and I look at some old game, and I may learn a few more things. At least I have experienced such a thing in other domains. Is this not likely with neural networks?
BrendanJNorman
Posts: 2564
Joined: Mon Feb 08, 2016 12:43 am
Full name: Brendan J Norman

Re: Best Nets Made from Human Games Data

Post by BrendanJNorman »

jdart wrote: Sat Jun 10, 2023 4:35 pm There are two issues with human games for training: first, humans blunder a lot. Even strong players. Secondly, you won't get enough games that way. You need billions of positions to get a well-trained net.
Really depends on the goal, doesn't it?

For example, a lot of chess engines are described (and marketed!) as "human-like" in their playing style, despite playing at 3000+ Elo and essentially flawless from a cp loss perspective vs humans.

If you *really* want to make a human-like net, wouldn't it be cool/better for it to actually blunder once in a while?

This is literally what makes "human-like", human-like. "To err is human".

I'd love to get hold of a net that occasionally "misses" something in a complicated position, chooses the wrong plan, etc, even if said position is only complicated at a human level.

Who cares about Elo at this stage?