Database snapshot

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
User avatar
Ovyron
Posts: 2833
Joined: Tue Jul 03, 2007 2:30 am

Re: Database snapshot

Post by Ovyron » Sat Aug 03, 2019 5:10 am

Ferdy wrote:
Fri Aug 02, 2019 11:26 am
Or try it at https://fsmosca.github.io/ChessDB-Online-Book/
Thanks, working perfectly. Is it possible to show a backsolved (minmax) score that comes from leaf nodes alongside the score shown now? Currently it's very hard to find the PV that the database scores best (say, after 1.e4 the best reply is 1...e5 with -7 score, but after 2.Nf3 the score falls to -10, which is worse than 1...e6's -9, forcing user to go back and try to find black's best scoring line, so it's currently impossible to find the mainline, because previous moves have a better score than in the current position.)
Great spirits have always encountered violent opposition from mediocre minds.

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Sat Aug 03, 2019 11:30 am

Ovyron wrote:
Sat Aug 03, 2019 5:10 am
Ferdy wrote:
Fri Aug 02, 2019 11:26 am
Or try it at https://fsmosca.github.io/ChessDB-Online-Book/
Thanks, working perfectly. Is it possible to show a backsolved (minmax) score that comes from leaf nodes alongside the score shown now? Currently it's very hard to find the PV that the database scores best (say, after 1.e4 the best reply is 1...e5 with -7 score, but after 2.Nf3 the score falls to -10, which is worse than 1...e6's -9, forcing user to go back and try to find black's best scoring line, so it's currently impossible to find the mainline, because previous moves have a better score than in the current position.)
Perhaps noobpwnftw can implement that.

I tried to get the score but only 4 plies ahead from the top move to easily verify and debug.

Image

The current score of c3 is 17, but if we follow the yellow-highlighted 4-move pv line, the score of e5 the 5th move in the pv is only worth 12.

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Sat Aug 03, 2019 12:27 pm

Another example but this time I get the score of the max depth less 1 move, if odd, I subtract 1 further, if even that will be it.

Image

I want to examine the score of Bb5, d4, Nc3 and Bc4 close to the leaf of best pv by moving each move on the board and record the score of black. The bestmove by score will be the move that has the worst score for black.

1. Move Bb5 on the board.
Score of a6 after 28 plies: -8

2. Move d4 on the board.
Score of exd4 after 22 plies: 0

3. Move Nc3 on the board.
Score of Nf6 after 32 plies: -6

4. Move Bc4 on the board.
Score of Nf6 after 14 plies: -14

So the bestmove of white according to the score is Bc4 because the score of black after moving it is -14.

noobpwnftw
Posts: 362
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Sat Aug 03, 2019 4:47 pm

One principle has to be noted that the score of a move provided only makes sense to those of the moves on the very same position.
The database handles aggregation of scores on subsequent lines internally and it does not follow mini-max calculation.

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Sat Aug 03, 2019 5:18 pm

noobpwnftw wrote:
Sat Aug 03, 2019 4:47 pm
One principle has to be noted that the score of a move provided only makes sense to those of the moves on the very same position.
The database handles aggregation of scores on subsequent lines internally and it does not follow mini-max calculation.
We know it does not follow mini-max. And Ovyron likes to have a mini-max score. What I did was to walk the best pv as close as possible to the leaf and then get its score and compare it with the current position score. If you can return the score of the leaf from following the top pv that would be great. So that we can have root position score and leaf position score, this could be a good substiture for mini-max.

zullil
Posts: 5684
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Database snapshot

Post by zullil » Sat Aug 03, 2019 6:50 pm

noobpwnftw wrote:
Sat Aug 03, 2019 4:47 pm
One principle has to be noted that the score of a move provided only makes sense to those of the moves on the very same position.
The database handles aggregation of scores on subsequent lines internally and it does not follow mini-max calculation.
But two distinct positions, say B and C, can both arise as children of some position A. If the scores coming from B are not comparable to those coming from A, then does it make sense to use these to determine a score for A?

I, too, would prefer to see the pure mini-max score given for each non-terminal node. After all, mini-max is the canonical way to assign values to nodes.

noobpwnftw
Posts: 362
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Sat Aug 03, 2019 7:37 pm

zullil wrote:
Sat Aug 03, 2019 6:50 pm
noobpwnftw wrote:
Sat Aug 03, 2019 4:47 pm
One principle has to be noted that the score of a move provided only makes sense to those of the moves on the very same position.
The database handles aggregation of scores on subsequent lines internally and it does not follow mini-max calculation.
But two distinct positions, say B and C, can both arise as children of some position A. If the scores coming from B are not comparable to those coming from A, then does it make sense to use these to determine a score for A?

I, too, would prefer to see the pure mini-max score given for each non-terminal node. After all, mini-max is the canonical way to assign values to nodes.
Most engines already do that, run an engine evaluation and you should get that score in matter of seconds.

zullil
Posts: 5684
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Database snapshot

Post by zullil » Sat Aug 03, 2019 7:46 pm

noobpwnftw wrote:
Sat Aug 03, 2019 7:37 pm
zullil wrote:
Sat Aug 03, 2019 6:50 pm
noobpwnftw wrote:
Sat Aug 03, 2019 4:47 pm
One principle has to be noted that the score of a move provided only makes sense to those of the moves on the very same position.
The database handles aggregation of scores on subsequent lines internally and it does not follow mini-max calculation.
But two distinct positions, say B and C, can both arise as children of some position A. If the scores coming from B are not comparable to those coming from A, then does it make sense to use these to determine a score for A?

I, too, would prefer to see the pure mini-max score given for each non-terminal node. After all, mini-max is the canonical way to assign values to nodes.
Most engines already do that, run an engine evaluation and you should get that score in matter of seconds.
Sure, but then we have no control over the shape of the tree or the determination of the scores for the terminal nodes.

It's your database, so of course you can construct it as you choose. But it seems odd to me not to provide true mini-max scores for the tree you've so carefully constructed.

noobpwnftw
Posts: 362
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Sat Aug 03, 2019 7:59 pm

zullil wrote:
Sat Aug 03, 2019 7:46 pm
noobpwnftw wrote:
Sat Aug 03, 2019 7:37 pm
zullil wrote:
Sat Aug 03, 2019 6:50 pm
noobpwnftw wrote:
Sat Aug 03, 2019 4:47 pm
One principle has to be noted that the score of a move provided only makes sense to those of the moves on the very same position.
The database handles aggregation of scores on subsequent lines internally and it does not follow mini-max calculation.
But two distinct positions, say B and C, can both arise as children of some position A. If the scores coming from B are not comparable to those coming from A, then does it make sense to use these to determine a score for A?

I, too, would prefer to see the pure mini-max score given for each non-terminal node. After all, mini-max is the canonical way to assign values to nodes.
Most engines already do that, run an engine evaluation and you should get that score in matter of seconds.
Sure, but then we have no control over the shape of the tree or the determination of the scores for the terminal nodes.

It's your database, so of course you can construct it as you choose. But it seems odd to me not to provide true mini-max scores for the tree you've so carefully constructed.
See, how then would one choose which moves to explore? And how to determine scores for terminal nodes reliably?
In my database, I use a shallow engine search on all moves to decide which moves to explore, then use a deeper engine search to decide the scores of terminal nodes. After that, a slightly different method than mini-max to back-propagate the scores.

I also keep the scores of terminal nodes unchanged(which are largely reproducible by using the same engine with the same parameters). So if you really have a problem with my method of calculating the scores of their parent nodes, then you can implement any method you'd like to calculate them and work with the database dump provided earlier, there is even a ply value indicating which position could be reached first from startpos to prevent possible chicken & egg problem with various ways of aggregating scores.

In essence, given the shape of the tree and scores of all the terminal nodes, others are just transient.

I'd be more than happy to discuss problems with the foundations of the project: how to get better query performance on a large database, how to store information more efficiently and how to make incremental/live database dumps so that everyone can get what they want.

Ferdy
Posts: 4113
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Mon Aug 05, 2019 10:26 am

Interface updated. Leaf node score of top 4 moves are shown (Be patient it is a bit slower due to some server requests). Also it is now possible to save game in pgn format.
https://fsmosca.github.io/ChessDB-Online-Book/

Image

Post Reply