Database snapshot

Discussion of anything and everything relating to chess playing software and machines.

Moderators: bob, hgm, Harvey Williamson

Forum rules
This textbox is used to restore diagrams posted with the [d] tag before the upgrade.
Post Reply
Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Fri Aug 02, 2019 2:00 pm

zullil wrote:
Fri Aug 02, 2019 12:34 pm
Ferdy wrote:
Fri Aug 02, 2019 11:26 am

Html interface is released at https://github.com/fsmosca/ChessDB-Online-Book

Download the whole thing and open index.html.

Or try it at https://fsmosca.github.io/ChessDB-Online-Book/
Make sure to enable javascript.
Works fine if I download and open index.html. But the version online you've linked to doesn't work for me.

[EDIT] OK, Chrome is saying "Insecure content blocked. This page is trying to load scripts from unauthenticated sources." If I allow Chrome to "load unsafe scripts", then it works.
Same for me, on chrome, I have to press "Load unsafe scripts" when using the github page. But it is fine, most scripts are from chessboardjs, and chessjs, and I have scripts added to probe the online book and show it on the page.

noobpwnftw
Posts: 360
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Fri Aug 02, 2019 2:31 pm

My API endpoint supports HTTPS, so you could just change the base URL and get rid of the unsafe script warnings.

zullil
Posts: 5683
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Database snapshot

Post by zullil » Fri Aug 02, 2019 3:02 pm

Ferdy wrote:
Fri Aug 02, 2019 1:57 pm
zullil wrote:
Fri Aug 02, 2019 1:10 pm
How was the "winrate" for each position determined?
I suspect it is logistic function based on score. The score is from
"the scores been back-propagated using a weighted averaging function" see first post of this thread.

In javascipt when winrate is missing (by design) for mate scores, I use

Code: Select all

// 2019-08-02: ChessDB does not return winrate when score is mate.
// So we will calculate winrate by logistic function
if (typeof winrate === "undefined") {
  const K = 1.2;
  winrate = 100 * 1/(1 + (10**(-K*score/400)));
  winrate = winrate.toFixed(2);
}
Even for non-mate scores, this function is close to the winrate from server.
OK, so the winrates are simply determined by the scores; no games are actually played. And all the scores are derived deterministically from the scores of the terminal nodes. And these leaf scores are determined by an engine search.

(1) How many terminal nodes are there in this tree?

(2) How, precisely, was the score determined for each terminal node? Deterministic search (one thread) by Stockfish to depth 22? With or without endgame tables? If with, 7-man?

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Fri Aug 02, 2019 3:06 pm

noobpwnftw wrote:
Fri Aug 02, 2019 2:31 pm
My API endpoint supports HTTPS, so you could just change the base URL and get rid of the unsafe script warnings.
Wow nice it worked thanks.

Ferdy
Posts: 4111
Joined: Sun Aug 10, 2008 1:15 pm
Location: Philippines

Re: Database snapshot

Post by Ferdy » Fri Aug 02, 2019 3:20 pm

zullil wrote:
Fri Aug 02, 2019 3:02 pm
Ferdy wrote:
Fri Aug 02, 2019 1:57 pm
zullil wrote:
Fri Aug 02, 2019 1:10 pm
How was the "winrate" for each position determined?
I suspect it is logistic function based on score. The score is from
"the scores been back-propagated using a weighted averaging function" see first post of this thread.

In javascipt when winrate is missing (by design) for mate scores, I use

Code: Select all

// 2019-08-02: ChessDB does not return winrate when score is mate.
// So we will calculate winrate by logistic function
if (typeof winrate === "undefined") {
  const K = 1.2;
  winrate = 100 * 1/(1 + (10**(-K*score/400)));
  winrate = winrate.toFixed(2);
}
Even for non-mate scores, this function is close to the winrate from server.
OK, so the winrates are simply determined by the scores; no games are actually played. And all the scores are derived deterministically from the scores of the terminal nodes. And these leaf scores are determined by an engine search.

(1) How many terminal nodes are there in this tree?

(2) How, precisely, was the score determined for each terminal node? Deterministic search (one thread) by Stockfish to depth 22? With or without endgame tables? If with, 7-man?
Perhaps noobpwnftw can answer your questions.

noobpwnftw
Posts: 360
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Fri Aug 02, 2019 3:28 pm

Ferdy wrote:
Fri Aug 02, 2019 3:20 pm
zullil wrote:
Fri Aug 02, 2019 3:02 pm
Ferdy wrote:
Fri Aug 02, 2019 1:57 pm
zullil wrote:
Fri Aug 02, 2019 1:10 pm
How was the "winrate" for each position determined?
I suspect it is logistic function based on score. The score is from
"the scores been back-propagated using a weighted averaging function" see first post of this thread.

In javascipt when winrate is missing (by design) for mate scores, I use

Code: Select all

// 2019-08-02: ChessDB does not return winrate when score is mate.
// So we will calculate winrate by logistic function
if (typeof winrate === "undefined") {
  const K = 1.2;
  winrate = 100 * 1/(1 + (10**(-K*score/400)));
  winrate = winrate.toFixed(2);
}
Even for non-mate scores, this function is close to the winrate from server.
OK, so the winrates are simply determined by the scores; no games are actually played. And all the scores are derived deterministically from the scores of the terminal nodes. And these leaf scores are determined by an engine search.

(1) How many terminal nodes are there in this tree?

(2) How, precisely, was the score determined for each terminal node? Deterministic search (one thread) by Stockfish to depth 22? With or without endgame tables? If with, 7-man?
Perhaps noobpwnftw can answer your questions.
1. One can iterate the database and find out the exact number.
2. In the code, it is defined as a one thread search to depth 22 by a given version of Stockfish and with contempt = 0, without tablebases and keeps a small hash for recent positions.

zullil
Posts: 5683
Joined: Mon Jan 08, 2007 11:31 pm
Location: PA USA
Full name: Louis Zulli

Re: Database snapshot

Post by zullil » Fri Aug 02, 2019 3:38 pm

noobpwnftw wrote:
Fri Aug 02, 2019 3:28 pm
Ferdy wrote:
Fri Aug 02, 2019 3:20 pm
zullil wrote:
Fri Aug 02, 2019 3:02 pm
Ferdy wrote:
Fri Aug 02, 2019 1:57 pm
zullil wrote:
Fri Aug 02, 2019 1:10 pm
How was the "winrate" for each position determined?
I suspect it is logistic function based on score. The score is from
"the scores been back-propagated using a weighted averaging function" see first post of this thread.

In javascipt when winrate is missing (by design) for mate scores, I use

Code: Select all

// 2019-08-02: ChessDB does not return winrate when score is mate.
// So we will calculate winrate by logistic function
if (typeof winrate === "undefined") {
  const K = 1.2;
  winrate = 100 * 1/(1 + (10**(-K*score/400)));
  winrate = winrate.toFixed(2);
}
Even for non-mate scores, this function is close to the winrate from server.
OK, so the winrates are simply determined by the scores; no games are actually played. And all the scores are derived deterministically from the scores of the terminal nodes. And these leaf scores are determined by an engine search.

(1) How many terminal nodes are there in this tree?

(2) How, precisely, was the score determined for each terminal node? Deterministic search (one thread) by Stockfish to depth 22? With or without endgame tables? If with, 7-man?
Perhaps noobpwnftw can answer your questions.
1. One can iterate the database and find out the exact number.
2. In the code, it is defined as a one thread search to depth 22 by a given version of Stockfish and with contempt = 0, without tablebases and keeps a small hash for recent positions.
Thanks. Nice idea, to automate the creation of an opening book. Unfortunately, depth 22 is way too low to provide a reliable foundation, since most of the leaf positions are quiet and positional in nature.

noobpwnftw
Posts: 360
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Fri Aug 02, 2019 4:43 pm

zullil wrote:
Fri Aug 02, 2019 3:38 pm
Thanks. Nice idea, to automate the creation of an opening book. Unfortunately, depth 22 is way too low to provide a reliable foundation, since most of the leaf positions are quiet and positional in nature.
Unfortunately, such a data set which is better in both quality and quantity does not yet exist.
The project code-base is open to process any UCI-compatible electricity tossing, so it shouldn't be too hard to build one given sufficient resources. :D

I have done some experiments with a dumb engine that understands only PSQT, which turns out that despite being very inefficient in the building process, the database eventually converges to similar understanding of chess at root nodes compared to ones built by strong engines.

Dann Corbit
Posts: 10198
Joined: Wed Mar 08, 2006 7:57 pm
Location: Redmond, WA USA
Contact:

Re: Database snapshot

Post by Dann Corbit » Fri Aug 02, 2019 6:18 pm

Worked for me with chrome.
More fun than a barrel of monkeys.
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.

noobpwnftw
Posts: 360
Joined: Sun Nov 08, 2015 10:10 pm

Re: Database snapshot

Post by noobpwnftw » Fri Aug 02, 2019 9:53 pm

Ferdy wrote:
Fri Aug 02, 2019 1:57 pm
zullil wrote:
Fri Aug 02, 2019 1:10 pm
How was the "winrate" for each position determined?
I suspect it is logistic function based on score. The score is from
"the scores been back-propagated using a weighted averaging function" see first post of this thread.

In javascipt when winrate is missing (by design) for mate scores, I use

Code: Select all

// 2019-08-02: ChessDB does not return winrate when score is mate.
// So we will calculate winrate by logistic function
if (typeof winrate === "undefined") {
  const K = 1.2;
  winrate = 100 * 1/(1 + (10**(-K*score/400)));
  winrate = winrate.toFixed(2);
}
Even for non-mate scores, this function is close to the winrate from server.
The exact algorithm used in the calculation of winrate from score is as follows:
https://github.com/noobpwnftw/chessdb/b ... hp#L84-L86

I personally think that in cases where the outcome is known, showing a percentage of winning rate makes no sense.

Post Reply