The problem here is depth.
For an opening position, 36 plies is bare minimum and any new calculations I perform go at least 40 (38 or 39 plies is really good enough, but I figure it will age too fast and be useless in 4 years).
If it is a rather closed position or a gambit, figure 42 plies before you can trust it.
I found that the orangutan opening needs 50 plies before the strongest engines even start to understand it.
Engines do understand 1.d4 pretty well
Engines also have trouble with all the fianchetto Indian style openings.
Polyglot extension tool
Moderators: hgm, Rebel, chrisw
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Polyglot extension tool
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 1056
- Joined: Thu Mar 09, 2006 4:15 pm
- Location: Long Island, NY, USA
Re: Polyglot extension tool
Dealing with a large epd file should also include some cleanup. It is likely that the file you are dealing with does not need to be cleaned up, but it wouldn't hurt to check. There are 3 cleanup issues that I check:Rebel wrote: ↑Thu Jan 10, 2019 7:35 amThere are several utils that will do that for you:Jonathan003 wrote: ↑Wed Jan 09, 2019 12:35 pm Hi thanks ED, that helps a lot.
I don't know how many positions there are in the bin book, how can I know this? If I export the bin book to pgn with Lucas Chess I get 5000 different opening lines. (5000 games).
How can I split the epd file in 5 parts? Is there some software to do that? Or do I have to do it in a text editor?
40H-EPD - https://40h.000webhostapp.com/
SOMU - http://rebel13.nl/download/utilities.html#somu
But if you only have 5000 EPD lines I would use the advice from Guenther.
1. Are there any records with false "en passant" target squares? If so, this could enable duplicate records to go undetected.
2. Are there any duplicate records? Using duplicates will give extra weight to certain opening positions if the file is used for openings.
3. Are there are records where there are an excessive number of promoted pieces (beyond 1 per side). Removes "composed" positions that are unrealistic.
In my epd tool collection, I have detection and remedies for these issues.
1. epdEnPass to detect false "en passant" target squares.
2. epdOccur to flag duplicates
3. epdExtra to flag records with excessive promotions.
40H-EPD - https://40h.000webhostapp.com/
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Polyglot extension tool
And another problem is volume.
But you have to start somewhere and keep updating. 10 million positions from 3000+ engines is not a bad start. Maybe in 2-3 years the minimum elo can be increased to 3100.
And with the stone-wall, probably with more strategic openings.Dann Corbit wrote: ↑Fri Jan 11, 2019 12:34 pm For an opening position, 36 plies is bare minimum and any new calculations I perform go at least 40 (38 or 39 plies is really good enough, but I figure it will age too fast and be useless in 4 years).
If it is a rather closed position or a gambit, figure 42 plies before you can trust it.
I found that the orangutan opening needs 50 plies before the strongest engines even start to understand it.
Engines do understand 1.d4 pretty well
Engines also have trouble with all the fianchetto Indian style openings.
Besides the comp advice there will be a human advice as well. I have a test database with the games of Magnus Carlsen only and then after 1.e4 c5 the overview looks as follows:
Code: Select all
Reference : consult\3000.dat
Positions : 10.013.033
Moves found for rnbqkbnr/pp1ppppp/8/2p5/4P3/8/PPPP1PPP/RNBQKBNR w KQkq c6
Move Depth Score Engine
g1f3 20 0.28 Stockfish 7
g1f3 21 0.28 Stockfish 6
g1f3 18 0.63 Stockfish 4
g1f3 20 0.34 Stockfish DD
g1f3 20 0.34 Komodo 10
g1f3 18 0.37 Komodo 9
b1c3 19 0.23 Komodo 8
g1f3 19 0.25 Komodo 7
b1c3 18 0.31 Komodo 6
g1f3 18 0.23 Komodo 6
g1f3 18 0.23 Komodo 5
b1c3 19 0.27 Komodo 5
g1f3 18 0.33 Houdini 3
b1c3 17 0.18 Fire 5
g1f3 16 0.28 Fizbo 1.7
g1f3 18 0.21 Fritz 15
b1c3 20 0.30 Fritz 15
g1f3 22 0.18 Chiron 2
g1f3 22 0.25 Chiron 3
g1f3 18 0.21 Gull 2
g1f3 16 0.17 Gull 3
b1c3 19 0.27 Equinox
g1f3 20 0.18 Equinox
g1f3 16 0.20 Critter 1.6
b1c3 16 0.15 Critter 1.6
g1f3 18 0.21 Texel 1.05
b1c3 16 0.46 Naum 4.6
h2h3 17 0.50 Naum 4.6
Reference : consult\gm.dat
Positions : 86.372
Moves found for rnbqkbnr/pp1ppppp/8/2p5/4P3/8/PPPP1PPP/RNBQKBNR w KQkq c6
Move Player Game
g1f3 d7d6 d2d4 Carlsen Carlsen - Gelfand ½-½
g1f3 b8c6 d2d4 Shirov Shirov - Carlsen ½-½
g1f3 b8c6 d2d4 Topalov Topalov - Carlsen ½-½
g1f3 b8c6 f1b5 Nunn Nunn - Carlsen 0-1
g1f3 b8c6 b1c3 Svidler Svidler - Carlsen 1-0
b1c3 g7g6 h2h4 Carlsen Carlsen - N.N. 0-1
g1f3 b8c6 d2d4 Morozevich Morozevich - Carlsen ½-½
g1f3 e7e6 d2d4 Giri Giri - Carlsen 1-0
b2b3 b8c6 c1b2 Carlsen Carlsen - Ehlvest 1-0
g1f3 b8c6 d2d4 Anand,V Anand,V - Carlsen ½-½
b1c3 b8c6 g1f3 Nakamura Nakamura - Carlsen ½-½
g1f3 d7d6 d2d4 Leko Leko - Carlsen ½-½
g1f3 b8c6 d2d4 Kamsky Kamsky - Carlsen 1-0
g1e2 d7d6 b1c3 Carlsen Carlsen - Nakamura 0-1
g1f3 d7d6 d2d4 Ivanchuk Ivanchuk - Carlsen ½-½
g1f3 d7d6 f1b5 Bacrot Bacrot - Carlsen 0-1
g1f3 d7d6 d2d4 Polgar,J Polgar,J - Carlsen 0-1
g1f3 d7d6 d2d4 Short Short - Carlsen ½-½
g1f3 b8c6 c2c3 Nakamura Nakamura - Carlsen ½-½
g1f3 d7d6 d2d4 Adams Adams - Carlsen ½-½
b1c3 b8c6 g2g3 Short Short - Carlsen ½-½
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 12540
- Joined: Wed Mar 08, 2006 8:57 pm
- Location: Redmond, WA USA
Re: Polyglot extension tool
I have a chest chuckle every time I see this:
"90% of coding is debugging, the other 10% is writing bugs."
"90% of coding is debugging, the other 10% is writing bugs."
Taking ideas is not a vice, it is a virtue. We have another word for this. It is called learning.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
But sharing ideas is an even greater virtue. We have another word for this. It is called teaching.
-
- Posts: 239
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Polyglot extension tool
Hi
Does someone know how it's possible to change the notation in a pgn database.
So the notation is like
1. d4 d5 2. Nf3 Nf6 3. Bf4
instead of
1.d4 d5 2.Nf3 Nf6 3.Bf4
To be able to copie and past the moves here
http://www.ficgs.com/wikichess.html
Does someone know how it's possible to change the notation in a pgn database.
So the notation is like
1. d4 d5 2. Nf3 Nf6 3. Bf4
instead of
1.d4 d5 2.Nf3 Nf6 3.Bf4
To be able to copie and past the moves here
http://www.ficgs.com/wikichess.html
-
- Posts: 239
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Polyglot extension tool
I find it myself.
It can be done by copying to a new pgn database in ChessX
I wanted to use the website for annotating my repertoire. But it's not very useful I think.
It can be done by copying to a new pgn database in ChessX
I wanted to use the website for annotating my repertoire. But it's not very useful I think.
-
- Posts: 239
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Polyglot extension tool
Hi Ed
Poly 1.1 is very useful for me for creating my chess repertoire for over the board games.
I use it in conjunction with the functions of Lucas Chess to import only best line or best lines from a bin book to a openings guide.
I use the big database that comes with powerbook 2019 from chessbase to change the weight of the moves of a bin book with poly 1.1
I just change the names of the player with chess assistant 2019, to be able to use the function 'weight on learn' from poly 1.1.
Can you explain me a bit how this function works? Do it use back solving or mini-max like they cal it?
In addition to be able to set the value 100% for the moves with the highest learn value, or with the highest Stockfish score. I would like to be able to to ad a high value to the best 2 or best 3 moves. (If it are good moves). So the score of the best moves would 50% and 50% or 33%, 33% and 33%.
I want to be able to choose for max one best move, two best moves or three best moves.
If two moves have a score of 50% booth moves of the bin book wil be imported in the opening lines, if I choose to import al best lines, for on color, from a bin book in Lucas Chess. If three moves with a score of 33%, 33% and 33% al three moves wil be imported.
I hope you can help me with that.
Poly 1.1 is very useful for me for creating my chess repertoire for over the board games.
I use it in conjunction with the functions of Lucas Chess to import only best line or best lines from a bin book to a openings guide.
I use the big database that comes with powerbook 2019 from chessbase to change the weight of the moves of a bin book with poly 1.1
I just change the names of the player with chess assistant 2019, to be able to use the function 'weight on learn' from poly 1.1.
Can you explain me a bit how this function works? Do it use back solving or mini-max like they cal it?
In addition to be able to set the value 100% for the moves with the highest learn value, or with the highest Stockfish score. I would like to be able to to ad a high value to the best 2 or best 3 moves. (If it are good moves). So the score of the best moves would 50% and 50% or 33%, 33% and 33%.
I want to be able to choose for max one best move, two best moves or three best moves.
If two moves have a score of 50% booth moves of the bin book wil be imported in the opening lines, if I choose to import al best lines, for on color, from a bin book in Lucas Chess. If three moves with a score of 33%, 33% and 33% al three moves wil be imported.
I hope you can help me with that.
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Polyglot extension tool
I am not following, first you wanted the best move based on the SF score always on top (100%) and now you want to vary. Surely it can be done but to avoid disappointment just tell me how you want to have it.
How should the weight percentages look like?
And after 1.e4 same question.
Code: Select all
Book Weight Score Depth Learn
d2d4 51.60% 6 20 0
e2e4 41.96% 19 20 0
c2c4 4.13% 5 20 0
g1f3 2.31% -1 20 0
And after 1.e4 same question.
Code: Select all
Book Weight Score Depth Learn
c7c5 38.85% -32 20 0
e7e5 22.38% -33 20 0
e7e6 15.20% -25 20 0
c7c6 8.63% -34 20 0
d7d6 5.71% -37 20 0
d7d5 5.46% -47 20 0
g8f6 3.77% -52 20 0
90% of coding is debugging, the other 10% is writing bugs.
-
- Posts: 239
- Joined: Fri Jul 06, 2018 4:23 pm
- Full name: Jonathan Cremers
Re: Polyglot extension tool
Hi Ed
I understand your confusion.
I try to explain why I want multiple best lines. (I want to be able to choose for one best move, two best move or tree best moves)
I want multiple best moves with the same score, because if I import al best moves (for one color) from a bin book, in Lucas Chess, and a moves have for exemple 51% weight and the other move has a weight of 49% only the move with the 51% weight wil be imported. The weight has to be 50% and 50% for booth moves to be imported,(Limitations of Lucas Chess at the moment). If tree moves have a score of 33% al tree moves will be imported if I choose to import al best moves in Lucas Chess.
I want to make two repertoires. A narrow repertoire with only one best candidate move, for the side where the repertoire is build for.
If I know that repertoire by hart. I want to widen my repertoire to allow more variety of play.
And I don't want to ad bad moves to the repertoire. That's why I want to use the 'weight on learn' function from poly.
I don't know anything about programming. So I don't know if and how this can be programmed.
Basically I want to make pgn files from bin books. For my white repertoire I want to ad only the best (or multiple best moves),for white and al possible answers for black. For my black repertoire I want to ad only the best (or multiple best moves), for black and al possible answers for white.
I hope you understand a bit better what I want now.
I understand your confusion.
I try to explain why I want multiple best lines. (I want to be able to choose for one best move, two best move or tree best moves)
I want multiple best moves with the same score, because if I import al best moves (for one color) from a bin book, in Lucas Chess, and a moves have for exemple 51% weight and the other move has a weight of 49% only the move with the 51% weight wil be imported. The weight has to be 50% and 50% for booth moves to be imported,(Limitations of Lucas Chess at the moment). If tree moves have a score of 33% al tree moves will be imported if I choose to import al best moves in Lucas Chess.
I want to make two repertoires. A narrow repertoire with only one best candidate move, for the side where the repertoire is build for.
If I know that repertoire by hart. I want to widen my repertoire to allow more variety of play.
And I don't want to ad bad moves to the repertoire. That's why I want to use the 'weight on learn' function from poly.
I don't know anything about programming. So I don't know if and how this can be programmed.
Basically I want to make pgn files from bin books. For my white repertoire I want to ad only the best (or multiple best moves),for white and al possible answers for black. For my black repertoire I want to ad only the best (or multiple best moves), for black and al possible answers for white.
I hope you understand a bit better what I want now.
-
- Posts: 6991
- Joined: Thu Aug 18, 2011 12:04 pm
Re: Polyglot extension tool
I understood the first time But look at the part I bolded. You can make a book from PGN but not the other way around, technically impossible. However your wish makes sense and I will figure out some formula that recalculates the weights that represents the SF10 scores.Jonathan003 wrote: ↑Wed Jan 16, 2019 11:19 pm Hi Ed
I understand your confusion.
I try to explain why I want multiple best lines. (I want to be able to choose for one best move, two best move or tree best moves)
I want multiple best moves with the same score, because if I import al best moves (for one color) from a bin book, in Lucas Chess, and a moves have for exemple 51% weight and the other move has a weight of 49% only the move with the 51% weight wil be imported. The weight has to be 50% and 50% for booth moves to be imported,(Limitations of Lucas Chess at the moment). If tree moves have a score of 33% al tree moves will be imported if I choose to import al best moves in Lucas Chess.
I want to make two repertoires. A narrow repertoire with only one best candidate move, for the side where the repertoire is build for.
If I know that repertoire by hart. I want to widen my repertoire to allow more variety of play.
And I don't want to ad bad moves to the repertoire. That's why I want to use the 'weight on learn' function from poly.
I don't know anything about programming. So I don't know if and how this can be programmed.
Basically I want to make pgn files from bin books. For my white repertoire I want to ad only the best (or multiple best moves),for white and al possible answers for black. For my black repertoire I want to ad only the best (or multiple best moves), for black and al possible answers for white.
I hope you understand a bit better what I want now.
So:
Code: Select all
Book Weight Score Depth Learn
d2d4 51.60% 0.06 20 0
e2e4 41.96% 0.19 20 0
c2c4 4.13% 0.05 20 0
g1f3 2.31% -0.01 20 0
Code: Select all
Book Weight Score Depth Learn
e2e4 80.00% 0.19 20 0
d2d4 10.00% 0.06 20 0
c2c4 10.00% 0.05 20 0
g1f3 0.00% -0.01 20 0
Stay tuned.
90% of coding is debugging, the other 10% is writing bugs.