Open Opening Book Standard (OOBS)

[Ferdy]

Your FEN string should be rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq e3 0 1
(missing enpassant)

Right I missed that e.p square. I use python-chess to generate the FEN and EPD and it already removed unnecessary e.p. square.

this is exactly why I asked about FEN canonicalization, obviously this task is left to the user.
actually, since you can't ep capture on e3, both FENs are actually legit and describe the same position.

in fact, engines get rid of invalid ep squares internally, so I actually prefer the "stripped" FEN - and by the way, polyglot also doesn't hash invalid ep

I see by doing that we may gain little benefit (save some tiny amount of space) but we need to add more code and may have trouble with compatibility with other software.

Entities that already supported the new FEN and EPD format on e.p square.
1. Chessbase
2. ChessOK
3. HCE (Hiarcs Chess Explorer)
4. COW/BookUp (Chess Opening Wizard)
5. CPT (Chess Position Trainer)
6. Cutechess
7. Lichess
8. Scid vs PC
9. pgn-extract
10. Python chess
11. Julia chess
12. Nibbler
13. PyChess
14. LucasChess

The top 5 are commercials.

[Robert Pope]

...Speed: I don’t have any speed benchmark of OOBS at the moment but they should be significantly slower than Polyglot ones...

An engine needs to query the book before starting each search. Speed becomes extremely important if you're testing ideas by playing games-in-ten-seconds. What would be the overhead of 60 queries?

Steve

I was wondering if it made sense to have two databases: one with moves that can be reached in the first 5 (or 10) moves for super-fast common queries, and a much larger database for everything else.

[phhnguyen]

Your FEN string should be rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq e3 0 1
(missing enpassant)

Right I missed that e.p square. I use python-chess to generate the FEN and EPD and it already removed unnecessary e.p. square.

this is exactly why I asked about FEN canonicalization, obviously this task is left to the user.
actually, since you can't ep capture on e3, both FENs are actually legit and describe the same position.

in fact, engines get rid of invalid ep squares internally, so I actually prefer the "stripped" FEN - and by the way, polyglot also doesn't hash invalid ep

I see by doing that we may gain little benefit (save some tiny amount of space) but we need to add more code and may have trouble with compatibility with other software.

Entities that already supported the new FEN and EPD format on e.p square.
1. Chessbase
2. ChessOK
3. HCE (Hiarcs Chess Explorer)
4. COW/BookUp (Chess Opening Wizard)
5. CPT (Chess Position Trainer)
6. Cutechess
7. Lichess
8. Scid vs PC
9. pgn-extract
10. Python chess
11. Julia chess
12. Nibbler
13. PyChess
14. LucasChess

The top 5 are commercials.

I think almost all GUIs/tools can understand FEN/EPD. You may give them a FEN/EPD with or without en-passant values and they can understand them correctly. However, they will GENERATE en-passant values as the traditional way, say, always create them (when a side advanced Pawns two rows), regardless if it is useful or useless (the opposite can/couldn't capture those Pawns).

[phhnguyen]

...Speed: I don’t have any speed benchmark of OOBS at the moment but they should be significantly slower than Polyglot ones...

An engine needs to query the book before starting each search. Speed becomes extremely important if you're testing ideas by playing games-in-ten-seconds. What would be the overhead of 60 queries?

Steve

I was wondering if it made sense to have two databases: one with moves that can be reached in the first 5 (or 10) moves for super-fast common queries, and a much larger database for everything else.

Use different paths?

At the moment, we create books from PGN/OCGDB files. Just change the output path and parameters you will have a new book. The process is quite fast, thus, creating a new one with new parameters when needed is not a painful process

[phhnguyen]

The project has just been updated:
- Use EPD (Extended Position Description) instead of FEN
- Supports creating PGN, EPD books (besides OOBS and Polyglot)
- More options to be set up

[phhnguyen]

Your FEN string should be rnbqkbnr/pppppppp/8/8/4P3/8/PPPP1PPP/RNBQKBNR b KQkq e3 0 1
(missing enpassant)

Right I missed that e.p square. I use python-chess to generate the FEN and EPD and it already removed unnecessary e.p. square.

this is exactly why I asked about FEN canonicalization, obviously this task is left to the user.
actually, since you can't ep capture on e3, both FENs are actually legit and describe the same position.

in fact, engines get rid of invalid ep squares internally, so I actually prefer the "stripped" FEN - and by the way, polyglot also doesn't hash invalid ep

OK, OOBS now supports EPD with "legal" en-passant. The main reason is to keep consistent with Polyglot hashkeys/books.

[phhnguyen]

Version Beta 1 released

This is the first release. The program could:

• create a new opening book from PGN or OCGDBS (Open Chess Game Database Standard, .ocgdb.db3) files. It supports some opening book formats: OOBS (Open Opening Book Standard, .obs.db3), Polyglot (.bin), PGN (.pgn), EPD (.epd)

• query information for given EPD strings from an OOBS book

• make benchmark

There are three binaries for Windows 64, macOS and Linux

[dangi12012]

Hey I just take a closer look at the current state of both Standards. Is it too much to ask to also support PQL here too?
So oobs -q "Q[e4] = 1" etc.

With this project we could also print the fen strings for positive matches which OCGDB cannot do.
OCGDB does not resolve each fen - but here in OOBS we could print the winrate and fens coupled with a PQL query.
This is very important for some queries.

With the above idea for instance its directly possible to query a winrate for any given piece being on a certain square in the opening.
A direct query for a Piece-Square Tables would be possible.

[phhnguyen]

Hey I just take a closer look at the current state of both Standards. Is it too much to ask to also support PQL here too?
So oobs -q "Q[e4] = 1" etc.

With this project we could also print the fen strings for positive matches which OCGDB cannot do.
OCGDB does not resolve each fen - but here in OOBS we could print the winrate and fens coupled with a PQL query.
This is very important for some queries.

With the above idea for instance its directly possible to query a winrate for any given piece being on a certain square in the opening.
A direct query for a Piece-Square Tables would be possible.

Yes, it is possible and easy to add to OOBS since we have all the code/functions already. I will implement it soon.

Actually, it is a very good idea because we have been targeting Opening book's researchers/editors, and high-rank players. Definitely, they need some more tools/features for studying. Thanks!

[phhnguyen]

Hey I just take a closer look at the current state of both Standards. Is it too much to ask to also support PQL here too?
So oobs -q "Q[e4] = 1" etc.

With this project we could also print the fen strings for positive matches which OCGDB cannot do.
OCGDB does not resolve each fen - but here in OOBS we could print the winrate and fens coupled with a PQL query.
This is very important for some queries.

With the above idea for instance its directly possible to query a winrate for any given piece being on a certain square in the opening.
A direct query for a Piece-Square Tables would be possible.

The project code has been updated with the ability of PQL querying. Compile then run it as:

Code: Select all

./oobs -in ./bookmb-3-45-30.obs.db3 -q "Q[e4]" -o printall -cpu 4