New Interchange Protocol / Alternative to PGN

[mar]

I guess that the degree of compaction of this binary format is too small compared to .zip or .gz.

That depends on what exactly you want to do.
If you have a very large database, compressing the whole thing is out of question, so most likely split into blocks (unless you just want a backup).
Also some sort of index to be able to do random access on separate games would be nice.
If you want a true database, then implementing this to allow dynamic changes, fast queries and compact storage is certainly far from trivial
and would probably require a real database engine (sqlite maybe?) or something close to it.

[arjuntemurnikar]

Why did you choose to go with XML rather than using something more modern and readable, but equally easy to parse like JSON or YAML?

[arjuntemurnikar]

Parallel discussion going on at reddit.

It may be interesting to read some of the comments from users point of view.

[gcramer]

Why did you choose to go with XML rather than using something more modern and readable, but equally easy to parse like JSON or YAML?

Please read the previous posts about the capability of XML.

Shortly summarized: XML (or any other structured language) is not really of interest here, this is only a tool, the format of C/CIF is of interest.

PS: For me it's obvious that JSON and YAML are not useful for the description of C/CIF, as the developer I can see the depth behind this. Arguments like 'modern' are useless, modern trash is not gold, and arguments like 'readable' are not of interest, C/CIF is not designed to be readable for humans.

PPS: The statement "modern trash is not gold" does not mean that JSON or YAML is trash, it means that the usage of JSON or YAML for the description of C/CIF is trash.

It may be interesting to read some of the comments from users point of view.

Yes, the comments about C/CIF are of interest, but not about the tool XML. The user will not see the content of C/CIF files, who cares whether it is readable? The fact that also an readable format exists is only an addition. I think the crux is to distinguish between PGN and C/CIF. A readable format PGN is still existing, why should I develop PGN 2? I've developed a format for data interchange between modern applications, and this is a different goal. The primary format CCIF is binary and NOT HUMAN READABLE AT ALL. But the format of C/CIF is described with the use of a textual format. Talking about XML is missing the point, the point is the structure of C/CIF.

PS: Some people are quite fanciful, in some posts I've read "a replacement of PGN", who is developing this?

PPS: In fact I think that some day PGN will be replaced, PGN is out of date, especially the uncontrolled growth (undocumented features) is a problem. And even ChessBase is generating corrupted PGN files under some circumstances, this is a result of syntactical problems in PGN. But C/CIF is not developed to be the replacement.

[gcramer]

In the protocol extension used in the WinBoard Alien Edition multi-leg moving is indicated by using the normal long-algebaric notation of WB protocol for each leg, and separate the legs by commas. This also captures multi-move variants like Marseillaise Chess, or non-standard castlings. For moves with a single piece that perform captures along the way, you could just concatenate all squares it visits.

I've did an update: cif-moveSection.html.

The idea of the VariantMen tag is that it would supply all the rule knowledge relating to piece movement. This should enable the SAN parser to do legality checking, and catch moves that were illegal according to the rules of the variant.

Ok, now I'm understanding the idea of the VariantMen tag. But do you have a solution how to disambiguate SAN moves without the use of a move generator? Implementing a move generator belongs to complex chess logic, and I want to avoid this in C/CIF. And even Scidb is still restricted in variant support, even with the betza notation, the Bitboard in Scidb is 8x8. So I do not yet see a way how to convert any chess variant without a huge effort. Even the check whether a move is legal requires very complex chess logic (although Scidb has this logic). Currently I do not see a way for a conversion of all variants to and from PGN.

[gcramer]

Gregor, I'm not so sure your proposed format will get any wide acceptance.

I'm not sure, too, but everything needs a start.

...If they don't understand the new format, they won't use it, no GUI will use it...

Fortunately application Scidb will use it!

...You don't have a choice but to support the clumsy but easily readable PGN format.

I don't have a choice? What a nonsense!

[hgm]

Ok, now I'm understanding the idea of the VariantMen tag. But do you have a solution how to disambiguate SAN moves without the use of a move generator?

If multiple pieces of the same type X are present, any SAN move with such a piece will need to determine which of the X can reach the indicated square (unless several can, and the disambiguation is explicitly added to the notation. This is unavoidable.

Implementing a move generator belongs to complex chess logic, and I want to avoid this in C/CIF.

Well, considering that a move generator that supports a pretty complex extension of Betza notation in XBoard takes only 230 lines of code, one should not overestimate the complexity. But I fully agree that for an efficient storage format for use by a program this is quite undisirable. In XBoard I also convert PGN databases to an internal format that avoids any need for disambiguation, to do position searches efficiently.

I did not want to suggest that PGN could be made efficient by the VariantMen tag. Just that it could make conversion of PGN of Chess variants to other formats possible without actually knowing the varant.

And even Scidb is still restricted in variant support, even with the betza notation, the Bitboard in Scidb is 8x8. So I do not yet see a way how to convert any chess variant without a huge effort. Even the check whether a move is legal requires very complex chess logic (although Scidb has this logic). Currently I do not see a way for a conversion of all variants to and from PGN.

I agree that it would be difficult to overcome the board-size limitation. I am not so sure about the importance of legality checking, though. That is not the task of a converter. Ambiguity would of course always be fatal, but if there is a game where a Queen makes a Knight jump, I wouldn't see that as a reason to refuse to convert it, or have XBoard find positions in it. Of course there is the tricky case where you would have to disambiguate between two pseudo-legal moves based on one of them leaving you in check. This is really the only case where I have XBoard check move legality, on loading a PGN database. But of course that does mean the code has to be present. But when you have a move generator for individual pieces, it is not very hard to run it on all pieces to check if one of those can capture the King. It just adds a dozen lines of code or so.