Scid error on CCRL pgn file imports

Discussion of anything and everything relating to chess playing software and machines.

Moderators: hgm, Rebel, chrisw

stevenaaus
Posts: 608
Joined: Wed Oct 13, 2010 9:44 am
Location: Australia

Re: Scid error on CCRL pgn file imports

Post by stevenaaus »

Adam Hair wrote:The reason for the unique Round names (numbers) is that it is used to identify each game. Games from the published database can be traced back to the submitted pgn. Which has been useful lately. Norm Pollock noticed some discrepancies recently in the databases and it was easy to determine who submitted those games (me :oops: ) and determine what mistake was made. Given this and that the PGN specifications given no limit on the number of Round names, I do not think that the CCRL will change its method.

Of course, I am not expecting you to change Scid vs PC if you feel 2^18 is enough Round names.
Sorry i can't up this limit. Scid has a very tight database format. The 2^18 Round Name limit is enforced a handful of 18 bit wide fields, and will be around until a new format is made (if ever). It was an oversight not to fix this when Pascal made si4 as it would have been realtively easy to make it max_unsigned_integer.

Anyway, here's a new windows snapshot
http://sourceforge.net/projects/scidvsp ... p/download
It gives a half-decent error report, and leaves the database in a better state. Other recent changes include three colour bar graphs to the Tree window (white wins draws black wins), and clickable tablebase moves in the gameinfo window.

Image
stevenaaus
Posts: 608
Joined: Wed Oct 13, 2010 9:44 am
Location: Australia

Re: Scid error on CCRL pgn file imports

Post by stevenaaus »

Note - if one does a global exchange , say changing
[Round -> [GameNumber
this seems to solve the problem because the new field is stored in a more flexible (and slower) data type.
User avatar
hgm
Posts: 27808
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Scid error on CCRL pgn file imports

Post by hgm »

IMO it would be more useful to just clip the round number, and give a warning, rather than aborting the save. So to games get the same round number. Big deal. Presumably that would not render the database completely useless to the user. Better than having software that doesn't want to do anything with it at all...
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Scid error on CCRL pgn file imports

Post by Adam Hair »

stevenaaus wrote:Note - if one does a global exchange , say changing
[Round -> [GameNumber
this seems to solve the problem because the new field is stored in a more flexible (and slower) data type.
Now that the source of the problem is known, people can work around it. Such as you suggest here.
Norm Pollock
Posts: 1056
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: Scid error on CCRL pgn file imports

Post by Norm Pollock »

Adam Hair wrote: The reason for the unique Round names (numbers) is that it is used to identify each game. Games from the published database can be traced back to the submitted pgn. Which has been useful lately. Norm Pollock noticed some discrepancies recently in the databases and it was easy to determine who submitted those games (me :oops: ) and determine what mistake was made. Given this and that the PGN specifications given no limit on the number of Round names, I do not think that the CCRL will change its method.

Of course, I am not expecting you to change Scid vs PC if you feel 2^18 is enough Round names.
I have to correct what I said about using pgnscid.exe as a round-about. It does indeed crash when you exceed 2^18 different Round values. When I had used pgnscid.exe previously, it was on subsets of the CCRL collections, below the 2^18 number of games.

I think the best round-about is to rename "[Round" with something like "[Rd". Assuming you can load the file into a text editor to do the "find and replace".
stevenaaus
Posts: 608
Joined: Wed Oct 13, 2010 9:44 am
Location: Australia

Re: Scid error on CCRL pgn file imports

Post by stevenaaus »

hgm wrote:IMO it would be more useful to just clip the round number, and give a warning, rather than aborting the save. So to games get the same round number. Big deal. Presumably that would not render the database completely useless to the user. Better than having software that doesn't want to do anything with it at all...
Maybe, but it's a hack to implement this, and it still leaves one with a database with maxed out Round Names that will again fail shortly.

CCRL are bastardising the PGN format really.
Norm Pollock wrote:I think the best round-about is to rename "[Round" with something like "[Rd". Assuming you can load the file into a text editor to do the "find and replace".
Rd sounds like a good idea. These extra tag fields are stored uncompressed, so keeping them small is best.
User avatar
hgm
Posts: 27808
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: Scid error on CCRL pgn file imports

Post by hgm »

True, using the round tag in a way for which it was never intended is really abuse of the PGN format.

Just as a service to CCRL you could built in an option to do the replacement Round -> Rd on parsing the PGN file, so that no separate session with a text editor is needed.
Norm Pollock
Posts: 1056
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: Scid error on CCRL pgn file imports

Post by Norm Pollock »

If the file is too big to be loaded into a text editor for "find & replace", then use my utility tool "gameSplit" to break the pgn file into smaller parts. It's in the "40H-PGN" package.
Adam Hair
Posts: 3226
Joined: Wed May 06, 2009 10:31 pm
Location: Fuquay-Varina, North Carolina

Re: Scid error on CCRL pgn file imports

Post by Adam Hair »

Norm Pollock wrote:If the file is too big to be loaded into a text editor for "find & replace", then use my utility tool "gameSplit" to break the pgn file into smaller parts. It's in the "40H-PGN" package.
That is what I already do :wink:

Do you take requests, Norm? I have a two or three ideas for possible tools. Maybe you would find one of them worth doing.
Norm Pollock
Posts: 1056
Joined: Thu Mar 09, 2006 4:15 pm
Location: Long Island, NY, USA

Re: Scid error on CCRL pgn file imports

Post by Norm Pollock »

Adam Hair wrote:
Norm Pollock wrote:If the file is too big to be loaded into a text editor for "find & replace", then use my utility tool "gameSplit" to break the pgn file into smaller parts. It's in the "40H-PGN" package.
That is what I already do :wink:

Do you take requests, Norm? I have a two or three ideas for possible tools. Maybe you would find one of them worth doing.
Sure, I'll take a look at your ideas. And if I use any, I will credit you for the idea. It's best to email using the email address on my site. Or you can PM me here.