engine evaluation and chess informant symbols

[Don]

I have a better idea. Someone could put together a sample of a few hundred GM annotated game positions from a variety of Grandmasters and that contain a variety of these glyphs. It could be packaged up in such a way as to take the grunt work out of it. You basically run the program against the glyphs and out pops the thresholds you should use for annotations that would be the most compatible.

Where would I get these annotated games? I will put it together if someone will provide the games or give me a pointer to them.

Interesting approach.
Not every annotator uses the same meaning for the signs, but averaging across a number of them would give you the average meaning.
Issue is where to find annotated games; annotated games are typically NOT free. Chess informant is an obvious high-quality source, chessbase possibly as well, but I dont have either in suitable digital form.

Yes, if someone can just send me a few games - I would like to sample at least 100 position (more is better) with the 3 primary glyphs and their negative counterparts (where black has the advantage.)

[Don]

I have a better idea. Someone could put together a sample of a few hundred GM annotated game positions from a variety of Grandmasters and that contain a variety of these glyphs. It could be packaged up in such a way as to take the grunt work out of it. You basically run the program against the glyphs and out pops the thresholds you should use for annotations that would be the most compatible.

Where would I get these annotated games? I will put it together if someone will provide the games or give me a pointer to them.

Interesting approach.
Not every annotator uses the same meaning for the signs, but averaging across a number of them would give you the average meaning.
Issue is where to find annotated games; annotated games are typically NOT free. Chess informant is an obvious high-quality source, chessbase possibly as well, but I dont have either in suitable digital form.

Yes, if someone can just send me a few games - I would like to sample at least 100 position (more is better) with the 3 primary glyphs and their negative counterparts (where black has the advantage.)

I would also like to test for white/black bias while I am at it. I suspect (but do not know for sure) that players will tend to give white a small advantage more or less automatically - especially if they do not look too hard. But a computer will not be predisposed to think either side has an advantage.

[casaschi]

Yes, if someone can just send me a few games - I would like to sample at least 100 position (more is better) with the 3 primary glyphs and their negative counterparts (where black has the advantage.)

I only have those sample games from chess informant
http://pgn4web.casaschi.net/chess-informant-sample.pgn
I'm afraid too few position with one of those marks and all from the same commentator (Huebner). There's also another issue: I tried one position where the comment was "with the idea of playing f4 resulting in +/-". Well, the engine assesses the position as slightly better for White, but it does not have f4 anywhere in the PV... does this means the engine is failing the analysis because it's missing the winning idea or what?

[Don]

Yes, if someone can just send me a few games - I would like to sample at least 100 position (more is better) with the 3 primary glyphs and their negative counterparts (where black has the advantage.)

I only have those sample games from chess informant
http://pgn4web.casaschi.net/chess-informant-sample.pgn
I'm afraid too few position with one of those marks and all from the same commentator (Huebner). There's also another issue: I tried one position where the comment was "with the idea of playing f4 resulting in +/-". Well, the engine assesses the position as slightly better for White, but it does not have f4 anywhere in the PV... does this means the engine is failing the analysis because it's missing the winning idea or what?

I found a large collection of annotated games, so I am going to check them out.

I don't know if it's possible to feed the positions I will extract to the javascript program but that will not be necessary. I can do a few tests of common position to build a conversion factor.

Don

[casaschi]

I found a large collection of annotated games, so I am going to check them out.

Could you share the file?

[Don]

I found a large collection of annotated games, so I am going to check them out.

Could you share the file?

They are broken up into many pgn files. Here is the source:

http://www.angelfire.com/games3/smartbridge/

I have not taken a look yet to see what sort of quality they are or if they are very rich with the annotation glyphs.

Don

[Don]

I found a large collection of annotated games, so I am going to check them out.

Could you share the file?

They are broken up into many pgn files. Here is the source:

http://www.angelfire.com/games3/smartbridge/

I have not taken a look yet to see what sort of quality they are or if they are very rich with the annotation glyphs.

Don

That quality of these PGN are not very high. The use parenthesis for comments which is non-standard. I can fix this easily enough with a sed script but I see that they are also nesting comments which is not allowed in PGN.

I can fix all these problems but generally once you see one or two kinds of problems you will see several more and it will end up being a major nightmare to clean it all up. pgn-extract does not manage to remove the parenthesis but maybe a combination of sed and pgn-extract will give me what I want.

They may not all be that way, but the first one I checked was.

Don

[casaschi]

Maybe you should not rush with low quality data (you know... garbage in... garbage out...) and wait if someone with some good chess informant or chessbase data comes forward...

[gladius]

The evaluation of this program is reasonable. I put a morra gambit in and the score is not ridiculous. A classic morra position should return a score close to zero despite being a pawn down and probably be considered approximately equal. I get that white is down about 0.1 which means the evaluation is good enough to see a lot of compensation.

Well, reasonable might be an overstatement . It's just psq-tables and mobility. But it gets the job done against most humans, even in javascript.

[Don]

Maybe you should not rush with low quality data (you know... garbage in... garbage out...) and wait if someone with some good chess informant or chessbase data comes forward...

I don't believe there will be a problem with these as they are all annoated by strong players. Not all are grandmasters but that is not going to make a huge difference. The difference between an IM and a Grandmaster may be a few hundred ELO but it's only a small percentage of moves. They will be wrong slightly more often that the GM's but we are not looking for ridiculous precision here, we want a general consensus and quantity is more important. Most of these games actually appear to be annotated by GM's so that is probably not much of an issue.

I see there are other problems with these games as I already suspected. Let me look at some of the other files.