As I see it you already analyzed more positions with SF. Problem is there are same positions with different ce values on same engine. What I am proposing is to delete the duplicates in alpha.epd where alpha.epd is the epd from alpha.pgn + update_alpha.pgn.Frank Quisinsky wrote:Hi Ferdinand,
Example:
If I have 4 times the EPD with 4 different values ...
reject with 0.50 / -0.30
1. Value = -0.20
2. Value = -0.25
3. Value = -0.30
4. Value = -0.31
I reject only 1/4 with Value = -0.31 after Stockfish analysis. Because I do it by hand with game number information under Chessbase GUI. I created from the 87.417 PGN file CBH database files. PGN and EPD have the same game numbers! With epdOrder by Norm I can sort the EPD file with CE Information and delete the game numbers by hand in CBH file.
Reject information can be found in my database v1.03 file in the Stockfish subdirectory: reject
With other words ... during this work I can't see that the position is 4 times in the database.
After I reject what Stockfish find out + the update database of 4.728 games I build the beta-1.pgn file.
In beta-1.pgn (82.704 games) is now three times the PGN included because only 1 time removed.
That is indeed a problem yes!
Because better is to reject 4/4.
Maybe possible with your programming?
If epd more as 1 time in database delete all of it if one of them higher as value in criteria.txt
---
Now Komodo analysed the database without doubles. I will not have the problem again. After Komodo all other engines will be analysed again without doubles.
You wrote:
This duplicates makes the calculation complicated.
I know that ... and not thinking about it at first.
Best
Frank
Much more easy is to do this one.
Forget the 87.417 alpha database.
New main database is the beta-1.pgn database after Stockfish analyses with 26.619 games without doubles or 82.704 with doubles. If Komodo is ready we have 26.619 positions in EPD with ce. too because Komodo analysesd not all ... only the smaller database without doubles.
Maybe it make more sense to work and compare results from your tool with the beta-1 database and not with the alpha.pgn database with or without the update I create.
I don't know!
In around 4 days Komodo is done and I can create the beta-2 file and Houdini will be the next.
Best
Frank
Here is the criteria on what epd's will be removed for same epd with different ce, acd and acn.
1. Criteria will be based on acd and acn which are already present in alpha.epd, acd is more important than acn.
2. All epd with lower acd will be deleted
3. If acd is the same then delete the epd with lower acn.
Example:
Code: Select all
pos1 ce -15; acd 29; acs 30; acn 1000000;
pos1 ce -30; acd 29; acs 30; acn 1200000;
pos1 ce -24; acd 30; acs 30; acn 1020000;
Since the acd of pos1 with ce -24 is high (30), then delete the first 2 pos1.Code: Select all
pos1 ce -15; acd 29; acs 30; acn 1000000;
pos1 ce -30; acd 29; acs 30; acn 1200000;
Since the acn of pos1 with ce -30 is high (1200000), then delete the first pos1.