jhaglund wrote:It is not clear just yet how many...
I'm shooting for 2k, but 1500 is more realistic.
Joshua
Only thing I worry about is that I would like about 50K games to reach that +/-4 error pretty solidly. 40K is not quite there, usually +/- 4 to 5. I am looking for new opponents, but I would also like to drop at least one from the current group as it is almost too low in rating to matter now.
This is a good time for me to revamp opponents and positions since we are starting a new version and changing all the Elo numbers won't hurt.
Only thing I worry about is that I would like about 50K games to reach that +/-4 error pretty solidly. 40K is not quite there, usually +/- 4 to 5. I am looking for new opponents, but I would also like to drop at least one from the current group as it is almost too low in rating to matter now.
This is a good time for me to revamp opponents and positions since we are starting a new version and changing all the Elo numbers won't hurt.
A test run is all...
Suggest playing more different time controls to add up to 50k. I think the results are still valid...
Only thing I worry about is that I would like about 50K games to reach that +/-4 error pretty solidly. 40K is not quite there, usually +/- 4 to 5. I am looking for new opponents, but I would also like to drop at least one from the current group as it is almost too low in rating to matter now.
This is a good time for me to revamp opponents and positions since we are starting a new version and changing all the Elo numbers won't hurt.
A test run is all...
Suggest playing more different time controls to add up to 50k. I think the results are still valid...
I don't think that will work. For example when I try 10s + 0.1 or 20s + 0.2, I don't see much difference at all in the final results. That means that I am essentially playing each positon 4 times, twice as black, twice as white, same opponent. This is already known to be bad from the very early test results I saw on our cluster. We were playing using 40 positions, and could play a match 16 times and not get a single duplicate game. But the results were wrong, because you have more games, but duplicate outcomes, which BayesElo does not understand.
Imagine what happens if you just play 80 games, and then copy them multiple times into one big PGN file. Now you have (say) 80,000 games (f you duplicate the file 1,000 times) and BayesElo will report a wonderfully low error bar, and it will be completely wrong, because you didn't really try 80,000 independent trials. Every set of 1,000 trials are perfectly correlated and it won't know that nor understand it.
I don't think that will work. For example when I try 10s + 0.1 or 20s + 0.2, I don't see much difference at all in the final results. That means that I am essentially playing each positon 4 times, twice as black, twice as white, same opponent.
If you get duplicates at these time controls you'll just have to remove one of the time controls and find another.... probably longer one.
Have you tried insta-games... 9999 moves in 1 min? ...about 5 seconds per game.
I'm about 60% done with the test file, I believe...
I don't think that will work. For example when I try 10s + 0.1 or 20s + 0.2, I don't see much difference at all in the final results. That means that I am essentially playing each positon 4 times, twice as black, twice as white, same opponent.
If you get duplicates at these time controls you'll just have to remove one of the time controls and find another.... probably longer one.
Have you tried insta-games... 9999 moves in 1 min? ...about 5 seconds per game.
I'm about 60% done with the test file, I believe...
I have tried all sorts of time controls, from game in one second on up. Using different time controls is far more risky than having enough positions so that this is not necessary. There must be some degree of correlation between the games even though the time controls are different, and that wrecks the Elo calculations.
I have tried all sorts of time controls, from game in one second on up. Using different time controls is far more risky than having enough positions so that this is not necessary. There must be some degree of correlation between the games even though the time controls are different, and that wrecks the Elo calculations.
I see... comparing apples and oranges isn't quite what you're looking todo.
Maybe you can just add my file to your's when it's done. To keep it simple this way...
But... I would like a separate test run, even if it's shorter than the norm...
Adding more opponents would be the other way to go...
I have tried all sorts of time controls, from game in one second on up. Using different time controls is far more risky than having enough positions so that this is not necessary. There must be some degree of correlation between the games even though the time controls are different, and that wrecks the Elo calculations.
I see... comparing apples and oranges isn't quite what you're looking todo.
Maybe you can just add my file to your's when it's done. To keep it simple this way...
But... I would like a separate test run, even if it's shorter than the norm...
Adding more opponents would be the other way to go...
I shall work on it today.
I have another couple of potential opponents. When a current test is over, I am going to see if I can add a couple more, and probably back off from 4,000 positions to keep the total around 40K or so. 2 more gives 7 opponents. Maybe 2500 positions instead of 4000 which gets me to 35,000 games...
As far as testing your positions, that's easy to do. Just let me know when.