Vinvin wrote:I tested with a merge version of CCRL, CEGT and WBEC (I'll produce a list next week with bayeselo.exe) : a little bit more than 1 Millions games and 3500 different players.
The program crashed ... is it because numbers are too big ?
Based on a recent discussion on the IPON rankings, I decided to clean up the command line interface and release it. It may be an alternative to BayesELO and ELOSTAT.
Miguel
It could certainly be the problem. I hardwired a limit, but the program should warn about it. Whatever it is, it is my fault. I am sorry, I rushed to get this out, otherwise it would not have happened. I will check and fix it tonight. Crashing is unacceptable.
The program did not converge well. This is rare (I have seen it only once before), but it may happen when the data is difficult to chew (for instance, I see KoK with 100% performance, which is infinite rating). Still, I would like to deal with tough situations like this. Is there any chance you can send me the file you used?
Vinvin wrote:I tested with a merge version of CCRL, CEGT and WBEC (I'll produce a list next week with bayeselo.exe) : a little bit more than 1 Millions games and 3500 different players.
The program crashed ... is it because numbers are too big ?
Based on a recent discussion on the IPON rankings, I decided to clean up the command line interface and release it. It may be an alternative to BayesELO and ELOSTAT.
Miguel
It could certainly be the problem. I hardwired a limit, but the program should warn about it. Whatever it is, it is my fault. I am sorry, I rushed to get this out, otherwise it would not have happened. I will check and fix it tonight. Crashing is unacceptable.
The program did not converge well. This is rare (I have seen it only once before), but it may happen when the data is difficult to chew (for instance, I see KoK with 100% performance, which is infinite rating). Still, I would like to deal with tough situations like this. Is there any chance you can send me the file you used?
Miguel
PS: I send you the email by PM
The file when compress with 7z weight 60 MB , probably too big to send by mail
Vinvin wrote:I tested with a merge version of CCRL, CEGT and WBEC (I'll produce a list next week with bayeselo.exe) : a little bit more than 1 Millions games and 3500 different players.
The program crashed ... is it because numbers are too big ?
Based on a recent discussion on the IPON rankings, I decided to clean up the command line interface and release it. It may be an alternative to BayesELO and ELOSTAT.
Miguel
It could certainly be the problem. I hardwired a limit, but the program should warn about it. Whatever it is, it is my fault. I am sorry, I rushed to get this out, otherwise it would not have happened. I will check and fix it tonight. Crashing is unacceptable.
The program did not converge well. This is rare (I have seen it only once before), but it may happen when the data is difficult to chew (for instance, I see KoK with 100% performance, which is infinite rating). Still, I would like to deal with tough situations like this. Is there any chance you can send me the file you used?
Miguel
PS: I send you the email by PM
The file when compress with 7z weight 60 MB , probably too big to send by mail
Vinvin wrote:I tested with a merge version of CCRL, CEGT and WBEC (I'll produce a list next week with bayeselo.exe) : a little bit more than 1 Millions games and 3500 different players.
The program crashed ... is it because numbers are too big ?
Based on a recent discussion on the IPON rankings, I decided to clean up the command line interface and release it. It may be an alternative to BayesELO and ELOSTAT.
Miguel
It could certainly be the problem. I hardwired a limit, but the program should warn about it. Whatever it is, it is my fault. I am sorry, I rushed to get this out, otherwise it would not have happened. I will check and fix it tonight. Crashing is unacceptable.
The program did not converge well. This is rare (I have seen it only once before), but it may happen when the data is difficult to chew (for instance, I see KoK with 100% performance, which is infinite rating). Still, I would like to deal with tough situations like this. Is there any chance you can send me the file you used?
Miguel
PS: I send you the email by PM
The file when compress with 7z weight 60 MB , probably too big to send by mail
This "10000" seems strange ... does that means "impossible to converge" ?
It means it is struggling and moving too slow to find the optimum. I limit the number of iterations per phase to 10000, and then, it moves to the next one with a higher resolution (smaller increment). Sometimes, that helps.
I downloaded WBEC results from here http://wbec-ridderkerk.nl/html/WBEC1to10.rar
and excluded
"Xadreco 5.6",
"GreKo 2.37",
"King Of Kings 1.95",
"Neurosis 1.1",
because they either won all games, or lost all games.
Even after that, the results converge, but slowly and the results look odd (too many weak engines on top). This is a symptom that there are several pools that are either not connected, or loosely connected (for instance, strong engines did not play weak engines, except one fluke game that was lost). I will examine this more carefully, but it I confirm what I say, the data is very problematic. In addition, too many engines played too few games.
Vinvin wrote:I tested with a merge version of CCRL, CEGT and WBEC (I'll produce a list next week with bayeselo.exe) : a little bit more than 1 Millions games and 3500 different players.
The program crashed ... is it because numbers are too big ?
Based on a recent discussion on the IPON rankings, I decided to clean up the command line interface and release it. It may be an alternative to BayesELO and ELOSTAT.
Miguel
It could certainly be the problem. I hardwired a limit, but the program should warn about it. Whatever it is, it is my fault. I am sorry, I rushed to get this out, otherwise it would not have happened. I will check and fix it tonight. Crashing is unacceptable.
The program did not converge well. This is rare (I have seen it only once before), but it may happen when the data is difficult to chew (for instance, I see KoK with 100% performance, which is infinite rating). Still, I would like to deal with tough situations like this. Is there any chance you can send me the file you used?
Miguel
PS: I send you the email by PM
The file when compress with 7z weight 60 MB , probably too big to send by mail
This "10000" seems strange ... does that means "impossible to converge" ?
It means it is struggling and moving too slow to find the optimum. I limit the number of iterations per phase to 10000, and then, it moves to the next one with a higher resolution (smaller increment). Sometimes, that helps.
I downloaded WBEC results from here http://wbec-ridderkerk.nl/html/WBEC1to10.rar
and excluded
"Xadreco 5.6",
"GreKo 2.37",
"King Of Kings 1.95",
"Neurosis 1.1",
because they either won all games, or lost all games.
Even after that, the results converge, but slowly and the results look odd (too many weak engines on top). This is a symptom that there are several pools that are either not connected, or loosely connected (for instance, strong engines did not play weak engines, except one fluke game that was lost). I will examine this more carefully, but it I confirm what I say, the data is very problematic. In addition, too many engines played too few games.
Miguel
Actually, it wasn't that bad. I messed up the pgn results when I wanted to quickly remove the results of some engines
After I removed
"Xadreco 5.6",
"GreKo 2.37",
"King Of Kings 1.95",
"Neurosis 1.1",
"Thor'sHammer 1.9",
"Pyotr Club Ed 1.0",
"MiniChessAI 1.02",
It converges, not super fast, but it does.
After you remove the results of an engine, all of the sudden, another shows up with a 0% or a 100% (because a victory or a defeat was removed). Maybe I should include a feature to automatically detect and remove the results of these engines.
SIMULATION
This version includes the swich -s that allows to perform simulations.
The program will will virtually 'replay' the games getting results
based on the probabilities given by the ratings previously obtained.
After the simulations have been run a standard deviation is calculated.
The error matrix file will give the error between each of the pairs of
players. A bare minimum number will be -s 10, but a reliable number would be
at least -s 100. Take into account that this will take time. Each simulation
takes the same amount of time as each rating calculation.
The rest of the switches available are:
usage: ordo [-OPTION]
-h print this help
-v print version number and exit
-L display the license information
-q quiet (no screen progress updates)
-a <avg> set general rating average
-p <file> input file in .pgn format
-c <file> output file (comma separated value format)
-o <file> output file (text format), goes to the screen if not present
-s # perform # simulations to calculate errors
-e <file> saves an error matrix, if -s was used
ORDOPREP
A utility is included that will shrink the pgn file to "results only".
In addition, it could discard players that won all games, or lost all
games (which generally bring problems). A typical use will be
ordoprep -d -p raw.pgn -o shrunk.pgn
'-d' tells ordoprep to discard players with 100% or 0% performance.
After this, 'shrunk.pgn' could be used as input for 'ordo'
SIMULATION
This version includes the swich -s that allows to perform simulations.
The program will will virtually 'replay' the games getting results
based on the probabilities given by the ratings previously obtained.
After the simulations have been run a standard deviation is calculated.
The error matrix file will give the error between each of the pairs of
players. A bare minimum number will be -s 10, but a reliable number would be
at least -s 100. Take into account that this will take time. Each simulation
takes the same amount of time as each rating calculation.
The rest of the switches available are:
usage: ordo [-OPTION]
-h print this help
-v print version number and exit
-L display the license information
-q quiet (no screen progress updates)
-a <avg> set general rating average
-p <file> input file in .pgn format
-c <file> output file (comma separated value format)
-o <file> output file (text format), goes to the screen if not present
-s # perform # simulations to calculate errors
-e <file> saves an error matrix, if -s was used
ORDOPREP
A utility is included that will shrink the pgn file to "results only".
In addition, it could discard players that won all games, or lost all
games (which generally bring problems). A typical use will be
ordoprep -d -p raw.pgn -o shrunk.pgn
'-d' tells ordoprep to discard players with 100% or 0% performance.
After this, 'shrunk.pgn' could be used as input for 'ordo'
Miguel
Thanks, I've ordopreped my big files (removed all moves, 0% and 100%). Ordo still have troubles to converge, I sent you the files (2 MB now).
I've a request : please make an option to remove engines with <5% and >95% results ...
Vinvin wrote:Thanks, I've ordopreped my big files (removed all moves, 0% and 100%). Ordo still have troubles to converge, I sent you the files (2 MB now).
I've a request : please make an option to remove engines with <5% and >95% results ...
You can have all players at 50% and still have the algorithm diverge. What is necessary to ensure is that every player has at least a path made of wins to any other player, and a path made of losses (a draw can count as a win or as a loss, as needed). Otherwise, the algorithm cannot converge to finite values.
bayeselo has the "connect" command that does this. Pick a player, and it will keep only the other players that can be compared to it.
I am not sure if it is sufficient to help ordo to converge, but I am sure it is necessary.
Vinvin wrote:Thanks, I've ordopreped my big files (removed all moves, 0% and 100%). Ordo still have troubles to converge, I sent you the files (2 MB now).
I've a request : please make an option to remove engines with <5% and >95% results ...
You can have all players at 50% and still have the algorithm diverge. What is necessary to ensure is that every player has at least a path made of wins to any other player, and a path made of losses (a draw can count as a win or as a loss, as needed). Otherwise, the algorithm cannot converge to finite values.
bayeselo has the "connect" command that does this. Pick a player, and it will keep only the other players that can be compared to it.
I am not sure if it is sufficient to help ordo to converge, but I am sure it is necessary.
Rémi
That was not my point
I simply want to remove very low engines ...
