Re: TCEC Division 1 results simulator

[JJJ]

40 rounds so far, Division 1, sim predictions for being promoted to Premier Division:
Ethereal 90%
Chiron 83%
Fizbo 19%
Laser 3%

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3374         3334       0.562  0.909
Chiron         3351         3288       0.392  0.835
Fizbo          3301         3287       0.040  0.193
Laser          3247         3226       0.004  0.030
Fritz          3231         3226       0.001  0.014
ChessBrainVB   3247         3279       0.001  0.016
Jonny          3139         3140       0.000  0.000
Booot          3218         3272       0.000  0.002

Based on some not very well researched estimate elos, chances of winning the Premier Division would then be:

Code: Select all

Stockfish      3441         3441       0.572  
Komodo         3405         3405       0.186
Houdini        3400         3400       0.159  
Ethereal       3374         3374       0.055 
Chiron         3351         3351       0.019  
Fire           3326         3326       0.005 
Ginkgo         3322         3322       0.004  
Andscacs       3245         3245       0.000

About the second table. First, you seem to put Ethereal and Chiron too high. Second, generally, that in 56 games each, the winner is not one of the big three, seems to be a very slim possibility, much smaller than 8%. Third, Stockfish to win seems in above 70% to me. It has now "contempt" and is not underperforming in these round-robins with weaker engines. Gut feeling: SF say 72%, Kom 15%, Hou (if not updated) 11%, rest 2%.

And why not going further: SF above 90% to win the TCEC 13 (if Hou is not updated).

I would say the same thing. Houdini updated might have some good chance here.

The big 3 seems impossible to dethrone, even at this sample of game. And fire seems still above Chiron and Ethereal and might be update as well.

[chrisw]

Code: Select all

Andscacs       3245         3245       0.000

I think and I hope that Andscacs will not be last. 0.94 has no elo at CCRL for the moment. Also I have some improvements.

I also hope not!
Of course, the sim, with no cross table, is influenced only be the given elos, and will rank the engines in elo order.
If you have some rough estimate of Andsacs elo progress and a possible elo estimate, we can use that figure instead

[chrisw]

40 rounds so far, Division 1, sim predictions for being promoted to Premier Division:
Ethereal 90%
Chiron 83%
Fizbo 19%
Laser 3%

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3374         3334       0.562  0.909
Chiron         3351         3288       0.392  0.835
Fizbo          3301         3287       0.040  0.193
Laser          3247         3226       0.004  0.030
Fritz          3231         3226       0.001  0.014
ChessBrainVB   3247         3279       0.001  0.016
Jonny          3139         3140       0.000  0.000
Booot          3218         3272       0.000  0.002

Based on some not very well researched estimate elos, chances of winning the Premier Division would then be:

Code: Select all

Stockfish      3441         3441       0.572  
Komodo         3405         3405       0.186
Houdini        3400         3400       0.159  
Ethereal       3374         3374       0.055 
Chiron         3351         3351       0.019  
Fire           3326         3326       0.005 
Ginkgo         3322         3322       0.004  
Andscacs       3245         3245       0.000

About the second table. First, you seem to put Ethereal and Chiron too high. Second, generally, that in 56 games each, the winner is not one of the big three, seems to be a very slim possibility, much smaller than 8%. Third, Stockfish to win seems in above 70% to me. It has now "contempt" and is not underperforming in these round-robins with weaker engines. Gut feeling: SF say 72%, Kom 15%, Hou (if not updated) 11%, rest 2%.

And why not going further: SF above 90% to win the TCEC 13 (if Hou is not updated).

Ethereal and Chiron elos are being initialised to their given elo from the previous round plus 50% of the tournament elo gain, where:
tournament game = sum of [+/-400] / game count including draws
elo = initial elo + 0.5 * tournament gain

The elos of the other six engines are more likely to be problematic (too low, because progress).
Progress is tricky to guess at. At the moment I'ld go for:
Stockfish (++) because worked on continuously and reporting progress
Komodo (+) because worked on, but the side project MCTS may be distracting
Houdini (++) or (+) or (??) because I don't know
Fire (??) no idea at the moment
Ginggo (??)
Andsacs (+) or (++) because worked on

The Monte Carlo rollouts for the Premier are adjusted for the doubled game count (I just checked that I remembered to do that), and that will have effect of increasing the top winning chances for the sim.

[chrisw]

53 rounds so far, Division 1, sim prediction of promotion:
Ethereal 95%
Chiron 94%
Fizbo 7%

First tie-break (assuming zero disconnects) is on direct result between the two, where Chiron is currently ahead against Ethereal.

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3378         3334       0.522  0.958
Chiron         3365         3288       0.467  0.946
Fizbo          3289         3287       0.009  0.072
ChessBrainVB   3256         3279       0.001  0.017
Fritz          3228         3226       0.000  0.006
Jonny          3152         3140       0.000  0.000
Booot          3216         3272       0.000  0.000
Laser          3210         3226       0.000  0.002

TCEC cross table

Code: Select all

 1 Chiron S13        3340 9.0 13 56.25 0 +49 69.2 ·· 1= == =1 1= 0  =1 11
 2 Ethereal 10.85    3341 9.0 13 52.75 0 +44 69.2 0= ·· == 11 1  =1 == 11
 3 Fizbo 2           3284 7.5 14 48.25 0 +19 53.6 == == ·· == == 1= == ==
 4 ChessBrainVB 3.70 3300 6.0 13 37.50 0 -24 46.2 =0 00 == ·· 1= =1 1= 0 
 5 Fritz 16.10       3294 6.0 13 35.50 0 -23 46.2 0= 0  == 0= ·· == 1= 1=
 6 Laser 180818      3194 5.5 13 36.25 0 +28 42.3 1  =0 0= =0 == ·· == ==
 7 Booot 6.3.1       3273 5.5 14 37.00 2 -37 39.3 =0 == == 0= 0= == ·· ==
 8 Jonny 8.1         3274 4.5 13 27.50 0 -55 34.6 00 00 == 1  0= == == ··

[Milos]

40 rounds so far, Division 1, sim predictions for being promoted to Premier Division:
Ethereal 90%
Chiron 83%
Fizbo 19%
Laser 3%

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3374         3334       0.562  0.909
Chiron         3351         3288       0.392  0.835
Fizbo          3301         3287       0.040  0.193
Laser          3247         3226       0.004  0.030
Fritz          3231         3226       0.001  0.014
ChessBrainVB   3247         3279       0.001  0.016
Jonny          3139         3140       0.000  0.000
Booot          3218         3272       0.000  0.002

Based on some not very well researched estimate elos, chances of winning the Premier Division would then be:

Code: Select all

Stockfish      3441         3441       0.572  
Komodo         3405         3405       0.186
Houdini        3400         3400       0.159  
Ethereal       3374         3374       0.055 
Chiron         3351         3351       0.019  
Fire           3326         3326       0.005 
Ginkgo         3322         3322       0.004  
Andscacs       3245         3245       0.000

About the second table. First, you seem to put Ethereal and Chiron too high. Second, generally, that in 56 games each, the winner is not one of the big three, seems to be a very slim possibility, much smaller than 8%. Third, Stockfish to win seems in above 70% to me. It has now "contempt" and is not underperforming in these round-robins with weaker engines. Gut feeling: SF say 72%, Kom 15%, Hou (if not updated) 11%, rest 2%.

And why not going further: SF above 90% to win the TCEC 13 (if Hou is not updated).

Ethereal and Chiron elos are being initialised to their given elo from the previous round plus 50% of the tournament elo gain, where:
tournament game = sum of [+/-400] / game count including draws
elo = initial elo + 0.5 * tournament gain

The elos of the other six engines are more likely to be problematic (too low, because progress).
Progress is tricky to guess at. At the moment I'ld go for:
Stockfish (++) because worked on continuously and reporting progress
Komodo (+) because worked on, but the side project MCTS may be distracting
Houdini (++) or (+) or (??) because I don't know
Fire (??) no idea at the moment
Ginggo (??)
Andsacs (+) or (++) because worked on

The Monte Carlo rollouts for the Premier are adjusted for the doubled game count (I just checked that I remembered to do that), and that will have effect of increasing the top winning chances for the sim.

Just a small illustration how ridiculous are your assumptions (regarding Premier Division):
http://talkchess.com/forum3/viewtopic.p ... 31#p772531

[jdart]

Is TCEC down? I am trying to load http://tcec.chessdom.com/season13/live.php and getting no response.

--Jon

[AndrewGrant]

Is TCEC down? I am trying to load http://tcec.chessdom.com/season13/live.php and getting no response.

--Jon

http://tcec.chessdom.com/beta/live.html

Admins are moving around the GUIs

[Uri Blass]

40 rounds so far, Division 1, sim predictions for being promoted to Premier Division:
Ethereal 90%
Chiron 83%
Fizbo 19%
Laser 3%

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3374         3334       0.562  0.909
Chiron         3351         3288       0.392  0.835
Fizbo          3301         3287       0.040  0.193
Laser          3247         3226       0.004  0.030
Fritz          3231         3226       0.001  0.014
ChessBrainVB   3247         3279       0.001  0.016
Jonny          3139         3140       0.000  0.000
Booot          3218         3272       0.000  0.002

Based on some not very well researched estimate elos, chances of winning the Premier Division would then be:

Code: Select all

Stockfish      3441         3441       0.572  
Komodo         3405         3405       0.186
Houdini        3400         3400       0.159  
Ethereal       3374         3374       0.055 
Chiron         3351         3351       0.019  
Fire           3326         3326       0.005 
Ginkgo         3322         3322       0.004  
Andscacs       3245         3245       0.000

About the second table. First, you seem to put Ethereal and Chiron too high. Second, generally, that in 56 games each, the winner is not one of the big three, seems to be a very slim possibility, much smaller than 8%. Third, Stockfish to win seems in above 70% to me. It has now "contempt" and is not underperforming in these round-robins with weaker engines. Gut feeling: SF say 72%, Kom 15%, Hou (if not updated) 11%, rest 2%.

And why not going further: SF above 90% to win the TCEC 13 (if Hou is not updated).

Ethereal and Chiron elos are being initialised to their given elo from the previous round plus 50% of the tournament elo gain, where:
tournament game = sum of [+/-400] / game count including draws
elo = initial elo + 0.5 * tournament gain

The elos of the other six engines are more likely to be problematic (too low, because progress).
Progress is tricky to guess at. At the moment I'ld go for:
Stockfish (++) because worked on continuously and reporting progress
Komodo (+) because worked on, but the side project MCTS may be distracting
Houdini (++) or (+) or (??) because I don't know
Fire (??) no idea at the moment
Ginggo (??)
Andsacs (+) or (++) because worked on

The Monte Carlo rollouts for the Premier are adjusted for the doubled game count (I just checked that I remembered to do that), and that will have effect of increasing the top winning chances for the sim.

Just a small illustration how ridiculous are your assumptions (regarding Premier Division):
http://talkchess.com/forum3/viewtopic.p ... 31#p772531

Note that TCEC is a different type of competition than 3+1 blitz.
I do not claim that Ethereal is better than Fire at TCEC conditions but only that I do not know.
Here is CCRL 40/40
http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I see
Fire 7.1 64-bit 4CPU 3326
Deep Shredder 13 64-bit 4CPU 3287
Ethereal 10.55 64-bit 4CPU 3283

I guess new ethereal is better than 10.55 and shredder13 so the gap at 40/40 seem to be smaller than 40 elo.

[Milos]

40 rounds so far, Division 1, sim predictions for being promoted to Premier Division:
Ethereal 90%
Chiron 83%
Fizbo 19%
Laser 3%

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3374         3334       0.562  0.909
Chiron         3351         3288       0.392  0.835
Fizbo          3301         3287       0.040  0.193
Laser          3247         3226       0.004  0.030
Fritz          3231         3226       0.001  0.014
ChessBrainVB   3247         3279       0.001  0.016
Jonny          3139         3140       0.000  0.000
Booot          3218         3272       0.000  0.002

Based on some not very well researched estimate elos, chances of winning the Premier Division would then be:

Code: Select all

Stockfish      3441         3441       0.572  
Komodo         3405         3405       0.186
Houdini        3400         3400       0.159  
Ethereal       3374         3374       0.055 
Chiron         3351         3351       0.019  
Fire           3326         3326       0.005 
Ginkgo         3322         3322       0.004  
Andscacs       3245         3245       0.000

About the second table. First, you seem to put Ethereal and Chiron too high. Second, generally, that in 56 games each, the winner is not one of the big three, seems to be a very slim possibility, much smaller than 8%. Third, Stockfish to win seems in above 70% to me. It has now "contempt" and is not underperforming in these round-robins with weaker engines. Gut feeling: SF say 72%, Kom 15%, Hou (if not updated) 11%, rest 2%.

And why not going further: SF above 90% to win the TCEC 13 (if Hou is not updated).

Ethereal and Chiron elos are being initialised to their given elo from the previous round plus 50% of the tournament elo gain, where:
tournament game = sum of [+/-400] / game count including draws
elo = initial elo + 0.5 * tournament gain

The elos of the other six engines are more likely to be problematic (too low, because progress).
Progress is tricky to guess at. At the moment I'ld go for:
Stockfish (++) because worked on continuously and reporting progress
Komodo (+) because worked on, but the side project MCTS may be distracting
Houdini (++) or (+) or (??) because I don't know
Fire (??) no idea at the moment
Ginggo (??)
Andsacs (+) or (++) because worked on

The Monte Carlo rollouts for the Premier are adjusted for the doubled game count (I just checked that I remembered to do that), and that will have effect of increasing the top winning chances for the sim.

Just a small illustration how ridiculous are your assumptions (regarding Premier Division):
http://talkchess.com/forum3/viewtopic.p ... 31#p772531

Note that TCEC is a different type of competition than 3+1 blitz.
I do not claim that Ethereal is better than Fire at TCEC conditions but only that I do not know.
Here is CCRL 40/40
http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I see
Fire 7.1 64-bit 4CPU 3326
Deep Shredder 13 64-bit 4CPU 3287
Ethereal 10.55 64-bit 4CPU 3283

I guess new ethereal is better than 10.55 and shredder13 so the gap at 40/40 seem to be smaller than 40 elo.

CCRL 40/40 is irrelevant. Ethereal only played 400 games there.
It played 50 games in TCEC and ppl nake even more ridiculous conclusions. It'a hype and nothing else. Chance for Ethereal to finish 4th in primary division is less than 30%. To finish in top 3 is less than 1%.

[Milos]

op

40 rounds so far, Division 1, sim predictions for being promoted to Premier Division:
Ethereal 90%
Chiron 83%
Fizbo 19%
Laser 3%

Code: Select all

Column 'First' is chance of winning, Column 'First two' is chance of being in first two
Engine,   Tournament Elo, Initial Elo, First, First two
Ethereal       3374         3334       0.562  0.909
Chiron         3351         3288       0.392  0.835
Fizbo          3301         3287       0.040  0.193
Laser          3247         3226       0.004  0.030
Fritz          3231         3226       0.001  0.014
ChessBrainVB   3247         3279       0.001  0.016
Jonny          3139         3140       0.000  0.000
Booot          3218         3272       0.000  0.002

Based on some not very well researched estimate elos, chances of winning the Premier Division would then be:

Code: Select all

Stockfish      3441         3441       0.572  
Komodo         3405         3405       0.186
Houdini        3400         3400       0.159  
Ethereal       3374         3374       0.055 
Chiron         3351         3351       0.019  
Fire           3326         3326       0.005 
Ginkgo         3322         3322       0.004  
Andscacs       3245         3245       0.000

About the second table. First, you seem to put Ethereal and Chiron too high. Second, generally, that in 56 games each, the winner is not one of the big three, seems to be a very slim possibility, much smaller than 8%. Third, Stockfish to win seems in above 70% to me. It has now "contempt" and is not underperforming in these round-robins with weaker engines. Gut feeling: SF say 72%, Kom 15%, Hou (if not updated) 11%, rest 2%.

And why not going further: SF above 90% to win the TCEC 13 (if Hou is not updated).

Ethereal and Chiron elos are being initialised to their given elo from the previous round plus 50% of the tournament elo gain, where:
tournament game = sum of [+/-400] / game count including draws
elo = initial elo + 0.5 * tournament gain

The elos of the other six engines are more likely to be problematic (too low, because progress).
Progress is tricky to guess at. At the moment I'ld go for:
Stockfish (++) because worked on continuously and reporting progress
Komodo (+) because worked on, but the side project MCTS may be distracting
Houdini (++) or (+) or (??) because I don't know
Fire (??) no idea at the moment
Ginggo (??)
Andsacs (+) or (++) because worked on

The Monte Carlo rollouts for the Premier are adjusted for the doubled game count (I just checked that I remembered to do that), and that will have effect of increasing the top winning chances for the sim.

Just a small illustration how ridiculous are your assumptions (regarding Premier Division):
http://talkchess.com/forum3/viewtopic.p ... 31#p772531

Note that TCEC is a different type of competition than 3+1 blitz.
I do not claim that Ethereal is better than Fire at TCEC conditions but only that I do not know.
Here is CCRL 40/40
http://www.computerchess.org.uk/ccrl/40 ... t_all.html

I see
Fire 7.1 64-bit 4CPU 3326
Deep Shredder 13 64-bit 4CPU 3287
Ethereal 10.55 64-bit 4CPU 3283

I guess new ethereal is better than 10.55 and shredder13 so the gap at 40/40 seem to be smaller than 40 elo.

Oh give me a break and try to say something useful for a change. Both Fire and Ethereal have identical LazySMP as SF, and scale pretty much identically. These claims that we don't know how something behaves at 10 or 40x longer TC is just BS. We do know. There is just compression of rating differences. There is never an inversion. No one ever showed one with any reasonably strong A/B engine.
CCRL 40/40 is irrelevant. Ethereal only played 400 games there. Probably real difference vs Fire is 50-60 Elo instead of 70-80. And that's perfectly fine for 10x longer TC. And it's a simple fact that you compress rating differences with longer TC but you also compress error margins. So probability of marginal events happening remains the same.
It played 50 games in TCEC and ppl make even more ridiculous conclusions. It's a hype and nothing else. Chance for Ethereal to finish 4th in primary division is less than 20%. To finish in top 3 is less than 2%.