Round robin tournaments... When should I stop?

[govert]

I understand that there may not be a clear answer to the following question, but I would suspect that a few of you have some good approximations for the following problem:

Assume a round-robin tournament with N engines, and M games per engine.
How big margin does the winning engine "A" need in order for us to conclude "Engine A is clearly better than all the others"? (95% confidence perhaps?)

[hgm]

Use the likely-hood of superiority matrix as reported by BayesElo, and multiply all the numbers in the first row, to get an approximate answer. This product should exceed your confidence level.

[govert]

Very good. Thank you!

[Michael Sherwin]

A Tale of Two Tournaments

... of about equal strength.

One, RomiChess led for 30 of 44 rounds to finish in 4th place.

The other RomiChess was in last place for most of the tournament to finish in one of the last places, but not the last.

If BayesElo were an android it would have smoke coming out of its ears!