A laymans guide to probability theory

[Michael Sherwin]

A laymans guide to probability theory for the computer chess enthusiast.

I am a layman at this subject as I have no formal education in this field. Nor did I ever

read any books on probability theory. However, I know far more about it than most people.

At one time I had a gambling problem. I concocted many a system for making money at the

tables that was sure to succeed. Needless to say, they all failed. Eventhough the primary

system made me money for weeks on end with out fail the 'imposible' odds of a certain

length of bad luck that would break the system would always happen. I will not give

anyone the details of the system so as to not tempt anyone to try it. I will explain the

loosing streak neccesary to break it though.

Assume roulette for this decription and that the chips were placed on either red or

black. A 50/50 bet, except when a zero was spun that kills both red and black. If a

streak of bad luck happened that had 4 losses (4L) for every 3 wins (3W) the system makes

money like crazy. No matter how long the streak last. At 3L for every 2W the system

breaks even no matter how long the streak. When the streak gets a little worse than 3L

for every 2W the system starts loosing money, but it takes a very long time to break the

system. Even at 2L for every 1W it takes hundreds of spins of the wheele to break the

system. The trouble is that even worse odds will persist more often and for longer

periods of time than one could ever imagine! It can happen when you first set down at the

table or weeks latter after you have turned your initial $300 into $30,000.

How did I beat this addiction? Glad you asked! I bought a Radio Shack hand held, 2KB, 4

bit processor, basic language computer and learned to program gambling simulations. Well,

I quickly graduated to a more powerful computer. It escapes my memory what it was though.

Those were rather desperate times for me! I know that the random function on a computer

is not really random so it can not be a perfect simulation tool. The variance to reality

would tend to skew the results in favor of the system. The pseudo random numbers from a

typical computer are more evenly distributed over a longer period of time than the

randomness from a roulette wheel and, therefore, could take simulated months or even

years before the system would go bust. But, it always did go bust, even on the computer.

I was cured!!!

Now for computer chess! The various ELO systems are matematical educated guesses and

nothing more. Everyone here knows that a strict say, 2500 rating that is given is not

likely exactly accurate. Therefore mathematicians use probability theory to create what

is called a margine (error bars) that the actual rating is most likely between. They make

it sound that the probability is that it is almost a certainty that the true rating is

between the error bars. This is the equivelent of my system making money for weaks on end

before going bust. There are many additional anomilies with ELO ratings than with pure

gambling. When a 'bust' happens in the ELO system (notice the word system) it is

propagated throughout the rating pool. If bunches of engines play together with very

little mixing then the relative differences between bunches can be way off, even by more

than a hundred points. Certain engines may play other certain engines that they just do

better against and in limited gauntlets this can be skewed wildly in favor of or to the

detriment of any engine running the gauntlet. Just another possible bust! I am sure that

there are many more bust situations that I could think up.

Professor Robert Hyatt, the author of Crafty, is conducting an experiment to determine

the number of games needed to know that version B of a program is better than version A.

Note that, that does not imply a rating. It is just for determining better or worse.

Sofar he has concluded that at least 2,560 games are needed to determine this accuratly.

Also note that he has removed or minimized many sources of randomness from his procedure,

such as fixed positions rather than opening books, long time controls rather than short

and hand picked opponent engines. But is 2560 games an absolute? No, because, the

'thousand monkey syndrome' can happen at any time. It is however sufficient for his

testing purpose. Who would imagine that that many games would be needed?

If professor Hyatt was trying to calculate the number of games needed for an accurate

rating of a large group of engines that all had their own random opening books and the

wild fluctuation of results thereof, the number of games needed could be in the millions!

What the ELO systems can not take into consideration accuratly is various random

fluctuations in results that can happen at any time. The thousand monkeys can hit any

keys at any time to any wild extent of probability and the ELO system is clueless about it!

MJS