Division And Chess Engine Tournaments

[swami]

First,I would like to get some opinion specifically from following members,and also Engine testers to see how their opinions differ and what is popular amongst followers of engine tournaments,provided they get time to respond:
Graham Banks,Uri Blass, H.G.Muller,Dann Corbitt and Tord Romstad
(Thanks in advance!)

but ofcourse anyone can reply!

I have been running engine tournaments with standard double round robin and division league type quite lately and I have observed the level of competition which is somewhat not interesting in my current format and I would like to tell you what it is that makes it not interesting.

I usually run 1 division with 26 engines,double round robin to get 50 games for each engine as well as the standings that show where they are ranked and if any of the engine get promoted,they get to play more games in upper division in the same edition and if they don't demote and maintain their standings then they are placed in that division itself for the next edition.

I wonder if 1 division with 21 engines(or less) is better than the league with 26 engines? since I noticed that there was a lot of rating difference in engine that finishes first and the engine that is ranked last in the particular division,resulting in lower rated engine getting demoted,getting first in lower division and getting promoted and vice versa.Maybe,Reducing the number of engines and increasing the number of leagues provide good solution to this rating difference problem but it seems that there would not be much competition between engines if the division has less number of participants or would there be?I'm confused.

I wonder what your suggestions are and which one of the format would you pick if you were to run the tournament and why?

Total 121 Engines :11 Divisions,11 Engines each Division

Total 128 Engines: 8 Divisions,16 Engines each Division.

Total 126 Engines: 6 Divisions,21 Engines each Division

Total 130 engines: 5 Divisions,26 Engines each Division

Total 124 Engines: 4 Divisions,31 Engines each Division


Thanks

[Graham Banks]

I voted for 8 divisions of 16 engines.

Of course, a smaller number of engines is always preferable, but then you'd have too many divisions.

Regards, Graham.

[Uri Blass]

I do not know what is best but I suggest 6 divisions with 21 engines

Uri

[swami]

Well,Thanks for your input guys,I don't know what to decide still but I would wait till I get the clear stats.

Thanks again

[Michael Sherwin]

Maybe this is why Leo at WBEC has promotion tournaments.

Personally, I never liked the sectional type tournament.

I like what I call a wave tournament instead.

Lets say that you like 26 engines for 50 games per engine, then a wave would work like such.

Start at the bottom of the rating list with the 26 lowest engines including new unrated engines and choose a wave size, lets say 18 is the size of the wave. Then take the 18 top finishers and add the next 8 from the rating list, but adjust as neccesary for the last group. Make a new rating list and start the next wave. This way you can be sure that there is plenty of room on each wave for any deserving engine to advance. Maybe switch to single round robin instead, but still leave the final group a double. Waves can be progressing on multiple computers. As soon as a wave clears the bottom of the list, ratings can be calcculated, and a new wave started on the next computer.

It would be very interesting for spectators/authors to watch their favorite engines surf the wave.

Many variations are possible. Including a start with a random list to see how many waves it takes to sort out the list. The list could be left in order that the engines fell of the wave.

Just some thoughts!

[swami]

Maybe this is why Leo at WBEC has promotion tournaments.

Personally, I never liked the sectional type tournament.

I like what I call a wave tournament instead.

Lets say that you like 26 engines for 50 games per engine, then a wave would work like such.

Start at the bottom of the rating list with the 26 lowest engines including new unrated engines and choose a wave size, lets say 18 is the size of the wave. Then take the 18 top finishers and add the next 8 from the rating list, but adjust as neccesary for the last group. Make a new rating list and start the next wave. This way you can be sure that there is plenty of room on each wave for any deserving engine to advance. Maybe switch to single round robin instead, but still leave the final group a double. Waves can be progressing on multiple computers. As soon as a wave clears the bottom of the list, ratings can be calcculated, and a new wave started on the next computer.

It would be very interesting for spectators/authors to watch their favorite engines surf the wave.

Many variations are possible. Including a start with a random list to see how many waves it takes to sort out the list. The list could be left in order that the engines fell of the wave.

Just some thoughts!

Hi Michael,

I do get what you meant but i wonder if it is more of manual type than automatic type?And I guess that it would take time than a normal round robin tournament of all engines.

Can you give me more example of what a wave tournament is like if you have time because I only have a slight idea of how it works as the concept is generally new to most of the people.

Is it like,I choose 8 engines from the bottom 26 according to their seedings/initial ratings?and make a tournament,run a single round robin as you mentioned and promote only 1 engine to the next cycle(a wave?) with 8 upper engines and vice versa till i reserve double round robin for final group of engines and promote few to upper division to begin the next wave tournament?Sounds cool.

Thanks.

[Michael Sherwin]

Hi Swami,

There are endless variities to the wave concept. Big wave, little wave, big ocean, little ocean. The more engines in each RR the larger the ocean. The lower the starting position is for the next RR the bigger the wave.

So lets say that you start with the 51 lowest engines and have a 50 RR. Then you keep the 21 (could be 31 for a bigger wave) best finishers and then move up the list 30 (20) spots and make another 50 RR. In the above example I like keeping 31 engines and only drop off 20 (20 fall off the wave) and then moving 20 places up the list for the next RR.

The whole idea is to get rid of class boundries and to allow more engines to ride the wave.

There is also the gauntlet wave ladder idea. Start at the bottom rung of the ladder and have a little gauntlet of surrounding engines for that engine. Rate the games and shuffle the ladder. If a new engine is now on that rung play a little gauntlet for it. When an engine has already had a gauntlet from that rung start over at the next higher rung.

Your wave idea sounds good too!

I imagine that someone would need to write a tournament manager that could be capable of the wave idea.

[swami]

Hi Michael,

There are endless variities to the wave concept. Big wave, little wave, big ocean, little ocean. The more engines in each RR the larger the ocean. The lower the starting position is for the next RR the bigger the wave.

So lets say that you start with the 51 lowest engines and have a 50 RR. Then you keep the 21 (could be 31 for a bigger wave) best finishers and then move up the list 30 (20) spots and make another 50 RR. In the above example I like keeping 31 engines and only drop off 20 (20 fall off the wave) and then moving 20 places up the list for the next RR.

This wave tournament sounds like a long division tournament which is already in the poll,11 engines 11 divisions and it seems that there is almost no difference between a wave cycle and a division of less engines.

and I guess that it is more of the manual type and user has to look for the result each time the single round robin of the lower wave gets finished and manually create a next wave,but if i get to work I wouldn't be looking after it but really the concept of wave tournament sounds cool.

There is also the gauntlet wave ladder idea. Start at the bottom rung of the ladder and have a little gauntlet of surrounding engines for that engine. Rate the games and shuffle the ladder. If a new engine is now on that rung play a little gauntlet for it. When an engine has already had a gauntlet from that rung start over at the next higher rung.

I guess this would take a lot of games and time for one engine and I would guess that it would give you exact strength of the engine and it's ratings.

Your wave idea sounds good too!

Thanks.

I imagine that someone would need to write a tournament manager that could be capable of the wave idea.

Yes,newer Arena first needs to get back its swiss system which was already in older versions and this wave tournament is far away from development and I would guess that no one has proposed this example to the author,maybe someone can?I guess Volker Anuss wrote an additional software for Arena and author Of Buzz wrote one for Winboard,tournament managaer or something,maybe these guys have an idea of how to create it if they get time?I'm sure you can do it,Michael

[CRoberson]

I don't know how to define "interesting" in this context. However, I
do suggest fewer and larger groups. From a ratings perspective, it
is better to have larger groups which increases the degrees of freedom
in the accuracy of the rating calculation.

I think the best system I've seen (for rating accuracy) was by the
author of the chess agent Holmes -- Andreas Herrmann.
He ran each new program against all the others in a gauntlet then
calcualted that rating disregarding the performance of previous
versions of that engine. Given the number of engines, this was
statistically accurate producing more accurate ratings than other
systems.

Of course this system is easily run in parallel with multiple machines
and there are several ways to do it in parallel.

For example, assuming 4 machines:
1) Split the number of guantlet opponents across 4 machines and
run the test engine on each machine. Doesn't necissarily finish
4 times faster due to static load balancing.
2) Run 4 simultaneous engine tests. You have 4 new engines to
test, so set up a full gauntlet on each machine.

To achieve reasonable static load balancing (for #1), I'd evenly distribute
engines by strength. Don't put all the weak programs on one machine
and all the strong programs on another.