On-line engine blitz (policy poll)

[hgm]

I would like to get some participant feedback on how we should treat "little-brother" engines, running on vastly slower hardware. So far I have used the policy to prevent participation of multiple copies of the same engine, or a "close derivative". And the same program running on a Raspberry Pi and Intel i7 would still be considered the same program.

Is this a sensible policy, or should there be no objection to participation of such Raspberry-Pie duplicats of other participants? Last tourney even rpiStockfish (which is a Stockfish running on Raspberry Pi) finished only in lower half of the field, so it would not really affect the battle at the top. They would instead fall in the (now little populated) mid-section of the field if they were top engines, and in the sub-microMax tail if they were just average engines.

The advantage of lifting restrictions on such participation would be that we have more participants, of a wider range of strengths. IDuring the last tourney there were 4 engines logged in that now were barred from joining:
NightmareA, Blieps (= Rookie on R.Pi), rpiFruit (because of FruitReloaded), and rpiTogaII (as derivative of Fruit). In the end I did join the latter to make an even number. rpiGlaurung would be excluded anyway because of rpiStockfish.

The danger is that allowing multiple copies is the first step on a sliding scale: 'weak hardware' will eventually get stronger as well. In the future there could be 8-core Raspberri Pies. And is an engine running single-threaded on a 2.4GHz i3 laptop sufficiently weaker than the same engine running on a 12-core i7? How much weaker does the hardware have to be to be qualified as a 'minor participant' that is allowed besides the top-of-the-line version?

This problem has a direct parallel in the definition of "close derivative". Is Stockfish still to be considered a derivative of Glaurung, or has it evolved so much now that there really is nothing against having rpiGlaurung and rpiStockfish participating both, even though they are on the same hardware?

[Henk]

If it is pure a software competition then any difference in hardware makes no difference and same software running on different hardware should be considered identical.

[Henk]

Also if it is only software that counts all engines should run on the same computer or cluster with enough cores. That would be fairest. But then developers still can not develop for that system for they can't use it for testing.

[Joost Buijs]

It depends solely on what you want to accomplish with this tournament.

If it is meant to be a testing ground where programmers can test the performance of their engines it would be a good idea to allow different versions of the same engine.
Also running the same version of an engine on much weaker hardware, gives weaker programs a better opportunity to see where they stand.

If it is pure about competition, who comes at first spot, it is not a good idea to allow different versions of the same engine.

You may also think about whether this has to be a programmers only tournament or not.
Personally I would like it better when there are only engines participating who are entered by the original programmer.

[hgm]

Well, the main purpose is to create an opportunity for people to have fun exercising their hobby. And I try to be pragmatic about that. As it is obviously not possible to conduct the event without participants, priority is on encouriging participation rather than forbidding it. So as long as one more person could have fun by participating, without spoiling the fun for more than one other, why deny him the opportunity?

It seems clear that 10 participants all running Stockfish on top-of-the-line machines would ruin the event. But this is not the situation at all, nor does it seem a likely future development. In fact we hardly ever even have a single Stockfish participating. Most of the participants have been programmers running their own engine for as long as I have been organizing the event. And besides programmers there are other people with a deep interest in computer Chess, and if they want to operate one or more engines, I see no gain whatsoever in excluding them. I'd rather have an operator that engages in the discussion than a programmer that just leave his machine unattended during the entire event. So IMO the issue is more that there should be sufficiently many programmers to have a viable technical discussion, and that having many non-programmers in addition doesn't really hurt anyone.

On a practical note, it would be impossible for me to check which engine people are using anyway. Excluding non-authors would just make people bent on winning the event fake some name for their 'self-written engine', and then run Stockfish or Komodo instead. When they can openly enter these engines, they don't have to do that, and at least we know who has been participating.

[Daniel Anulliero]

I wish your tournament stay for own work of plain of amateurs.
You're right hg , if we have tons of stockfish compile this will ruines the évent and will become boring ...
I want to see new chess programers come with own work, THAT is very funny 😊

[jdart]

I'd like to limit the number of "engine types" that are participating, so two of the same kind are not desirable.

In addition, I think it is problematic if you add more participants w/o increasing the number of rounds. You get a result but then many engines have not had a chance to play each other.

--Jon

[hgm]

Well, 9 rounds of Swiss should be enough for 2^7 = 128 participants, not?

I think the problem is exactly the opposite: the participant strength varies over a very wide range, and pairing a top engine against an average one is basically a waste of time, as the outcome is 99% certain. But due to the comparatively large number of rounds the top engines run out of reasonable opponents pretty fast, (given the black/white restrictions), so that such pairings have to be made.

[Joost Buijs]

I think the problem is exactly the opposite: the participant strength varies over a very wide range, and pairing a top engine against an average one is basically a waste of time, as the outcome is 99% certain. But due to the comparatively large number of rounds the top engines run out of reasonable opponents pretty fast, (given the black/white restrictions), so that such pairings have to be made.

The last few tournaments there were at least 10 or 12 (top) engines of comparable strength, and about the same number of weaker engines.

I don't know anything about Swiss systems at all, but with this high number of participants would it not be possible to split the tournament into 2 groups, one for the stronger engines and one for the weaker ones?
This also gives you the opportunity to add some of the low level hardware engines to the 2nd group.
I guess it would not involve much extra work for the admin because most of it is handled by mamer anyway.

It seems logical to me that a Swiss tournament can only give a reasonable outcome with respect to ranking when the participants are of comparable strength.
If you only want to determine who is the strongest then it doesn't matter of course.

[Henk]

I think the problem is exactly the opposite: the participant strength varies over a very wide range, and pairing a top engine against an average one is basically a waste of time, as the outcome is 99% certain. But due to the comparatively large number of rounds the top engines run out of reasonable opponents pretty fast, (given the black/white restrictions), so that such pairings have to be made.

The last few tournaments there were at least 10 or 12 (top) engines of comparable strength, and about the same number of weaker engines.

I don't know anything about Swiss systems at all, but with this high number of participants would it not be possible to split the tournament into 2 groups, one for the stronger engines and one for the weaker ones?
This also gives you the opportunity to add some of the low level hardware engines to the 2nd group.
I guess it would not involve much extra work for the admin because most of it is handled by mamer anyway.

It seems logical to me that a Swiss tournament can only give a reasonable outcome with respect to ranking when the participants are of comparable strength.
If you only want to determine who is the strongest then it doesn't matter of course.

Even 2 groups may not be enough. But I also do not like to play against chess engines that give away pieces almost instantly. Even there is a difference between bad playing and very bad playing engines.