Problem with UCI engines hash in Arena

[Graham Banks]

Also, Hash is not an UCI parameter that is avaliable in the Configuration window, probably because it is configured globally.

I believe it works as follows:

UCI Hash is available to the user unless
'Engines'
'Manage'
'UCI'
'Common hashtable size'
is checked

That is exactly how it works to set it globally. So, if the user chose 256 MB for all UCI engines, Arena sends the command 'setoption name Hash value 256' to each UCI engine upon start up.

I think he's saying not to tick the common hash table size, but to set each engine up individually.

[Adam Hair]

Also, Hash is not an UCI parameter that is avaliable in the Configuration window, probably because it is configured globally.

I believe it works as follows:

UCI Hash is available to the user unless
'Engines'
'Manage'
'UCI'
'Common hashtable size'
is checked

That is exactly how it works to set it globally. So, if the user chose 256 MB for all UCI engines, Arena sends the command 'setoption name Hash value 256' to each UCI engine upon start up.

I think he's saying not to tick the common hash table size, but to set each engine up individually.

And indeed Norman (and you) are correct. I never tried that before. I have always wanted the engines to generally use the same amount of hash, so I never gave thought to how to assign different hash sizes to different UCI engines.

Code: Select all

 Tab Sheet UCI: 

Here you can set common parameters for all your UCI-engines. 

You can specify a common hash table and table base size here as well as a common path for the endgame table bases (Nalimov Table bases). Also whether the engines should use their own book, you can set here. It is recommended to use 'Own book' for the engines, because Arena itself currently doesn' t have an own book. The advantage is that you don't have to set these options individually for every engine. 

If you do not check a check box here the values you set individually for each engine are used. These values are set by right-clicking on the engine mainlines, configure engine or Ctrl+1 or Ctrl+2 for the first or the second engine, as described in Configuring an Engine . 

If you do check a check box here, the values are used, regardless if a different setting has been provided in the engine-configuration dialogue before or not. Settings made here can not be overwritten by the engines. 

[Sylwy]

RTFM ! This always helps !


Michael
http://www.playwitharena.com

People who say "RTFM!" might be considered rude ! Amen !


SilvianR (being in psycho breakdown because of Miky)

[Lavir]

Can you think of any other options as

-usePolyglotBook
-polyglotBook
-bookDepth
-bookVariation
-defaultHashSize
-defaultCacheSizeEGTB
-smpCores

that would have to be saved per tourney?

I think these (along the Permanent Brain setting) would be perfectly fine. For others more specific things, as you said, an user can just edit the file.

If I understand you correctly, this sounds like something very complex to do. In WinBoard, the engines have no identity other than described in the list of installed engines, which is only acessed the moment the engines are actually loaded because they have to play a game.

Could you not make a section for the options to give for every changed engine?

For example, let's pretend the user changed options "on the fly" for Houdini and Stockfish, for the former enabling learning and for the latter increasing aggressiveness, using something like:

Name=Houdini 3 (this the reference to the WB nickname)
[OPTIONS]
Learning=true
Learning File=try.lrn

Name=Stockfish 2.3.1
[OPTIONS]
Aggressiveness=200

Where you just take the values in the options sections and you give them with simple "set option *"?

You still would have to go through something like the Load Engine dialog for every on-the-fly engine.

Sure, but since it is facultative you would have to do only for those engines options you want to change and if you want to change them you have a reason to do so.

But anyway this is a very specialized thing, not so primary (differently from the indipendent tournament settings above). It would be a nice touch but nothing more, in definitive. Many users I don't think will dabble with it very much and yours being a "jack of all trades" GUI it would be mostly wasted, I think.

But, in case you want to expand even further the engine vs engine capability of WB in the future it's a "professional" touch you can take in consideration.

[hgm]

I think these (along the Permanent Brain setting) would be perfectly fine.

Ah yes. Forgot that one. Well, consider it done.

Could you not make a section for the options to give for every changed engine?

For example, let's pretend the user changed options "on the fly" for Houdini and Stockfish, for the former enabling learning and for the latter increasing aggressiveness, using something like:

Name=Houdini 3 (this the reference to the WB nickname)
[OPTIONS]
Learning=true
Learning File=try.lrn

Name=Stockfish 2.3.1
[OPTIONS]
Aggressiveness=200

Where you just take the values in the options sections and you give them with simple "set option *"?

Well, this is exactly what is made by Polyglot when you install an engine, and then change its options through Engine Settings. What you suggests basically amounts to reinstalling the engine a second time, possibly with modified engine settings, in a tempoary way (to be automatically deleted afte the tourney finishes).

... But, in case you want to expand even further the engine vs engine capability of WB in the future it's a "professional" touch you can take in consideration.

Well, at least I undrstand what you want, and I will give it some thought to see if this could somehow be reconciled with the current way things work. I should make some changes in that area anyway, because currently WinBoard entirely relies on Polyglot for storing engine options modified through the Engine Settings dialog. Which means that option settings for WinBoard engines cannot be saved, unless the engine povides a way to save them by itself.

So at some point I should mae a Save button in the Engine Settings as a standard button, which would cause WinBoard to store all modified engine options in the engine list. Perhaps this could be done in a way that distinguishes permanent settings from temporary settings.

[Lavir]

Just two last little things.

I noticed that in the time control panel, when you set the "tournament" times (i.e. n move in n min) there is only one single TC for all, there's no way to set something like 40/120 + 20/60 + 30 min for rest of the game.

Why did you leave that option aside? I suppose that you have done so purposely, but almost all GUIs have separate options settable by the user there, and IMO (especially the possibility of setting an "n time for rest of game" after a time control) it's somewhat important.

The other is the possibility of enabling a max depth for PGN positions. You can now choose random/sequential but not a max amount for moves if so desired from the positions.

After this, I will bore you no more, promised

[Don]

Would you want to have the individual controls of the latter dialogs as duplicates in the Tounament Options, rather than a quick way to go there? I am afraid this would overload the dialog, which is already quite full. I am also not sure what you mean by 'options'. The General Options?

Because making it as an option there would easily give the possibility of setting the thing individually engine by engine if one so want. The "common engine" has hash and cores, but hash IMO should be given also the possibility of setting it indipendently by engine (for example one 256 and another 512 if one wants) and it would be good even if it was possible to to do so for threads (think about it: in this way your GUI would be the first to have this option so easily achievable). If you can do these things easily from a simple window it's all another thing.

This is why I built my own automated tester. Every engine is configured separately and that even includes the time control. For example you can test Fischer time control against fixed depth and you can give each engine a separate hash size.

Most GUI's will only play 1 game at a time so you have to start many instances of the GUI - my tester let's you set the number of simultaneous matches to run and manages everything for you. Nothing beats a custom solution and I don't have to depend on others to fix problems.

We also stop games that end with king + (non-pawn) for both sides - provided a few moves have been played while being in that exact ending. Our tester allows any number of players to be configured up to 1024 I think. It does not do Swiss but it does allow you to assign a family designation to each player with the rule family doesn't play family. This allows complex or simple gauntlets. In fact by default all programs have an implied family designation which is the name used to specify the program. The name used to specify the program is the handle you assign to each entity so the same binary (or script) can be configured multiple times with different setups.

State is maintained via the PGN output of the tester. You can stop it and restart it without further consideration. For each move I record the depth, time and nodes in the pgn file and I have tools which make sense of the output.

One way to do this easily is to set a simple check as for example the Shredder gui does. It lets you simply chose if you want all engines have these common options or have them changed engine by engine (by simply selecting them). Sadly the Shredder GUI has only this option for book and hash.

It would not "clutter" the interface that much, at all, and on the contrary I actually think the more options you give immediately to the user without sub-dialogs there the better (the most important ones naturallly, the ones that people use most of the times, and surely these are the cores, the hash, books/lines and permanent brain - another thing you have to set in Winboard in yet another window); the user that has not to search everywhere (this btw is what it happens with ChessGUI, that is, in fact, a little troublesome for beginners) and everything of most importance is there, immediately.

Other less frequently used (and more specific) options I agree should be setted elsewhere but these ones (hash, cores, PB, book/lines) IMO should be easily finded without going to search for them (along with possibility of changing engine parameters on the fly, that would be a nice touch) in the tournament/match section that is the section where they are most handy to change.

I personally think that one should specialize in the type of GUI one wants to make: jack of all trades master of none I don't think it works so well, but naturally people can have a different opinion on this. If I had to actually create a GUI myself I would make it the most specialized possible for what it should, indeed, do. I think Winboard, for the "philosophical" way it is created would make a perfect GUI specialized for engine matches. Do you imagine if it was also able to make concurrency games automatically (as CuteChess does, but actually being able to look at the games) so that you could watch/make 4 games side by side while they are being played?

It does already do that, not? If you start a match from the menu, that match will stop after 'default match games', setable from the Common Engine dialog (or, as 'games per pairing', from the Tournament Options dialog). Unless you mean something else than I think.

I meant something like in Arena when there's an option to end the tournament after the game is played or the round/cycle at the touch of a button. It is a nice touch, especially when you play LTC games.

But maybe doing this in Winboard would be more troublesome since I don't know where you could place the option. The minimalistic approach of Winboard is very good for some things but less for others.

In Chessmaster there was even the option to adjourn engine games, a thing I've not seen in any other GUI.

Another thing concerning this: I think that when you abort a game it should not be saved in the pgn if less than certain moves).

Yes, please. I always appreciate feedback. My e-mail address is h DOT g DOT muller ATSIGN hccnet DOT nl. I cannot promise that I will always agree with you, but it always very useful to hear what users want, even if in the end it would only be a minority opinion. (Sometimes irreconcilable differences in user demands can be solved by putting features under option control.)

Ok, I will send you an email ASAP with all the things that IMO would make a nice addition to Winboard for what it concerns engine vs engine matches.

[hgm]

For example you can test Fischer time control against fixed depth and you can give each engine a separate hash size.

Well, you can do most of that in WinBoard through the standard work-arounds (setting an -initString which includes the required protocol commands). But is it really useful? To me it seemed that work-arounds are good enough for that kind of exotic stuff, and that the work-arounds are a bit clumsy and discourage people to do such things is an added benefit...

Most GUI's will only play 1 game at a time so you have to start many instances of the GUI -

I always considered that a large benefit. Because the different instances are independent, errors won't propagate so easily. And I like to layout all the boards on my display, so that I can watch all the games. Of course you can always make a single program behave like multiple indpendent programs, by giving it multiple windows, etc. But that makes things needlessly complicated. I like the conceptual simplicity of the 1-game-1-GUI design.

my tester let's you set the number of simultaneous matches to run and manages everything for you. Nothing beats a custom solution and I don't have to depend on others to fix problems.

No argument with that at all. But of course that is just what WinBoard/XBoard is: my own custom solution for all board games, from mini-Shogi to Tai Shogi and Ultima. Why start from scratch if there is open source?

[Don]

For example you can test Fischer time control against fixed depth and you can give each engine a separate hash size.

Well, you can do most of that in WinBoard through the standard work-arounds (setting an -initString which includes the required protocol commands). But is it really useful? To me it seemed that work-arounds are good enough for that kind of exotic stuff, and that the work-arounds are a bit clumsy and discourage people to do such things is an added benefit...

It's not a work-around for us and for 99% of the developers it's a must have feature. Since the early days of Doch we have never played even matches against foreign opponents as it's a huge waste of time. Instead we would pick a very strong opponent and give it a time handicap. In this way you spend almost all of your time testing YOUR program, not everyone else's.

Some people just try to find opponents of the same general strength and play matches that way - but that is throwing away half of your CPU resources.

I would suggest that all aspiring chess program authors consider doing the same, pick opponents such as Stockfish and handicap appropriately - measure your progress based on that. Once you have gained superiority at that handicap modify it to get to the next level.

We no longer have that luxury however. So we currently play even time control matches with the very best programs. For Houdini we are playing up a little but we have usually preferred playing up to playing down. If Komodo gets much stronger we will have to consider being the one that accepts the handicap in order to get the variety we need.

Most GUI's will only play 1 game at a time so you have to start many instances of the GUI -

I always considered that a large benefit. Because the different instances are independent, errors won't propagate so easily. And I like to layout all the boards on my display, so that I can watch all the games. Of course you can always make a single program behave like multiple indpendent programs, by giving it multiple windows, etc. But that makes things needlessly complicated. I like the conceptual simplicity of the 1-game-1-GUI design.

Nothing wrong with that, but for us our needs are different. We don't want to have to fuss with all of that. Also, a GUI is a waste of time for our needs. There are times when I want a GUI and I use xboard for those times. If I had games being displayed I would want to watch them instead of working

my tester let's you set the number of simultaneous matches to run and manages everything for you. Nothing beats a custom solution and I don't have to depend on others to fix problems.

No argument with that at all. But of course that is just what WinBoard/XBoard is: my own custom solution for all board games, from mini-Shogi to Tai Shogi and Ultima. Why start from scratch if there is open source?

Because nothing beats a custom solution. It's likely I would need to support some feature nobody else cares about and I have no interest in keeping my changes in sync with other developers. For me it's a tool, a means to an end. I'm not a tool developer except when I have to be.

[hgm]

It's not a work-around for us and for 99% of the developers it's a must have feature. Since the early days of Doch we have never played even matches against foreign opponents as it's a huge waste of time. Instead we would pick a very strong opponent and give it a time handicap. In this way you spend almost all of your time testing YOUR program, not everyone else's.

Indeed, this is good strategy. But of course WinBoard supports straightforward time-odds too, even through the GUI menus. What I doubt the usefulness of is playing fixed-depth versus time, or incremental TC vs classical. That just seems to serve no point. WinBoard is actually ideally equiped for implementing such things; afte each move it does "playerClock += TimeQuota(moveNr, tcString)/playerTimeOdds", where tcString can be something like "40/3600:20/900:300+1". Currently both players use the same tcString, built from the conventional -tc, -mps and -inc options, but it would be totally trivial to provide string options -first/secondTC that would overule those, when non-empty, and could then be installed with the engine. So it is not like the work discourages me. I just think it is a bad thing to have. More is not always better, and the 'maximum-flexibility-minimum-usefullness principle' is always looming around the corner.

If I had games being displayed I would want to watch them instead of working

Yes, I know the feeling. Blitz Chess is strongly hypnotic.

Because nothing beats a custom solution. It's likely I would need to support some feature nobody else cares about and I have no interest in keeping my changes in sync with other developers. For me it's a tool, a means to an end. I'm not a tool developer except when I have to be.

That is pretty contradictory once you remove the misconception that starting your own fork from an open-source solution should imply that you must become maintainer of the main branch. That it worked out for me that way is really an unintended coincidence of circumstances, mainly because WinBoard was really orphaned at the time I needed it, and my general susceptability to requests for help that I can easily satisfy. I would not even have had to open the source if I had just used it only privately. If I would have started from scratch, I would have had to spend years before even having a vey basic GUI, if I could have done it at all. (I had never done any GUI progamming whatsoever at that time.) By starting from WinBoard it took me about 1 week before I could do what I originally intended. (Which was playing with 7 piece types rather than 6.)