CuteChess-cli "...connection stalls"

[Steve Maughan]

Maverick's UCI interface code seems to be stable. I'm not getting reports of crashes and when you play with the engine in analysis mode I can zip back and forth through the game, while its thinking, with no problem. However, when I stress-test it using CuteChess-cli and 8 concurrent games (on a 4 core + HT Intel i7) Maverick falls over and CuteChess-cli reports "Black's connection stalls" and finishes the test. I'm using the ultra fast "tc=40/2+0.01" time control!

Maverick uses one thread to "think" and the other to capture the input. So in the 8 concurrent game scenario there are really 16 threads spinning - I don't know if this is a problem. When I reduce the concurrent games to 4 it seems OK - strange!

I also set the buffers:

Code: Select all

    setbuf(stdout, NULL);
    setbuf(stdin, NULL);
    setvbuf(stdout, NULL, _IONBF, 0);
    setvbuf(stdin, NULL, _IONBF, 0);

Could this be a problem - are the buffers being overloaded and information lost?

I'm assuming this is a problem with Maverick (has anyone else had this problem with CuteChess-cli). The most obvious possibility is a rare race condition between the two threads. But it's difficult to debug as I cannot even write to a text file when there are 8 concurrent games going on.

Any thoughts or suggestions would be appreciated!

Thanks,

Steve

[tpetzke]

Hi Steve,

I recommend to use only as much threads as you have real cores, so limit it to 4 in your setup.

Thomas...

[tvrzsky]

Hi Steve,
it seems that I faced some similar problems (buffering issues, synchronization of input, output and working threads, instability of Cutechess). Maybe I could give you few hints.
From your other post I deduce you are running Windows. My expierence with C runtime library and output buffering was bad. With MinGW compiler I was not able to find buffer settings which would be stable and reliable. Many experiments, using fflush(), nothing worked. Always there were broken output lines (I mean lines which reached other side of the pipe broken to more parts). I noticed this also with other engines which rely on C output functions (for example Cheng3). This was big problem when I worked on my own tournament manager because parsing output of those engines was complicated and generated big overhead (5-7 % of CPU time contrary to good behaving ones, where overhead was almost negligible, 0,1-0,2 %). Only solution which worked rock solid for me was using of Windows WriteFile() function for output instead of fwrite() and this stuff.
For debugging was wery usefull to switch on debug logging in tournament manager (my one has almost the same as Cutechess) and add notification of various events in your engine to its stdout, which allows somewhat trace synchronization issues. For very detailed monitoring of threads and timing issues was very usefull to work with Perfmon utility from Microsoft. It shows very clearly that with such fast time controls where you practically have no more than tens of milliseconds per move and are running all CPU cores then stealing of just one time slice from your engine thread by some system process can lead to time forfeit, and you can do very little to prevent this, maybe except of toying with process or thread priorities which is not good solution indeed.
Last but not least I was also unlucky with Cutechess which was in my test setup crashing too often to be usable. I am really ashamed that I did not report this to Cutechess team in time in order to help in development but I was lazy and shy to do it because no one else reported any problems with it. Finally my solution was creation of my own manager. However I do not want to defame Cutechess in any way, it is an excellent tool.

[Steve Maughan]

Thanks Thomas & Filip!

Interesting stuff Filip. I'll look into WriteFIle(). Maybe I should switch to Microsoft's compiler and see if it helps. Maybe it's the depth command for CuteChess.

Thanks,

Steve

[tvrzsky]

Ooops, my bad, I did not mean Perfmon but Xperf.
Filip

[hgm]

Just out of curiosity: do you have similar problems when you run WinBoard (or XBoard) in -noGUI mode?

[Steve Maughan]

Hi H.G, - I've never used Windboard in -noGUI mode. I'll download and give it a try.

Thanks,

Steve