hgm wrote:I am not sure this is useful. From glancing at the documentation it seems that processes are able to change the process group to which they belong. Of course Polyglot could be made to spawn the engine process in the same group (if it doesn't already), and the GUI to change the group of the spawned engine/adapter to a unique one that could be selectively stopped or killed, but a malicious engine could escape to a different process group (if I understand this concept correctly). Or just spawn a process in a different group to consume resources while the opponent is thinking.
Nevertheless, I should dig into this. XBoard now kills engines that do not respond to quit, but I think it would only kill its direct decendant. It would be nice if it would also kill all non-malicious indirect offspring.
STOP also seems a very useful way for implementing the GUI Pause feature on engines that do not support the 'pause'/'resume' commands (for which I now only pause after they give their move), but also only if it also reaches beyond the adapter.
Even if an engine beyond the adapter does not belong to the same process group, I think it would stop as well as soon as it attempts to read from/write to the pipe over which it is connected to the adapter. The pipe is not broken, and there is no EOF condition, but the reading end of a pipe of which the writing end is in a stopped process should appear like an "empty" pipe, and a process that tries to read from an empty pipe will block. The other direction should work in a similar way. So SIGSTOP would in this case not stop the engine itself immediately, therefore it might only be useful for "pause/resume" under the "same process group" condition. But SIGSTOP would work for Bob's scenario in the adapter case.
I think pipe buffers are pretty large nowadays (64K?), and it would take a lot of engine output before they will fill up and the engine would block during write. Even in the seventies they were 4KB, which would get a WB engine a long way into its ponder search before it blocked. I don't think writing on a pipe to a stopped process is treated any different than to a process that is not currently reading for any other reason. (But I could be wrong.)
hgm wrote:I think pipe buffers are pretty large nowadays (64K?), and it would take a lot of engine output before they will fill up and the engine would block during write. Even in the seventies they were 4KB, which would get a WB engine a long way into its ponder search before it blocked. I don't think writing on a pipe to a stopped process is treated any different than to a process that is not currently reading for any other reason. (But I could be wrong.)
But you don't really use buffered output for engine->GUI communication that needs to arrive in time, do you? Either you switch off buffering, or you do fflush(stdout) immediately after sending your move (otherwise you might lose on time).
Also pondering is not the relevant scenario for using SIGSTOP in Bob's approach, he was talking about ponder=off matches where he described SIGSTOP as a way to prevent engines from doing anything while not on move. Maybe it is different for your pause/resume case which obviously should work for various user scenarios.
The engine might not buffer, but the OS will. A pipe is not something inside the engine. It is a memory buffer (or even a disk file) in the OS, similar to sectors in the disk cache. The user program does not have any control over this, neither the program that writes, nor the program that reads. setbuf or fflush only determine when the user program will use the write() system call to hand the data to the OS (namely immediately), but not what the OS does with that data once it passed though the write().
hgm wrote:The engine might not buffer, but the OS will. A pipe is not something inside the engine. It is a memory buffer (or even a disk file) in the OS, similar to sectors in the disk cache. The user program does not have any control over this, neither the program that writes, nor the program that reads. setbuf or fflush only determine when the user program will use the write() system call to hand the data to the OS (namely immediately), but not what the OS does with that data once it passed though the write().
Sounds right in principle, but then how do you think xboard interacts with an engine to avoid delays when receiving engine moves (or other commands)? You don't want to say that the GUI might happen to process commands from the engine only after the corresponding pipe buffer is full?
Does xboard use real pipes or pseudo-ttys on Linux? I would assume it uses real pipes, but in the latter case we would already have the explanation why it works (for pseudo-ttys line buffering is used) ...
hgm wrote:The engine might not buffer, but the OS will. A pipe is not something inside the engine. It is a memory buffer (or even a disk file) in the OS, similar to sectors in the disk cache. The user program does not have any control over this, neither the program that writes, nor the program that reads. setbuf or fflush only determine when the user program will use the write() system call to hand the data to the OS (namely immediately), but not what the OS does with that data once it passed though the write().
Sounds right in principle, but then how do you think xboard interacts with an engine to avoid delays when receiving engine moves (or other commands)? You don't want to say that the GUI might happen to process commands from the engine only after the corresponding pipe buffer is full?
Does xboard use real pipes or pseudo-ttys on Linux? I would assume it uses real pipes, but in the latter case we would already have the explanation why it works (for pseudo-ttys line buffering is used) ...
I think the explanation why xboard is not affected by pipe buffering is in "backend.c", function "StartChessProgram()". There you have a call to AddInputSource() with parameter "lineByLine" set to TRUE, so I'd say that this code manages somehow to always receive engine input line by line. Would you agree?
hgm wrote:The engine might not buffer, but the OS will. A pipe is not something inside the engine. It is a memory buffer (or even a disk file) in the OS, similar to sectors in the disk cache. The user program does not have any control over this, neither the program that writes, nor the program that reads. setbuf or fflush only determine when the user program will use the write() system call to hand the data to the OS (namely immediately), but not what the OS does with that data once it passed though the write().
Sounds right in principle, but then how do you think xboard interacts with an engine to avoid delays when receiving engine moves (or other commands)? You don't want to say that the GUI might happen to process commands from the engine only after the corresponding pipe buffer is full?
Does xboard use real pipes or pseudo-ttys on Linux? I would assume it uses real pipes, but in the latter case we would already have the explanation why it works (for pseudo-ttys line buffering is used) ...
I think the explanation why xboard is not affected by pipe buffering is in "backend.c", function "StartChessProgram()". There you have a call to AddInputSource() with parameter "lineByLine" set to TRUE, so I'd say that this code manages somehow to always receive engine input line by line. Would you agree?
It works because in case of "lineByLine" xboard uses read() which reads immediately what is available in the pipe, in this case up to the next line ending.
So to return to the original point regarding SIGSTOP: you are right, the pipe buffer will fill up with commands sent by the engine that the adapter does not read since it has received a SIGSTOP. 64k pipe buffer size (Linux) are a lot, engines usually do not produce that much output in a short time frame. So the direction engine -> adapter would not cause an implicit blocking of the engine. What about the other direction, adapter -> engine? The adapter is stopped so it does not write anything into the pipe from which the engine tries to read commands. So the engine processes all lines that made it into the pipe, then it sees no further input is available. What happens now would depend on the input processing strategy of the engine: it may or may not do a blocking read; usually it won't. So this direction will probably not cause implicit blocking as well.
Therefore only the "same process group" approach would help for the SIGSTOP approach.
HGM 1 - Sven 0
or (to obtain a better ELO for me)
HGM 99 - Sven 98
bob wrote:In Unix, the IDEAL GUI would probably send a STOP signal to a process once it receives a move from it, and then send a CONT signal before sending it the next move, then the engine could not do any funny stuff.
An interesting idea. Unfortunately it would not work for engines that run through adapters such as Polyglot. The GUI would just stop the adapter, but not its engine child process. Or are there commands in Unix to stop an entire process tree?
If parent and child process are in the same process group (i.e. have the same PG ID) then it is possible to send a signal (like STOP) to all processes of the group with one command.
Only drawback is not all unix/linux versions support process groups. New ones do, but no guarantees on older ones. Also some screw up the groups so that everything goes under one (main reason for groups was fair-share scheduling where each user gets an equal quantum, not each individual process, which would let me spawn 50 and take over the system effectively.)
hgm wrote:I am not sure this is useful. From glancing at the documentation it seems that processes are able to change the process group to which they belong. Of course Polyglot could be made to spawn the engine process in the same group (if it doesn't already), and the GUI to change the group of the spawned engine/adapter to a unique one that could be selectively stopped or killed, but a malicious engine could escape to a different process group (if I understand this concept correctly). Or just spawn a process in a different group to consume resources while the opponent is thinking.
Nevertheless, I should dig into this. XBoard now kills engines that do not respond to quit, but I think it would only kill its direct decendant. It would be nice if it would also kill all non-malicious indirect offspring.
STOP also seems a very useful way for implementing the GUI Pause feature on engines that do not support the 'pause'/'resume' commands (for which I now only pause after they give their move), but also only if it also reaches beyond the adapter.
Even if an engine beyond the adapter does not belong to the same process group, I think it would stop as well as soon as it attempts to read from/write to the pipe over which it is connected to the adapter. The pipe is not broken, and there is no EOF condition, but the reading end of a pipe of which the writing end is in a stopped process should appear like an "empty" pipe, and a process that tries to read from an empty pipe will block. The other direction should work in a similar way. So SIGSTOP would in this case not stop the engine itself immediately, therefore it might only be useful for "pause/resume" under the "same process group" condition. But SIGSTOP would work for Bob's scenario in the adapter case.
Sven
It might not do a read, it might do a select() to see if there is input and do no read if there is none..
hgm wrote:The engine might not buffer, but the OS will. A pipe is not something inside the engine. It is a memory buffer (or even a disk file) in the OS, similar to sectors in the disk cache. The user program does not have any control over this, neither the program that writes, nor the program that reads. setbuf or fflush only determine when the user program will use the write() system call to hand the data to the OS (namely immediately), but not what the OS does with that data once it passed though the write().
Sounds right in principle, but then how do you think xboard interacts with an engine to avoid delays when receiving engine moves (or other commands)? You don't want to say that the GUI might happen to process commands from the engine only after the corresponding pipe buffer is full?
Does xboard use real pipes or pseudo-ttys on Linux? I would assume it uses real pipes, but in the latter case we would already have the explanation why it works (for pseudo-ttys line buffering is used) ...
I think the explanation why xboard is not affected by pipe buffering is in "backend.c", function "StartChessProgram()". There you have a call to AddInputSource() with parameter "lineByLine" set to TRUE, so I'd say that this code manages somehow to always receive engine input line by line. Would you agree?
It works because in case of "lineByLine" xboard uses read() which reads immediately what is available in the pipe, in this case up to the next line ending.
So to return to the original point regarding SIGSTOP: you are right, the pipe buffer will fill up with commands sent by the engine that the adapter does not read since it has received a SIGSTOP. 64k pipe buffer size (Linux) are a lot, engines usually do not produce that much output in a short time frame. So the direction engine -> adapter would not cause an implicit blocking of the engine. What about the other direction, adapter -> engine? The adapter is stopped so it does not write anything into the pipe from which the engine tries to read commands. So the engine processes all lines that made it into the pipe, then it sees no further input is available. What happens now would depend on the input processing strategy of the engine: it may or may not do a blocking read; usually it won't. So this direction will probably not cause implicit blocking as well.
Therefore only the "same process group" approach would help for the SIGSTOP approach.
HGM 1 - Sven 0
or (to obtain a better ELO for me)
HGM 99 - Sven 98
For the record, read reads EVERYTHING up to either (a) max byte count given in the read or (b) EVERYTHING available in the pipe/socket/file/etc... It doesn't stop at a linefeed.
