unbuffered input/ouput

[Daniel Shawul]

Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.

[Karlo Bala]

Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.

Could it be because of the size of the PIPE buffer (GUI)?

[Sven]

Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.

Could it be because of the size of the PIPE buffer (GUI)?

No, please refer to my non-chess, non-GUI, non-PIPE example in my previous post.

Sven

[bob]

Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.

The simple answer is to use read()/write(). There is no limit to the number of bytes you can read, within reason, with a single read. I can't imagine why you would not just go to the basic I/O level and use read(). That is the method of input in Crafty for all things that are read, and it works just fine, whether it be reading from a file, across the network, or from the keyboard.

[bob]

Yep. So far only crafty & tscp passed the test. Tscp doesn't do polling though.

read() is the correct way to deal with this. It eliminates one level of indirection, because everything else from scanf, to gets, to whatever all use read() internally...

[Daniel Shawul]

That is one of the four methods recommended in the winboard communication protocol. The question is why one of the methods there does not work as intended on some operating system. It also seems many engines have this problem too. A "better" option which avoids hassle with unbuffered streams is to use a separate thread for input/ouput processing. I will use one of the working methods eventually but do you know why the current method doesn't work.

A second way to fix the problem might be to ask your I/O library not to buffer on input. It should then be safe to poll the underlying file descriptor as described above. With C, you can try calling setbuf(stdin, NULL). However, I have never tried this. Also, there could be problems if you use scanf(), at least with certain patterns, because scanf() sometimes needs to read one extra character and "push it back" into the buffer; hence, there is a one-character pushback buffer even if you asked for stdio to be unbuffered. With C++, you can try cin.rdbuf()->setbuf(NULL, 0), but again, I have never tried this.

[bob]

That is one of the four methods recommended in the winboard communication protocol. The question is why one of the methods there does not work as intended on some operating system. It also seems many engines have this problem too. A "better" option which avoids hassle with unbuffered streams is to use a separate thread for input/ouput processing. I will use one of the working methods eventually but do you know why the current method doesn't work.

A second way to fix the problem might be to ask your I/O library not to buffer on input. It should then be safe to poll the underlying file descriptor as described above. With C, you can try calling setbuf(stdin, NULL). However, I have never tried this. Also, there could be problems if you use scanf(), at least with certain patterns, because scanf() sometimes needs to read one extra character and "push it back" into the buffer; hence, there is a one-character pushback buffer even if you asked for stdio to be unbuffered. With C++, you can try cin.rdbuf()->setbuf(NULL, 0), but again, I have never tried this.

There are two levels of I/O. System-level and library-level. You are using library-level I/O which leaves it up to the library developers as to what kind of internal buffer sizes they use. You disabled internal buffering. Yet the data has to be read somewhere.

This might be a legacy issue from some systems. 20+ years ago, many systems stored character strings as a byte value indicating how many characters are present, followed by the characters. That would limit "strings" to 255 bytes, max. fgets has to add a NULL to the end today, which might explain the 254.

I don't see why anyone would choose to use gets/fgets over read() which avoids the library buffering issues completely. Disabling buffering makes sense for output operations if you are unwilling to use flush/fflush. But if you read from that stream, it seems risky.

My read() code works on every system I have tried, from windows, to all the unix variants, VMS, etc. Why keep fooling around and dodging the problem? If you use read(), all this will work flawlessly and you can move on to other issues...

[Daniel Shawul]

My read() code works on every system I have tried, from windows, to all the unix variants, VMS, etc. Why keep fooling around and dodging the problem? If you use read(), all this will work flawlessly and you can move on to other issues...

I could solve the problem in three other ways too but I/we wanted to know why fgets() doesn't work.. is that hard to understand? I used read() in the past too but replaced it with fgets() since the former required removal of the extra new line character at the end. Thanks for the rest of your post though, which seems to explain the cause of this problem.

[bob]

My read() code works on every system I have tried, from windows, to all the unix variants, VMS, etc. Why keep fooling around and dodging the problem? If you use read(), all this will work flawlessly and you can move on to other issues...

I could solve the problem in three other ways too but I/we wanted to know why fgets() doesn't work.. is that hard to understand? I used read() in the past too but replaced it with fgets() since the former required removal of the extra new line character at the end. Thanks for the rest of your post though, which seems to explain the cause of this problem.

If it doesn't work, why does it matter why? Move on to something that does, and always has. This is called "beating your head against a brick wall". It is not a very effective way of developing code.

Removing the N/L is trivial:

*strchr(buffer, '\n') = NULL;

will do the trick. If you are not sure a n/l character is present,

if ((p = strchr(buffer, '\n')) *p = NULL;

will work for cases where it is there or not there...

[Daniel Shawul]

If it doesn't work, why does it matter why? Move on to something that does, and always has. This is called "beating your head against a brick wall". It is not a very effective way of developing code.

Wrong again. It hasn't stopped me from moving on playing on a 19x19 board. Infact I had already something in place when I first asked the question. If it doesn't interest you that I pursue this, _you_ move on.
Would be a lot more easier if you concentrate on the question.

*strchr(buffer, '\n') = NULL;

will do the trick. If you are not sure a n/l character is present,

if ((p = strchr(buffer, '\n')) *p = NULL;

This is not enough for winboard gui.

read(fileno(stdin),buffer,MAX_STR);
if ((p = strchr(buffer, '\n')) *p = NULL;

can not replace a simple fgets() yet. So you see the inconviniece has already grown...