Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.
unbuffered input/ouput
Moderators: hgm, Rebel, chrisw
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
-
- Posts: 373
- Joined: Wed Mar 22, 2006 10:17 am
- Location: Novi Sad, Serbia
- Full name: Karlo Balla
Re: unbuffered input/ouput
Could it be because of the size of the PIPE buffer (GUI)?Daniel Shawul wrote:Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.
Best Regards,
Karlo Balla Jr.
Karlo Balla Jr.
-
- Posts: 4052
- Joined: Thu May 15, 2008 9:57 pm
- Location: Berlin, Germany
- Full name: Sven Schüle
Re: unbuffered input/ouput
No, please refer to my non-chess, non-GUI, non-PIPE example in my previous post.Karlo Bala wrote:Could it be because of the size of the PIPE buffer (GUI)?Daniel Shawul wrote:Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.
Sven
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: unbuffered input/ouput
The simple answer is to use read()/write(). There is no limit to the number of bytes you can read, within reason, with a single read. I can't imagine why you would not just go to the basic I/O level and use read(). That is the method of input in Crafty for all things that are read, and it works just fine, whether it be reading from a file, across the network, or from the keyboard.Daniel Shawul wrote:Hope you will finish the job with the ReadFile(). I have surrendered already
Why I need it is due to this really long FEN on a 19x19 board which could be as long as 400 bytes.
All other engine-gui commands are really short which is probably why this problem went unnoticed.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: unbuffered input/ouput
read() is the correct way to deal with this. It eliminates one level of indirection, because everything else from scanf, to gets, to whatever all use read() internally...Daniel Shawul wrote:Yep. So far only crafty & tscp passed the test. Tscp doesn't do polling though.
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: unbuffered input/ouput
That is one of the four methods recommended in the winboard communication protocol. The question is why one of the methods there does not work as intended on some operating system. It also seems many engines have this problem too. A "better" option which avoids hassle with unbuffered streams is to use a separate thread for input/ouput processing. I will use one of the working methods eventually but do you know why the current method doesn't work.
A second way to fix the problem might be to ask your I/O library not to buffer on input. It should then be safe to poll the underlying file descriptor as described above. With C, you can try calling setbuf(stdin, NULL). However, I have never tried this. Also, there could be problems if you use scanf(), at least with certain patterns, because scanf() sometimes needs to read one extra character and "push it back" into the buffer; hence, there is a one-character pushback buffer even if you asked for stdio to be unbuffered. With C++, you can try cin.rdbuf()->setbuf(NULL, 0), but again, I have never tried this.
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: unbuffered input/ouput
There are two levels of I/O. System-level and library-level. You are using library-level I/O which leaves it up to the library developers as to what kind of internal buffer sizes they use. You disabled internal buffering. Yet the data has to be read somewhere.Daniel Shawul wrote:That is one of the four methods recommended in the winboard communication protocol. The question is why one of the methods there does not work as intended on some operating system. It also seems many engines have this problem too. A "better" option which avoids hassle with unbuffered streams is to use a separate thread for input/ouput processing. I will use one of the working methods eventually but do you know why the current method doesn't work.A second way to fix the problem might be to ask your I/O library not to buffer on input. It should then be safe to poll the underlying file descriptor as described above. With C, you can try calling setbuf(stdin, NULL). However, I have never tried this. Also, there could be problems if you use scanf(), at least with certain patterns, because scanf() sometimes needs to read one extra character and "push it back" into the buffer; hence, there is a one-character pushback buffer even if you asked for stdio to be unbuffered. With C++, you can try cin.rdbuf()->setbuf(NULL, 0), but again, I have never tried this.
This might be a legacy issue from some systems. 20+ years ago, many systems stored character strings as a byte value indicating how many characters are present, followed by the characters. That would limit "strings" to 255 bytes, max. fgets has to add a NULL to the end today, which might explain the 254.
I don't see why anyone would choose to use gets/fgets over read() which avoids the library buffering issues completely. Disabling buffering makes sense for output operations if you are unwilling to use flush/fflush. But if you read from that stream, it seems risky.
My read() code works on every system I have tried, from windows, to all the unix variants, VMS, etc. Why keep fooling around and dodging the problem? If you use read(), all this will work flawlessly and you can move on to other issues...
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: unbuffered input/ouput
I could solve the problem in three other ways too but I/we wanted to know why fgets() doesn't work.. is that hard to understand? I used read() in the past too but replaced it with fgets() since the former required removal of the extra new line character at the end. Thanks for the rest of your post though, which seems to explain the cause of this problem.My read() code works on every system I have tried, from windows, to all the unix variants, VMS, etc. Why keep fooling around and dodging the problem? If you use read(), all this will work flawlessly and you can move on to other issues...
-
- Posts: 20943
- Joined: Mon Feb 27, 2006 7:30 pm
- Location: Birmingham, AL
Re: unbuffered input/ouput
If it doesn't work, why does it matter why? Move on to something that does, and always has. This is called "beating your head against a brick wall". It is not a very effective way of developing code.Daniel Shawul wrote:I could solve the problem in three other ways too but I/we wanted to know why fgets() doesn't work.. is that hard to understand? I used read() in the past too but replaced it with fgets() since the former required removal of the extra new line character at the end. Thanks for the rest of your post though, which seems to explain the cause of this problem.My read() code works on every system I have tried, from windows, to all the unix variants, VMS, etc. Why keep fooling around and dodging the problem? If you use read(), all this will work flawlessly and you can move on to other issues...
Removing the N/L is trivial:
*strchr(buffer, '\n') = NULL;
will do the trick. If you are not sure a n/l character is present,
if ((p = strchr(buffer, '\n')) *p = NULL;
will work for cases where it is there or not there...
-
- Posts: 4185
- Joined: Tue Mar 14, 2006 11:34 am
- Location: Ethiopia
Re: unbuffered input/ouput
If it doesn't work, why does it matter why? Move on to something that does, and always has. This is called "beating your head against a brick wall". It is not a very effective way of developing code.
Wrong again. It hasn't stopped me from moving on playing on a 19x19 board. Infact I had already something in place when I first asked the question. If it doesn't interest you that I pursue this, _you_ move on.
Would be a lot more easier if you concentrate on the question.
This is not enough for winboard gui.*strchr(buffer, '\n') = NULL;
will do the trick. If you are not sure a n/l character is present,
if ((p = strchr(buffer, '\n')) *p = NULL;
can not replace a simple fgets() yet. So you see the inconviniece has already grown...
read(fileno(stdin),buffer,MAX_STR);
if ((p = strchr(buffer, '\n')) *p = NULL;