unbuffered input/ouput

Discussion of chess software programming and technical issues.

Moderators: hgm, Rebel, chrisw

Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

unbuffered input/ouput

Post by Daniel Shawul »

With unbuffered input/ouput why is it that I can only type in 256 characters .
for instance

Code: Select all

setboard wwwwwwwwwwwwwww... (stops at 256th character) 
At first I thought the problem was because of the limit in maximum length of characters, but it seems it is because of

Code: Select all

setbuf(stdout,NULL);
setbuf(stdin,NULL);
Gian-Carlo Pascutto
Posts: 1243
Joined: Sat Dec 13, 2008 7:00 pm

Re: unbuffered input/ouput

Post by Gian-Carlo Pascutto »

Operating system?

Shell?
User avatar
hgm
Posts: 27808
Joined: Fri Mar 10, 2006 10:06 am
Location: Amsterdam
Full name: H G Muller

Re: unbuffered input/ouput

Post by hgm »

I remember that under Unix in the seventies the keyoard buffer was only 256 bytes. So if you did not put the tty in raw mode (and if you wanted to beable to do line editing, you didn't, as even backspacing would not work then), everything you typed after the 256th character was simply lost.
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: unbuffered input/ouput

Post by Daniel Shawul »

It is good that you are here. At first I thought it was OS restriction but the limit is actually 8192 for dos command line. I asked my question elsewhere first thinking it was dos's problem http://www.codeguru.com/forum/showthread.php?t=511777
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: unbuffered input/ouput

Post by Daniel Shawul »

I also thought it was OS problem at first but it seems it is not...
http://www.codeguru.com/forum/showthread.php?t=511777
By the way, I tested Stockfish,Sjeng12 and my engine as well all fail.
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: unbuffered input/ouput

Post by Sven »

I can confirm it happens under Windows XP at least, with various UCI or WinBoard engines. No idea why.

It can be tested also with Fruit2.1 which uses setvbuf(stdin,NULL,_IONBF,0) to define unbuffered input, and reads input lines with fgets() as usual. Just start the engine in console mode and hold down one key, like 'w'. I observed that it always stops after 254 characters.

Time to start the debugger ...

Sven
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: unbuffered input/ouput

Post by Daniel Shawul »

Yep. So far only crafty & tscp passed the test. Tscp doesn't do polling though.
Gian-Carlo Pascutto
Posts: 1243
Joined: Sat Dec 13, 2008 7:00 pm

Re: unbuffered input/ouput

Post by Gian-Carlo Pascutto »

This is a Windows limitation. I observe no such problem on Unix/bash.
Daniel Shawul
Posts: 4185
Joined: Tue Mar 14, 2006 11:34 am
Location: Ethiopia

Re: unbuffered input/ouput

Post by Daniel Shawul »

Agreed. All seem to work fine under linux with select(). It must be _kbhit() and unbuffered input on dos ?
Sven
Posts: 4052
Joined: Thu May 15, 2008 9:57 pm
Location: Berlin, Germany
Full name: Sven Schüle

Re: unbuffered input/ouput

Post by Sven »

Daniel Shawul wrote:Agreed. All seem to work fine under linux with select(). It must be _kbhit() and unbuffered input on dos ?
Unbuffered input: yes. _kbhit() and polling: no. It happens also with my very old engine version Surprise 4.2.8 which has no polling for input during search.

You can easily reproduce it with this non-chess program:

Code: Select all

#include <stdio.h>

int main&#40;int argc, char * argv&#91;&#93;)
&#123;
	if &#40;argc == 1&#41; &#123;
		setbuf&#40;stdin, 0&#41;;
	&#125;
	char line&#91;65536&#93;;
	printf&#40;"input&#58; ");
	fflush&#40;stdout&#41;;
	&#40;void&#41; fgets&#40;line, sizeof&#40;line&#41;, stdin&#41;;
	return 0;
&#125;
which has no polling for input or whatsoever, just plain fgets(). Call the program with any argument to get the buffered version.

At least the MSVC implementation of fgets() internally uses ReadFile(). By stepping through it with the debugger I could see this behaviour when ReadFile() was called: I was able to type 254 'x' characters but not more.

The final explanation why ReadFile() behaves that way with an unbuffered input stream has not been found yet, though.

TSCP seems to use buffered input, at least I did not see any setbuf() or similar. Crafty does not use fgets() but read(). Most others are using fgets(), getline() or similar.

Sven