I'm trying to write a client for an existing server (an Asterisk
"Call Manager" server) that converses in text over a bog-standard
bi-directional TCP connection. I haven't done much sockets programming
and need some advice.

I successfully establish my socket connection to the server using
getaddrinfo() (to build the IPV4 address, socket() (to establish
an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
me to read a text line from the socket before commencing my client
requests.

For the line read, I decided to try stdio, so I fdopen(,"r") using
the acquired socket. The next steps are the ones that are currently
causing me issues.

IF I

char vrm[16];
fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);

to extract the AMI version number that follows the text slash, my
fscanf() hangs.

BUT, if I

char buffer[1024], vrm[16];
fgets(buffer,sizeof buffer,fd);
sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm);

I manage to obtain the appropriate data. A dump of the
buffer array shows that I did indeed capture the introduction line
from the server.

So, why the difference in behaviour? Obviously, in the fscanf() version,
I've not set something up right. Any hints as to what I've done wrong
would be greatly appreciated.

Thanks in advance for your help
--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 01:44:53 +0000, Lew Pitcher wrote:

> I'm trying to write a client for an existing server (an Asterisk
> "Call Manager" server) that converses in text over a bog-standard
> bi-directional TCP connection. I haven't done much sockets programming
> and need some advice.
[snip]

Here's the code, and some trial runs

First, the code. It embodies three different ways of using stdio to read
the incoming stream.

With no compile options, it dumps (as hex values and as ASCII characters)
the first 29 octets read from the socket. This should (and does) represent
the introduction text sent by the server.

With the -DUSE_FGETS compile option, it uses fgets() to retrieve the first
text line from the server, and sscanf() to parse it. This works.

With the -DUSE_FSCANF compile option, it uses fscanf() to retrieve and
parse the first text line from the server. This hangs.

8<-------------------- main.c -------------------------------------->8
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <ctype.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netdb.h>

/*
** NB: compile with
** -DUSE_FSCANF for fscanf() parse of 1st input line, or
** -DUSE_FGETS for fgets()/sscanf() parse of 1st input line
** otherwise code will use fgetc() to read the first 29 bytes of input
*/

int netOpen(char *host, char *port);

int main(void)
{ int status = 1;
int s_ami; /* base socket from netOpen(), fd for amiIn */

if ((s_ami = netOpen("localhost","5038")) >= 0)
{
FILE *amiIn;

if (amiIn = fdopen(s_ami,"r"))
{
#ifdef USE_FSCANF
{ /* use fscanf() to retrieve and parse a full line */
char vrm[16];

if (fscanf(amiIn,"Asterisk Call Manager/%s\r\n",vrm) == 1)
{
printf("Got introduction: AMI %s\n",vrm);
status = 1;
}
else puts("Cant retrieve VRM");
}
#else
#ifdef USE_FGETS
{ /* use fgets() to retrieve a full line, then sscanf() to parse it */
char buffer[256], vrm[16];

if (fgets(buffer,sizeof buffer,amiIn))
{
if (sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm) == 1)
{
printf("Got introduction: AMI %s\n",vrm);
status = 1;
}
else puts("Cant retrieve VRM");
}
else puts("Cant get introduction");
}
#else
{ /* naive dump of the first 29 bytes of input */
int byte;

for (int count = 0; count < 29; ++count)
{
if ((byte = fgetc(amiIn)) == EOF) break;
byte &= 0x7f;
printf("0x%02.2x %c\n",byte,((byte < 32)||(byte == 127))?' ':byte);
}
status = 1;
}
#endif
#endif
}
fclose(amiIn);
}
else fprintf(stderr,"Cant open AMI connection\n");

return status;
}

int netOpen(char *host, char *port)
{ int Socket;
struct addrinfo hint,
*address;

/* prepare the address */
memset(&hint,0,sizeof hint);
hint.ai_family = AF_INET;
hint.ai_socktype = SOCK_STREAM;
hint.ai_protocol = 0;
hint.ai_flags = 0;

if (getaddrinfo(host,port,&hint,&address) == 0)
{
if ((Socket = socket(AF_INET,SOCK_STREAM,0)) != -1)
{
if (connect(Socket,address->ai_addr,address->ai_addrlen) != 0)
{
close(Socket);
Socket = -1; /* connect() failure */
}
freeaddrinfo(address);
}
}
else Socket = -1; /* getaddrinfo() failure */

return Socket;
}

8<-------------------- main.c -------------------------------------->8

Now for the tests.

First off, a telnet session showing telnet's view of the server interaction.
23:43 $ telnet localhost 5038
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Asterisk Call Manager/7.0.0
^]
telnet> close
Connection closed.
23:43 $

The server sends "Asterisk Call Manager/7.0.0\r\n"

Next, the test program, compiled to dump the first 29 octets received
23:43 $ cc -o test1 main.c
23:44 $ ./test1
0x41 A
0x73 s
0x74 t
0x65 e
0x72 r
0x69 i
0x73 s
0x6b k
0x20
0x43 C
0x61 a
0x6c l
0x6c l
0x20
0x4d M
0x61 a
0x6e n
0x61 a
0x67 g
0x65 e
0x72 r
0x2f /
0x37 7
0x2e .
0x30 0
0x2e .
0x30 0
0x0d
0x0a

OK, so fgetc() can properly read the socket, and confirms the text including
the terminating \r\n combination.

Now, for the fgets()/sscanf() version:
23:44 $ cc -o test2 -DUSE_FGETS main.c
23:44 $ ./test2
Got introduction: AMI 7.0.0

That works as well.

Finally, the fscanf() version:
23:44 $ cc -o test3 -DUSE_FSCANF main.c
23:44 $ ./test3
^C
23:44 $ exit

Which hung. I let it sit for a few seconds, then killed it.

OK, you've seen the code, and you've seen the test results.

So, what did I do wrong in the fscanf() version? Any suggestions?

--
Lew Pitcher
"In Skills We Trust"

Am 13.02.24 um 02:44 schrieb Lew Pitcher:
> For the line read, I decided to try stdio, so I fdopen(,"r") using
> the acquired socket. The next steps are the ones that are currently
> causing me issues.

You do not want to use the buffered FILE API for bidirectional sockets
with message based protocols. This API tries to fill its buffer before
any other processing. And if the other endpoint does not send more data
so far it will block. This is a deadlock if the other endpoint will not
send more unless it gets your response.

Use the low level API (read, write etc.) for sockets. It will simply
return partial data if no more is available for now. Then you can decide
whether you expect more or not.

Typically the best is to use an appropriate library for the protocol you
need.

Marcel

On Tue, 13 Feb 2024 01:44:53 -0000 (UTC), Lew Pitcher wrote:

> For the line read, I decided to try stdio ...

Don’t. Your code is probably reading past the end of the response and
trying to get more.

Also, remember that Asterisk AMI terminates lines with CR/LF.

Sample Python code, from <https://gitlab.com/ldo/seaskirt/>, that
decodes responses properly:

while True :
endpos = self.buff.find(NL + NL)
if endpos >= 0 :
# got at least one complete response
resp = self.buff[:endpos + len(NL)] # include one NL at end
self.buff = self.buff[endpos + 2 * len(NL):]
response = {}
while True :
split, resp = resp.split(NL, 1)
if split != "" :
if split.endswith(":") :
keyword = split[:-1]
value = ""
else :
keyword, value = split.split(": ", 1)
#end if
if keyword in response :
response[keyword] += "\n" + value
else :
response[keyword] = value
#end if
if resp == "" :
break
#end if
#end while
break
#end if
# need more input
if self.EOF :
raise EOFError("Asterisk Manager connection EOF")
#end if
more = await self.sock.recv(IOBUFSIZE, timeout)
if more == None :
# timed out
break
self.buff += more.decode()
if len(more) == 0 :
self.EOF = True
#end if
#end while

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

> I'm trying to write a client for an existing server (an Asterisk
> "Call Manager" server) that converses in text over a bog-standard
> bi-directional TCP connection. I haven't done much sockets
> programming and need some advice. [...]

I've done a fair amount of simple sockets programming. I wouldn't
call myself an expert but I'm certainly not a novice. I have
several bits of advice to offer.

Use read() and write() on any sockets. Avoid stdio.

Do the appropriate calls to make socket I/O non-blocking. At some
point you may want to get fancy and use select() or something
similar, but starting out it's probably good enough to do the
read()s and write()s in a loop, with a sleep for a hundredth of a
second in the loop so the program doesn't burn up cpu cycles.

Put all the calls to esoteric functions like getaddrinfo() in a
separate .c file, so the main body of code can be compiled with
options -std=c?? -pedantic, and only the one special .c file
needs all the special enabling to get non-standard functions.
(IIRC read() and write() do not need that kind of special
treatment, and can be mixed in with ISO conforming code with
no difficulty.)

Also, a general recommendation to avoid fscanf() altogether
and use sscanf() to do whatever scanning needs doing.

Lew Pitcher , dans le message <uqehik$1mdue$4@dont-email.me>, a écrit :
> char vrm[16];
> fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);

And wham the security flaw.

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
> I'm trying to write a client for an existing server (an Asterisk
> "Call Manager" server) that converses in text over a bog-standard
> bi-directional TCP connection. I haven't done much sockets programming
> and need some advice.
>
> I successfully establish my socket connection to the server using
> getaddrinfo() (to build the IPV4 address, socket() (to establish
> an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
> me to read a text line from the socket before commencing my client
> requests.
>
> For the line read, I decided to try stdio, so I fdopen(,"r") using
> the acquired socket. The next steps are the ones that are currently
> causing me issues.

I did some tests with this with the built-in inetd echo server and the
fscanf, regardless of buffering mode the stream has been set to, simply
does another read immediately after the first which returned the
data. As no more data will be received, this other read just blocks
forever. fgets happens to work because the call returns after a complete
line has been read. But AFAIK, that's not mandated behaviour and a
different implementation may well hang in fgets, too.

The basic problem is that you're doing real-time I/O using an
I/O-facility not intended to operate in real-time. If real-time
operation is desired, which is generally the case for all kinds of
request-response protocol operations, you'll need to use read/write or
send/recv.

Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>
>> I'm trying to write a client for an existing server (an Asterisk
>> "Call Manager" server) that converses in text over a bog-standard
>> bi-directional TCP connection. I haven't done much sockets
>> programming and need some advice. [...]
>
> I've done a fair amount of simple sockets programming. I wouldn't
> call myself an expert but I'm certainly not a novice. I have
> several bits of advice to offer.
>
> Use read() and write() on any sockets. Avoid stdio.
>
> Do the appropriate calls to make socket I/O non-blocking.

Non-blocking I/O is supposed to be used in situation where a
single-threaded program (or a single thread of a multi-threaded one)
needs to deal with data from multiple sources/ file descriptors. It's
not some kind of magic fairy dust for socket programming.

> At some
> point you may want to get fancy and use select() or something
> similar, but starting out it's probably good enough to do the
> read()s and write()s in a loop, with a sleep for a hundredth of a
> second in the loop so the program doesn't burn up cpu cycles.

BSD terrorcoding at its finest. Never do anything properly, that's way
to much efforts! But hide the deficienies of your code well, they might
find you out otherwise!

If the program doesn't have anything else to do, it should block waiting
for something to do.

> Put all the calls to esoteric functions like getaddrinfo() in a
> separate .c file, so the main body of code can be compiled with
> options -std=c?? -pedantic, and only the one special .c file
> needs all the special enabling to get non-standard functions.

getaddrinfo is a standard function, just not an ISO-C standard function.

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

> I'm trying to write a client for an existing server (an Asterisk
> "Call Manager" server) that converses in text over a bog-standard
> bi-directional TCP connection. I haven't done much sockets programming
> and need some advice.
>
> I successfully establish my socket connection to the server using
> getaddrinfo() (to build the IPV4 address, socket() (to establish
> an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
> me to read a text line from the socket before commencing my client
> requests.
>
> For the line read, I decided to try stdio, so I fdopen(,"r") using
> the acquired socket. The next steps are the ones that are currently
> causing me issues.
>
> IF I
>
> char vrm[16];
> fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
>
> to extract the AMI version number that follows the text slash, my
> fscanf() hangs.
>
> BUT, if I
>
> char buffer[1024], vrm[16];
> fgets(buffer,sizeof buffer,fd);
> sscanf(buffer,"Asterisk Call Manager/%s\r\n",vrm);
>
> I manage to obtain the appropriate data. A dump of the
> buffer array shows that I did indeed capture the introduction line
> from the server.
>
> So, why the difference in behaviour?

You've had some good advice, but no one has answered the question.

The scanf functions are not intuitive -- you really have to read the
manual carefully. The trouble you are having is that space characters
are directives and are not treated literally. The \r directs fscanf to
read one or more space characters, and it won't stop trying until it
reads a non-space character. That's more reading than you want in a
protocol stream.

Though you might be able to fix it (you could read exactly two single
characters and check that they are \r and \n) it's generally better to
read a response and use sscanf if that suits the application

(The only advice I'd question is that of using non-block I/O by default.
I would be surprised if that helps you in this project, and it might
make things more complicated.)

> Obviously, in the fscanf() version,
> I've not set something up right. Any hints as to what I've done wrong
> would be greatly appreciated.

--
Ben.

On Mon, 12 Feb 2024 23:39:28 -0800, Tim Rentsch wrote:

> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>
>> I'm trying to write a client for an existing server (an Asterisk
>> "Call Manager" server) that converses in text over a bog-standard
>> bi-directional TCP connection. I haven't done much sockets
>> programming and need some advice. [...]
>
> I've done a fair amount of simple sockets programming. I wouldn't
> call myself an expert but I'm certainly not a novice. I have
> several bits of advice to offer.

[snip very good advice]

Thanks, Tim, for the advice; I'll be sure to take it. It matches with
advice that I've seen from other sources, while I've been researching
this little problem.

The general impression that I've got is that stdio on sockets is...
tricky. (in my uniformed opinion, it /shouldn't/ be tricky; it is
very similar to stdio on a terminal device, in that there's no
lookahead, no unget, and must wait for it's input, just like a
terminal stream. But.... stdio on sockets /is/ tricky.

Looks like I'll have to use other means to access this socket stream.

Thanks again for the advice.
--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 12:13:23 +0000, Rainer Weikusat wrote:

> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>> I'm trying to write a client for an existing server (an Asterisk
>> "Call Manager" server) that converses in text over a bog-standard
>> bi-directional TCP connection. I haven't done much sockets programming
>> and need some advice.
>>
>> I successfully establish my socket connection to the server using
>> getaddrinfo() (to build the IPV4 address, socket() (to establish
>> an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
>> me to read a text line from the socket before commencing my client
>> requests.
>>
>> For the line read, I decided to try stdio, so I fdopen(,"r") using
>> the acquired socket. The next steps are the ones that are currently
>> causing me issues.
>
> I did some tests with this with the built-in inetd echo server and the
> fscanf, regardless of buffering mode the stream has been set to, simply
> does another read immediately after the first which returned the
> data. As no more data will be received, this other read just blocks
> forever. fgets happens to work because the call returns after a complete
> line has been read. But AFAIK, that's not mandated behaviour and a
> different implementation may well hang in fgets, too.
>
> The basic problem is that you're doing real-time I/O using an
> I/O-facility not intended to operate in real-time. If real-time
> operation is desired, which is generally the case for all kinds of
> request-response protocol operations, you'll need to use read/write or
> send/recv.

Thanks, Rainer

As I suspected, stdio on sockets is tricky, and not the correct interface
to use. But, I had to try :-)

Thanks again for your help
--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 07:05:03 +0000, Lawrence D'Oliveiro wrote:

> On Tue, 13 Feb 2024 01:44:53 -0000 (UTC), Lew Pitcher wrote:
>
>> For the line read, I decided to try stdio ...
>
> Don’t. Your code is probably reading past the end of the response and
> trying to get more.

Thanks, Lawrence

That's what I suspect, as well.

> Also, remember that Asterisk AMI terminates lines with CR/LF.

I know this all too well. But, thanks for the reminder

> Sample Python code, from <https://gitlab.com/ldo/seaskirt/>, that
> decodes responses properly:
[snip python code]

Thanks for that sample; I'll spend some time getting to know it.

Thanks again for your advice and help
--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 06:59:21 +0100, Marcel Mueller wrote:

> Am 13.02.24 um 02:44 schrieb Lew Pitcher:
>> For the line read, I decided to try stdio, so I fdopen(,"r") using
>> the acquired socket. The next steps are the ones that are currently
>> causing me issues.
>
> You do not want to use the buffered FILE API for bidirectional sockets
> with message based protocols.
[snip]
> Typically the best is to use an appropriate library for the protocol you
> need.

Thanks, Marcel, for the advice.

However, outside of a couple of out of maintenance, 3rd-party projects from
Asterisk users trying to do the same thing as I am, there's /no/ official
"library for the protocol" available, that I can find.

So, I either adapt one of those 3rd-party projects to my needs, or I roll
my own. And, since I need something to do, and something to learn, I decided
to roll my own protocol. :-)

Thanks again for the advice.
--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:

> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>
>> I'm trying to write a client for an existing server (an Asterisk
>> "Call Manager" server) that converses in text over a bog-standard
>> bi-directional TCP connection. I haven't done much sockets programming
>> and need some advice.
>>
>> I successfully establish my socket connection to the server using
>> getaddrinfo() (to build the IPV4 address, socket() (to establish
>> an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
>> me to read a text line from the socket before commencing my client
>> requests.
>>
>> For the line read, I decided to try stdio, so I fdopen(,"r") using
>> the acquired socket. The next steps are the ones that are currently
>> causing me issues.
>>
>> IF I
>>
>> char vrm[16];
>> fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
>>
>> to extract the AMI version number that follows the text slash, my
>> fscanf() hangs.
[snip]
>> So, why the difference in behaviour?
>
> You've had some good advice, but no one has answered the question.
>
> The scanf functions are not intuitive -- you really have to read the
> manual carefully. The trouble you are having is that space characters
> are directives and are not treated literally. The \r directs fscanf to
> read one or more space characters, and it won't stop trying until it
> reads a non-space character. That's more reading than you want in a
> protocol stream.

AHA! That makes sense. Thanks, Ben, for the clarification. I had forgotten
that fscanf() treats \r and \n as whitespace. And, that explains why the
sscanf() doen't fail; it encounters the end-of-string and treats it as
the equivalent of an EOF or non-space character. And so, doesn't hang.

> Though you might be able to fix it (you could read exactly two single
> characters and check that they are \r and \n) it's generally better to
> read a response and use sscanf if that suits the application

Again, that makes sense. A good suggestion.

> (The only advice I'd question is that of using non-block I/O by default.
> I would be surprised if that helps you in this project, and it might
> make things more complicated.)

I've done less programming with non-blocking I/O than I've done with
sockets. I have heard the advice to use non-blocking I/O, and will
consider it, but I don't want to take on too much in one shot. So,
for now, I'll stick with blocking I/O and a foreknowledge of what
is (or should be) coming down the wire. To me, at this time,
non-blocking I/O is like adding a regex to the mix ("And, now you
have /two/ problems." :-) )

Thanks for the advice. It certainly explains the differences between
the three trial programs, and gives me something to think about wrt
how I handle this sockets I/O.

--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 08:08:11 +0000, Nicolas George wrote:

> Lew Pitcher , dans le message <uqehik$1mdue$4@dont-email.me>, a écrit :
>> char vrm[16];
>> fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
>
> And wham the security flaw.

I suspect that you mean "buffer overflow" flaw.

Yes, in the wild, it would be.

But, this is an experiment (acknowledged as an experiment) under
laboratory conditions, where I /know/that the buffer is big enough
for the data (the string extracted will, in this case, /always/ be
"7.0.0", 6 octets, including the end-of-string character).

I will, however, fix this if and when my experiment grows into
something more useful.

Thanks for the catch.
--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 13:54:22 +0000, Lew Pitcher wrote:

> On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
>
>> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>>
>>> I'm trying to write a client for an existing server (an Asterisk
>>> "Call Manager" server) that converses in text over a bog-standard
>>> bi-directional TCP connection. I haven't done much sockets programming
>>> and need some advice.
>>>
>>> I successfully establish my socket connection to the server using
>>> getaddrinfo() (to build the IPV4 address, socket() (to establish
>>> an AF_INET,SOCK_STREAM socket) and connect(). The protocol requires
>>> me to read a text line from the socket before commencing my client
>>> requests.
>>>
>>> For the line read, I decided to try stdio, so I fdopen(,"r") using
>>> the acquired socket. The next steps are the ones that are currently
>>> causing me issues.
>>>
>>> IF I
>>>
>>> char vrm[16];
>>> fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
>>>
>>> to extract the AMI version number that follows the text slash, my
>>> fscanf() hangs.
> [snip]
>>> So, why the difference in behaviour?
>>
>> You've had some good advice, but no one has answered the question.
>>
>> The scanf functions are not intuitive -- you really have to read the
>> manual carefully. The trouble you are having is that space characters
>> are directives and are not treated literally. The \r directs fscanf to
>> read one or more space characters, and it won't stop trying until it
>> reads a non-space character. That's more reading than you want in a
>> protocol stream.
>
> AHA! That makes sense. Thanks, Ben, for the clarification. I had forgotten
> that fscanf() treats \r and \n as whitespace. And, that explains why the
> sscanf() doen't fail; it encounters the end-of-string and treats it as
> the equivalent of an EOF or non-space character. And so, doesn't hang.

And BINGO, when I change the line from
fscanf(amiIn,"Asterisk Call Manager/%s\r\n",vrm)
to
fscanf(amiIn,"Asterisk Call Manager/%s",vrm)
I get the expected results, and confirm Ben's observation.

Thanks again, Ben
That solved my puzzle.

And, now on to reworking my design, based on the advice I've seen here.

Thanks, all, for the help.
--
Lew Pitcher
"In Skills We Trust"

Ben Bacarisse <ben.usenet@bsb.me.uk> writes:
> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

[...]

>> IF I
>>
>> char vrm[16];
>> fscanf(fd,"Asterisk Call Manager/%s\r\n",vrm);
>>
>> to extract the AMI version number that follows the text slash, my
>> fscanf() hangs.

[...]

> The scanf functions are not intuitive -- you really have to read the
> manual carefully. The trouble you are having is that space characters
> are directives and are not treated literally. The \r directs fscanf to
> read one or more space characters, and it won't stop trying until it
> reads a non-space character.

This is specifically demanded by the UNIX specification:

,----
| A directive composed of one or more white-space characters shall be
| executed by reading input until no more valid input can be read, or up
| to the first byte which is not a white-space character, which remains
| unread.
`----

OTOH, the stream may be fully buffered or line buffered (unspecified)
and - AFAICT - no specific behaviour is manadated for input from either
a fully buffered or a line buffered stream. Eg, the implementation is
free to do readahead to whatever degree someone considers useful. Even
for an unbuffered stream, it's just said that

,----
| bytes are intended to appear from the source or at the destination as
| soon as possible;
`----

which means setting a stream to unbuffered is basically a hint to the
implementation: If that's deemed possible, please send the data now or, for
input, return as soon as enough the call can complete successfully and
the implementation is totally free to take a Kinderwille ist Dreck!¹
stance on that.

¹ German for "Children may wish for anything. Doesn't mean they'll get
it!"

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
> On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
>> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:

[...]

>> (The only advice I'd question is that of using non-block I/O by default.
>> I would be surprised if that helps you in this project, and it might
>> make things more complicated.)
>
> I've done less programming with non-blocking I/O than I've done with
> sockets. I have heard the advice to use non-blocking I/O, and will
> consider it, but I don't want to take on too much in one shot.

The idea behind non-blocking I/O is usually that of a single-threaded
program (or a single thread of a program) which needs to handle
real-time input¹ on more than one file descriptor and thus, cannot block
waiting for input on one of them as data might arrive at one of the
others first. Hence, it'll set all to non-blocking and then blocks
select/ poll/ epoll etc so that it gets notified when input is available
on any of them.

A somewhat simpler use case is I/O on only one file descriptor but with
a need to impose timeouts to cope with unreliable communication
partners, although there are other options available for this (like
alarm signals or socket read timeouts).

¹ Principally also to avoid blocking for buffer space to become
available to complete an output operations but that's a rare case.

Lew Pitcher , dans le message <uqfsgu$23ni8$6@dont-email.me>, a écrit :
> But, this is an experiment (acknowledged as an experiment) under
> laboratory conditions

You posted it on Usenet.

On Tue, 13 Feb 2024 16:12:01 +0000, Rainer Weikusat wrote:

> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>> On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
>>> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>
> [...]
>
>>> (The only advice I'd question is that of using non-block I/O by default.
>>> I would be surprised if that helps you in this project, and it might
>>> make things more complicated.)
>>
>> I've done less programming with non-blocking I/O than I've done with
>> sockets. I have heard the advice to use non-blocking I/O, and will
>> consider it, but I don't want to take on too much in one shot.
>
> The idea behind non-blocking I/O is usually that of a single-threaded
> program (or a single thread of a program) which needs to handle
> real-time input¹ on more than one file descriptor and thus, cannot block
> waiting for input on one of them as data might arrive at one of the
> others first.
[snip]
> A somewhat simpler use case is I/O on only one file descriptor but with
> a need to impose timeouts to cope with unreliable communication
> partners
[snip]

For this mini-project, both client and server live on the same machine,
and communicate through the loopback ("localhost"). There won't be
any significant latency issues or other communications interference.

I'm only concerned with writing the client, and it will only interact
over one channel in a simple request/reply type protocol. There doesn't
seem to be a pressing need for nonblocking I/O.

But, as it has been suggested here, I will look into nonblocking I/O.
Perhaps I'm missing something that others have seen.

--
Lew Pitcher
"In Skills We Trust"

On Tue, 13 Feb 2024 16:13:05 +0000, Nicolas George wrote:

> Lew Pitcher , dans le message <uqfsgu$23ni8$6@dont-email.me>, a écrit :
>> But, this is an experiment (acknowledged as an experiment) under
>> laboratory conditions
>
> You posted it on Usenet.

Indeed, I did. And your point is?

--
Lew Pitcher
"In Skills We Trust"

Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
> On Tue, 13 Feb 2024 16:12:01 +0000, Rainer Weikusat wrote:
>
>> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>>> On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:
>>>> Lew Pitcher <lew.pitcher@digitalfreehold.ca> writes:
>>
>> [...]
>>
>>>> (The only advice I'd question is that of using non-block I/O by default.
>>>> I would be surprised if that helps you in this project, and it might
>>>> make things more complicated.)
>>>
>>> I've done less programming with non-blocking I/O than I've done with
>>> sockets. I have heard the advice to use non-blocking I/O, and will
>>> consider it, but I don't want to take on too much in one shot.
>>
>> The idea behind non-blocking I/O is usually that of a single-threaded
>> program (or a single thread of a program) which needs to handle
>> real-time input¹ on more than one file descriptor and thus, cannot block
>> waiting for input on one of them as data might arrive at one of the
>> others first.
> [snip]
>> A somewhat simpler use case is I/O on only one file descriptor but with
>> a need to impose timeouts to cope with unreliable communication
>> partners
> [snip]
>
> For this mini-project, both client and server live on the same machine,
> and communicate through the loopback ("localhost"). There won't be
> any significant latency issues or other communications interference.
>
> I'm only concerned with writing the client, and it will only interact
> over one channel in a simple request/reply type protocol. There doesn't
> seem to be a pressing need for nonblocking I/O.
>
> But, as it has been suggested here, I will look into nonblocking I/O.
> Perhaps I'm missing something that others have seen.

If you don't want to implement I/O timeouts, non-blocking I/O is uselees
for your case. I think it's often just being (mis-)used because it exists and
the idea of a program which just waits until woken up by the kernel
instead of Doing Someting[tm] make many people nervous.

On Tue, 13 Feb 2024 12:54:23 +0000, Ben Bacarisse wrote:

> (The only advice I'd question is that of using non-block I/O by default.
> I would be surprised if that helps you in this project, and it might
> make things more complicated.)

This is why we have the “async” paradigm, a.k.a. the resurgence of
coroutines. It leads to fewer race conditions than threads, and works well
where the bottleneck is network I/O (or, in a GUI app, the user
interaction), rather than the CPU.

On Tue, 13 Feb 2024 13:45:36 -0000 (UTC), Lew Pitcher wrote:

> However, outside of a couple of out of maintenance, 3rd-party projects
> from Asterisk users trying to do the same thing as I am, there's /no/
> official "library for the protocol" available, that I can find.

My Python library, Seaskirt (link posted elsewhere) is the only one I know
of that covers AMI, AGI, Async-AGI, ARI and even the console interface. It
offers both synchronous and asynchronous (async/await) versions of all the
main API classes, and even allows for SSL/TLS connections where Asterisk
will accept these.

On Tue, 13 Feb 2024 20:35:54 +0000, Lawrence D'Oliveiro wrote:

> On Tue, 13 Feb 2024 13:45:36 -0000 (UTC), Lew Pitcher wrote:
>
>> However, outside of a couple of out of maintenance, 3rd-party projects
>> from Asterisk users trying to do the same thing as I am, there's /no/
>> official "library for the protocol" available, that I can find.
>
> My Python library, Seaskirt (link posted elsewhere) is the only one I know
> of that covers AMI, AGI, Async-AGI, ARI and even the console interface. It
> offers both synchronous and asynchronous (async/await) versions of all the
> main API classes, and even allows for SSL/TLS connections where Asterisk
> will accept these.

I will check it out. Thanks

--
Lew Pitcher
"In Skills We Trust"