Is there any special logic within tcl, when stdin & -out have
been closed? maybe, if next command is "exec" ?

I've got a tcl-script that gets started with stdin and stdout
connected to a socket.

Under some circumstances, these sockets get closed, and at that
point the script is only supposed to [exec ...] a (shortly
running) command, and then exit.

But instead, it keeps "hanging" after the exec.

Running it with "truss" shows closing of stdin & -out,
and immediately afterwards a pipe is created, and some
flag is set for them:

55572: 33847: close(0) = 0
55572: 33847: close(1) = 0

not sure if next 3 lines are aftermath of close, or part of [exec]

55572: 33847: pipe(0x2FF21FD8) = 0
55572: 33847: kfcntl(0, F_SETFD, 0x00000001) = 0
55572: 33847: kfcntl(1, F_SETFD, 0x00000001) = 0

.... [exec ...]: kfork, childprocess does: ..., execve, ..., _exit(0)

and then the main process goes on to:

55572: 33847: kread(0, 0x205D2324, 4096) (sleeping...)

Reading from a pipe, whose write-side is kept by the same process.

I don't know where the read on stdin comes from. According to
the script it should do an "exit" right after the "exec".

So far, I cannot yet tell, if this is a "heisen-" or "bohr-" bug.
I see it happens a couple of times, but I don't get to see the
cases where it doesn't happen, if those exist.

Andreas Leitgeb <avl@logic.at> wrote:
> Is there any special logic within tcl, when stdin & -out have
> been closed? maybe, if next command is "exec" ?

I can now reproduce it on linux (originally it showed on AIX, but
apparently isn't specific to it)

simple Test-Script:
close stdin; close stdout
exec true
exit

The strace-output starting from the "close"s:

close(0) = 0
close(1) = 0
pipe([0, 1]) = 0
fcntl(0, F_SETFD, FD_CLOEXEC) = 0
fcntl(1, F_SETFD, FD_CLOEXEC) = 0
stat("/tmp", {st_mode=S_IFDIR|S_ISVTX|0777, st_size=12288, ...}) = 0
access("/tmp", W_OK) = 0
getpid() = 4239
openat(AT_FDCWD, "/tmp/tcl_kEjgNd", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
unlink("/tmp/tcl_kEjgNd") = 0
fcntl(3, F_SETFD, FD_CLOEXEC) = 0
pipe([4, 5]) = 0
fcntl(4, F_SETFD, FD_CLOEXEC) = 0
fcntl(5, F_SETFD, FD_CLOEXEC) = 0
clone(child_stack=NULL,
flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD,
child_tidptr=0x7f8497acda10) = 4240
close(5) = 0
read(4, "", 223) = 0
close(4) = 0
read(0, 0x561f2dd4e4f8, 4096) = ? ERESTARTSYS (To be restarted
if SA_RESTART is set)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=4240,
si_uid=1000, si_status=0, si_utime=0, si_stime=0} ---
read(0,

The strace output on linux gives some more information, than truss on
AIX, namely that the main process starts reading the pipe even before
the child-process' exit is reported as SIGCHLD.

If I comment out the initial "close"s - any(!) of them - then all is fine:
the pipe gets created on higher fds, and the "read" that previously hung,
now immediately returns "". Then the pipe gets closed, and the exit done.

So, the question is no longer "Why is it reading from the pipe?"
but:
Why is there *no* other thread writing to it,
exactly if *both* stdin&stdout were previously closed?"

Also, I've now found a workaround: If I open /dev/null right after the
two closes, then the rest appears to work again.

After some more experiments, I've now created this ticket:

https://core.tcl-lang.org/tcl/tktview/21dbc600ae0a005a7f879a05580550a8da257b14

so, this thread here is now obsolete.

Andreas Leitgeb <avl@logic.at> wrote:
> Andreas Leitgeb <avl@logic.at> wrote:
>> Is there any special logic within tcl, when stdin & -out have
>> been closed? maybe, if next command is "exec" ?
> I can now reproduce it on linux (originally it showed on AIX, but
> apparently isn't specific to it)
> simple Test-Script:
> close stdin; close stdout
> exec true
> [...]

On 12/6/2022 6:59 AM, Andreas Leitgeb wrote:
> After some more experiments, I've now created this ticket:
>
> https://core.tcl-lang.org/tcl/tktview/21dbc600ae0a005a7f879a05580550a8da257b14
>
> so, this thread here is now obsolete.
>

I wonder if this is really a "bug". From the documentation page for exec:

> If standard input is not redirected with “<”, “<<” or “<@” then
> the standard input for the first command in the pipeline is taken
> from the application's current standard input.

You have closed both stdin and stdout; but exec needs those as well as
sterr. The above paragraph also explains the work-around you have
discovered: you are simply redirecting it to somewhere else.

saitology9 <saitology9@gmail.com> wrote:
> On 12/6/2022 6:59 AM, Andreas Leitgeb wrote:
>> After some more experiments, I've now created this ticket:
>>
>> https://core.tcl-lang.org/tcl/tktview/21dbc600ae0a005a7f879a05580550a8da257b14
>> so, this thread here is now obsolete.
>
> I wonder if this is really a "bug". From the documentation page for exec:
>
>> If standard input is not redirected with “<”, “<<” or “<@” then
>> the standard input for the first command in the pipeline is taken
>> from the application's current standard input.
>
> You have closed both stdin and stdout; but exec needs those as well as
> sterr. The above paragraph also explains the work-around you have
> discovered: you are simply redirecting it to somewhere else.
>

Good point, but then why does it also depend on stdout having been
closed as well, to trigger that behaviour?

If I merely close stdin, or close both and reopen stdin from /dev/null
no hang happens.

Neither, if I close stdout, open /dev/null, then close stdin: no hang.

In these cases, missing stdin but not missing stdout, the child process
gets started fine without an fd 0, so exec doesn't really seem to depend
specifically on a valid stdin for starting new processes.

It really seems like there is some hardcoded special case: if the pipe
returns the endpoint-fds as [0, 1], then do something "unoptimal."

On 12/6/2022 1:12 PM, Andreas Leitgeb wrote:
>
> Good point, but then why does it also depend on stdout having been
> closed as well, to trigger that behaviour?
>
> If I merely close stdin, or close both and reopen stdin from /dev/null
> no hang happens.
>
> Neither, if I close stdout, open /dev/null, then close stdin: no hang.
>
> In these cases, missing stdin but not missing stdout, the child process
> gets started fine without an fd 0, so exec doesn't really seem to depend
> specifically on a valid stdin for starting new processes.
>
> It really seems like there is some hardcoded special case: if the pipe
> returns the endpoint-fds as [0, 1], then do something "unoptimal."
>

It could be; I am not sure. I always thought of exec as starting a new
program, where tcl takes care of some of the setup including the
input/output channels.

The man page says this for stdout:

> If standard output has not been redirected then the exec command returns
> the standard output from the last command in the pipeline, unless “2>@1”
> was specified, in which case standard error is included as well.

Notice the phrase "stdout of the last pipeline command". So, I would
assume that exec needs stdin to start things off (which makes sense),
and stdout only on demand when it is needed. Perhaps in your specific
case, there is a single exec'ed command and it doesn't use stdout at all.

Howdy Andreas,

interesting problem indeed. The cause seems to be that the write end of the initial pipe becomes stdout (fd=1) and is never closed.

The following snippet added right after pipe() in TclpCreatePipe() and similarly in Tcl_CreatePipe() might remedy this problem be crossing out the stdout (fd=1) case.

if (pipeIds[1] == 1) {
int newId = dup(pipeIds[1]);

if (newId > pipeIds[1]) {
close(pipeIds[1]);
pipeIds[1] = newId;
} else {
close(pipeIds[0]);
close(pipeIds[1]);
errno = EMFILE;
return 0;
}
}

BR,
Christian