Message-ID:

Just go with the flow control, roll with the crunches, and, when you get a prompt, type like hell.

devel / comp.lang.python / Re: Python child process in while True loop blocks parent

Re: Python child process in while True loop blocks parent

<mailman.62.1638225113.9869.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=16284&group=comp.lang.python#16284

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: jenk...@tutanota.com (Jen Kris)
Newsgroups: comp.lang.python
Subject: Re: Python child process in while True loop blocks parent
Date: Mon, 29 Nov 2021 23:31:51 +0100 (CET)
Lines: 111
Message-ID: <mailman.62.1638225113.9869.python-list@python.org>
References: <MphOu5i--3-2@tutanota.com>
<B05343F2-30B5-470C-8CFC-EBFA7C5EFD6A@barrys-emacs.org>
<MphoqOf--3-2@tutanota.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de MPHQ7BjEQQJawJ78XyTGBQfF5Knl8IrN4+DoTrM9eT1A==
Return-Path: <jenkris@tutanota.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=tutanota.com header.i=@tutanota.com header.b=zKpZwCKE;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.004
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; 'this:': 0.03; 'yet.':
0.04; 'loop': 0.07; 'parent': 0.07; 'blocking': 0.09; 'cc:addr
:python-list': 0.09; 'child': 0.09; 'ideas.': 0.09; 'set,': 0.09;
'subject:Python': 0.12; 'import': 0.15; 'url:mailman': 0.15;
'affects': 0.16; 'barry': 0.16; 'cameron': 0.16; 'cc:name:python
list': 0.16; 'default.': 0.16; 'events:': 0.16; 'length.': 0.16;
'looping': 0.16; 'pipe': 0.16; 'subject:True': 0.16;
'subject:child': 0.16; 'subject:loop': 0.16; 'subject:parent':
0.16; 'subject:while': 0.16; 'writes': 0.16; 'wrote:': 0.16;
'python': 0.16; 'uses': 0.19; 'libraries': 0.19;
'cc:addr:python.org': 0.20; 'code': 0.23; 'received:de': 0.23;
'url-ip:188.166.95.178/32': 0.25; 'url-ip:188.166.95/24': 0.25;
'url:listinfo': 0.25; 'cc:2**0': 0.25; 'url-ip:188.166/16': 0.25;
'perform': 0.26; 'email addr:python.org>': 0.28; 'example,':
0.28; 'suggest': 0.28; 'it,': 0.29; 'putting': 0.31; 'url-
ip:188/8': 0.31; 'program': 0.31; 'question': 0.32; 'assume':
0.32; 'end.': 0.32; 'python-list': 0.32; 'signal': 0.32;
'received:192.168.1': 0.32; 'but': 0.32; "i'm": 0.33; 'someone':
0.34; 'header:In-Reply-To:1': 0.34; 'running': 0.34; 'functions':
0.36; 'source': 0.36; 'received:192.168': 0.37; 'way': 0.38;
'could': 0.38; 'put': 0.38; 'read': 0.38; '8bit%:14': 0.38;
'thanks': 0.38; 'handle': 0.39; 'case.': 0.40; 'program.': 0.40;
'event': 0.40; 'best': 0.61; 'likely': 0.61; 'here.': 0.61;
'skip:o 10': 0.61; 'skip:o 20': 0.63; 'true': 0.63; 'events':
0.63; 'your': 0.64; 'plan': 0.64; 'process.': 0.65; 'similar':
0.65; 'named': 0.65; 'skip:e 20': 0.67; 'right': 0.68; 'skip:#
10': 0.68; 'tip': 0.69; 'communicates': 0.84; 'email name:&lt
;python-list': 0.84; 'parent.': 0.84; 'pipes': 0.84; 'true:':
0.84; 'receives': 0.91; 'worry': 0.95
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1638225111;
s=s1; d=tutanota.com;
h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender;
bh=642MxYMpZ7ZmdIOh+PfecJrEA75LKRUenKZb2FSILho=;
b=zKpZwCKEQZbW3fqiOZ/uOMvZS6mYOZzRNNdZ/nnPpw27CXoCPD53JgZ4XFzvlUD1
k8kHsqBOQaCesRWygZA5K0ugtpoCAZaHvTA6CWxJltD1xTEX1vEXAtRDVBXG3KOoNUM
UflCAJw1i6wCjyOG6xnXNU+YwDvJYuAbmpoLvB4a3OcmAq0AprEA0+u0/CauwdwgP9R
onV3+A4kPnGqATYItJKLpGLw4R2f6kNHnOlGnWFRRxzxKwPtfP+kdkFOQHcAQUCwZd6
nFkGRj6axoGsqh5z/rsN+XJq6YcNQSmiM7TI62t1P4HcdlHOdRX2ZdoPsc+197YGA/C
za3lwNm+Gw==
In-Reply-To: <B05343F2-30B5-470C-8CFC-EBFA7C5EFD6A@barrys-emacs.org>
X-Content-Filtered-By: Mailman/MimeDel 2.1.37
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.37
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <MphoqOf--3-2@tutanota.com>
X-Mailman-Original-References: <MphOu5i--3-2@tutanota.com>
<B05343F2-30B5-470C-8CFC-EBFA7C5EFD6A@barrys-emacs.org>

by: Jen Kris - Mon, 29 Nov 2021 22:31 UTC

Thanks to you and Cameron for your replies. The C side has an epoll_ctl set, but no event loop to handle it yet. I'm putting that in now with a pipe write in Python-- as Cameron pointed out that is the likely source of blocking on C. The pipes are opened as rdwr in Python because that's nonblocking by default. The child will become more complex, but not in a way that affects polling. And thanks for the tip about the c-string termination.

Nov 29, 2021, 14:12 by barry@barrys-emacs.org:

>
>
>> On 29 Nov 2021, at 20:36, Jen Kris via Python-list <python-list@python.org> wrote:
>>
>> I have a C program that forks to create a child process and uses execv to call a Python program. The Python program communicates with the parent process (in C) through a FIFO pipe monitored with epoll().
>>
>> The Python child process is in a while True loop, which is intended to keep it running while the parent process proceeds, and perform functions for the C program only at intervals when the parent sends data to the child -- similar to a daemon process.
>>
>> The C process writes to its end of the pipe and the child process reads it, but then the child process continues to loop, thereby blocking the parent.
>>
>> This is the Python code:
>>
>> #!/usr/bin/python3
>> import os
>> import select
>>
>> #Open the named pipes
>> pr = os.open('/tmp/Pipe_01', os.O_RDWR)
>>
> Why open rdwr if you are only going to read the pipe?
>
>> pw = os.open('/tmp/Pipe_02', os.O_RDWR)
>>
> Only need to open for write.
>
>>
>> ep = select.epoll(-1)
>> ep.register(pr, select.EPOLLIN)
>>
>
> Is the only thing that the child does this:
> 1. Read message from pr
> 2. Process message
> 3. Write result to pw.
> 4. Loop from 1
>
> If so as Cameron said you do not need to worry about the poll.
> Do you plan for the child to become more complex?
>
>>
>> while True:
>>
>> events = ep.poll(timeout=2.5, maxevents=-1)
>> #events = ep.poll(timeout=None, maxevents=-1)
>>
>> print("child is looping")
>>
>> for fileno, event in events:
>> print("Python fileno")
>> print(fileno)
>> print("Python event")
>> print(event)
>> v = os.read(pr,64)
>> print("Pipe value")
>> print(v)
>>
>> The child process correctly receives the signal from ep.poll and correctly reads the data in the pipe, but then it continues looping. For example, when I put in a timeout:
>>
>> child is looping
>> Python fileno
>> 4
>> Python event
>> 1
>> Pipe value
>> b'10\x00'
>>
> The C code does not need to write a 0 bytes at the end.
> I assume the 0 is from the end of a C string.
> UDS messages have a length.
> In the C just write 2 byes in the case.
>
> Barry
>
>> child is looping
>> child is looping
>>
>> That suggests that a while True loop is not the right thing to do in this case. My question is, what type of process loop is best for this situation? The multiprocessing, asyncio and subprocess libraries are very extensive, and it would help if someone could suggest the best alternative for what I am doing here.
>>
>> Thanks very much for any ideas.
>>
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>>

Subject	Author
Re: Python child process in while True loop blocks parent	Jen Kris