Message-ID:

The first sign of maturity is the discovery that the volume knob also turns to the left.

devel / comp.lang.python / Data unchanged when passing data to Python in multiprocessing shared memory

Data unchanged when passing data to Python in multiprocessing shared memory

<mailman.8.1643747049.2976.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=16841&group=comp.lang.python#16841

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: jenk...@tutanota.com (Jen Kris)
Newsgroups: comp.lang.python
Subject: Data unchanged when passing data to Python in multiprocessing
shared memory
Date: Tue, 1 Feb 2022 21:14:31 +0100 (CET)
Lines: 52
Message-ID: <mailman.8.1643747049.2976.python-list@python.org>
References: <Muqv9er--3-2@tutanota.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de NxHFcCn1nAKOHv6QdOFUBg2rPP5tCENtDmtKXZMXeUtw==
Return-Path: <jenkris@tutanota.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=tutanota.com header.i=@tutanota.com header.b=pLvy82s8;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.045
X-Spam-Evidence: '*H*': 0.91; '*S*': 0.00; 'python?': 0.03;
'subject:when': 0.07; 'python.': 0.08; '8bit%:3': 0.09; 'memory.':
0.09; 'subject:Python': 0.12; 'memory': 0.15; 'char': 0.16;
'integer': 0.16; 'language?': 0.16; 'protocol.': 0.16;
'subject:shared': 0.16; 'subject:skip:m 10': 0.16; 'unsigned':
0.16; 'want,': 0.16; 'python': 0.16; "can't": 0.17; 'uses': 0.19;
'to:addr:python-list': 0.20; 'language': 0.21; 'code': 0.23;
'received:de': 0.23; 'binary': 0.26; 'program': 0.31; 'question':
0.32; 'received:192.168.1': 0.32; 'but': 0.32; 'following': 0.35;
'thanks,': 0.36; 'using': 0.37; 'received:192.168': 0.37; 'read':
0.38; 'text': 0.39; 'shared': 0.39; 'four': 0.60; 'skip:\xc2 10':
0.62; 'skip:i 20': 0.62; 'send': 0.63; 'between': 0.63; 'pass':
0.64; 'verify': 0.64; '&': 0.65; 'back': 0.67; 'exactly':
0.68; 'refers': 0.69; 'subject:Data': 0.71; '8bit%:100': 0.76;
'points': 0.84; 'naturally': 0.84; 'pointer': 0.84; 'subject: \n
': 0.84; 'subject:memory': 0.84; 'want.': 0.84
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1643746471;
s=s1; d=tutanota.com;
h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Date:Date:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:Sender;
bh=rFIj8dUu7jCxfZeiWPC1f6EQRm9GLedL/UysGUyZsac=;
b=pLvy82s8Upkf0lOX2otxj8uPJGoScKklKNk6Ics/GKtVdIoq0C7YY6BQ5J5YPz9g
aqJhiGhcjuZ6FrYUkIx8BsFEX79rZHeswXjqyNTMwacK/t1aKOPUJK29yoQnXxRe3M9
saBqPmrzf0f9e94GZQwJNnSX5Q/qwlFBhQIxRVanJC5lOVXj87phklSH/VLCqQDFypr
HFnVtP3EWoHessWtj76ZMuvR8w3bdCE0LHiA3W1BiuRK4LRbhzVbzM9tcuAdv3RGSQF
nyUMnGFCzsJdX/nadrrI0vMmH95IPcUO/HYoK/fiDb9Aqu8NajT+2GehCh60pfxA4EZ
lqWXtEa+gA==
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <Muqv9er--3-2@tutanota.com>

by: Jen Kris - Tue, 1 Feb 2022 20:14 UTC

I am using multiprocesssing.shared_memory to pass data between NASM and Python. The shared memory is created in NASM before Python is called. Python connects to the shm: shm_00 = shared_memory.SharedMemory(name='shm_object_00',create=False).

I have used shared memory at other points in this project to pass text data from Python back to NASM with no problems. But now this time I need to pass a 32-bit integer (specifically 32,894) from NASM to Python.

First I convert the integer to bytes in a C program linked into NASM:

unsigned char bytes[4]
unsigned long int_to_convert = 32894;

    bytes[0] = (int_to_convert >> 24) & 0xFF;
    bytes[1] = (int_to_convert >> 16) & 0xFF;
    bytes[2] = (int_to_convert >> 8) & 0xFF;
    bytes[3] = int_to_convert & 0xFF;
    memcpy(outbuf, bytes, 4);

where outbuf is a pointer to the shared memory. On return from C to NASM, I verify that the first four bytes of the shared memory contain what I want, and they are 0, 0, -128, 126 which is binary 00000000 00000000 10000000 01111110, and that's correct (32,894).

Next I send a message to Python through a FIFO to read the data from shared memory. Python uses the following code to read the first four bytes of the shared memory:

        byte_val = shm_00.buf[:4]
        print(shm_00.buf[0])
        print(shm_00.buf[1])
        print(shm_00.buf[2])
        print(shm_00.buf[3])

But the bytes show as 40 39 96 96, which is exactly what the first four bytes of this shared memory contained before I called C to overwrite them with the bytes 0, 0, -128, 126. So Python does not see the updated bytes, and naturally int.from_bytes(byte_val, "little") does not return the result I want.

I know that Python refers to shm00.buf, using the buffer protocol. Is that the reason that Python can't see the data that has been updated by another language?

So my question is, how can I alter the data in shared memory in a non-Python language to pass back to Python?

Thanks,

Jen

Subject	Author
Data unchanged when passing data to Python in multiprocessing	Jen Kris