Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

6 May, 2024: The networking issue during the past two days has been identified and appears to be fixed. Will keep monitoring.


devel / comp.lang.python / Re: PyObject_CallFunctionObjArgs segfaults

SubjectAuthor
o Re: PyObject_CallFunctionObjArgs segfaultsJen Kris

1
Re: PyObject_CallFunctionObjArgs segfaults

<mailman.502.1664491301.20444.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=19611&group=comp.lang.python#19611

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: jenk...@tutanota.com (Jen Kris)
Newsgroups: comp.lang.python
Subject: Re: PyObject_CallFunctionObjArgs segfaults
Date: Fri, 30 Sep 2022 00:41:39 +0200 (CEST)
Lines: 301
Message-ID: <mailman.502.1664491301.20444.python-list@python.org>
References: <ND8xHA---3-2@tutanota.com>
<fbd7fcc1-6095-e381-4721-bc5c09975202@mrabarnett.plus.com>
<ND9prTQ--3-2@tutanota.com-ND9qgBV----2> <ND9zJoA--7-2@tutanota.com>
<b60d1d3b-1381-8646-fd43-15496bf993dd@mrabarnett.plus.com>
<NDAPOvd--3-2@tutanota.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de qax30fej+0MSQQZFrXHPTgzNG3AhHDItgDXrVonpQtcQ==
Return-Path: <jenkris@tutanota.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=tutanota.com header.i=@tutanota.com header.b=qdLZIbTa;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'comments': 0.03;
'argument': 0.04; '3.8': 0.05; 'library.': 0.05; 'random': 0.05;
'below)': 0.07; 'string': 0.07; 'used.': 0.07; 'wrong.': 0.07;
'cc:addr:python-list': 0.09; 'didn\xe2\x80\x99t': 0.09; 'moved':
0.09; 'segmentation': 0.09; 'url:mailman': 0.15; 'problem.': 0.15;
'(3)': 0.16; '2022,': 0.16; 'answer.': 0.16; 'arguments': 0.16;
'comments.': 0.16; 'directory.': 0.16; 'email addr:python.org:':
0.16; 'incremented': 0.16; 'initialize': 0.16; 'modules,': 0.16;
'nltk': 0.16; 'pointers': 0.16; 'program:': 0.16; 'pyobject*':
0.16; 'skip:. 20': 0.16; 'solved': 0.16; 'value?': 0.16; 'wrote:':
0.16; 'python': 0.16; 'api': 0.17; 'values': 0.17; 'instead':
0.17; 'solve': 0.19; 'cc:addr:python.org': 0.20; 'written': 0.22;
'problem,': 0.22; 'returns': 0.22; 'code': 0.23; 'command': 0.23;
'skip:p 30': 0.23; 'received:de': 0.23; 'url-
ip:188.166.95.178/32': 0.25; 'url-ip:188.166.95/24': 0.25;
'url:listinfo': 0.25; 'cc:2**0': 0.25; 'url-ip:188.166/16': 0.25;
'anyone': 0.25; 'seems': 0.26; 'tried': 0.26; 'library': 0.26;
'object': 0.26; 'else': 0.27; 'ideas': 0.28; 'recently': 0.29;
'error': 0.29; 'program,': 0.31; 'url-ip:188/8': 0.31; 'program':
0.31; 'question': 0.32; '(as': 0.32; 'it.\xc2\xa0': 0.32;
'objects': 0.32; 'python-list': 0.32; 'sep': 0.32; 'signal': 0.32;
'received:192.168.1': 0.32; 'but': 0.32; 'same': 0.34; 'header:In-
Reply-To:1': 0.34; '8bit%:40': 0.35; 'errors': 0.36; 'using':
0.37; "it's": 0.37; 'this.': 0.37; 'received:192.168': 0.37;
'file': 0.38; 'way': 0.38; '8bit%:14': 0.38; 'thanks': 0.38;
'list': 0.39; 'use': 0.39; 'still': 0.40; '(see': 0.40;
'program.': 0.40; 'both': 0.40; 'something': 0.40; 'reference':
0.60; 'best': 0.61; 'load': 0.62; 'here': 0.62; 'pass': 0.64;
'clear': 0.64; 'verified': 0.64; 'your': 0.64; 'well': 0.65;
'now,': 0.67; 'number:': 0.67; 'right': 0.68; 'before,': 0.69;
'cast': 0.69; 'created.': 0.69; 'generator': 0.69; 'refers': 0.69;
'below': 0.69; '8bit%:38': 0.70; '8bit%:6': 0.71; 'skip:\xe2 10':
0.71; 'documented': 0.76; 'correct.\xc2\xa0': 0.84; 'exactly.':
0.84; 'imports': 0.84; 'pointer': 0.84; 'revised': 0.91;
'here\xe2\x80\x99s': 0.93
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1664491299;
s=s1; d=tutanota.com;
h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender;
bh=Jk3GNFPjSKQKTebukP8B4ln2r82SDJLRPDUrll0Va7U=;
b=qdLZIbTaR4MDl/T2Xsw+wIOZvqKS74w2lHKTuhwqDMRnRqkmHNs4MWoM7lXsVQyU
x7758AdFWZvsZguQrx9zwBW1L5FtPOGDmxPmcp7k1KMuKeTeWoC5EP+lPGGYlk3in1R
1+ENTzt87bdCfRE+kMkURfgVSKTgBd/Ei38DYrK7QUMDIHJFFoGa+mwHwyNqboVD3zQ
nkuRU5soOVa3hvQBy5TsH+2IvdoWdcnHQ2Rn7XydOcOso5x/+vRFaMsyQR5XWQU88Wh
YXP53uw6M4TrebEHVj1QDw4hvAhwsZ+kUnIQ+MtAAjDnYFoxiLg535ocrSHVAdePjtr
Rg8r6LycoA==
In-Reply-To: <b60d1d3b-1381-8646-fd43-15496bf993dd@mrabarnett.plus.com>
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <NDAPOvd--3-2@tutanota.com>
X-Mailman-Original-References: <ND8xHA---3-2@tutanota.com>
<fbd7fcc1-6095-e381-4721-bc5c09975202@mrabarnett.plus.com>
<ND9prTQ--3-2@tutanota.com-ND9qgBV----2> <ND9zJoA--7-2@tutanota.com>
<b60d1d3b-1381-8646-fd43-15496bf993dd@mrabarnett.plus.com>
 by: Jen Kris - Thu, 29 Sep 2022 22:41 UTC

I just solved this C API problem, and I’m posting the answer to help anyone else who might need it. 

The errors were:

(1) we must call Py_INCREF on each object when it’s created.

(2) in C_API_2 (see below) we don’t cast value_1 as I did before with PyObject * value_ptr = (PyObject * )value_1.  Instead we use PyObject * value_ptr = PyLong_FromLong(value_1);

(3) The command string to PyObject_CallFunctionObjArgs must be null terminated.

Here’s the revised code:

First we load the modules, and increment the reference to each object: 

int64_t Get_LibModules(int64_t * return_array)
{ PyObject * pName_random = PyUnicode_FromString("random");
PyObject * pMod_random = PyImport_Import(pName_random);

Py_INCREF(pName_random);
Py_INCREF(pMod_random);

if (pMod_random == 0x0){
PyErr_Print();
return 1;}

PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed");
PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random, "randrange");

Py_INCREF(pAttr_seed);
Py_INCREF(pAttr_randrange);

return_array[0] = (int64_t)pAttr_seed;
return_array[1] = (int64_t)pAttr_randrange;

return 0;
}

Next we call a program to initialize the random number generator with random.seed(), and increment the reference to its return value p_seed_calc:

int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1)
{ PyObject * value_ptr = PyLong_FromLong(value_1);
PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed, value_ptr, NULL);

// _________

if (p_seed_calc == 0x0){
    PyErr_Print();
    return 1;}

Py_INCREF(p_seed_calc);

return 0;
}

Now we call another program to get a random number:

int64_t C_API_12(PyObject * pAttr_randrange, Py_ssize_t value_1)
{ PyObject * value_ptr = PyLong_FromLong(value_1);
PyObject * p_randrange_calc = PyObject_CallFunctionObjArgs(pAttr_randrange, value_ptr, NULL);

if (p_randrange_calc == 0x0){
    PyErr_Print();
    return 1;}

//Prepare return values
long return_val = PyLong_AsLong(p_randrange_calc);

return return_val;
}

That returns 28, which is what I get from the Python command line. 

Thanks again to MRAB for helpful comments. 

Jen

Sep 29, 2022, 15:31 by python@mrabarnett.plus.com:

> On 2022-09-29 21:47, Jen Kris wrote:
>
>> To update my previous email, I found the problem, but I have a new problem.
>>
>> Previously I cast PyObject * value_ptr = (PyObject * )value_1 but that's not correct.  Instead I used PyObject * value_ptr = PyLong_FromLong(value_1) and that works.  HOWEVER, while PyObject_CallFunctionObjArgs does work now, it returns -1, which is not the right answer for random.seed.  I use "long return_val = PyLong_AsLong(p_seed_calc);" to convert it to a long.
>>
> random.seed returns None, so when you call PyObject_CallFunctionObjArgs it returns a new reference to Py_None.
>
> If you then pass to PyLong_AsLong a reference to something that's not a PyLong, it'll set an error and return -1.
>
>> So my question is why do I get -1 as return value?  When I query p_seed calc : get:
>>
>> (gdb) p p_seed_calc
>> $2 = (PyObject *) 0x7ffff69be120 <_Py_NoneStruct>
>>
> Exactly. It's Py_None, not a PyLong.
>
>> Thanks again.
>>
>> Jen
>>
>>
>>
>>
>> Sep 29, 2022, 13:02 by python-list@python.org:
>>
>> Thanks very much to @MRAB for taking time to answer.  I changed my
>> code to conform to your answer (as best I understand your comments
>> on references), but I still get the same error.  My comments
>> continue below the new code immediately below.
>>
>> int64_t Get_LibModules(int64_t * return_array)
>> {
>> PyObject * pName_random = PyUnicode_FromString("random");
>> PyObject * pMod_random = PyImport_Import(pName_random);
>>
>> Py_INCREF(pName_random);
>> Py_INCREF(pMod_random);
>>
>> if (pMod_random == 0x0){
>> PyErr_Print();
>> return 1;}
>>
>> PyObject * pAttr_seed = PyObject_GetAttrString(pMod_random, "seed");
>> PyObject * pAttr_randrange = PyObject_GetAttrString(pMod_random,
>> "randrange");
>>
>> Py_INCREF(pAttr_seed);
>> Py_INCREF(pAttr_randrange);
>>
>> return_array[0] = (int64_t)pAttr_seed;
>> return_array[1] = (int64_t)pAttr_randrange;
>>
>> return 0;
>> }
>>
>> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1)
>> {
>> PyObject * value_ptr = (PyObject * )value_1;
>> PyObject * p_seed_calc = PyObject_CallFunctionObjArgs(pAttr_seed,
>> value_ptr, NULL);
>>
>> if (p_seed_calc == 0x0){
>>     PyErr_Print();
>>     return 1;}
>>
>> //Prepare return values
>> long return_val = PyLong_AsLong(p_seed_calc);
>>
>> return return_val;
>> }
>>
>> So I incremented the reference to all objects in Get_LibModules,
>> but I still get the same segfault at
>> PyObject_CallFunctionObjArgs.  Unfortunately, reference counting
>> is not well documented so I’m not clear what’s wrong.
>>
>>
>>
>>
>> Sep 29, 2022, 10:06 by python@mrabarnett.plus.com:
>>
>> On 2022-09-29 16:54, Jen Kris via Python-list wrote:
>>
>> Recently I completed a project where I used
>> PyObject_CallFunctionObjArgs extensively with the NLTK
>> library from a program written in NASM, with no problems.
>> Now I am on a new project where I call the Python random
>> library.  I use the same setup as before, but I am getting
>> a segfault with random.seed.
>>
>> At the start of the NASM program I call a C API program
>> that gets PyObject pointers to “seed” and “randrange” in
>> the same way as I did before:
>>
>> int64_t Get_LibModules(int64_t * return_array)
>> {
>> PyObject * pName_random = PyUnicode_FromString("random");
>> PyObject * pMod_random = PyImport_Import(pName_random);
>>
>> Both PyUnicode_FromString and PyImport_Import return new
>> references or null pointers.
>>
>> if (pMod_random == 0x0){
>> PyErr_Print();
>>
>>
>> You're leaking a reference here (pName_random).
>>
>> return 1;}
>>
>> PyObject * pAttr_seed >> PyObject_GetAttrString(pMod_random, "seed");
>> PyObject * pAttr_randrange >> PyObject_GetAttrString(pMod_random, "randrange");
>>
>> return_array[0] = (int64_t)pAttr_seed;
>> return_array[1] = (int64_t)pAttr_randrange;
>>
>>
>> You're leaking 2 references here (pName_random and pMod_random).
>>
>> return 0;
>> }
>>
>> Later in the same program I call a C API program to call
>> random.seed:
>>
>> int64_t C_API_2(PyObject * pAttr_seed, Py_ssize_t value_1)
>> {
>> PyObject * p_seed_calc >> PyObject_CallFunctionObjArgs(pAttr_seed, value_1);
>>
>>
>> It's expecting all of the arguments to be PyObject*, but
>> value_1 is Py_ssize_t instead of PyObject* (a pointer to a
>> _Python_ int).
>>
>> The argument list must end with a null pointer.
>>
>> It returns a new reference or a null pointer.
>>
>>
>> if (p_seed_calc == 0x0){
>>     PyErr_Print();
>>     return 1;}
>>
>> //Prepare return values
>> long return_val = PyLong_AsLong(p_seed_calc);
>>
>> You're leaking a reference here (p_seed_calc).
>>
>> return return_val;
>> }
>>
>> The first program correctly imports “random” and gets
>> pointers to “seed” and “randrange.”  I verified that the
>> same pointer is correctly passed into C_API_2, and the
>> seed value (1234) is passed as  Py_ssize_t value_1.  But I
>> get this segfault:
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00007ffff64858d5 in _Py_INCREF (op=0x4d2) at
>> ../Include/object.h:459
>> 459     ../Include/object.h: No such file or directory.
>>
>> So I tried Py_INCREF in the first program:
>>
>> Py_INCREF(pMod_random);
>> Py_INCREF(pAttr_seed);
>>
>> Then I moved Py_INCREF(pAttr_seed) to the second program. 
>> Same segfault.
>>
>> Finally, I initialized “random” and “seed” in the second
>> program, where they are used.  Same segfault.
>>
>> The segfault refers to Py_INCREF, so this seems to do with
>> reference counting, but Py_INCREF didn’t solve it.
>>
>> I’m using Python 3.8 on Ubuntu.
>>
>> Thanks for any ideas on how to solve this.
>>
>> Jen
>>
>>
>> -- https://mail.python.org/mailman/listinfo/python-list
>>
>>
>> -- https://mail.python.org/mailman/listinfo/python-list
>>
> --
> https://mail.python.org/mailman/listinfo/python-list
>


Click here to read the complete article
1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor