Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Conquest is easy. Control is not. -- Kirk, "Mirror, Mirror", stardate unknown

devel / comp.lang.python / Re: C API PyObject_Call segfaults with string

SubjectAuthor
o Re: C API PyObject_Call segfaults with stringJen Kris

1
Re: C API PyObject_Call segfaults with string

<mailman.97.1644523447.7010.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=17023&group=comp.lang.python#17023

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: jenk...@tutanota.com (Jen Kris)
Newsgroups: comp.lang.python
Subject: Re: C API PyObject_Call segfaults with string
Date: Thu, 10 Feb 2022 21:04:05 +0100 (CET)
Lines: 107
Message-ID: <mailman.97.1644523447.7010.python-list@python.org>
References: <MvW3s4j--3-2@tutanota.com>
<CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
<MvW9RVE--3-2@tutanota.com>
<CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
<MvWBjNR--3-2@tutanota.com>
<CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
<MvWGgaJ--B-2@tutanota.com>
<36767883-dafa-84f0-ee65-478965a6ae8f@mrabarnett.plus.com>
<Mv_E4Xm--3-2@tutanota.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de 5aIfuwpxFSxmWVSmo45zgQuLro6DVwfJP9ur+SBurWUA==
Return-Path: <jenkris@tutanota.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=tutanota.com header.i=@tutanota.com header.b=Jbak7e5W;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'looks': 0.02; 'comments':
0.03; '3.8': 0.05; 'chances': 0.05; 'variable': 0.05;
'subject:API': 0.07; '"""': 0.09; 'cc:addr:python-list': 0.09;
'const': 0.09; 'example:': 0.09; 'subject:skip:P 10': 0.09;
'url:mailman': 0.15; '2022,': 0.16; '[snip]': 0.16; 'api.': 0.16;
'arguments': 0.16; 'cc:name:python list': 0.16; 'char': 0.16;
'easiest': 0.16; 'goto': 0.16; 'inada': 0.16; 'naoki': 0.16;
'reference,': 0.16; 'solved': 0.16; 'subject:string': 0.16;
'use:': 0.16; 'wrote:': 0.16; 'problem': 0.16; 'python': 0.16;
'feb': 0.17; 'cc:addr:python.org': 0.20; "i've": 0.22; 'skip:p
30': 0.23; 'received:de': 0.23; '(and': 0.25; 'url-
ip:188.166.95.178/32': 0.25; 'url-ip:188.166.95/24': 0.25;
'help.': 0.25; 'url:listinfo': 0.25; 'cc:2**0': 0.25; 'url-
ip:188.166/16': 0.25; 'tried': 0.26; 'object': 0.26; 'function':
0.27; 'greatly': 0.28; 'url-ip:188/8': 0.31; 'objects': 0.32;
'python-list': 0.32; 'received:192.168.1': 0.32; 'but': 0.32;
"i'm": 0.33; '0);': 0.33; 'header:In-Reply-To:1': 0.34; 'track':
0.35; 'using': 0.37; "it's": 0.37; 'received:192.168': 0.37;
'could': 0.38; 'means': 0.38; 'thanks': 0.38; 'valid': 0.39;
'list': 0.39; 'difficult': 0.40; 'reference': 0.60; "there's":
0.61; 'format': 0.62; 'showing': 0.62; 'once': 0.63; 'finished':
0.64; 'your': 0.64; 'addition': 0.71; 'longer': 0.71; 'see,':
0.76; 'reply': 0.77; 'leaked': 0.84; 'lot.': 0.84; 'reduces': 0.84
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1644523445;
s=s1; d=tutanota.com;
h=From:From:To:To:Subject:Subject:Content-Description:Content-ID:Content-Type:Content-Type:Content-Transfer-Encoding:Cc:Cc:Date:Date:In-Reply-To:In-Reply-To:MIME-Version:MIME-Version:Message-ID:Message-ID:Reply-To:References:References:Sender;
bh=V7FLanIu4nP9NG+wjnrImx1uExtU89RTXf0GHrioSDI=;
b=Jbak7e5WYYySrVfIQW16zYX9agT9Bs9fCcki7KLi9+3rfMfV10Opb4Rg9VTDtDva
DkrLmyJ+gIXmb9C2eMUDGkpc0Uz/K/MLGseu9f187h3WWZElHkO99FPZohpibfRAwVz
Y6grksOgE7LHtmhPY9UfPLdIMYQn1AZ1L39iCselruo1YxQQNsdi0NKhZOQYsexfZnl
N3EVdYvOoT2T3UtT1lEuXccnqgiOvtdfWvjOKtxMc4YR1gZc7jH2Shh/CJnSIFX0Z7E
jl6J66zQhP/UdwgAc61nIFZNfmDr4p7cy2P+VbyFDw6pkeGrqmzlVYirVTc15zdHhpT
dKFOO/THFA==
In-Reply-To: <36767883-dafa-84f0-ee65-478965a6ae8f@mrabarnett.plus.com>
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <Mv_E4Xm--3-2@tutanota.com>
X-Mailman-Original-References: <MvW3s4j--3-2@tutanota.com>
<CAEfz+TxXDn975BWxhG87O2GfVNpZj6tJ4vA-=ub44CcwHeY=hA@mail.gmail.com>
<MvW9RVE--3-2@tutanota.com>
<CAEfz+Ty7MhhLZugDgnEjrQX_UmN7sLUifbBPz3gtB73QFe5TWQ@mail.gmail.com>
<MvWBjNR--3-2@tutanota.com>
<CAEfz+Ty3HGXymXt2VMAeVXzR7rw14HPu6rBrrARLV1UGXQEE=w@mail.gmail.com>
<MvWGgaJ--B-2@tutanota.com>
<36767883-dafa-84f0-ee65-478965a6ae8f@mrabarnett.plus.com>
 by: Jen Kris - Thu, 10 Feb 2022 20:04 UTC

Hi and thanks very much for your comments on reference counting.  Since I'm new to the C_API that will help a lot.  I know that reference counting is one of the difficult issues with the C API. 

I just posted a reply to Inada Naoki showing how I solved the problem I posted yesterday. 

Thanks much for your help.

Jen

Feb 9, 2022, 18:43 by python@mrabarnett.plus.com:

> On 2022-02-10 01:37, Jen Kris via Python-list wrote:
>
>> I'm using Python 3.8 so I tried your second choice:
>>
>> pSents = PyObject_CallFunctionObjArgs(pSentMod, pListItem);
>>
>> but pSents is 0x0.  pSentMod and pListItem are valid pointers.
>>
> 'PyObject_CallFunction' looks like a good one to use:
>
> """PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...)
>
> Call a callable Python object callable, with a variable number of C arguments. The C arguments are described using a Py_BuildValue() style format string. The format can be NULL, indicating that no arguments are provided.
> """
>
> [snip]
>
> What I do is add comments to keep track of what objects I have references to at each point and whether they are new references or could be NULL.
>
> For example:
>
> pName = PyUnicode_FromString("nltk.corpus");
> //> pName+?
>
> This means that 'pName' contains a reference, '+' means that it's a new reference, and '?' means that it could be NULL (usually due to an exception, but not always) so I need to check it.
>
> Continuing in this vein:
>
> pModule = PyImport_Import(pName);
> //> pName+? pModule+?
>
> pSubMod = PyObject_GetAttrString(pModule, "gutenberg");
> //> pName+? pModule+? pSubMod+?
> pFidMod = PyObject_GetAttrString(pSubMod, "fileids");
> //> pName+? pModule+? pSubMod+? pFidMod+?
> pSentMod = PyObject_GetAttrString(pSubMod, "sents");
> //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+?
>
> pFileIds = PyObject_CallObject(pFidMod, 0);
> //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? PyObject_CallObject+?
> pListItem = PyList_GetItem(pFileIds, listIndex);
> //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? PyObject_CallObject+? pListItem?
> pListStrE = PyUnicode_AsEncodedString(pListItem, "UTF-8", "strict");
> //> pName+? pModule+? pSubMod+? pFidMod+? pSentMod+? PyObject_CallObject+? pListItem? pListStrE+?
>
> As you can see, there's a lot of leaked references building up.
>
> Note how after:
>
> pListItem = PyList_GetItem(pFileIds, listIndex);
>
> the addition is:
>
> //> pListItem?
>
> This means that 'pListItem' contains a borrowed (not new) reference, but could be NULL.
>
> I find it easiest to DECREF as soon as I no longer need the reference and remove a name from the list as soon I no longer need it (and DECREFed where).
>
> For example:
>
> pName = PyUnicode_FromString("nltk.corpus");
> //> pName+?
> if (!pName)
> goto error;
> //> pName+
> pModule = PyImport_Import(pName);
> //> pName+ pModule+?
> Py_DECREF(pName);
> //> pModule+?
> if (!pModule)
> goto error;
> //> pModule+
>
> I find that doing this greatly reduces the chances of getting the reference counting wrong, and I can remove the comments once I've finished the function I'm writing.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor