novaBBS - comp.lang.python - Re: Puzzling behaviour of Py

Re: Puzzling behaviour of Py_IncRef

<mailman.257.1643149687.3079.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=16808&group=comp.lang.python#16808

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: bar...@barrys-emacs.org (Barry)
Newsgroups: comp.lang.python
Subject: Re: Puzzling behaviour of Py_IncRef
Date: Tue, 25 Jan 2022 22:28:04 +0000
Lines: 89
Message-ID: <mailman.257.1643149687.3079.python-list@python.org>
References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
<1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
Mime-Version: 1.0 (1.0)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de fUXfYsRKe0yE4nmoWk88hQlszmZWBanAELFO9agGoQ7Q==
Return-Path: <barry@barrys-emacs.org>
X-Original-To: Python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=none reason="no signature";
dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; '2022': 0.05; 'despite':
0.05; 'angelico': 0.09; 'cc:addr:python-list': 0.09;
'effectively': 0.09; 'from:addr:barry': 0.09; 'obviously': 0.09;
'received:217.70': 0.09; 'received:217.70.178': 0.09;
'received:gandi.net': 0.09; 'received:mail.gandi.net': 0.09;
'cc:no real name:2**0': 0.14; 'url:mailman': 0.15; 'problem.':
0.15; '*second*': 0.16; '2022,': 0.16; '>>>>': 0.16; '>>>>>':
0.16; 'assuming': 0.16; 'barry': 0.16; 'behaviour': 0.16;
'chrisa': 0.16; 'count.': 0.16; 'drops': 0.16; 'from:addr:barrys-
emacs.org': 0.16; 'greg': 0.16; 'indeed': 0.16; 'message-id
:@barrys-emacs.org': 0.16; 'nodes': 0.16; 'python\xe2\x80\x99s':
0.16; 'wrote:': 0.16; 'python': 0.16; 'calls': 0.19;
'cc:addr:python.org': 0.20; 'fri,': 0.22; 'code': 0.23; '(and':
0.25; 'url-ip:188.166.95.178/32': 0.25; 'url-ip:188.166.95/24':
0.25; 'actual': 0.25; 'saying': 0.25; 'url:listinfo': 0.25;
'cc:2**0': 0.25; 'url-ip:188.166/16': 0.25; 'seems': 0.26; 'do,':
0.26; 'library': 0.26; "wasn't": 0.26; 'else': 0.27; 'function':
0.27; '>>>': 0.28; 'chris': 0.28; 'expect': 0.28; 'purpose': 0.28;
'suggest': 0.28; 'code,': 0.31; 'am,': 0.31; 'url-ip:188/8': 0.31;
'do.': 0.32; 'point,': 0.32; 'python-list': 0.32; 'returning':
0.32; 'but': 0.32; 'header:In-Reply-To:1': 0.34; 'trying': 0.35;
'yes,': 0.35; 'cases': 0.36; 'count': 0.36; 'change': 0.36;
"it's": 0.37; 'way': 0.38; 'could': 0.38; 'means': 0.38; 'two':
0.39; 'setting': 0.39; 'break': 0.39; 'seeing': 0.39; 'still':
0.40; 'double': 0.40; 'counts': 0.60; 'reference': 0.60;
"there's": 0.61; 'here': 0.62; 'email': 0.63; 'clear': 0.64;
'imagine': 0.64; 'your': 0.64; 'named': 0.65; 'back': 0.67;
'received:217': 0.67; 'accept': 0.67; 'right': 0.68; 'within':
0.69; '2nd': 0.70; 'interesting': 0.71; 'anthony': 0.84;
'maintains': 0.84; 'parent.': 0.84; 'referenced': 0.84; 'tony':
0.84; 'deliberately': 0.91; 'return.': 0.91
In-Reply-To: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>
X-Mailer: iPad Mail (19C63)
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <1DCDD357-E281-4024-95F8-0C9F066F1439@barrys-emacs.org>
X-Mailman-Original-References: <112a6b03-9a98-2d12-3492-67e4327ad32f@btinternet.com>

by: Barry - Tue, 25 Jan 2022 22:28 UTC

> On 25 Jan 2022, at 14:50, Tony Flury via Python-list <python-list@python.org> wrote:
>
>
>> On 20/01/2022 23:12, Chris Angelico wrote:
>>> On Fri, 21 Jan 2022 at 10:10, Greg Ewing <greg.ewing@canterbury.ac.nz> wrote:
>>> On 20/01/22 12:09 am, Chris Angelico wrote:
>>>> At this point, the refcount has indeed been increased.
>>>>
>>>>> return self;
>>>>> }
>>>> And then you say "my return value is this object".
>>>>
>>>> So you're incrementing the refcount, then returning it without
>>>> incrementing the refcount. Your code is actually equivalent to "return
>>>> self".
>>> Chris, you're not making any sense. This is C code, so there's no
>>> way that "return x" can change the reference count of x.
>> Yeah, I wasn't clear there. It was equivalent to *the Python code*
>> "return self". My apologies.
>>
>>> > The normal thing to do is to add a reference to whatever you're
>>> > returning. For instance, Py_RETURN_NONE will incref None and then
>>> > return it.
>>> >
>>>
>>> The OP understands that this is not a normal thing to do. He's
>>> trying to deliberately leak a reference for the purpose of diagnosing
>>> a problem.
>>>
>>> It would be interesting to see what the actual refcount is after
>>> calling this function.
>
> After calling this without a double increment in the function the ref count is still only 1 - which means that the 'return self' effectively does a double decrement. My original message includes the Python code which calls this 'leaky' function and you can see that despite the 'leaky POC' doing an increment ref count drops back to one after the return.
>
> You are right this is not a normal thing to do, I am trying to understand the behaviour so my library does the correct thing in all cases - for example - imagine you have two nodes in a tree :
>
> A --- > B
>
> And your Python code has a named reference to A, and B also maintains a reference to A as it's parent.
>
> In this case I would expect A to have a reference count of 2 (counted as 3 through sys.getrefcount() - one for the named reference in the Python code - and one for the link from B back to A; I would also expect B to have a reference count here of 1 (just the reference from A - assuming nothing else referenced B).
>
> My original code was incrementing the ref counts of A and B and then returning A. within the Python test code A had a refcount of 1 (and not the expected 2), but the refcount from B was correct as far as I could tell.
>
>
>> Yes, and that's why I was saying it would need a *second* incref.
>>
>> ChrisA
>
> Thank you to all of you for trying to help - I accept that the only way to make the code work is to do a 2nd increment.
>
> I don't understand why doing a 'return self' would result in a double decrement - that seems utterly bizzare behaviour - it obviously works, but why.

The return self in C will not change the ref count.

I would suggest setting a break point in your code and stepping out of the function and seeing that python’s code does to the ref count.

Barry
>
>
>
> --
> Anthony Flury
> email : anthony.flury@btinternet.com
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>

The world is no nursery. -- Sigmund Freud

devel / comp.lang.python / Re: Puzzling behaviour of Py_IncRef

Subject	Author
Re: Puzzling behaviour of Py_IncRef	Barry