Message-ID:

6 May, 2024: The networking issue during the past two days has been identified and appears to be fixed. Will keep monitoring.

devel / comp.lang.python / Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)

Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)

<mailman.545.1665226263.20444.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=19675&group=comp.lang.python#19675

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!news.szaf.org!fu-berlin.de!uni-berlin.de!not-for-mail
From: gweathe...@uchc.edu (Weatherby,Gerard)
Newsgroups: comp.lang.python
Subject: Re: Ref-strings in logging messages (was: Performance issue with
CPython 3.10 + Cython)
Date: Sat, 8 Oct 2022 10:50:51 +0000
Lines: 43
Message-ID: <mailman.545.1665226263.20444.python-list@python.org>
References: <f583eefa-ef36-a59b-7b33-928d67e49f06@mrabarnett.plus.com>
<FC32E4B4-6B96-44BC-ACE6-C0EF2E93242B@barrys-emacs.org>
<SA1PR14MB5855B09F91EF5250E297E9D4B95E9@SA1PR14MB5855.namprd14.prod.outlook.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
X-Trace: news.uni-berlin.de GSvF8p7kSuvMF/cdeOciuw0yKXMGCx5ll0t72VshRhbQ==
Return-Path: <prvs=0280b9b6e5=gweatherby@uchc.edu>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=uchc.edu header.i=@uchc.edu header.b=Zv5VlPRH;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'def': 0.04; '2022': 0.05;
'>>>': 0.07; 'debugging': 0.07; 'cc:addr:python-list':
0.09; 'enabled': 0.09; 'lazy': 0.09;
'received:namprd14.prod.outlook.com': 0.09; 'skip:` 20': 0.09;
'url-ip:13.107.213.67/32': 0.09; 'url-ip:13.107.213/24': 0.09;
'url-ip:13.107.246.67/32': 0.09; 'url-ip:13.107.246/24': 0.09;
'log': 0.12; '>': 0.14; 'url:mailman': 0.15; '***': 0.16;
'1:30': 0.16; '2022,': 0.16; '3.10': 0.16; 'arguments': 0.16;
'barry': 0.16; 'cpython': 0.16; 'efficiently': 0.16; 'evaluating':
0.16; 'montanaro': 0.16; 'subject:3.10': 0.16; 'subject:CPython':
0.16; 'subject:Cython': 0.16; 'subject:issue': 0.16;
'subject:logging': 0.16; 'url:urldefense': 0.16; 'url:v3': 0.16;
'wrote:': 0.16; 'october': 0.17; 'calls': 0.19;
'cc:addr:python.org': 0.20; 'issue': 0.21; 'fri,': 0.22; 'skip:_
10': 0.22; 'discussion': 0.25; 'url:listinfo': 0.25; 'cc:2**0':
0.25; 'friday,': 0.26; 'object': 0.26; 'opening': 0.26;
'received:edu': 0.26; '>>>': 0.28; 'email addr:python.org>':
0.28; 'am,': 0.31; 'module': 0.31; 'python-list': 0.32; "i'm":
0.33; 'there': 0.33; 'header:In-Reply-To:1': 0.34;
'header:Received:8': 0.36; 'received:filterd': 0.37;
'received:pps.filterd': 0.37; 'class': 0.37; 'url-ip:13.107/16':
0.38; 'way': 0.38; 'could': 0.38; 'date:': 0.39; 'use': 0.39;
'case.': 0.40; 'gone': 0.40; 'email.': 0.61; 'from:': 0.62; 'me.':
0.62; 'to:': 0.62; 'pass': 0.64; 're:': 0.64; 'cc:': 0.69;
'obvious': 0.69; 'url-ip:52.6/16': 0.69; 'performance': 0.71;
'clicking': 0.76;
'received:nam10-bn7-obe.outbound.protection.outlook.com': 0.76;
'links.': 0.81; 'admit': 0.84; 'attention:': 0.84; 'email name:&lt
;python-list': 0.84; 'enabled.': 0.84; 'generation.': 0.84;
'skip:& 50': 0.84; 'subject: \n ': 0.84; 'subject:Performance':
0.84; 'expensive': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uchc.edu;
h=from : to : cc :
subject : date : message-id : references : in-reply-to : content-type :
mime-version; s=UCHC03162020;
bh=s8oAbOviB8jsD0/jCzy/UfpWK2iIAtvqESuiv4RW+M0=;
b=Zv5VlPRHhMlf7k01vceHSuA9R5iybAoVXg5R7vv8qliWWFybn8nuKwp4qjt58WGPFcY7
oHpnashTxqsADMKxS7ayvV/lr3WIU1TMxK4eTZs/M8b4/KfJm4jNJx63rF7FKtxvkFMJ
XjLi6Xu1bOMXZ4/F0M3/kfoOHAu55vdOJSmC+HOC8pLj7wBRZCe52lw1kfh2t4raAjIF
59EwWuM1d8S9OvrUKH+1U3/Q6iBfXJ3DSz9JL+dlnWZb1dWamO6Q0pfy4KYKDXOjPwX8
GvGINjY4RCGumDoC3HlDks67p3rqZ2R5yhTzFJc/v4oeM1Pm2hKsFWsUWpECPjzxg1cn Yg==
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=eo1D1sTmye1B5p5WMf8/2IP/09/8eHGK/vxdYzfUXXQM0agFcxbgD9EXaJ416hti0Sj5RCGTfWEH5cwhfi8q5VqNhFIJ2BqGKy0+fGcmPmA6G0soLMR/q8S36JnT4PARb5KAJyFsv4c7ZSHfUemNXMaWFnI0nDOjig75mQX47zkmEOlxvi8O8PKX+OxNHeIyxzI0Qe8xsZ/77ZqHctzNlSeAPhCSLaN9PQc09fgMGcunkKrGRNytOIBvJCIMzz/g2pi8/9crMTMuDibEeSGk1jiz0c+tqJ45qhr5DiIgPqRIrwW6Qp2vccuFj7E45v8+4gejAOWg/EA06IBUtHSidA==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=s8oAbOviB8jsD0/jCzy/UfpWK2iIAtvqESuiv4RW+M0=;
b=hg3HbcDZU+IxDs08fdAyoBVJvlzJVqPJur73ADZ6RBYURTEND9E5WjUA8pkuGHUCGFU1LHwoOzN5+G26pwodxN6iLgB90QmlazNA075PxG2hF6qDstihcTA/lwdzP6fVVSxH1PZ0VHyj3K8H1c+TGoeY2MEvo0hUTu9LgciI2M0mrnBDD/WLXq7/bm7GCu0IYm5vBMzMkxwrbfqArKejJJfiCgAgCEYRil9nlzhwoFzgcuf+kDTEZz2PNXc+M4K+nrIPOAxzx+EhDCwjRFW3Jz0W9dK4UR0DhKEnz7qDMQDiddqJPFXKJQNYFkxoK1ANXZFXJNJMp+qAZBC2/wiZvg==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=uchc.edu; dmarc=pass action=none header.from=uchc.edu;
dkim=pass header.d=uchc.edu; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uchc.onmicrosoft.com;
s=selector2-uchc-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=s8oAbOviB8jsD0/jCzy/UfpWK2iIAtvqESuiv4RW+M0=;
b=KZxQdQQHqPKjy/c5tEvdA12+l0IgnrpaJRWdbTv9amifCM2Jv71G7f/jRZjD4anJjB05TNJqfIywPtdoO0j/Uhv2yc6+3XCkYCCbj0djh3znlml8MhXdOZnSYoK+4/ZqOaAn+GiX/m5nIu6QnPOj4s1whYztVVASu4Qpt8pdTAE=
Thread-Topic: Ref-strings in logging messages (was: Performance issue with
CPython 3.10 + Cython)
Thread-Index: AQHY2mQiMCrmoE+o502VgGYTOhAxmq4DKqKAgAAE6ACAASIEkw==
In-Reply-To: <FC32E4B4-6B96-44BC-ACE6-C0EF2E93242B@barrys-emacs.org>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: SA1PR14MB5855:EE_|BL0PR14MB3617:EE_
x-ms-office365-filtering-correlation-id: b8e4bb25-40f1-4a0c-371d-08daa91af7f2
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: TadhW7zIvm6Mz0bmuE78+OxwFGx1i1DfJxMMuk9lIwYVpRFvOVjQ3cjYz/vHvcXaasa4RhHbLnHqdQnCjm5T4+3nC6AwuR/g1/3mMp+OG1I/Kei9kpWe8lQr5UcfdzjgtBYLAwJzIikWafJbndrg7H6gtP+hckDLDm1iFiD/L3ywEtaWFONKpqG1uMaYq8nZa8E17RAs5hxRQJcWg4hoiUCcwOmN2nfDSm2Dyzl3FoVlZD8H1ARSb+w2BsNHUhZmU78etcb6bV77uQPUrwW/tLTxEqWmBnIQms/Y25IN5g07Y7Io7gmvOCAX2s43HRsFXypIqoHDoIZ3+kB9qx65IkskWHyP7OGJP18EMP/a7qfJQeCZls30eMJaQUydUXwXDZbQLtwqLCqkqNsTVmv729rikTumfT8ANJIUZ3B1GVZx4aAER3g7DxRGZlBN0zb4LCw5RPcka/IL2iIWEjGmGYwcYpX82dxUojwjT1pGFaVoivnY6n/FTeKMNV1FTLONYeC9pARD/FWP3vCrgpQ0+aNPFzDQr4niamUmi5AKNlRwvSLwuvPTSfd2YHTL2mMan5Yk72ePTZT61x0WdrSYd7LaUHLo/W/BbupQ9rZMuLSsJROiRJe+3ZiWeqE9LfFMl2lQTmayl7d7gVbGuf4bjBiKe2ZFQ6tlNUwb1m4eyRuYvFSc66d0YuD/669XPFq8GTL04m4a78MiWcmmknXCPA2vWD6JRkde+JqExyyT5ci8pQEHAo5JinZNuQ/gEpYA0wXJMCk1Kj+IiC1lPBvdipAGOiMkWuALw7EFQ+MZfo4cDjFTLZvIkMt/cXUWBquocIFgkjrEAgf49sJ44CE28g==
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:SA1PR14MB5855.namprd14.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230022)(376002)(396003)(346002)(366004)(39850400004)(136003)(451199015)(71200400001)(53546011)(15650500001)(478600001)(33656002)(38100700002)(64756008)(6506007)(4326008)(122000001)(966005)(166002)(83380400001)(66446008)(8676002)(7696005)(38070700005)(66476007)(52536014)(76116006)(5660300002)(66946007)(316002)(55016003)(8936002)(786003)(86362001)(41300700001)(91956017)(186003)(66556008)(26005)(110136005)(9686003)(2906002)(41320700001)(75432002)(11970500018);
DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: DyI8hPrrNZkCxVyP79AJCL7ARgT/Gus0SHdISGEZaLHKa
ovsGJGXKiDdrs8FYT0CbqLCaS2ybf7bHjdbPF9jm1Fd23
Zj7Cn+/DiO9+AzBniehpxBXp3OCYq1lrgJCLitgNKW/Tb
9X2cvELq6ANOvS4DXTxuZq+9KhwOr3gCq+OYNSz+kecVj
O5f5AZPpDGBWyohtsmnLYZatDFoe7chRimdl8s9qswPhi
b6qVu7oklMzRM0BYfwFFk6Ep6ROONlHYKq9bC76YlSEt4
CCn/bLWhrK6iNrYSmj8mkp4uOmFIisKfTMst1tmmrJ9xf
zgWcaHMyoHWo43r+UJt76ByHX0JuCGOAJLRWN7GusPBZP
yzpa7pyA+VQppZKB3lm68QmaT0eV/wsTB7I750lugWnTD
iiWf5z6k6piPQt/y4stvkKh7S+siACXInDG19OuFUgv+v
eGfzooktq4/yMwUU098zMMdNt8Nry3qwmHMm0hl5qPxne
HQf22YjynczCozA5tAu8/j4GJZ8XpMbZCfrSEbefiwcV7
UeD1IRRjXJ+ARY706XWnyTZ6lVTxSXvq38M0rOjJe3lqt
C4+zk6kBVrLxWQ3Tt2X44p92JlWcfuVbR0tjw3LeK+Bk9
0drBkOL+UqWlMuB8YtGvOTAnU431eixarAgf3zz2CYV7Q
6M1tFsrARSLVaCsR0FzlbNxI/nf4Hvc2574+5V3ogthal
dAAjX24n6qRi2qjV6o4+Ozhob7Pyl2g9hxybaLRWPPtdg
PLzLmF3zDKjyY7z545lZUopbUa8130RaxlPbFuizJcf4y
0wglr2zJZGw5iGPi6cvA9LjgLhMPmi7XJHlGbE4XbdLEg
FWyenxyTkqyfUWiqBeB58O09ziF3LIXEk9tKtPfwjZF71
3MTwHn62Wp48oFDs5/gl9vxaZTzRRvkgOY4Wv12btjkjX
EhIbj1R+9UEEC0rgeZjOwLSH4dDoAlGi+emgOteIWBNXC
OUmiflI010Lw9WE/RcJhUoxgM1iGsAUUYVtdCbqZsEmq7
M1DIojpQvxaqmMTLpocV672yHV4OFSqeq8wVaJEClBz0u
NLn7SUU0ZCSWnusbrhf90oZZzhxi8c0Kw9J5USZ0or1Kr
vdscLkPA0/fv8jVXsUys+evE5zchdGqipgkgfYJNakCsT
Hxhn4sSs5vCOOk/W/wg2uC+TrDbF7ou7sLk/R0/pF/OeK
543K2NmIXeHDvRufl8nTQ7uvgZAW/br07Az4+U7oYHEK2
tEdbeobTfcJ6KOw7uwv2xQ4OltaLTQjKngbxc3It+spCV
belaM3X3hWWB4Cy8UAEK0lGEM9HTSyfeEa7cVz/A1DPJg
3SPrxrFgTYzNkq2gFnVChvOGsES+lJPJ1n0XxtGlhp+h+
cKXo9gBdkM1w54F465RJcLshqcB5vsqxow1QsDzzGe30k
iwxzHjZj94HVqifAMr6yfBV9gdZMfHbPNeelic0koOOXP
G7oC9li/VBI+0lCxstqtsqpV10pilorg9Q6S+fzbPk+qi
tQKOnKS8MkK00VB8NTZXUQVJJq3sirin8kqUGl7xWRXhy
briZKchDw8iMNzARezRYwL8AwuIzDSnRSIFGXfY9KB/kw
uQ==
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SA1PR14MB5855.namprd14.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b8e4bb25-40f1-4a0c-371d-08daa91af7f2
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Oct 2022 10:50:51.9967 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5c82d83a-818a-4c16-b540-ded2344a7ad3
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: PRnLDxV7ylTh9tFoG9B8qqSigjIKKxkXSjzBugPpNYkV/tcXnDVDcdV73U5HG+jfRsHXNJVKx7Wb8VzMZmcD5Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR14MB3617
X-Proofpoint-GUID: oLf8MZfqOdds-wfkwjpXtaYMG0UzNHMp
X-Proofpoint-ORIG-GUID: oLf8MZfqOdds-wfkwjpXtaYMG0UzNHMp
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.205,Aquarius:18.0.895,Hydra:6.0.528,FMLib:17.11.122.1
definitions=2022-10-07_04,2022-10-07_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
clxscore=1011 malwarescore=0
mlxscore=0 impostorscore=0 bulkscore=0 lowpriorityscore=0 suspectscore=0
phishscore=0 spamscore=0 priorityscore=1501 mlxlogscore=999 adultscore=0
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2209130000
definitions=main-2210080068
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <SA1PR14MB5855B09F91EF5250E297E9D4B95E9@SA1PR14MB5855.namprd14.prod.outlook.com>
X-Mailman-Original-References: <f583eefa-ef36-a59b-7b33-928d67e49f06@mrabarnett.plus.com>
<FC32E4B4-6B96-44BC-ACE6-C0EF2E93242B@barrys-emacs.org>

by: Weatherby,Gerard - Sat, 8 Oct 2022 10:50 UTC

Logging does support passing a callable, if indirectly. It only calls __str__ on the object passed if debugging is enabled.
class Defer:
def __init__(self,fn):
self.fn = fn
def __str__(self):
return self.fn()
def some_expensive_function():
return "hello"
logging.basicConfig()
logging.debug(Defer(some_expensive_function))

From: Python-list <python-list-bounces+gweatherby=uchc.edu@python.org> on behalf of Barry <barry@barrys-emacs.org>
Date: Friday, October 7, 2022 at 1:30 PM
To: MRAB <python@mrabarnett.plus.com>
Cc: python-list@python.org <python-list@python.org>
Subject: Re: Ref-strings in logging messages (was: Performance issue with CPython 3.10 + Cython)
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***
> On 7 Oct 2022, at 18:16, MRAB <python@mrabarnett.plus.com> wrote:
>
> On 2022-10-07 16:45, Skip Montanaro wrote:
>>> On Fri, Oct 7, 2022 at 9:42 AM Andreas Ames <andreas.0815.qwertz@gmail.com>
>>> wrote:
>>> 1. The culprit was me. As lazy as I am, I have used f-strings all over the
>>> place in calls to `logging.logger.debug()` and friends, evaluating all
>>> arguments regardless of whether the logger was enabled or not.
>>>
>> I thought there was some discussion about whether and how to efficiently
>> admit f-strings to the logging package. I'm guessing that's not gone
>> anywhere (yet).
> Letting you pass in a callable to call might help because that you could use lambda.
Yep, that’s the obvious way to avoid expensive log data generation.
Would need logging module to support that use case.
Barry
> --
> https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$>
>
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!mrESxAj9YCHsdtNAfkNiY-Zf6U3WTIqaNrgBmbw1ELlQy51ilob43dD0ONsqvg4a94MEdOdwomgyqfyABbvRnA$>

Subject	Author
Re: Ref-strings in logging messages (was: Performance issue with	Weatherby,Gerard