Message-ID:

All the existing 2.0.x kernels are to buggy for 2.1.x to be the main goal. -- Alan Cox

devel / comp.lang.python / Re: Debugging reason for python running unreasonably slow when adding numbers

Re: Debugging reason for python running unreasonably slow when adding numbers

<mailman.2332.1678900408.20444.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=22447&group=comp.lang.python#22447

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: gweathe...@uchc.edu (Weatherby,Gerard)
Newsgroups: comp.lang.python
Subject: Re: Debugging reason for python running unreasonably slow when adding
numbers
Date: Wed, 15 Mar 2023 17:13:13 +0000
Lines: 86
Message-ID: <mailman.2332.1678900408.20444.python-list@python.org>
References: <41f44497-bf2d-44e3-a99d-e7dddc1fa2c4@Spark>
<ebf2ea3b-9273-48ed-b6fc-29aee295d201@Spark>
<20230314215211.kq4ohsrnz6vyigxw@hjp.at>
<CAPTjJmqMfDAArxKG8pAdKk1P-kjzWKbd_SunCQORP+tJDNpZZA@mail.gmail.com>
<DB7PR07MB3916933F1AA08AA930F94F6F87BF9@DB7PR07MB3916.eurprd07.prod.outlook.com>
<CAPTjJmq_ZnQOKy=kW23tL_W-kEwpuBjy23noUxNvcnWGnpy_uw@mail.gmail.com>
<d4821123-a90d-5048-5dfa-b5782dc9074f@tompassin.net>
<CAPTjJmrJ26b=9UYo8gzT_yR=vtxtm1m5PsmBQNnsZVXaeYoBdQ@mail.gmail.com>
<DB7PR07MB3916029644F69D13A5021A8A87BF9@DB7PR07MB3916.eurprd07.prod.outlook.com>
<SJ0PR14MB586405B8C2F95F6F0B2888FEB9BF9@SJ0PR14MB5864.namprd14.prod.outlook.com>
<SJ0PR14MB586493402E4FDD90DB504C53B9BF9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de Ry8N0TUUp/wjq3dmeIbikQcG8jWH9lcZTGn0rFz//sIw==
Return-Path: <prvs=0438124b24=gweatherby@uchc.edu>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=uchc.edu header.i=@uchc.edu header.b=RMmp8vzH;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.020
X-Spam-Evidence: '*H*': 0.96; '*S*': 0.00; 'def': 0.04;
'subject:python': 0.06; '2023': 0.07; 'debugging': 0.07; 'loop':
0.07; 'subject:when': 0.07; 'input:': 0.09;
'received:namprd14.prod.outlook.com': 0.09; 'subject:numbers':
0.09; '>': 0.14; 'import': 0.15; 'url:mailman': 0.15; '***':
0.16; 'confused': 0.16; 'iteration': 0.16; 'print(': 0.16; 'slow':
0.16; 'subject:running': 0.16; 'url:urldefense': 0.16; 'url:v3':
0.16; 'weights': 0.16; 'python': 0.16; 'to:addr:python-list':
0.20; 'to:name:python-list@python.org': 0.24; 'url:listinfo':
0.25; 'opening': 0.26; 'received:edu': 0.26; 'wednesday,': 0.26;
'email addr:python.org>': 0.28; 'program': 0.31; 'elements':
0.32; 'python-list': 0.32; 'skip:7 10': 0.32; "i'm": 0.33;
'subject:for': 0.33; 'march': 0.33; 'header:In-Reply-To:1': 0.34;
'running': 0.34; 'header:Received:8': 0.36; 'received:filterd':
0.37; 'received:pps.filterd': 0.37; 'url-ip:13.107/16': 0.38;
'adding': 0.39; 'enough': 0.39; 'date:': 0.39; 'list': 0.39;
'use': 0.39; 'case.': 0.40; 'wants': 0.40; 'email.': 0.61;
"there's": 0.61; 'from:': 0.62; 'to:': 0.62; 'here': 0.62;
'expert': 0.64; 'key': 0.64; 're:': 0.64; 'numbers': 0.67; 'url-
ip:104.18/16': 0.67; 'generator': 0.69; 'lifting': 0.69; 'url-
ip:52.6/16': 0.69; 'url-ip:lookup error': 0.70; 'performance':
0.71; 'clicking': 0.76; 'links.': 0.81; 'attention:': 0.84; 'email
name:<python-list': 0.84; 'indexes': 0.84; 'skip:& 50': 0.84;
'subject: \n ': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uchc.edu;
h=from : to : subject :
date : message-id : references : in-reply-to : content-type :
mime-version; s=UCHC03162020;
bh=GxybWuvtlSKV1imc7sc78vNrSCTjgfupl0E0MTBq2SM=;
b=RMmp8vzHQTvGouhWGL9+MszZYLoQiKLblgr+2WDnY44gbJqBbrO4Txl3/BNYbnWIa932
QC+W4ngCyw9t1C3sWezFHjsXelbA0DLQjMyzrigbmyPxzOMO9QmC9KDjtSmvxvyLGesd
hHKRP2+3TXnJtPfaEmSR8+PR+v3CkrQQthbPEce3VGsr8870eK6PSbK6L4VkmQ75sWqr
cjO4KTW/MXTd+0kxaXCJpEZFu7zMjg72W9hR7vAGZzd6gBywPKEgwX/iNUSYg7y4kWKc
Q+ViYt0/SKqeXE1yeIeb4/BoZLVuYw19hMVQZba6C+UzOYZlZmSq+poBycIBgJ3+Vn34 WA==
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none;
b=mcBA+jSJbDvbuJSkGZvTUBNbJAkGzE/At+fir4CaZczH1ImkiWFw/xD4IOfYOsRD4lno6Q4eofRbdZNdBMs/PWwwD1AYtr7/ILKRToWcXnjLSu/FXbmndFcvjZFohINP8QadWqTKuChw9g4dVP2k2bAL5pxfl5QXWBBkRyU0MIe6d53Bwq80XIzWwe8xqoQ+MNhdCCmsNgsGSPBiFNFYk4ludSZI8SvkdWYqaw83mZw6271LWmNd0mz2/kFBXbRyjFRG4xPN3qVYCo3BMvqtuZ/ZTtE+aHMRHz5kNbvlKcG64nwhlaLm0TWASO2TiQov1if/miBr7RfzfKcDnRoQmw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com;
s=arcselector9901;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1;
bh=GxybWuvtlSKV1imc7sc78vNrSCTjgfupl0E0MTBq2SM=;
b=CEcj9qFvZfidYkHzTTBjzJmo+gp6/e/+wQCClKszQ649r3d6MaIuADVYBnGPMSSDValxa4QIs87Ym9sjyarDsf9nt3ppP+3/IyaPL9hoNKMVxAgtypNGQN8MEmCwSEFB4YE5y+z9tzWiC+3twwsZEUhk33ie5Vy4Kh/pwq+xxFcr0hZpLuIOVoD4e6oWs2Bv3GeYzTKwOCedZbNgUDbOxk/tt/d6udT6qIGTDy0k3C5CjoZURiFzNTJrewBsNNg33B7XmazpjRErMx9ikZDU/6/StP7UVn/3YTU7ZK04fSl+JfOKaBfcqguk8ammq59im8QVrex39l1nKfoeQxqlPA==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass
smtp.mailfrom=uchc.edu; dmarc=pass action=none header.from=uchc.edu;
dkim=pass header.d=uchc.edu; arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=uchc.onmicrosoft.com;
s=selector2-uchc-onmicrosoft-com;
h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck;
bh=GxybWuvtlSKV1imc7sc78vNrSCTjgfupl0E0MTBq2SM=;
b=UGhvyQN5Nvv3BdzSjPJgFJNf7r9AEPzxWCWW8vuZTKFVnUtxtmPLX/no3WexIFAgHI4GtKEL8qLVjH6chSObOyd6AfZCmKXcCsHi96sorcC+dhlLTr8JYObkp7mSjcy5Vdl4Y+Y6YgLm6ghvdbWWwzdqsp8QCFdqnx0yJ54+Zbs=
Thread-Topic: Debugging reason for python running unreasonably slow when
adding numbers
Thread-Index: AQHZVpHOz+gHPSfqQ0+JmpMenqTifq760VKAgAADywCAARF5AIAAClkAgAADHwCAAAI0gIAABr6AgAATVECAAAUepA==
In-Reply-To: <SJ0PR14MB586405B8C2F95F6F0B2888FEB9BF9@SJ0PR14MB5864.namprd14.prod.outlook.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-ms-publictraffictype: Email
x-ms-traffictypediagnostic: SJ0PR14MB5864:EE_|PH7PR14MB5366:EE_
x-ms-office365-filtering-correlation-id: cf467127-00f0-484b-6620-08db25788fa7
x-ms-exchange-senderadcheck: 1
x-ms-exchange-antispam-relay: 0
x-microsoft-antispam: BCL:0;
x-microsoft-antispam-message-info: NVcwrTDITSHG/TstDgmiMvqNVSc9y3HEnPb7DzyFItsmVJFDlwVVqFq/GdD3d8v+1ZLiPuNFqoqo5+sd+qUO0WL0RtGJVDgwboM/6/4KHLENRfsf16vgIE8YKytZIDwKPNfs3bTxPX6xPau5NllcV75vmdTGWmJbJPuDXtu0VvFMnMfSCMwvSjFj4mPm0tRStvhkOLf12xNnn4iODtVQVgdlSemne5ruQqJ4VKXTyNaFUfWP54C93U2eZ0q9SIi/D84fa4btF1oWTYAM3Tu8578dDoCzj3YeoBCS1x5WAuOczhz93tXe+/lVgS+2I2Em5BxIAE5jCdpPYpDjlSHxMg7vzqYyKpJZ2BhecNn7kfr+7Lqor5mennKwgrYx2M0SAFyGKnVtCF0mPXoAhxYsXr7RBmtdyJBLwqP/mJn/61INdSNMcJCwD7je2y1RZUFVEzbqRInnujlTRrzJFjZuzqAQA7gnaIzIBAfVHdppQAtyBdf5Uy/JJMYDrExboJZZRU4sneelmLaOXakXoPmw2WhR2RNpN1ktbu0sVs9atGWED+DoD0rYv9XNyJ/lElGU64yVXJnDui7JfZD2jayxKcwQF28352HjY0aIzhWBSoHqTetuuqLC297JaxTVRzhCEbkooj6T+leI0v/Geda24S7w/pM1+PoXVcGZsB4szvU8zNXmfzhzPGDXYRIqqir8
x-forefront-antispam-report: CIP:255.255.255.255; CTRY:; LANG:en; SCL:1; SRV:;
IPV:NLI; SFV:NSPM; H:SJ0PR14MB5864.namprd14.prod.outlook.com; PTR:; CAT:NONE;
SFS:(13230025)(396003)(39860400002)(376002)(136003)(366004)(346002)(451199018)(91956017)(786003)(316002)(478600001)(86362001)(55016003)(33656002)(122000001)(38070700005)(166002)(41320700001)(38100700002)(966005)(26005)(75432002)(7696005)(53546011)(9686003)(71200400001)(2940100002)(186003)(6506007)(66476007)(5660300002)(6916009)(64756008)(8676002)(76116006)(66446008)(66556008)(66946007)(2906002)(41300700001)(8936002)(52536014);
DIR:OUT; SFP:1101;
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: BAFUUsp7ScXgZiN2hjNCHkZRp8cJuKwl//jJhyYtPxCe80V8nVAm1c/glTA3
SQuj6QxCdIuIosRAx0MKOqNBuB08noS8GocNGgBC6fOBNtwk54AHxJ91BfoF
M6fKtre2tjgkPJbDHsE/0YXjkfqQBGhmAwj9zQUtb7oBB5Zm+YVu7kYTA75W
6nlt7yO0ivDjRfJIifwmdIMfwpcDuARDgEYPdDSFGbV79YAT7VKSz4V4g8ja
U3bP6MB4c7H8ObMeLqhEaQQ5haozfsavGOMrmZlw3jnzeT0bGpJ3iMFsjG59
dsQbzlobCF3AxcN2D6eabHr5+P5RSt+0aSAwq7lE8ggPcTE6u4n5ny25we4e
FNPohXKkOFpxPJO1rUIe81sYa44+PAuEJVUqTIc0nYVxXvz/mx8SRBlMUX3O
J16TOW2k6vfZGAAnkTeCC5+n15V3zu2ESDMfonG+gr8JA1esxWeHrPa2TfWy
H1eqL3K76FyMHNOk1QU6Y1vXQoJB7XAfj9tXdWTO8WnIVI0LfIs1pdQQAyc/
vHHBE4Iigwtl3/hNGXA8vReL47FehNY/rrv2MeSSSs0IyHW8yQ1BPu7+ul6E
LX/AyDcWWbuTZ1H4DMvLUm2Xb5/e5RtpGbirDGAfkcQBH7jfIHIB8CdFVyBe
1ML7Ft/Iy/cHQ8Tbunr6P9zHZdFSkBQfA139P7L5XJwllEmOfa3YtNgp9XIT
mB54I02wWfiRFyu0V/4OafiqPPAOvmbPO+BVvXIzcFpZpGW7bt5yCLfMsGY+
+u6E3oxP3EEScFtKJI577oVSVlCuFww1pLBWrZjG/PntD/7uKcXhgF/ZTqZA
G8gP0azFIyixlZBNWV9nMDkvWg5DbJXHK3Ie3j3puFngMQreuBRO7hEtZb9L
NssciF87T6TE5sSa0rltFUnN8PvGGkJwlN//HQBpcWK3Rmc8J6NdNXVTVT2I
vko/0lYYcGs8z+QJME2XuQVqDOSGGlQJDzbJQ8IdnS31C39BlYdNbvscmtR7
xDCRgPfaIT6g6Unumrq+whhBZyatPRWqDNeiLSjZpzfE9LsV1ze92L/Ttrm3
ftsLcurlH0MruyU3OrIsB80BmzbSwhzjsdrsUl2Lr7cMg7GS5sG23D8dGkvj
A3oBiSjrINrpRDt1bvwNdlQvc1tgtos1RrVFsHpZvE9b0nQZ5x3WsM0fBq0J
ZhCOUhHaB4gKIbLs/Ot1qCCB5geJniMqq/KRRN0lTc3giQlHLTdOPrGxPp6P
Fk1+bQmt32M6V/UY2khxoekDGe4IMAOu6NjCuYgfM3M4LwPPfq4dnC6U4zxW
gOYJDetnBcJ6rxO4QAs4TO2Q4J55LuY4Q4f7tpN8FQxu7NvPnkVez3AFzCwL
AeYYenYUdoPLxkY8+xrhYAJOi6ZDFrVRzEo5le/wIW4HP2OOqeJ/bYqxRNbF
5zFkJCK9h3qNQKbDopYqtTCRCOadKbpUt8DUq+uWTmzPTdF+Z098j172WHf+
UhQpMUBIONIiIFh7Cqdtg4sX7vK9cpxbit370GRD+A6xNbP2XYnF6git1/9e
Vn/hT+5pEsL1wwgEPGXG0qm8AOQkBoFR9VFQFGV+Jn05uFt2g9KVMvSUwfil
og==
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR14MB5864.namprd14.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cf467127-00f0-484b-6620-08db25788fa7
X-MS-Exchange-CrossTenant-originalarrivaltime: 15 Mar 2023 17:13:13.9160 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 5c82d83a-818a-4c16-b540-ded2344a7ad3
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 0ZJOHvg9QVboGlcBc66aHgm/fImRJqXtIWamcisOEFeTyW1+niYCBrTCY8k2ymgZaJBgmHbUhJeBbr8CtfrbZw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR14MB5366
X-Proofpoint-GUID: 2Gu4Bnm6dHlbD3Z5-5TruYlrz5ttxIw7
X-Proofpoint-ORIG-GUID: 2Gu4Bnm6dHlbD3Z5-5TruYlrz5ttxIw7
X-Proofpoint-Virus-Version: vendor=baseguard
engine=ICAP:2.0.254,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
definitions=2023-03-15_08,2023-03-15_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0
malwarescore=0 suspectscore=0
priorityscore=1501 mlxlogscore=869 mlxscore=0 bulkscore=0 phishscore=0
spamscore=0 impostorscore=0 lowpriorityscore=0 clxscore=1015 adultscore=0
classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2302240000
definitions=main-2303150144
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <SJ0PR14MB586493402E4FDD90DB504C53B9BF9@SJ0PR14MB5864.namprd14.prod.outlook.com>
X-Mailman-Original-References: <41f44497-bf2d-44e3-a99d-e7dddc1fa2c4@Spark>
<ebf2ea3b-9273-48ed-b6fc-29aee295d201@Spark>
<20230314215211.kq4ohsrnz6vyigxw@hjp.at>
<CAPTjJmqMfDAArxKG8pAdKk1P-kjzWKbd_SunCQORP+tJDNpZZA@mail.gmail.com>
<DB7PR07MB3916933F1AA08AA930F94F6F87BF9@DB7PR07MB3916.eurprd07.prod.outlook.com>
<CAPTjJmq_ZnQOKy=kW23tL_W-kEwpuBjy23noUxNvcnWGnpy_uw@mail.gmail.com>
<d4821123-a90d-5048-5dfa-b5782dc9074f@tompassin.net>
<CAPTjJmrJ26b=9UYo8gzT_yR=vtxtm1m5PsmBQNnsZVXaeYoBdQ@mail.gmail.com>
<DB7PR07MB3916029644F69D13A5021A8A87BF9@DB7PR07MB3916.eurprd07.prod.outlook.com>
<SJ0PR14MB586405B8C2F95F6F0B2888FEB9BF9@SJ0PR14MB5864.namprd14.prod.outlook.com>

by: Weatherby,Gerard - Wed, 15 Mar 2023 17:13 UTC

Moving the generator out:

import timeit

thedata = [i for i in range(1_000_000)]
def sum1():
s = 0
for i in thedata:
s += i
return s

def sum2():
return sum(thedata)

print('For Loop Sum:', timeit.timeit(sum1, number=100))
print( 'Built-in Sum:', timeit.timeit(sum2, number=100))

---
For Loop Sum: 6.984986504539847
Built-in Sum: 0.5175364706665277

From: Weatherby,Gerard <gweatherby@uchc.edu>
Date: Wednesday, March 15, 2023 at 1:09 PM
To: python-list@python.org <python-list@python.org>
Subject: Re: Debugging reason for python running unreasonably slow when adding numbers
Sum is faster than iteration in the general case.

Lifting a test program from Stack Overflow https://stackoverflow.com/questions/24578896/python-built-in-sum-function-vs-for-loop-performance,

import timeit

def sum1():
s = 0
for i in range(1000000):
s += i
return s

def sum2():
return sum(range(1000000))

print('For Loop Sum:', timeit.timeit(sum1, number=100))
print( 'Built-in Sum:', timeit.timeit(sum2, number=100))

---

For Loop Sum: 7.726335353218019
Built-in Sum: 1.0398506000638008

---

From: Python-list <python-list-bounces+gweatherby=uchc.edu@python.org> on behalf of David Raymond <David.Raymond@tomtom.com>
Date: Wednesday, March 15, 2023 at 11:46 AM
To: python-list@python.org <python-list@python.org>
Subject: RE: Debugging reason for python running unreasonably slow when adding numbers
*** Attention: This is an external email. Use caution responding, opening attachments or clicking on links. ***

> Then I'm very confused as to how things are being done, so I will shut
> up. There's not enough information here to give performance advice
> without actually being a subject-matter expert already.

Short version: In this specific case "weights" is a 5,147 element list of floats, and "input" is a 10 element list of integers which has the indexes of the 10 elements in weights that he wants to add up.

sum_ = 0
for key in input:
sum_ += weights[key]

sum_ = sum(weights[key] for key in input)

vs... other ways
--
https://urldefense.com/v3/__https://mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ikhJJxqJnllDHh6JKaMX8g2K-Ceq6ZiRDJxX7AbS-1AiBIrdAmA2qjBtYZbxel2mktyno1s9iJGo_zAl5alBIWxoVXE$<https://urldefense.com/v3/__https:/mail.python.org/mailman/listinfo/python-list__;!!Cn_UX_p3!ikhJJxqJnllDHh6JKaMX8g2K-Ceq6ZiRDJxX7AbS-1AiBIrdAmA2qjBtYZbxel2mktyno1s9iJGo_zAl5alBIWxoVXE$>

Subject	Author
Re: Debugging reason for python running unreasonably slow when adding	Weatherby,Gerard