Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

6 May, 2024: The networking issue during the past two days has been identified and appears to be fixed. Will keep monitoring.


devel / comp.lang.python / Implementation of an lru_cache() decorator that ignores the first argument

SubjectAuthor
* Implementation of an lru_cache() decorator that ignores the firstRobert Latest
+* Re: Implementation of an lru_cache() decorator that ignores the firstChris Angelico
|`* Re: Implementation of an lru_cache() decorator that ignores theRobert Latest
| `- Re: Implementation of an lru_cache() decorator that ignores the firstHeinrich Kruger
`- Re: Implementation of an lru_cache() decorator that ignores the firstdn

1
Implementation of an lru_cache() decorator that ignores the first argument

<jpjhmnFi048U1@mid.individual.net>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=19592&group=comp.lang.python#19592

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!usenet.goja.nl.eu.org!3.eu.feeder.erje.net!feeder.erje.net!news2.arglkargh.de!news.karotte.org!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: boblat...@yahoo.com (Robert Latest)
Newsgroups: comp.lang.python
Subject: Implementation of an lru_cache() decorator that ignores the first
argument
Date: 28 Sep 2022 18:22:15 GMT
Lines: 81
Message-ID: <jpjhmnFi048U1@mid.individual.net>
X-Trace: individual.net D+IgzxMtKy1XdPFcetpPJQ0HkyB9kELVMpLuRzk/l9wLMa50HB
Cancel-Lock: sha1:+T0idZmkMNqC/mniooDiKL21yHg=
User-Agent: slrn/1.0.3 (Linux)
 by: Robert Latest - Wed, 28 Sep 2022 18:22 UTC

Hi all,

in a (Flask) web application I often find that many equal (SQLAlchemy) queries
are executed across subsequent requests. So I tried to cache the results of
those queries on the module level like this:

@lru_cache()
def query_db(db, args):
# do the "expensive" query
return result

This obviously doesn't work because each request uses a new database session,
so the db argument always changes from one call to the next, triggering a new
query against the database. But even if that weren't so, the function would
keep returning the same value forever (unless it's kicked out of the cache) and
not reflect the (infrequent) changes on the database. So what I need is some
decorator that can be used like this:

@lru_ignore_first(timeout=10)
def query_db(db, args):
# do the "expensive" query
return result

This is what I came up with. I'm quite happy with it so far. Question: Am I
being too clever? is it too complicated? Am I overlooking something that will
come back and bite me later? Thanks for any comments!

from functools import wraps, lru_cache
from time import time, sleep

def lru_ignore_first(timeout=0, **lru_args):

class TimeCloak():
'''All instances compare equal until timeout expires'''
__slots__ = ('x', 't', 'timeout')

def __init__(self, timeout):
self.timeout = timeout
self.t = 0
self.x = None

def __hash__(self):
return self.t

def __eq__(self, other):
return self.t == other.t

def update(self, x):
self.x = x
if self.timeout:
t = int(time())
if t >= self.t + self.timeout:
self.t = t

cloak = TimeCloak(timeout)

def decorator(func):

@lru_cache(**lru_args)
def worker(cloak, *a, **b):
return func(cloak.x, *a, **b)

@wraps(func)
def wrapped(first, *a, **kw):
cloak.update(first)
return worker(cloak, *a, **kw)

return wrapped

return decorator

@lru_ignore_first(3)
def expensive(first, par):
'''This takes a long time'''
print('Expensive:', first, par)
return par * 2

for i in range(10):
r = expensive(i, 100)
sleep(1)
print(r)

Re: Implementation of an lru_cache() decorator that ignores the first argument

<mailman.493.1664399529.20444.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=19595&group=comp.lang.python#19595

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: ros...@gmail.com (Chris Angelico)
Newsgroups: comp.lang.python
Subject: Re: Implementation of an lru_cache() decorator that ignores the first
argument
Date: Thu, 29 Sep 2022 07:11:55 +1000
Lines: 41
Message-ID: <mailman.493.1664399529.20444.python-list@python.org>
References: <jpjhmnFi048U1@mid.individual.net>
<CAPTjJmqk9vs5-n2CT1RkTMbuC5ovkTT-f5jXm1Js51=hdFrp1w@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de O+TpcjGqHR0XNNj8aSJBMwToyN4CFcfGItmVB6KGj2RQ==
Return-Path: <rosuav@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=SWjc491n;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.014
X-Spam-Evidence: '*H*': 0.97; '*S*': 0.00; 'this:': 0.03; 'def': 0.04;
'2022': 0.05; 'fairly': 0.05; 'question:': 0.09; 'requests.':
0.09; 'subject:first': 0.09; 'approach,': 0.16; 'args):': 0.16;
'bite': 0.16; 'caching': 0.16; 'chrisa': 0.16; 'especially,':
0.16; 'executed': 0.16; 'expense': 0.16; 'from:addr:rosuav': 0.16;
'from:name:chris angelico': 0.16; 'hint': 0.16; 'later?': 0.16;
'lookup.': 0.16; 'received:209.85.218': 0.16; 'reusable': 0.16;
'sept': 0.16; 'simpler,': 0.16; 'subject:() ': 0.16; 'though:':
0.16; 'wrote:': 0.16; 'probably': 0.17; "aren't": 0.19; 'thu,':
0.19; 'to:addr:python-list': 0.20; 'code': 0.23; 'tried': 0.26;
'robert': 0.26; "isn't": 0.27; 'function': 0.27; 'series': 0.28;
'module': 0.31; 'putting': 0.31; 'think': 0.32; 'python-list':
0.32; 'simple,': 0.32; 'message-id:@mail.gmail.com': 0.32; 'but':
0.32; "i'm": 0.33; 'able': 0.34; 'header:In-Reply-To:1': 0.34;
'received:google.com': 0.34; 'running': 0.34; 'trying': 0.35;
'subject:that': 0.35; 'from:addr:gmail.com': 0.35; 'those': 0.36;
'really': 0.37; 'using': 0.37; "it's": 0.37; 'received:209.85':
0.37; 'way': 0.38; 'could': 0.38; 'means': 0.38; 'thanks': 0.38;
'received:209': 0.39; 'quite': 0.39; 'single': 0.39; 'mentioned':
0.39; 'something': 0.40; 'higher': 0.60; 'mode': 0.62; 'come':
0.62; 'overall': 0.64; 'transaction': 0.64; 'your': 0.64; 'top':
0.65; 'came': 0.65; 'back': 0.67; 'entire': 0.67; 'cost': 0.69;
'performance,': 0.69; 'times': 0.69; 'terms': 0.70; 'costs': 0.74;
'easy': 0.74; 'guarantee': 0.76; 'subsequent': 0.76; 'database':
0.80; 'queries': 0.81; '(first': 0.84; 'feature,': 0.84;
'performance.': 0.84; 'transaction.': 0.84; 'transactions': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=to:subject:message-id:date:from:in-reply-to:references:mime-version
:from:to:cc:subject:date;
bh=q6OoG3Ob59wc3NlBn763OAXpqbZDxjeqFrtgRpaKdiQ=;
b=SWjc491nNBnezvyv1Y9RQGBo/SPBwBMbjfFQTUqi+HDkWCr1Tts5RXihonmF+D6gdD
undEeK+dSwe08tcPijM6uZnLqVEaZ52GmG3eGuxVVrk4ITpeGiPgDzShFPQNx/q2NybA
DQqm/g23zfMXhw89BSGT80UREZcmG4Yc+NsZFOh8bfCwfj7LePUpbsIHI+RlfB68PsLD
RdxoukT22dkYaxnCs1wrOqfLmVq3Bg3rIc81h/5RYj/k33AuarbOyTwVkVM02FNKIihQ
A9p+Qm2oVXICDufNI57aWdGqC7ZWZfvLEoVkHP5VL/FF63em2DHnmUfYy6ij3cKdj+4B
G0kA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=to:subject:message-id:date:from:in-reply-to:references:mime-version
:x-gm-message-state:from:to:cc:subject:date;
bh=q6OoG3Ob59wc3NlBn763OAXpqbZDxjeqFrtgRpaKdiQ=;
b=iBJTevc3CBhZwq/8V0GCYRxblAKYM6gOItyBJShRrYKSjtGaDtrRB0dR4u+uJsO7Lv
MBTShUis7+yZGauW3EcVmhsAhHGcWSeOLQDK+m2dNnUtdpXcAeemQMCNv/BJ7E97XqRJ
uzeaF4+2fkCEh5GDkq/m2T63VG3kf5L52zaTabwaySWJJ5rDKnYWU1VzWAB/MQFTCH8V
rY507/E5AtfRxcU8vHAW1Xd/xHNHquzxzxtc1Q5lpdEtjBdpEOJe84fAThv6Iin7InFb
mJWe8NBLVAOAvTvhLlupF8UTVCWCrqSSdYawGu7BEL7/bZgce8Z8esqVVXVZw9i9d1wM
UhiA==
X-Gm-Message-State: ACrzQf3UwrS3eo7FbMXqk1fdTA4t1iqrpZ2c1OQQjvjWEx2id8h5t8Lg
3UOm2+06O+ZyvgNUmr+Q3d8yeaIJGty6NCRb3cZtUhL8D5k=
X-Google-Smtp-Source: AMsMyM4VPHNfEuOhnpvLHKmuvACrz+uepZcDQ8+Q6ivs7wPzLjJCFHk7Ejq2qZG64jUXhDNP/sATA2FsZv/+/Pj7DOo=
X-Received: by 2002:a17:907:8a23:b0:782:6a5:6dd5 with SMTP id
sc35-20020a1709078a2300b0078206a56dd5mr28020375ejc.581.1664399526995; Wed, 28
Sep 2022 14:12:06 -0700 (PDT)
In-Reply-To: <jpjhmnFi048U1@mid.individual.net>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAPTjJmqk9vs5-n2CT1RkTMbuC5ovkTT-f5jXm1Js51=hdFrp1w@mail.gmail.com>
X-Mailman-Original-References: <jpjhmnFi048U1@mid.individual.net>
 by: Chris Angelico - Wed, 28 Sep 2022 21:11 UTC

On Thu, 29 Sept 2022 at 05:36, Robert Latest via Python-list
<python-list@python.org> wrote:
> in a (Flask) web application I often find that many equal (SQLAlchemy) queries
> are executed across subsequent requests. So I tried to cache the results of
> those queries on the module level like this:
>
> @lru_cache()
> def query_db(db, args):
> # do the "expensive" query
> return result
>
> ...
> This is what I came up with. I'm quite happy with it so far. Question: Am I
> being too clever? is it too complicated? Am I overlooking something that will
> come back and bite me later? Thanks for any comments!
>
> def lru_ignore_first(timeout=0, **lru_args):
> ...

I think this code is fairly specific to what you're doing, which means
the decorator won't be as reusable (first hint of that is the entire
"timeout" feature, which isn't mentioned at all in the function's
name). So it's probably not worth trying to do this multi-layered
approach, and it would be as effective, and a lot simpler, to just
have code at the top of the query_db function to do the cache lookup.
But you may find that your database is *itself* able to do this
caching for you, and it will know when to evict from cache. If you
really have to do it yourself, keep it really really simple, but have
an easy way *in your own code* to do the cache purge; that way, you
guarantee correctness, even at the expense of some performance.

In terms of overall database performance, though: are you using
transactions correctly? With PostgreSQL, especially, the cost of doing
a series of queries in one transaction is barely higher than doing a
single query in a transaction; or, putting it the other way around,
doing several sequential transactions costs several times as much as
doing one combined transaction. Check to see that you aren't
accidentally running in autocommit mode or anything. It could save you
a lot of hassle!

ChrisA

Re: Implementation of an lru_cache() decorator that ignores the first argument

<mailman.494.1664404706.20444.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=19596&group=comp.lang.python#19596

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: PythonL...@DancesWithMice.info (dn)
Newsgroups: comp.lang.python
Subject: Re: Implementation of an lru_cache() decorator that ignores the first
argument
Date: Thu, 29 Sep 2022 11:38:16 +1300
Organization: DWM
Lines: 28
Message-ID: <mailman.494.1664404706.20444.python-list@python.org>
References: <jpjhmnFi048U1@mid.individual.net>
<bec95539-b431-a92a-b556-ee7713ca9a0d@DancesWithMice.info>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
X-Trace: news.uni-berlin.de dJm1/eL3uZJ+L+44kCiqZg/UeaGCLQk1wUAdammX/97w==
Return-Path: <PythonList@DancesWithMice.info>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=danceswithmice.info header.i=@danceswithmice.info
header.b=k3s9EB06; dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.033
X-Spam-Evidence: '*H*': 0.93; '*S*': 0.00; '(which': 0.04; '=dn':
0.09; 'from:addr:danceswithmice.info': 0.09;
'from:addr:pythonlist': 0.09; 'question:': 0.09; 'subject:first':
0.09; 'bite': 0.16; 'later?': 0.16; 'message-
id:@DancesWithMice.info': 0.16; 'received:51.254': 0.16;
'received:51.254.211': 0.16; 'received:51.254.211.219': 0.16;
'received:cloud': 0.16; 'received:rangi.cloud': 0.16; 'solved':
0.16; 'subject:() ': 0.16; 'wrote:': 0.16; 'problem': 0.16;
"can't": 0.17; 'to:addr:python-list': 0.20; 'seems': 0.26;
'robert': 0.26; 'bit': 0.27; 'header:User-Agent:1': 0.30;
'header:Organization:1': 0.31; 'python-list': 0.32;
'received:192.168.1': 0.32; "i'm": 0.33; 'header:In-Reply-To:1':
0.34; 'subject:that': 0.35; 'received:192.168': 0.37; 'way': 0.38;
'put': 0.38; 'thanks': 0.38; 'quite': 0.39; 'something': 0.40;
'likely': 0.61; "there's": 0.61; 'come': 0.62; 'skip:r 20': 0.64;
'received:51': 0.64; 'upon': 0.64; 'came': 0.65; 'skip:t 20':
0.66; 'back': 0.67; 'depending': 0.70; 'front': 0.70; 'easy':
0.74; '....': 0.76; '"yes",': 0.84; 'say,': 0.84; 'subject: \n ':
0.84; 'criticism': 0.91
DKIM-Filter: OpenDKIM Filter v2.11.0 vps.rangi.cloud 0CD737D99
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=danceswithmice.info;
s=staff; t=1664404704;
bh=uimDnRjKtyEc4tFVSeRtd/+NjEvTkTqbCvHWUfeHm1A=;
h=Date:Subject:To:References:From:In-Reply-To:From;
b=k3s9EB06kCGwozDJbPgUGFlCPfTga6YDihb9M1E2fnf8pnymRR6l0iO6oovdX/QIO
bEERjKrY9XqZqQVEl7AKYA+U5cneKX3YpFwVqkfdedrdTmE3LTwcmZOgeamAgEWJxw
399NwytRQFUt6Y2aPflUC5UWp6LAEli2DAjS9Bkwy0AFAxwKKKuAU+UUvag8QWW0ru
H+C13iHFES0ZIfVSXLuT+0Dcndys3HMn3kHRPlI+W9iEuuaqR7Tin/U9L2J2pMCblV
lAcaHr5Vkl+tCHDogaQD+dNwFOvSl/5slfNz85BLNH8x001/1Re1p2x5smpmvCTpgm
My3gVy1ncKYTw==
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
Thunderbird/91.13.1
Content-Language: en-GB
In-Reply-To: <jpjhmnFi048U1@mid.individual.net>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <bec95539-b431-a92a-b556-ee7713ca9a0d@DancesWithMice.info>
X-Mailman-Original-References: <jpjhmnFi048U1@mid.individual.net>
 by: dn - Wed, 28 Sep 2022 22:38 UTC

On 29/09/2022 07.22, Robert Latest via Python-list wrote:
....

> This is what I came up with. I'm quite happy with it so far. Question: Am I
> being too clever? is it too complicated? Am I overlooking something that will
> come back and bite me later? Thanks for any comments!

Thank you for the chuckle: "Yes", you are clever; and "yes", this is
likely a bit too clever (IMHO).

The impression is that LRU will put something more-concrete, 'in front
of' SQLAlchemy - which is more abstract, and which is in-turn 'in front
of' the RDBMS (which is concrete...). Is this the code-smell making
one's nose suspicious?

The criticism is of SQLAlchemy. If the problem can't be solved with that
tool, perhaps it is not the right-tool-for-the-job...

Bias: With decades of SQL/RDBMS experience, it is easy to say, "drop the
tool".

+1 @Chris: depending upon how many transactions-between, it seems likely
find that the RDBMS will cache sufficiently, as SOP.

YMMV, ie there's only one way to find-out!
--
Regards,
=dn

Re: Implementation of an lru_cache() decorator that ignores the first argument

<jpkrlsFo220U1@mid.individual.net>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=19598&group=comp.lang.python#19598

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!individual.net!not-for-mail
From: boblat...@yahoo.com (Robert Latest)
Newsgroups: comp.lang.python
Subject: Re: Implementation of an lru_cache() decorator that ignores the
first argument
Date: 29 Sep 2022 06:18:36 GMT
Lines: 22
Message-ID: <jpkrlsFo220U1@mid.individual.net>
References: <jpjhmnFi048U1@mid.individual.net>
<CAPTjJmqk9vs5-n2CT1RkTMbuC5ovkTT-f5jXm1Js51=hdFrp1w@mail.gmail.com>
<mailman.493.1664399529.20444.python-list@python.org>
X-Trace: individual.net 8HGKEzPPhxvQT5ZXIaOPmQjQbLo66BX42W1Ed6XHi4L5ECpeY/
Cancel-Lock: sha1:XJn9BJZBjy5jV1oYT15PguxUph8=
User-Agent: slrn/1.0.3 (Linux)
 by: Robert Latest - Thu, 29 Sep 2022 06:18 UTC

Hi Chris and dh,

thanks for your --as usually-- thoughtful and interesting answers. Indeed, when
doing these web applications I find that there are several layers of useful,
maybe less useful, and unknown caching. Many of my requests rely on a
notoriously unreliable read-only database outside of my control, so I cache the
required data into a local DB on my server, then I do some in-memory caching of
expensive data plots because I haven't figured out how to reliably exploit the
client-side caching ... then every middleware on that path may or may not
implement its own version of clever or not-so-clever caching. Probably not a
good idea to try and outsmart that by adding yet another thing that may break
or not be up-to-date at the wrong moment.

That said, the only caching that SQLAlchemy does (to my knowledge) is that it
stores retrieved DB items by their primary keys in the session. Not worth much
since the session gets created and dumped on each request by SQA's unit of work
paradigm. But the DB backend itself may be caching repeated queries.

Back to Python-theory: The "Cloak" object is the only way I could think of to
sneak changing data past lru_cache's key lookup mechanism. Is there some other
method? Just curious.

Re: Implementation of an lru_cache() decorator that ignores the first argument

<mailman.507.1664562752.20444.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=19617&group=comp.lang.python#19617

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: heindsi...@kruger.dev (Heinrich Kruger)
Newsgroups: comp.lang.python
Subject: Re: Implementation of an lru_cache() decorator that ignores the first
argument
Date: Fri, 30 Sep 2022 18:32:17 +0000
Lines: 114
Message-ID: <mailman.507.1664562752.20444.python-list@python.org>
References: <jpjhmnFi048U1@mid.individual.net>
<CAPTjJmqk9vs5-n2CT1RkTMbuC5ovkTT-f5jXm1Js51=hdFrp1w@mail.gmail.com>
<mailman.493.1664399529.20444.python-list@python.org>
<jpkrlsFo220U1@mid.individual.net>
<VxwwUMut0HQx-WU8_QteznSThRbqj1-FewPA73lKwhfbiuKsWPzFyGuh5MOjUekwKyETR572nB8rq2HfPeMnlsRwVioj9EFbtlLKfAmSN0I=@kruger.dev>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de OdamUIue0wNL4mkQeiJwpghums6osIbPNwj4ydryJ+qg==
Return-Path: <heindsight@kruger.dev>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=kruger.dev header.i=@kruger.dev header.b=yYX/X3Xu;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'this:': 0.03; 'def': 0.04;
'2022': 0.05; '(to': 0.07; 'backend': 0.09; 'cc:addr:python-list':
0.09; 'lookup': 0.09; 'plots': 0.09; 'skip:_ 20': 0.09;
'subject:first': 0.09; 'though.': 0.09; 'timeout': 0.09; 'cc:no
real name:2**0': 0.14; 'import': 0.15; 'url:mailman': 0.15; '(1,':
0.16; 'caching': 0.16; 'client-side': 0.16; 'exploit': 0.16;
'layers': 0.16; 'mechanism.': 0.16; 'method?': 0.16; 'paradigm.':
0.16; 'queries.': 0.16; 'repeated': 0.16; 'server,': 0.16;
'subject:() ': 0.16; 'wrapper': 0.16; 'wrote:': 0.16;
'applications': 0.17; 'probably': 0.17; 'implement': 0.19;
'cc:addr:python.org': 0.20; 'first,': 0.22; 'maybe': 0.22;
'version': 0.23; 'idea': 0.24; 'past': 0.25; 'url-
ip:188.166.95.178/32': 0.25; 'url-ip:188.166.95/24': 0.25;
'url:listinfo': 0.25; 'cc:2**0': 0.25; 'url-ip:188.166/16': 0.25;
'object': 0.26; 'received:185.70': 0.26; 'robert': 0.26; 'local':
0.27; 'function': 0.27; 'chris': 0.28; 'output': 0.28; 'example,':
0.28; 'requests': 0.28; 'wrong': 0.28; 'url-ip:188/8': 0.31;
'think': 0.32; 'python-list': 0.32; 'received:185.70.40': 0.32;
'said,': 0.32; 'unknown': 0.32; 'but': 0.32; 'there': 0.33;
'path': 0.33; 'header:In-Reply-To:1': 0.34; 'received:ch': 0.35;
'subject:that': 0.35; 'request': 0.35; 'necessarily': 0.37;
"it's": 0.37; 'way': 0.38; 'could': 0.38; 'thanks': 0.38;
'adding': 0.39; 'use': 0.39; 'break': 0.39; 'something': 0.40;
'try': 0.40; 'should': 0.40; 'key': 0.64; 'to:addr:yahoo.com':
0.64; 'your': 0.64; 'required': 0.65; 'less': 0.65; 'back': 0.67;
'outside': 0.67; 'primary': 0.67; 'items': 0.68; 'control,': 0.69;
'following:': 0.69; 'stores': 0.69; 'interesting': 0.71; 'skip:f
30': 0.71; 'received:185': 0.71; 'database': 0.80; 'unit': 0.81;
'header:Received:2': 0.84; '(first,': 0.84; 'expensive': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=kruger.dev;
s=protonmail3; t=1664562743; x=1664821943;
bh=0Mp1tjrNhPCVv0jlZqsue9c986mnB91+nkBBDqsHKIg=;
h=Date:To:From:Cc:Subject:Message-ID:In-Reply-To:References:
Feedback-ID:From:To:Cc:Date:Subject:Reply-To:Feedback-ID:
Message-ID;
b=yYX/X3XuYSv2MdrW0KtHUH8xi8fDYFAFoMxbDqg9kuX+ExqaVC5ccC02vktpJBVHc
3KtlBK7/aF0Kz6dicucaX2U/8QZQwhWXs1NQA49V0VC/lEu0uKDVtRPXQuEtfS5DSx
8iDfUOgvyizqbz8q+BYWjGgHctcuL6iEIshMI+oOapvvCyvmuu05KW0k0rC0n23Bg+
ulAqyresel/du6uE2RRtwQqy44TBuWZcCPi/kXC1UpKrg603Mnxl2l/+leDLvlJdOF
6NtiANEVPhwNqMzAD2AkVwHX1G6HO2Kw1CYCMljdUl+gIpQRcz4BhsQKORVkewcA7A
s1pok28VkC3Og==
In-Reply-To: <jpkrlsFo220U1@mid.individual.net>
Feedback-ID: 12191497:user:proton
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <VxwwUMut0HQx-WU8_QteznSThRbqj1-FewPA73lKwhfbiuKsWPzFyGuh5MOjUekwKyETR572nB8rq2HfPeMnlsRwVioj9EFbtlLKfAmSN0I=@kruger.dev>
X-Mailman-Original-References: <jpjhmnFi048U1@mid.individual.net>
<CAPTjJmqk9vs5-n2CT1RkTMbuC5ovkTT-f5jXm1Js51=hdFrp1w@mail.gmail.com>
<mailman.493.1664399529.20444.python-list@python.org>
<jpkrlsFo220U1@mid.individual.net>
 by: Heinrich Kruger - Fri, 30 Sep 2022 18:32 UTC

On Thursday, September 29th, 2022 at 07:18, Robert Latest via Python-list <python-list@python.org> wrote:

> Hi Chris and dh,
>
> thanks for your --as usually-- thoughtful and interesting answers. Indeed, when
> doing these web applications I find that there are several layers of useful,
> maybe less useful, and unknown caching. Many of my requests rely on a
> notoriously unreliable read-only database outside of my control, so I cache the
> required data into a local DB on my server, then I do some in-memory caching of
> expensive data plots because I haven't figured out how to reliably exploit the
> client-side caching ... then every middleware on that path may or may not
> implement its own version of clever or not-so-clever caching. Probably not a
> good idea to try and outsmart that by adding yet another thing that may break
> or not be up-to-date at the wrong moment.
>
> That said, the only caching that SQLAlchemy does (to my knowledge) is that it
> stores retrieved DB items by their primary keys in the session. Not worth much
> since the session gets created and dumped on each request by SQA's unit of work
> paradigm. But the DB backend itself may be caching repeated queries.
>
> Back to Python-theory: The "Cloak" object is the only way I could think of to
> sneak changing data past lru_cache's key lookup mechanism. Is there some other
> method? Just curious.
>
> --
> https://mail.python.org/mailman/listinfo/python-list

You could use closures. For example, something like this:

import functools
import time

def my_cache(timeout):
start = time.monotonic()

def cache_decorator(func):
wrapper = _my_cache_wrapper(func, timeout, start)
return functools.update_wrapper(wrapper, func)

return cache_decorator

def _my_cache_wrapper(func, timeout, start):
first = None

@functools.cache
def _cached(timeout_factor, *args):
print("In the _cached function")
return func(first, *args)

def wrapper(*args):
print("In the wrapper")
nonlocal first
first, *rest = args

elapsed = time.monotonic() - start
timeout_factor = elapsed // timeout

return _cached(timeout_factor, *rest)

return wrapper

@my_cache(3)
def expensive(first, second, third):
print("In the expensive function")
return (first, second, third)

if __name__ == "__main__":
print(expensive(1, 2, 3))
print()
time.sleep(2)
print(expensive(2, 2, 3))
print()
time.sleep(2)
print(expensive(3, 2, 3))

This should output the following:

In the wrapper
In the _cached function
In the expensive function
(1, 2, 3)

In the wrapper
(1, 2, 3)

In the wrapper
In the _cached function
In the expensive function
(3, 2, 3)

It's not necessarily better than your version though. :D

Kind regards,
Heinrich Kruger

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor