Message-ID:

We are drowning in information but starved for knowledge. -- John Naisbitt, Megatrends

devel / comp.lang.python / Re: dict.get_deep()

Re: dict.get_deep()

<mailman.38.1649058050.20749.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=17710&group=comp.lang.python#17710

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: kirill.r...@devoteam.com (Kirill Ratkin)
Newsgroups: comp.lang.python
Subject: Re: dict.get_deep()
Date: Mon, 4 Apr 2022 10:40:47 +0300
Lines: 132
Message-ID: <mailman.38.1649058050.20749.python-list@python.org>
References: <CABbU2U8tdtEW8M7WB0E3nB_9cAHrZNY6cZYWgyjUHrUzMk8yhw@mail.gmail.com>
<525b0547-0ed2-776d-e862-b1d9193bd70e@devoteam.com>
<20220403194512.kcqo6zn5yjryvxz5@hjp.at>
<CABbU2U_oXPYYznc0u2rA02RbC6KJUcMqFnBO7j7kvmgCK5amGA@mail.gmail.com>
<1617313993.737226.1649028275904@mail.yahoo.com>
<33bfa854-004f-30e2-1ebe-b964797d1e73@devoteam.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de xobVUikwdYb3MD7ZR/42MAFOJu2D+L1k8JRRnRzg8ZFQ==
Return-Path: <kirill.ratkin@devoteam.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=devoteam.com header.i=@devoteam.com header.b=gbqh4C1a;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.001
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; 'argument': 0.04; 'def':
0.04; '2022': 0.05; 'containing': 0.05; 'parameter': 0.05; 'pypi':
0.05; 'matching': 0.07; 'sun,': 0.07; '"""': 0.09; 'else:': 0.09;
'modules.': 0.09; 'represents': 0.09; 'request?': 0.09; 'url-
ip:151.101.0.223/32': 0.09; 'url-ip:151.101.128.223/32': 0.09;
'url-ip:151.101.192.223/32': 0.09; 'url-ip:151.101.64.223/32':
0.09; '(so': 0.16; '>>>>': 0.16; '??.': 0.16; 'args[0]': 0.16;
'argument"': 0.16; 'arguments': 0.16; 'avi': 0.16; 'dict': 0.16;
'dictionary.': 0.16; 'empty"': 0.16; 'expects': 0.16; 'gross':
0.16; 'holzer': 0.16; 'key,': 0.16; 'key.': 0.16; 'missed': 0.16;
'mistake.': 0.16; 'n-1': 0.16; 'nested': 0.16; 'parameter?': 0.16;
'protocol.': 0.16; 'received:209.85.208.171': 0.16; 'received
:mail-lj1-f171.google.com': 0.16; 'something.': 0.16; 'stdlib':
0.16; 'tries': 0.16; 'understand.': 0.16; 'url:project': 0.16;
'url:pypi': 0.16; 'wrote:': 0.16; 'python': 0.16; 'values': 0.17;
"can't": 0.17; 'it?': 0.19; 'to:addr:python-list': 0.20;
'language': 0.21; 'maybe': 0.22; 'way.': 0.22; 'lines': 0.23;
'(and': 0.25; 'skip:- 10': 0.25; 'seems': 0.26; 'again,': 0.26;
'object': 0.26; 'pattern': 0.26; 'function': 0.27; 'expect': 0.28;
'ideas': 0.28; 'it,': 0.29; 'header:User-Agent:1': 0.30;
'approach': 0.31; 'deep': 0.31; 'default': 0.31; 'raise': 0.31;
'think': 0.32; 'question': 0.32; 'fine.': 0.32; 'objects': 0.32;
'point,': 0.32; 'python-list': 0.32; 'replies': 0.32; 'retrieve':
0.32; 'returning': 0.32; 'right,': 0.32; 'structure': 0.32; 'but':
0.32; "i'm": 0.33; 'there': 0.33; 'received:192.168.0': 0.33;
'march': 0.33; "didn't": 0.34; 'mean': 0.34; 'package': 0.34;
'header:In-Reply-To:1': 0.34; 'received:google.com': 0.34; 'one.':
0.35; 'subject:skip:d 10': 0.35; 'url:)': 0.35; 'yes,': 0.35;
'final': 0.35; 'request': 0.35; 'built': 0.36; 'errors': 0.36;
'those': 0.36; 'using': 0.37; "it's": 0.37; 'received:209.85':
0.37; 'class': 0.37; 'received:192.168': 0.37; 'way': 0.38;
'miss': 0.62; 'url-ip:151.101.0/24': 0.62; 'url-
ip:151.101.128/24': 0.62; 'url-ip:151.101.192/24': 0.62; 'url-
ip:151.101.64/24': 0.62; 'here': 0.62; 'subject': 0.63; 'true':
0.63; 'pass': 0.64; 'key': 0.64; 'about.': 0.64; 're:': 0.64;
'his': 0.65; 'process.': 0.65; 'similar': 0.65; 'look': 0.65;
'well': 0.65; 'back': 0.67; 'that,': 0.67; 'cc:': 0.69; 'deeply':
0.69; 'generator': 0.69; 'it:': 0.69; 'ps:': 0.69; 'remaining':
0.69; 'solutions': 0.70; 'too.': 0.70; 'street': 0.71; 'plus':
0.73; 'operate': 0.75; 'yourself': 0.75; 'skip:f 20': 0.75;
'finds': 0.76; 'implemented': 0.76; 'sent:': 0.78; 'returned':
0.81; 'eventually': 0.84; 'indexes': 0.84; 'obj': 0.84; 'sulla':
0.84; 'valid,': 0.84; 'behind': 0.88; 'interested,': 0.95
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=devoteam.com; s=google;
h=message-id:date:mime-version:user-agent:subject:content-language:to
:references:from:in-reply-to:content-transfer-encoding;
bh=mzwR5UqW0sYvnk5H5nMPz1YRhG49t3B8ShOKt/whclU=;
b=gbqh4C1aIMjPWNi6ZiJSNp0kwfNBImG8usSRra4YnSPV0J3PqyuD9L/JS0OOcs7h+P
eAuChF7DYHiVIAebnTgPjTttPcnadtx9bdSJZEO+Ud9rYgCvwzdbEkSl/bQ97YT5l6aW
eByo6FQK80noaE7B0yNj8EcV9e4dDVP7zsJf3H1GxFgn6CiHDoiyFIud6D2cXzNz/qLd
nJ+NPhFS8nbmNPZQqc/oqCUHdIzWeL0DQrlrPJ2gXNyuLbxiNUC/w3MWJskMjwwfICOQ
krIdyXbkKJYqrCRGqoltZwdvcTp2NQKNK18VdAdbp3Pq0VLwCqyj5CymkEhHr0buHNxB
9MXw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:message-id:date:mime-version:user-agent:subject
:content-language:to:references:from:in-reply-to
:content-transfer-encoding;
bh=mzwR5UqW0sYvnk5H5nMPz1YRhG49t3B8ShOKt/whclU=;
b=TKyLSKPdaFzWQXnQScslx6NfI0gjfAl4nb4DSRYBpAYgZQxwjJiXiaYXeBm3S40ZSl
Dh+CDws3rsIM2EMYEMAWyqe7cHj4+CDYBh1scw5L4lkWrsEtGV/VEBq6ZZ/lG40wkF13
ODn0feWjAu1jtQDDTRshD+B/GJOvZYw9K5ldE0FX8N5ezrJAfy/4jNY/KGNh4a62g6dN
PNTr2aHB05rJxqEMcVdT54eBx5Is/T6lfAvBcI1qAn8e2SbVCyXXFvb9yAMmHAvMbD5z
H1CPchoeEBmvIQw/ccgK/l1dxCQrJuI4ClrW8q0JfuVAhFxT+70WvIkJLdJ9ikkBOOwx
V4JQ==
X-Gm-Message-State: AOAM5333PEiYi2xMeh6BtzmLUqeAGleiYBkLr9QnnybfGvbnPL1tNlFJ
jiC9mTMNj+UohB5sBcOBkBtg3GVHQqOYi9eE11srwqcUJtVuAPuipbxCzhGcRj/NcvBCA5Qf5Zs
5f55HYPtNCQiH6+oDSnYqrsSsRv6mlZpxeW7jMgeEGo+p3baSPAB6rieZ3XPI2V2H6Ld2xYvmhd
TJ3TjO8A==
X-Google-Smtp-Source: ABdhPJxqCX1ANFDSLQ3Tzu/RPYdmf8EqRVGnw1Enjp6OTZKnC8tqB6ACZ7BVO4B1d3X5AZmjHMeVJw==
X-Received: by 2002:a2e:9e43:0:b0:24b:3c6:3832 with SMTP id
g3-20020a2e9e43000000b0024b03c63832mr8692369ljk.63.1649058048080;
Mon, 04 Apr 2022 00:40:48 -0700 (PDT)
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.7.0
Content-Language: ru
In-Reply-To: <1617313993.737226.1649028275904@mail.yahoo.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <33bfa854-004f-30e2-1ebe-b964797d1e73@devoteam.com>
X-Mailman-Original-References: <CABbU2U8tdtEW8M7WB0E3nB_9cAHrZNY6cZYWgyjUHrUzMk8yhw@mail.gmail.com>
<525b0547-0ed2-776d-e862-b1d9193bd70e@devoteam.com>
<20220403194512.kcqo6zn5yjryvxz5@hjp.at>
<CABbU2U_oXPYYznc0u2rA02RbC6KJUcMqFnBO7j7kvmgCK5amGA@mail.gmail.com>
<1617313993.737226.1649028275904@mail.yahoo.com>

by: Kirill Ratkin - Mon, 4 Apr 2022 07:40 UTC

Hello,

Yes, I misunderstood as well because started to think about pattern
matching which is good but this is not subject the question was about.

Sorry for my mistake.

Because question was about 'builtin' function which means stdlib
function implemented in python itself or even in C.

It seems, maybe I miss again, but we are talking about similar ideas
behind 'xpath' or 'jsonpath' or even 'LINQ'. We want to find some 'dsl'
which give us simple and safe way to get deeply nested values from dict.

There are several similar solutions on pypi
(https://pypi.org/project/dpath/, https://pypi.org/project/path-dict/).

But maybe (and maybe I miss again) we talk about language embedded
solution like operator ? or ??.

For example deep dict extraction could look like: street =
data["users"]?[0]?["address"]?["street"]?.

// BR

04.04.2022 2:24, Avi Gross via Python-list пишет:
> I may have misunderstood something.
> The original post in this subject sounded to ME likethey had nested dictionaries and wanted to be ableto ask a method in the first dictionary totake an unspecified number of arguments thatwould be successive keys and return the results.
> I mean if A was a dictionary containing saycities and it had an alphabetical index of lettersA to Z and those contained dictionaries ofsay last names as additional dictionaries andso on, then they wanted to perhaps say;
> A.getdeep("Boston", "S", "Smith", "Address", default="None")
> But the replies I am seeing look so different that I mayhave missed something as it seems more about usingpattern matching on the data used to make the dictionariesor something.
> So I was happy to see Marco suggesting a function alongthe lines of my thought process. But I have another thought.A stand-alone function along his lines might be fine. Buta method built into a general Dictionary class is anotherthing as it asks a method in one dictionary to march aroundinto other dictionaries. So I wonder if a better methodis sort of recursive.
> If you had a class like dictionary that had a getdeep function,and it got called with N arguments, and perhaps a namedargument supplying a default, then would it make sensefor the function checking to see if the FIRST argument canbe found as a key to the current dictionary.
> If arguments remain then it should expect to finda result that is a dictionary (or perhaps some otherobject that supports the getdeep() protocol and ask thatobject to operate on the N-1 remaining arguments, passingthe default along too.
> If the request is valid, after some iterations an object willhave a method invoked with a single argument (plus default)and a value passed back up the chain. For any errors alongthe way, the default would be returned.
> Is this closer to the spirit of the request? I view this versionof nested dictionaries as a sort of tree structure with variablebranches along the way. So an approach like this could makesense and perhaps Python could be updated eventually to havesome objects support such a protocol.
> Of course you could sort of do it yourself by subclassing somethingand making changes but that may not work for what is already asort of built-in data structure but could work for one of many variantsalready implemented in modules.
>
>
>
> -----Original Message-----
> From: Marco Sulla <Marco.Sulla.Python@gmail.com>
> To: Peter J. Holzer <hjp-python@hjp.at>
> Cc: python-list@python.org
> Sent: Sun, Apr 3, 2022 5:17 pm
> Subject: Re: dict.get_deep()
>
> On Sun, 3 Apr 2022 at 21:46, Peter J. Holzer <hjp-python@hjp.at> wrote:
>>>> data.get_deep("users", 0, "address", "street", default="second star")
>> Yep. Did that, too. Plus pass the final result through a function before
>> returning it.
> I didn't understand. Have you added a func parameter?
>
>> I'm not sure whether I considered this when I wrote it, but a function
>> has the advantage of working with every class which can be indexed. A
>> method must be implemented on any class (so at least dict and list to be
>> useful).
> You're right, but where to put it? I don't know if an iterableutil package
> exists. If included in the stdlib, I don't know where to put it. In
> collections maybe?
>
> PS: if you're interested, here is my implementation:
>
> def get_deep(self, *args, default=_sentinel):
> r"""
> Get a nested element of the dictionary.
>
> The method accepts multiple arguments or a single one. If a single
> argument is passed, it must be an iterable. This represents the
> keys or indexes of the nested element.
>
> The method first tries to get the value v1 of the dict using the
> first key. If it finds v1 and there's no other key, v1 is
> returned. Otherwise, the method tries to retrieve the value from v1
> associated with the second key/index, and so on.
>
> If in any point, for any reason, the value can't be retrieved, the
> `default` parameter is returned if specified. Otherwise, a
> KeyError or an IndexError is raised.
> """
>
> if len(args) == 1:
> single = True
>
> it_tpm = args[0]
>
> try:
> len(it_tpm)
> it = it_tpm
> except Exception:
> # maybe it's a generator
> try:
> it = tuple(it_tpm)
> except Exception:
> err = (
> f"`{self.get_deep.__name__}` called with a single " +
> "argument supports only iterables"
> )
>
> raise TypeError(err) from None
> else:
> it = args
> single = False
>
> if not it:
> if single:
> raise ValueError(
> f"`{self.get_deep.__name__}` argument is empty"
> )
> else:
> raise TypeError(
> f"`{self.get_deep.__name__}` expects at least one argument"
> )
>
> obj = self
>
> for k in it:
> try:
> obj = obj[k]
> except (KeyError, IndexError) as e:
> if default is _sentinel:
> raise e from None
>
> return default
>
> return obj