Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Don't compare floating point numbers solely for equality.

devel / comp.lang.python / Re: All permutations from 2 lists

SubjectAuthor
o Re: All permutations from 2 listsAvi Gross

1
Re: All permutations from 2 lists

<mailman.124.1646326845.2329.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=17301&group=comp.lang.python#17301

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: avigr...@verizon.net (Avi Gross)
Newsgroups: comp.lang.python
Subject: Re: All permutations from 2 lists
Date: Thu, 3 Mar 2022 17:00:39 +0000 (UTC)
Lines: 121
Message-ID: <mailman.124.1646326845.2329.python-list@python.org>
References: <1388685953.247061.1646275320253.ref@mail.yahoo.com>
<1388685953.247061.1646275320253@mail.yahoo.com>
<CACwCsY75Y4TtcY+xySf-UBoE=Mh3XbvQ7jdBaYfbhg7o_jnyGw@mail.gmail.com>
<1497529778.363953.1646326839989@mail.yahoo.com>
Reply-To: Avi Gross <avigross@verizon.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de oIzY6sW6LVLNBQ7MtIDKhQzSjXxY4iVST05NMuFlsHag==
Return-Path: <avigross@verizon.net>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=verizon.net header.i=@verizon.net header.b=r4o2AwXV;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.009
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'def': 0.04; '2022': 0.05;
'containing': 0.05; 'fairly': 0.05; 'is.': 0.05; 'explanation':
0.07; 'mar': 0.07; 'matches': 0.07; 'modules': 0.07; 'python.':
0.08; 'cc:addr:python-list': 0.09; 'describe': 0.09; 'expression':
0.09; 'filtering': 0.09; 'originally': 0.09; 'smaller': 0.09;
'those.': 0.09; 'that.': 0.15; 'another.': 0.16; 'answer.': 0.16;
'avi': 0.16; 'case:': 0.16; 'constraints.': 0.16; 'context.':
0.16; 'database.': 0.16; 'dict': 0.16; 'disclosed': 0.16;
'expectation': 0.16; 'frustration': 0.16; 'generates': 0.16;
'generating': 0.16; 'gross': 0.16; 'indeed': 0.16; 'large.': 0.16;
'nested': 0.16; 'similarities': 0.16; 'specify': 0.16;
'subject:lists': 0.16; 'vector': 0.16; 'words.': 0.16; 'wrote:':
0.16; 'problem': 0.16; 'values': 0.17; 'instead': 0.17; 'solve':
0.19; 'implement': 0.19; 'thu,': 0.19; 'cc:addr:python.org': 0.20;
'issue': 0.21; 'science': 0.22; 'version': 0.23; 'code': 0.23;
'anything': 0.25; 'skip:- 10': 0.25; 'saying': 0.25; 'cc:2**0':
0.25; 'anyone': 0.25; 'interests': 0.26; 'pattern': 0.26;
'solution.': 0.26; 'creating': 0.27; 'function': 0.27; 'sense':
0.28; 'version.': 0.28; 'example,': 0.28; 'wrong': 0.28;
'computer': 0.29; 'code,': 0.31; 'seem': 0.31; 'approach': 0.31;
'looked': 0.31; 'module': 0.31; 'think': 0.32; 'empty': 0.32;
'guess': 0.32; 'lists,': 0.32; 'python-list': 0.32; 'words,':
0.32; 'skip:2 10': 0.32; 'but': 0.32; 'there': 0.33; 'someone':
0.34; 'same': 0.34; 'work.': 0.34; 'header:In-Reply-To:1': 0.34;
'trying': 0.35; 'words': 0.35; 'people': 0.36; 'those': 0.36;
'lists': 0.37; 'presented': 0.37; 'subject:from': 0.37; 'using':
0.37; "it's": 0.37; 'hard': 0.37; 'others': 0.37; 'this.': 0.37;
'simply': 0.63; 'leading': 0.63; 'pass': 0.64; 'clear': 0.64;
'down': 0.64; 'imagine': 0.64; 're:': 0.64; 'source.': 0.64;
'your': 0.64; 'came': 0.65; 'produce': 0.65; 'saw': 0.65;
'similar': 0.65; 'named': 0.65; 'decided': 0.67; 'outside': 0.67;
'accept': 0.67; 'time,': 0.67; 'exactly': 0.68; 'items': 0.68;
'cc:': 0.69; 'natural': 0.69; 'presenting': 0.69; 'sequence':
0.69; 'square': 0.69; 'within': 0.69; 'solutions': 0.70; 'too.':
0.70; 'product': 0.71; 'little': 0.73; 'five': 0.75; 'cross':
0.76; 'out,': 0.78; 'sent:': 0.78; 'header:Reply-To:1': 0.79;
'position': 0.81; 'known': 0.84; 'need.': 0.84; 'brackets': 0.84;
'characters': 0.84; 'combinations': 0.84; 'converted': 0.84;
'dealt': 0.84; 'field,': 0.84; 'mysterious': 0.84; 'pairs': 0.84;
'received:74.6.132.123': 0.84;
'received:sonic314-13.consmr.mail.bf2.yahoo.com': 0.84;
'subject:All': 0.84; 'waited': 0.84; 'us,': 0.89;
'received:74.6.132': 0.91; 'skip:q 20': 0.95
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=verizon.net; s=a2048;
t=1646326842; bh=Nx2fsaVc4XvKcBQf7QpFizhfghBdTPNnTaf3NsNO1TU=;
h=Date:From:Reply-To:Cc:In-Reply-To:References:Subject:From:Subject:Reply-To;
b=r4o2AwXVPEwRv8P91IC0NYiLnoT45DnetDiZvmqNtG5ppCa73+ye6g8gHlQpshh92leLEXs3Ry+Nmro0NlsyeuzLhJ0xzpQqsfuedn8MnM/FBn71/KnX2tpU0Eps6k0SqUcBZ3S0ITpGM+7pMr3s7+p2rWxxNuLBeEJU6ibx58TNSeXGpzZz3pkF6fiX1SGBHE1lQIlVj2vD///UCPzRwMtD5H7Z4ClQsDJ9+xnCyNIoVOiMvOapR17zhMwGJgHmdzwLmEkObXKEvYyie51uCnFacfdLfuvW4tJHhQ0t7hgTrGYgUPF/gr+UW9uk6wgjJdHaya4tYjjXTbXNCTyZGQ==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048;
t=1646326842; bh=73CkQW/iGZX4LC55R5kvTwGXwfPUsJ4U5m8gOTUrJ0F=;
h=X-Sonic-MF:Date:From:Subject:From:Subject;
b=ozrouEEZeMQhr9Y5urd0mWmi69spyQczFRCpi6iO1kOzOL4yfsp7CyQk9Y5FGrVAzcu4UYSRzPrmbdoHKYjGrVBOEpSBRLyEX9GSYgK/PqfCobzPZICC0eugA/kMmviZe/rAVqWppuf8aLj4c+hmxnIj8FQTSVDlxjWB6rxmpnuSQDhWvXWwVSx+G3YVc9JYLkQ84upifqshzR+Ga8TlGD2ADIlAXjhxSMSNMUgnnWZfFNYIgx+zLIl1pSrb1cgcjLqRscpovoWKg548weA/7suRfOMQZSmlktrkVBd17tNFsqeKpfRkZDsP5t5ZLT2rKQDG63KOMHlMCsveUqqi1g==
X-YMail-OSG: geQXbL4VM1ntaZTa1l2_UykTAHnP9c6CyWAo.DIcZuz0VO8LbAajG53aKAXMOUq
9OaqpcXZGa0Y8hjAGQmo6bKalR9PyJTz6C5m35QPm8KruB54PRxMOAyurlkzhpGLXm3d9mn0zi1s
yy.qUIF8RTZBmE.8tWOJWSYNE4_oZCmME0u0tM_RcRhtqcy8QCF9vibS9BFtZ.v9aLcyPUvG3Vk4
4qBKpW78Iakh_0M9rQ4m2_FosvDzov3oYVrB2nbMpiSBBdJ3uUAwJPmsbeiyVetBQjIL3L3DoBmG
3eGkT_ZUKWmxCFRuny_uI4WxrNMer6v2_TfCjMdpvpv1AbUpPSp0Pdb2wz6t2uRc8wVRohcmTvR7
gTUREytJAAudM9W1Ux5uSJnUaJ8cMVKzEnWMVA5ighFvahnGk8uU6lfRzSVNFiPrj3LqQXUdCbv_
B4ivu5WxZv4mz7yhwo1UJX6UnaCVS3t9eTKGaFNgm0GTJ4d6D_pMYpbnlKuvzf5PTsr4yEdUN312
tmLr2.2zmJGJM77_O_uAxQL.avanWFJ2JNdiQxK0OimvxdqU1SGv6qsD0izTRqP8AtXOlou047K3
TVbmA9Y3QZ2YJ1.aMR3yfBTX589rEyV4SkIzwEn8yFPHYscwlLbuvCacFstKXbRfNs8dJfYzARej
lxXLL2j6t6.GexV3TckPiULme15s8vlgN0qiOfF_5.SF6n5knvFdOAWnC0szlwoY0F0R1FH1JSOS
sWkl5DiCZ0E8xxXBxuJ8TZq8ltNo7HDXQjjhwb1_ojD.Y4JGxuS9asUae4knVww0Mu6aLPi3mGzm
IpqDo24mQ1u6XS8ZFnYcnMfWNv2t2R17RuVfMMOZxh0Bjt962Vwl9cfGvO0q57JX4IOEiibemcaL
X4lVIRKOvPQ592ZOAruI3OpEmXi.xaU.P6jkOju1G5sU9SGHOKOohN1Rgoz3FpsGEQTpF_BpNnkv
KqjtOSj6FqAdQkswc8PoPbfLcavemh864G89aqtlDNEC9aFz1QPYwhIv70G4uu7KE3yhHGVgjH3r
.WoPhrXm7LocQVATxAMXbwArT3cUuJWCOOMm7ClfDIS1xAPgy8xA8NXRk559rlq6z413osPfh7zC
YFIDLg1JUJGMNE9u8XpP8svGTA2qRPG5hmmIkfKVstBRmCm8sSe0R2arj.wPUv1vH60y8JOxHLYT
cpi3cb0YnqxHKSQ0NNVlRTz6DIPVvCTcKoKha46dnWpPOy.WGV3WR_WuqvmGH3277lq.fVG04PMC
u099TTXMfNsF9Ep_YMnGMUHfoDZW2mRFxLkGUf4AMVdbGeaDmyXC8o.lFk1Eq.C9IUIZ5AMU5u3X
SW.Jao3jrJBgGR52O_lnpK1z7oDN_U1j4xa8JLDUTIfIaBPPnJur_d9hmwGi6Tu8LHHgwBDx5QAS
W.MUQl9s49TRzKbYkfCl7PZXbGySI8T9N9dNzO4hXTbKpWO17p15_Nd2KX7.0KwZN9rGEynjH2yu
z8bgeUDpmWfOaC3IMus2RDICvKKSX5EesdwdBD_2HTt5LDrkzaMFf9NYMEZ7tUuI1hLfuEfUoOGA
PElTg.qZXwIv60Qi9YT7f8s.NLgqEq54bF9pmOJUOc8K4qi_l6cKDCjZb5lZ2yW_zA8xUVXd1H0B
10E_qLhVwWsaTHNZX7Zt3eW4_PmTB9T1fveHRdEIAAJt1kxoHqrb02uUfnIlld8M7T1IIRge9eeC
Q7Q1MUe0e6kBs8okU3HnHKUtY7LIf4EDnDkYLbqLwcdu3wUaMUCF9.LkuEvRJ.Ts12rE3d457Uji
kvdYyX2fQ915FI.T200M9gV7mWtKLsTqaedV.dJ1FNxB4yOuUtdjMYpl7OYxCFxtO5uOdLwK3hh.
OJrO5S6bLWzC3jayvSzjg5DBX4ZKO5R0yme69wr9wf00IkL.PX6GnFtSg6yZGW1oEnUpli5FBdOH
2p.5uI4x1FPWX4ipvN4uttmJGyQTglcMhQhUs5lLg5j09x51PGV39W_GsyzKFvdoPfW4tXXLGuM.
DcERoSM19SudqkZIjUJh1GQM217Ufi9eUPJ15J5gKSbFuc7UDOWTUycHF8ujRxq5oWdTARRkBiyD
HSKf8K_xWedyLhDnGUvMhgdpJWJBhBBl23wvgsaqgEp.QhG7mCZwCas7LW5Jzumj4O6bznXw2TVp
q55MeD8lSIcJAS.CT9cDUjN8vuEKXoNxBeq2LLjD0z8e3Nkqa5PhC2ByFzSLE1kGRg2oXSOo-
X-Sonic-MF: <avigross@verizon.net>
In-Reply-To: <CACwCsY75Y4TtcY+xySf-UBoE=Mh3XbvQ7jdBaYfbhg7o_jnyGw@mail.gmail.com>
X-Mailer: WebService/1.1.19724 aolwebmail
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <1497529778.363953.1646326839989@mail.yahoo.com>
X-Mailman-Original-References: <1388685953.247061.1646275320253.ref@mail.yahoo.com>
<1388685953.247061.1646275320253@mail.yahoo.com>
<CACwCsY75Y4TtcY+xySf-UBoE=Mh3XbvQ7jdBaYfbhg7o_jnyGw@mail.gmail.com>
 by: Avi Gross - Thu, 3 Mar 2022 17:00 UTC

Larry,

That explanation made more sense and provided context.

I fully agree with you that generating the cross product of multiple lists can be messy and large and best avoided.

As an example, someone on an R forum presented their version of a way to see what are potential solutions to the game WORDLE at any given time, given the current constraints. The details are not important except that their process makes multiple vectors of the characters that can be allowed for letters "one" through "five" and then generates a data.frame of all combinations. Early in the game you know little and the number of combinations can be as high as 26*26*26*26*26 in the English version. Within a few moves, you may know more but even 15*18*... can be large. So you have data.frames with sometimes millions of rows that are then converted rowwise to five letter words to make a long vector than you query a huge dictionary for each word and produce a list of possible words. Now imagine the same game looking instead for 6 letter words and 7 letter words ...

I looked at it and decided it was the wrong approach and in, brief, made a much smaller dictionary containing only the five letter words, and made a regular expression that looked like"

"^[letters][^letters]S[more][^this]$"

The above has 5 matches that may be for a specific letter you know is there (the S in position 3) or a sequence of letters in square brackets saying any one of those match, or the same with a leading caret saying anything except those. You then simply use the R grep() function to search the list of valid 5-letter words using that pattern and in one sweep get them all without creating humongous data structures.

What you describe has some similarities as you searched for an alternate way to do something and it is now clearer why you did not immediately vocalize exactly what you anticipated. But your solution was not a solution to what anyone trying to help was working on. It was a solution to a different problem and what people would have had to know about how you were using a dictionary to pass to a mysterious function was not stated, till now. I would have appreciated it if you had simply stated you decided to use a different way and if anyone is curious, here it is.

For the rest of us, I think what we got from the exchange may vary. Some saw it as a natural fit with using something like a nested comprehension, albeit empty lists might need to be dealt with. Others saw a module designed to do such things as an answer. I saw other modules in numpy/pandas as reasonable. Some thought iterators were a part of a solution. The reality is that making permutations and combinations is a fairly common occurance in computer science and it can be expected that many implement one solution or another.

But looking at your code, I am amused that you seem to already have not individual lists but a dictionary of named lists. Code similar to what you show now could trivially have removed dictionary items that held only an empty list. And as I pointed out, some of the solutions we came up with that could generalize to any number of lists, happily would accept such a dictionary and generate all combinations.

My frustration was not about you asking how to solve a very reasonable problem in Python. It was about the process and what was disclosed and then the expectation that we should have known about things not shared. Certainly sharing too much is a problem too. Your title alone was very concrete asking about 2 lists. It is clear that was not quite your real need.

-----Original Message-----
From: Larry Martell <larry.martell@gmail.com>
To: Avi Gross <avigross@verizon.net>
Cc: python-list@python.org <python-list@python.org>
Sent: Thu, Mar 3, 2022 9:07 am
Subject: Re: All permutations from 2 lists

On Wed, Mar 2, 2022 at 9:42 PM Avi Gross via Python-list
<python-list@python.org> wrote:
>
> Larry,
>
> i waited patiently to see what others will write and perhaps see if you explain better what you need. You seem to gleefully swat down anything offered. So I am not tempted to engage.

But then you gave in to the temptation.

> And it is hard to guess as it is not clear what you will do with this.

In the interests of presenting a minimal example I clearly
oversimplified. This is my use case: I get a dict from an outside
source. The dict contains key/value pairs that I need to use to query
a mongodb database. When the values in the dict are all scalar I can
pass the dict directly into the query, e.g.:
self._db_conn[collection_name].find(query). But if any of the values
are lists that does not work. I need to query with something like the
cross product of all the lists. It's not a true product since if a
list is empty it means no filtering on that field, not no filtering on
all the fields.  Originally I did not know I could generate a single
query that did that. So I was trying to come up with a way to generate
a list of all the permutations and was going to issue a query for each
individually.  Clearly that would become very inefficient if the lists
were long or there were a lot of lists. I then found that I could
specify a list with the "$in" clause, hence my solution.

> def query_lfixer(query):
>    for k, v in query.items():
>        if type(v)==list:
>            query[k] = {"$in": v}
>    return query
>
> self._db_conn[collection_name].find(query_lfixer(query))
>
>
> So why did so many of us bother?

Indeed - so why did you bother?

1
server_pubkey.txt

rocksolid light 0.9.8
clearnet tor