Message-ID:

The reason computer chips are so small is computers don't eat much.

devel / comp.lang.python / Re: Select columns based on dates - Pandas

Select columns based on dates - Pandas

<9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com>

https://www.novabbs.com/devel/article-flat.php?id=14919&group=comp.lang.python#14919

copy link Newsgroups: comp.lang.python

X-Received: by 2002:ac8:5845:: with SMTP id h5mr4562377qth.91.1630610911574;
Thu, 02 Sep 2021 12:28:31 -0700 (PDT)
X-Received: by 2002:ac8:6c3:: with SMTP id j3mr4518870qth.182.1630610911322;
Thu, 02 Sep 2021 12:28:31 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.lang.python
Date: Thu, 2 Sep 2021 12:28:31 -0700 (PDT)
Injection-Info: google-groups.googlegroups.com; posting-host=66.76.20.130; posting-account=1YW3JAoAAAB0wczvpXQKtnF7cFYul9GA
NNTP-Posting-Host: 66.76.20.130
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com>
Subject: Select columns based on dates - Pandas
From: medina...@gmail.com (Richard Medina)
Injection-Date: Thu, 02 Sep 2021 19:28:31 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 10

by: Richard Medina - Thu, 2 Sep 2021 19:28 UTC

Re: Select columns based on dates - Pandas

<mailman.604.1630688212.4164.python-list@python.org>

copy mid

https://www.novabbs.com/devel/article-flat.php?id=14956&group=comp.lang.python#14956

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: martinp....@gmail.com (Martin Di Paola)
Newsgroups: comp.lang.python
Subject: Re: Select columns based on dates - Pandas
Date: Fri, 3 Sep 2021 16:56:44 +0000
Lines: 75
Message-ID: <mailman.604.1630688212.4164.python-list@python.org>
References: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com>
<20210903165644.kpgyvpits5hjlmbc@gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii; format=flowed
X-Trace: news.uni-berlin.de KJfX4D4Xlyxpq3bdcRB0WgqK7kZY/l89FSq5eqeKmPGQ==
Return-Path: <martinp.dipaola@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=oaSowiL4;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.012
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'this:': 0.03; 'row': 0.05;
'filter': 0.07; 'url:mailman': 0.09; '-0700,': 0.09; 'pandas':
0.09; 'received:209.85.219': 0.09; 'represents': 0.09; '02,':
0.16; 'columns': 0.16; 'dataset': 0.16; 'frame': 0.16; 'objects.':
0.16; 'subject:Pandas': 0.16; 'wrote:': 0.16; 'url:listinfo':
0.16; 'message-id:@gmail.com': 0.19; 'thu,': 0.20; 'url-
ip:188.166.95.178/32': 0.20; 'url-ip:188.166.95/24': 0.20; 'url-
ip:188.166/16': 0.23; 'to:addr:python-list': 0.23; 'header': 0.24;
'single': 0.28; 'url-ip:188/8': 0.29; '2019,': 0.31; 'but': 0.31;
'thanks,': 0.31; 'march': 0.32; '(this': 0.32; 'assume': 0.32;
'sep': 0.32; 'header:In-Reply-To:1': 0.33; 'received:google.com':
0.34; 'following': 0.35; '22,': 0.35; 'understood': 0.35;
'from:addr:gmail.com': 0.35; 'cases': 0.37; 'received:209.85':
0.38; 'received:209': 0.38; 'could': 0.40; "skip:' 10": 0.40;
'range': 0.64; 'per': 0.64; 'your': 0.64; 'covid-19': 0.65; '20,':
0.71; '2021': 0.84; '(three': 0.84; 'martin.': 0.84; 'medina':
0.84; 'strings': 0.84; 'subject:based': 0.84; 'country': 0.88;
'format:': 0.93
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=date:from:to:subject:message-id:mail-followup-to:'user-agent
:references:mime-version:content-disposition:in-reply-to;
bh=olkxlPK0lHaAuC9J6C938gnSvGgC4pEze18OeaDQwQo=;
b=oaSowiL4Bch3+OXq1WLSDguhRcHCqbGEc9xtmD0IG7KMYA5UbbVnFlqvYabVVbVJGi
ws7V9bef7teZ3cTcikntPupw2vgXs0Dl3gd62EcHRswHLlYcmwaHRwiKbWwNsA6YdI9L
p6/RfNuF4iW4yETghlI+8ETyWbwWus6Gwl0tdmaPHXtTbBfe4AEn7Y/EUqCWbug/yIC/
NqtP6bFR3ydMO2MIB5Eq1sNBdsN7wPpSdmIypmnf6NXThNJOY2Bo1t7W9pyIRUY6cjXR
G4LK0pSjgYmG5EPjJS7KqPZkzHpqMzTflMBxCCsQPXw2lRWnzhfFbIU6HJ+orNd0ca/5
plkg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:date:from:to:subject:message-id:mail-followup-to
:'user-agent:references:mime-version:content-disposition:in-reply-to;
bh=olkxlPK0lHaAuC9J6C938gnSvGgC4pEze18OeaDQwQo=;
b=AoPdqmZeUHRpLjoWffq1R/z4LqdwmeGXN88lXR7G/CzUrOZcrYZKmWZPkrjPDebR8h
D5qiCjZeeMKpuwcWox2tI0cgV5L6M99d3ocoJzF84bNZ6lyQvM7kepAtDb9ddIi43wYU
hyW/UazbQGMNnSTpo005HRVwMPG8qWf3KATSd6TtK6Av+Sju1vkhDieIyv5DhF1Ex158
GTa6S+8EgbnhW16DwssD4uXur1BJByQBCRT0OMjgsvQx3Oyz4hNJ/jOEqcX1uE3mOv4u
iG7WWlyKfMJpFXpbtOW3a8qEsFLf/J9vtCD5Q5pqY5waePQDCOcA49R+PPEJ+nZFEpno
ZolA==
X-Gm-Message-State: AOAM531czy4Zi9BtdvXYADSqKWWKt+0UTR96Govu+Iqym3mWFEy4Ijy3
xA7BtQsOpSvhX9hxQfg/eNRsA4YrI3HRzw==
X-Google-Smtp-Source: ABdhPJwtCOha9jCqa4dPM/g+N1KNrVrt1jT5e0sCWBxB3Z45qAbHZoBC6jltSu18pE3gsWJzOJk70A==
X-Received: by 2002:ad4:5cc2:: with SMTP id iu2mr341qvb.41.1630688209965;
Fri, 03 Sep 2021 09:56:49 -0700 (PDT)
Mail-Followup-To: python-list@python.org
'User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:52.0) Gecko/20100101
Thunderbird/52.5.2 Lightning/5.4.5.2'
Content-Disposition: inline
In-Reply-To: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com>
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.34
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <20210903165644.kpgyvpits5hjlmbc@gmail.com>
X-Mailman-Original-References: <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com>

by: Martin Di Paola - Fri, 3 Sep 2021 16:56 UTC

You may want to reshape the dataset to a tidy format: Pandas works
better with that format.

Let's assume the following dataset (this is what I understood from your
message):

In [34]: df = pd.DataFrame({
....: 'Country': ['us', 'uk', 'it'],
....: '01/01/2019': [10, 20, 30],
....: '02/01/2019': [12, 22, 32],
....: '03/01/2019': [14, 24, 34],
....: })

In [35]: df
Out[35]:
Country 01/01/2019 02/01/2019 03/01/2019
0 us 10 12 14
1 uk 20 22 24
2 it 30 32 34

Then, reshape it to a tidy format. Notice how each row now represents
a single measure.

In [43]: pd.melt(df, id_vars=['Country'], var_name='Date',
value_name='Cases')
Out[43]:
Country Date Cases
0 us 01/01/2019 10
1 uk 01/01/2019 20
2 it 01/01/2019 30
3 us 02/01/2019 12
4 uk 02/01/2019 22
5 it 02/01/2019 32
6 us 03/01/2019 14
7 uk 03/01/2019 24
8 it 03/01/2019 34

I used strings to represent the dates but it is much handy work
with real date objects.

In [44]: df2 = _
In [45]: df2['Date'] = pd.to_datetime(df2['Date'])

Now we can filter by date:

In [50]: df2[df2['Date'] < '2019-03-01']
Out[50]:
Country Date Cases
0 us 2019-01-01 10
1 uk 2019-01-01 20
2 it 2019-01-01 30
3 us 2019-02-01 12
4 uk 2019-02-01 22
5 it 2019-02-01 32

With that you could create three dataframes, one per month.

Thanks,
Martin.

On Thu, Sep 02, 2021 at 12:28:31PM -0700, Richard Medina wrote:
>Hello, forum,
>I have a data frame with covid-19 cases per month from 2019 - 2021 like a header like this:
>
>Country, 01/01/2019, 2/01/2019, 01/02/2019, 3/01/2019, ... 01/01/2021, 2/01/2021, 01/02/2021, 3/01/2021
>
>I want to filter my data frame for columns of a specific month range of march to September of 2019, 2020, and 2021 only (three data frames).
>
>Any ideas?
>Thank you
>
>
>--
>https://mail.python.org/mailman/listinfo/python-list

Re: Select columns based on dates - Pandas

<ee973b8c-d9fc-468d-877a-7f3dd2eafd27n@googlegroups.com>

copy mid

https://www.novabbs.com/devel/article-flat.php?id=15006&group=comp.lang.python#15006

copy link Newsgroups: comp.lang.python

X-Received: by 2002:ac8:5290:: with SMTP id s16mr5226331qtn.412.1630796788125;
Sat, 04 Sep 2021 16:06:28 -0700 (PDT)
X-Received: by 2002:a37:9307:: with SMTP id v7mr4937112qkd.371.1630796787844;
Sat, 04 Sep 2021 16:06:27 -0700 (PDT)
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!news.misty.com!border2.nntp.dca1.giganews.com!nntp.giganews.com!news-out.google.com!nntp.google.com!postnews.google.com!google-groups.googlegroups.com!not-for-mail
Newsgroups: comp.lang.python
Date: Sat, 4 Sep 2021 16:06:27 -0700 (PDT)
In-Reply-To: <mailman.604.1630688212.4164.python-list@python.org>
Injection-Info: google-groups.googlegroups.com; posting-host=74.195.46.149; posting-account=1YW3JAoAAAB0wczvpXQKtnF7cFYul9GA
NNTP-Posting-Host: 74.195.46.149
References: <20210903165644.kpgyvpits5hjlmbc@gmail.com> <9971a1ad-eec9-444d-ac6f-1f170a6572den@googlegroups.com>
<mailman.604.1630688212.4164.python-list@python.org>
User-Agent: G2/1.0
MIME-Version: 1.0
Message-ID: <ee973b8c-d9fc-468d-877a-7f3dd2eafd27n@googlegroups.com>
Subject: Re: Select columns based on dates - Pandas
From: medina...@gmail.com (Richard Medina)
Injection-Date: Sat, 04 Sep 2021 23:06:28 +0000
Content-Type: text/plain; charset="UTF-8"
Lines: 76

by: Richard Medina - Sat, 4 Sep 2021 23:06 UTC

On Friday, September 3, 2021 at 11:57:12 AM UTC-5, Martin Di Paola wrote:
> You may want to reshape the dataset to a tidy format: Pandas works
> better with that format.
>
> Let's assume the following dataset (this is what I understood from your
> message):
>
> In [34]: df = pd.DataFrame({
> ...: 'Country': ['us', 'uk', 'it'],
> ...: '01/01/2019': [10, 20, 30],
> ...: '02/01/2019': [12, 22, 32],
> ...: '03/01/2019': [14, 24, 34],
> ...: })
>
> In [35]: df
> Out[35]:
> Country 01/01/2019 02/01/2019 03/01/2019
> 0 us 10 12 14
> 1 uk 20 22 24
> 2 it 30 32 34
>
> Then, reshape it to a tidy format. Notice how each row now represents
> a single measure.
>
> In [43]: pd.melt(df, id_vars=['Country'], var_name='Date',
> value_name='Cases')
> Out[43]:
> Country Date Cases
> 0 us 01/01/2019 10
> 1 uk 01/01/2019 20
> 2 it 01/01/2019 30
> 3 us 02/01/2019 12
> 4 uk 02/01/2019 22
> 5 it 02/01/2019 32
> 6 us 03/01/2019 14
> 7 uk 03/01/2019 24
> 8 it 03/01/2019 34
>
> I used strings to represent the dates but it is much handy work
> with real date objects.
>
> In [44]: df2 = _
> In [45]: df2['Date'] = pd.to_datetime(df2['Date'])
>
> Now we can filter by date:
>
> In [50]: df2[df2['Date'] < '2019-03-01']
> Out[50]:
> Country Date Cases
> 0 us 2019-01-01 10
> 1 uk 2019-01-01 20
> 2 it 2019-01-01 30
> 3 us 2019-02-01 12
> 4 uk 2019-02-01 22
> 5 it 2019-02-01 32
>
> With that you could create three dataframes, one per month.
>
> Thanks,
> Martin.
> On Thu, Sep 02, 2021 at 12:28:31PM -0700, Richard Medina wrote:
> >Hello, forum,
> >I have a data frame with covid-19 cases per month from 2019 - 2021 like a header like this:
> >
> >Country, 01/01/2019, 2/01/2019, 01/02/2019, 3/01/2019, ... 01/01/2021, 2/01/2021, 01/02/2021, 3/01/2021
> >
> >I want to filter my data frame for columns of a specific month range of march to September of 2019, 2020, and 2021 only (three data frames).
> >
> >Any ideas?
> >Thank you
> >
> >
> >--
> >https://mail.python.org/mailman/listinfo/python-list
Thank you, Martin,
Is it possible to select/filter the dates (like columns from a date range) from the columns without reshaping them?
Thank you for your answer.

Subject	Author
Select columns based on dates - Pandas	Richard Medina
Re: Select columns based on dates - Pandas	Martin Di Paola
Re: Select columns based on dates - Pandas	Richard Medina