Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Feeling amorous, she looked under the sheets and cried, "Oh, no, it's Microsoft!"


devel / comp.lang.python / pandas (in jupyter?) problem

SubjectAuthor
o pandas (in jupyter?) problemPaulo da Silva

1
pandas (in jupyter?) problem

<t53rsb$kho$1@gioia.aioe.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=18155&group=comp.lang.python#18155

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!aioe.org!r2jfXpftVyFE0Vzaeaalag.user.46.165.242.75.POSTED!not-for-mail
From: p_d_a_s_...@nonetnoaddress.pt (Paulo da Silva)
Newsgroups: comp.lang.python
Subject: pandas (in jupyter?) problem
Date: Fri, 6 May 2022 20:11:07 +0100
Organization: Aioe.org NNTP Server
Message-ID: <t53rsb$kho$1@gioia.aioe.org>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Info: gioia.aioe.org; logging-data="21048"; posting-host="r2jfXpftVyFE0Vzaeaalag.user.gioia.aioe.org"; mail-complaints-to="abuse@aioe.org";
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
Firefox/91.0 Thunderbird/91.8.1
Content-Language: en-US
X-Notice: Filtered by postfilter v. 0.9.2
 by: Paulo da Silva - Fri, 6 May 2022 19:11 UTC

Hi all!

I'm having the following problem. Consider the code (the commented or
the not commented which I think do the same things):

#for col in missing_cols:
# df[col] = np.nan

df=df.copy()
df[missing_cols]=np.nan

df has about 20000 cols and len(missing_cols) is about 18000.

I'm getting lots (1 by missing_col?) of the following message from
ipykernel:

"PerformanceWarning: DataFrame is highly fragmented. This is usually
the result of calling `frame.insert` many times, which has poor
performance. Consider joining all columns at once using
pd.concat(axis=1) instead. To get a de-fragmented frame, use `newframe =
frame.copy()`
df[missing_cols]=np.nan"

At first I didn't have df=df.copy(). I added it later, but the same problem.

This slows down the code a lot, perhaps because jupyter is taking too
much time issuing these messages!

Thanks for any comments.

1
server_pubkey.txt

rocksolid light 0.9.8
clearnet tor