Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Disc space -- the final frontier!

devel / comp.lang.python / Problem with concatenating two dataframes

SubjectAuthor
o Problem with concatenating two dataframesMahmood Naderan

1
Problem with concatenating two dataframes

<mailman.184.1636217955.23718.python-list@python.org>

  copy mid

https://www.novabbs.com/devel/article-flat.php?id=15923&group=comp.lang.python#15923

  copy link   Newsgroups: comp.lang.python
Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: nt_mahm...@yahoo.com (Mahmood Naderan)
Newsgroups: comp.lang.python
Subject: Problem with concatenating two dataframes
Date: Sat, 6 Nov 2021 16:16:15 +0000 (UTC)
Lines: 112
Message-ID: <mailman.184.1636217955.23718.python-list@python.org>
References: <1670105892.779098.1636215375137.ref@mail.yahoo.com>
<1670105892.779098.1636215375137@mail.yahoo.com>
Reply-To: Mahmood Naderan <nt_mahmood@yahoo.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de BCwbzn9pEuneQk/m7N9rwQjUcCXOZbbh6fL1ddzRl8Sg==
Return-Path: <nt_mahmood@yahoo.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=yahoo.com header.i=@yahoo.com header.b=VbflTNKM;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.012
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; 'argument': 0.04; 'error:':
0.05; 'that?': 0.07; '8bit%:62': 0.09; 'objects,': 0.09; 'pandas':
0.09; 'skip:\xc2 20': 0.09; 'subject:two': 0.09; 'typeerror:':
0.09; 'append': 0.16; 'appended': 0.16; 'batch': 0.16; 'column':
0.16; 'dataframe.': 0.16; 'dict': 0.16; 'dictionary.': 0.16;
'from:addr:nt_mahmood': 0.16; 'from:name:mahmood naderan': 0.16;
'iterable': 0.16; 'key-value': 0.16; 'mahmood': 0.16; 'values':
0.17; 'to:addr:python-list': 0.20; 'to:name:python-
list@python.org': 0.24; 'object': 0.26; 'creating': 0.27; "isn't":
0.27; 'error': 0.29; 'code,': 0.31; "doesn't": 0.32; 'skip:7 10':
0.32; 'trying': 0.35; 'following': 0.35; 'fix': 0.36; 'file':
0.38; 'read': 0.38; 'wrote': 0.39; 'beginning': 0.40;
'otherwise,': 0.40; 'should': 0.40; '8bit%:29': 0.60; 'reply-
to:addr:yahoo.com': 0.61; 'skip:\xc2 10': 0.62; 'skip:i 20': 0.62;
'skip:b 20': 0.63; 'key': 0.64; 'definition': 0.64; 'compare':
0.69; 'created.': 0.69; '8bit%:38': 0.70; '8bit%:31': 0.73;
'8bit%:94': 0.75; '8bit%:100': 0.76; 'header:Reply-To:1': 0.79;
'8bit%:67': 0.81; '8bit%:97': 0.84; 'pairs': 0.84; 'skip:d 30':
0.86
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048;
t=1636215376; bh=Sec1AqOIe/lbmIqjB5oK7nbMqD6ytEapUXceQenZC1E=;
h=Date:From:Reply-To:To:Subject:References:From:Subject:Reply-To;
b=VbflTNKMz6go4eQVEljBR8bVe2E9fNaTt2Z3HmFcZqWzvVIzAxDwbokz0JRZYczim0rpmFiPuBSYfxvsOWFqT1OA2t9g539QhyT+rJ9ulirJBmepVwloEUpQbjAzDxRnk2OmJRU+KEDPHxHKwxvxYh2tO7rWjxX5yeNq2hlf0f+3jHGzYQxHDvuOii/LreibHcllldLVfHttPDKDLNuBHYq2VzKYgcE5fzbCOnhY054VoC2Y0/vPp7iAbdqMDPqoOgRbDXBZSufGkt+Tf5rjzo0ZdUpg5U7fUchmDCRNOT0cv9ZN6aOdehkTvQyVouXdeCIpQ1UDbzDgzJbCFE9Seg==
X-SONIC-DKIM-SIGN: v=1; a=rsa-sha256; c=relaxed/relaxed; d=yahoo.com; s=s2048;
t=1636215376; bh=ld/JbGJ/wB4E3tsA3fbv0iCGKYj3rv1f/pj0UDjm/6j=;
h=X-Sonic-MF:Date:From:To:Subject:From:Subject;
b=OtxY2U0Siw6gApeHT4NnQ/P/nvt6VmTXlNxe2AFikMzidnZLwPNNQhPXwCn6R9eaKlM6pWOTEXMaj5HUAaLU3Och1ItubW+8LeUsfn2O/oIls3HgvX8t4G9uVW/75yYnG23by4XJ4GtpDdJq3ZEy/1Ne8GVtHKr5WMmPrLyp7vkrNWtHlcADJXPFdIKAaRNBYwdvMieeiSi6UfXkLwHP+s1j4V7Vme5l3BgaU6/Ko5vwhb0BYSUc4PR8K/0JcDimf8YWyt7tAjZDv8nVGNesmYsrlfI6GrAje0fCL69BWMUJshPkw5Lp7VyL0OwLzPfU1ZnjfUMVS0FGN36qzKbfvA==
X-YMail-OSG: bZA3dI0VM1kyiFQnu2pZu8uDEQ.UucQNDBpWbyxwol4yRefHJIM4mRwo2Z0wx5X
iUzGlGIl8jq_nD3_U7UnU_r4czl3ROBkWwKFkVX3u0zdwXASErmmRkIOuhGueUDndmcsA53.paR8
6G4dr9HAYCUB4rQKE31lr3tD85OvFai2b7vzMiCQuXn4IB7Bj3oWhh0NpfK4VSEnWn1m75TRljfk
RG4P4.ZfI46qEmX3phwjD77lDn20oXxxfIVmG.E6JWIFI07cpI3y2xHAWG5hyeD5SszI1EHRUpun
2EHQGgytfddWxAeR3hacaOapvitkBikcU7ktkiXjj2bZrUqgnXPEF5x367XV6EagnW7knYsW_zjW
7pW1JAVeAyZKHfOWWovDZoNtThIU9Q2uwFEWCyEQw1wzxaMESo_6F.zUeFJ0.eZ0l4tOnwO0IaEH
dzmYgL290YcdBBv9jC2_is3M_CAxFFgokaWhiop92q.ftc6J5VyIZ92JEjWXkEaJGIORyNfGPxAe
gVHg5UEzuqVOgVQgBx6CEENLzQvE1jeIMecunUffov5hDd1PJ2ijtDvvtmI9sclTqPtcNnpzOyAO
Y2QflSYJUr7a7KC1BXs0DZ4EEyyTl3MkeIbXW2DtQZVuyQunWul5t4Bvjm7A4kiNqa7Ao85Ee314
zlf6YCsiEdFXvr9Az5hw1QqjVc8YFlz82cRleks1zCAoK3StrvPR_eyzepzdxnoj2_B3uSe0EREb
XhAwL5yjgOZudnlxutAl0UCNlk8MMqtDMxgr0mjbT3qP6zTEdY9crV5TNyixsmfuGvSw321qhYpn
9bs_HD0NoBwm3cmxICOuVBZZhjmPPf0BYzvU3KmJzoDciBf4DluxecgYDgv9SSMx6lnDxZLCRvR2
r3rY9_uQzKlWydk_Q3fEyILoPRUdKsq4RsT6EvxLni3Jc3Chx08NUcpvNTeWZ4JxVbuprQU0nrbc
Fq__m4l_hKw4W6FYOrvXDpblS72LK.TBZhUX240khI62cculNpFBEXC15kAGC6WdNPKE7RLtRNbn
NXSRpTBycgm2vI3NR5S_ApYuwBvdiHJK9vVHO1WmbbhmMq.6s6EE7Uu8TYYAf5kvgNf0jZn_BHxb
8.tyaI4WrXpuHIlbOPIk8xBS4VCogxZbHD4Isn.2UxgKNChU49dLhio3zFoAzFYVV9d.MjMNm7pw
DdHTh_8vyDCq_.WVFcL9GEXFH19nSzrMwz2XYdGbp0CsEwevL6KVVG3a7qabT9Im_xdAzmyHy82T
aWSq43ethrO3hEIEr7jP.eeZAqny_YODN9JG9dW6kSvWsFYXvwLuyM2XnYroOdf8SGKm.78Wc54M
MNtvf7ZYuWFtQ2lOr.az3GSSiWD0FCY633m2y0oYKh4BxpB1QgoEcNykWJgzybJJWeKdEzRFDuWf
XxVy_rU0wIqikeaE9hw6RvqNNCrLXxjcofY_Rd31j7CL1ki3vHdiluHpRo6iVe_7riINILyb62AA
Ads0ywoY.FOT8h0mlNzRmxHhPD07x14.1F_KKWTXEct3ew4DJinU_4vlT.dmHIkuRuXGydhlcEtc
u6zok4AETMyqDrBXVjh5PglzpNMSzlP.8oyg.ljdQIN8zbef5pAamNJvAZSOPy0ntw5euouxXagp
BX_8Ztqs0QIB9uHXMJqpXf60iR9Xu5016LCx8WxE2mKa7YKu.dTO7ym9DImvgxxep_ERlNx_ZMxX
JShLtcLzCd7fsoYQX5yYQNFe7d15zEJ4kY0X5gVAni1V2yC.JeQts6VUt0EKQo.G4EXHJxRBSOIK
zUlz2TIF3m7NH11ysSJIaDgfs.zyXnnqJpavFxgQ7lWyx2D3pW487Bammy.wk0tT7MdseUEKu7nB
qHPH.KIE_TWr5kkUEE3r9g2ejM1dn8Fhdp8f.IC6Qd53qY595W76eSZAejXDVgOvNTUuYpsfdm29
PGzRXuEs_TFqIwdbdLaYy7nr6iO_XHnssUdgQETZ3UTOLoZ74rpwokOWp88Sr95BKX5jFghGlKX4
k2G3hTKI8Lg1C54W8CFec3HMhxonwIj6sYO7PzJo8995tQUc684xxse72pFPhfoRdp1CdKJxhyoz
iQblcCN3dYYrQ.p74YPFLqoPsn4BHPyUZTX76V1hVzd7YED7oV_WfLqVui5.Cg3dQ7b7_i7CkPLL
B7enoieLWNNoSb1KxoH4tDxjBo81qATNNJleQ7.tWu6atYD_q5jNKtzuUrtyJjVGQR_k1Md7z6_P
OxpKtfI2wdBwehzHZGt9yo8I1CQZX47I7g.JGktRmOYY34N9F8Q--
X-Sonic-MF: <nt_mahmood@yahoo.com>
X-Mailer: WebService/1.1.19266 YMailNorrin
X-Mailman-Approved-At: Sat, 06 Nov 2021 12:59:13 -0400
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.35
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <1670105892.779098.1636215375137@mail.yahoo.com>
X-Mailman-Original-References: <1670105892.779098.1636215375137.ref@mail.yahoo.com>
 by: Mahmood Naderan - Sat, 6 Nov 2021 16:16 UTC

In the following code, I am trying to create some key-value pairs in a dictionary where the first element is a name and the second element is a dataframe.

# Creating a dictionary
data = {'Value':[0,0,0]}
kernel_df = pd.DataFrame(data, index=['M1','M2','M3'])
dict = {'dummy':kernel_df}
# dummy  ->          Value
#               M1      0
#               M2      0
#               M3      0

Then I read a file and create some batches and compare the name in the batch with the stored names in dictionary. If it doesn't exist, a new key-value (name and dataframe) is created. Otherwise, the Value column is appended to the existing dataframe.

df = pd.read_csv('test.batch.csv')
print(df)
for i in range(0, len(df), 3):
    print("\n------BATCH BEGIN")
    batch_df = df.iloc[i:i+3]
    name = batch_df.loc[i].at["Name"]
    values = batch_df.loc[:,["Value"]]
    print(name)
    print(values)
    print("------BATCH END")
    if name in dict:
        # Append values to the existing key
        dict[name] = pd.concat( dict[name],values )   #### ERROR
    else:
        # Create a new pair in dictionary
        dict[name] = values;

As you can see in the output, the join statement has error.

   ID Name Metric  Value
0   0   K1     M1     10
1   0   K1     M2      5
2   0   K1     M3     10
3   1   K2     M1     20
4   1   K2     M2     10
5   1   K2     M3     15
6   2   K1     M1      2
7   2   K1     M2      2
8   2   K1     M3      2

------BATCH BEGIN
K1
   Value
0     10
1      5
2     10
------BATCH END

------BATCH BEGIN
K2
   Value
3     20
4     10
5     15
------BATCH END

------BATCH BEGIN
K1
   Value
6      2
7      2
8      2
------BATCH END

As it reaches the contact() statement, I get this error:

TypeError: first argument must be an iterable of pandas objects, you passed an object of type "DataFrame"

Based on the definition I wrote in the beginning of the code, "dict[name]" should be a dataframe. Isn't that?

How can I fix that?

Regards,
Mahmood

1
server_pubkey.txt

rocksolid light 0.9.81
clearnet tor