novaBBS - comp.lang.python

Re: tail

<mailman.341.1651951338.20749.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=18180&group=comp.lang.python#18180

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!feeder8.news.weretis.net!lilly.ping.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: pyt...@mrabarnett.plus.com (MRAB)
Newsgroups: comp.lang.python
Subject: Re: tail
Date: Sat, 7 May 2022 20:19:04 +0100
Lines: 58
Message-ID: <mailman.341.1651951338.20749.python-list@python.org>
References: <CABbU2U-_Z546umxtnZXL8b1LUERCnyOxYw6osKTvKncOHFkJ3A@mail.gmail.com>
<60454E09-0ADA-4881-A84B-6C11397D244F@barrys-emacs.org>
<CABbU2U99Jpa6nuYg0sXw6=GjBEKVk9u-_oyxSoL8hLrW_2FoBA@mail.gmail.com>
<561ac7a8-2034-c1ce-6fca-f4280baac409@mrabarnett.plus.com>
<CABbU2U-N=YiRYfVkjpv8RP6BCo4VOLL7SWK=vNq8oje7nwuyUw@mail.gmail.com>
<c1131c59-2283-c78a-49c8-6d05c6eb7832@mrabarnett.plus.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Trace: news.uni-berlin.de ju7Ze/qiLfF1CVWwd5wtYAsJj70jLASEy3sn+cVzFpAQ==
Return-Path: <python@mrabarnett.plus.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=plus.com header.i=@plus.com header.b=RsAbl6ZR;
dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.002
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; '(which': 0.04; 'argument':
0.04; '2022': 0.05; '(assuming': 0.09; 'bin': 0.09; 'characters,':
0.09; 'from:addr:python': 0.09; 'ok,': 0.09;
'received:192.168.1.64': 0.09; '>>>>': 0.16; 'assuming': 0.16;
'barry': 0.16; 'characters.': 0.16; 'determining': 0.16;
'encoding': 0.16; 'from:addr:mrabarnett.plus.com': 0.16;
'from:name:mrab': 0.16; 'furthermore,': 0.16; 'input.': 0.16;
'message-id:@mrabarnett.plus.com': 0.16; 'received:84.93': 0.16;
'received:84.93.230': 0.16; 'received:plus.net': 0.16; 'specify':
0.16; 'treats': 0.16; 'unit,': 0.16; 'wrote:': 0.16; 'to:addr
:python-list': 0.20; 'sat,': 0.22; 'code': 0.23; 'to:name:python-
list@python.org': 0.24; 'binary': 0.26; '>>>': 0.28; 'wrong':
0.28; 'header:User-Agent:1': 0.30; 'think': 0.32; 'specified':
0.32; 'received:192.168.1': 0.32; 'but': 0.32; 'there': 0.33;
"didn't": 0.34; 'skip:" 20': 0.34; 'header:In-Reply-To:1': 0.34;
'handling': 0.35; 'cases': 0.36; 'those': 0.36; "skip:' 10": 0.37;
"it's": 0.37; 'hard': 0.37; 'received:192.168': 0.37; 'file':
0.38; 'way': 0.38; 'could': 0.38; 'added': 0.39; 'text': 0.39;
'handle': 0.39; 'use': 0.39; 'want': 0.40; 'method': 0.61; 'mode':
0.62; 'skip:b 20': 0.63; 'skip:b 10': 0.63; "you'd": 0.64; 'down':
0.64; 'per': 0.68; 'choice': 0.76; 'attribute': 0.84; 'opened,':
0.84; 'skip:" 40': 0.84; 'sulla': 0.84; 'trick': 0.91
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=plus.com; s=042019;
t=1651951151; bh=pBk+h/znPAHiuMpqcTvcvsax79WU5dRCjMIswJF3hj0=;
h=Date:Subject:To:References:From:In-Reply-To;
b=RsAbl6ZR8vMVAlUBPbyWURj39KS/QD3DUXxDJZCcwnaql/RARkfDq/I5ZbdPaf8ul
2f5GmQlEgi1L9X/8Iz/hkuUu26RQ3nFi/PVDipByhSNGmUIzORRVEaZrYP3cKtP6aU
Pdq+3dS3Osi6pAVmyrtuYiaCOhyJb8OSZ1U0gn4/cJcLGb5Ydg9xWh80kbsPMslK7p
7zRU3uSQvr5IfEzYDaV7fZXUX+d1xEvmRFe1huM2CZrpcKlSyOBhrT7myFeEqT+SHj
m/OBibMSuggEGYzO+mtvfTUJGM0mczAWKHfE6c4Oqh5ACgb4UAuUSjCwWKZ5P7A+0P
3YCBGTuBk7x+g==
X-Clacks-Overhead: "GNU Terry Pratchett"
X-CM-Score: 0.00
X-CNFS-Analysis: v=2.4 cv=HttlpmfS c=1 sm=1 tr=0 ts=6276c62f
a=0nF1XD0wxitMEM03M9B4ZQ==:117 a=0nF1XD0wxitMEM03M9B4ZQ==:17
a=IkcTkHD0fZMA:10 a=EBOSESyhAAAA:8 a=pJ0dDMpWAAAA:8 a=VS8GM9xs-4iah7_XOoEA:9
a=QEXdDO2ut3YA:10 a=yJM6EZoI5SlJf8ks9Ge_:22 a=7HhFaE7EYeXGtY87cTDK:22
X-AUTH: mrabarnett@:2500
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
Thunderbird/91.9.0
Content-Language: en-GB
In-Reply-To: <CABbU2U-N=YiRYfVkjpv8RP6BCo4VOLL7SWK=vNq8oje7nwuyUw@mail.gmail.com>
X-CMAE-Envelope: MS4xfFFy2kkXtAGcB688VRhrq0lLP6gIcoXXFATlGVAcnghD6Q4MP5W/rAfoiQmtswId2UciImrHmkNIGGywFZ68BonB99ICcDK/vmjIfkUn9X8tdmNkta4J
clxSu/U3Ih94JNmD5S2xSm+mf8r/LjxZpSkROb1lrOWE9iAME2+iNM1EVyq3VqFbrCTPdDdMTk9CY08UwBDNbMduSsVMspGiAvw=
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <c1131c59-2283-c78a-49c8-6d05c6eb7832@mrabarnett.plus.com>
X-Mailman-Original-References: <CABbU2U-_Z546umxtnZXL8b1LUERCnyOxYw6osKTvKncOHFkJ3A@mail.gmail.com>
<60454E09-0ADA-4881-A84B-6C11397D244F@barrys-emacs.org>
<CABbU2U99Jpa6nuYg0sXw6=GjBEKVk9u-_oyxSoL8hLrW_2FoBA@mail.gmail.com>
<561ac7a8-2034-c1ce-6fca-f4280baac409@mrabarnett.plus.com>
<CABbU2U-N=YiRYfVkjpv8RP6BCo4VOLL7SWK=vNq8oje7nwuyUw@mail.gmail.com>

by: MRAB - Sat, 7 May 2022 19:19 UTC

On 2022-05-07 19:35, Marco Sulla wrote:
> On Sat, 7 May 2022 at 19:02, MRAB <python@mrabarnett.plus.com> wrote:
> >
> > On 2022-05-07 17:28, Marco Sulla wrote:
> > > On Sat, 7 May 2022 at 16:08, Barry <barry@barrys-emacs.org> wrote:
> > >> You need to handle the file in bin mode and do the handling of line endings and encodings yourself. It’s not that hard for the cases you wanted.
> > >
> > >>>> "\n".encode("utf-16")
> > > b'\xff\xfe\n\x00'
> > >>>> "".encode("utf-16")
> > > b'\xff\xfe'
> > >>>> "a\nb".encode("utf-16")
> > > b'\xff\xfea\x00\n\x00b\x00'
> > >>>> "\n".encode("utf-16").lstrip("".encode("utf-16"))
> > > b'\n\x00'
> > >
> > > Can I use the last trick to get the encoding of a LF or a CR in any encoding?
> >
> > In the case of UTF-16, it's 2 bytes per code unit, but those 2 bytes
> > could be little-endian or big-endian.
> >
> > As you didn't specify which you wanted, it defaulted to little-endian
> > and added a BOM (U+FEFF).
> >
> > If you specify which endianness you want with "utf-16le" or "utf-16be",
> > it won't add the BOM:
> >
> > >>> # Little-endian.
> > >>> "\n".encode("utf-16le")
> > b'\n\x00'
> > >>> # Big-endian.
> > >>> "\n".encode("utf-16be")
> > b'\x00\n'
>
> Well, ok, but I need a generic method to get LF and CR for any
> encoding an user can input.
> Do you think that
>
> "\n".encode(encoding).lstrip("".encode(encoding))
>
> is good for any encoding?
'.lstrip' is the wrong method to use because it treats its argument as a
set of characters, so it might strip off too many characters. A better
choice is '.removeprefix'.
> Furthermore, is there a way to get the encoding of an opened file object?
>
How was the file opened?

If it was opened as a text file, use the '.encoding' attribute (which
just tells you what encoding was specified when it was opened, and you'd
be assuming that it's the correct one).

If it was opened as a binary file, all you know is that it contains
bytes, and determining the encoding (assuming that it is a text file) is
down to heuristics (i.e. guesswork).

Badges? We don't need no stinking badges.

devel / comp.lang.python / Re: tail