novaBBS - comp.lang.python

Re: tail

<mailman.347.1651998060.20749.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=18187&group=comp.lang.python#18187

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: bar...@barrys-emacs.org (Barry)
Newsgroups: comp.lang.python
Subject: Re: tail
Date: Sun, 8 May 2022 09:04:13 +0100
Lines: 45
Message-ID: <mailman.347.1651998060.20749.python-list@python.org>
References: <CABbU2U99Jpa6nuYg0sXw6=GjBEKVk9u-_oyxSoL8hLrW_2FoBA@mail.gmail.com>
<A3773CDA-B6FE-4A51-8D75-362397220F67@barrys-emacs.org>
Mime-Version: 1.0 (1.0)
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Trace: news.uni-berlin.de tYdDgh9yO+vAoWfZhNT9YguxLcWLo4LdJm15hB2DHskQ==
Return-Path: <barry@barrys-emacs.org>
X-Original-To: Python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=none reason="no signature";
dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.005
X-Spam-Evidence: '*H*': 0.99; '*S*': 0.00; '2022': 0.05; 'utf-8':
0.07; 'bin': 0.09; 'byte': 0.09; 'cc:addr:python-list': 0.09;
'difficult.': 0.09; 'from:addr:barry': 0.09; 'received:217.70':
0.09; 'received:217.70.178': 0.09; 'received:gandi.net': 0.09;
'received:mail.gandi.net': 0.09; 'cc:no real name:2**0': 0.14;
'2022,': 0.16; '>>>>': 0.16; 'barry': 0.16; 'encoding': 0.16;
'from:addr:barrys-emacs.org': 0.16; 'message-id:@barrys-
emacs.org': 0.16; 'unicode': 0.16; 'wrote:': 0.16; 'values': 0.17;
'solve': 0.19; 'cc:addr:python.org': 0.20; 'sat,': 0.22;
'cc:2**1': 0.23; 'series': 0.28; 'issues.': 0.32; 'but': 0.32;
'there': 0.33; 'skip:" 20': 0.34; 'header:In-Reply-To:1': 0.34;
'handling': 0.35; 'cases': 0.36; 'using': 0.37; 'hard': 0.37;
'file': 0.38; 'handle': 0.39; 'use': 0.39; 'mode': 0.62; 'once':
0.63; 'skip:b 20': 0.63; 'skip:b 10': 0.63; 'your': 0.64; 'look':
0.65; 'received:217': 0.67; 'little': 0.73; 'deal': 0.73; 'base,':
0.84; 'skip:" 40': 0.84; 'sulla': 0.84; 'you?': 0.88; 'trick':
0.91
In-Reply-To: <CABbU2U99Jpa6nuYg0sXw6=GjBEKVk9u-_oyxSoL8hLrW_2FoBA@mail.gmail.com>
X-Mailer: iPad Mail (19E258)
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <A3773CDA-B6FE-4A51-8D75-362397220F67@barrys-emacs.org>
X-Mailman-Original-References: <CABbU2U99Jpa6nuYg0sXw6=GjBEKVk9u-_oyxSoL8hLrW_2FoBA@mail.gmail.com>

by: Barry - Sun, 8 May 2022 08:04 UTC

> On 7 May 2022, at 17:29, Marco Sulla <Marco.Sulla.Python@gmail.com> wrote:
>
> On Sat, 7 May 2022 at 16:08, Barry <barry@barrys-emacs.org> wrote:
>> You need to handle the file in bin mode and do the handling of line endings and encodings yourself. It’s not that hard for the cases you wanted.
>
>>>> "\n".encode("utf-16")
> b'\xff\xfe\n\x00'
>>>> "".encode("utf-16")
> b'\xff\xfe'
>>>> "a\nb".encode("utf-16")
> b'\xff\xfea\x00\n\x00b\x00'
>>>> "\n".encode("utf-16").lstrip("".encode("utf-16"))
> b'\n\x00'
>
> Can I use the last trick to get the encoding of a LF or a CR in any encoding?

In a word no.

There are cases that you just have to know the encoding you are working with..
utf-16 because you have deal with the data in 2 byte units and know if
it is big endian or little endian.

There will be other encoding that will also be difficult.

But if you are working with encoding that are using ASCII as a base,
like unicode encoded as utf-8 or iso-8859 series then you can just look
for NL and CR using the ASCII values of the byte.

In short once you set your requirements then you can know what problems
you can avoid and which you must solve.

Is utf-16 important to you? If not no need to solve its issues.

Barry

All laws are simulations of reality. -- John C. Lilly

devel / comp.lang.python / Re: tail