novaBBS - comp.lang.python

Re: tail

<mailman.231.1650758599.20749.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=18011&group=comp.lang.python#18011

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: cs...@cskk.id.au (Cameron Simpson)
Newsgroups: comp.lang.python
Subject: Re: tail
Date: Sun, 24 Apr 2022 10:03:13 +1000
Lines: 23
Message-ID: <mailman.231.1650758599.20749.python-list@python.org>
References: <CAPTjJmo7oNKuuiQPa_t3X9Sws3veDuX4660We5ZKFPeajC9+Ng@mail.gmail.com>
<YmSTwYXFZqpZt3a7@cskk.homeip.net>
Mime-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Trace: news.uni-berlin.de chw/A55l/0gHyJXi6+1+XAgo36HIqxd+btlRfWesxYzg==
Return-Path: <cameron@cskk.id.au>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=none reason="no signature";
dkim-adsp=none (unprotected policy); dkim-atps=neutral
X-Spam-Status: OK 0.000
X-Spam-Evidence: '*H*': 1.00; '*S*': 0.00; '2022': 0.05; 'sun,': 0.07;
'angelico': 0.09; 'byte': 0.09; 'though.': 0.09; 'cheers,': 0.11;
'cameron': 0.16; 'chunking,': 0.16; 'elaborate': 0.16;
'from:addr:cs': 0.16; 'from:addr:cskk.id.au': 0.16;
'from:name:cameron simpson': 0.16; 'locate': 0.16; 'message-
id:@cskk.homeip.net': 0.16; 'predicting': 0.16; 'received:13.237':
0.16; 'received:13.237.201': 0.16; 'received:13.237.201.189':
0.16; 'received:cskk.id.au': 0.16; 'received:id.au': 0.16;
'received:mail.cskk.id.au': 0.16; 'simpson': 0.16; 'vague': 0.16;
'wrote:': 0.16; 'code.': 0.17; 'to:addr:python-list': 0.20; "i'd":
0.24; 'idea': 0.24; 'chris': 0.28; 'expect': 0.28; 'header:User-
Agent:1': 0.30; 'approach': 0.31; 'think': 0.32; 'zero': 0.32;
'but': 0.32; "i'm": 0.33; "didn't": 0.34; 'header:In-Reply-To:1':
0.34; 'received:au': 0.35; "it's": 0.37; 'file': 0.38; 'read':
0.38; 'single': 0.39; 'use': 0.39; 'true.': 0.40; 'both': 0.40;
'something': 0.40; 'received:13': 0.64; 'received:userid': 0.66;
'performance': 0.71; 'job,': 0.84
Mail-Followup-To: Python List <python-list@python.org>
Content-Disposition: inline
In-Reply-To: <CAPTjJmo7oNKuuiQPa_t3X9Sws3veDuX4660We5ZKFPeajC9+Ng@mail.gmail.com>
User-Agent: Mutt/2.2.1 (2022-02-19)
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <YmSTwYXFZqpZt3a7@cskk.homeip.net>
X-Mailman-Original-References: <CAPTjJmo7oNKuuiQPa_t3X9Sws3veDuX4660We5ZKFPeajC9+Ng@mail.gmail.com>

by: Cameron Simpson - Sun, 24 Apr 2022 00:03 UTC

On 24Apr2022 08:21, Chris Angelico <rosuav@gmail.com> wrote:
>On Sun, 24 Apr 2022 at 08:18, Cameron Simpson <cs@cskk.id.au> wrote:
>> An approach I think you both may have missed: mmap the file and use
>> mmap.rfind(b'\n') to locate line delimiters.
>> https://docs.python.org/3/library/mmap.html#mmap.mmap.rfind
>
>Yeah, I made a vague allusion to use of mmap, but didn't elaborate
>because I actually have zero idea of how efficient this would be.
>Would it be functionally equivalent to the chunking, but with the
>chunk size defined by the system as whatever's most optimal? It would
>need to be tested.

True. I'd expect better than single byte seek/read though.

>I've never used mmap for this kind of job, so it's not something I'm
>comfortable predicting the performance of.

Fair.

But it would be much easier to read code.

Cheers,
Cameron Simpson <cs@cskk.id.au>

"If it ain't broke, don't fix it." -- Bert Lantz

devel / comp.lang.python / Re: tail