Message-ID:

Math is like love -- a simple idea but it can get complicated. -- R. Drabek

Re: tail

<mailman.338.1651942483.20749.python-list@python.org>

https://www.novabbs.com/devel/article-flat.php?id=18174&group=comp.lang.python#18174

copy link Newsgroups: comp.lang.python

Path: i2pn2.org!i2pn.org!news.swapon.de!fu-berlin.de!uni-berlin.de!not-for-mail
From: drsali...@gmail.com (Dan Stromberg)
Newsgroups: comp.lang.python
Subject: Re: tail
Date: Sat, 7 May 2022 09:54:28 -0700
Lines: 80
Message-ID: <mailman.338.1651942483.20749.python-list@python.org>
References: <CABbU2U98YKdcnJkDPfzE3Pqso+6LL72usB8hrSBVR0WbhauRoQ@mail.gmail.com>
<CAGGBd_qKfULfPaKWLYbFo6wNBU9WK7tfGw0xCov2dFN4MguhCA@mail.gmail.com>
Mime-Version: 1.0
Content-Type: text/plain; charset="UTF-8"
X-Trace: news.uni-berlin.de rrBQoUeR9GTHBaoxfXpSJwr8byCDWrq/3gJgQ88bwZjQ==
Return-Path: <drsalists@gmail.com>
X-Original-To: python-list@python.org
Delivered-To: python-list@mail.python.org
Authentication-Results: mail.python.org; dkim=pass
reason="2048-bit key; unprotected key"
header.d=gmail.com header.i=@gmail.com header.b=Mo+Xw2zD;
dkim-adsp=pass; dkim-atps=neutral
X-Spam-Status: OK 0.008
X-Spam-Evidence: '*H*': 0.98; '*S*': 0.00; '2022': 0.05; '"""': 0.09;
'cc:addr:python-list': 0.09; 'skip:# 30': 0.09; 'import': 0.15;
'url:mailman': 0.15; 'backward': 0.16; 'bottom?': 0.16;
'cc:name:python list': 0.16; 'commons': 0.16;
'from:addr:drsalists': 0.16; 'from:name:dan stromberg': 0.16;
"it'd": 0.16; 'scanning': 0.16; 'streams': 0.16; 'wrote:': 0.16;
'cc:addr:python.org': 0.20; 'sat,': 0.22; 'lines': 0.23; 'skip:p
30': 0.23; "i'd": 0.24; 'url-ip:188.166.95.178/32': 0.25; 'url-
ip:188.166.95/24': 0.25; 'url:listinfo': 0.25; 'cc:2**0': 0.25;
'url-ip:188.166/16': 0.25; 'output': 0.28; 'url-ip:188/8': 0.31;
'message-id:@mail.gmail.com': 0.32; 'but': 0.32; 'header:In-Reply-
To:1': 0.34; 'received:google.com': 0.34; 'from:addr:gmail.com':
0.35; 'received:209.85': 0.37; 'received:209': 0.39; 'text': 0.39;
'use': 0.39; 'break': 0.39; 'received:209.85.222': 0.39;
'advantage': 0.40; 'something': 0.40; 'method': 0.61; 'skip:o 10':
0.61; 'skip:r 20': 0.64; 'introducing': 0.64; 'skip:e 20': 0.67;
'del': 0.70; '11:36': 0.84; 'sulla': 0.84
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
h=mime-version:references:in-reply-to:from:date:message-id:subject:to
:cc; bh=PxL32M+zUsCV4YeSEvgEi75xB4Q+yOD19RfG1MT25tU=;
b=Mo+Xw2zDjBDd+3TCjm0aUd4vONzYnAPIEMk01l9Fs28CGYkvBPHU79JgTYIIb3pth4
IPluh6YMcZS7OBQmJhMadBSDSy9NKrgtgMpvYN7vkWtrdCoXL6Lk/ekGdyJ8jZOwQMV+
8q91uKtebCWWG4+KWwwcQRMOMuSLutGOwV+Nd34QEr3wS8RxaOs6eNbIjifnBf1ojfaC
sWEbq2FpcWFb7pFHhSx3gg+mx2u5hSvfKYDr6LCuKmDAtXQJ6SCWieq0CZSrmN9k99fc
/G3LDBOnwTu1Ps8/oLr2lWHb8juy92r1kX22guLEN9q9OsPdH5MevPE+6tB6kwnEbVjq
rXxg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20210112;
h=x-gm-message-state:mime-version:references:in-reply-to:from:date
:message-id:subject:to:cc;
bh=PxL32M+zUsCV4YeSEvgEi75xB4Q+yOD19RfG1MT25tU=;
b=Xtd7vwYSNOuEH+gHgRZX7BUglLB5ibIqE+eiRGH8cHRT7zkMG9yCAI5GzeMaiWBJsm
NcwY6iMGNP7ky3YC6ouutEHTg2U2ODfbGgdJvWxA0szCXPOxeD/8KYWUE5dV26lofno3
GWADM9HtBN/BzjHTtFIbjXyqNO6PT3gox4XcfPgzcWaTZi48IUHIghSoTxSLIeeiJefu
ZxZmNesZATi2Ja2StRweVzjc/d99rF5J178H8sIDBR/GKXsf4QQ8dKDZnUStYnLpUBK8
BMFN4q9ifsQqYy5Sf6PME/gVqIe6KPp8wzj2LIeJtxmUJK63hKhAkeCaeWCUGy0/mrOe
ujNQ==
X-Gm-Message-State: AOAM531aVGFY/94GTx9A54Cgmg6TG786jgI3QjcqXm58yxUrd3CwTd5V
UkYdyE+19YUOg8yw18v2+3tgNX/u4C0h4WS1pJpT5XyH4WU=
X-Google-Smtp-Source: ABdhPJwu0trbjMcPWWiWfYRmYofUwPKucU7FbXqT/7glDCrnoAfdyRLxUYgzOHRGSLdTxvHousIzDqkfZmZEKoI2jWQ=
X-Received: by 2002:ab0:407:0:b0:35f:ef75:81e8 with SMTP id
7-20020ab00407000000b0035fef7581e8mr4853420uav.91.1651942480548; Sat, 07 May
2022 09:54:40 -0700 (PDT)
In-Reply-To: <CABbU2U98YKdcnJkDPfzE3Pqso+6LL72usB8hrSBVR0WbhauRoQ@mail.gmail.com>
X-Content-Filtered-By: Mailman/MimeDel 2.1.39
X-BeenThere: python-list@python.org
X-Mailman-Version: 2.1.39
Precedence: list
List-Id: General discussion list for the Python programming language
<python-list.python.org>
List-Unsubscribe: <https://mail.python.org/mailman/options/python-list>,
<mailto:python-list-request@python.org?subject=unsubscribe>
List-Archive: <https://mail.python.org/pipermail/python-list/>
List-Post: <mailto:python-list@python.org>
List-Help: <mailto:python-list-request@python.org?subject=help>
List-Subscribe: <https://mail.python.org/mailman/listinfo/python-list>,
<mailto:python-list-request@python.org?subject=subscribe>
X-Mailman-Original-Message-ID: <CAGGBd_qKfULfPaKWLYbFo6wNBU9WK7tfGw0xCov2dFN4MguhCA@mail.gmail.com>
X-Mailman-Original-References: <CABbU2U98YKdcnJkDPfzE3Pqso+6LL72usB8hrSBVR0WbhauRoQ@mail.gmail.com>

by: Dan Stromberg - Sat, 7 May 2022 16:54 UTC

I believe I'd do something like:

#!/usr/local/cpython-3.10/bin/python3

"""
Output the last 10 lines of a potentially-huge file.

O(n). But technically so is scanning backward from the EOF.

It'd be faster to use a dict, but this has the advantage of working for
huge num_lines.
"""

import dbm

import os

import sys

tempfile = f'/tmp/{os.path.basename(sys.argv[0])}.{os.getpid()}'

db = dbm.open(tempfile, 'n')

num_lines = 10

for cur_lineno, line in enumerate(sys.stdin):

db[str(cur_lineno)] = line.encode('utf-8')

max_lineno = cur_lineno

str_age_out_lineno = str(cur_lineno - num_lines - 1)

if str_age_out_lineno in db:

del db[str_age_out_lineno]

for lineno in range(max_lineno, max_lineno - num_lines, -1):

str_lineno = str(lineno)

if str_lineno not in db:

break

print(db[str(lineno)].decode('utf-8'), end='')

db.close()

os.unlink(tempfile)

On Sat, Apr 23, 2022 at 11:36 AM Marco Sulla <Marco.Sulla.Python@gmail.com>
wrote:

> What about introducing a method for text streams that reads the lines
> from the bottom? Java has also a ReversedLinesFileReader with Apache
> Commons IO.
> --
> https://mail.python.org/mailman/listinfo/python-list
>

Math is like love -- a simple idea but it can get complicated. -- R. Drabek

devel / comp.lang.python / Re: tail