Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

<wiggy> in a stunning new move I actually tested this upload


computers / news.software.readers / Re: Bug in SLRN trunk: not seen in 1.0.3.

SubjectAuthor
* Re: Bug in SLRN trunk: not seen in 1.0.3.Julien ÉLIE
+- Re: Bug in SLRN trunk: not seen in 1.0.3.Retro Guy
+* Re: Bug in SLRN trunk: not seen in 1.0.3.Russ Allbery
|`* Re: Bug in SLRN trunk: not seen in 1.0.3.Julien ÉLIE
| `- Re: Bug in SLRN trunk: not seen in 1.0.3.Russ Allbery
`- Re: Bug in SLRN trunk: not seen in 1.0.3.Julien ÉLIE

1
Re: Bug in SLRN trunk: not seen in 1.0.3.

<udpkdo$2sh67$1@news.trigofacile.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1433&group=news.software.readers#1433

 copy link   Newsgroups: news.software.nntp news.software.readers
Path: i2pn2.org!i2pn.org!nntp.comgw.net!weretis.net!feeder8.news.weretis.net!news.trigofacile.com!.POSTED.176.143-2-105.abo.bbox.fr!not-for-mail
From: iul...@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp,news.software.readers
Subject: Re: Bug in SLRN trunk: not seen in 1.0.3.
Date: Tue, 12 Sep 2023 14:09:28 +0200
Organization: Groupes francophones par TrigoFACILE
Message-ID: <udpkdo$2sh67$1@news.trigofacile.com>
References: <20230911185353.581@kylheku.com>
<slrnufvq4m.15l9v.rayban@raybanana.net> <20230911214521.831@kylheku.com>
<20230911222037.955@kylheku.com> <877cowj6sm.fsf@hope.eyrie.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 12 Sep 2023 12:09:28 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="176.143-2-105.abo.bbox.fr:176.143.2.105";
logging-data="3032263"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
Gecko/20100101 Thunderbird/102.15.0
Cancel-Lock: sha1:xw5g0wP66HQNY5pwInwZHHD+ELk= sha256:nv7Hl6RBr+BwfYwyVVnAhWKb8Bt6KxeMroDsTKFdQGM=
sha1:vvUTTtweNvBb83wsLeVKMLKTa4E= sha256:ERciJEJeIhK+sEx9CTUJryNRG4Y8G71urkxsLeSfwj0=
In-Reply-To: <877cowj6sm.fsf@hope.eyrie.org>
 by: Julien ÉLIE - Tue, 12 Sep 2023 12:09 UTC

Hi Russ,

> The basic problem is that from the perspective of slrn it's an
> inconsistency in the overview database (and maybe the active file; I'm not
> sure precisely what the database looks like since it's been a while since
> I've looked at this).

The high water mark never decreases in the active file as this
information is used by innd to assign the next unused article number.

Though overview data may be handled differently, I think the same rule
is also followed, and the expiry process does not decrease high water
marks in overview data, but only updates low water marks, article counts
and removes old articles. It is what I have in mind from recent
investigation in the 4 overview methods when looking at how they handle
empty newsgroups.
Just to be sure, I have just cancelled an article in a local testing
group, and will report tomorrow.

> The thing is, slrn doesn't know which way that inconsistency will
> eventually resolve. We all know that it's probably because the articles
> were deleted and thus this will be resolved by deleting the overview
> records as well, and this isn't done because due to the structure of the
> overview database in the most common configurations deletions are
> expensive and are therefore batched. But from slrn's perspective, it's
> possible that the articles are going to show up later and make the
> overview correct (and there are indeed some configurations where that can
> possibly happen). So it's making the conservative decision to keep
> retrying those articles to see if they start working, until the overview
> records are removed.

In order to improve the experience of news clients, wouldn't a
groupexacthigh parameter in readers.conf be useful in INN?
The idea would be that nnrpd would give a high water mark corresponding
to an existing article when responding to GROUP, LISTGROUP, LIST ACTIVE
and LIST COUNTS.

In Kaz's example where ~/.jewsrc has "comp.lang.c: 1-254212" but the
news server reports that the highest available article number is 254214,
using "groupexacthigh: 5" in readers.conf would result in the news
server reporting that the highest available article number is 254212.
It would probe the existence of articles from 254214 to (254214-5), and
report the highest existing one, or (254214-5), to the client.

In the latest INN 2.7.1 release, a groupexactcount parameter was added
to report the real article *count* instead of an estimate. When the
estimated count of articles is strictly smaller than groupexactcount,
nnrpd recounts the number of still existing articles, and report the
exact value. If groupexactcount is set to 0, nnrpd always recounts. If
set to 1, it never recounts.
Though exact article counts are not required by the NNTP protocol, they
may be useful to distinguish empty newsgroups by reporting an exact
count of 0 instead of an estimate of 1 (when it happens, the news client
may show that 1 article is present in the newsgroup, then it tries to
retrieve the article, and finally does not show anything to the user).
The default value for this parameter is 5.

I suggest a similar parameter for the high water mark, as it also sounds
useful (independently of whether there is a regression in slrn 1.0.4,
other news clients may also report that 2 new articles are available in
comp.lang.c but on entering the group and downloading its real overview
data, the number of new articles to read becomes 0).

I add the news.software.readers newsgroup in this discussion. Maybe
some people have there information about news clients reporting
available articles, and then nothing when entering the group?

--
Julien ÉLIE

« Life is short… so eat dessert first! »

Re: Bug in SLRN trunk: not seen in 1.0.3.

<369f4305b571e2c5dd663da796d82ef6@news.novabbs.org>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1434&group=news.software.readers#1434

 copy link   Newsgroups: news.software.nntp news.software.readers
Path: i2pn2.org!.POSTED!not-for-mail
From: retro....@rocksolidbbs.com (Retro Guy)
Newsgroups: news.software.nntp,news.software.readers
Subject: Re: Bug in SLRN trunk: not seen in 1.0.3.
Date: Tue, 12 Sep 2023 14:02:41 +0000
Organization: Rocksolid Light
Message-ID: <369f4305b571e2c5dd663da796d82ef6@news.novabbs.org>
References: <20230911185353.581@kylheku.com> <slrnufvq4m.15l9v.rayban@raybanana.net> <20230911214521.831@kylheku.com> <20230911222037.955@kylheku.com> <877cowj6sm.fsf@hope.eyrie.org> <udpkdo$2sh67$1@news.trigofacile.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org;
logging-data="1668811"; mail-complaints-to="usenet@i2pn2.org";
posting-account="PGd4t4cXnWwgUWG9VtTiCsm47oOWbHLcTr4rYoM0Edo";
User-Agent: Rocksolid Light 0.9.1
X-Rslight-Posting-User: 91053d4a47d51b416144568e5a1040f05e31ed1b
X-Rslight-Site: $2y$10$Z3zGEFc/qmwZ7D8lJ64EbeouKQVmTWqk3bIvCrpxxwJInVckvmdB2
X-Spam-Checker-Version: SpamAssassin 4.0.0 (2022-12-13) on i2pn2.org
X-Face: .&YR-G(w(DZ$$,}%k=]*5*!p'=(anr"IT`wZG'2VWdfl\r)l[42u7JH`n(JUQ*e5*A|XCDf
?&\X&uwkl38"CYX3O8m}C8E4p'%N$2#kSTVzx{Ly|DjLT\Vk7NE}NQ(VC$Yq]i:7|z[.9iv^g>*8_B
H0=hZt'[%)4kG|
 by: Retro Guy - Tue, 12 Sep 2023 14:02 UTC

Julien_ÉLIE wrote:

> Hi Russ,

>> The basic problem is that from the perspective of slrn it's an
>> inconsistency in the overview database (and maybe the active file; I'm not
>> sure precisely what the database looks like since it's been a while since
>> I've looked at this).

> The high water mark never decreases in the active file as this
> information is used by innd to assign the next unused article number.

>> The thing is, slrn doesn't know which way that inconsistency will
>> eventually resolve. We all know that it's probably because the articles
>> were deleted and thus this will be resolved by deleting the overview
>> records as well, and this isn't done because due to the structure of the
>> overview database in the most common configurations deletions are
>> expensive and are therefore batched. But from slrn's perspective, it's
>> possible that the articles are going to show up later and make the
>> overview correct (and there are indeed some configurations where that can
>> possibly happen). So it's making the conservative decision to keep
>> retrying those articles to see if they start working, until the overview
>> records are removed.

> In order to improve the experience of news clients, wouldn't a
> groupexacthigh parameter in readers.conf be useful in INN?
> The idea would be that nnrpd would give a high water mark corresponding
> to an existing article when responding to GROUP, LISTGROUP, LIST ACTIVE
> and LIST COUNTS.

This is how my simple nnrpd server handles this, and clients seem to respond
fine to it. Missing articles (deleted, cancelled, NoCeM) are not listed
and not counted. When the highest article is deleted, it is noted in a file
and not reused.

Here's a sample:

200 Rocksolid Light NNTP Server ready (no posting)
group rocksolid.test.test
211 180 2 214 rocksolid.test.test
quit
205 closing connection - goodbye!

Then, I delete the highest article (214)

200 Rocksolid Light NNTP Server ready (no posting)
group rocksolid.test.test
211 179 2 213 rocksolid.test.test
quit
205 closing connection - goodbye!

Then I post a new article

200 Rocksolid Light NNTP Server ready (no posting)
group rocksolid.test.test
211 180 2 215 rocksolid.test.test
quit
Connection closed by foreign host.

The same counting applies to any nnrpd commands requesting count
infomation. So far I have so far not seen any issues with clients.

--
Retro Guy

Re: Bug in SLRN trunk: not seen in 1.0.3.

<87v8cfgxyo.fsf@hope.eyrie.org>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1439&group=news.software.readers#1439

 copy link   Newsgroups: news.software.nntp news.software.readers
Path: i2pn2.org!i2pn.org!paganini.bofh.team!news.killfile.org!news.eyrie.org!.POSTED!not-for-mail
From: eag...@eyrie.org (Russ Allbery)
Newsgroups: news.software.nntp,news.software.readers
Subject: Re: Bug in SLRN trunk: not seen in 1.0.3.
Date: Tue, 12 Sep 2023 09:25:51 -0700
Organization: The Eyrie
Message-ID: <87v8cfgxyo.fsf@hope.eyrie.org>
References: <20230911185353.581@kylheku.com>
<slrnufvq4m.15l9v.rayban@raybanana.net>
<20230911214521.831@kylheku.com> <20230911222037.955@kylheku.com>
<877cowj6sm.fsf@hope.eyrie.org> <udpkdo$2sh67$1@news.trigofacile.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Injection-Info: hope.eyrie.org;
logging-data="17090"; mail-complaints-to="news@eyrie.org"
User-Agent: Gnus/5.13 (Gnus v5.13)
Cancel-Lock: sha1:5h1P6Lyge3BI7E6ih7wOgk1d4Xk=
 by: Russ Allbery - Tue, 12 Sep 2023 16:25 UTC

Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:

> The high water mark never decreases in the active file as this
> information is used by innd to assign the next unused article number.

The initial number of unread articles is probably based entirely on the
high water mark, but to me the question is why slrn doesn't mark those
articles as read the first time you enter the group and they don't exist.
I am assuming that this is because it sees phantom OVER entries for them,
but then the articles are missing when it goes to retrieve them. This
could indicate, say, an overchan setup where the overview was written
before the article was written, so it is arguably correct to not mark such
articles as read.

If the OVER command also doesn't return any data for those articles,
though, I think slrn really should just mark them as read and move on.
Articles that are both missing and don't have OVER entries probably have
been deleted. (Although I guess this could be wrong if somehow the high
water mark got updated before *either* the article was written or the
overview database was updated. I don't remember if that can happen.)

> Though overview data may be handled differently, I think the same rule
> is also followed, and the expiry process does not decrease high water
> marks in overview data, but only updates low water marks, article counts
> and removes old articles. It is what I have in mind from recent
> investigation in the 4 overview methods when looking at how they handle
> empty newsgroups.

I thought the overview data for the deleted article would be dropped from
the database during nightly expire, so will not appear in the OVER output,
but it's been a while since I looked at it so I could be wrong. In other
words, I think you're correct that the high water marks would not change,
since that's how INN ensures that it never reuses an article number, but
when you run OVER on the group, I believe the entries for those articles
should be missing.

> In order to improve the experience of news clients, wouldn't a
> groupexacthigh parameter in readers.conf be useful in INN? The idea
> would be that nnrpd would give a high water mark corresponding to an
> existing article when responding to GROUP, LISTGROUP, LIST ACTIVE and
> LIST COUNTS.

Yes, that would also solve the problem. It won't avoid the appearance of
three unread articles and then seeing only one unread article the next
time an article arrives that isn't deleted, but I think that's unavoidable
in the protocol.

--
Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

Re: Bug in SLRN trunk: not seen in 1.0.3.

<udq8js$2ss6l$2@news.trigofacile.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1440&group=news.software.readers#1440

 copy link   Newsgroups: news.software.nntp news.software.readers
Path: i2pn2.org!i2pn.org!news.nntp4.net!news.gegeweb.eu!gegeweb.org!news.trigofacile.com!.POSTED.san13-h02-176-143-2-105.dsl.sta.abo.bbox.fr!not-for-mail
From: iul...@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp,news.software.readers
Subject: Re: Bug in SLRN trunk: not seen in 1.0.3.
Date: Tue, 12 Sep 2023 19:54:04 +0200
Organization: Groupes francophones par TrigoFACILE
Message-ID: <udq8js$2ss6l$2@news.trigofacile.com>
References: <20230911185353.581@kylheku.com>
<slrnufvq4m.15l9v.rayban@raybanana.net> <20230911214521.831@kylheku.com>
<20230911222037.955@kylheku.com> <877cowj6sm.fsf@hope.eyrie.org>
<udpkdo$2sh67$1@news.trigofacile.com> <87v8cfgxyo.fsf@hope.eyrie.org>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Tue, 12 Sep 2023 17:54:04 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="san13-h02-176-143-2-105.dsl.sta.abo.bbox.fr:176.143.2.105";
logging-data="3043541"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
Gecko/20100101 Thunderbird/102.15.0
Cancel-Lock: sha1:TfVkZGKqidY8yw2A3ukMi4Ocqiw= sha256:Gy+ZdyRnzPBqG0Y+yxRL7qgWI/uZX+ujui9bI/xXick=
sha1:45YH9wg7A38t/a+puHqqmap+s0g= sha256:WqC7g3D6dd0ASxnGIJ1OXCth5nGyJnIs+OaYnRVXVb4=
In-Reply-To: <87v8cfgxyo.fsf@hope.eyrie.org>
 by: Julien ÉLIE - Tue, 12 Sep 2023 17:54 UTC

Hi Russ,

>> Though overview data may be handled differently, I think the same rule
>> is also followed, and the expiry process does not decrease high water
>> marks in overview data, but only updates low water marks, article counts
>> and removes old articles. It is what I have in mind from recent
>> investigation in the 4 overview methods when looking at how they handle
>> empty newsgroups.
>
> I thought the overview data for the deleted article would be dropped from
> the database during nightly expire, so will not appear in the OVER output,
> but it's been a while since I looked at it so I could be wrong. In other
> words, I think you're correct that the high water marks would not change,
> since that's how INN ensures that it never reuses an article number, but
> when you run OVER on the group, I believe the entries for those articles
> should be missing.

The entries for cancelled articles no longer are in the overview data
returned by an OVER request. When processing a cancel, the rows are
deleted in ovdb/ovsqlite database and the index entry is deleted in
tradindexed. OVER no longer shows them.
The expiration process will then properly remove the overview data in
tradindexed.

The use case when the overview data is still present but no longer the
article, is when using CNFS and the article has been overwritten. OVER
will report the article until the next expire run.

>> In order to improve the experience of news clients, wouldn't a
>> groupexacthigh parameter in readers.conf be useful in INN? The idea
>> would be that nnrpd would give a high water mark corresponding to an
>> existing article when responding to GROUP, LISTGROUP, LIST ACTIVE and
>> LIST COUNTS.
>
> Yes, that would also solve the problem. It won't avoid the appearance of
> three unread articles and then seeing only one unread article the next
> time an article arrives that isn't deleted, but I think that's unavoidable
> in the protocol.

Ah yes, indeed, there may be gaps in the available article numbers. The
problem solved here would be to show unread articles whereas there are
no new articles to read, but not the exact count of unread articles,
which is impossible to know from a mere GROUP command.
The client may send a "LISTGROUP latest high-new advertised high"
command and counts itself, but it will slow down a bit the reading
experience.

--
Julien ÉLIE

« Sum, ergo bibo ; bibo, ergo sum. »

Re: Bug in SLRN trunk: not seen in 1.0.3.

<878r9bgt6c.fsf@hope.eyrie.org>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1441&group=news.software.readers#1441

 copy link   Newsgroups: news.software.nntp news.software.readers
Path: i2pn2.org!i2pn.org!paganini.bofh.team!news.killfile.org!news.eyrie.org!.POSTED!not-for-mail
From: eag...@eyrie.org (Russ Allbery)
Newsgroups: news.software.nntp,news.software.readers
Subject: Re: Bug in SLRN trunk: not seen in 1.0.3.
Date: Tue, 12 Sep 2023 11:09:15 -0700
Organization: The Eyrie
Message-ID: <878r9bgt6c.fsf@hope.eyrie.org>
References: <20230911185353.581@kylheku.com>
<slrnufvq4m.15l9v.rayban@raybanana.net>
<20230911214521.831@kylheku.com> <20230911222037.955@kylheku.com>
<877cowj6sm.fsf@hope.eyrie.org> <udpkdo$2sh67$1@news.trigofacile.com>
<87v8cfgxyo.fsf@hope.eyrie.org> <udq8js$2ss6l$2@news.trigofacile.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Injection-Info: hope.eyrie.org;
logging-data="17090"; mail-complaints-to="news@eyrie.org"
User-Agent: Gnus/5.13 (Gnus v5.13)
Cancel-Lock: sha1:6fDqtsppmyYG6lUWTCsYIp5ygvs=
 by: Russ Allbery - Tue, 12 Sep 2023 18:09 UTC

Julien ÉLIE <iulius@nom-de-mon-site.com.invalid> writes:

> The entries for cancelled articles no longer are in the overview data
> returned by an OVER request. When processing a cancel, the rows are
> deleted in ovdb/ovsqlite database and the index entry is deleted in
> tradindexed. OVER no longer shows them.

Oh! Okay, so my diagnosis of the problem was wrong because I forgot that
we did hide the overview information immediately.

So you're right, this probably is entirely about the high water mark.

--
Russ Allbery (eagle@eyrie.org) <https://www.eyrie.org/~eagle/>

Please post questions rather than mailing me directly.
<https://www.eyrie.org/~eagle/faqs/questions.html> explains why.

Re: Bug in SLRN trunk: not seen in 1.0.3.

<uds2fp$2ua8g$1@news.trigofacile.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=1442&group=news.software.readers#1442

 copy link   Newsgroups: news.software.nntp news.software.readers
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!news.trigofacile.com!.POSTED.san13-h02-176-143-2-105.dsl.sta.abo.bbox.fr!not-for-mail
From: iul...@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp,news.software.readers
Subject: Re: Bug in SLRN trunk: not seen in 1.0.3.
Date: Wed, 13 Sep 2023 12:21:45 +0200
Organization: Groupes francophones par TrigoFACILE
Message-ID: <uds2fp$2ua8g$1@news.trigofacile.com>
References: <20230911185353.581@kylheku.com>
<slrnufvq4m.15l9v.rayban@raybanana.net> <20230911214521.831@kylheku.com>
<20230911222037.955@kylheku.com> <877cowj6sm.fsf@hope.eyrie.org>
<udpkdo$2sh67$1@news.trigofacile.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Wed, 13 Sep 2023 10:21:45 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="san13-h02-176-143-2-105.dsl.sta.abo.bbox.fr:176.143.2.105";
logging-data="3090704"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
Gecko/20100101 Thunderbird/102.15.0
Cancel-Lock: sha1:STl4Pf37k25AbbVOJKIpjJ+8Ow8= sha256:BEhj/Q5V9hkk7tqa5q8M4MCdjpMfJQb6tXzqi289g9w=
sha1:chYSkzDCYXnI1n9xYt2Op8X3kwA= sha256:1QsYVmUInNQL1F5PMGCkV4qaNRs5tdqo13pj6wY3T5o=
In-Reply-To: <udpkdo$2sh67$1@news.trigofacile.com>
 by: Julien ÉLIE - Wed, 13 Sep 2023 10:21 UTC

Responding to my previous article:
> The high water mark never decreases in the active file as this
> information is used by innd to assign the next unused article number.
>
> Though overview data may be handled differently, I think the same rule
> is also followed, and the expiration process does not decrease high water
> marks in overview data, but only updates low water marks, article counts
> and removes old articles.
> Just to be sure, I have just cancelled an article in a local testing
> group, and will report tomorrow.

I confirm the high water mark does not decrease in the overview data
after an expiration run. Same thing as for the active file.
The expiration process properly cleans up the related (hidden) expired
overview data, as discussed yesterday.

--
Julien ÉLIE

« Donec eris felix, multos numerabis amicos. » (Ovide)

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor