Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

<moshez> ok, I will not marry Jo-Con-El's cow.


computers / news.software.nntp / Re: perl-nocem not processing all NoCeMs by same issuer

SubjectAuthor
* perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
`* Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 +* Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 |`* Re: perl-nocem not processing all NoCeMs by same issuerIvo Gandolfo
 | `* Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 |  `* Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 |   +* Re: perl-nocem not processing all NoCeMs by same issuerIvo Gandolfo
 |   |`- Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 |   `* Re: perl-nocem not processing all NoCeMs by same issuerJulien ÉLIE
 |    +* Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 |    |+- Re: perl-nocem not processing all NoCeMs by same issuerIvo Gandolfo
 |    |`- Re: perl-nocem not processing all NoCeMs by same issuerJulien ÉLIE
 |    `* Re: perl-nocem not processing all NoCeMs by same issuerJesse Rehmer
 |     `- Re: perl-nocem not processing all NoCeMs by same issuerJulien ÉLIE
 `* Re: perl-nocem not processing all NoCeMs by same issuerAdam W.
  `- Re: perl-nocem not processing all NoCeMs by same issuerAdam W.

1
perl-nocem not processing all NoCeMs by same issuer

<ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2069&group=news.software.nntp#2069

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 12:12:41 -0000 (UTC)
Organization: BlueWorld Hosting Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 2 Sep 2023 12:12:41 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com; posting-account="k8cWG9+Y/93vxQYza75s9JQFoL8rgVF3P1Yluveoqs0";
logging-data="37794"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:h29SBVosd6oI+JJqD1b0EJuy62o= sha256:gqmshp8Wujqf7UbR1RwgBSvux2oScYK1kiC0Ro1tl84=
sha1:AndaLIk/sM5ADa1hxxZ9XVcw4KM= sha256:683IzWFZFIavKZzCkBXcMwO6lxMWQoluFLPss0INgFU=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Sat, 2 Sep 2023 12:12 UTC

This morning I added nocem-fr@alphanet.ch's key to the keyring and bit to the
nocem.ctl file. I tried to reprocess some of the recent NoCeMs but found that
a good portion of them don't do anything. I'm not sure if this is something
I'm misunderstanding, or if the notices aren't being generated properly, or if
there is an issue with perl-nocem?

For example, when I process this one cancels are generated as expected:

grephistory '<nocem.ALPHANET.fr.20230902123509.18620.584@nocem.alphanet.ch>' |
perl-nocem

However, many other NoCeMs from the same issuer do nothing:

grephistory '<nocem.ALPHANET.fr.20230902045542.24754.171@nocem.alphanet.ch>' |
perl-nocem
grephistory '<nocem.ALPHANET.fr.20230902125503.23507.509@nocem.alphanet.ch>' |
perl-nocem
grephistory '<nocem.ALPHANET.fr.20230902125530.23877.24@nocem.alphanet.ch>' |
perl-nocem

The only thing I see in the logs related to those is the following:

nocem[25391]: starting up

Re: perl-nocem not processing all NoCeMs by same issuer

<ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2070&group=news.software.nntp#2070

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 12:28:10 -0000 (UTC)
Organization: BlueWorld Hosting Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 2 Sep 2023 12:28:10 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com; posting-account="k8cWG9+Y/93vxQYza75s9JQFoL8rgVF3P1Yluveoqs0";
logging-data="41522"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:E45Pw2PgzGSvIQuie9mUlQY+FGA= sha256:l7ZLCwFF4c0tLsYPIxeTofOwEEeUKe9/iuG47D87JTE=
sha1:cqGIWf++lzUwtcos8Z2IfCibQ6c= sha256:mp+2ZL42bnMWP28DXf/6K6R8Xw5+AKovI4p2L+MyPLo=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Sat, 2 Sep 2023 12:28 UTC

On Sep 2, 2023 at 7:12:41 AM CDT, "Jesse Rehmer"
<jesse.rehmer@blueworldhosting.com> wrote:

> This morning I added nocem-fr@alphanet.ch's key to the keyring and bit to the
> nocem.ctl file. I tried to reprocess some of the recent NoCeMs but found that
> a good portion of them don't do anything. I'm not sure if this is something
> I'm misunderstanding, or if the notices aren't being generated properly, or if
> there is an issue with perl-nocem?
>
> For example, when I process this one cancels are generated as expected:
>
> grephistory '<nocem.ALPHANET.fr.20230902123509.18620.584@nocem.alphanet.ch>' |
> perl-nocem
>
> However, many other NoCeMs from the same issuer do nothing:
>
> grephistory '<nocem.ALPHANET.fr.20230902045542.24754.171@nocem.alphanet.ch>' |
> perl-nocem
> grephistory '<nocem.ALPHANET.fr.20230902125503.23507.509@nocem.alphanet.ch>' |
> perl-nocem
> grephistory '<nocem.ALPHANET.fr.20230902125530.23877.24@nocem.alphanet.ch>' |
> perl-nocem
>
> The only thing I see in the logs related to those is the following:
>
> nocem[25391]: starting up

Determined this is because the issuer uses 'Types' that are not "spam", so I
had to adjust my nocem.ctl to include those types:

nocem-fr@alphanet.ch:spam,cleanfeed-emp,cleanfeed-too-many-newsgroups

Next question, is there an easier way for me to process the notices I have in
news.lists.filters without grabbing Message-IDs one-by-one?

Re: perl-nocem not processing all NoCeMs by same issuer

<ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2071&group=news.software.nntp#2071

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 12:47:25 -0000 (UTC)
Organization: BlueWorld Hosting Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 2 Sep 2023 12:47:25 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com; posting-account="k8cWG9+Y/93vxQYza75s9JQFoL8rgVF3P1Yluveoqs0";
logging-data="78571"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:fAScEGIQe9tcUW4VMEmyGE+5/Jw= sha256:y3oxwdX225iLBxvE2mvv0twQnMH279Im1gJv/e5uhMw=
sha1:LuFrRwjNWraxL09yERx7GfQBLtE= sha256:eZlWgkXlvmOi9DFHc7c5V9Cfs46Fi+57tLSXNjB98Mk=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Sat, 2 Sep 2023 12:47 UTC

On Sep 2, 2023 at 7:28:10 AM CDT, "Jesse Rehmer"
<jesse.rehmer@blueworldhosting.com> wrote:

> On Sep 2, 2023 at 7:12:41 AM CDT, "Jesse Rehmer"
> <jesse.rehmer@blueworldhosting.com> wrote:
>
>> This morning I added nocem-fr@alphanet.ch's key to the keyring and bit to the
>> nocem.ctl file. I tried to reprocess some of the recent NoCeMs but found that
>> a good portion of them don't do anything. I'm not sure if this is something
>> I'm misunderstanding, or if the notices aren't being generated properly, or if
>> there is an issue with perl-nocem?
>>
>> For example, when I process this one cancels are generated as expected:
>>
>> grephistory '<nocem.ALPHANET.fr.20230902123509.18620.584@nocem.alphanet.ch>' |
>> perl-nocem
>>
>> However, many other NoCeMs from the same issuer do nothing:
>>
>> grephistory '<nocem.ALPHANET.fr.20230902045542.24754.171@nocem.alphanet.ch>' |
>> perl-nocem
>> grephistory '<nocem.ALPHANET.fr.20230902125503.23507.509@nocem.alphanet.ch>' |
>> perl-nocem
>> grephistory '<nocem.ALPHANET.fr.20230902125530.23877.24@nocem.alphanet.ch>' |
>> perl-nocem
>>
>> The only thing I see in the logs related to those is the following:
>>
>> nocem[25391]: starting up
>
> Determined this is because the issuer uses 'Types' that are not "spam", so I
> had to adjust my nocem.ctl to include those types:
>
> nocem-fr@alphanet.ch:spam,cleanfeed-emp,cleanfeed-too-many-newsgroups
>
> Next question, is there an easier way for me to process the notices I have in
> news.lists.filters without grabbing Message-IDs one-by-one?

Also, shouldn't perl-nocem be logging when it receives a notice for which one
does not have a key?

I see the logging in the perl-nocem code, but I never get "not in keyring" in
my logs even though I was receiving messages for which were not in my keyring.

Re: perl-nocem not processing all NoCeMs by same issuer

<ucvcfo$ppi7$1@paganini.bofh.team>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2072&group=news.software.nntp#2072

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!paganini.bofh.team!not-for-mail
From: use...@bofh.team (Ivo Gandolfo)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 15:14:32 +0200
Organization: To protect and to server
Message-ID: <ucvcfo$ppi7$1@paganini.bofh.team>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
<ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>
<ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Sat, 2 Sep 2023 13:14:32 -0000 (UTC)
Injection-Info: paganini.bofh.team; logging-data="845383"; posting-host="EScGJCTPxkS5vKB7h2AV4w.user.paganini.bofh.team"; mail-complaints-to="usenet@bofh.team"; posting-account="NnMoDEm3qkIJGKBtBYRUeg";
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.15.0
Cancel-Lock: sha256:w5FHi6jclvXiaoE3SC9ZaRiVBGm2gxY8FCnxof1Maso=
X-Notice: Filtered by postfilter v. 0.9.3
Content-Language: en-US
 by: Ivo Gandolfo - Sat, 2 Sep 2023 13:14 UTC

On 02/09/2023 14:47, Jesse Rehmer wrote:
>>
>> Next question, is there an easier way for me to process the notices I have in
>> news.lists.filters without grabbing Message-IDs one-by-one?
>

This is a script I use when I want to re-run the notice again:

http://paganini.bofh.team/ru_nocem_again.pl

Just adapt them to your system

> Also, shouldn't perl-nocem be logging when it receives a notice for which one
> does not have a key?
>
> I see the logging in the perl-nocem code, but I never get "not in keyring" in
> my logs even though I was receiving messages for which were not in my keyring.

Almost prob. you have same my problem, the "debug" doent's work. Just
edit your perl-nocem script, and change all log "debug" with "err" or
"notice", and u find all on news.err or news.notice. Reload the config
on inn with cltinnd reload

Have fun.

Sincerely

--
Ivo Gandolfo

Re: perl-nocem not processing all NoCeMs by same issuer

<ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2073&group=news.software.nntp#2073

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 13:31:05 -0000 (UTC)
Organization: BlueWorld Hosting Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com> <ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com> <ucvcfo$ppi7$1@paganini.bofh.team>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 2 Sep 2023 13:31:05 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com; posting-account="k8cWG9+Y/93vxQYza75s9JQFoL8rgVF3P1Yluveoqs0";
logging-data="55432"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:D2h6wH6vkwltjBr/K9szTOKqA48= sha256:S3EpnA3mdDQmJ+Yzf/ACW42lOnWPhRc1G7UZA4vqgN8=
sha1:z0ZhDy1IB75qkU6FcIJDdob21Hc= sha256:wPXS/R4NTgTiGPndaCehsYq0bcracHuUFI2si6cpY1g=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Sat, 2 Sep 2023 13:31 UTC

On Sep 2, 2023 at 8:14:32 AM CDT, "Ivo Gandolfo" <usenet@bofh.team> wrote:

> On 02/09/2023 14:47, Jesse Rehmer wrote:
>>>
>>> Next question, is there an easier way for me to process the notices I have in
>>> news.lists.filters without grabbing Message-IDs one-by-one?
>>
>
> This is a script I use when I want to re-run the notice again:
>
> http://paganini.bofh.team/ru_nocem_again.pl
>
> Just adapt them to your system

This will work, thank you!

>
>> Also, shouldn't perl-nocem be logging when it receives a notice for which one
>> does not have a key?
>>
>> I see the logging in the perl-nocem code, but I never get "not in keyring" in
>> my logs even though I was receiving messages for which were not in my keyring.
>
> Almost prob. you have same my problem, the "debug" doent's work. Just
> edit your perl-nocem script, and change all log "debug" with "err" or
> "notice", and u find all on news.err or news.notice. Reload the config
> on inn with cltinnd reload

I see what you mean about some logging set to 'debug', but this portion of the
code that should log the one I mention doesn't specify a level:

if (/^\[GNUPG:\]\s+GOODSIG\s+\S+\s+(.*)/m) {
return 1 if $1 =~ /\Q$issuer\E/;
logmsg("Article $msgid: signed by $1 instead of $issuer");
} elsif (/^\[GNUPG:\]\s+NO_PUBKEY\s+(\S+)/m) {
logmsg("Article $msgid: $issuer (ID $1) not in keyring");
} elsif (/^\[GNUPG:\]\s+BADSIG\s+\S+\s+(.*)/m) {
logmsg("Article $msgid: bad signature from $1");
} elsif (/^\[GNUPG:\]\s+BADARMOR/m or /^\[GNUPG:\]\s+UNEXPECTED/m) {
logmsg("Article $msgid: malformed signature");
} elsif (/^\[GNUPG:\]\s+ERRSIG\s+(\S+)/m) {
# safety net: we get there if we don't know about some token
logmsg("Article $msgid: unknown error (ID $1)");
} else {
# some other error we don't know about happened.
# 126 is returned by the child if exec fails.
s/ at \S+ line \d+\.\n$//;
s/\n/_/;
if ($INN::Config::gpg) {
logmsg(
"Article $msgid: $INN::Config::gpg exited "
. (($status == 126) ? "($_)" : "with status $status"),
'err'
);
} else {
logmsg(
"Article $msgid: $INN::Config::gpgv exited "
. (($status == 126) ? "($_)" : "with status $status"),
'err'
);
}
}
return 0;
}

Re: perl-nocem not processing all NoCeMs by same issuer

<ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2074&group=news.software.nntp#2074

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 13:47:02 -0000 (UTC)
Organization: BlueWorld Hosting Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com> <ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com> <ucvcfo$ppi7$1@paganini.bofh.team> <ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 2 Sep 2023 13:47:02 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com; posting-account="k8cWG9+Y/93vxQYza75s9JQFoL8rgVF3P1Yluveoqs0";
logging-data="48836"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:EmieC6HiuTjMIOg6PeAEISR/CFg= sha256:X8qJrrjvqIfCcjPTboxp/0b89hdOr4ftScyVOIXIgCk=
sha1:Uw0TPelGP5uN1miHTJoiID6n3BI= sha256:1+LsLsXR0vewbYW/HHQHiNWG2/FZlAGLDH4wex9zxno=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Sat, 2 Sep 2023 13:47 UTC

On Sep 2, 2023 at 8:31:05 AM CDT, "Jesse Rehmer"
<jesse.rehmer@blueworldhosting.com> wrote:

> On Sep 2, 2023 at 8:14:32 AM CDT, "Ivo Gandolfo" <usenet@bofh.team> wrote:
>
>> On 02/09/2023 14:47, Jesse Rehmer wrote:
>>>>
>>>> Next question, is there an easier way for me to process the notices I have in
>>>> news.lists.filters without grabbing Message-IDs one-by-one?
>>>
>>
>> This is a script I use when I want to re-run the notice again:
>>
>> http://paganini.bofh.team/ru_nocem_again.pl
>>
>> Just adapt them to your system
>
> This will work, thank you!
>
>>
>>> Also, shouldn't perl-nocem be logging when it receives a notice for which one
>>> does not have a key?
>>>
>>> I see the logging in the perl-nocem code, but I never get "not in keyring" in
>>> my logs even though I was receiving messages for which were not in my keyring.
>>
>> Almost prob. you have same my problem, the "debug" doent's work. Just
>> edit your perl-nocem script, and change all log "debug" with "err" or
>> "notice", and u find all on news.err or news.notice. Reload the config
>> on inn with cltinnd reload
>
> I see what you mean about some logging set to 'debug', but this portion of the
> code that should log the one I mention doesn't specify a level:
> <SNIP>

I see they do end up in debug, if I add news.debug to my syslog configuration
I do see the messages in that log.

When I run your script I notice a lot of occurrences of this error from
perl-nocem, but things seem to be working.

Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
183, <ART> line 44.
Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
183, <ART> line 45.
Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
183, <ART> line 46.

Re: perl-nocem not processing all NoCeMs by same issuer

<ucvg8i$ppi7$2@paganini.bofh.team>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2075&group=news.software.nntp#2075

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!paganini.bofh.team!not-for-mail
From: use...@bofh.team (Ivo Gandolfo)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 16:18:58 +0200
Organization: To protect and to server
Message-ID: <ucvg8i$ppi7$2@paganini.bofh.team>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
<ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>
<ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com>
<ucvcfo$ppi7$1@paganini.bofh.team>
<ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
<ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Sat, 2 Sep 2023 14:18:58 -0000 (UTC)
Injection-Info: paganini.bofh.team; logging-data="845383"; posting-host="EScGJCTPxkS5vKB7h2AV4w.user.paganini.bofh.team"; mail-complaints-to="usenet@bofh.team"; posting-account="NnMoDEm3qkIJGKBtBYRUeg";
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
Thunderbird/102.15.0
Cancel-Lock: sha256:85Y4Zv74Ep0y3MccmmeySRthawVA7zv2ejkAOeciJW8=
X-Notice: Filtered by postfilter v. 0.9.3
Content-Language: en-US
 by: Ivo Gandolfo - Sat, 2 Sep 2023 14:18 UTC

On 02/09/2023 15:47, Jesse Rehmer wrote:
[cut]

Just add where you want the log(, "err") for example:

if (/^\[GNUPG:\]\s+GOODSIG\s+\S+\s+(.*)/m) {
return 1 if $1 =~ /\Q$issuer\E/;
logmsg("Article $msgid: signed by $1 instead of $issuer",
"notice");
} elsif (/^\[GNUPG:\]\s+NO_PUBKEY\s+(\S+)/m) {
logmsg("Article $msgid: $issuer (ID $1) not in keyring", "err");

etc :)

>
> When I run your script I notice a lot of occurrences of this error from
> perl-nocem, but things seem to be working.
>
> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
> 183, <ART> line 44.
> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
> 183, <ART> line 45.
> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
> 183, <ART> line 46.

You can safe ignore them.

Sincerely

--
Ivo Gandolfo

Re: perl-nocem not processing all NoCeMs by same issuer

<ucvh1t$2mvp$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2076&group=news.software.nntp#2076

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 2 Sep 2023 14:32:29 -0000 (UTC)
Organization: BlueWorld Hosting Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ucvh1t$2mvp$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com> <ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com> <ucvg8i$ppi7$2@paganini.bofh.team>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 2 Sep 2023 14:32:29 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com; posting-account="k8cWG9+Y/93vxQYza75s9JQFoL8rgVF3P1Yluveoqs0";
logging-data="89081"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:iZ4orv5yxi1rgXn8s5U84zIa1TU= sha256:bV5SqRw/XiDdemea4NWr9zUAkhmciXTGG2dCV11eOmE=
sha1:/MdOGifmNDBR9n8s/a67ssZhh40= sha256:ca9zIdRsSy9sGgsXliA/UA06FGjAjzpJylDaPNyut6E=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Sat, 2 Sep 2023 14:32 UTC

On Sep 2, 2023 at 9:18:58 AM CDT, "Ivo Gandolfo" <usenet@bofh.team> wrote:

> On 02/09/2023 15:47, Jesse Rehmer wrote:
> [cut]
>
> Just add where you want the log(, "err") for example:
>
> if (/^\[GNUPG:\]\s+GOODSIG\s+\S+\s+(.*)/m) {
> return 1 if $1 =~ /\Q$issuer\E/;
> logmsg("Article $msgid: signed by $1 instead of $issuer",
> "notice");
> } elsif (/^\[GNUPG:\]\s+NO_PUBKEY\s+(\S+)/m) {
> logmsg("Article $msgid: $issuer (ID $1) not in keyring", "err");
>
> etc :)
>
>
>>
>> When I run your script I notice a lot of occurrences of this error from
>> perl-nocem, but things seem to be working.
>>
>> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
>> 183, <ART> line 44.
>> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
>> 183, <ART> line 45.
>> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
>> 183, <ART> line 46.
>
>
> You can safe ignore them.
>
>
> Sincerely

Thank you, Ivo!

Re: perl-nocem not processing all NoCeMs by same issuer

<ud2u8f$2cj5i$1@news.trigofacile.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2077&group=news.software.nntp#2077

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!weretis.net!feeder8.news.weretis.net!proxad.net!feeder1-2.proxad.net!usenet-fr.net!news.trigofacile.com!.POSTED.176.143-2-105.abo.bbox.fr!not-for-mail
From: iul...@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sun, 3 Sep 2023 23:36:15 +0200
Organization: Groupes francophones par TrigoFACILE
Message-ID: <ud2u8f$2cj5i$1@news.trigofacile.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
<ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>
<ucvast$2cnb$1@nnrp.usenet.blueworldhosting.com>
<ucvcfo$ppi7$1@paganini.bofh.team>
<ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
<ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sun, 3 Sep 2023 21:36:15 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="176.143-2-105.abo.bbox.fr:176.143.2.105";
logging-data="2510002"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
Gecko/20100101 Thunderbird/102.14.0
Cancel-Lock: sha1:rM4TpVonY16/OZMqnnZdtxhtGbg= sha256:a1qJ2dxPFoaeQjwg4QAqvU8+oW5nCpoXcrIHBEEhu5M=
sha1:6F6we07cHsaLdrxy+z4KSwcTVPU= sha256:NXE/mDlD4fAgs0qJEooXTY6Cw9QFvXL4J1n2jxcFetM=
In-Reply-To: <ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
 by: Julien ÉLIE - Sun, 3 Sep 2023 21:36 UTC

Hi Jesse,

>>>>> Next question, is there an easier way for me to process the notices I have in
>>>>> news.lists.filters without grabbing Message-IDs one-by-one?

Besides Ivo's script working for the tradspool storage method, there is
also a straight-forward way to do that depending on the overview method
you're using.

With ovsqlite, you obtain all these storage tokens with:

ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' '

See tdx-util and ovdb_stat for a similar dump with tradindexed and ovdb.
I can add a paragraph about that in the perl-nocem documentation as it
is an interesting use case to document.

> Use of uninitialized value $key in lc at /usr/local/news/bin/perl-nocem line
> 183, <ART> line 44.

Noted. I'll fix that warning in the next release.

>>>> Also, shouldn't perl-nocem be logging when it receives a notice for which one
>>>> does not have a key?
>>>>
>>>> I see the logging in the perl-nocem code, but I never get "not in keyring" in
>>>> my logs even though I was receiving messages for which were not in my keyring.

The "not in keyring" log should really be in your news.notice file as
the default log level for the logmsg() function in perl-nocem is notice:
syslog($lvl || 'notice', '%s', $msg);

Are you sure the nocem-fr@alphanet.ch entry was in nocem.ctl when you
did that test?
I think the log you expected was another one, for a issuer not present
in nocem.ctl:

logmsg(
"Article $msgid: unwanted ($hdrs{issuer}/$hdrs{type})",
'debug'
);

In that case, yes, this log is forced at the debug level. I believe the
rationale is for not repeating every couple of minutes that you don't
process the notices or types from one or several issuers. It's not
necessary when you already know that.
Yet, I understand it may be useful to be aware of new issuers or a new
type sent by a known issuer.
Should this very log be changed to be at the notice level? I'm OK with
that move unless you see any drawback. It seems more helpful for admins.

--
Julien ÉLIE

« On ne peut jamais être neutre. Le silence est une opinion. » (Henri
Moret)

Re: perl-nocem not processing all NoCeMs by same issuer

<ufmm3l$2mjm$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2261&group=news.software.nntp#2261

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Thu, 5 Oct 2023 15:52:21 -0000 (UTC)
Organization: BWH Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ufmm3l$2mjm$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com> <ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com> <ud2u8f$2cj5i$1@news.trigofacile.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 5 Oct 2023 15:52:21 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com;
logging-data="88694"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:gsNk5k8WCpAOUbOYZeSFwk7uiZs= sha256:xWQ+HSs3h28gP9q5D4+kVAdT0jmb9yyslP0R7fcupG8=
sha1:YeONAh1d6UevXoKxNnZOGBvz3Aw= sha256:BobjNrK/k0+5BWC1ArJjklRmhaxzODpLzwwq5UyBi40=
X-Usenapp: v1.27.1/d - Full License
 by: Jesse Rehmer - Thu, 5 Oct 2023 15:52 UTC

On Sep 3, 2023 at 4:36:15 PM CDT, "Julien ÉLIE"
<iulius@nom-de-mon-site.com.invalid> wrote:

> Hi Jesse,
>
>>>>>> Next question, is there an easier way for me to process the notices I have in
>>>>>> news.lists.filters without grabbing Message-IDs one-by-one?
>
> Besides Ivo's script working for the tradspool storage method, there is
> also a straight-forward way to do that depending on the overview method
> you're using.
>
> With ovsqlite, you obtain all these storage tokens with:
>
> ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' '
>
> See tdx-util and ovdb_stat for a similar dump with tradindexed and ovdb.
> I can add a paragraph about that in the perl-nocem documentation as it
> is an interesting use case to document.

If it isn't overly complicated to provide an example for tdx-util that would
be helpful to me and I'm sure others. I'll get around to figuring it out at
some point. I'm currently using tradspool so Ivo's script works fine for now.

>>>>> Also, shouldn't perl-nocem be logging when it receives a notice for which one
>>>>> does not have a key?
>>>>>
>>>>> I see the logging in the perl-nocem code, but I never get "not in keyring" in
>>>>> my logs even though I was receiving messages for which were not in my keyring.
>
> The "not in keyring" log should really be in your news.notice file as
> the default log level for the logmsg() function in perl-nocem is notice:
> syslog($lvl || 'notice', '%s', $msg);
>
> Are you sure the nocem-fr@alphanet.ch entry was in nocem.ctl when you
> did that test?
> I think the log you expected was another one, for a issuer not present
> in nocem.ctl:
>
> logmsg(
> "Article $msgid: unwanted ($hdrs{issuer}/$hdrs{type})",
> 'debug'
> );
>
> In that case, yes, this log is forced at the debug level. I believe the
> rationale is for not repeating every couple of minutes that you don't
> process the notices or types from one or several issuers. It's not
> necessary when you already know that.
> Yet, I understand it may be useful to be aware of new issuers or a new
> type sent by a known issuer.
> Should this very log be changed to be at the notice level? I'm OK with
> that move unless you see any drawback. It seems more helpful for admins.

You are correct, what I was looking for then would have been related to
missing issuer in nocem.ctl. I don't think changing it would spam logs too
much?

Re: perl-nocem not processing all NoCeMs by same issuer

<ufmq31$sh74$1@paganini.bofh.team>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2263&group=news.software.nntp#2263

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!nntp.comgw.net!paganini.bofh.team!not-for-mail
From: use...@bofh.team (Ivo Gandolfo)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Thu, 5 Oct 2023 19:00:17 +0200
Organization: To protect and to server
Message-ID: <ufmq31$sh74$1@paganini.bofh.team>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
<ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
<ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
<ud2u8f$2cj5i$1@news.trigofacile.com>
<ufmm3l$2mjm$1@nnrp.usenet.blueworldhosting.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Injection-Date: Thu, 5 Oct 2023 17:00:17 -0000 (UTC)
Injection-Info: paganini.bofh.team; logging-data="935140"; posting-host="D3rsw1DzOU/UxOS1tx8Krg.user.paganini.bofh.team"; mail-complaints-to="usenet@bofh.team"; posting-account="NnMoDEm3qkIJGKBtBYRUeg";
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha256:w+F2IrbW3QxQz9xn3cvMyd2CwXM1z8+AUrGtn9PbUP8=
Content-Language: en-US
X-Notice: Filtered by postfilter v. 0.9.3
 by: Ivo Gandolfo - Thu, 5 Oct 2023 17:00 UTC

On 05/10/2023 17:52, Jesse Rehmer wrote:
> You are correct, what I was looking for then would have been related to
> missing issuer in nocem.ctl. I don't think changing it would spam logs too
> much?

I have areally modified, and this is a 30 day result (I log everything,
and passed 30 day's the old log was moved to a backup server for history
and legal reason):

root@paganini:/var/log# du -hs news/
123M news/
root@paganini:/var/log#

Sincerely

--
Ivo Gandolfo

Re: perl-nocem not processing all NoCeMs by same issuer

<ufmvle$76bs$1@news.trigofacile.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2266&group=news.software.nntp#2266

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!nntp.comgw.net!weretis.net!feeder8.news.weretis.net!news.trigofacile.com!.POSTED.176.143-2-105.abo.bbox.fr!not-for-mail
From: iul...@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Thu, 5 Oct 2023 20:35:26 +0200
Organization: Groupes francophones par TrigoFACILE
Message-ID: <ufmvle$76bs$1@news.trigofacile.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
<ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
<ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
<ud2u8f$2cj5i$1@news.trigofacile.com>
<ufmm3l$2mjm$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Thu, 5 Oct 2023 18:35:26 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="176.143-2-105.abo.bbox.fr:176.143.2.105";
logging-data="235900"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:rLjdjGUZnH9voniKi6T/yOw3Zbw= sha256:PpYaCRPN0XyiJxSub+x4h/autOqv/UJCtTAwEpbXCcI=
sha1:o3JGB1M2nCiXnrPcKRZXPDOUK4M= sha256:tvug6EHhsGyJ2HVDd9ImEzWbhFoxwy6VcjbUV72m0dI=
In-Reply-To: <ufmm3l$2mjm$1@nnrp.usenet.blueworldhosting.com>
 by: Julien ÉLIE - Thu, 5 Oct 2023 18:35 UTC

Hi Jesse,

>>> Next question, is there an easier way for me to process the notices I have in
>>> news.lists.filters without grabbing Message-IDs one-by-one?
>>
>> Besides Ivo's script working for the tradspool storage method, there is
>> also a straight-forward way to do that depending on the overview method
>> you're using.
>>
>> With ovsqlite, you obtain all these storage tokens with:
>>
>> ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' '
>>
>> See tdx-util and ovdb_stat for a similar dump with tradindexed and ovdb.
>> I can add a paragraph about that in the perl-nocem documentation as it
>> is an interesting use case to document.
>
> If it isn't overly complicated to provide an example for tdx-util that would
> be helpful to me and I'm sure others.

Sure. As for tdx-util, the storage token is the 6th field in the output:

tdx-util -g -n news.lists.filters | cut -f 6 -d' '

Piping that into perl-nocem will re-process all the notices in this
newsgroup.

With the Berkeley DB database, it is less straight-forward as the
ovdb_stat program works differently. We have to retrieve the overview
data ("-r 1-" for all the articles, or "-r low-high" for a specific
range of articles), find the Message-ID in the 5th field, and convert
them to storage tokens.

ovdb_stat -r 1- trigofacile.test | cut -f 5 | grephistory -s

I'll add these commands in the perl-nocem documentation as they seem to
prove useful.

--
Julien ÉLIE

« Le vrai danger, ce n'est pas quand les ordinateurs penseront comme les
hommes, c'est quand les hommes penseront comme les ordinateurs. »

Re: perl-nocem not processing all NoCeMs by same issuer

<uh70r2$hfd$3$arnold@news.chmurka.net>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2411&group=news.software.nntp#2411

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!news.chmurka.net!.POSTED.s.v.chmurka.net!not-for-mail
From: gof-cut-...@cut-this-chmurka.net.invalid (Adam W.)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Mon, 23 Oct 2023 23:49:54 -0000 (UTC)
Organization: news.chmurka.net
Message-ID: <uh70r2$hfd$3$arnold@news.chmurka.net>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com>
NNTP-Posting-Host: s.v.chmurka.net
Injection-Date: Mon, 23 Oct 2023 23:49:54 -0000 (UTC)
Injection-Info: news.chmurka.net; posting-account="arnold"; posting-host="s.v.chmurka.net:172.24.44.20";
logging-data="17901"; mail-complaints-to="abuse-news.(at).chmurka.net"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.32-v7+ (armv7l))
Cancel-Lock: sha1:dDr6oxJg8oUCDRUtxidAhmxbwrk=
sha1:mUmtqeYes0R/6CSXpzVf+h05w5s= sha256:IuIvy5V7qtC0PjCvfO3mYdHH27ztm+Hl78nhciKT2a4=
sha1:vcQl+ufKUHJrIPVQgOtwcfD4DiQ= sha256:arHM077w6hFYbUoBeX2LAYhu12Co1PVcS7f53hrragE=
 by: Adam W. - Mon, 23 Oct 2023 23:49 UTC

Jesse Rehmer <jesse.rehmer@blueworldhosting.com> wrote:

> Next question, is there an easier way for me to process the notices I have in
> news.lists.filters without grabbing Message-IDs one-by-one?

I see the date of the post and you probably already solved it (and forgot
about it), but here's my solution. Works for traindexed overview.

for f in $(cat spool/overview/n/l/f/news.lists.filters.DAT |awk -F'\t' '{print $5}' ); do bin/grephistory $f | bin/perl-nocem; done

It will re-feed the full content of news.lists.filters to perl-nocem.
Quite resource intensive.

Re: perl-nocem not processing all NoCeMs by same issuer

<uh70v6$hfd$4$arnold@news.chmurka.net>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2412&group=news.software.nntp#2412

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!news.chmurka.net!.POSTED.s.v.chmurka.net!not-for-mail
From: gof-cut-...@cut-this-chmurka.net.invalid (Adam W.)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Mon, 23 Oct 2023 23:52:06 -0000 (UTC)
Organization: news.chmurka.net
Message-ID: <uh70v6$hfd$4$arnold@news.chmurka.net>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucv9oq$18hi$1@nnrp.usenet.blueworldhosting.com> <uh70r2$hfd$3$arnold@news.chmurka.net>
NNTP-Posting-Host: s.v.chmurka.net
Injection-Date: Mon, 23 Oct 2023 23:52:06 -0000 (UTC)
Injection-Info: news.chmurka.net; posting-account="arnold"; posting-host="s.v.chmurka.net:172.24.44.20";
logging-data="17901"; mail-complaints-to="abuse-news.(at).chmurka.net"
User-Agent: tin/2.6.1-20211226 ("Convalmore") (Linux/5.15.32-v7+ (armv7l))
Cancel-Lock: sha1:Oy6K1X1jNwd22orv9Do+ppYCnU4=
sha1:qC1T2MAt86OlZ8KCfLQMrCr3wMY= sha256:ypgpO51bbvBjjohH7KZ2lnCjyQHoizcN8zL7/vquTVo=
sha1:dUP2wLfxnc+j2d5Jk5gIra5xi/E= sha256:UbYzxJU53NmpFQbC0DJoCJsYgs21tEcO+ng+QF+kzWo=
 by: Adam W. - Mon, 23 Oct 2023 23:52 UTC

Adam W. <gof-cut-this-news@cut-this-chmurka.net.invalid> wrote:

> for f in $(cat spool/overview/n/l/f/news.lists.filters.DAT |awk -F'\t' '{print $5}' ); do bin/grephistory $f | bin/perl-nocem; done

Sorry, old version. It's better this way (runs perl-nocem only once,
not for every message):

for f in $(cat spool/overview/n/l/f/news.lists.filters.DAT |awk -F'\t' '{print $5}' ); do bin/grephistory $f; done | bin/perl-nocem

Re: perl-nocem not processing all NoCeMs by same issuer

<ui34jh$aqo$1@nnrp.usenet.blueworldhosting.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2464&group=news.software.nntp#2464

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!usenet.blueworldhosting.com!diablo1.usenet.blueworldhosting.com!nnrp.usenet.blueworldhosting.com!.POSTED!not-for-mail
From: jesse.re...@blueworldhosting.com (Jesse Rehmer)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Fri, 3 Nov 2023 15:45:53 -0000 (UTC)
Organization: BWH Usenet (https://usenet.blueworldhosting.com)
Message-ID: <ui34jh$aqo$1@nnrp.usenet.blueworldhosting.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com> <ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com> <ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com> <ud2u8f$2cj5i$1@news.trigofacile.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=fixed
Content-Transfer-Encoding: 8bit
Injection-Date: Fri, 3 Nov 2023 15:45:53 -0000 (UTC)
Injection-Info: nnrp.usenet.blueworldhosting.com;
logging-data="11096"; mail-complaints-to="usenet@blueworldhosting.com"
User-Agent: Usenapp for MacOS
Cancel-Lock: sha1:mgQ8Tzl9OuAfdV5V6WEUJusxEWg= sha256:BpcnQA/UCe39iHdUKS41QxYU5XlTwYQ/EpGaN7TTKGU=
sha1:V7WMjSgUNSSAIWhYiYrd79qgLJ0= sha256:1h8s49rpAo4C5EHhi43zaPPiVoVx245uvgDgsml0ey4=
X-Usenapp: v1.27.2/d - Full License
 by: Jesse Rehmer - Fri, 3 Nov 2023 15:45 UTC

On Sep 3, 2023 at 4:36:15 PM CDT, "Julien ÉLIE"
<iulius@nom-de-mon-site.com.invalid> wrote:

> Hi Jesse,
>
>>>>>> Next question, is there an easier way for me to process the notices I have in
>>>>>> news.lists.filters without grabbing Message-IDs one-by-one?
>
> Besides Ivo's script working for the tradspool storage method, there is
> also a straight-forward way to do that depending on the overview method
> you're using.
>
> With ovsqlite, you obtain all these storage tokens with:
>
> ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' '

Hi Julien,

Finally got my history and overview rebuilt with ovsqlite...

When I use the command above piped to perl-nocem, innd grinds to a halt. It
seems to process a few nocem articles and then everything stops. No articles
processed from peers and incoming connections hang, nothing reported in the
logs except perl-nocem starting. I don't see anything with truss against innd
or perl-nocem, and as soon as I cancel the command everything springs back to
life.

If I take the output of the ovqlite-util command to a file and do the
following it seems to run fine:

ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' ' > list
perl-nocem < list

When I run it like this everything hangs:

ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' ' | perl-nocem

Re: perl-nocem not processing all NoCeMs by same issuer

<ui57rq$1kh2p$1@news.trigofacile.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=2465&group=news.software.nntp#2465

 copy link   Newsgroups: news.software.nntp
Path: i2pn2.org!i2pn.org!news.hispagatos.org!eternal-september.org!feeder2.eternal-september.org!eternal-september.org!news.trigofacile.com!.POSTED.san13-h02-176-143-2-105.dsl.sta.abo.bbox.fr!not-for-mail
From: iul...@nom-de-mon-site.com.invalid (Julien ÉLIE)
Newsgroups: news.software.nntp
Subject: Re: perl-nocem not processing all NoCeMs by same issuer
Date: Sat, 4 Nov 2023 11:53:46 +0100
Organization: Groupes francophones par TrigoFACILE
Message-ID: <ui57rq$1kh2p$1@news.trigofacile.com>
References: <ucv8rp$14t2$1@nnrp.usenet.blueworldhosting.com>
<ucvdep$1m48$1@nnrp.usenet.blueworldhosting.com>
<ucvecm$1fm4$1@nnrp.usenet.blueworldhosting.com>
<ud2u8f$2cj5i$1@news.trigofacile.com>
<ui34jh$aqo$1@nnrp.usenet.blueworldhosting.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Date: Sat, 4 Nov 2023 10:53:46 -0000 (UTC)
Injection-Info: news.trigofacile.com; posting-account="julien"; posting-host="san13-h02-176-143-2-105.dsl.sta.abo.bbox.fr:176.143.2.105";
logging-data="1721433"; mail-complaints-to="abuse@trigofacile.com"
User-Agent: Mozilla Thunderbird
Cancel-Lock: sha1:2mE2RkhmWfOIfj7OY4E/rVqVNA8= sha256:6VA82QSrbW+Ubm6cgDkdtvigSqgpSphAADBi34x0r8U=
sha1:xRv4ZV7tBjZw4y3ctUTnZjrfDZ4= sha256:qpRn4lv/GePAY3YXXgSWnZaguDVH9Zvzfe9Y5YvJjnU=
In-Reply-To: <ui34jh$aqo$1@nnrp.usenet.blueworldhosting.com>
 by: Julien ÉLIE - Sat, 4 Nov 2023 10:53 UTC

Hi Jesse,

>> With ovsqlite, you obtain all these storage tokens with:
>>
>> ovsqlite-util -g -n news.lists.filters | cut -f 5 -d' '
>
> Finally got my history and overview rebuilt with ovsqlite...

Good news :)

> When I use the command above piped to perl-nocem, innd grinds to a halt. It
> seems to process a few nocem articles and then everything stops. No articles
> processed from peers and incoming connections hang, nothing reported in the
> logs except perl-nocem starting. I don't see anything with truss against innd
> or perl-nocem, and as soon as I cancel the command everything springs back to
> life.

I also see the same behaviour, indeed, when thousands of articles are
processed.
After investigation, perl-nocem hangs at the receival of a response from
innd:

foreach (@$ids) {
print NNTP "$_\r\n";
if (($r = <NNTP>) !~ /^289/) {
$r =~ s/\r\n$//;
logmsg("cannot cancel $_: $r", 'err');
goto ERR;
}
}

<NNTP> waits for more bytes.

Yet, innd has properly seen the article, as enabling traces (ctlinnd
trace innd y) shows at debug level:

Nov 4 11:23:53 news innd[1620966]: localhost:46 NCproc state=15 next
"<O9KdnVDIDbin9V"
Nov 4 11:23:53 news innd[1620966]: localhost:46 <
<O9KdnVDIDbin9VP_nZ2dnUU7_83NnZ2d@giganews.com>

and innd logs a response only when I stop perl-nocem:

Nov 4 11:24:01 news innd[1620966]: localhost:46 NCwritereply
26=write(46, "289 Article can", 26)
Nov 4 11:24:01 news innd[1620966]: localhost:46 > 289 Article cancelled OK

As you noted, the behaviour is different when running "perl-nocem <
file" instead of piping the ovsqlite-util result into perl-nocem.
There's no hang in that case.

I am unsure what should be fixed there; innd shouldn't really be
indefinitely blocked in reading in the Unix-domain socket opened by
perl-nocem. Seems like after NCwritedone(), the channel is added to be
read (RCHANadd) but nothing is to be read as perl-nocem did not receive
any response for whatever reason.

--
Julien ÉLIE

« There's a way of transferring funds that is even faster than
electronic banking. It's called marriage. » (Sam Kinison)

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor