Hi all,

I have a fairly large killfile. Frequently, Gnus fetches the news and shows
me that there are groups with unread messages, but when I try to go into one
of those groups, Gnus fetches more information and realizes that all of the
unread articles have been scored down below the threshold of being visible.

This is a bit annoying. I'd like to get Gnus to apply my scores in each
group as soon as it has fetched news.

This is my attempt to get that behavior:

(defun enter-all-groups-to-force-scoring ()
(gnus-message 5 "Scoring...")
(catch 'done-looping
(let ((seen-newsgroups '())
(most-recent-newsgroup nil))
(while t
(gnus-group-select-group)
(setq most-recent-newsgroup gnus-newsgroup-name)
(gnus-summary-exit)
(if (seq-contains-p seen-newsgroups most-recent-newsgroup)
(throw 'done-looping nil)
(setq seen-newsgroups (cons most-recent-newsgroup
seen-newsgroups))))))
(gnus-message 5 "Scoring... done"))

(add-hook 'gnus-group-prepare-hook
'enter-all-groups-to-force-scoring)
(add-hook 'gnus-after-getting-new-news-hook
'enter-all-groups-to-force-scoring)

This seems to work *sometimes*--I can watch as groups that had unread
articles are reduced to zero unread articles. Other times, though, it seems
that scores in a particular group were not applied, and I run into the
sequence of events described at the top of this post.

(This solution also opens the last newsgroup twice, because I wasn't sure
how else to determine that we're done looping through the groups!)

Is there a better way of accomplishing this bit of configuration?

Thanks,

Benjamin

(I also asked this question on the Emacs Stack Exchange site at
<https://emacs.stackexchange.com/q/77110/24262>.)

Benjamin Esham wrote:

> I have a fairly large killfile. Frequently, Gnus fetches the news and
> shows me that there are groups with unread messages, but when I try to
> go into one of those groups, Gnus fetches more information and
> realizes that all of the unread articles have been scored down below
> the threshold of being visible.

did you try `M-x g-ba-s' from group buffer?

if that was not what you had in mind, you could also post on
<nntp://news.eternal-september.org/gnu.emacs.gnus>.

On Mon, 08 May 2023 16:36:58 -0400, Benjamin Esham wrote:

> Hi all,
>
> I have a fairly large killfile. Frequently, Gnus fetches the news and shows
> me that there are groups with unread messages, but when I try to go into one
> of those groups, Gnus fetches more information and realizes that all of the
> unread articles have been scored down below the threshold of being visible.
>
> This is a bit annoying. I'd like to get Gnus to apply my scores in each
> group as soon as it has fetched news.
>
> This is my attempt to get that behavior:
>
> (defun enter-all-groups-to-force-scoring ()
> (gnus-message 5 "Scoring...")
> (catch 'done-looping
> (let ((seen-newsgroups '())
> (most-recent-newsgroup nil))
> (while t
> (gnus-group-select-group)
> (setq most-recent-newsgroup gnus-newsgroup-name)
> (gnus-summary-exit)
> (if (seq-contains-p seen-newsgroups most-recent-newsgroup)
> (throw 'done-looping nil)
> (setq seen-newsgroups (cons most-recent-newsgroup
> seen-newsgroups))))))
> (gnus-message 5 "Scoring... done"))
>
> (add-hook 'gnus-group-prepare-hook
> 'enter-all-groups-to-force-scoring)
> (add-hook 'gnus-after-getting-new-news-hook
> 'enter-all-groups-to-force-scoring)
>
> This seems to work *sometimes*--I can watch as groups that had unread
> articles are reduced to zero unread articles. Other times, though, it seems
> that scores in a particular group were not applied, and I run into the
> sequence of events described at the top of this post.
>
> (This solution also opens the last newsgroup twice, because I wasn't sure
> how else to determine that we're done looping through the groups!)
>
> Is there a better way of accomplishing this bit of configuration?
>
> Thanks,
>
> Benjamin
>
> (I also asked this question on the Emacs Stack Exchange site at
> <https://emacs.stackexchange.com/q/77110/24262>.)

I'm not a LISP guy, but are you by chance scoring on article
headers (or the article body) that are not available until the
whole article is retrieved?

News servers only dish out a subset of article headers when
getting a list of articles for perusal and initial scoring(*).
Only when the entire article is retrieved for full viewing can it
be scored on "non-XOVER" headers (or the body text).

(*) To see the list of headers your news server provides
initially, telnet into your news server (usually at port 119, at
least) and give it a "list overview.fmt" command.

Doubtful you're on Windows 1[0-1], but if you are it would look
something like this:

Microsoft Windows [Version 10.0.19044.2965]
(c) Microsoft Corporation. All rights reserved.

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"

C:\Users\Sqwertz>telnet usnews.blocknews.net 119
'telnet' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>cmd.exe
'cmd.exe' is not recognized as an internal or external command,
operable program or batch file.

C:\Users\Sqwertz>command.com
'command.com' is not recognized as an internal or external
command, operable program or batch file.

C:\Users\Sqwertz>fuck you
'fuck' is not recognized as an internal or external command,
operable program or batch file.

So it's a good thing you're not using Windows, eh?

-sw

Sqwertz <sqwertzme@gmail.compost> wrote:
> On Mon, 08 May 2023 16:36:58 -0400, Benjamin Esham wrote:
[...]
> I'm not a LISP guy, but are you by chance scoring on article
> headers (or the article body) that are not available until the
> whole article is retrieved?
>
> News servers only dish out a subset of article headers when
> getting a list of articles for perusal and initial scoring(*).
> Only when the entire article is retrieved for full viewing can it
> be scored on "non-XOVER" headers (or the body text).
>
> (*) To see the list of headers your news server provides
> initially, telnet into your news server (usually at port 119, at
> least) and give it a "list overview.fmt" command.
>
> Doubtful you're on Windows 1[0-1], but if you are it would look
> something like this:
>
> Microsoft Windows [Version 10.0.19044.2965]
> (c) Microsoft Corporation. All rights reserved.
>
> C:\Users\Sqwertz>telnet usnews.blocknews.net 119
> 'telnet' is not recognized as an internal or external command,
> operable program or batch file.
>
> C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"
>
> C:\Users\Sqwertz>telnet usnews.blocknews.net 119
> 'telnet' is not recognized as an internal or external command,
> operable program or batch file.

On Windows - also on 10 and 11 - you can still enable Telnet:

Control Panel -> Programs -> Programs and Features -> Turn Windows
features on and off. Towards the bottom of the list there is 'Telnet
Client' with a tick-box.

> C:\Users\Sqwertz>cmd.exe
> 'cmd.exe' is not recognized as an internal or external command,
> operable program or batch file.

Hmmm!? Strange! Apparently C:\WINDOWS\system32 is not in your Path
variable.

Are you using a Windows Terminal window instead of a Command Prompt
window? (I only use the latter.)

> C:\Users\Sqwertz>command.com
> 'command.com' is not recognized as an internal or external
> command, operable program or batch file.

IIRC, command.com is from the pre-/non-NT era.

> C:\Users\Sqwertz>fuck you
> 'fuck' is not recognized as an internal or external command,
> operable program or batch file.
>
> So it's a good thing you're not using Windows, eh?

Well, I'm running tin and Hamster on Windows, so ... :-)

On 12 May 2023 14:32:25 GMT, Frank Slootweg wrote:

> Sqwertz <sqwertzme@gmail.compost> wrote:
>> On Mon, 08 May 2023 16:36:58 -0400, Benjamin Esham wrote:
> [...]
>> I'm not a LISP guy, but are you by chance scoring on article
>> headers (or the article body) that are not available until the
>> whole article is retrieved?
>>
>> News servers only dish out a subset of article headers when
>> getting a list of articles for perusal and initial scoring(*).
>> Only when the entire article is retrieved for full viewing can it
>> be scored on "non-XOVER" headers (or the body text).
>>
>> (*) To see the list of headers your news server provides
>> initially, telnet into your news server (usually at port 119, at
>> least) and give it a "list overview.fmt" command.
>>
>> Doubtful you're on Windows 1[0-1], but if you are it would look
>> something like this:
>>
>> Microsoft Windows [Version 10.0.19044.2965]
>> (c) Microsoft Corporation. All rights reserved.
>>
>> C:\Users\Sqwertz>telnet usnews.blocknews.net 119
>> 'telnet' is not recognized as an internal or external command,
>> operable program or batch file.
>>
>> C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"
>>
>> C:\Users\Sqwertz>telnet usnews.blocknews.net 119
>> 'telnet' is not recognized as an internal or external command,
>> operable program or batch file.
>
> On Windows - also on 10 and 11 - you can still enable Telnet:
>
> Control Panel -> Programs -> Programs and Features -> Turn Windows
> features on and off. Towards the bottom of the list there is 'Telnet
> Client' with a tick-box.

C:\Users\Sqwertz>pkgmgr /iu:"TelnetClient"

....does the same thing.

>> C:\Users\Sqwertz>cmd.exe
>> 'cmd.exe' is not recognized as an internal or external command,
>> operable program or batch file.
>
> Hmmm!? Strange! Apparently C:\WINDOWS\system32 is not in your Path
> variable.
>
> Are you using a Windows Terminal window instead of a Command Prompt
> window? (I only use the latter.)
>
>> C:\Users\Sqwertz>command.com
>> 'command.com' is not recognized as an internal or external
>> command, operable program or batch file.
>
> IIRC, command.com is from the pre-/non-NT era.

I needed to fork a new shell after the pkgadd for telnet to work.

Either one would have sufficed, but I was being lazy and was
demonstrating the irony of not including a standard OS
command/utility that only occupies a measly 132K of disk space in
Windows 10+ ... considering all the other bloated space-wasting
trash including in the OS.

(AND....I forgot the sequence to get into a newserver as a guest
for its overview.fmt and/or active list without posting my USER
and PASS creds <heh>)

-sw

Sqwertz wrote:

> (AND....I forgot the sequence to get into a newserver as a guest
> for its overview.fmt and/or active list without posting my USER
> and PASS creds <heh>)

i believe that's not inherent in op's case. point is, gnus by default
applies score rules once you enter groups, and there's a logic behind
this; a way to have the actual #'s of unread articles in group (or
topic) buffer is by using the function i pointed out.

issdr wrote:

> Benjamin Esham wrote:
>
>> I have a fairly large killfile. Frequently, Gnus fetches the news and
>> shows me that there are groups with unread messages, but when I try to
>> go into one of those groups, Gnus fetches more information and
>> realizes that all of the unread articles have been scored down below
>> the threshold of being visible.
>
> did you try `M-x g-ba-s' from group buffer?

I had not tried that, thank you! (The full name is gnus-batch-score, for my
future reference.) I tried to run that function automatically with

(add-hook 'gnus-started-hook 'gnus-batch-score)
(add-hook 'gnus-after-getting-new-news-hook 'gnus-batch-score)

but that didn't seem to work: there was a very long delay after I started
Gnus--maybe as long as a minute--but still not all of my scores had been
applied.

I went into the Gnus source and extracted the "main part" of that function,
which is

(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit)))))

and found that if I ran this interactively (at the M-: prompt) after news
had been fetched, then the scores would really, truly be applied. This is
promising! Now I just need to put the code back into a function and figure
out exactly when and how to call it in order to make all of this automatic.

> if that was not what you had in mind, you could also post on
> <nntp://news.eternal-september.org/gnu.emacs.gnus>.

I'm pretty sure I've posted in that group before, and yet Gnus is telling me
that it doesn't exist on the server. But that will be a separate thing to
debug...

Thanks,

Benjamin

Benjamin Esham wrote:

> issdr wrote:

[...]

> (add-hook 'gnus-started-hook 'gnus-batch-score)
> (add-hook 'gnus-after-getting-new-news-hook 'gnus-batch-score)
>
> but that didn't seem to work: there was a very long delay after I started
> Gnus--maybe as long as a minute--but still not all of my scores had been
> applied.

those vars are labeled as risky in local files. second one generated an
endless loop, as `gnus-batch-score' does get new news itself.

> I went into the Gnus source and extracted the "main part" of that function,
> which is
>
> (let (info group newsrc unread)
> (setq newsrc (cdr gnus-newsrc-alist))
> (while (setq info (pop newsrc))
> (setq group (gnus-info-group info)
> unread (gnus-group-unread group))
> (when (and (<= (gnus-info-level info) gnus-level-subscribed)
> (and unread
> (or (eq unread t)
> (not (zerop unread)))))
> (ignore-errors
> (gnus-summary-read-group group nil t nil t))
> (when (eq (current-buffer) (get-buffer gnus-summary-buffer))
> (gnus-summary-exit)))))
>
> and found that if I ran this interactively (at the M-: prompt) after news
> had been fetched, then the scores would really, truly be applied.

good. define a new function with it, then *add* it to the
after-getting-new-news hook (you were replacing it, and it's non empty
by default), like

(eval-after-load "gnus-start"
'(add-to-list 'gnus-after-getting-new-news-hook 'ben-gnus-score t))

[...]

>> <nntp://news.eternal-september.org/gnu.emacs.gnus>.
>
> I'm pretty sure I've posted in that group before, and yet Gnus is
> telling me that it doesn't exist on the server. But that will be a
> separate thing to debug...

i believe i have an account there, i'll check when i reach my pc

issdr wrote:

> Benjamin Esham wrote:
>
>> issdr wrote:
>
> [...]
>
>> (add-hook 'gnus-started-hook 'gnus-batch-score)
>> (add-hook 'gnus-after-getting-new-news-hook 'gnus-batch-score)
>>
>> but that didn't seem to work: there was a very long delay after I started
>> Gnus--maybe as long as a minute--but still not all of my scores had been
>> applied.
>
> those vars are labeled as risky in local files. second one generated an
> endless loop, as `gnus-batch-score' does get new news itself.

Can you tell me where/how they're labeled as risky? I'm pretty new to Emacs
Lisp.

>> I went into the Gnus source and extracted the "main part" of that
>> function, which is
>>
>> (let (info group newsrc unread)
>> (setq newsrc (cdr gnus-newsrc-alist))
>> (while (setq info (pop newsrc))
>> (setq group (gnus-info-group info)
>> unread (gnus-group-unread group))
>> (when (and (<= (gnus-info-level info) gnus-level-subscribed)
>> (and unread
>> (or (eq unread t)
>> (not (zerop unread)))))
>> (ignore-errors
>> (gnus-summary-read-group group nil t nil t))
>> (when (eq (current-buffer) (get-buffer gnus-summary-buffer))
>> (gnus-summary-exit)))))
>>
>> and found that if I ran this interactively (at the M-: prompt) after news
>> had been fetched, then the scores would really, truly be applied.
>
> good. define a new function with it, then *add* it to the
> after-getting-new-news hook (you were replacing it, and it's non empty
> by default), like
>
> (eval-after-load "gnus-start"
> '(add-to-list 'gnus-after-getting-new-news-hook 'ben-gnus-score t))
>
> [...]

Are you sure? The Elisp manual [1] seems to make it pretty clear that
add-hook will *append* the given function to the given hook, not replace any
previously-added functions.

>>> <nntp://news.eternal-september.org/gnu.emacs.gnus>.
>>
>> I'm pretty sure I've posted in that group before, and yet Gnus is
>> telling me that it doesn't exist on the server. But that will be a
>> separate thing to debug...
>
> i believe i have an account there, i'll check when i reach my pc

It looks like I misremembered the error I had gotten. Gnus told me "Group
gnu.emacs.gnus contains no messages", which is surprising (*no one* has
posted recently enough for Eternal September to retain the message?) but not
completely implausible.

Benjamin

[1] https://www.gnu.org/software/emacs/manual/html_node/elisp/Setting-Hooks.html#index-add_002dhook

Benjamin Esham wrote:

> issdr wrote:

[...]

>> those vars are labeled as risky in local files. second one generated an
>> endless loop, as `gnus-batch-score' does get new news itself.
>
> Can you tell me where/how they're labeled as risky? I'm pretty new to Emacs
> Lisp.

go `C-h v RET' while on one

[...]

>> (you were replacing it, and it's non empty by default), like
>>
>> (eval-after-load "gnus-start"
>> '(add-to-list 'gnus-after-getting-new-news-hook 'ben-gnus-score t))
>>
>> [...]
>
> Are you sure?

nope, my bad. that must have been me using a setq in the rush.

> The Elisp manual [1] seems to make it pretty clear that add-hook will
> *append* the given function to the given hook, not replace any
> previously-added functions.

thanks for the refresher, pretty useful function btw.

i tried the piece of code you isolated, it works both at startup and
when checking news again, it just gives warnings of no group found once
it's thru.

[...]

> It looks like I misremembered the error I had gotten. Gnus told me "Group
> gnu.emacs.gnus contains no messages", which is surprising (*no one* has
> posted recently enough for Eternal September to retain the message?) but not
> completely implausible.

no high volume indeed, but on my server there's a relatively fresh
post. i asked on es's support group what's going on

issdr wrote:

> i tried the piece of code you isolated, it works both at startup and
> when checking news again, it just gives warnings of no group found once
> it's thru.

Yes, the warnings are a bit annoying. Maybe it's possible to suppress them
by setting gnus-verbose locally? Otherwise, though, this works well:

; Apply scores as soon as news is fetched
(defun apply-scores-to-subscribed-groups ()
(let (info group newsrc unread)
(setq newsrc (cdr gnus-newsrc-alist))
(while (setq info (pop newsrc))
(setq group (gnus-info-group info)
unread (gnus-group-unread group))
(when (and (<= (gnus-info-level info) gnus-level-subscribed)
(and unread
(or (eq unread t)
(not (zerop unread)))))
(ignore-errors
(gnus-summary-read-group group nil t nil t))
(when (eq (current-buffer) (get-buffer gnus-summary-buffer))
(gnus-summary-exit))))))

(add-hook 'gnus-started-hook
'apply-scores-to-subscribed-groups)
(add-hook 'gnus-after-getting-new-news-hook
'apply-scores-to-subscribed-groups)

Thank you for the assistance!

Benjamin

Benjamin Esham wrote:

> issdr wrote:
>
>> i tried the piece of code you isolated, it works both at startup and
>> when checking news again, it just gives warnings of no group found once
>> it's thru.
>
> Yes, the warnings are a bit annoying. Maybe it's possible to suppress them
> by setting gnus-verbose locally?

i changed this from 3 to 4 and re-byte-compiled, but could have broken
something else, i don't know (no bad signs so far):

~ $ rgrep -n "No articles in the group" emacs/
emacs/lisp/gnus/gnus-sum.el:7463: (gnus-message 4 "No articles in the group")
grep: emacs/lisp/gnus/gnus-sum.elc: binary file matches
~ $

> Otherwise, though, this works well:
>
> ; Apply scores as soon as news is fetched
> (defun apply-scores-to-subscribed-groups ()
> (let (info group newsrc unread)
> (setq newsrc (cdr gnus-newsrc-alist))
> (while (setq info (pop newsrc))
> (setq group (gnus-info-group info)
> unread (gnus-group-unread group))
> (when (and (<= (gnus-info-level info) gnus-level-subscribed)
> (and unread
> (or (eq unread t)
> (not (zerop unread)))))
> (ignore-errors
> (gnus-summary-read-group group nil t nil t))
> (when (eq (current-buffer) (get-buffer gnus-summary-buffer))
> (gnus-summary-exit))))))
>
> (add-hook 'gnus-started-hook
> 'apply-scores-to-subscribed-groups)
> (add-hook 'gnus-after-getting-new-news-hook
> 'apply-scores-to-subscribed-groups)
>
> Thank you for the assistance!

great! no prob.

from e-s.support:

,----
| The groups still exist on E-S, but they are empty, presumably because
| nobody bothered to send rmgroups when the Stanford news server hosting
| the mail2news gateway for the GNU mailing lists shut down in September
| 2020.
|
| Time to remove the gnu.* groups, I reckon.
`----

even though i see an active gnu.emacs.help, where gnus topics are
discussed.

issdr wrote:

> Benjamin Esham wrote:
>
>> issdr wrote:
>>
>>> i tried the piece of code you isolated, it works both at startup and
>>> when checking news again, it just gives warnings of no group found once
>>> it's thru.
>>
>> Yes, the warnings are a bit annoying. Maybe it's possible to suppress
>> them by setting gnus-verbose locally?
>
> i changed this from 3 to 4 and re-byte-compiled, but could have broken
> something else, i don't know (no bad signs so far):
>
> ~ $ rgrep -n "No articles in the group" emacs/
> emacs/lisp/gnus/gnus-sum.el:7463: (gnus-message 4 "No articles in the group")
> grep: emacs/lisp/gnus/gnus-sum.elc: binary file matches
> ~ $

It certainly seems like a safe change to me.

> [snip]
>
> from e-s.support:
>
> ,----
> | The groups still exist on E-S, but they are empty, presumably because
> | nobody bothered to send rmgroups when the Stanford news server hosting
> | the mail2news gateway for the GNU mailing lists shut down in September
> | 2020.
> |
> | Time to remove the gnu.* groups, I reckon.
> `----
>
> even though i see an active gnu.emacs.help, where gnus topics are
> discussed.

Ah, that's a shame. One more piece of news infrastructure that has crumbled
away...

Benjamin

On Sat, 13 May 2023 13:32:13 +0200, issdr wrote:

> Sqwertz wrote:
>
>> (AND....I forgot the sequence to get into a newserver as a guest
>> for its overview.fmt and/or active list without posting my USER
>> and PASS creds <heh>)
>
> i believe that's not inherent in op's case. point is, gnus by default
> applies score rules once you enter groups, and there's a logic behind
> this; a way to have the actual #'s of unread articles in group (or
> topic) buffer is by using the function i pointed out.

I think you quoted the wrong paragraph of mine and from the wrong
post. But it's whatever <shrug>

-sw

Sqwertz wrote:

> On Sat, 13 May 2023 13:32:13 +0200, issdr wrote:
>
>> Sqwertz wrote:
>>
>>> (AND....I forgot the sequence to get into a newserver as a guest
>>> for its overview.fmt and/or active list without posting my USER
>>> and PASS creds <heh>)
>>
>> i believe that's not inherent in op's case. point is, gnus by default
>> applies score rules once you enter groups, and there's a logic behind
>> this; a way to have the actual #'s of unread articles in group (or
>> topic) buffer is by using the function i pointed out.
>
> I think you quoted the wrong paragraph of mine and from the wrong
> post. But it's whatever <shrug>

were you trying to access a news server to check overview headers? that
wasn't relevant to op's problem.

Benjamin Esham wrote:

>> ~ $ rgrep -n "No articles in the group" emacs/
>> emacs/lisp/gnus/gnus-sum.el:7463: (gnus-message 4 "No articles in the group")
>> grep: emacs/lisp/gnus/gnus-sum.elc: binary file matches
>> ~ $
>
> It certainly seems like a safe change to me.

it does break the `e' (or `D e') command in the drafts group, dunno why.