On Tue, 06 Jul 2021 18:03:17 +0100, wibble@bibble.com.invalid wrote:

> I have noticed that my [6003,6003] and [0,0] filters do not get
> automatically run. I am having to used Tools, Apply Filters to
> Folder.
>
> I am sure I must be missing a setting somewhere.

Are those multipart messages? And when you wrote "a lot have exactly
6003 lines", was that 6003 lines *before* joining or 6003 lines *after*
joining ?

Filters which automatically run when you get new headers, run *before*
joining.

--
Kind regards
Ralph

On Wed, 07 Jul 2021 06:32:05 +1200, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Tue, 06 Jul 2021 18:03:17 +0100, wibble@bibble.com.invalid wrote:
>
>> I have noticed that my [6003,6003] and [0,0] filters do not get
>> automatically run. I am having to used Tools, Apply Filters to
>> Folder.
>>
>> I am sure I must be missing a setting somewhere.
>
>
>Are those multipart messages? And when you wrote "a lot have exactly
>6003 lines", was that 6003 lines *before* joining or 6003 lines *after*
>joining ?
>
>Filters which automatically run when you get new headers, run *before*
>joining.

Why can't he just make one such as [900,6500]?

That should get any part of a mulitpart message. Right?

On Tue, 06 Jul 2021 15:11:58 -0500, Merle@invalid.com wrote:

> On Wed, 07 Jul 2021 06:32:05 +1200, Ralph Fox <-rf-nz-@-.invalid>
> wrote:
>
>> On Tue, 06 Jul 2021 18:03:17 +0100, wibble@bibble.com.invalid wrote:
>>
>>> I have noticed that my [6003,6003] and [0,0] filters do not get
>>> automatically run. I am having to used Tools, Apply Filters to
>>> Folder.
>>>
>>> I am sure I must be missing a setting somewhere.
>>
>>
>> Are those multipart messages? And when you wrote "a lot have exactly
>> 6003 lines", was that 6003 lines *before* joining or 6003 lines *after*
>> joining ?
>>
>> Filters which automatically run when you get new headers, run *before*
>> joining.
>
> Why can't he just make one such as [900,6500]?
>
> That should get any part of a mulitpart message. Right?

That would depend on which of these wibble wants to do:

(a) kill only the gibberish posts and leave the ones which are not gibberish;

(b) kill _all_ multipart messages, whether or not they are gibberish.

--
Kind regards
Ralph

On Wed, 07 Jul 2021 20:54:25 +1200, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Tue, 06 Jul 2021 15:11:58 -0500, Merle@invalid.com wrote:
>
>> On Wed, 07 Jul 2021 06:32:05 +1200, Ralph Fox <-rf-nz-@-.invalid>
>> wrote:
>>
>>> On Tue, 06 Jul 2021 18:03:17 +0100, wibble@bibble.com.invalid wrote:
>>>
>>>> I have noticed that my [6003,6003] and [0,0] filters do not get
>>>> automatically run. I am having to used Tools, Apply Filters to
>>>> Folder.
>>>>
>>>> I am sure I must be missing a setting somewhere.
>>>
>>>
>>> Are those multipart messages? And when you wrote "a lot have exactly
>>> 6003 lines", was that 6003 lines *before* joining or 6003 lines *after*
>>> joining ?
>>>
>>> Filters which automatically run when you get new headers, run *before*
>>> joining.
>>
>> Why can't he just make one such as [900,6500]?
>>
>> That should get any part of a mulitpart message. Right?
>
>
>That would depend on which of these wibble wants to do:
>
> (a) kill only the gibberish posts and leave the ones which are not gibberish;
>
> (b) kill _all_ multipart messages, whether or not they are gibberish.

I was assuming he was tryng to downlaod normal headers from a "
"conversational" newsgroup. If he's downloading from binary groups,
that's a different story. I thought some jerk was just flooding a
"normal" group with those huge posts.

If he's in a binary group, lots of luck guessing the size of future
unwanted posts.

On Wed, 07 Jul 2021 06:32:05 +1200, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Tue, 06 Jul 2021 18:03:17 +0100, wibble@bibble.com.invalid wrote:
>
>> I have noticed that my [6003,6003] and [0,0] filters do not get
>> automatically run. I am having to used Tools, Apply Filters to
>> Folder.
>>
>> I am sure I must be missing a setting somewhere.
>
>
>Are those multipart messages? And when you wrote "a lot have exactly
>6003 lines", was that 6003 lines *before* joining or 6003 lines *after*
>joining ?
>
>Filters which automatically run when you get new headers, run *before*
>joining.

Thanks Ralf.

Most parts are exactly 6003 lines of a multi-part post.

On Sun, 04 Jul 2021 10:15:47 +0200, none@given.com wrote:

>On Sat, 03 Jul 2021 17:12:32 -0500, merle@invalid.com wrote:
>
>>But Wibble got lucky in that the posts he's contending with now have a
>>repeatable line number of 6003 for a line length. The [0,0] type
>>filter worked perfectly when that number was used in it.
>
>I'm not understanding this 6003 filter. To clarify, is it the case
>that the filter
>
>[6003,6003]
>
> filters out ONLY the gibberish posts of that exact length, but leaves
>alone everything both smaller and larger than that?
>
>ie, leaving untouched everything else but that crap?
>
>I tried the 6003 on a couple of the affected groups which were showing
>hundreds of apparently 0-line posts, and it seemed to do nothing at
>all.
>
>However, using the [0,0] filter seems to do the trick.

I have tried [0,0] and it helps a lot but does not filter every 0
length posting

On Wed, 07 Jul 2021 06:32:05 +1200, Ralph Fox <-rf-nz-@-.invalid>
wrote:

>On Tue, 06 Jul 2021 18:03:17 +0100, wibble@bibble.com.invalid wrote:
>
>> I have noticed that my [6003,6003] and [0,0] filters do not get
>> automatically run. I am having to used Tools, Apply Filters to
>> Folder.
>>
>> I am sure I must be missing a setting somewhere.
>
>
>Are those multipart messages? And when you wrote "a lot have exactly
>6003 lines", was that 6003 lines *before* joining or 6003 lines *after*
>joining ?
>
>Filters which automatically run when you get new headers, run *before*
>joining.

In my case the 0 length that weren't filtered were incomplete messages

On Wed, 07 Jul 2021 07:56:23 -0700, The Horny Goat <lcraver@home.ca>
wrote:

>I have tried [0,0] and it helps a lot but does not filter every 0
>length posting

Yes, odd, that. A few of them still come through.

But what is strange is that when you "apply filters" to one of these
newsgroups where some still got through, now the exact same filter
gets them.

Possibly this filter applied globally may be simply too much for Agent
to cope with when there's more than one or two groups involved?

wibble@bibble.com.invalid wrote:

>The OP on this thread mention the zero length posts.
>
>The posts in question are in the book groups (possibly others) and
>have 'random' characters in the Subject and Author fields.
>
>Example:
>
>Subject: nN2oOu3daxxkDhx [$/10] "???" yEnc (*/#)
>From: PzC8PJLdsWAEmu9fcjGq@-CYJNP7EZw69.0T7
>Date: Sat, 03 Jul 2021 10:43:45 +0100
>Lines: 6003
>
>
>
>For a long time I have successfully used the filter:
>{[a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9][a-z0-9]}
>
>on Subject and/or Author with success, but as the OP stated, the
>latest wave is not blocked by that filter.

Perhaps, a different approach is needed. Instead of blacklisting
(filtering) the articles you don't want, use a whitelist identifying the
articles you do want. In other words mark read everything except the
articles you want.

The ones you want might have a Subject containing "req" or "epub" or
"treesaver" or "bloodnut"
!subject: (req|epub|treesaver|bloodnut)
What this filter does is mark read everything not containg the words
req, epub, treesaver or bloodnut.

Add other wanted articles as needed, e.g. "ebooks pack".

Not perfect and you might miss some interesting articles, so it will
take a bit of monitoring and modifying to not mark read the articles you
want to see, but those long strings of random characters containing no
words are marked read by default.
You can't fix stupid.

On Thu, 08 Jul 2021 07:42:34 +0200, none@given.com wrote:

>On Wed, 07 Jul 2021 07:56:23 -0700, The Horny Goat <lcraver@home.ca>
>wrote:
>
>>I have tried [0,0] and it helps a lot but does not filter every 0
>>length posting
>
>Yes, odd, that. A few of them still come through.
>
>But what is strange is that when you "apply filters" to one of these
>newsgroups where some still got through, now the exact same filter
>gets them.
>
>Possibly this filter applied globally may be simply too much for Agent
>to cope with when there's more than one or two groups involved?

Could be though I have pretty much a maximal setup for Agent in terms
of both memory and disk.

On Sun, 04 Jul 2021 09:57:48 +1200, Ralph Fox <-rf-nz-@-.invalid> wrote:

>On Sat, 03 Jul 2021 04:27:51 -0500, merle@invalid.com wrote:
>
>> I screwed up. I didn't notice that you said some posts had zero line
>> content. Use this - it works in Agent 1.9x
>>
>> [0.0]
>>
>> Don NOT use Subject: in the filter. Use only what i show here.
>>
>> I downloaded some years back of files for this group and found those
>> zero line content files. The filter above worked perfectly and did
>> not filter out any other posts.
>
>
>Just be aware that some legitimate posts appear to have zero lines
>when the header is downloaded. The actual number of lines appears
>only when the body is downloaded.
>
>The [0,0] filter will also hit those legitimate posts.

A lot of those posts are legitimate files using a new obsfucation method to
hide copyrighted posts from the agents who take them down. Nearly all
posts involving tv shows and movies and similar content is being posted
using this method. Not only does each and every rar of the post have a
completely different random assortment of letters and numbers in the
subject, but it aso falsely shows a randomized number of lines.

Unfortunately, together both of those randomizations messes up Agent, which
can't figure out what is what and shows errors and ends up saving multiple
small, incomplete files to your default saved files folder, and the various
rar files to individual sub-folders. To access the shows and movies, you
need to move every one of the rar files from their individual sub-folders
to the default folder, then unrar it.

It's a huge pain in the butt. Some other app like Newzbin, Nzbget and
Sabnzbd supposedly can handle these posts.

Has anyone had any luck getting these files with Agent?

Is Forte, Inc even aware of this? I sent a feature request, but only got
an automated reply.

On Tue, 10 Aug 2021 03:14:15 -0400,
davidhhh@NOSPAMMERSbellatlantic.net wrote:

(I apologize for the extensive quoting but the full context is really
necessary. My responses inline.)

> A lot of those posts are legitimate files using a new obsfucation method to
> hide copyrighted posts from the agents who take them down. Nearly all
> posts involving tv shows and movies and similar content is being posted
> using this method. Not only does each and every rar of the post have a
> completely different random assortment of letters and numbers in the
> subject, but it aso falsely shows a randomized number of lines.

First off, thank you for that explanation. I was wondering how the new
obfuscation method worked.

> Unfortunately, together both of those randomizations messes up Agent, which
> can't figure out what is what and shows errors and ends up saving multiple
> small, incomplete files to your default saved files folder, and the various
> rar files to individual sub-folders. To access the shows and movies, you
> need to move every one of the rar files from their individual sub-folders
> to the default folder, then unrar it.

> It's a huge pain in the butt. Some other app like Newzbin, Nzbget and
> Sabnzbd supposedly can handle these posts.

I can vouch that SABnzbd does it brilliantly. The only thing that it
doesn't do is rename the NewObfsd file: Instead, it puts it into a
folder with the file's name so that you end up with something like
this:

Housebroken.S01E08.720p.HEVC.x265-MeGusta\ [folder name]
010a8b69fc8a4218a29a5e1716f6efdf.mkv [name of only file contained in
that folder]

> Has anyone had any luck getting these files with Agent?

I did for about a week. It was OK when downloading a single episode of
a single show (eg one episode of Full Frontal) or single episodes of
several shows (eg one episode each of Last Week Tonight, 60 Minutes,
and Dateline). It gets really messy however when downloading several
episodes of several shows (eg three episodes each of Antiques Road
Trip, Rick and Morty, and South Park) because of the difficulty of
identifying which episode is which. Like you said: It's A Huge Pain In
The Butt. I have since moved on to SABnzbd.

However before moving on I discovered an interesting thing that I
asked about in this NG on 16 July:

<2021.07.16>
I've noticed that when Agent downloads a complete set of these files,
you get something that looks like the following. (The numbers at the
end of each line are file sizes.)

08ffc6dd4af04287b981f4d0ee106177(1).mkv 0
08ffc6dd4af04287b981f4d0ee106177(10).mkv 0
....
08ffc6dd4af04287b981f4d0ee106177(15).mkv 15,204,352
08ffc6dd4af04287b981f4d0ee106177(16).mkv 0
....
08ffc6dd4af04287b981f4d0ee106177.mkv 0
08ffc6dd4af04287b981f4d0ee106177.par2 7,720
08ffc6dd4af04287b981f4d0ee106177.part01.rar 15,360,000
....
08ffc6dd4af04287b981f4d0ee106177.part11.rar 15,360,000
08ffc6dd4af04287b981f4d0ee106177.part12.rar 318,980

Notice that all the .mkv files except one are 0 bytes long. That is a
preview that's been packed into the set. If you play it you'll get
about two minutes of the start of the video. Every new-obfuscated set
that I've downloaded with Agent has one of these but it does not
always occur in the same position in the set.
</2021.07.16>

If we assume that this preview is being included in the set for a
reason, then there may be a way of extracting it before downloading
the complete set. I've asked about this in the SABnzbd forum but had
no response.

> Is Forte, Inc even aware of this? I sent a feature request, but only got
> an automated reply.

I thnk Forte has gone into dormancy.

--
Bob
St Francis would have done better to preach to the cats

Opinicus <gezgin@spamcop.net.which.is.not.quite.invalid> on Wed, 11
Aug 2021 08:39:36 +0300 typed in alt.usenet.offline-reader.forte-agent
the following:
>
>> Is Forte, Inc even aware of this? I sent a feature request, but only got
>> an automated reply.
>
>I thnk Forte has gone into dormancy.

Forte has been ignoring Agent since at least '17, when the epoch
for the "temp" filters ended, and temporary filters no longer worked.
--
pyotr filipivich
This Week's Panel: Us & Them - Eliminating Them.
Next Month's Panel: Having eliminated the old Them(tm)
Selecting who insufficiently Woke(tm) as to serve as the new Them(tm)

On Tue, 10 Aug 2021 03:14:15 -0400, davidhhh@NOSPAMMERSbellatlantic.net
wrote:

> A lot of those posts are legitimate files using a new obsfucation method to
> hide copyrighted posts from the agents who take them down.

How come, "new"?

--
s|b