Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

linux: because a PC is a terrible thing to waste (ksh@cis.ufl.edu put this on Tshirts in '93)


computers / rocksolid.nodes / Re: hey, what do know ? vichan supports nntp

SubjectAuthor
* hey, what do know ? vichan supports nntpanon
+* hey, what do know ? vichan supports nntpanon
|`* hey, what do know ? vichan supports nntpRetro Guy
| `* hey, what do know ? vichan supports nntpanon
|  `* hey, what do know ? vichan supports nntpRetro Guy
|   `* hey, what do know ? vichan supports nntpanon
|    `* hey, what do know ? vichan supports nntpanon
|     `* hey, what do know ? vichan supports nntpRetro Guy
|      `* hey, what do know ? vichan supports nntpanon
|       `* hey, what do know ? vichan supports nntpanon
|        +* hey, what do know ? vichan supports nntpanon
|        |`* hey, what do know ? vichan supports nntpanonymous
|        | +* hey, what do know ? vichan supports nntpRetro Guy
|        | |`* hey, what do know ? vichan supports nntpanon
|        | | `* hey, what do know ? vichan supports nntpRetro Guy
|        | |  `* hey, what do know ? vichan supports nntpanonymous
|        | |   `* hey, what do know ? vichan supports nntpRetro Guy
|        | |    `* hey, what do know ? vichan supports nntpanonymous
|        | |     `* hey, what do know ? vichan supports nntpanon
|        | |      +* hey, what do you know: vichan supports nntpanon
|        | |      |+- hey, what do you know: vichan supports nntpRetro Guy
|        | |      |`* hey, what do you know: vichan supports nntpanon
|        | |      | `- hey, what do you know: vichan supports nntpRetro Guy
|        | |      `- hey, what do know ? vichan supports nntpRetro Guy
|        | `* hey, what do know ? vichan supports nntpRetro Guy
|        |  `* hey, what do know ? vichan supports nntpanonymous
|        |   `* hey, what do know ? vichan supports nntpanonymous
|        |    `* hey, what do know ? vichan supports nntpanonymous
|        |     +* hey, what do know ? vichan supports nntpanonymous
|        |     |`* hey, what do know ? vichan supports nntpanonymous
|        |     | `* hey, what do know ? vichan supports nntpanonymous
|        |     |  `* hey, what do know ? vichan supports nntpanonymous
|        |     |   `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |    `* hey, what do know ? vichan supports nntpanonymous
|        |     |     `* hey, what do know ? vichan supports nntpanonymous
|        |     |      `* hey, what do know ? vichan supports nntpanonymous
|        |     |       `* hey, what do know ? vichan supports nntpanonymous
|        |     |        `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |         `* hey, what do know ? vichan supports nntpanonymous
|        |     |          `* hey, what do know ? vichan supports nntpanonymous
|        |     |           `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |            `* hey, what do know ? vichan supports nntpanonymous
|        |     |             `* hey, what do know ? vichan supports nntpanonymous
|        |     |              `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |               `* hey, what do know ? vichan supports nntpanon
|        |     |                `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |                 `* hey, what do know ? vichan supports nntpanonymous
|        |     |                  `* hey, what do know ? vichan supports nntpanonymous
|        |     |                   `* hey, what do know ? vichan supports nntpanonymous
|        |     |                    `* hey, what do know ? vichan supports nntpanonymous
|        |     |                     `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |                      `* hey, what do know ? vichan supports nntpanonymous
|        |     |                       +* hey, what do know ? vichan supports nntpanonymous
|        |     |                       |+- hey, what do know ? vichan supports nntpanonymous
|        |     |                       |`- hey, what do know ? vichan supports nntpRetro Guy
|        |     |                       `* hey, what do know ? vichan supports nntpRetro Guy
|        |     |                        `- hey, what do know ? vichan supports nntpanonymous
|        |     `- hey, what do know ? vichan supports nntpRetro Guy
|        `- hey, what do know ? vichan supports nntpRetro Guy
+- hey, what do know ? vichan supports nntpanon
+* hey, what do know ? vichan supports nntpanon
|`* hey, what do know ? vichan supports nntpanonymous
| +- hey, what do you know....anon
| `* hey, what do know ? vichan supports nntpanonymous
|  `* hey, what do know ? vichan supports nntpRetro Guy
|   `- hey, what do know ? vichan supports nntpanon
+* basics are workingAnonymous
|`- basics are workingRetro Guy
+* guess i will open for business soonAnonymous
|+* guess i will open for business soonMei
||`- guess i will open for business soonRetro Guy
|`- guess i will open for business soonRetro Guy
+- opsec check is goodAnonymous
+- > goo to go.Anonymous
+* yes that should workAnonymous
|`- yes that should workRetro Guy
+* cron or not ?Anonymous
|`- cron or not ?Retro Guy
+- themes on def2.i2pAnonymous
`* ok, ok, i switched the default themeAnonymous
 `* ok, ok, i switched the default themeMei
  `- ok, ok, i switched the default themeanon

Pages:1234
Re: hey, what do know ? vichan supports nntp

<dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=281&group=rocksolid.nodes#281

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Fri, 22 May 2020 19:24 UTC
Organization: def2
Message-ID: <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="15333"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$rGh9eMhRH2jdD62G0wRSWuk5GAW8HG7ZiEGsuwR8sKt3m/9MiPfdm
 by: anonymous - Fri, 22 May 2020 19:24 UTC

ok, that was easy, i took message_read and put all articles into something like this:

{
"header": {
"number": 100,
"id": "<96042908114f73b92a4b25ecaf31371d@def4>",
"from": "anon@anon.com",
"name": "anon",
"subject": "Re: sync trest def2",
"newsgroups": "rocksolid.shared.test",
"followup": null,
"date": 1589462807,
"date_thread": null,
"organization": "def5",
"xnoarchive": null,
"references": [
"<46f0e0c1993dc510ec8653ac72cb4445$1@z5bqfv5v75kxy7pj.onion>"
],
"bestreference": null,
"content_transfer_encoding": "8bit",
"mime_version": null,
"content_type": [
"text/plain"
],
"content_type_charset": [
"utf-8"
],
"content_type_name": [
"unnamed"
],
"content_type_boundary": null,
"content_type_format": [
"flowed"
],
"answers": null,
"isAnswer": false,
"username": "anon",
"user_agent": null,
"isReply": null,
"threadsize": null
},
"body": [
"nok, seen on def4nnPosted on def4n"
]
}

that is the json encoded array that you get when you typecaste the object that message_read returns.
now i only read this into my import routine from vichan.

right now i work on a batch import function, and try to wrap my head around some of the details of vichan (which is harder for me than it probably sounds). :-)

cheers

trw
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=282&group=rocksolid.nodes#282

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Fri, 22 May 2020 20:53 UTC
Organization: def2
Message-ID: <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="26469"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$bqzCKQpbCQDNi7RaKw6J8uZG/4BkAlSvB2KCnvYqCdyLGO8TASZm6
 by: anonymous - Fri, 22 May 2020 20:53 UTC

ok, i got a kind of lame import routine working (lame because it has to be called per message). but at least it works. got all the boards created too.

cheers

trw
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=283&group=rocksolid.nodes#283

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Fri, 22 May 2020 21:50 UTC
Organization: def2
Message-ID: <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="1041"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$mfpxtJLZgGL0Vklj5S8pB.P7Mmrg/KfVpXE2n0FWhAEeplPsWLc1W
 by: anonymous - Fri, 22 May 2020 21:50 UTC

one thing vichan is clearly not prepared to do is to deal with replies to (yet) non existing messages. so no automagic fixing of threads or anything, like ff does.
so the options are: either delete the reference and import such message as a new thread start, or refuse to take it.
the second one looks cleaner to me, and gives the chance to later try it again, so I guess I will opt for this one.
In any event, vichan is sensitive here (should not be a problem once the thing is steadily updated).
Another thing I am contemplating is how many messages to keep. By design I think it keeps 100 messages per board (or something like that), and after starts to delete the oldest ones.
Would be different from the rest of the nodes, which keep the messages forever. Of course, that could be done with vichan as well. But I kind of like the idea of a volatile message base on one of the nodes.

What do you think, Retro ?

cheers

trw
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<208a215d095e2a90acdc545066307dbe$1@news.novabbs.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=284&group=rocksolid.nodes#284

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro ...@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sat, 23 May 2020 08:17 UTC
Organization: Rocksolid Light
Message-ID: <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="1354"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Spam-Level: *
X-Rslight-Site: $2y$10$NYZNLkptm7.nGmndLEPW6eVpVDNKyvRQfr6RBxiJkfne2J9K5yMaO
 by: Retro Guy - Sat, 23 May 2020 08:17 UTC

anonymous wrote:

> one thing vichan is clearly not prepared to do is to deal with replies to (yet) non existing messages. so no automagic fixing of threads or anything, like ff does.
> so the options are: either delete the reference and import such message as a new thread start, or refuse to take it.
> the second one looks cleaner to me, and gives the chance to later try it again, so I guess I will opt for this one.

Do you mean put the message into some holding location and try again in the future to apply it to vichan's database of articles? One thing you can do is grep overview for the message-id of the referenced article. If the server has it, it will be in overview and you can find it. Every group containing articles has a groupname-overview (rocksolid.nodes-overview). Reading this file, you can crossreference message-id to article number, etc. It is tab separated:
160 Re: ddos defense for novabbs.com anonymous@anon.com (anonymous) Fri, 22 May 2020 10:50 UTC <e4cfbed07f97837a031302573d5ebefc$1@z5bqfv5v75kxy7pj.onion> <733615c1436ceffa04cd2d7eb3ec2546$1@news.novabbs.com> <f0ba1870346ce5f36eb9365850311bf7$1@z5bqfv5v75kxy7pj.onion> <715287e8bd4cc83e628a4e69ce6bd304$1@news.novabbs.com> 1709 12 Xref: debiantest rocksolid.nodes:160

> In any event, vichan is sensitive here (should not be a problem once the thing is steadily updated).
> Another thing I am contemplating is how many messages to keep. By design I think it keeps 100 messages per board (or something like that), and after starts to delete the oldest ones.
> Would be different from the rest of the nodes, which keep the messages forever. Of course, that could be done with vichan as well. But I kind of like the idea of a volatile message base on one of the nodes.

> What do you think, Retro ?

Expiring my count of messages is nicer than by date, so a group never becomes empty. This is also what Synchronet does and it works well. I like it.

Retro Guy

--
Posted on: Rocksolid Light
news.novabbs.com

Re: hey, what do know ? vichan supports nntp

<0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=285&group=rocksolid.nodes#285

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sat, 23 May 2020 15:34 UTC
Organization: def2
Message-ID: <0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion> <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="22718"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$aSq.3cTlJqMmP3dtgqDVNOh4IO3I1ro6uNUM8upYzpkWT5g7ECH4C
 by: anonymous - Sat, 23 May 2020 15:34 UTC

>Do you mean put the message into some holding location and try again in the future to apply it to vichan's database of articles?

Yes, this is more or less what I meant.

>One thing you can do is grep overview for the message-id of the referenced article.

I used message_read for now, and made one new file to follow the status of the exports. Actually, this kind of thing would be better done in one of the existing code bases, this was just the fastest way for me to test the concept.

The code lines below export any new message that is found into a format that is translated later in the new post.php of vichan.

So I guess I could call this script at the end of spoolnews (or maybe just insert it).

On the other end, I have a modified post.php, which reads the oldest message (by name) and imports it. I could not modify it in a way yet that would allow to cycle through all existing messages, so the whole setup is just an interim solution yet (but it is importing the messages right now).

later more,

trw

<?php

include "config.inc.php";
include "$file_newsportal";
include 'message.inc.php';

$groups = array(
'rocksolid.nodes',
'rocksolid.nodes.help',
'rocksolid.nodes.announce',
'rocksolid.shared.encryption',
'rocksolid.shared.entertainment',
'rocksolid.shared.freenet',
'rocksolid.shared.general',
'rocksolid.shared.hacking',
'rocksolid.shared.helpdesk',
'rocksolid.shared.i2p',
'rocksolid.shared.linux',
'rocksolid.shared.news',
'rocksolid.shared.offtopic',
'rocksolid.programming',
'rocksolid.shared.rocksolid',
'rocksolid.shared.security',
'rocksolid.social',
'rocksolid.shared.test',
'rocksolid.shared.tor',
);
$watermarks = json_decode(file_get_contents('watermarks.json'), true);
$update = false;
foreach($groups as $group){
echo($group . "nr");
for ($i = 0; $i < 1000; $i++){
if(!in_array($i,$watermarks[$group])){
$message = message_read($i, 0, $group);
if($message){
echo("processing message # " . $i . " from group " . $group . "nr");
$paddedI = str_pad($i, 5, '0', STR_PAD_LEFT);
$jsonFile = fopen('/var/www/html/vi/vichan-master/debug/' . $paddedI . '_' . $group . '.json', 'w');
fwrite($jsonFile, json_encode((array) $message, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonFile);
array_push($watermarks[$group],$i);
$update = true;
}
}
}
} if($update){
echo("saving updated watermarksnr");
$jsonWatermarks = fopen('watermarks.json', 'w');
fwrite($jsonWatermarks, json_encode((array) $watermarks, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonWatermarks);
} else {
echo("no new files foundnr");
} exit;
?>

--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<a4e01850071af95292d537f931769051$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=286&group=rocksolid.nodes#286

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sat, 23 May 2020 21:33 UTC
Organization: def2
Message-ID: <a4e01850071af95292d537f931769051$1@z5bqfv5v75kxy7pj.onion>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion> <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com> <0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="4232"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$PQ8ptntp7D/bDZJ.O4dj7ejAizDmkFBcnyuI693YMSscHibDIQuT2
 by: anonymous - Sat, 23 May 2020 21:33 UTC

so, i have got it basically running, so messages gets synced to and from vichan. it is still sort of clumsy right now...
the fields "date" and "from" are not acknowledged for some reason, so posts get the timestamp of the import, and are "forced anonymous". guess i can live with both, but will try to get it done right.
the work is a bit tedious, but will be definitely be worth it in the end. vichan is super easy to configure, modify, moderate, and it is just looking very nice (still playing with all the css that come with it, but will most likely stay on the dark one).

the main challenge to modify the code from vichan and run it differently seems to be that the context changes depending on how the scripts are run, and in most cases this lets any file inclusions fail (calling from the cli works as well as calling from the webserver, but running it from another script fails, even if the directory is changed).

cheers

trw
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<00cc63d3bc43ff8fc63add025b35edd5$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=287&group=rocksolid.nodes#287

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sat, 23 May 2020 22:12 UTC
Organization: def2
Message-ID: <00cc63d3bc43ff8fc63add025b35edd5$1@z5bqfv5v75kxy7pj.onion>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion> <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com> <0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion> <a4e01850071af95292d537f931769051$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="8741"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$MzeOUOs18Wdg3KJC51LB1urXl47NH09i.q07LxZW974UnCCD9uEN2
 by: anonymous - Sat, 23 May 2020 22:12 UTC

>> Do you mean put the message into some holding location and try again in >>the future to apply it to vichan's database of articles?
>Yes, this is more or less what I meant.

Tried that one, and for sure it did not work. Either a message which was rejected once is not considered anymore, or there is some serious issue with the references.
While the first would not be an issue for new threads, the second one would require some coding to rewrite the references (shudder...).
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<93c5cc225e028d5fc5b73a9f51939c5b$1@www.novabbs.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=288&group=rocksolid.nodes#288

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro ...@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sun, 24 May 2020 00:51 UTC
Organization: Rocksolid Light
Message-ID: <93c5cc225e028d5fc5b73a9f51939c5b$1@www.novabbs.com>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion> <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com> <0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion> <a4e01850071af95292d537f931769051$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="24652"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$nc5HOiC4toNdUdhR5x6WAetXyTZZNpID4kSBsNrLI8mkS5Su23sd.
 by: Retro Guy - Sun, 24 May 2020 00:51 UTC

anonymous wrote:

> so, i have got it basically running, so messages gets synced to and from vichan. it is still sort of clumsy right now...
> the fields "date" and "from" are not acknowledged for some reason, so posts get the timestamp of the import, and are "forced anonymous". guess i can live with both, but will try to get it done right.

Did you change something since you wrote the above? I'm seeing usernames in r.s.test starting with this post:
Re: Does this encryption thing work ? Retro Guy@rslight.i2p 05/23/20 (Sat) 23:29:02 No.141

> the work is a bit tedious, but will be definitely be worth it in the end. vichan is super easy to configure, modify, moderate, and it is just looking very nice (still playing with all the css that come with it, but will most likely stay on the dark one).

It is looking great!

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

Re: hey, what do know ? vichan supports nntp

<6b6cb9f2c92ff8fb564128e73a2f66e3$1@www.novabbs.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=289&group=rocksolid.nodes#289

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro ...@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sun, 24 May 2020 00:57 UTC
Organization: Rocksolid Light
Message-ID: <6b6cb9f2c92ff8fb564128e73a2f66e3$1@www.novabbs.com>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion> <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com> <0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="25294"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$1XJ3MkMZyL55I8fbyiUVAeUFexaXpF5ATBcHN5sUy8BwvR4L3Ntlq
 by: Retro Guy - Sun, 24 May 2020 00:57 UTC

anonymous wrote:

>>One thing you can do is grep overview for the message-id of the referenced article.

> I used message_read for now, and made one new file to follow the status of the exports. Actually, this kind of thing would be better done in one of the existing code bases, this was just the fastest way for me to test the concept.

Same as the way I test. I just write a test script and get it working, then apply to existing codebase. Much easier to not break existing code that way.

> The code lines below export any new message that is found into a format that is translated later in the new post.php of vichan.

> So I guess I could call this script at the end of spoolnews (or maybe just insert it).

What I'd like to see, if it's possible, is calling from spoolnews. We just put an $enable_vichan variable in config so if using vichan it will run the code and send the data needed by vichan. If disabled, it does nothing.

> On the other end, I have a modified post.php, which reads the oldest message (by name) and imports it. I could not modify it in a way yet that would allow to cycle through all existing messages, so the whole setup is just an interim solution yet (but it is importing the messages right now).

Since spoolnews loops through every message that it imports, it should work to just make the call to format and send to vichan in that loop. Every message rslight decides it wants (let rslight determine if a message is new or already existing), will then get sent to vichan. Thoughts?

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

Re: hey, what do know ? vichan supports nntp

<361949e1ee56e582fef5974d3a8fac8f$1@def2.i2p>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=290&group=rocksolid.nodes#290

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sun, 24 May 2020 11:23 UTC
Organization: def2
Message-ID: <361949e1ee56e582fef5974d3a8fac8f$1@def2.i2p>
References: <56148a22de20564af15e4ee0c7a2536e$1@news.novabbs.com> <c4b1c865ef480a6f4d9459af1391a116@def4> <4a2c95c3090fd2779b68fe50d43b2854$1@news.novabbs.com> <1b2bbbcb354ea383343edde8d8bdffe3$1@z5bqfv5v75kxy7pj.onion> <dfc3ccdd0ba2d83aa3febd427357defd$1@z5bqfv5v75kxy7pj.onion> <47921547e66d1985844c048b2f4a280f$1@z5bqfv5v75kxy7pj.onion> <746ad48c7dcd992903b0f09a875f10fd$1@z5bqfv5v75kxy7pj.onion> <208a215d095e2a90acdc545066307dbe$1@news.novabbs.com> <0aa95e99d0630d9dff411f409db00e5e$1@z5bqfv5v75kxy7pj.onion> <6b6cb9f2c92ff8fb564128e73a2f66e3$1@www.novabbs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="32653"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$xIKpsUtvGQdrcbnx8i0Rm..Mmk5WR3dIMXRPVs5Dh5TtOVngsKCae
 by: anonymous - Sun, 24 May 2020 11:23 UTC

>Since spoolnews loops through every message that it imports, it should work to just make the call to format and send to vichan in that loop. Every message rslight decides it wants (let rslight determine if a message is new or already existing), will then get sent to vichan. Thoughts?

Yes, that is the right approach imo. I just need to dig out the id of the message that started the thread before I can send it, and insert it as the first reference.

The usernames in the messages where easy to switch on.
--
Posted on def2
def2.i2p

Re: hey, what do know ? vichan supports nntp

<96872abfea21a7f3d415a30e15921a3b@def4>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=291&group=rocksolid.nodes#291

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!rocksolid2!def5!POSTED.localhost!not-for-mail
From: ano...@anon.com (anon)
Newsgroups: rocksolid.nodes
Message-ID: <96872abfea21a7f3d415a30e15921a3b@def4>
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sun, 24 May 2020 16:22:42+0000
Organization: def5
In-Reply-To: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
 by: anon - Sun, 24 May 2020 16:22 UTC

to dig out the thread starting message, this code should do it, right ? (from article-flat.php):

if($message) {
// load thread-data and get IDs of the actual subthread
$thread=thread_load($group);
$subthread=thread_getsubthreadids($message->header->id,$thread);
if($thread_articles == false) {
sort($subthread);
}

trw

Posted on def4

Re: hey, what do know ? vichan supports nntp

<1f58d56998423c513ee9c6e5e98df25f$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=292&group=rocksolid.nodes#292

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Sun, 24 May 2020 21:23 UTC
Organization: def2
Message-ID: <1f58d56998423c513ee9c6e5e98df25f$1@z5bqfv5v75kxy7pj.onion>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4> <96872abfea21a7f3d415a30e15921a3b@def4>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="10701"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$V02o9AfgAbNYaKuY4ltWWurGO9EdrvTq0go1Uyyjktz8v7vtFTWD6
 by: anonymous - Sun, 24 May 2020 21:23 UTC

I did my own code to do the threading like vichan expects it, wasn't so hard after all (one recursive function, basically).
I just redo the message import, still get mistakes, but a lot less. seems more and more usable. If only I knew what vichan is expecting exactly as a date (this is the last (obvious) thing to solve).

cheers

trw
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do you know....

<6b0ef1d5b22aa3329e5020265a6a3f2b@def4>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=293&group=rocksolid.nodes#293

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!rocksolid2!def5!POSTED.localhost!not-for-mail
From: ano...@anon.com (anon)
Newsgroups: rocksolid.nodes
Message-ID: <6b0ef1d5b22aa3329e5020265a6a3f2b@def4>
Subject: Re: hey, what do you know....
Date: Sun, 24 May 2020 23:38:30+0000
Organization: def5
In-Reply-To: <1f58d56998423c513ee9c6e5e98df25f$1@z5bqfv5v75kxy7pj.onion>
References: <1f58d56998423c513ee9c6e5e98df25f$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
 by: anon - Sun, 24 May 2020 23:38 UTC

>If only I knew what vichan is expecting exactly as a date (this is the last (obvious) thing to solve).

Turns out that it expected what I had been trying at the very start: a unix timestamp. ...it just doesn't respect it !
So for some reason the date of the import is always displayed as the messages time, no matter what the original time is... which is no problem if updates happen in timely manner, but for rebuilding an existing message base it is not so good (at least one context of the messages is gone).
guess the image boards have always been fixated on short term, low latency interaction...

Posted on def4

Re: hey, what do know ? vichan supports nntp

<5647b0460b715d292fd9a74f64a0be13$1@z5bqfv5v75kxy7pj.onion>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=294&group=rocksolid.nodes#294

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: anonym...@anon.com (anonymous)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Mon, 25 May 2020 21:49 UTC
Organization: def2
Message-ID: <5647b0460b715d292fd9a74f64a0be13$1@z5bqfv5v75kxy7pj.onion>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4> <96872abfea21a7f3d415a30e15921a3b@def4> <1f58d56998423c513ee9c6e5e98df25f$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="27972"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Rslight-Site: $2y$10$Afzq/Yq0do2nvWdI0I6l0.aNTtt6uTikwPWylFElmx6JOCu2kGubS
 by: anonymous - Mon, 25 May 2020 21:49 UTC

had to rewrite the re-indexing code due to some bugs, this is the data exchange format I use now:

---snip

"<04a2efaf39f90b7fd445b3d34b04e849$1@retrobbs.rocksolidbbs.com>": {
"parents": [
"<5fa5f7ef9a71f09f4a486c3d579083bf@def4.com>",
"<fb56c3093f6c0f918ca58cd34ad42065@def4.com>",
"<prlg96$jgf$1@def3.retrobbs.com>"
],
"op": false,
"ref": "<prlg96$jgf$1@def3.retrobbs.com>",
"ref_org": "<5fa5f7ef9a71f09f4a486c3d579083bf@def4.com>",
"solved": true
},
"<d11adec78757e4dad3e4b7d2f46880f5$1@dkzerogt6z6ybhcj.onion>": {
"parents": [],
"op": true
},
"<b5d27591156eeb51329e572177153951$1@www.novabbs.com>": {
"parents": [],
"op": true
},
"<7c83a8ef135c1022d441adde9029441b$1@www.novabbs.com>": {
"parents": [
"<b5d27591156eeb51329e572177153951$1@www.novabbs.com>"
],
"op": false,
"ref": "<b5d27591156eeb51329e572177153951$1@www.novabbs.com>",
"ref_org": "<b5d27591156eeb51329e572177153951$1@www.novabbs.com>",
"solved": true
},

---snap

with this implemented properly, most of the threads should work on vichan as well. than i just have to reindex all messages.
but first the rslight upgrade.

cheers

trw
--
Posted on def2
z5bqfv5v75kxy7pj.onion

Re: hey, what do know ? vichan supports nntp

<23446718aac5c9bb5100df609426140f$1@www.novabbs.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=295&group=rocksolid.nodes#295

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro ...@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: hey, what do know ? vichan supports nntp
Date: Tue, 26 May 2020 09:07 UTC
Organization: Rocksolid Light
Message-ID: <23446718aac5c9bb5100df609426140f$1@www.novabbs.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4> <96872abfea21a7f3d415a30e15921a3b@def4> <1f58d56998423c513ee9c6e5e98df25f$1@z5bqfv5v75kxy7pj.onion> <5647b0460b715d292fd9a74f64a0be13$1@z5bqfv5v75kxy7pj.onion>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="4591"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$dWDFsiqoV6ItI2LRGxPlOeV3chnlxLhYJFh/QQ61utotCgW5fGVQO
 by: Retro Guy - Tue, 26 May 2020 09:07 UTC

anonymous wrote:

> had to rewrite the re-indexing code due to some bugs, this is the data exchange format I use now:

> ---snip

> with this implemented properly, most of the threads should work on vichan as well. than i just have to reindex all messages.
> but first the rslight upgrade.

This is awesome! You're really making some great mods to vichan!

Once you have some code you want applied to spoolnews let me know. I'd like to either get most of it into a function (or two), or a script itself (vichan.php?)

I'll just put a toggle in config like I do with synchronet ($synchronet=true/false).

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

Re: hey, what do know ? vichan supports nntp

<6effa0f2ee99f0f0c4d5ce67b9849296@def4>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=296&group=rocksolid.nodes#296

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!rocksolid2!def5!POSTED.localhost!not-for-mail
From: ano...@anon.com (anon)
Newsgroups: rocksolid.nodes
Message-ID: <6effa0f2ee99f0f0c4d5ce67b9849296@def4>
Subject: Re: hey, what do know ? vichan supports nntp
Date: Tue, 26 May 2020 10:50:56+0000
Organization: def5
In-Reply-To: <23446718aac5c9bb5100df609426140f$1@www.novabbs.com>
References: <23446718aac5c9bb5100df609426140f$1@www.novabbs.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
 by: anon - Tue, 26 May 2020 10:50 UTC

>This is awesome! You're really making some great mods to vichan!

Thanks for the kind words, it is really fun to work on that.

Just in case some vichan or nntpchan dev should ever read this thread: I'm still standing on the proverbial "shoulders of giants". Anyway, thanks for this nice piece of software.

cheers

trw

Posted on def4

basics are working

<n.232.195omy@anon.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=297&group=rocksolid.nodes#297

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: pos...@anon.com (Anonymous)
Newsgroups: rocksolid.nodes
Subject: basics are working
Date: Tue, 26 May 2020 08:34:01 -0700
Organization: i2pn2 (i2pn.org)
Message-ID: <n.232.195omy@anon.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Mime-Version: 1.0
Content-Type: multipart/mixed; boundary=d299d1fb9af5b5d0014ff7992bc19505ae56652d
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="20677"; mail-complaints-to="usenet@i2pn2.org"
 by: Anonymous - Tue, 26 May 2020 15:34 UTC
Attachments: "bg_jyc_new.png"; name="attachment" (image/png)

with some 11 exceptions my routine could glue all the threads together again.
this is still far from being elegant, mind you....

this code handles all the exporting from rslight to vichan.
-it checks the articles in the spool,
-builds an index file and a watermark file (if necessary),
-then resolves all the broken threads (fixed 10 iterations, have to make that dynamic later),
-then exports everything to vichan.

on the vichan side there is a modified post.php to get the messages.

so far the three parts (rslight, vichan and the script) are fully decoupled and individually called from cron.

in the long run code could be merged of course.

import script:

<?php

include "config.inc.php";
include "$file_newsportal";
include 'message.inc.php';

$groups = array(
'rocksolid.nodes',
'rocksolid.nodes.help',
'rocksolid.nodes.announce',
'rocksolid.shared.encryption',
'rocksolid.shared.entertainment',
'rocksolid.shared.freenet',
'rocksolid.shared.general',
'rocksolid.shared.hacking',
'rocksolid.shared.helpdesk',
'rocksolid.shared.i2p',
'rocksolid.shared.linux',
'rocksolid.shared.news',
'rocksolid.shared.offtopic',
'rocksolid.programming',
'rocksolid.shared.rocksolid',
'rocksolid.shared.security',
'rocksolid.social',
'rocksolid.shared.test',
'rocksolid.shared.tor',
);

if(!file_exists('xt.json')){
echo("no index found, building it \n\r");
$threads = build_threads($groups);
} else {
$threads = json_decode(file_get_contents('xt.json'), true);
$watermarks = json_decode(file_get_contents('xw.json'), true);
$threads = update_threads($groups, $threads, $watermarks);
} $thread_keys = array_keys($threads);
for ($i = 0; $i < 10; $i++){
$threads = stage($thread_keys, $threads);
}

$jsonFile = fopen('xt.json', 'w');
fwrite($jsonFile, json_encode($threads, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonFile);

update_sync($threads);

exit;

function build_threads($groups){
$threads = array();
$watermarks = array();
foreach($groups as $group){
$watermarks[$group] = array();
echo("building index for group " . $group . "\n\r");
for ($i = 0; $i < 1000; $i++){
$message = message_read($i, 0, $group);
if($message){
echo("processing message # " . $i . " from group " . $group . "\r");
$rawData = json_decode(json_encode((array) $message, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES), true);
$ID = $rawData['header']['id'];
$Ref = $rawData['header']['references']['0'];
$firstRef = explode("<", $Ref);
$Ref = ("<" . $firstRef[1]);
$threads[$ID] = array();
$threads[$ID]['parents'] = array();
$threads[$ID]['synced'] = false;
$threads[$ID]['number'] = $i;
$threads[$ID]['group'] = $group;
$threads[$ID]['message'] = array();
$threads[$ID]['message'] = $rawData;
if($Ref == "<"){
$threads[$ID]['op'] = true;
$threads[$ID]['solved'] = true;
} else {
$threads[$ID]['op'] = false;
$threads[$ID]['ref'] = $Ref;
$threads[$ID]['ref_org'] = $rawData['header']['references']['0'];
$threads[$ID]['solved'] = false;
}
array_push($watermarks[$group],$i);
}
}
}
$jsonFile = fopen('xt.json', 'w');
fwrite($jsonFile, json_encode($threads, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonFile);
$jsonWatermarks = fopen('xw.json', 'w');
fwrite($jsonWatermarks, json_encode((array) $watermarks, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonWatermarks);
return $threads;
} function update_threads($groups, $threads, $watermarks){
$update = false;
foreach($groups as $group){
echo("updating index for group " . $group . "\n\r");
for ($i = 0; $i < 1000; $i++){
if(!in_array($i,$watermarks[$group])){
$message = message_read($i, 0, $group);
if($message){
echo("processing message # " . $i . " from group " . $group . "\r");
$update = true;
$rawData = json_decode(json_encode((array) $message, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES), true);
$ID = $rawData['header']['id'];
$Ref = $rawData['header']['references']['0'];
$firstRef = explode("<", $Ref);
$Ref = ("<" . $firstRef[1]);
$threads[$ID] = array();
$threads[$ID]['parents'] = array();
$threads[$ID]['synced'] = false;
$threads[$ID]['number'] = $i;
$threads[$ID]['group'] = $group;
$threads[$ID]['message'] = array();
$threads[$ID]['message'] = $rawData;
if($Ref == "<"){
$threads[$ID]['op'] = true;
$threads[$ID]['solved'] = true;
} else {
$threads[$ID]['op'] = false;
$threads[$ID]['ref'] = $Ref;
$threads[$ID]['ref_org'] = $rawData['header']['references']['0'];
$threads[$ID]['solved'] = false;
}
array_push($watermarks[$group],$i);
}
}
}
}
if($update){
echo("saving updated threading\n\r");
$jsonFile = fopen('xt.json', 'w');
fwrite($jsonFile, json_encode($threads, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonFile);
echo("saving updated watermarks\n\r");
$jsonWatermarks = fopen('xw.json', 'w');
fwrite($jsonWatermarks, json_encode((array) $watermarks, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonWatermarks);
} else {
echo("no new files found\n\r");
}
return $threads;
}

function stage($thread_keys, $threads){
foreach($thread_keys as $thread_key){
if( ($threads[$thread_key]['solved'] !== true) && (threads[$thread_key]['op'] !== true) ){
echo("checking message" . $thread_key . "\r");
if ($threads[$thread_key]['parents']){
$to_check = end($threads[$thread_key]['parents']);
if(!array_key_exists($to_check, $threads)){
$threads[$thread_key]['op'] = true;
$threads[$thread_key]['ref'] = array();
$threads[$thread_key]['missing_ref'] = $to_check;
$threads[$thread_key]['solved'] = true;
} else {
if($threads[$to_check]['op'] == true){
$threads[$thread_key]['ref'] = $to_check;
$threads[$thread_key]['solved'] = true;
} else {
array_push($threads[$thread_key]['parents'], $threads[$to_check]['ref']);
array_filter($threads[$thread_key]['parents'], static function($var){return $var !== null;} );
}
}
} elseif ($threads[$thread_key]['ref']){
array_push($threads[$thread_key]['parents'], $threads[$thread_key]['ref']);
}
}
}
return $threads;
}

function update_sync($threads){
$update = false;
$thread_keys = array_keys($threads);
foreach($thread_keys as $thread_key){
if( ($threads[$thread_key]['synced'] == false) && ($threads[$thread_key]['solved'] == true) ){
echo("processing message # " . $threads[$thread_key]['number'] . " from group " . $threads[$thread_key]['group'] . "\n\r");
$rawData = $threads[$thread_key]['message'];
if($threads[$thread_key]['ref']){
$rawData['header']['references']['0'] = $threads[$thread_key]['ref'];
} else {
unset($rawData['header']['references']);
}
$paddedI = str_pad($threads[$thread_key]['number'], 5, '0', STR_PAD_LEFT);
$jsonFile = fopen('/var/www/html/vi/vichan-master/debug/' . $paddedI . '_' . $threads[$thread_key]['group'] . '.json', 'w');
fwrite($jsonFile, json_encode((array) $rawData, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonFile);
chown($jsonFile, 'www-data');
chgrp($jsonFile, 'www-data');
$update = true;
$threads[$thread_key]['synced'] = true;
echo($thread['synced'] . "\n\r");
}
}
if($update){
echo("saving updated syncing\n\r");
$jsonFile = fopen('xt.json', 'w');
fwrite($jsonFile, json_encode($threads, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES));
fclose($jsonFile);
} else {
echo("no new files found for syncing\n\r");
}
}
Click here to read the complete articleAttachments:  

guess i will open for business soon

<n.233.40xx8o@anon.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=298&group=rocksolid.nodes#298

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: pos...@anon.com (Anonymous)
Newsgroups: rocksolid.nodes
Subject: guess i will open for business soon
Date: Tue, 26 May 2020 09:24:55 -0700
Organization: i2pn2 (i2pn.org)
Message-ID: <n.233.40xx8o@anon.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Content-Type: text/plain; charset=UTF-8
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="26956"; mail-complaints-to="usenet@i2pn2.org"
 by: Anonymous - Tue, 26 May 2020 16:24 UTC

....and switch def2.i2p to vichan

just a little security research before, guess shouldn't take too long.

cheers

trw

opsec check is good

<n.234.1ks519@anon.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=299&group=rocksolid.nodes#299

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: pos...@anon.com (Anonymous)
Newsgroups: rocksolid.nodes
Subject: opsec check is good
Date: Tue, 26 May 2020 12:44:12 -0700
Organization: i2pn2 (i2pn.org)
Message-ID: <n.234.1ks519@anon.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Content-Type: text/plain; charset=UTF-8
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="21161"; mail-complaints-to="usenet@i2pn2.org"
 by: Anonymous - Tue, 26 May 2020 19:44 UTC

no bugs listed at bugtraq, none in my favorite hacker forum, none in google, no known hacks that I would heard about.
seems like we're goo to go.

cheers

trw

> goo to go.

<n.235.3s4rn9@anon.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=300&group=rocksolid.nodes#300

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: pos...@anon.com (Anonymous)
Newsgroups: rocksolid.nodes
Subject: > goo to go.
Date: Tue, 26 May 2020 12:47:12 -0700
Organization: i2pn2 (i2pn.org)
Message-ID: <n.235.3s4rn9@anon.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Content-Type: text/plain; charset=UTF-8
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="21641"; mail-complaints-to="usenet@i2pn2.org"
 by: Anonymous - Tue, 26 May 2020 19:47 UTC

yeah, we are googoogoogoo...
good to go. spelling is difficult.

Re: basics are working

<af0db666e6c7dab4838ec58b91d4418e$1@www.novabbs.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=301&group=rocksolid.nodes#301

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro ...@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: basics are working
Date: Tue, 26 May 2020 19:45 UTC
Organization: Rocksolid Light
Message-ID: <af0db666e6c7dab4838ec58b91d4418e$1@www.novabbs.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4> <n.232.195omy@anon.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="21772"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$g7YtmFskVIFxmMkGAojHlOKYpZO46BfnmFNJpEirjYB5v/DP.aOZW
 by: Retro Guy - Tue, 26 May 2020 19:45 UTC

Anonymous wrote:

> with some 11 exceptions my routine could glue all the threads together again.
> this is still far from being elegant, mind you....

> this code handles all the exporting from rslight to vichan.
> -it checks the articles in the spool,
> -builds an index file and a watermark file (if necessary),
> -then resolves all the broken threads (fixed 10 iterations, have to make that dynamic later),
> -then exports everything to vichan.

> on the vichan side there is a modified post.php to get the messages.

> so far the three parts (rslight, vichan and the script) are fully decoupled and individually called from cron.

That looks like a great way to handle it. So when complete, if I just add a $enable_vichan in config, then reference that in cron.php, I can just use that to decide whether to run the code at all. Is that correct?

Retro Guy

--
Posted on: Rocksolid Light
www.novabbs.com

yes that should work

<n.237.1zueiu@anon.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=302&group=rocksolid.nodes#302

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: pos...@anon.com (Anonymous)
Newsgroups: rocksolid.nodes
Subject: yes that should work
Date: Wed, 27 May 2020 00:58:20 -0700
Organization: i2pn2 (i2pn.org)
Message-ID: <n.237.1zueiu@anon.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Content-Type: text/plain; charset=UTF-8
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="7931"; mail-complaints-to="usenet@i2pn2.org"
 by: Anonymous - Wed, 27 May 2020 07:58 UTC

>I can just use that to decide whether to run the code at all. Is that correct?

should work as far my export script is concerned.
on the vichan side of things i think the import script (the modified post.php) has to run from crontab directly, when i called post.php from another php script, I was getting errors (which could be fixed, i guess, but i could not find how, it was missing includes, most likely because the path was not right).

cheers

trw

Re: yes that should work

<1c6e36fd6d47d1387dfac1f87369534e$1@www.novabbs.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=303&group=rocksolid.nodes#303

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Retro ...@rslight.i2p (Retro Guy)
Newsgroups: rocksolid.nodes
Subject: Re: yes that should work
Date: Wed, 27 May 2020 09:10 UTC
Organization: Rocksolid Light
Message-ID: <1c6e36fd6d47d1387dfac1f87369534e$1@www.novabbs.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4> <n.237.1zueiu@anon.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="17160"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$ljiXzlIoeiYgyYafSULqXe51sRIaCw/Ru/kwSuIG2/dm74J3S.dh6
 by: Retro Guy - Wed, 27 May 2020 09:10 UTC

Anonymous wrote:

>>I can just use that to decide whether to run the code at all. Is that correct?

> should work as far my export script is concerned.
> on the vichan side of things i think the import script (the modified post.php) has to run from crontab directly, when i called post.php from another php script, I was getting errors (which could be fixed, i guess, but i could not find how, it was missing includes, most likely because the path was not right).

That sounds likely. Running from cron.php is no problem, so no real reason to change it.

--
Posted on: Rocksolid Light
www.novabbs.com

cron or not ?

<n.239.3xbw1r@anon.com>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=304&group=rocksolid.nodes#304

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: pos...@anon.com (Anonymous)
Newsgroups: rocksolid.nodes
Subject: cron or not ?
Date: Wed, 27 May 2020 05:03:12 -0700
Organization: i2pn2 (i2pn.org)
Message-ID: <n.239.3xbw1r@anon.com>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4>
Content-Type: text/plain; charset=UTF-8
Injection-Info: i2pn2.org; posting-account="def2";
logging-data="6378"; mail-complaints-to="usenet@i2pn2.org"
 by: Anonymous - Wed, 27 May 2020 12:03 UTC

>That sounds likely. Running from cron.php is no problem, so no real reason to change it.

There would be one reason, which is that running from cron, you can only run the script blindly, regardless if there messages to import or not. And in case of many messages, you can only import one per minute (limitation of cron).
Both is fine for now, but actually you can also use a browser to call the script locally, and this could be done from another script. I just have to figure out how to make curl disregard the proxy options of the system and send the request directly, than I can implement that directly in my import script.
Could use wget as well, but prefer not to use any exec if I don't have to.

cheers

trw

Re: guess i will open for business soon

<9215a36f37b5d9c947c3414131d9abe3$1@rslight.i2p>

 copy mid

https://www.novabbs.com/computers/article-flat.php?id=305&group=rocksolid.nodes#305

 copy link   Newsgroups: rocksolid.nodes
Path: i2pn2.org!.POSTED!not-for-mail
From: Mei...@rslight.i2p (Mei)
Newsgroups: rocksolid.nodes
Subject: Re: guess i will open for business soon
Date: Thu, 28 May 2020 07:56 UTC
Organization: Rocksolid Light
Message-ID: <9215a36f37b5d9c947c3414131d9abe3$1@rslight.i2p>
References: <7b6448e9ac3e8f40f5ff6b8fa9b433b6@def4> <n.233.40xx8o@anon.com>
Mime-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Injection-Info: i2pn2.org; posting-account="retrobbs1";
logging-data="15623"; mail-complaints-to="usenet@i2pn2.org"
User-Agent: Rocksolid Light (news.novabbs.com/getrslight)
X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on novabbs.com
X-Rslight-Site: $2y$10$cZdsjA53cTvRkEEAv4sl..7TwIuG1zsVRtgAgWlWYzDxTtj5MYvYW
 by: Mei - Thu, 28 May 2020 07:56 UTC

As my sight begins to return I curse you for making it so bright. We're on the DARK web, you can't have light themes here!

What do you mean "that's not what that means?" It is now!
--
Posted on: Rocksolid Light
rslight.i2p

Pages:1234
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor