Rocksolid Light

Welcome to novaBBS (click a section below)

mail  files  register  newsreader  groups  login

Message-ID:  

Bus error -- please leave by the rear door.


devel / comp.unix.bsd.freebsd.misc / Hard crashes and swap issues

SubjectAuthor
* Hard crashes and swap issuesLouis Epstein
`* Re: Hard crashes and swap issuesLouis Epstein
 `* Re: Hard crashes and swap issuesLouis Epstein
  `* Re: Hard crashes and swap issuesWinston
   `* Re: Hard crashes and swap issuesLouis Epstein
    +* Re: Hard crashes and swap issuesLouis Epstein
    |`* Re: Hard crashes and swap issuesLouis Epstein
    | `* Re: Hard crashes and swap issuesbob prohaska
    |  `* Re: Hard crashes and swap issuesLouis Epstein
    |   `- Re: Hard crashes and swap issuesbob prohaska
    `* Re: Hard crashes and swap issuesWinston
     `* Re: Hard crashes and swap issuesLouis Epstein
      `* Re: Hard crashes and swap issuesWinston
       +- Re: Hard crashes and swap issuesWinston
       `* Re: Hard crashes and swap issuesLouis Epstein
        +- Re: Hard crashes and swap issuesLouis Epstein
        `* Re: Hard crashes and swap issuesWinston
         `* Re: Hard crashes and swap issuesLouis Epstein
          `* Re: Hard crashes and swap issuesWinston
           `* Re: Hard crashes and swap issuesLouis Epstein
            `* Re: Hard crashes and swap issuesWinston
             `- Re: Hard crashes and swap issuesLouis Epstein

1
Hard crashes and swap issues

<ubuiqq$4dq$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=432&group=comp.unix.bsd.freebsd.misc#432

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Hard crashes and swap issues
Date: Mon, 21 Aug 2023 02:40:26 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <ubuiqq$4dq$1@reader2.panix.com>
Injection-Date: Mon, 21 Aug 2023 02:40:26 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="4538"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p2 (amd64))
 by: Louis Epstein - Mon, 21 Aug 2023 02:40 UTC

For some time now when I run synth builds lang/rust
never completes,running out of swap space and making
over a dozen other ports that wait for it to get
built be skipped.

In the last few days,however,the swap space (I have 19G)
running out has not only caused a failure of rust or
a termination of synth,but made other applications terminate
("unable to recover memory") and logged me out,and when
I log in I discover that programs such as httpd and sendmail
are no longer running...

and furthermore,when I try a shutdown to reboot,it doesn't
complete the process,just sits after syncer is finished
and never powers down from shutdown -p
(and despite all the controlled shutdowns that it has
gone through,when I manually power down and up I have
to fsck everything).

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<uc2tfs$gbo$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=433&group=comp.unix.bsd.freebsd.misc#433

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Tue, 22 Aug 2023 18:06:52 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <uc2tfs$gbo$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com>
Injection-Date: Tue, 22 Aug 2023 18:06:52 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="16760"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p2 (amd64))
 by: Louis Epstein - Tue, 22 Aug 2023 18:06 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> For some time now when I run synth builds lang/rust
> never completes,running out of swap space and making
> over a dozen other ports that wait for it to get
> built be skipped.
>
> In the last few days,however,the swap space (I have 19G)
> running out has not only caused a failure of rust or
> a termination of synth,but made other applications terminate
> ("unable to recover memory") and logged me out,and when
> I log in I discover that programs such as httpd and sendmail
> are no longer running...
>
> and furthermore,when I try a shutdown to reboot,it doesn't
> complete the process,just sits after syncer is finished
> and never powers down from shutdown -p
> (and despite all the controlled shutdowns that it has
> gone through,when I manually power down and up I have
> to fsck everything).

shutdown -r also stops cold after sync-ing.
> -=-=-
> The World Trade Center towers MUST rise again,
> at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<uc5iad$5q8$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=434&group=comp.unix.bsd.freebsd.misc#434

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Wed, 23 Aug 2023 18:14:37 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <uc5iad$5q8$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
Injection-Date: Wed, 23 Aug 2023 18:14:37 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="5960"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p2 (amd64))
 by: Louis Epstein - Wed, 23 Aug 2023 18:14 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>> For some time now when I run synth builds lang/rust
>> never completes,running out of swap space and making
>> over a dozen other ports that wait for it to get
>> built be skipped.
>>
>> In the last few days,however,the swap space (I have 19G)
>> running out has not only caused a failure of rust or
>> a termination of synth,but made other applications terminate
>> ("unable to recover memory") and logged me out,and when
>> I log in I discover that programs such as httpd and sendmail
>> are no longer running...
>>
>> and furthermore,when I try a shutdown to reboot,it doesn't
>> complete the process,just sits after syncer is finished
>> and never powers down from shutdown -p
>> (and despite all the controlled shutdowns that it has
>> gone through,when I manually power down and up I have
>> to fsck everything).
>
> shutdown -r also stops cold after sync-ing.

And killing the rust build when 7% of swap space is in use
still leads to an unable to get swap space message just
as killing it when 99.9% of swap space is in use before
the shutdown.

The swp_pager message comes after the "all buffers synced"
and I note a message above that saying that rc.shutdown
had an abnormal termination and it's going to single user
mode but then it never gets there.

>> -=-=-
>> The World Trade Center towers MUST rise again,
>> at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<ydfs49f02v.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=436&group=comp.unix.bsd.freebsd.misc#436

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Thu, 24 Aug 2023 02:04:08 -0400
Organization: A noiseless patient Spider
Lines: 19
Message-ID: <ydfs49f02v.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="f631cf3ed589af523aa47b4f20b21122";
logging-data="3535152"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19CaPBzwU2yIfhOkJ0it6ji"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix)
Cancel-Lock: sha1:/HRllA5VGF0N5wAMUdwUp6OwOIY=
sha1:X/2HZ6XlRhE8H4JOfCh/5Bu4ok8=
Mail-Copies-To: never
 by: Winston - Thu, 24 Aug 2023 06:04 UTC

Louis Epstein <le@main.lekno.ws> wrote:
>>> In the last few days,however,the swap space (I have 19G)
>>> running out has not only caused a failure of rust or
>>> a termination of synth,but made other applications terminate
>>> ("unable to recover memory") and logged me out,and when
>>> I log in I discover that programs such as httpd and sendmail
>>> are no longer running...

and later added:
> And killing the rust build when 7% of swap space is in use
> still leads to an unable to get swap space message just
> as killing it when 99.9% of swap space is in use before
> the shutdown.

Wild stab in the dark ...

If you're using tmpfs and it's an unknown tmpfs bug,
perhaps disable tmpfs and see if that makes a difference?
-WBE

Re: Hard crashes and swap issues

<uelvmi$68p$2@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=437&group=comp.unix.bsd.freebsd.misc#437

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sat, 23 Sep 2023 06:13:38 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <uelvmi$68p$2@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
Injection-Date: Sat, 23 Sep 2023 06:13:38 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="6425"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p3 (amd64))
 by: Louis Epstein - Sat, 23 Sep 2023 06:13 UTC

Winston <wbe@ubeblock.psr.com.invalid> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>>>> In the last few days,however,the swap space (I have 19G)
>>>> running out has not only caused a failure of rust or
>>>> a termination of synth,but made other applications terminate
>>>> ("unable to recover memory") and logged me out,and when
>>>> I log in I discover that programs such as httpd and sendmail
>>>> are no longer running...
>
> and later added:
>> And killing the rust build when 7% of swap space is in use
>> still leads to an unable to get swap space message just
>> as killing it when 99.9% of swap space is in use before
>> the shutdown.
>
> Wild stab in the dark ...
>
> If you're using tmpfs and it's an unknown tmpfs bug,
> perhaps disable tmpfs and see if that makes a difference?
> -WBE

Will have to consider that,tmpfs is certainly
shown on a pre-shutdown df with lots of synth material.

The problem had gone away for a while but returned recently.

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<ugd7mk$45n$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=439&group=comp.unix.bsd.freebsd.misc#439

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sat, 14 Oct 2023 05:07:32 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <ugd7mk$45n$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com>
Injection-Date: Sat, 14 Oct 2023 05:07:32 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="4279"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Sat, 14 Oct 2023 05:07 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> Winston <wbe@ubeblock.psr.com.invalid> wrote:
>> Louis Epstein <le@main.lekno.ws> wrote:
>>>>> In the last few days,however,the swap space (I have 19G)
>>>>> running out has not only caused a failure of rust or
>>>>> a termination of synth,but made other applications terminate
>>>>> ("unable to recover memory") and logged me out,and when
>>>>> I log in I discover that programs such as httpd and sendmail
>>>>> are no longer running...
>>
>> and later added:
>>> And killing the rust build when 7% of swap space is in use
>>> still leads to an unable to get swap space message just
>>> as killing it when 99.9% of swap space is in use before
>>> the shutdown.
>>
>> Wild stab in the dark ...
>>
>> If you're using tmpfs and it's an unknown tmpfs bug,
>> perhaps disable tmpfs and see if that makes a difference?
>> -WBE
>
> Will have to consider that,tmpfs is certainly
> shown on a pre-shutdown df with lots of synth material.
>
> The problem had gone away for a while but returned recently.

Now it may indeed by gone...after a rust config-option
deletion.
> -=-=-
> The World Trade Center towers MUST rise again,
> at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<uhhkik$1aa$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=440&group=comp.unix.bsd.freebsd.misc#440

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sat, 28 Oct 2023 00:28:04 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <uhhkik$1aa$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <ugd7mk$45n$1@reader2.panix.com>
Injection-Date: Sat, 28 Oct 2023 00:28:04 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="1354"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Sat, 28 Oct 2023 00:28 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>> Winston <wbe@ubeblock.psr.com.invalid> wrote:
>>> Louis Epstein <le@main.lekno.ws> wrote:
>>>>>> In the last few days,however,the swap space (I have 19G)
>>>>>> running out has not only caused a failure of rust or
>>>>>> a termination of synth,but made other applications terminate
>>>>>> ("unable to recover memory") and logged me out,and when
>>>>>> I log in I discover that programs such as httpd and sendmail
>>>>>> are no longer running...
>>>
>>> and later added:
>>>> And killing the rust build when 7% of swap space is in use
>>>> still leads to an unable to get swap space message just
>>>> as killing it when 99.9% of swap space is in use before
>>>> the shutdown.
>>>
>>> Wild stab in the dark ...
>>>
>>> If you're using tmpfs and it's an unknown tmpfs bug,
>>> perhaps disable tmpfs and see if that makes a difference?
>>> -WBE
>>
>> Will have to consider that,tmpfs is certainly
>> shown on a pre-shutdown df with lots of synth material.
>>
>> The problem had gone away for a while but returned recently.
>
> Now it may indeed by gone...after a rust config-option
> deletion.

Now it seems to have returned...

>> -=-=-
>> The World Trade Center towers MUST rise again,
>> at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<uhhufq$2n04j$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=441&group=comp.unix.bsd.freebsd.misc#441

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: bp...@www.zefox.net (bob prohaska)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sat, 28 Oct 2023 03:17:14 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 9
Message-ID: <uhhufq$2n04j$1@dont-email.me>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <ugd7mk$45n$1@reader2.panix.com> <uhhkik$1aa$1@reader2.panix.com>
Injection-Date: Sat, 28 Oct 2023 03:17:14 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="61a16efaeb830a51bb723f88cb95d465";
logging-data="2850963"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19p+3i13JMJGo5McI7kUPvtD8bHR96aowA="
User-Agent: tin/2.4.4-20191224 ("Millburn") (FreeBSD/12.4-STABLE (arm))
Cancel-Lock: sha1:udX8bADA8cdW7oYmitTrwly2wjc=
 by: bob prohaska - Sat, 28 Oct 2023 03:17 UTC

Louis Epstein <le@main.lekno.ws> wrote:
>
> Now it seems to have returned...
>

Is there a description of your setup somewhere? I didn't see it.

bob prohaska

Re: Hard crashes and swap issues

<uhm5ji$h8i$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=442&group=comp.unix.bsd.freebsd.misc#442

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sun, 29 Oct 2023 17:43:14 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <uhm5ji$h8i$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <ugd7mk$45n$1@reader2.panix.com> <uhhkik$1aa$1@reader2.panix.com> <uhhufq$2n04j$1@dont-email.me>
Injection-Date: Sun, 29 Oct 2023 17:43:14 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="17682"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Sun, 29 Oct 2023 17:43 UTC

bob prohaska <bp@www.zefox.net> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>>
>> Now it seems to have returned...
>>
>
> Is there a description of your setup somewhere? I didn't see it.

What details do you need?
I'm running 13.2-RELEASE-p4 on an Intel i7-3770K.
8G DDR3 RAM and 19G swap that gets swallowed by these crashes.

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<uhmu4l$4eau$1@dont-email.me>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=443&group=comp.unix.bsd.freebsd.misc#443

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: bp...@www.zefox.net (bob prohaska)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Mon, 30 Oct 2023 00:41:57 -0000 (UTC)
Organization: A noiseless patient Spider
Lines: 22
Message-ID: <uhmu4l$4eau$1@dont-email.me>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <ugd7mk$45n$1@reader2.panix.com> <uhhkik$1aa$1@reader2.panix.com> <uhhufq$2n04j$1@dont-email.me> <uhm5ji$h8i$1@reader2.panix.com>
Injection-Date: Mon, 30 Oct 2023 00:41:57 -0000 (UTC)
Injection-Info: dont-email.me; posting-host="62bb84ad0a7512879c0ec9c0ec8b3475";
logging-data="145758"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19M6uFg/b/+D5lbn/7EIsxSBUM1/UJFEfs="
User-Agent: tin/2.4.4-20191224 ("Millburn") (FreeBSD/12.4-STABLE (arm))
Cancel-Lock: sha1:k91XBzXbfayAPcBDHQu0EVq2Dd8=
 by: bob prohaska - Mon, 30 Oct 2023 00:41 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> bob prohaska <bp@www.zefox.net> wrote:
>> Louis Epstein <le@main.lekno.ws> wrote:
>>>
>>> Now it seems to have returned...
>>>
>>
>> Is there a description of your setup somewhere? I didn't see it.
>
> What details do you need?
> I'm running 13.2-RELEASE-p4 on an Intel i7-3770K.
> 8G DDR3 RAM and 19G swap that gets swallowed by these crashes.
>
Oops, I won't do you much good. The problems described sounded a bit
like me trying to compile large ports (think www/chromium) on Raspberry
Pi systems. I thought perhaps there might be some similarities, but
that seem most unlikely.

Apologies for intruding,

bob prohaska

Re: Hard crashes and swap issues

<yd7cn4fbgd.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=444&group=comp.unix.bsd.freebsd.misc#444

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!rocksolid2!news.neodome.net!news.mixmin.net!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Mon, 30 Oct 2023 10:10:10 -0400
Organization: A noiseless patient Spider
Lines: 39
Message-ID: <yd7cn4fbgd.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
<uelvmi$68p$2@reader2.panix.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="d5f892f459e1dc20302599e29e4896bc";
logging-data="523328"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX19MDaVcA0+mU86P6yjGKxbz"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix)
Cancel-Lock: sha1:OrHFKAX9PXR55gkI7yW/Kk1V3lQ=
sha1:Xos5ptUfP78CVmyea9z9Zpsp154=
Mail-Copies-To: never
 by: Winston - Mon, 30 Oct 2023 14:10 UTC

Louis Epstein <le@main.lekno.ws> wrote:
>>>>> In the last few days,however,the swap space (I have 19G)
>>>>> running out has not only caused a failure of rust or
>>>>> a termination of synth,but made other applications terminate
>>>>> ("unable to recover memory") and logged me out,and when
>>>>> I log in I discover that programs such as httpd and sendmail
>>>>> are no longer running...

and later added:
>>> And killing the rust build when 7% of swap space is in use
>>> still leads to an unable to get swap space message just
>>> as killing it when 99.9% of swap space is in use before
>>> the shutdown.

to which I replied:
>> Wild stab in the dark ...

>> If you're using tmpfs and it's an unknown tmpfs bug,
>> perhaps disable tmpfs and see if that makes a difference?

Louis Epstein <le@main.lekno.ws> then replied:
> Will have to consider that,tmpfs is certainly
> shown on a pre-shutdown df with lots of synth material.

> The problem had gone away for a while but returned recently.

Did you ever try the experiment of disabling tmpfs to see if
that helps?

FWIW, your issue sounds similar to the title of bug 219399
from 2017 which is still Open:

https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219399

In that bug, though, the problem was seen with an early AMD Ryzen
processor, and (skimming through the discussion), it seems like
the problem was a hardware problem where temperature caused
various RAM and CPU errors.
-WBE

Re: Hard crashes and swap issues

<uhrb8k$56i$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=445&group=comp.unix.bsd.freebsd.misc#445

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Tue, 31 Oct 2023 16:50:28 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <uhrb8k$56i$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com>
Injection-Date: Tue, 31 Oct 2023 16:50:28 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="5330"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Tue, 31 Oct 2023 16:50 UTC

Winston <wbe@ubeblock.psr.com.invalid> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>>>>>> In the last few days,however,the swap space (I have 19G)
>>>>>> running out has not only caused a failure of rust or
>>>>>> a termination of synth,but made other applications terminate
>>>>>> ("unable to recover memory") and logged me out,and when
>>>>>> I log in I discover that programs such as httpd and sendmail
>>>>>> are no longer running...
>
> and later added:
>>>> And killing the rust build when 7% of swap space is in use
>>>> still leads to an unable to get swap space message just
>>>> as killing it when 99.9% of swap space is in use before
>>>> the shutdown.
>
> to which I replied:
>>> Wild stab in the dark ...
>
>>> If you're using tmpfs and it's an unknown tmpfs bug,
>>> perhaps disable tmpfs and see if that makes a difference?
>
> Louis Epstein <le@main.lekno.ws> then replied:
>> Will have to consider that,tmpfs is certainly
>> shown on a pre-shutdown df with lots of synth material.
>
>> The problem had gone away for a while but returned recently.
>
> Did you ever try the experiment of disabling tmpfs to see if
> that helps?

Given that using tmpfs is supposed to speed things up and
the synth builds already take hours and hours when the
batch is large and the swap-swallows prevent using the
computer for anything else,I was cautious about this.

Is there a way to dismount the /tmpfs entries of df
that are there when I kill synth before the swap-swallow
has crashed the system,so I know if they play a part in
the failure of the shutdown program to execute completely?
shutdown -h and shutdown -p both end up in a circle.
> FWIW, your issue sounds similar to the title of bug 219399
> from 2017 which is still Open:
>
> https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=219399
>
> In that bug, though, the problem was seen with an early AMD Ryzen
> processor, and (skimming through the discussion), it seems like
> the problem was a hardware problem where temperature caused
> various RAM and CPU errors.
> -WBE

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<yd34xpfsdq.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=447&group=comp.unix.bsd.freebsd.misc#447

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Wed, 01 Nov 2023 10:41:21 -0400
Organization: A noiseless patient Spider
Lines: 58
Message-ID: <yd34xpfsdq.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
<uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com>
<uhrb8k$56i$1@reader2.panix.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="23b89ea94d7e807aa558c5a93e84bc7b";
logging-data="1751458"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX18HW/GTh+hGeeJfR12fluh6"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix)
Cancel-Lock: sha1:ZXcrz6ji9SVoyEvSNBzQKi0yZWQ=
sha1:5D78XhiqCEbAaNDR3zY0YROwUU0=
Mail-Copies-To: never
 by: Winston - Wed, 1 Nov 2023 14:41 UTC

As part of the thread, I asked:
>> Did you ever try the experiment of disabling tmpfs to see if
>> that helps?

to which Louis Epstein <le@main.lekno.ws> responded:
> Given that using tmpfs is supposed to speed things up and
> the synth builds already take hours and hours when the
> batch is large and the swap-swallows prevent using the
> computer for anything else,I was cautious about this.

> Is there a way to dismount the /tmpfs entries of df
> that are there when I kill synth before the swap-swallow
> has crashed the system, so I know if they play a part in
> the failure of the shutdown program to execute completely?
> shutdown -h and shutdown -p both end up in a circle.

I'm not confident I understand what you're asking, but
perhaps one or more of the following address your question.

While I would think that disabling tmpfs is the only
definitive way to test whether it's a tmpfs bug, if you
don't wish to disable tmpfs:

* You could run pstat or swapinfo from time to see how much
swap space is in use.

* If you have >1 swap area, you could consider suspending
the build and then using swapoff to remove one of them.
It'll take a while to move the active content to the
remaining swap space. Then use swapon to re-add it.
Hopefully, tmpfs handles swap space addition and removal
well.

See if the reported swap space used changes significantly.
If so, maybe it's a tmpfs bug (not freeing no-longer-used
swap space).

* protect(1) can be use to keep processes from being killed
when swap space runs out, but, as the man page warns,
that can also lead to the system hanging instead of
crashing.

* Maybe check temperatures from time to time while the
build is running, too.

* Periodic logging: maybe write a trivial shell loop to run
every few seconds or so to write the output of swapinfo,
df -h /tmp, temperatures, etc. to a file which you could
examine after rebooting.

If swap space doesn't grow gradually, it starts to look
more like a bit error or really-hard-to-find bug.

In the case of bug 219399, small increases in voltages
reduced, but didn't eliminate, bit errors caused by heat.

HTH,
-WBE

Re: Hard crashes and swap issues

<ydy1fhe6qy.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=448&group=comp.unix.bsd.freebsd.misc#448

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!paganini.bofh.team!eternal-september.org!feeder2.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Wed, 01 Nov 2023 13:13:57 -0400
Organization: A noiseless patient Spider
Lines: 3
Message-ID: <ydy1fhe6qy.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
<uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com>
<uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="23b89ea94d7e807aa558c5a93e84bc7b";
logging-data="1807554"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+LqhkoPrN063iFK2NNp2f2"
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.2 (berkeley-unix)
Cancel-Lock: sha1:fDaou2ThtjoaFswx4Rd2EOCmni8=
sha1:GTHsSdRYbTWGushct7HtMhJ8yXc=
Mail-Copies-To: never
 by: Winston - Wed, 1 Nov 2023 17:13 UTC

Also in the "in case it's a hardware problem" vein,
maybe also record voltages if you're able to.
-WBE

Re: Hard crashes and swap issues

<ui5r2a$1r3$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=449&group=comp.unix.bsd.freebsd.misc#449

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.12.144.5.2!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sat, 4 Nov 2023 16:21:30 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <ui5r2a$1r3$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com> <uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com>
Injection-Date: Sat, 4 Nov 2023 16:21:30 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="12.144.5.2";
logging-data="1891"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Sat, 4 Nov 2023 16:21 UTC

Winston <wbe@ubeblock.psr.com.invalid> wrote:
> As part of the thread, I asked:
>>> Did you ever try the experiment of disabling tmpfs to see if
>>> that helps?
>
> to which Louis Epstein <le@main.lekno.ws> responded:
>> Given that using tmpfs is supposed to speed things up and
>> the synth builds already take hours and hours when the
>> batch is large and the swap-swallows prevent using the
>> computer for anything else,I was cautious about this.
>
>> Is there a way to dismount the /tmpfs entries of df
>> that are there when I kill synth before the swap-swallow
>> has crashed the system, so I know if they play a part in
>> the failure of the shutdown program to execute completely?
>> shutdown -h and shutdown -p both end up in a circle.
>
> I'm not confident I understand what you're asking, but
> perhaps one or more of the following address your question.
>
> While I would think that disabling tmpfs is the only
> definitive way to test whether it's a tmpfs bug, if you
> don't wish to disable tmpfs:
>
> * You could run pstat or swapinfo from time to see how much
> swap space is in use.

The "dashboard" on synth includes a steadily updating
percentage of how much swap space is in use...I also
run top on another terminal session at times to see
the figure there.

> * If you have >1 swap area, you could consider suspending
> the build and then using swapoff to remove one of them.

I don't think synth takes an interrupt.

> It'll take a while to move the active content to the
> remaining swap space. Then use swapon to re-add it.
> Hopefully, tmpfs handles swap space addition and removal
> well.
>
> See if the reported swap space used changes significantly.
> If so, maybe it's a tmpfs bug (not freeing no-longer-used
> swap space).
>
> * protect(1) can be use to keep processes from being killed
> when swap space runs out, but, as the man page warns,
> that can also lead to the system hanging instead of
> crashing.

But would this solve what makes shutdown unable to complete
after I have killed synth between errors but before the
whole system has gone down?

> * Maybe check temperatures from time to time while the
> build is running, too.
>
> * Periodic logging: maybe write a trivial shell loop to run
> every few seconds or so to write the output of swapinfo,
> df -h /tmp, temperatures, etc. to a file which you could
> examine after rebooting.
>
> If swap space doesn't grow gradually, it starts to look
> more like a bit error or really-hard-to-find bug.
>
> In the case of bug 219399, small increases in voltages
> reduced, but didn't eliminate, bit errors caused by heat.
>
> HTH,
> -WBE

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<ujhegi$q6n$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=451&group=comp.unix.bsd.freebsd.misc#451

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.main.lekno.ws!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Tue, 21 Nov 2023 05:17:06 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <ujhegi$q6n$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com> <uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com> <ui5r2a$1r3$1@reader2.panix.com>
Injection-Date: Tue, 21 Nov 2023 05:17:06 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="main.lekno.ws:12.144.5.2";
logging-data="26839"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Tue, 21 Nov 2023 05:17 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> Winston <wbe@ubeblock.psr.com.invalid> wrote:
>> As part of the thread, I asked:
>>>> Did you ever try the experiment of disabling tmpfs to see if
>>>> that helps?
>>
>> to which Louis Epstein <le@main.lekno.ws> responded:
>>> Given that using tmpfs is supposed to speed things up and
>>> the synth builds already take hours and hours when the
>>> batch is large and the swap-swallows prevent using the
>>> computer for anything else,I was cautious about this.
>>
>>> Is there a way to dismount the /tmpfs entries of df
>>> that are there when I kill synth before the swap-swallow
>>> has crashed the system, so I know if they play a part in
>>> the failure of the shutdown program to execute completely?
>>> shutdown -h and shutdown -p both end up in a circle.
>>
>> I'm not confident I understand what you're asking, but
>> perhaps one or more of the following address your question.
>>
>> While I would think that disabling tmpfs is the only
>> definitive way to test whether it's a tmpfs bug, if you
>> don't wish to disable tmpfs:
>>
>> * You could run pstat or swapinfo from time to see how much
>> swap space is in use.
>
> The "dashboard" on synth includes a steadily updating
> percentage of how much swap space is in use...I also
> run top on another terminal session at times to see
> the figure there.
>
>> * If you have >1 swap area, you could consider suspending
>> the build and then using swapoff to remove one of them.
>
> I don't think synth takes an interrupt.
>
>> It'll take a while to move the active content to the
>> remaining swap space. Then use swapon to re-add it.
>> Hopefully, tmpfs handles swap space addition and removal
>> well.
>>
>> See if the reported swap space used changes significantly.
>> If so, maybe it's a tmpfs bug (not freeing no-longer-used
>> swap space).
>>
>> * protect(1) can be use to keep processes from being killed
>> when swap space runs out, but, as the man page warns,
>> that can also lead to the system hanging instead of
>> crashing.
>
> But would this solve what makes shutdown unable to complete
> after I have killed synth between errors but before the
> whole system has gone down?

Further experimentation has revealed that while synth is still
running before the system crashes,doing a kill -9 rather than
plain kill saves the shutdown program from becoming unable to
complete,
and doing umount -a before a shutdown after it HAS crashed
doesn't save the shutdown program or remove the /tmpfs entries
from df,but it DOES avoid all the fsck'ing of the drives after
the power-cycling.
>> * Maybe check temperatures from time to time while the
>> build is running, too.
>>
>> * Periodic logging: maybe write a trivial shell loop to run
>> every few seconds or so to write the output of swapinfo,
>> df -h /tmp, temperatures, etc. to a file which you could
>> examine after rebooting.
>>
>> If swap space doesn't grow gradually, it starts to look
>> more like a bit error or really-hard-to-find bug.
>>
>> In the case of bug 219399, small increases in voltages
>> reduced, but didn't eliminate, bit errors caused by heat.
>>
>> HTH,
>> -WBE
>
> -=-=-
> The World Trade Center towers MUST rise again,
> at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<ydttp9jzae.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=452&group=comp.unix.bsd.freebsd.misc#452

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Sat, 25 Nov 2023 16:31:05 -0500
Organization: A noiseless patient Spider
Lines: 67
Message-ID: <ydttp9jzae.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
<uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com>
<uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com>
<ui5r2a$1r3$1@reader2.panix.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="ab3ff0b6d16beee1b40df6152cc8cf06";
logging-data="3102368"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1+itG088NMcAD0lRl5whwsC"
User-Agent: Gnus/5.13 (Gnus v5.13)
Cancel-Lock: sha1:qcFWTK1MhoZPErNWblnmD9rGodc=
sha1:QEMKnCFRKmyIUOurDpmeqxpiWRw=
Mail-Copies-To: never
 by: Winston - Sat, 25 Nov 2023 21:31 UTC

I'm still mostly just taking stabs in the dark, but here goes ...

As part of this thread, I previously suggested:
>> * You could run pstat or swapinfo from time to see how much
>> swap space is in use.

to which Louis Epstein <le@main.lekno.ws> replied:
> The "dashboard" on synth includes a steadily updating
> percentage of how much swap space is in use...I also
> run top on another terminal session at times to see
> the figure there.

I misread that for a long time. I read "steadily decreasing"
rather than what you actually wrote: steadily updating.

So, is swap space decreasing as the build progresses? If so,
does df /tmp steadily increase to match? If it does, check
what's filling up /tmp(fs).

Louis followed up what I quoted above by adding:
> Further experimentation has revealed that while synth is
> still running before the system crashes, doing a kill -9
> rather than plain kill saves the shutdown program from
> becoming unable to complete, and doing umount -a before a
> shutdown after it HAS crashed doesn't save the shutdown
> program or remove the /tmpfs entries from df,but it DOES
> avoid all the fsck'ing of the drives after the power-cycling.

Since part of shutting down is syncing the vnodes, if it
doesn't get that far, it's not surprising that a fsck could be
needed. That's probably why the umount helps. I'm a little
surprised that umount /tmp(fs) wouldn't remove the df entry,
though.

After rebooting, can you resume the build from the point of the
crash (and does it complete?), or do you start over?

I note that /sbin/shutdown runs /usr/bin/wall and execs
/sbin/reboot, /sbin/halt, etc. In those cases where your
system is out of swap space, that may contribute to shutdown
not working, ... Even a subroutine call that needs heap space
and can't get it could cause trouble.

But, several articles ago, you wrote:
>>>> And killing the rust build when 7% of swap space is in
>>>> use still leads to an unable to get swap space message
>>>> just as killing it when 99.9% of swap space is in use
>>>> before the shutdown.

So then it's not strictly an out-of-swap-space issue.

This from the "man tmpfs" might be relevant:

Metadata, including the directory content, is never
swapped out by the current implementation. Keep this
in mind when planning the mount limits, especially when
expecting to place many small files on a tmpfs mount.

I can imagine a case where a smaller RAM is full from the
combination of locked memory, active processes, and lots of
tmpfs file metadata, even when there's unused swap space. As
such a situation develops, I would think there'd be a *lot* of
swapping of whichever processes are still swappable, making
things really slow.

Trying to help, but not confident I'm succeeding, :)
-WBE

Re: Hard crashes and swap issues

<ukl0mc$sf2$1@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=470&group=comp.unix.bsd.freebsd.misc#470

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.main.lekno.ws!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Mon, 4 Dec 2023 17:02:04 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <ukl0mc$sf2$1@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com> <uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com> <ui5r2a$1r3$1@reader2.panix.com> <ydttp9jzae.fsf@UBEblock.psr.com>
Injection-Date: Mon, 4 Dec 2023 17:02:04 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="main.lekno.ws:12.144.5.2";
logging-data="29154"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Mon, 4 Dec 2023 17:02 UTC

Winston <wbe@ubeblock.psr.com.invalid> wrote:
> I'm still mostly just taking stabs in the dark, but here goes ...
>
> As part of this thread, I previously suggested:
>>> * You could run pstat or swapinfo from time to see how much
>>> swap space is in use.
>
> to which Louis Epstein <le@main.lekno.ws> replied:
>> The "dashboard" on synth includes a steadily updating
>> percentage of how much swap space is in use...I also
>> run top on another terminal session at times to see
>> the figure there.
>
> I misread that for a long time. I read "steadily decreasing"
> rather than what you actually wrote: steadily updating.
>
> So, is swap space decreasing as the build progresses? If so,
> does df /tmp steadily increase to match? If it does, check
> what's filling up /tmp(fs).

Not quite linearly but it was ultimately reaching 99.9% swap
in use before the crashes.

The df has shown tons of things in tmpfs.

I just got rust to build overnight after increasing the
swap space on the machine.
> Louis followed up what I quoted above by adding:
>> Further experimentation has revealed that while synth is
>> still running before the system crashes, doing a kill -9
>> rather than plain kill saves the shutdown program from
>> becoming unable to complete, and doing umount -a before a
>> shutdown after it HAS crashed doesn't save the shutdown
>> program or remove the /tmpfs entries from df,but it DOES
>> avoid all the fsck'ing of the drives after the power-cycling.
>
> Since part of shutting down is syncing the vnodes, if it
> doesn't get that far, it's not surprising that a fsck could be
> needed. That's probably why the umount helps. I'm a little
> surprised that umount /tmp(fs) wouldn't remove the df entry,
> though.

I had not tried that but umount -a removes the filesystems
in etc/fstab which do not include /tmpfs.

> After rebooting, can you resume the build from the point of the
> crash (and does it complete?), or do you start over?
>
> I note that /sbin/shutdown runs /usr/bin/wall and execs
> /sbin/reboot, /sbin/halt, etc. In those cases where your
> system is out of swap space, that may contribute to shutdown
> not working, ... Even a subroutine call that needs heap space
> and can't get it could cause trouble.
>
> But, several articles ago, you wrote:
>>>>> And killing the rust build when 7% of swap space is in
>>>>> use still leads to an unable to get swap space message
>>>>> just as killing it when 99.9% of swap space is in use
>>>>> before the shutdown.
>
> So then it's not strictly an out-of-swap-space issue.
>
> This from the "man tmpfs" might be relevant:
>
> Metadata, including the directory content, is never
> swapped out by the current implementation. Keep this
> in mind when planning the mount limits, especially when
> expecting to place many small files on a tmpfs mount.
>
> I can imagine a case where a smaller RAM is full from the
> combination of locked memory, active processes, and lots of
> tmpfs file metadata, even when there's unused swap space. As
> such a situation develops, I would think there'd be a *lot* of
> swapping of whichever processes are still swappable, making
> things really slow.
>
> Trying to help, but not confident I'm succeeding, :)
> -WBE

So far the increased swap space has done the immediate need,
get lang/rust to complete the build process.

I'm now doing a general synth prepare-system to see if it
follows through and builds all the things that would always
wait on rust to build.

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<ydttoxp2iz.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=471&group=comp.unix.bsd.freebsd.misc#471

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Tue, 05 Dec 2023 01:49:24 -0500
Organization: A noiseless patient Spider
Lines: 11
Message-ID: <ydttoxp2iz.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
<uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com>
<uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com>
<ui5r2a$1r3$1@reader2.panix.com> <ydttp9jzae.fsf@UBEblock.psr.com>
<ukl0mc$sf2$1@reader2.panix.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="a549d690420595ef025d8f985b39be12";
logging-data="4024455"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1/E36gX4du3uc74dchvImrg"
User-Agent: Gnus/5.13 (Gnus v5.13)
Cancel-Lock: sha1:A8XoNBYQM6Fei5rwZs+YXMFn48Y=
sha1:5uZWG/YpL18dXSITN6Ze85OOkfs=
Mail-Copies-To: never
 by: Winston - Tue, 5 Dec 2023 06:49 UTC

Louis Epstein <le@main.lekno.ws> wrote:
> I just got rust to build overnight after increasing the
> swap space on the machine.

> So far the increased swap space has done the immediate need,
> get lang/rust to complete the build process.

So, just to wrap things up ... Once the build was finished,
the swap space all reappeared or was accounted for by tmpfs
files?
-WBE

Re: Hard crashes and swap issues

<ulqp84$pcf$3@reader2.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=490&group=comp.unix.bsd.freebsd.misc#490

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!news.neodome.net!weretis.net!feeder6.news.weretis.net!panix!.POSTED.main.lekno.ws!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Tue, 19 Dec 2023 00:48:04 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <ulqp84$pcf$3@reader2.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com> <uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com> <ui5r2a$1r3$1@reader2.panix.com> <ydttp9jzae.fsf@UBEblock.psr.com> <ukl0mc$sf2$1@reader2.panix.com> <ydttoxp2iz.fsf@UBEblock.psr.com>
Injection-Date: Tue, 19 Dec 2023 00:48:04 -0000 (UTC)
Injection-Info: reader2.panix.com; posting-host="main.lekno.ws:12.144.5.2";
logging-data="25999"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p4 (amd64))
 by: Louis Epstein - Tue, 19 Dec 2023 00:48 UTC

Winston <wbe@ubeblock.psr.com.invalid> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>> I just got rust to build overnight after increasing the
>> swap space on the machine.
>
>> So far the increased swap space has done the immediate need,
>> get lang/rust to complete the build process.
>
> So, just to wrap things up ... Once the build was finished,
> the swap space all reappeared or was accounted for by tmpfs
> files?
> -WBE

Synth runs now continue past the building of rust,
and complete the rebuilding of the local repository.

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

Re: Hard crashes and swap issues

<yd5y0uw8th.fsf@UBEblock.psr.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=491&group=comp.unix.bsd.freebsd.misc#491

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!eternal-september.org!feeder3.eternal-september.org!news.eternal-september.org!.POSTED!not-for-mail
From: wbe...@UBEBLOCK.psr.com.invalid (Winston)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Tue, 19 Dec 2023 09:44:42 -0500
Organization: A noiseless patient Spider
Lines: 20
Message-ID: <yd5y0uw8th.fsf@UBEblock.psr.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com>
<uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com>
<uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com>
<uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com>
<ui5r2a$1r3$1@reader2.panix.com> <ydttp9jzae.fsf@UBEblock.psr.com>
<ukl0mc$sf2$1@reader2.panix.com> <ydttoxp2iz.fsf@UBEblock.psr.com>
<ulqp84$pcf$3@reader2.panix.com>
MIME-Version: 1.0
Content-Type: text/plain
Injection-Info: dont-email.me; posting-host="a87e9d6022e8c4af3763fe82a2dde46e";
logging-data="97483"; mail-complaints-to="abuse@eternal-september.org"; posting-account="U2FsdGVkX1826OCgLERrVj/CqDoDF1OD"
User-Agent: Gnus/5.13 (Gnus v5.13)
Cancel-Lock: sha1:yBzqy7/ZTkx6YAvalvCOpR4cEao=
sha1:Uj0LlIQisA27ZCKTsXxKj//R2H4=
Mail-Copies-To: never
 by: Winston - Tue, 19 Dec 2023 14:44 UTC

Louis Epstein <le@main.lekno.ws> wrote:
>>> I just got rust to build overnight after increasing the
>>> swap space on the machine.

>>> So far the increased swap space has done the immediate need,
>>> get lang/rust to complete the build process.

prompting me to ask:
>> So, just to wrap things up ... Once the build was finished,
>> the swap space all reappeared or was accounted for by tmpfs
>> files?

Louis Epstein <le@main.lekno.ws> replied:
> Synth runs now continue past the building of rust,
> and complete the rebuilding of the local repository.

OK, but my question (above) was more about whether you saw
swap + tmpfs space lost along the way (bug), or was
swap + tmpfs space all back to normal afterward (no bug)?
-WBE

Re: Hard crashes and swap issues

<umbdmp$oig$2@reader1.panix.com>

 copy mid

https://www.novabbs.com/devel/article-flat.php?id=519&group=comp.unix.bsd.freebsd.misc#519

 copy link   Newsgroups: comp.unix.bsd.freebsd.misc
Path: i2pn2.org!i2pn.org!weretis.net!feeder6.news.weretis.net!panix!.POSTED.main.lekno.ws!not-for-mail
From: le...@main.lekno.ws (Louis Epstein)
Newsgroups: comp.unix.bsd.freebsd.misc
Subject: Re: Hard crashes and swap issues
Date: Mon, 25 Dec 2023 08:15:21 -0000 (UTC)
Organization: PANIX Public Access Internet and UNIX, NYC
Message-ID: <umbdmp$oig$2@reader1.panix.com>
References: <ubuiqq$4dq$1@reader2.panix.com> <uc2tfs$gbo$1@reader2.panix.com> <uc5iad$5q8$1@reader2.panix.com> <ydfs49f02v.fsf@UBEblock.psr.com> <uelvmi$68p$2@reader2.panix.com> <yd7cn4fbgd.fsf@UBEblock.psr.com> <uhrb8k$56i$1@reader2.panix.com> <yd34xpfsdq.fsf@UBEblock.psr.com> <ui5r2a$1r3$1@reader2.panix.com> <ydttp9jzae.fsf@UBEblock.psr.com> <ukl0mc$sf2$1@reader2.panix.com> <ydttoxp2iz.fsf@UBEblock.psr.com> <ulqp84$pcf$3@reader2.panix.com> <yd5y0uw8th.fsf@UBEblock.psr.com>
Injection-Date: Mon, 25 Dec 2023 08:15:21 -0000 (UTC)
Injection-Info: reader1.panix.com; posting-host="main.lekno.ws:12.144.5.2";
logging-data="25168"; mail-complaints-to="abuse@panix.com"
User-Agent: tin/2.6.2-20221225 ("Pittyvaich") (FreeBSD/13.2-RELEASE-p8 (amd64))
 by: Louis Epstein - Mon, 25 Dec 2023 08:15 UTC

Winston <wbe@ubeblock.psr.com.invalid> wrote:
> Louis Epstein <le@main.lekno.ws> wrote:
>>>> I just got rust to build overnight after increasing the
>>>> swap space on the machine.
>
>>>> So far the increased swap space has done the immediate need,
>>>> get lang/rust to complete the build process.
>
> prompting me to ask:
>>> So, just to wrap things up ... Once the build was finished,
>>> the swap space all reappeared or was accounted for by tmpfs
>>> files?
>
> Louis Epstein <le@main.lekno.ws> replied:
>> Synth runs now continue past the building of rust,
>> and complete the rebuilding of the local repository.
>
> OK, but my question (above) was more about whether you saw
> swap + tmpfs space lost along the way (bug), or was
> swap + tmpfs space all back to normal afterward (no bug)?
> -WBE

I don't think I checked the tmpfs closely enough to answer.

-=-=-
The World Trade Center towers MUST rise again,
at least as tall as before...or terror has triumphed.

1
server_pubkey.txt

rocksolid light 0.9.7
clearnet tor