I regularly back up my work files to an external hard disk using rsync. It's worked well for the six months since I moved to Linux Mint. Just in the last couple of days, though, the rsync routine produces errors such as

rsync: [receiver] rename "/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/.vleads.msf.h9nPNm" -> "Mail/Releases.sbd/vleads.msf": No such file or directory (2)

Most of the path name above (/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/) is correct in that there really is a directory called that on my source disk. And there is a file in the corresponding directory on the destination disk (i.e. the backup disk) called vleads.msf. However that file on the backup disk is corrupt in some way. I can't delete it, rename it or move it, even as root; I get the "No such file or directory" error message. However "ls" finds it, and the properties->permissions function says it is read-write, and I can open it with a text editor (even though what's inside it is a jpeg file, which it shouldn't be).

Seems to me that its existence, and its corrupt nature, is somehow buggering up rsync. But I can't see how to get rid of it.

On 11/16/23 1:01 PM, Handsome Jack wrote:
> I regularly back up my work files to an external hard disk using rsync. It's worked well for the six months since I moved to Linux Mint. Just in the last couple of days, though, the rsync routine produces errors such as
>
> rsync: [receiver] rename "/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/.vleads.msf.h9nPNm" -> "Mail/Releases.sbd/vleads.msf": No such file or directory (2)
>
>
> Most of the path name above (/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/) is correct in that there really is a directory called that on my source disk. And there is a file in the corresponding directory on the destination disk (i.e. the backup disk) called vleads.msf. However that file on the backup disk is corrupt in some way. I can't delete it, rename it or move it, even as root; I get the "No such file or directory" error message. However "ls" finds it, and the properties->permissions function says it is read-write, and I can open it with a text editor (even though what's inside it is a jpeg file, which it shouldn't be).
>
> Seems to me that its existence, and its corrupt nature, is somehow buggering up rsync. But I can't see how to get rid of it.
>
Have you checked if the usb disk is full?

Nic <Nic@none.net> wrote:
> On 11/16/23 1:01 PM, Handsome Jack wrote:
>> I regularly back up my work files to an external hard disk using rsync. It's worked well for the six months since I moved to Linux Mint. Just in the last couple of days, though, the rsync routine produces errors such as
>>
>> rsync: [receiver] rename "/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/.vleads.msf.h9nPNm" -> "Mail/Releases.sbd/vleads.msf": No such file or directory (2)
>>
>>
>> Most of the path name above (/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/) is correct in that there really is a directory called that on my source disk. And there is a file in the corresponding directory on the destination disk (i.e. the backup disk) called vleads.msf. However that file on the backup disk is corrupt in some way. I can't delete it, rename it or move it, even as root; I get the "No such file or directory" error message. However "ls" finds it, and the properties->permissions function says it is read-write, and I can open it with a text editor (even though what's inside it is a jpeg file, which it shouldn't be).
>>
>> Seems to me that its existence, and its corrupt nature, is somehow buggering up rsync. But I can't see how to get rid of it.
>>
> Have you checked if the usb disk is full?

It's not full.

On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
> Nic <Nic@none.net> wrote:
<snip>
>>> Seems to me that its existence, and its corrupt nature, is
>>> somehow buggering up rsync. But I can't see how to get rid of it.
>>>
>> Have you checked if the usb disk is full?
>
> It's not full.
>

I would umount the external drive and run e2fsck on the partition/
filesystem to make sure all is good with the file system.
Determine the device node the partition/filesystem resides on by running:
fdisk -l
from a root term
then run
e2fsck -vfy /dev/sdXx
from a root term.
Replace sdXx with the actual device node found via the
output of fdisk -l.

Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>
> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>> Nic <Nic@none.net> wrote:
> <snip>
>>>> Seems to me that its existence, and its corrupt nature, is
>>>> somehow buggering up rsync. But I can't see how to get rid of it.
>>>>
>>> Have you checked if the usb disk is full?
>>
>> It's not full.
>>
>
> I would umount the external drive and run e2fsck on the partition/
> filesystem to make sure all is good with the file system.
> Determine the device node the partition/filesystem resides on by running:
> fdisk -l
> from a root term
> then run
> e2fsck -vfy /dev/sdXx
> from a root term.
> Replace sdXx with the actual device node found via the
> output of fdisk -l.

OK thanks. The result of { e2fsck -vfy /dev/sdf1 } was
{ e2fsck 1.46.5 (30-Dec-2021)
ext2fs_open2: Bad magic number in super-block
e2fsck: Superblock invalid, trying backup blocks...
e2fsck: Bad magic number in super-block while trying to open /dev/sdf1

The superblock could not be read or does not describe a valid ext2/ext3/ext4
filesystem. If the device is valid and it really contains an ext2/ext3/ext4
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate superblock:
e2fsck -b 8193 <device>
or
e2fsck -b 32768 <device>
}

I tried doing both the latter, and got the same "super-block" error.

BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT

On 2023-11-16, Handsome Jack wrote:
> I regularly back up my work files to an external hard disk using
> rsync. It's worked well for the six months since I moved to Linux
> Mint. Just in the last couple of days, though, the rsync routine
> produces errors such as
>
> rsync: [receiver] rename
> "/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/.vleads.msf.h9nPNm"
> -> "Mail/Releases.sbd/vleads.msf": No such file or directory (2)
>
>
> Most of the path name above
> (/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/) is
> correct in that there really is a directory called that on my source
> disk. And there is a file in the corresponding directory on the
> destination disk (i.e. the backup disk) called vleads.msf. However
> that file on the backup disk is corrupt in some way. I can't delete
> it, rename it or move it, even as root; I get the "No such file or
> directory" error message. However "ls" finds it, and the
> properties->permissions function says it is read-write, and I can open
> it with a text editor (even though what's inside it is a jpeg file,
> which it shouldn't be).

Then the file is not actually "vleasds.msf" perhaps there's a hidden
special character (newline, etc) . Are you using tab-completion so that
the shell grabs everything?

--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

On 17-11-2023 11:28, Dan Purgert wrote:
> On 2023-11-16, Handsome Jack wrote:
>> I regularly back up my work files to an external hard disk using
>> rsync. It's worked well for the six months since I moved to Linux
>> Mint. Just in the last couple of days, though, the rsync routine
>> produces errors such as
>>
>> rsync: [receiver] rename
>> "/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/.vleads.msf.h9nPNm"
>> -> "Mail/Releases.sbd/vleads.msf": No such file or directory (2)
>>
>>
>> Most of the path name above
>> (/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/) is
>> correct in that there really is a directory called that on my source
>> disk. And there is a file in the corresponding directory on the
>> destination disk (i.e. the backup disk) called vleads.msf. However
>> that file on the backup disk is corrupt in some way. I can't delete
>> it, rename it or move it, even as root; I get the "No such file or
>> directory" error message. However "ls" finds it, and the
>> properties->permissions function says it is read-write, and I can open
>> it with a text editor (even though what's inside it is a jpeg file,
>> which it shouldn't be).
>
> Then the file is not actually "vleasds.msf" perhaps there's a hidden
> special character (newline, etc) . Are you using tab-completion so that
> the shell grabs everything?
>
>
Or a unicode character which looks exactly like you would expect, but
still would
be regarded as different by the system.

On 17/11/2023 06:09, Handsome Jack wrote:
> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>
>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>> Nic <Nic@none.net> wrote:
>> <snip>
>>>>> Seems to me that its existence, and its corrupt nature, is
>>>>> somehow buggering up rsync. But I can't see how to get rid of it.
>>>>>
>>>> Have you checked if the usb disk is full?
>>>
>>> It's not full.
>>>
>>
>> I would umount the external drive and run e2fsck on the partition/
>> filesystem to make sure all is good with the file system.
>> Determine the device node the partition/filesystem resides on by running:
>> fdisk -l
>> from a root term
>> then run
>> e2fsck -vfy /dev/sdXx
>> from a root term.
>> Replace sdXx with the actual device node found via the
>> output of fdisk -l.
>
> OK thanks. The result of { e2fsck -vfy /dev/sdf1 } was
> {
> e2fsck 1.46.5 (30-Dec-2021)
> ext2fs_open2: Bad magic number in super-block
> e2fsck: Superblock invalid, trying backup blocks...
> e2fsck: Bad magic number in super-block while trying to open /dev/sdf1
>
> The superblock could not be read or does not describe a valid ext2/ext3/ext4
> filesystem. If the device is valid and it really contains an ext2/ext3/ext4
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt, and you might try running e2fsck with an alternate superblock:
> e2fsck -b 8193 <device>
> or
> e2fsck -b 32768 <device>
> }
>
> I tried doing both the latter, and got the same "super-block" error.
>
> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>
>

Really?!!!
That's what you get for blindly following instructions on the internet.
Exactly what incarnation of rsync are you using?

--
Chris Elvidge, England
I WILL NOT DRAW NAKED LADIES IN CLASS

Chris Elvidge <chris@mshome.net> wrote:
> On 17/11/2023 06:09, Handsome Jack wrote:

>> OK thanks. The result of { e2fsck -vfy /dev/sdf1 } was
>> {
>> e2fsck 1.46.5 (30-Dec-2021)
>> ext2fs_open2: Bad magic number in super-block
>> e2fsck: Superblock invalid, trying backup blocks...
>> e2fsck: Bad magic number in super-block while trying to open /dev/sdf1
>>
>> The superblock could not be read or does not describe a valid ext2/ext3/ext4
>> filesystem. If the device is valid and it really contains an ext2/ext3/ext4
>> filesystem (and not swap or ufs or something else), then the superblock
>> is corrupt, and you might try running e2fsck with an alternate superblock:
>> e2fsck -b 8193 <device>
>> or
>> e2fsck -b 32768 <device>
>> }
>>
>> I tried doing both the latter, and got the same "super-block" error.
>>
>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>
>>
>
> Really?!!!
> That's what you get for blindly following instructions on the internet.

Is it bad? How long have I got, Doc?

> Exactly what incarnation of rsync are you using?

rsync version 3.2.7 protocol version 31. Worked OK for the past six months.

On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>
>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
<snip>
> I tried doing both the latter, and got the same "super-block" error.
>
> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>
<snip>

If it is a NTFS partition/file-system you would use your Windows
chkdsk program to check/repair the NTFS file system.

What is the source file system? Another NTFS file system mounted using 3g-ntfs
via Linux or is it a Linux file system where the source files reside?
What type of file system is the source?
What file system type does the file reside on that you can not delete?

On 11/16/2023 1:01 PM, Handsome Jack wrote:
> I regularly back up my work files to an external hard disk using rsync. It's worked well for the six months since I moved to Linux Mint. Just in the last couple of days, though, the rsync routine produces errors such as
>
> rsync: [receiver] rename "/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/.vleads.msf.h9nPNm" -> "Mail/Releases.sbd/vleads.msf": No such file or directory (2)
>
>
> Most of the path name above (/media/jack/Backups/LatestBackup/Archive/Mail/Releases.sbd/) is correct in that there really is a directory called that on my source disk. And there is a file in the corresponding directory on the destination disk (i.e. the backup disk) called vleads.msf. However that file on the backup disk is corrupt in some way. I can't delete it, rename it or move it, even as root; I get the "No such file or directory" error message. However "ls" finds it, and the properties->permissions function says it is read-write, and I can open it with a text editor (even though what's inside it is a jpeg file, which it shouldn't be).
>
> Seems to me that its existence, and its corrupt nature, is somehow buggering up rsync. But I can't see how to get rid of it.
>

This appears to be the alternate representation of info in Thunderbird.
Namely, to store messages as separate files, rather than to store them
in an MBOX format.

A lot of people here, would have Inbox and Inbox.msf, which is a raw MBOX
plus a Mork file which serves as the "generated" index file for the MBOX.

An item with .sbd appears to be a SuBDirectory, so you may find a whole
bunch of separate messages in it. However, a file at the same level, may
be a Mork .msf file, which is an index Thunderbird generates.

https://askubuntu.com/questions/1319671/thunderbird-sbd-files

If you had a folder with just an Inbox in it, Thunderbird would
make an Inbox.msf by scanning the Inbox file. If you were to Import content
from an older Thunderbird to a newer Thunderbird, sure, the Inbox might
be copied, but the .msf, if it was missing, could be generated on the fly.
Presumably, something similar happens with a .sbf representation.

*******

Now, the next issue, is Thunderbird performance.

Let's say a user has 20GB of email, collected from 40 years of working
at the same company.

If Thunderbird opened that pile of shit, plus generated or manipulated
..msf files all at once, it would need a fuck-ton of RAM to do that.

If you looked in the Bugzilla long ago, there were discussions about
"well, what should we do". They settled on a dynamic method, of
keeping some boxes open, and the rest remained at rest.

Once there is no activity in a box, for some number of seconds (the
user has left the room), the box is "put away". The footprint drops
as time passes.

This is an attempt, to give "RAM-like speeds" while working on a box,
without using all the memory in the computer. It is a compromise.

*******

Thunderbird has been (partially) re-written. Are boxes handled the same way as before ?
Could they be using MMAP ? What would the consequences of the new mechanism be ?

I don't know the answers to these questions. Am I going to open the
source tarball and find out ? NO :-)

Would there be shards left in the file system, for RSync to trip over ???

From this, we learn a general principle.

"Backup works best, for data at rest"

If the OS is not running, if the applications are not open, guess what
happens. No funny business, that's what.

*******

Windows has a mechanism, VSS, which is a means of preparing a partition
for backup. This works pretty well... but it misses stuff. Any program
which refuses to honor a request to quiesce, the files are in an indeterminate
state. The File Indexer is an example of a culprit. Macrium, for example,
does not even attempt to back up Windows.edb (File Indexer), because it is
known to be "open". For most of the rest of the computer contents, you
can back up most everything.

Now, certain Linux file systems, also possess a useful capability. I'm
not knowledgeable about these things, but it is possible ZFS can make
an "instantaneous snapshot" which is very similar to how VSS makes a snapshot
(in terms of semantics). I think at least one or two people here, know
how to use those, and capture the output for later.

But for the rest of us, booting a LiveDVD and running our rsync, might
be safer, until more adventurous means are proved out.

*******

Summary: Sure, you can run fsck if you want, but... I don't think that's it.
Try closing the Thunderbird window, run a ps command and verify it
is really exited, now run your Rsync. What happens ? Good copy ?

I don't know if Thunderbird uses any services in the background,
so do a thorough sweep so the .thunderbird folder is at rest.
As far as I know, once the main process exits, the data is at rest.

It could be, that exiting most all of your desktop programs, is
the best set of conditions for your Rsync.

But for a "guaranteed" result, boot a Linux LiveDVD and rsync from there.

Paul

On 17/11/2023 16:46, Handsome Jack wrote:
> Chris Elvidge <chris@mshome.net> wrote:
>> On 17/11/2023 06:09, Handsome Jack wrote:
>
>>> OK thanks. The result of { e2fsck -vfy /dev/sdf1 } was
>>> {
>>> e2fsck 1.46.5 (30-Dec-2021)
>>> ext2fs_open2: Bad magic number in super-block
>>> e2fsck: Superblock invalid, trying backup blocks...
>>> e2fsck: Bad magic number in super-block while trying to open /dev/sdf1
>>>
>>> The superblock could not be read or does not describe a valid ext2/ext3/ext4
>>> filesystem. If the device is valid and it really contains an ext2/ext3/ext4
>>> filesystem (and not swap or ufs or something else), then the superblock
>>> is corrupt, and you might try running e2fsck with an alternate superblock:
>>> e2fsck -b 8193 <device>
>>> or
>>> e2fsck -b 32768 <device>
>>> }
>>>
>>> I tried doing both the latter, and got the same "super-block" error.
>>>
>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>
>>>
>>
>> Really?!!!
>> That's what you get for blindly following instructions on the internet.
>
> Is it bad? How long have I got, Doc?

Didn't the name of the program give you an inkling? e2fsck = check an
ext{2,3,4} filesystem (from man e2fsck)

>
>> Exactly what incarnation of rsync are you using?
>
> rsync version 3.2.7 protocol version 31. Worked OK for the past six months.
>

No. What's the command line you used? Switches, options etc.

>

Can you successfully do the operation with an ext{2,3,4} device as the
target? If so, it's a mismatch of what filename(s) are allowed on
NTFS/exFAT.

--
Chris Elvidge, England
LOOSE TEETH DON'T NEED MY HELP

Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>
> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>
>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
> <snip>
>> I tried doing both the latter, and got the same "super-block" error.
>>
>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>
> <snip>
>
> If it is a NTFS partition/file-system you would use your Windows
> chkdsk program to check/repair the NTFS file system.

Thanks, this seems to have solved it. Luckily I still have access to a Windows machine into which I could plug this NTFS removable disk.

What would I have done if I hadn't? Is there not an equivalent of chkdsk that runs on Linux to fix NTFS disks?

>
> What is the source file system? Another NTFS file system mounted using 3g-ntfs
> via Linux or is it a Linux file system where the source files reside?

Linux.

> What file system type does the file reside on that you can not delete?
>

NTFS. Deleted them now.

On 2023-11-19, Handsome Jack wrote:
> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>
>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>
>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>> <snip>
>>> I tried doing both the latter, and got the same "super-block" error.
>>>
>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>
>> <snip>
>>
>> If it is a NTFS partition/file-system you would use your Windows
>> chkdsk program to check/repair the NTFS file system.
>
>
> Thanks, this seems to have solved it. Luckily I still have access to a
> Windows machine into which I could plug this NTFS removable disk.
>
> What would I have done if I hadn't? Is there not an equivalent of
> chkdsk that runs on Linux to fix NTFS disks?

Not really, the linux equivalent just resets the NTFS dirty bit. Best
to not use NTFS in a linux-only environment.

--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

On 11/19/23 7:58 AM, Dan Purgert wrote:
> On 2023-11-19, Handsome Jack wrote:
>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>> <snip>
>>>> I tried doing both the latter, and got the same "super-block" error.
>>>>
>>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>>
>>> <snip>
>>>
>>> If it is a NTFS partition/file-system you would use your Windows
>>> chkdsk program to check/repair the NTFS file system.
>>
>> Thanks, this seems to have solved it. Luckily I still have access to a
>> Windows machine into which I could plug this NTFS removable disk.
>>
>> What would I have done if I hadn't? Is there not an equivalent of
>> chkdsk that runs on Linux to fix NTFS disks?
> Not really, the linux equivalent just resets the NTFS dirty bit. Best
> to not use NTFS in a linux-only environment.
>
>
What would you format the storage drive to?

On 2023-11-19, Nic wrote:
> On 11/19/23 7:58 AM, Dan Purgert wrote:
>> On 2023-11-19, Handsome Jack wrote:
>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>>> <snip>
>>>>> I tried doing both the latter, and got the same "super-block" error.
>>>>>
>>>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>>>
>>>> <snip>
>>>>
>>>> If it is a NTFS partition/file-system you would use your Windows
>>>> chkdsk program to check/repair the NTFS file system.
>>>
>>> Thanks, this seems to have solved it. Luckily I still have access to a
>>> Windows machine into which I could plug this NTFS removable disk.
>>>
>>> What would I have done if I hadn't? Is there not an equivalent of
>>> chkdsk that runs on Linux to fix NTFS disks?
>> Not really, the linux equivalent just resets the NTFS dirty bit. Best
>> to not use NTFS in a linux-only environment.
>>
>>
> What would you format the storage drive to?

I've never had ext{2,3,4} really cause any problems, or at least none
that weren't insurmountable. But I tend to only use external media as
backup-of-backup targets (so even if I have to blow the whole thing
away, it's still duplicated "somewhere").

--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

On 11/19/23 8:26 AM, Dan Purgert wrote:
> On 2023-11-19, Nic wrote:
>> On 11/19/23 7:58 AM, Dan Purgert wrote:
>>> On 2023-11-19, Handsome Jack wrote:
>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>>>> <snip>
>>>>>> I tried doing both the latter, and got the same "super-block" error.
>>>>>>
>>>>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>>>>
>>>>> <snip>
>>>>>
>>>>> If it is a NTFS partition/file-system you would use your Windows
>>>>> chkdsk program to check/repair the NTFS file system.
>>>> Thanks, this seems to have solved it. Luckily I still have access to a
>>>> Windows machine into which I could plug this NTFS removable disk.
>>>>
>>>> What would I have done if I hadn't? Is there not an equivalent of
>>>> chkdsk that runs on Linux to fix NTFS disks?
>>> Not really, the linux equivalent just resets the NTFS dirty bit. Best
>>> to not use NTFS in a linux-only environment.
>>>
>>>
>> What would you format the storage drive to?
> I've never had ext{2,3,4} really cause any problems, or at least none
> that weren't insurmountable. But I tend to only use external media as
> backup-of-backup targets (so even if I have to blow the whole thing
> away, it's still duplicated "somewhere").
>
>
So in my specific case, I have a primary ssd and a secondary spin drive
4TB (ntfs) where I store my data (videos,PDF,text files and image
files.). In your opinion it would be better to format the secondary
drive to ext.4.

Dan Purgert <dan@djph.net> wrote:
> On 2023-11-19, Handsome Jack wrote:
>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>
>>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>>
>>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>> <snip>
>>>> I tried doing both the latter, and got the same "super-block" error.
>>>>
>>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>>
>>> <snip>
>>>
>>> If it is a NTFS partition/file-system you would use your Windows
>>> chkdsk program to check/repair the NTFS file system.
>>
>>
>> Thanks, this seems to have solved it. Luckily I still have access to a
>> Windows machine into which I could plug this NTFS removable disk.
>>
>> What would I have done if I hadn't? Is there not an equivalent of
>> chkdsk that runs on Linux to fix NTFS disks?
>
> Not really, the linux equivalent just resets the NTFS dirty bit. Best
> to not use NTFS in a linux-only environment.

But the whole point of the removable backup drive is that one morning I will turn on my Linux machine and it won't start; and then I plug in my removable drive to my Windows laptop and get on with my work. If Windows can't read the removable drive then I won't be able to do that. It has to be readable by *both* OSs.

Op 19-11-2023 om 17:23 schreef Handsome Jack:

> But the whole point of the removable backup drive is that one morning I will turn on my Linux machine and it won't start; and then I plug in my removable drive to my Windows laptop and get on with my work. If Windows can't read the removable drive then I won't be able to do that. It has to be readable by *both* OSs.

I'm perfectly happy with NTFS, but exFAT may work too.

On 2023-11-19, Handsome Jack <Jack@handsome.com> wrote:
> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>
>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>
>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>> <snip>
>>> I tried doing both the latter, and got the same "super-block" error.
>>>
>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>
>> <snip>
>>
>> If it is a NTFS partition/file-system you would use your Windows
>> chkdsk program to check/repair the NTFS file system.
>
>
> Thanks, this seems to have solved it. Luckily I still have access
> to a Windows machine into which I could plug this NTFS removable disk.
>
> What would I have done if I hadn't? Is there not an equivalent
> of chkdsk that runs on Linux to fix NTFS disks?

No, there is no Linux application that will do a proper job of it.
There was a small (<50mb) LiveCD called "Recovery Console" back about
Win XP era that could be used. I did not own any Windows
machines myself so used it to run chkdsk to fix another persons laptop
that had a damaged NTFS filesystem. Don't know if it would work on
current NTFS file systems. Bottom line is you have to use a Windows
version of chkdsk when dealing with NTFS.
I think you can create a bootable ISO from your Windows computer.

>>
>> What is the source file system? Another NTFS file system mounted
>> using 3g-ntfs via Linux or is it a Linux file system
>> where the source files reside?
>
> Linux.

Not a good idea. If you HAVE to store linux files on a NTFS
file system, you should first tar up the directory/s first then rsync
the tared file/s over to the NTFS file system. If it were I, I would
resize the NTFS partition on the backup drive to make room for a Linux
partition formatting it as EXT3 or EXT4 file system to back up
the Linux system files.

>> What file system type does the file reside on that you can not delete?
>>
>
> NTFS. Deleted them now.

Good.

On 2023-11-19, Nic wrote:
> On 11/19/23 8:26 AM, Dan Purgert wrote:
>> On 2023-11-19, Nic wrote:
>>> On 11/19/23 7:58 AM, Dan Purgert wrote:
>>>> [...]
>>>> Not really, the linux equivalent just resets the NTFS dirty bit. Best
>>>> to not use NTFS in a linux-only environment.
>>>>
>>>>
>>> What would you format the storage drive to?
>> I've never had ext{2,3,4} really cause any problems, or at least none
>> that weren't insurmountable. But I tend to only use external media as
>> backup-of-backup targets (so even if I have to blow the whole thing
>> away, it's still duplicated "somewhere").
>>
>>
> So in my specific case, I have a primary ssd and a secondary spin drive
> 4TB (ntfs) where I store my data (videos,PDF,text files and image
> files.). In your opinion it would be better to format the secondary
> drive to ext.4.

Well, if it was never leaving a linux-only environment, that'd most
likely by my first action. But you do whatever you're comfortable with.

--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

On 2023-11-19, Handsome Jack wrote:
> Dan Purgert <dan@djph.net> wrote:
>> [...]
>> Not really, the linux equivalent just resets the NTFS dirty bit. Best
>> to not use NTFS in a linux-only environment.
>
>
> But the whole point of the removable backup drive is that one morning
> I will turn on my Linux machine and it won't start; and then I plug in
> my removable drive to my Windows laptop and get on with my work. If
> Windows can't read the removable drive then I won't be able to do
> that. It has to be readable by *both* OSs.

Then you're not in a "linux-only environment".

--
|_|O|_|
|_|_|O| Github: https://github.com/dpurgert
|O|O|O| PGP: DDAB 23FB 19FA 7D85 1CC1 E067 6D65 70E5 4CE7 2860

On 11/19/2023 2:27 PM, Kirk_Rockstein wrote:
> On 2023-11-19, Handsome Jack <Jack@handsome.com> wrote:
>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>
>>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>>
>>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>> <snip>
>>>> I tried doing both the latter, and got the same "super-block" error.
>>>>
>>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>>
>>> <snip>
>>>
>>> If it is a NTFS partition/file-system you would use your Windows
>>> chkdsk program to check/repair the NTFS file system.
>>
>>
>> Thanks, this seems to have solved it. Luckily I still have access
>> to a Windows machine into which I could plug this NTFS removable disk.
>>
>> What would I have done if I hadn't? Is there not an equivalent
>> of chkdsk that runs on Linux to fix NTFS disks?
>
> No, there is no Linux application that will do a proper job of it.
> There was a small (<50mb) LiveCD called "Recovery Console" back about
> Win XP era that could be used. I did not own any Windows
> machines myself so used it to run chkdsk to fix another persons laptop
> that had a damaged NTFS filesystem. Don't know if it would work on
> current NTFS file systems. Bottom line is you have to use a Windows
> version of chkdsk when dealing with NTFS.
> I think you can create a bootable ISO from your Windows computer.
>
>>>
>>> What is the source file system? Another NTFS file system mounted
>>> using 3g-ntfs via Linux or is it a Linux file system
>>> where the source files reside?
>>
>> Linux.
>
> Not a good idea. If you HAVE to store linux files on a NTFS
> file system, you should first tar up the directory/s first then rsync
> the tared file/s over to the NTFS file system. If it were I, I would
> resize the NTFS partition on the backup drive to make room for a Linux
> partition formatting it as EXT3 or EXT4 file system to back up
> the Linux system files.
>
>
>>> What file system type does the file reside on that you can not delete?
>>>
>>
>> NTFS. Deleted them now.
>
> Good.
>

A part of selecting filesystems, is a "planetary model".

The Windows file system, if used, gets maintenance when it is
connected to Windows.

An EXT4 stands a good chance of getting proper maintenance if
connected to Linux. A "reader" may ignore the maintenance
needs and just perform a reading function.

You may not be aware of the details regarding your file system
of choice, when you selected it. Linux, as far as I know,
invalidates the NTFS journal, because it doesn't have code for the
journal, and any interaction with the file system, means the
journal no longer matches file system state.

If the power goes off in mid-session on NTFS, the next stop
is Windows. Then CHKDSK or autochk will look into it for you.

*******

There are all sorts of possibilities for adventures, with computers.

https://devblogs.microsoft.com/commandline/access-linux-filesystems-in-windows-and-wsl-2/

https://en.wikipedia.org/wiki/Universal_Disk_Format

[Picture]

https://i.postimg.cc/2SX3bX3S/why-we-select-obscure-things.gif

In the example, I have stored a file on a "portable" filesystem,
but I have no idea, not the foggiest clue, as to whether
any maintenance tools are available for my choice or not.
It's "UDF, as mystery meat".

Paul

On 11/19/23 7:51 PM, Dan Purgert wrote:
> On 2023-11-19, Nic wrote:
>> On 11/19/23 8:26 AM, Dan Purgert wrote:
>>> On 2023-11-19, Nic wrote:
>>>> On 11/19/23 7:58 AM, Dan Purgert wrote:
>>>>> [...]
>>>>> Not really, the linux equivalent just resets the NTFS dirty bit. Best
>>>>> to not use NTFS in a linux-only environment.
>>>>>
>>>>>
>>>> What would you format the storage drive to?
>>> I've never had ext{2,3,4} really cause any problems, or at least none
>>> that weren't insurmountable. But I tend to only use external media as
>>> backup-of-backup targets (so even if I have to blow the whole thing
>>> away, it's still duplicated "somewhere").
>>>
>>>
>> So in my specific case, I have a primary ssd and a secondary spin drive
>> 4TB (ntfs) where I store my data (videos,PDF,text files and image
>> files.). In your opinion it would be better to format the secondary
>> drive to ext.4.
> Well, if it was never leaving a linux-only environment, that'd most
> likely by my first action. But you do whatever you're comfortable with.
>
>
Yeah well, we all know about m$ invasive business model.

On 11/20/23 12:37 AM, Paul wrote:
> On 11/19/2023 2:27 PM, Kirk_Rockstein wrote:
>> On 2023-11-19, Handsome Jack <Jack@handsome.com> wrote:
>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>> On 2023-11-17, Handsome Jack <Jack@handsome.com> wrote:
>>>>> Kirk_Rockstein <Kirk_Rockstein@nowhere.invalid> wrote:
>>>>>> On 2023-11-16, Handsome Jack <Jack@handsome.com> wrote:
>>>> <snip>
>>>>> I tried doing both the latter, and got the same "super-block" error.
>>>>>
>>>>> BTW the disk is a real (spinning) disk and is formatted as HPFS/NTFS/exFAT
>>>>>
>>>> <snip>
>>>>
>>>> If it is a NTFS partition/file-system you would use your Windows
>>>> chkdsk program to check/repair the NTFS file system.
>>>
>>> Thanks, this seems to have solved it. Luckily I still have access
>>> to a Windows machine into which I could plug this NTFS removable disk.
>>>
>>> What would I have done if I hadn't? Is there not an equivalent
>>> of chkdsk that runs on Linux to fix NTFS disks?
>> No, there is no Linux application that will do a proper job of it.
>> There was a small (<50mb) LiveCD called "Recovery Console" back about
>> Win XP era that could be used. I did not own any Windows
>> machines myself so used it to run chkdsk to fix another persons laptop
>> that had a damaged NTFS filesystem. Don't know if it would work on
>> current NTFS file systems. Bottom line is you have to use a Windows
>> version of chkdsk when dealing with NTFS.
>> I think you can create a bootable ISO from your Windows computer.
>>
>>>> What is the source file system? Another NTFS file system mounted
>>>> using 3g-ntfs via Linux or is it a Linux file system
>>>> where the source files reside?
>>> Linux.
>> Not a good idea. If you HAVE to store linux files on a NTFS
>> file system, you should first tar up the directory/s first then rsync
>> the tared file/s over to the NTFS file system. If it were I, I would
>> resize the NTFS partition on the backup drive to make room for a Linux
>> partition formatting it as EXT3 or EXT4 file system to back up
>> the Linux system files.
>>
>>
>>>> What file system type does the file reside on that you can not delete?
>>>>
>>> NTFS. Deleted them now.
>> Good.
>>
> A part of selecting filesystems, is a "planetary model".
>
> The Windows file system, if used, gets maintenance when it is
> connected to Windows.
>
> An EXT4 stands a good chance of getting proper maintenance if
> connected to Linux. A "reader" may ignore the maintenance
> needs and just perform a reading function.
>
> You may not be aware of the details regarding your file system
> of choice, when you selected it. Linux, as far as I know,
> invalidates the NTFS journal, because it doesn't have code for the
> journal, and any interaction with the file system, means the
> journal no longer matches file system state.
>
> If the power goes off in mid-session on NTFS, the next stop
> is Windows. Then CHKDSK or autochk will look into it for you.
>
> *******
>
> There are all sorts of possibilities for adventures, with computers.
>
> https://devblogs.microsoft.com/commandline/access-linux-filesystems-in-windows-and-wsl-2/
>
> https://en.wikipedia.org/wiki/Universal_Disk_Format
>
> [Picture]
>
> https://i.postimg.cc/2SX3bX3S/why-we-select-obscure-things.gif
>
> In the example, I have stored a file on a "portable" filesystem,
> but I have no idea, not the foggiest clue, as to whether
> any maintenance tools are available for my choice or not.
> It's "UDF, as mystery meat".
>
> Paul
>

re:If the power goes off in mid-session on NTFS, the next stop is
Windows. Then CHKDSK or autochk will look into it for you.

What happens if the filesystem is ext.4?

When I do a 'properties' on my secondary ntfs driveD: I get 'fuse'