Yesterday evening I started transferring 26 gigs of files from one
external drive to another. "a simple procedure" involving Robocopy.
I'd done a similar transfer a couple days ago, it took six hours.

Last night, I woke up in the middle of the night and Windows had
gone to sleep. Halting the transfer _and_ closing the logfile. (The
logfile for the creation of the directory tree was ~8,000 k, this one
was ~2000.) Woke up Windows and it resumes transfer. As far as I can
figure, I had the power saving options set so that windows shuts the
hard drives (and the system) down after a half hour. I reset that to
"never". when it is completed in another hour or so .. "elapsed time"
is 6 hrs)

This morning, I reformatted drive G, moved drive L to the other
USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
around on the computer keeping it awake.

So, is the "shut the system down if the hard drive is not
involved" a feature or a bug?

arggle bargle. Yes I remember when the fastest way to move large
amounts of data was to keep in mind "the bandwidth of a Toyota with a
duffle bag full of floppies." I forgot what I figured it worked out
but a six hour drive wasn't so bad. I was going there anyway.

--
APL is a mistake, carried through to perfection. It is the language of the
future for the programming techniques of the past: it creates a new generation
of coding bums.
-- Edsger W. Dijkstra, SIGPLAN Notices, Volume 17, Number 5

On 2/10/2023 3:57 PM, pyotr filipivich wrote:
>
> Yesterday evening I started transferring 26 gigs of files from one
> external drive to another. "a simple procedure" involving Robocopy.
> I'd done a similar transfer a couple days ago, it took six hours.
>
> Last night, I woke up in the middle of the night and Windows had
> gone to sleep. Halting the transfer _and_ closing the logfile. (The
> logfile for the creation of the directory tree was ~8,000 k, this one
> was ~2000.) Woke up Windows and it resumes transfer. As far as I can
> figure, I had the power saving options set so that windows shuts the
> hard drives (and the system) down after a half hour. I reset that to
> "never". when it is completed in another hour or so .. "elapsed time"
> is 6 hrs)
>
> This morning, I reformatted drive G, moved drive L to the other
> USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
> around on the computer keeping it awake.
>
> So, is the "shut the system down if the hard drive is not
> involved" a feature or a bug?
>
>
> arggle bargle. Yes I remember when the fastest way to move large
> amounts of data was to keep in mind "the bandwidth of a Toyota with a
> duffle bag full of floppies." I forgot what I figured it worked out
> but a six hour drive wasn't so bad. I was going there anyway.
>
>
Robocopy spends a lot of time keeping a full log. I suggest looking at
its logging options and also /MT, /R, and /W.

I use Robocopy in an unattended overnight batch with /NDL /NFL /NP /MT:8
/R:0 /W:0 for very fast operation and check if the ERRORLEVEL is above 7
indicating a problem and then have the batch report ONLY problems to
review in the morning. If I cannot fix error then rerun with a full log.

Zaidy036 <Zaidy036@air.isp.spam> on Fri, 10 Feb 2023 16:38:46 -0500
typed in alt.windows7.general the following:
>On 2/10/2023 3:57 PM, pyotr filipivich wrote:
>>
>> Yesterday evening I started transferring 26 gigs of files from one
>> external drive to another. "a simple procedure" involving Robocopy.
>> I'd done a similar transfer a couple days ago, it took six hours.
>>
>> Last night, I woke up in the middle of the night and Windows had
>> gone to sleep. Halting the transfer _and_ closing the logfile. (The
>> logfile for the creation of the directory tree was ~8,000 k, this one
>> was ~2000.) Woke up Windows and it resumes transfer. As far as I can
>> figure, I had the power saving options set so that windows shuts the
>> hard drives (and the system) down after a half hour. I reset that to
>> "never". when it is completed in another hour or so .. "elapsed time"
>> is 6 hrs)
>>
>> This morning, I reformatted drive G, moved drive L to the other
>> USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
>> around on the computer keeping it awake.
>>
>> So, is the "shut the system down if the hard drive is not
>> involved" a feature or a bug?
>>
>>
>> arggle bargle. Yes I remember when the fastest way to move large
>> amounts of data was to keep in mind "the bandwidth of a Toyota with a
>> duffle bag full of floppies." I forgot what I figured it worked out
>> but a six hour drive wasn't so bad. I was going there anyway.
>>
>>
>Robocopy spends a lot of time keeping a full log. I suggest looking at
>its logging options and also /MT, /R, and /W.

I've limited Robocopy's logging to extra / new /newer file, size
and filename & part. /R: is 5 /W:5
>
>I use Robocopy in an unattended overnight batch with /NDL /NFL /NP /MT:8
>/R:0 /W:0 for very fast operation and check if the ERRORLEVEL is above 7
>indicating a problem and then have the batch report ONLY problems to
>review in the morning. If I cannot fix error then rerun with a full log.

I thought I could do that. But it appears that unless something
joggles Windows elbow on a regular schedule, it figures nothing is
happening and turn off the lights and goes home.
--
pyotr filipivich
This Week's Panel: Us & Them - Eliminating Them.
Next Month's Panel: Having eliminated the old Them(tm)
Selecting who insufficiently Woke(tm) as to serve as the new Them(tm)

On 2/11/2023 12:23 AM, pyotr filipivich wrote:
> Zaidy036 <Zaidy036@air.isp.spam> on Fri, 10 Feb 2023 16:38:46 -0500
> typed in alt.windows7.general the following:
>> On 2/10/2023 3:57 PM, pyotr filipivich wrote:
>>>
>>> Yesterday evening I started transferring 26 gigs of files from one
>>> external drive to another. "a simple procedure" involving Robocopy.
>>> I'd done a similar transfer a couple days ago, it took six hours.
>>>
>>> Last night, I woke up in the middle of the night and Windows had
>>> gone to sleep. Halting the transfer _and_ closing the logfile. (The
>>> logfile for the creation of the directory tree was ~8,000 k, this one
>>> was ~2000.) Woke up Windows and it resumes transfer. As far as I can
>>> figure, I had the power saving options set so that windows shuts the
>>> hard drives (and the system) down after a half hour. I reset that to
>>> "never". when it is completed in another hour or so .. "elapsed time"
>>> is 6 hrs)
>>>
>>> This morning, I reformatted drive G, moved drive L to the other
>>> USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
>>> around on the computer keeping it awake.
>>>
>>> So, is the "shut the system down if the hard drive is not
>>> involved" a feature or a bug?
>>>
>>>
>>> arggle bargle. Yes I remember when the fastest way to move large
>>> amounts of data was to keep in mind "the bandwidth of a Toyota with a
>>> duffle bag full of floppies." I forgot what I figured it worked out
>>> but a six hour drive wasn't so bad. I was going there anyway.
>>>
>>>
>> Robocopy spends a lot of time keeping a full log. I suggest looking at
>> its logging options and also /MT, /R, and /W.
>
> I've limited Robocopy's logging to extra / new /newer file, size
> and filename & part. /R: is 5 /W:5
>>
>> I use Robocopy in an unattended overnight batch with /NDL /NFL /NP /MT:8
>> /R:0 /W:0 for very fast operation and check if the ERRORLEVEL is above 7
>> indicating a problem and then have the batch report ONLY problems to
>> review in the morning. If I cannot fix error then rerun with a full log.
>
> I thought I could do that. But it appears that unless something
> joggles Windows elbow on a regular schedule, it figures nothing is
> happening and turn off the lights and goes home.

So set your screen saver time to something beyond that necessary.

I use a program called PTFBpro <https://www.ptfbpro.com/download.shtml>
Not free. It can reset the time during the batch and do anything else
you would do manually.

During my batch I turn off the monitors(2) and at the end set up the
programs I want running in the morning and then lock the PC. I do not
shut off my PC except when traveling away from home.

Zaidy036 <Zaidy036@air.isp.spam> on Sat, 11 Feb 2023 11:41:34 -0500
typed in alt.windows7.general the following:
>On 2/11/2023 12:23 AM, pyotr filipivich wrote:
>> Zaidy036 <Zaidy036@air.isp.spam> on Fri, 10 Feb 2023 16:38:46 -0500
>> typed in alt.windows7.general the following:
>>> On 2/10/2023 3:57 PM, pyotr filipivich wrote:
>>>>
>>>> Yesterday evening I started transferring 26 gigs of files from one
>>>> external drive to another. "a simple procedure" involving Robocopy.
>>>> I'd done a similar transfer a couple days ago, it took six hours.
>>>>
>>>> Last night, I woke up in the middle of the night and Windows had
>>>> gone to sleep. Halting the transfer _and_ closing the logfile. (The
>>>> logfile for the creation of the directory tree was ~8,000 k, this one
>>>> was ~2000.) Woke up Windows and it resumes transfer. As far as I can
>>>> figure, I had the power saving options set so that windows shuts the
>>>> hard drives (and the system) down after a half hour. I reset that to
>>>> "never". when it is completed in another hour or so .. "elapsed time"
>>>> is 6 hrs)
>>>>
>>>> This morning, I reformatted drive G, moved drive L to the other
>>>> USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
>>>> around on the computer keeping it awake.
>>>>
>>>> So, is the "shut the system down if the hard drive is not
>>>> involved" a feature or a bug?
>>>>
>>>>
>>>> arggle bargle. Yes I remember when the fastest way to move large
>>>> amounts of data was to keep in mind "the bandwidth of a Toyota with a
>>>> duffle bag full of floppies." I forgot what I figured it worked out
>>>> but a six hour drive wasn't so bad. I was going there anyway.
>>>>
>>>>
>>> Robocopy spends a lot of time keeping a full log. I suggest looking at
>>> its logging options and also /MT, /R, and /W.
>>
>> I've limited Robocopy's logging to extra / new /newer file, size
>> and filename & part. /R: is 5 /W:5
>>>
>>> I use Robocopy in an unattended overnight batch with /NDL /NFL /NP /MT:8
>>> /R:0 /W:0 for very fast operation and check if the ERRORLEVEL is above 7
>>> indicating a problem and then have the batch report ONLY problems to
>>> review in the morning. If I cannot fix error then rerun with a full log.
>>
>> I thought I could do that. But it appears that unless something
>> joggles Windows elbow on a regular schedule, it figures nothing is
>> happening and turn off the lights and goes home.
>
>So set your screen saver time to something beyond that necessary.

I did : "Never"
>
>I use a program called PTFBpro <https://www.ptfbpro.com/download.shtml>
>Not free. It can reset the time during the batch and do anything else
>you would do manually.
>
>During my batch I turn off the monitors(2) and at the end set up the
>programs I want running in the morning and then lock the PC. I do not
>shut off my PC except when traveling away from home.
--
pyotr filipivich
This Week's Panel: Us & Them - Eliminating Them.
Next Month's Panel: Having eliminated the old Them(tm)
Selecting who insufficiently Woke(tm) as to serve as the new Them(tm)

On 2/11/2023 10:04 PM, pyotr filipivich wrote:

>> So set your screen saver time to something beyond that necessary.
>
> I did : "Never"

VLC and Windows Media Player will set the execution properties
such that the OS won't shut off. This allows a 2 hour movie
to play, without "jiggling the mouse".

As for what kind of media content to use, the video players should
have a "loop mode". This means you can use a short video clip,
and the players will just keep playing it forever. And at the
same time, keeping the properties needed, asserted.

( Not all OSes have sample files )

C:\users\public\videos\sample videos\wildlife.wmv

(it's from a Windows 7 package in WinSxS...)

You might also evaluate your drives, using the read-test of the
HDTune benchmark. And determine whether the drives work in a
decently healthy way.

https://www.hdtune.com/files/hdtune_255.exe

When you transfer a half million tiny files, the drives just don't
handle that well, so you cannot expect miracles from spinning rust.
And it takes time to zip up a half million files as well, so you
cannot "bypass" the issue by using a ZIP file. The conversion to ZIP,
is just as expensive as copying the half million files.

A modern 3.5" drive can do 200-225MB/sec.
A 15K drive, the best of them could do 300MB/sec
(but that was probably a SAS drive as well).
Some of the older hard drives (right when SATA came out),
they used an IDE to SATA chip, with a 100MB/sec limitation,
so that's a reason/excuse for suckage. I have just one
drive with that chip on the controller board.

My first PC drive, ran at the amazing speed of 5MB/sec
and even today, the drive is still healthy. Now, imagine
how painful a transfer would be on that thing.

As for smooth transfers, a Macrium partition backup
may be smoother than a Robocopy file-by-file. If the
files represent most of a partition, cloning the
partition will be faster than copying the files
one by one. The difference is all the directory
updating in the file-by-file case.

I use that method for special test cases. I use Macrium
to make a backup copy of a partition, because if I let
File Explorer anywhere near it, the transfer will be as slow
as molasses.

Paul

On Fri, 10 Feb 2023 12:57:01 -0800, pyotr filipivich
<phamp@mindspring.com> wrote:

>
>Yesterday evening I started transferring 26 gigs of files from one
>external drive to another. "a simple procedure" involving Robocopy.
>I'd done a similar transfer a couple days ago, it took six hours.
>
> Last night, I woke up in the middle of the night and Windows had
>gone to sleep. Halting the transfer _and_ closing the logfile. (The
>logfile for the creation of the directory tree was ~8,000 k, this one
>was ~2000.) Woke up Windows and it resumes transfer. As far as I can
>figure, I had the power saving options set so that windows shuts the
>hard drives (and the system) down after a half hour. I reset that to
>"never". when it is completed in another hour or so .. "elapsed time"
>is 6 hrs)
>
> This morning, I reformatted drive G, moved drive L to the other
>USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
>around on the computer keeping it awake.
>
> So, is the "shut the system down if the hard drive is not
>involved" a feature or a bug?
>
>
>arggle bargle. Yes I remember when the fastest way to move large
>amounts of data was to keep in mind "the bandwidth of a Toyota with a
>duffle bag full of floppies." I forgot what I figured it worked out
>but a six hour drive wasn't so bad. I was going there anyway.
>
>

Just go in and set all the power saving features to "never". I have my
disk tool machine set up that way. I suppose you could save it as a
power scheme that you can select and then go back to your "save the
planet" settings when you are done.
What is so great about robocopy over simply dragging and dropping?
I use file synchronizer (not free but cheap) when I am selectively
moving stuff but if I just copying I drag and drop.
Sometimes I will go into command line and use Xcopy. (lots of options
there)

On 2/12/2023 12:50 PM, gfretwell@aol.com wrote:

> What is so great about robocopy over simply dragging and dropping?

The answer to that is /MIR .

Only new files are copied and the command execution
time should be much shorter. (The directory trees
have to be scanned for differences and this takes time.)

The "help" for Robocopy, has a warning paragraph in it
that the program can *delete* files, and /MIR deletes
destination files which are not in the source tree, as part of
synchronization (mirroring).

Robocopy also has overlapping I/O. It can be reading the next
file, while writing the current file. This is an advantage when
"two spindles are involved".

Paul

gfretwell@aol.com on Sun, 12 Feb 2023 12:50:34 -0500 typed in
alt.windows7.general the following:
>On Fri, 10 Feb 2023 12:57:01 -0800, pyotr filipivich
><phamp@mindspring.com> wrote:
>>Yesterday evening I started transferring 26 gigs of files from one
>>external drive to another. "a simple procedure" involving Robocopy.
>>I'd done a similar transfer a couple days ago, it took six hours.
>>
>> Last night, I woke up in the middle of the night and Windows had
>>gone to sleep. Halting the transfer _and_ closing the logfile. (The
>>logfile for the creation of the directory tree was ~8,000 k, this one
>>was ~2000.) Woke up Windows and it resumes transfer. As far as I can
>>figure, I had the power saving options set so that windows shuts the
>>hard drives (and the system) down after a half hour. I reset that to
>>"never". when it is completed in another hour or so .. "elapsed time"
>>is 6 hrs)
>>
>> This morning, I reformatted drive G, moved drive L to the other
>>USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
>>around on the computer keeping it awake.
>>
>> So, is the "shut the system down if the hard drive is not
>>involved" a feature or a bug?
>>
>>
>>arggle bargle. Yes I remember when the fastest way to move large
>>amounts of data was to keep in mind "the bandwidth of a Toyota with a
>>duffle bag full of floppies." I forgot what I figured it worked out
>>but a six hour drive wasn't so bad. I was going there anyway.
>>
>
>Just go in and set all the power saving features to "never". I have my
>disk tool machine set up that way. I suppose you could save it as a
>power scheme that you can select and then go back to your "save the
>planet" settings when you are done.
>What is so great about robocopy over simply dragging and dropping?

Automation. "Fire and forget".
Drag and drop wants to ask each and every time there is a
duplicate file name, do I want to replace, not replace, or copy as a
second instance? When one has a "Brazilian" files, that is beyond
tedious. Granted Robocopy can take a while to set up in a batch file,
but once done, I don't have to give instructions again.
Also, it seems to me that Drag and Drop has a lot of overhead.
(Sometimes it seems to me that Windows wants to make sure all files
which are to be copied, moved, or deleted must first be checked for
errors and if they are in proper alphabetical order. Operative word
"seems". Annoying, regardless.)

>I use file synchronizer (not free but cheap) when I am selectively
>moving stuff but if I just copying I drag and drop.
>Sometimes I will go into command line and use Xcopy. (lots of options
>there)

I used to use Xcopy too. I don't know if it had an option to only
copy "newer" files. I started with Robocopy back when I was trying to
keep the class files on my computer synced with the ones in class.
Which sometimes required not just assignment files but the
configuration files, because I could not be assured that (A) I'd have
the same machine and (B) a previous user hadn't "done something
idiosyncratic and weird."
I could set up Robocopy to get just the files which had changed
(or were new) and ignore the rest, thereby shortening the process of
getting out of here. Lot of late nights in the lab. Not quite on a
name basis with security but "Is it that late? okay, let me log out."
Anyway, after a while, I sort of forgot about Xcopy, etc.

So, Robocopy is my "goto program" for anything more than "just
move this handful of files."
--
pyotr filipivich
This Week's Panel: Us & Them - Eliminating Them.
Next Month's Panel: Having eliminated the old Them(tm)
Selecting who insufficiently Woke(tm) as to serve as the new Them(tm)

Paul <nospam@needed.invalid> on Sun, 12 Feb 2023 14:41:29 -0500 typed
in alt.windows7.general the following:
>On 2/12/2023 12:50 PM, gfretwell@aol.com wrote:
>> What is so great about robocopy over simply dragging and dropping?
>
>The answer to that is /MIR .
>
>Only new files are copied and the command execution
>time should be much shorter. (The directory trees
>have to be scanned for differences and this takes time.)
>
>The "help" for Robocopy, has a warning paragraph in it
>that the program can *delete* files, and /MIR deletes
>destination files which are not in the source tree, as part of
>synchronization (mirroring).

Robocopy also has the option to /CREATE the directory tree and
entering all the file names (but length zero files).
>
>Robocopy also has overlapping I/O. It can be reading the next
>file, while writing the current file. This is an advantage when
>"two spindles are involved".
>
> Paul
--
pyotr filipivich
This Week's Panel: Us & Them - Eliminating Them.
Next Month's Panel: Having eliminated the old Them(tm)
Selecting who insufficiently Woke(tm) as to serve as the new Them(tm)

On 12/02/2023 17:50, gfretwell@aol.com wrote:
> What is so great about robocopy over simply dragging and dropping?
> I use file synchronizer (not free but cheap) when I am selectively
> moving stuff but if I just copying I drag and drop.
> Sometimes I will go into command line and use Xcopy. (lots of options
> there)

I use Microsoft SyncToy (free) which does incremental copies: it
compares the source folder with the destination folder (which takes a
little while) and then only copies the files that are new/changed.

In the OP's situation of having a partial copy of the tree, SyncToy
would have identified which files already existed in the destination
folder and only copied the rest.

It may be an old program that hasn't been changed for years, but it does
the job. The only time it is slow is when there are a lot of files to
compare - eg when adding/changing a few files in a tree structure where
90% already exists on the destination. And then it's a matter of ten
minutes or so.

It normally compares timestamps and file size to determine changed
files, but I think there is an option to compare file contents at the
expense of taking a lot longer to determine whether to copy (with a
large file, you hope that the first difference between source and
destination is near the beginning of the file!).

On 2/10/2023 2:57 PM, pyotr filipivich wrote:
>
> Yesterday evening I started transferring 26 gigs of files from one
> external drive to another. "a simple procedure" involving Robocopy.
> I'd done a similar transfer a couple days ago, it took six hours.
>
> Last night, I woke up in the middle of the night and Windows had
> gone to sleep. Halting the transfer _and_ closing the logfile. (The
> logfile for the creation of the directory tree was ~8,000 k, this one
> was ~2000.) Woke up Windows and it resumes transfer. As far as I can
> figure, I had the power saving options set so that windows shuts the
> hard drives (and the system) down after a half hour. I reset that to
> "never". when it is completed in another hour or so .. "elapsed time"
> is 6 hrs)
>
> This morning, I reformatted drive G, moved drive L to the other
> USB 3 port, and repeated. This time it was 2 .5 hours while I fussed
> around on the computer keeping it awake.
>
> So, is the "shut the system down if the hard drive is not
> involved" a feature or a bug?
>
>
> arggle bargle. Yes I remember when the fastest way to move large
> amounts of data was to keep in mind "the bandwidth of a Toyota with a
> duffle bag full of floppies." I forgot what I figured it worked out
> but a six hour drive wasn't so bad. I was going there anyway.

This is why I do not let the PCs in my shop hibernate.

Lynn