>> No, people want that kind of stuff found at compile time.
> Not possible in the more general case, as you know.

Of course it's possible. You just have to design your language
accordingly, but lots of languages have solved most of those problems
(tho many have introduced others).

Stefan

Michael S wrote:
> On Sunday, September 19, 2021 at 3:09:39 PM UTC+3, Terje Mathisen
> wrote:
>> Anton Ertl wrote:
>>> Terje Mathisen <terje.m...@tmsw.no> writes:
>>>> This problem has almost nothing to do with REP MOVS, the
>>>> two-byte version should have been solved with code that
>>>> replicated the pattern into an 8-byte register, then storing
>>>> that repeatedly.
>>>
>>> There are alternative ways to do it, yes, but that's beside the
>>> point. So we already went from an implementation that uses REP
>>> MOVSB for an overlapping case to an implementation that uses it
>>> only for the non-overlapping case (i.e., use REP MOVSB for
>>> memcpy() functionality). And it's still much slower than a not
>>> particularly sophisticated implementation of the same
>>> functionality with simpler instructions.
>>>
>>> Every time you write that one should use something else supports
>>> the position that CISCy instructions are not as great as Brett
>>> thinks.
>> I do agree that a fast REP MOVS should be able to handle all cases
>> efficiently, including the one where it is used to unpack an
>> RLL-encoded string by replicating a pattern.
>>
>> However, I don't see any way to make this faster than the cache
>> hierarchy/forwarding network would let you repeatedly re-read and
>> write to the same cache line, and if the pattern has a short
>> length, then the "fast strings" hw must in fact detect this
>> situation and use it to internally unroll/replicate the pattern and
>> turn it into the same code I wrote by hand, i.e. a repeated block
>> store.
>>
>> Until that happens I would be quite happy with simply having a
>> fast non-overlapping copy that matches or beats individual
>> hand-coded instructions, even if the partly overlapping case is
>> reduced to running at a byte or two per cycle.
>>
>> Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can
>> be viewed as an exercise in caching"
>
> For copies of small integer count of words (where 'word' is either of
> GPR size or of VR size) where count is known at compile time, your
> performance expectation is unrealistic.
>

Urealistic, yeah. Impossible? Maybe not.

The main issue is in fact the need to setup the block move opcode, i.e.
you need at least to load a register with the # of bytes to move, even
if both source and dest address already exists in registers usable by
the block move instruction.

I would however be perfectly happy with a comiler which generated a pair
of MOVs for small struct copies and fell back on REP MOVS for
longer/variable length blocks.

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

MitchAlsup <MitchAlsup@aol.com> schrieb:
> On Sunday, September 19, 2021 at 5:38:51 PM UTC-5, Thomas Koenig wrote:
>> MitchAlsup <Mitch...@aol.com> schrieb:
>> > On Sunday, September 19, 2021 at 5:16:03 PM UTC-5, Thomas Koenig wrote:
>> >> Stefan Monnier <mon...@iro.umontreal.ca> schrieb:
>> >> >> There is no specification that a compiler writer could follow.
>> >> >> The respective language standards are irrelevant. Past behavior
>> >> >> is all that counts, and pointing out that his program is errnoneous
>> >> >> is a no-no.
>> >> >>
>> >> >> The consequence is clear - compiler writers should stop fixing
>> >> >> bugs because fixing a bug might change the observable behavior of
>> >> >> existing programs, which is bad.
>> >> >
>> >> > I think Mitch had another conclusion which I would paraphrase as
>> >> > "a serious API shouldn't have any `undefined` part".
>> >> Mandatory bounds checking? Mandatory checking for -32768, with
>> >> a specification what happens with a bounds overrun? Exact
>> >> language specification of what happens if memory on the system
>> >> runs out (i.e. no memory overcommit on Linux)?
>> >>
>> >> And even so, according to Linus T's definition, if code like
>> >>
>> >> int x[10];
>> >> int a;
>> >>
>> >> ...
>> >>
>> >> x[10] = 42;
>> >> assert (a == 42);
>> >>
>> >> ever "worked" on any system, it would have to be kept that way -
>> >> mandating stack layout of variables, keeping a from being kept in
>> >> a register etc.
>> >>
>> >> Is that really what people want?
>> ><
>> > No, people want that kind of stuff found at compile time.
>> Not possible in the more general case, as you know.
><
> ADA does not seem to have this problem. Perhaps because they actually care.

You wrote "at compile time". Testing the correctnes of a program
written in Turing-complete languge is the equivalent of solving
the halting problem.

Besides:

Does Ada catch the equivalent of C's

int x[10];
int i;
scanf ("%d", &i);
x[i] = 42;

at compile-time because x[i] might overflow?

At run-time is a different kettle of fish, of course. Even so,
how does Ada guarantee that there is no overcommitment of memory
on Linux, the OOM killer comes along and kills the process?

Stefan Monnier <monnier@iro.umontreal.ca> schrieb:
>>> No, people want that kind of stuff found at compile time.
>> Not possible in the more general case, as you know.
>
> Of course it's possible. You just have to design your language
> accordingly, but lots of languages have solved most of those problems
> (tho many have introduced others).

Not at compile-time (which is what Mitch wrote above).

MitchAlsup <MitchAlsup@aol.com> schrieb:
> On Sunday, September 19, 2021 at 11:57:06 AM UTC-5, Thomas Koenig wrote:
>> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
>> > Then Ulrich Drepper thought it was a good idea to use backwards
>> > stride, and that code broke; and his really great idea was to make the
>> > stride dependent on the CPU model at run-time. So it might test fine
>> > on the original developer's machine, and break on a user's machine,
>> > which is instruction-set compatible with the developer's machine, only
>> > library-incompatible.
><
>> Shared library-incompatible, to be exact. Welcome to shared library
>> hell.
><
> I missed this before::
><
> Is the change in observed behavior because of some out-of-specification
> use of shared library, or a change in the in-specification use of shared the
> library?

The first.

Using memcpy with overlapping arguments is undefined per the C
standard.

George Neuner <gneuner2@comcast.net> schrieb:
> On Sun, 19 Sep 2021 13:28:10 -0700 (PDT), MitchAlsup
><MitchAlsup@aol.com> wrote:
>
>>We can conclude from this that "computer science" is not science--it fails
>>to learn form its own past.
>
> A good rule of thumb to keep in mind is that anything that calls
> itself "science" probably isn’t.
> -- John R. Searle

In numerical analysis, optimization means finding a minimizer
for some objective function. In compilation, optimization means
running some series of passes & declaring victory. The definition
from numerical analysis seems more rigorous.

-- Keith Cooper

On 19/09/2021 13:38, Anton Ertl wrote:
> Thomas Koenig <tkoenig@netcologne.de> writes:
>> Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>
>>> Unless they want to have problems like the glibc maintainers
>>> encountered <https://bugzilla.redhat.com/show_bug.cgi?id=638477>
>>> <https://sourceware.org/bugzilla/show_bug.cgi?id=12518> when they
>>> relied on their "it's undefined behaviour" fantasy, they better
>>> should.
>>
>> So, if memcpy happens to work like memmove for a certain implementation,
>> it should work that way forever?
>
> Exactly.
>
> However, my educated guess at what happened is: memcpy()
> implementations up to that time were using forward stride, and were
> safe for overlapping copies to lower addresses, so some uses of this
> kind appeared in code, of course; after all, the code passed the
> tests.
>
> Then Ulrich Drepper thought it was a good idea to use backwards
> stride, and that code broke; and his really great idea was to make the
> stride dependent on the CPU model at run-time. So it might test fine
> on the original developer's machine, and break on a user's machine,
> which is instruction-set compatible with the developer's machine, only
> library-incompatible. This would have stayed a minor episode if
> Ulrich Drepper had accepted the bug report and fixed the bug, instead
> of being an obstinate fool; admittedly he was not alone, there were
> (and are still) lots of fans of undefined behaviour, but that's no
> excuse.

Imagine the audacity! He made an implementation of the function that
does what the standard says it should do, and does it in the fastest way
it can on the machine running the code! Any programmer who does that
should, of course, be fired - their job is to read the minds of every
third-rate coder out there and figure out what that person /meant/ to
write, with a total disregard for the efficiency of code written by
people who know what they are doing.

Still, in this particular case I do understand that there are so many
cases where people have got this wrong, and used memcpy when they meant
memmove, and that the efficiency cost for making memcpy act like memmove
is so low, that it is probably best to pander to the erroneous code and
make memcpy work like memmove. Libraries could perhaps include extra
functions like "__memcpy_disjoint", "__memmove_forward",
"__memmove_backward" that are particularly optimised, and that a
compiler like gcc can call rather than general "memmove" when asked for
a "memmove" or "memcpy" and it knows something about the relationships
between the pointers.

>
> I suffer from the fallout of this to this day: When I build Gforth on
> a recent system, and try to run it on an ancient one, it does not run,
> complaining that the binary needs a newer version of the glibc .so
> than the old system has. That's because Ulrich Drepper's solution to
> the problem was to have two versions of memcpy() for newer glibc: for
> old binaries, memcpy() does memmove(); for newer binaries, memcpy()
> does something else. And of course the linker magic that they use for
> that is not set up to fall back to the old memcpy() when trying to
> link a newer binary with an old library, even though the new memcpy()
> does not give any better guarantees than the old one.

That does sound a messy "solution".

>
>> There is no justification for having memcpy at all, then.
>
> There isn't. Undefined behaviour (and therefore memcpy()) is just a
> bad idea in a world where testing is the an accepted methodology for
> finding bugs in programs.
>

On that, we disagree. Undefined behaviour is a natural part of computer
science (and the mathematics behind it), and attempts to remove it from
languages always come at a cost (though the cost is not always significant).

>> Just use
>> -Dmemcpy=memmove and be done with it.
>
> Unfortunately, that's not sufficient to get rid of memcpy(); we tried,
> but the library incompatibility problem outlined above persisted.
> IIRC gcc produces calls to memcpy() when you copy some structs or
> somesuch.
>
> - anton
>

On 2021-09-20 10:52, Thomas Koenig wrote:
> MitchAlsup <MitchAlsup@aol.com> schrieb:
>> On Sunday, September 19, 2021 at 5:38:51 PM UTC-5, Thomas Koenig wrote:
>>> MitchAlsup <Mitch...@aol.com> schrieb:
>>>> On Sunday, September 19, 2021 at 5:16:03 PM UTC-5, Thomas Koenig wrote:
>>>>> Stefan Monnier <mon...@iro.umontreal.ca> schrieb:
>>>>>>> There is no specification that a compiler writer could follow.
>>>>>>> The respective language standards are irrelevant. Past behavior
>>>>>>> is all that counts, and pointing out that his program is errnoneous
>>>>>>> is a no-no.
>>>>>>>
>>>>>>> The consequence is clear - compiler writers should stop fixing
>>>>>>> bugs because fixing a bug might change the observable behavior of
>>>>>>> existing programs, which is bad.
>>>>>>
>>>>>> I think Mitch had another conclusion which I would paraphrase as
>>>>>> "a serious API shouldn't have any `undefined` part".
>>>>> Mandatory bounds checking? Mandatory checking for -32768, with
>>>>> a specification what happens with a bounds overrun? Exact
>>>>> language specification of what happens if memory on the system
>>>>> runs out (i.e. no memory overcommit on Linux)?
>>>>>
>>>>> And even so, according to Linus T's definition, if code like
>>>>>
>>>>> int x[10];
>>>>> int a;
>>>>>
>>>>> ...
>>>>>
>>>>> x[10] = 42;
>>>>> assert (a == 42);
>>>>>
>>>>> ever "worked" on any system, it would have to be kept that way -
>>>>> mandating stack layout of variables, keeping a from being kept in
>>>>> a register etc.
>>>>>
>>>>> Is that really what people want?
>>>> <
>>>> No, people want that kind of stuff found at compile time.
>>> Not possible in the more general case, as you know.
>> <
>> ADA does not seem to have this problem. Perhaps because they actually care.
>
> You wrote "at compile time". Testing the correctnes of a program
> written in Turing-complete languge is the equivalent of solving
> the halting problem.
>
> Besides:
>
> Does Ada catch the equivalent of C's
>
> int x[10];
> int i;
> scanf ("%d", &i);
> x[i] = 42;
>
> at compile-time because x[i] might overflow?

Ada compilers would normally not warn about that, but the static
analysis tools (CodePeer, SPARK) often used for critical Ada projects
would do that. So would, of course, similar static analysis tools for C,
but they might find it more difficult in more complex cases because of
C's weak type system for arrays and their indices.

> At run-time is a different kettle of fish, of course. Even so,
> how does Ada guarantee that there is no overcommitment of memory
> on Linux, the OOM killer comes along and kills the process?

Ada cannot guarantee that, of course (the question was probably
rhetorical anyway).

On 19/09/2021 19:50, EricP wrote:
> MitchAlsup wrote:
>> On Sunday, September 19, 2021 at 7:14:52 AM UTC-5, Anton Ertl wrote:
>>> Unfortunately, that's not sufficient to get rid of memcpy(); we
>>> tried, but the library incompatibility problem outlined above
>>> persisted. IIRC gcc produces calls to memcpy() when you copy some
>>> structs or somesuch.
>> <
>> Yes but !!
>> <
>> a structure assignment: s1 = s2; is never in a position where s1 and
>> s2 overlap; and therefore nobody cares which direction, or how wide, any
>> data transfer happens to be.
>
> An array of S's can be shifting elements with overlapping slices.
>
> If s1 and s2 are structs in different areas of a union and passed
> to a subroutine void DoCopy (struct S *s1, struct S *s2);
>
>   union
>   {  struct S s1;
>      struct
>      {  int  i;
>         struct S s2;
>      } foo;
>   } u;
>
> DoCopy cannot prove its args do not overlap so should use memmove.
>

I think there is a subtle point here.

I did some testing with:

struct S { int a; int b[103]; };

union
{ struct S s1;
struct
{ int i;
struct S s2;
} foo;
} u;

void foo1(void) {
u.s1 = u.foo.s2;
}

void foo2(void) {
u.foo.s2 = u.s1;
}

void DoCopy (struct S *s1, struct S *s2) {
*s1 = *s2;
}

Testing with gcc and clang shows that they both use memcpy (or a "rep
movsq" sequence on x86 that I /think/ assumes there is no overlap - but
I am not an expert on x86 assembly).

It's not often that the gcc folk are wrong about this kind of thing.
(Overly pedantic or unhelpful, perhaps, but not wrong.) It's even rarer
for both gcc and clang to be wrong.

I believe the answer is in the C standards, 6.5.16.1p3, regarding the
semantics of the assignment operator:

"""
If the value being stored in an object is read from another object that
overlaps in any way the storage of the first object, then the overlap
shall be exact and the two objects shall have qualified or unqualified
versions of a compatible type; otherwise, the behavior is undefined.
"""

So if you have "s1 = s2", then either they do not overlap at all, or
they are the same object (in which case memcpy will still be fine in
practice).

Niklas Holsti <niklas.holsti@tidorum.invalid> schrieb:
> On 2021-09-20 10:52, Thomas Koenig wrote:

>> At run-time is a different kettle of fish, of course. Even so,
>> how does Ada guarantee that there is no overcommitment of memory
>> on Linux, the OOM killer comes along and kills the process?
>
> Ada cannot guarantee that, of course (the question was probably
> rhetorical anyway).

Actually, it could.

On Linux, any Ada program could check /proc/sys/vm/overcommit_memory
and refuse to start unless the value found there is 2. A few
other systems do not overcommit memory, so they would be fine.

Of course, if the Ada standard mandated this behavior, it might
lead to a sharp drop in the popularity of the language.

On 20/09/2021 00:10, Thomas Koenig wrote:
> Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>> Thomas Koenig <tkoenig@netcologne.de> writes:
>>> Just to get the general philosphy behind Linus T's reasoning.
>>>
>>> A compiler change that introduces a bug in a user program is bad.
>>> It does not matter if the program relied on an implementation
>>> detail not specified either by the compiler or the standard, or
>>> if another compiler or another architecture does this differently.
>>>
>>> Did I sum that up correctly?
>>
>> Correct.
>>
>>> There is no specification that a compiler writer could follow.
>>> The respective language standards are irrelevant. Past behavior
>>> is all that counts, and pointing out that his program is errnoneous
>>> is a no-no.
>>
>> Correct.
>
> You subsccibe to "language standards are irrelevant"?

It sounds more like the "a language is defined by the way
implementations work" philosophy that is common in languages without
standards or governing bodies, and which applied to C before ANSI (or
perhaps before K&R).

It is always tough to find out that code you have used happily for years
is actually erroneous, but the errors didn't manifest themselves as
user-visible bugs before. And there is no doubt that there are times in
C programming where it is impossible to write code that is efficient and
gives the desired behaviour on both a weaker and laxer compiler, and a
stronger but fussier compiler.

To me, the answer has always been that the language (and library)
specifications (including toolchain documentation) form the contract
between the programmer and the toolchain. Stray outside of that, and
you have no grounds for expecting consistency. And that also means that
if you can't rely on the code having been written fully to the C
standards (and few people can), then your testing is only valid for the
combinations of toolchain version, compiler flags, and target that you
used for testing. This is why a project like the Linux kernel expects
only a limited selection of compilers and versions, and is widely tested
on different platforms before being considered "tested and working". It
is why on my one projects (which are a very different kind of work), I
consider the toolchain to be a part of the project and do not change
toolchains in the lifetime of the project.

So if you don't know your code works correctly with the latest gcc-11,
but you are sure it works with gcc-5, then install gcc-5 and use that.
Or install a later version and test it, fixing the code or reporting
bugs as appropriate.

>
> That's a tough to swallow. that almost leaves me without
> a grounds to discuss this topic further with you.
>
> Just one question: Is there a clear specification anywhere that,
> in your opinion, compiler writers should adhere to, some sort of
> additional standard document (approved by whom?) that clearly,
> explicitly and unabiguously defines the areas left undefined by
> the C standard?
>
> Or is it restricted to the bugs that Linus Torvalds and yourself
> find irritating?

And that is the key problem with "it should work like it always worked"
or "it should work like I expect it to work" attitudes. How is one
compiler supposed to guess how a different compiler worked? How is a
compiler supposed to read your mind? And are new tools supposed to be
limited by mistakes other people made in the past?

>
>>> The consequence is clear - compiler writers should stop fixing
>>> bugs because fixing a bug might change the observable behavior of
>>> existing programs, which is bad.
>>
>> Somehow you took a wrong turn here. The issue is not whether it *might*
>> change the observable behaviour
>
> Trivially, any wrong-code bug changes observable behavior.
>
>> (that's Bedenkentraeger thinking), but
>> whether it *does*, and whether the user experiences that as regression.
>
> Users complain about compiler bugs all the time. Sometimes they
> are right, most of the time they are not.
>

That is my experience.

Just today, there was a post on the gcc development mailing list by
someone suspecting a compiler bug, when the issue was that their code
assumed signed integer arithmetic has two's complement wrapping semantics.

I don't actually do any development work on gcc (I've never had the time
to get into it), but I sometimes answer posts like that on the mailing
list. The bug is not in the compiler, but in the programmer's
understanding of the language - and hopefully that is now fixed.

>> Note that Linux bugs are fixed all the time, despite the "We don't
>> break userspace" rule in Linux.
>
> We are talking compilers here.
>

(Actually, I would we are mostly talking about C standard libraries -
"toolchains" would be the best term. But with that fix, I agree with
your point.)

On 19/09/2021 22:41, Stefan Monnier wrote:
>> There is no specification that a compiler writer could follow.
>> The respective language standards are irrelevant. Past behavior
>> is all that counts, and pointing out that his program is errnoneous
>> is a no-no.
>>
>> The consequence is clear - compiler writers should stop fixing
>> bugs because fixing a bug might change the observable behavior of
>> existing programs, which is bad.
>
> I think Mitch had another conclusion which I would paraphrase as
> "a serious API shouldn't have any `undefined` part".
>

I don't think that concept makes any sense without a clear definition of
what is meant by an "API". On a boundary between significantly separate
parts of a system, such as system calls to the OS or a library
interface, it can make sense to check parameters thoroughly - and thus
give defined behaviour to invalid parameters. It could be worth the
cost there, and you can define an error-handling strategy (like "crash
the program with an error message").

For internal functions, that would be a pointless inefficiency. And the
C standard library functions are internal to the language.

There is little point in checking for mistakes in parameters or
operands, if you don't have anything sensible to do in the event of an
error. For some languages, there may be appropriate error handling
methods (such as exceptions, or printing run-time error messages on the
screen). For C, there is no method that could reasonably be specified.

I am strongly against the idea of defining behaviour just so that you
can say it is defined and not undefined. If there is a "right" answer
for a given set of inputs, then that should be part of the specification
of the function - if there is no right answer, then defining a wrong
answer helps no one. It just limits optimisers and implementers, and
more importantly it limits any static or run-time debugging aids.

When I was taught programming, we learned that functions (or operators,
or whatever) have a pre-condition on the inputs, and a post-condition on
the outputs. The user guarantees that the inputs will match the
pre-condition, and in return the function guarantees that the
post-condition will apply when the function returns. If you don't match
the pre-conditions, the fault lies with the caller. If the
pre-conditions match but the post-condition does not, the bug is in the
function. Clear and simple.

It certainly beats "functions should do what /I/ think is sensible no
matter what I ask of them".

On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
> On 19/09/2021 13:38, Anton Ertl wrote:
> > Thomas Koenig <tko...@netcologne.de> writes:
> >> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
> >>
> >>> Unless they want to have problems like the glibc maintainers
> >>> encountered <https://bugzilla.redhat.com/show_bug.cgi?id=638477>
> >>> <https://sourceware.org/bugzilla/show_bug.cgi?id=12518> when they
> >>> relied on their "it's undefined behaviour" fantasy, they better
> >>> should.
> >>
> >> So, if memcpy happens to work like memmove for a certain implementation,
> >> it should work that way forever?
> >
> > Exactly.
> >
> > However, my educated guess at what happened is: memcpy()
> > implementations up to that time were using forward stride, and were
> > safe for overlapping copies to lower addresses, so some uses of this
> > kind appeared in code, of course; after all, the code passed the
> > tests.
> >
> > Then Ulrich Drepper thought it was a good idea to use backwards
> > stride, and that code broke; and his really great idea was to make the
> > stride dependent on the CPU model at run-time. So it might test fine
> > on the original developer's machine, and break on a user's machine,
> > which is instruction-set compatible with the developer's machine, only
> > library-incompatible. This would have stayed a minor episode if
> > Ulrich Drepper had accepted the bug report and fixed the bug, instead
> > of being an obstinate fool; admittedly he was not alone, there were
> > (and are still) lots of fans of undefined behaviour, but that's no
> > excuse.
> Imagine the audacity! He made an implementation of the function that
> does what the standard says it should do, and does it in the fastest way
> it can on the machine running the code!

May be, fastest on his particular machine with his set of tests.
IIRC, according to Linus, it was not faster on other machines or with other sets of tests.

> Any programmer who does that
> should, of course, be fired - their job is to read the minds of every
> third-rate coder out there and figure out what that person /meant/ to
> write, with a total disregard for the efficiency of code written by
> people who know what they are doing.
>
> Still, in this particular case I do understand that there are so many
> cases where people have got this wrong, and used memcpy when they meant
> memmove,

I am afraid, it's worse.
Those people used memcpy() when they meant
for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
And they were relying on replication effect in case when src < dst < src+len.

> and that the efficiency cost for making memcpy act like memmove
> is so low, that it is probably best to pander to the erroneous code and
> make memcpy work like memmove. Libraries could perhaps include extra
> functions like "__memcpy_disjoint", "__memmove_forward",
> "__memmove_backward" that are particularly optimised, and that a
> compiler like gcc can call rather than general "memmove" when asked for
> a "memmove" or "memcpy" and it knows something about the relationships
> between the pointers.
> >
> > I suffer from the fallout of this to this day: When I build Gforth on
> > a recent system, and try to run it on an ancient one, it does not run,
> > complaining that the binary needs a newer version of the glibc .so
> > than the old system has. That's because Ulrich Drepper's solution to
> > the problem was to have two versions of memcpy() for newer glibc: for
> > old binaries, memcpy() does memmove(); for newer binaries, memcpy()
> > does something else. And of course the linker magic that they use for
> > that is not set up to fall back to the old memcpy() when trying to
> > link a newer binary with an old library, even though the new memcpy()
> > does not give any better guarantees than the old one.
>
> That does sound a messy "solution".
>
> >
> >> There is no justification for having memcpy at all, then.
> >
> > There isn't. Undefined behaviour (and therefore memcpy()) is just a
> > bad idea in a world where testing is the an accepted methodology for
> > finding bugs in programs.
> >
>
> On that, we disagree. Undefined behaviour is a natural part of computer
> science (and the mathematics behind it), and attempts to remove it from
> languages always come at a cost (though the cost is not always significant).
> >> Just use
> >> -Dmemcpy=memmove and be done with it.
> >
> > Unfortunately, that's not sufficient to get rid of memcpy(); we tried,
> > but the library incompatibility problem outlined above persisted.
> > IIRC gcc produces calls to memcpy() when you copy some structs or
> > somesuch.
> >
> > - anton
> >

On 20/09/2021 12:18, Michael S wrote:
> On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
>> On 19/09/2021 13:38, Anton Ertl wrote:
>>> Thomas Koenig <tko...@netcologne.de> writes:
>>>> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
>>>>
>>>>> Unless they want to have problems like the glibc maintainers
>>>>> encountered <https://bugzilla.redhat.com/show_bug.cgi?id=638477>
>>>>> <https://sourceware.org/bugzilla/show_bug.cgi?id=12518> when they
>>>>> relied on their "it's undefined behaviour" fantasy, they better
>>>>> should.
>>>>
>>>> So, if memcpy happens to work like memmove for a certain implementation,
>>>> it should work that way forever?
>>>
>>> Exactly.
>>>
>>> However, my educated guess at what happened is: memcpy()
>>> implementations up to that time were using forward stride, and were
>>> safe for overlapping copies to lower addresses, so some uses of this
>>> kind appeared in code, of course; after all, the code passed the
>>> tests.
>>>
>>> Then Ulrich Drepper thought it was a good idea to use backwards
>>> stride, and that code broke; and his really great idea was to make the
>>> stride dependent on the CPU model at run-time. So it might test fine
>>> on the original developer's machine, and break on a user's machine,
>>> which is instruction-set compatible with the developer's machine, only
>>> library-incompatible. This would have stayed a minor episode if
>>> Ulrich Drepper had accepted the bug report and fixed the bug, instead
>>> of being an obstinate fool; admittedly he was not alone, there were
>>> (and are still) lots of fans of undefined behaviour, but that's no
>>> excuse.
>> Imagine the audacity! He made an implementation of the function that
>> does what the standard says it should do, and does it in the fastest way
>> it can on the machine running the code!
>
> May be, fastest on his particular machine with his set of tests.
> IIRC, according to Linus, it was not faster on other machines or with other sets of tests.
>

I don't know the details of the machines in question. But as I
understand it, on some processors it was faster to copy the memory
backwards rather than forwards. So the library routine would adapt to
the machine running the code, and run the copies forwards or backwards
as appropriate. (Library routines often have different versions that
are picked according to the target machine - as do some functions in
Linux. The RAID6 routines, for example, are provided in multiple
versions that are picked at boot time.)

>> Any programmer who does that
>> should, of course, be fired - their job is to read the minds of every
>> third-rate coder out there and figure out what that person /meant/ to
>> write, with a total disregard for the efficiency of code written by
>> people who know what they are doing.
>>
>> Still, in this particular case I do understand that there are so many
>> cases where people have got this wrong, and used memcpy when they meant
>> memmove,
>
> I am afraid, it's worse.
> Those people used memcpy() when they meant
> for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
> And they were relying on replication effect in case when src < dst < src+len.
>

That is what memmove is for. Memmove works even if the source and
destination overlap.

On Monday, September 20, 2021 at 1:40:03 PM UTC+3, David Brown wrote:
> On 20/09/2021 12:18, Michael S wrote:
> > On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
> >> On 19/09/2021 13:38, Anton Ertl wrote:
> >>> Thomas Koenig <tko...@netcologne.de> writes:
> >>>> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
> >>>>
> >>>>> Unless they want to have problems like the glibc maintainers
> >>>>> encountered <https://bugzilla.redhat.com/show_bug.cgi?id=638477>
> >>>>> <https://sourceware.org/bugzilla/show_bug.cgi?id=12518> when they
> >>>>> relied on their "it's undefined behaviour" fantasy, they better
> >>>>> should.
> >>>>
> >>>> So, if memcpy happens to work like memmove for a certain implementation,
> >>>> it should work that way forever?
> >>>
> >>> Exactly.
> >>>
> >>> However, my educated guess at what happened is: memcpy()
> >>> implementations up to that time were using forward stride, and were
> >>> safe for overlapping copies to lower addresses, so some uses of this
> >>> kind appeared in code, of course; after all, the code passed the
> >>> tests.
> >>>
> >>> Then Ulrich Drepper thought it was a good idea to use backwards
> >>> stride, and that code broke; and his really great idea was to make the
> >>> stride dependent on the CPU model at run-time. So it might test fine
> >>> on the original developer's machine, and break on a user's machine,
> >>> which is instruction-set compatible with the developer's machine, only
> >>> library-incompatible. This would have stayed a minor episode if
> >>> Ulrich Drepper had accepted the bug report and fixed the bug, instead
> >>> of being an obstinate fool; admittedly he was not alone, there were
> >>> (and are still) lots of fans of undefined behaviour, but that's no
> >>> excuse.
> >> Imagine the audacity! He made an implementation of the function that
> >> does what the standard says it should do, and does it in the fastest way
> >> it can on the machine running the code!
> >
> > May be, fastest on his particular machine with his set of tests.
> > IIRC, according to Linus, it was not faster on other machines or with other sets of tests.
> >
> I don't know the details of the machines in question. But as I
> understand it, on some processors it was faster to copy the memory
> backwards rather than forwards. So the library routine would adapt to
> the machine running the code,

In theory.
In practice the change was applied too widely and there was sizeable amount of performance regressions.
And amount of cases with measurable improvement was not nearly as big as claimed by the person that introduced the change.

> and run the copies forwards or backwards
> as appropriate. (Library routines often have different versions that
> are picked according to the target machine - as do some functions in
> Linux. The RAID6 routines, for example, are provided in multiple
> versions that are picked at boot time.)
> >> Any programmer who does that
> >> should, of course, be fired - their job is to read the minds of every
> >> third-rate coder out there and figure out what that person /meant/ to
> >> write, with a total disregard for the efficiency of code written by
> >> people who know what they are doing.
> >>
> >> Still, in this particular case I do understand that there are so many
> >> cases where people have got this wrong, and used memcpy when they meant
> >> memmove,
> >
> > I am afraid, it's worse.
> > Those people used memcpy() when they meant
> > for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
> > And they were relying on replication effect in case when src < dst < src+len.
> >
> That is what memmove is for. Memmove works even if the source and
> destination overlap.

No, memmove has no replication effect that they wanted.

On Monday, September 20, 2021 at 10:12:07 AM UTC+3, Terje Mathisen wrote:
> Michael S wrote:
> > On Sunday, September 19, 2021 at 3:09:39 PM UTC+3, Terje Mathisen
> > wrote:
> >> Anton Ertl wrote:
> >>> Terje Mathisen <terje.m...@tmsw.no> writes:
> >>>> This problem has almost nothing to do with REP MOVS, the
> >>>> two-byte version should have been solved with code that
> >>>> replicated the pattern into an 8-byte register, then storing
> >>>> that repeatedly.
> >>>
> >>> There are alternative ways to do it, yes, but that's beside the
> >>> point. So we already went from an implementation that uses REP
> >>> MOVSB for an overlapping case to an implementation that uses it
> >>> only for the non-overlapping case (i.e., use REP MOVSB for
> >>> memcpy() functionality). And it's still much slower than a not
> >>> particularly sophisticated implementation of the same
> >>> functionality with simpler instructions.
> >>>
> >>> Every time you write that one should use something else supports
> >>> the position that CISCy instructions are not as great as Brett
> >>> thinks.
> >> I do agree that a fast REP MOVS should be able to handle all cases
> >> efficiently, including the one where it is used to unpack an
> >> RLL-encoded string by replicating a pattern.
> >>
> >> However, I don't see any way to make this faster than the cache
> >> hierarchy/forwarding network would let you repeatedly re-read and
> >> write to the same cache line, and if the pattern has a short
> >> length, then the "fast strings" hw must in fact detect this
> >> situation and use it to internally unroll/replicate the pattern and
> >> turn it into the same code I wrote by hand, i.e. a repeated block
> >> store.
> >>
> >> Until that happens I would be quite happy with simply having a
> >> fast non-overlapping copy that matches or beats individual
> >> hand-coded instructions, even if the partly overlapping case is
> >> reduced to running at a byte or two per cycle.
> >>
> >> Terje -- - <Terje.Mathisen at tmsw.no> "almost all programming can
> >> be viewed as an exercise in caching"
> >
> > For copies of small integer count of words (where 'word' is either of
> > GPR size or of VR size) where count is known at compile time, your
> > performance expectation is unrealistic.
> >
>
> Urealistic, yeah. Impossible? Maybe not.
>

I'd dare to say that if it is possible in general case then underlying OoO machinery is not particularly good.

> The main issue is in fact the need to setup the block move opcode, i.e.
> you need at least to load a register with the # of bytes to move, even
> if both source and dest address already exists in registers usable by
> the block move instruction.

IMHO, the main issue is that memcpy instruction can't start until at least two out of three pieces of the puzzle,
i.e src and count, are in place. In practice, in order to reduce complexity, implementation will likely wait for dst, too.
On the other hand, load+store based fixed-length copy needs only src to initiate loads.

Another factor is related to compiler's optimization.
For such short copies it's quite common that source-side data is already in registers, either partially or completely.
Compiler can and should take advantage of that, but semantics of memcpy instruction preclude it.

>
> I would however be perfectly happy with a comiler which generated a pair
> of MOVs for small struct copies and fell back on REP MOVS for
> longer/variable length blocks.
>

IMHO, even the best implementation of memcpy instruction would be slower than individual load/store not just for
"a pair of MOVs", but for a dozen of moves as well. At least, for a dozen of MOVs it likely would be more compact.
Not so for 2 or 4 MOVs.
For cases of 6 and 8 moves, what variant is more compact would likely depend on details surrounding code.

> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

On 20/09/2021 13:00, Michael S wrote:
> On Monday, September 20, 2021 at 1:40:03 PM UTC+3, David Brown
> wrote:
>> On 20/09/2021 12:18, Michael S wrote:
>>> On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown
>>> wrote:
>>>> On 19/09/2021 13:38, Anton Ertl wrote:
>>>>> Thomas Koenig <tko...@netcologne.de> writes:
>>>>>> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
>>>>>>
>>>>>>> Unless they want to have problems like the glibc
>>>>>>> maintainers encountered
>>>>>>> <https://bugzilla.redhat.com/show_bug.cgi?id=638477>
>>>>>>> <https://sourceware.org/bugzilla/show_bug.cgi?id=12518>
>>>>>>> when they relied on their "it's undefined behaviour"
>>>>>>> fantasy, they better should.
>>>>>>
>>>>>> So, if memcpy happens to work like memmove for a certain
>>>>>> implementation, it should work that way forever?
>>>>>
>>>>> Exactly.
>>>>>
>>>>> However, my educated guess at what happened is: memcpy()
>>>>> implementations up to that time were using forward stride,
>>>>> and were safe for overlapping copies to lower addresses, so
>>>>> some uses of this kind appeared in code, of course; after
>>>>> all, the code passed the tests.
>>>>>
>>>>> Then Ulrich Drepper thought it was a good idea to use
>>>>> backwards stride, and that code broke; and his really great
>>>>> idea was to make the stride dependent on the CPU model at
>>>>> run-time. So it might test fine on the original developer's
>>>>> machine, and break on a user's machine, which is
>>>>> instruction-set compatible with the developer's machine, only
>>>>> library-incompatible. This would have stayed a minor episode
>>>>> if Ulrich Drepper had accepted the bug report and fixed the
>>>>> bug, instead of being an obstinate fool; admittedly he was
>>>>> not alone, there were (and are still) lots of fans of
>>>>> undefined behaviour, but that's no excuse.
>>>> Imagine the audacity! He made an implementation of the function
>>>> that does what the standard says it should do, and does it in
>>>> the fastest way it can on the machine running the code!
>>>
>>> May be, fastest on his particular machine with his set of tests.
>>> IIRC, according to Linus, it was not faster on other machines or
>>> with other sets of tests.
>>>
>> I don't know the details of the machines in question. But as I
>> understand it, on some processors it was faster to copy the memory
>> backwards rather than forwards. So the library routine would adapt
>> to the machine running the code,
>
> In theory. In practice the change was applied too widely and there
> was sizeable amount of performance regressions. And amount of cases
> with measurable improvement was not nearly as big as claimed by the
> person that introduced the change.
>

Code for routines like memcpy always has varying speeds. Any
implementation that will be good for large copies is going to have a lot
of setup, and thus be inefficient for small copies. No matter what you
do, any change is going to have regressions for some cases even if it
speeds up others. And it is extremely difficult to define an "average"
case to get good weightings and balance. (I have no idea about the
actual cases involved here.)

>> and run the copies forwards or backwards as appropriate. (Library
>> routines often have different versions that are picked according to
>> the target machine - as do some functions in Linux. The RAID6
>> routines, for example, are provided in multiple versions that are
>> picked at boot time.)
>>>> Any programmer who does that should, of course, be fired -
>>>> their job is to read the minds of every third-rate coder out
>>>> there and figure out what that person /meant/ to write, with a
>>>> total disregard for the efficiency of code written by people
>>>> who know what they are doing.
>>>>
>>>> Still, in this particular case I do understand that there are
>>>> so many cases where people have got this wrong, and used memcpy
>>>> when they meant memmove,
>>>
>>> I am afraid, it's worse. Those people used memcpy() when they
>>> meant for (int i=0; i < len, ++i)
>>> ((char*)dst)[i]=((char*)src)[i]; And they were relying on
>>> replication effect in case when src < dst < src+len.
>>>
>> That is what memmove is for. Memmove works even if the source and
>> destination overlap.
>
> No, memmove has no replication effect that they wanted.
>

Sorry, I had misunderstood you. Yes, that is even worse if people were
relying on a particular undefined and undocumented behaviour of a
particular memcpy() implementation when it is used outside of its
specifications.

On Monday, September 20, 2021 at 1:21:03 AM UTC+3, MitchAlsup wrote:
> On Sunday, September 19, 2021 at 4:30:47 PM UTC-5, Michael S wrote:
> > On Sunday, September 19, 2021 at 10:21:46 PM UTC+3, Anton Ertl wrote:
> > > Michael S <already...@yahoo.com> writes:
> > > >On Sunday, September 19, 2021 at 8:38:28 PM UTC+3, Anton Ertl wrote:
> > > >> It is an issue: When I do static linking, the binary does not work on
> > > >> the old system, either, because the new glibc does not work on old
> > > >> kernels.
> > > ...
> > > >Then do dynamic linking and supply .so together with executive.
> > > What makes you think that the new library will work on an old kernel
> > > when dynamically linked, if it does not when statically linked?
> > > - anton
> > > --
> > > 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> > > Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
> > My hope was that kernel<->user interface is the most stable part of any OS.
> <
> You might be a comedian on TV !! if you can continuously come up with
> lines like that.

I disconnected my home TV set almost two years ago.
Was not watching comedians on TV for much longer that that.
And never ever watched American TV comedians, in part because my American English is not particularly good, but I'd guess that even if it was good I wouldn't find them funny because of differences in cultural background.

So, I don't quite get your comment; can't even figure out if it is a criticism or praise.

On Monday, September 20, 2021 at 10:52:34 AM UTC+3, Thomas Koenig wrote:
> MitchAlsup <Mitch...@aol.com> schrieb:
> > On Sunday, September 19, 2021 at 5:38:51 PM UTC-5, Thomas Koenig wrote:
> >> MitchAlsup <Mitch...@aol.com> schrieb:
> >> > On Sunday, September 19, 2021 at 5:16:03 PM UTC-5, Thomas Koenig wrote:
> >> >> Stefan Monnier <mon...@iro.umontreal.ca> schrieb:
> >> >> >> There is no specification that a compiler writer could follow.
> >> >> >> The respective language standards are irrelevant. Past behavior
> >> >> >> is all that counts, and pointing out that his program is errnoneous
> >> >> >> is a no-no.
> >> >> >>
> >> >> >> The consequence is clear - compiler writers should stop fixing
> >> >> >> bugs because fixing a bug might change the observable behavior of
> >> >> >> existing programs, which is bad.
> >> >> >
> >> >> > I think Mitch had another conclusion which I would paraphrase as
> >> >> > "a serious API shouldn't have any `undefined` part".
> >> >> Mandatory bounds checking? Mandatory checking for -32768, with
> >> >> a specification what happens with a bounds overrun? Exact
> >> >> language specification of what happens if memory on the system
> >> >> runs out (i.e. no memory overcommit on Linux)?
> >> >>
> >> >> And even so, according to Linus T's definition, if code like
> >> >>
> >> >> int x[10];
> >> >> int a;
> >> >>
> >> >> ...
> >> >>
> >> >> x[10] = 42;
> >> >> assert (a == 42);
> >> >>
> >> >> ever "worked" on any system, it would have to be kept that way -
> >> >> mandating stack layout of variables, keeping a from being kept in
> >> >> a register etc.
> >> >>
> >> >> Is that really what people want?
> >> ><
> >> > No, people want that kind of stuff found at compile time.
> >> Not possible in the more general case, as you know.
> ><
> > ADA does not seem to have this problem. Perhaps because they actually care.
> You wrote "at compile time". Testing the correctnes of a program
> written in Turing-complete languge is the equivalent of solving
> the halting problem.
>
> Besides:
>
> Does Ada catch the equivalent of C's
>
> int x[10];
> int i;
> scanf ("%d", &i);
> x[i] = 42;
>
> at compile-time because x[i] might overflow?
>
> At run-time is a different kettle of fish, of course. Even so,
> how does Ada guarantee that there is no overcommitment of memory
> on Linux, the OOM killer comes along and kills the process?

I think, that's not what Mitch meant.
Seems like he meant that in Ada compiler can guarantee, in compile time, that code below never prints Hello.
So, there is no chance that code that relies on printing hello would be ever shipped.

int x[10];
int a=41;
int i;
scanf ("%d", &i);
x[i] = 42;
if (a==42) printf("Hello\n");

MitchAlsup wrote:
> On Sunday, September 19, 2021 at 2:20:16 PM UTC-5, EricP wrote:
>> MitchAlsup wrote:
>>> On Sunday, September 19, 2021 at 11:19:10 AM UTC-5, EricP wrote:
>>>> MOVC3 accepts arbitrary registers to specify the length in bytes,
>>> <
>>> Could the length be provided by an immediate ? If not why ?
>> Yes.
> <
> Then how is MOVC3 interruptable ?
> I can see setting FirstPartDone, and updating the address pointers,
> but how does one update the constant ? Or is this why R0..R5 get consumed
> in the execution of MOVC3 ?

In order to perform an interruptible, overlap-safe, MOVC3 (MemMove)
that after completion leaves the source and dest address registers
pointing at the next bytes to be read and written, requires they
save 4 data items: the start or current source and dest addresses,
the original buffer size, and the current copied count.

PSW.FPD First Part Done flag is normally clear.

At MOVC3 instruction start it copies the args into R0..R5 then
sets PSW.FPD and does not increment the program counter.
R0,R1 = len1,addr1 R2,R3 = len2,addr2 R4,R5 = len3,addr3

After that it uses temp regs until lengths = 0 then
clear PSW.FPD and increments PC.

On interrupt or exception R0..R5 are updated for changes and
the non-incremented PC and PSW are saved on stack.
On Return From Interrupt the PC & PSW is restored and it re-fetches the
MOVC instruction. It checks the PSW.FPD and finds it set so skips
copying args into R0..R5, just uses them as is.

There are details like auto-increment of arg registers that are
likely also controlled by PSW.FPD too as the source args must be
evaluated serially and copied before execution of the MOVC starts
in case there are arg register or memory dependencies.

So the instruction args must be evaluated, serially, once at the start
and not again as you might be MOVC'ing over top of some args that use
auto-increment double indirect addressing (with byte overlaps).

Of course the whole thing could have been simplified if they had
just defined that the string instruction args are in registers R0..R5.
But that would not be the VAX way.

Michael S wrote:
> On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
>> Any programmer who does that
>> should, of course, be fired - their job is to read the minds of every
>> third-rate coder out there and figure out what that person /meant/ to
>> write, with a total disregard for the efficiency of code written by
>> people who know what they are doing.
>>
>> Still, in this particular case I do understand that there are so many
>> cases where people have got this wrong, and used memcpy when they meant
>> memmove,
>
> I am afraid, it's worse.
> Those people used memcpy() when they meant
> for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
> And they were relying on replication effect in case when src < dst < src+len.

No, they were like me who read many years ago that the code above is
_exactly_ how memcpy() is supposed to work.

I.e. the pointer-based version below is more or less the definition from
my textbooks (modulo void vs char, return value etc):

void memcpy(char *dst, char *src, int len)
{ while (len--) *dst++ = *src++;
}

This would be in the same chapter as strcpy() where the loop would be
soemthing like:

do { char c = *dst++ = *src++; } while (c);

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

David Brown wrote:
> On 19/09/2021 19:50, EricP wrote:
>> MitchAlsup wrote:
>>> On Sunday, September 19, 2021 at 7:14:52 AM UTC-5, Anton Ertl wrote:
>>>> Unfortunately, that's not sufficient to get rid of memcpy(); we
>>>> tried, but the library incompatibility problem outlined above
>>>> persisted. IIRC gcc produces calls to memcpy() when you copy some
>>>> structs or somesuch.
>>> <
>>> Yes but !!
>>> <
>>> a structure assignment: s1 = s2; is never in a position where s1 and
>>> s2 overlap; and therefore nobody cares which direction, or how wide, any
>>> data transfer happens to be.
>> An array of S's can be shifting elements with overlapping slices.
>>
>> If s1 and s2 are structs in different areas of a union and passed
>> to a subroutine void DoCopy (struct S *s1, struct S *s2);
>>
>> union
>> { struct S s1;
>> struct
>> { int i;
>> struct S s2;
>> } foo;
>> } u;
>>
>> DoCopy cannot prove its args do not overlap so should use memmove.
>>
>
> I think there is a subtle point here.
>
> I did some testing with:
>
>
> struct S { int a; int b[103]; };
>
> union
> { struct S s1;
> struct
> { int i;
> struct S s2;
> } foo;
> } u;
>
> void foo1(void) {
> u.s1 = u.foo.s2;
> }
>
> void foo2(void) {
> u.foo.s2 = u.s1;
> }
>
> void DoCopy (struct S *s1, struct S *s2) {
> *s1 = *s2;
> }
>
>
> Testing with gcc and clang shows that they both use memcpy (or a "rep
> movsq" sequence on x86 that I /think/ assumes there is no overlap - but
> I am not an expert on x86 assembly).
>
> It's not often that the gcc folk are wrong about this kind of thing.
> (Overly pedantic or unhelpful, perhaps, but not wrong.) It's even rarer
> for both gcc and clang to be wrong.
>
>
> I believe the answer is in the C standards, 6.5.16.1p3, regarding the
> semantics of the assignment operator:
>
> """
> If the value being stored in an object is read from another object that
> overlaps in any way the storage of the first object, then the overlap
> shall be exact and the two objects shall have qualified or unqualified
> versions of a compatible type; otherwise, the behavior is undefined.
> """
>
> So if you have "s1 = s2", then either they do not overlap at all, or
> they are the same object (in which case memcpy will still be fine in
> practice).

Try it with -fno-strict-aliasing.
It should switch to the equivalent of memove which would test
if (source_addr < dest_addr) and CLD/STD clear/set the direction flag
before REP MOVSQ.

From what I've read, Linux and other OS's must compile
with -fno-strict-aliasing to function as programmers intended.

Most of my applications would too if my MS compiler
didn't assume this anyway.

On Monday, September 20, 2021 at 3:53:36 PM UTC+3, Terje Mathisen wrote:
> Michael S wrote:
> > On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
> >> Any programmer who does that
> >> should, of course, be fired - their job is to read the minds of every
> >> third-rate coder out there and figure out what that person /meant/ to
> >> write, with a total disregard for the efficiency of code written by
> >> people who know what they are doing.
> >>
> >> Still, in this particular case I do understand that there are so many
> >> cases where people have got this wrong, and used memcpy when they meant
> >> memmove,
> >
> > I am afraid, it's worse.
> > Those people used memcpy() when they meant
> > for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
> > And they were relying on replication effect in case when src < dst < src+len.
> No, they were like me who read many years ago that the code above is
> _exactly_ how memcpy() is supposed to work.
>
> I.e. the pointer-based version below is more or less the definition from
> my textbooks (modulo void vs char, return value etc):
>

What is a name of the textbook?

> void memcpy(char *dst, char *src, int len)
> {
> while (len--) *dst++ = *src++;
> }
>
> This would be in the same chapter as strcpy() where the loop would be
> soemthing like:
>
> do { char c = *dst++ = *src++; } while (c);
> Terje
>
> --
> - <Terje.Mathisen at tmsw.no>
> "almost all programming can be viewed as an exercise in caching"

Michael S wrote:
> On Monday, September 20, 2021 at 1:40:03 PM UTC+3, David Brown wrote:
>> On 20/09/2021 12:18, Michael S wrote:
>>> On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
>>>>
>>>> Still, in this particular case I do understand that there are so many
>>>> cases where people have got this wrong, and used memcpy when they meant
>>>> memmove,
>>> I am afraid, it's worse.
>>> Those people used memcpy() when they meant
>>> for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
>>> And they were relying on replication effect in case when src < dst < src+len.
>>>
>> That is what memmove is for. Memmove works even if the source and
>> destination overlap.
>
> No, memmove has no replication effect that they wanted.

The definition in the C12 standard says

"The memmove function copies n characters from the object pointed to
by s2 into the object pointed to by s1. Copying takes place as if
the n characters from the object pointed to by s2 are first copied
into a temporary array of n characters that does not overlap the
objects pointed to by s1 and s2, and then the n characters from the
temporary array are copied into the object pointed to by s1."

That definition would seem to explicitly *exclude* replication.

Is there any implementation of memmove that actually works by
using a temp array for overlap? Certainly the REP MOVS don't do this.

On Monday, September 20, 2021 at 4:53:34 PM UTC+3, EricP wrote:
> Michael S wrote:
> > On Monday, September 20, 2021 at 1:40:03 PM UTC+3, David Brown wrote:
> >> On 20/09/2021 12:18, Michael S wrote:
> >>> On Monday, September 20, 2021 at 11:13:14 AM UTC+3, David Brown wrote:
> >>>>
> >>>> Still, in this particular case I do understand that there are so many
> >>>> cases where people have got this wrong, and used memcpy when they meant
> >>>> memmove,
> >>> I am afraid, it's worse.
> >>> Those people used memcpy() when they meant
> >>> for (int i=0; i < len, ++i) ((char*)dst)[i]=((char*)src)[i];
> >>> And they were relying on replication effect in case when src < dst < src+len.
> >>>
> >> That is what memmove is for. Memmove works even if the source and
> >> destination overlap.
> >
> > No, memmove has no replication effect that they wanted.
> The definition in the C12 standard says
>
> "The memmove function copies n characters from the object pointed to
> by s2 into the object pointed to by s1. Copying takes place as if
> the n characters from the object pointed to by s2 are first copied
> into a temporary array of n characters that does not overlap the
> objects pointed to by s1 and s2, and then the n characters from the
> temporary array are copied into the object pointed to by s1."
>
> That definition would seem to explicitly *exclude* replication.
>

That what I said above. David Brown undoubtedly knows it as well.

> Is there any implementation of memmove that actually works by
> using a temp array for overlap? Certainly the REP MOVS don't do this.