EricP wrote:
> David Brown wrote:
>>
>> It makes no difference (which is not a surprise, since it is not an
>> aliasing issue - the types of s1 and s2 are the same).
>
> - For MS compiler, no matter what options I tried, the assembler for
> DoCopy1, DoCopy2, DoCopy3 are always identical and the same as
> foo1 and foo2.
>
> - For foo1, foo2, DoCopy1, DoCopy2, DoCopy3 it always assumes the
> structs are not overlapped and either generates a memcpy as
> REP MOVSB
> with no direction test, so forward only,
> or an inline move sequence using XMM registers,
> again ignoring the byte offset overlap.
>
> - For DoCopy4 it generates a jump to memmove library routine.
>
> Bottom line: MS VC compiler is ignoring all memory aliasing,
> even in a union where it should know that it overlaps,
> and generating moves that assume no overlap.
>
>
> I see reference on StackOverflow that MS removed the compile option
> Assume No aliasing /Oa in VS 2008 and replaced it with
> __declspec(restrict) and __declspec(noalias) directives.
>
> Optimization best practices
> https://docs.microsoft.com/en-us/cpp/build/optimization-best-practices?view=msvc-160
>
>
> It seems they think that it should assume memory aliasing unless
> explicitly told not to, though it does not appear to function that way.
>
> I don't know why this hasn't broken all Win32, kernel and driver code.

I tried this on my 10+ year old MSVC compiler and it works the same but
without the recent optimizations, that is it uses a forward REP MOVSB
except for DoCopy4.

I can only assume that everyone who want to do block struct moves
uses memmove not *ptr = *ptr.

On Wednesday, September 22, 2021 at 12:40:19 AM UTC+3, EricP wrote:
> EricP wrote:
> > David Brown wrote:
> >>
> >> It makes no difference (which is not a surprise, since it is not an
> >> aliasing issue - the types of s1 and s2 are the same).
> >
> > - For MS compiler, no matter what options I tried, the assembler for
> > DoCopy1, DoCopy2, DoCopy3 are always identical and the same as
> > foo1 and foo2.
> >
> > - For foo1, foo2, DoCopy1, DoCopy2, DoCopy3 it always assumes the
> > structs are not overlapped and either generates a memcpy as
> > REP MOVSB
> > with no direction test, so forward only,
> > or an inline move sequence using XMM registers,
> > again ignoring the byte offset overlap.
> >
> > - For DoCopy4 it generates a jump to memmove library routine.
> >
> > Bottom line: MS VC compiler is ignoring all memory aliasing,
> > even in a union where it should know that it overlaps,
> > and generating moves that assume no overlap.
> >
> >
> > I see reference on StackOverflow that MS removed the compile option
> > Assume No aliasing /Oa in VS 2008 and replaced it with
> > __declspec(restrict) and __declspec(noalias) directives.
> >
> > Optimization best practices
> > https://docs.microsoft.com/en-us/cpp/build/optimization-best-practices?view=msvc-160
> >
> >
> > It seems they think that it should assume memory aliasing unless
> > explicitly told not to, though it does not appear to function that way.
> >
> > I don't know why this hasn't broken all Win32, kernel and driver code.
> I tried this on my 10+ year old MSVC compiler and it works the same but
> without the recent optimizations, that is it uses a forward REP MOVSB
> except for DoCopy4.
>
> I can only assume that everyone who want to do block struct moves
> uses memmove not *ptr = *ptr.

More likely, nobody uses overlapped structures of the same type.
Can you give me one reason why anybody would?

On Tuesday, September 21, 2021 at 4:40:19 PM UTC-5, EricP wrote:
> EricP wrote:
> > David Brown wrote:
> >>
> >> It makes no difference (which is not a surprise, since it is not an
> >> aliasing issue - the types of s1 and s2 are the same).
> >
> > - For MS compiler, no matter what options I tried, the assembler for
> > DoCopy1, DoCopy2, DoCopy3 are always identical and the same as
> > foo1 and foo2.
> >
> > - For foo1, foo2, DoCopy1, DoCopy2, DoCopy3 it always assumes the
> > structs are not overlapped and either generates a memcpy as
> > REP MOVSB
> > with no direction test, so forward only,
> > or an inline move sequence using XMM registers,
> > again ignoring the byte offset overlap.
> >
> > - For DoCopy4 it generates a jump to memmove library routine.
> >
> > Bottom line: MS VC compiler is ignoring all memory aliasing,
> > even in a union where it should know that it overlaps,
> > and generating moves that assume no overlap.
> >
> >
> > I see reference on StackOverflow that MS removed the compile option
> > Assume No aliasing /Oa in VS 2008 and replaced it with
> > __declspec(restrict) and __declspec(noalias) directives.
> >
> > Optimization best practices
> > https://docs.microsoft.com/en-us/cpp/build/optimization-best-practices?view=msvc-160
> >
> >
> > It seems they think that it should assume memory aliasing unless
> > explicitly told not to, though it does not appear to function that way.
> >
> > I don't know why this hasn't broken all Win32, kernel and driver code.
> I tried this on my 10+ year old MSVC compiler and it works the same but
> without the recent optimizations, that is it uses a forward REP MOVSB
> except for DoCopy4.
>
> I can only assume that everyone who want to do block struct moves
> uses memmove not *ptr = *ptr.
<
Brian's compiler compiles *ptr1=*ptr2 into MM instruction
Brian's compiler also compiles memmove(ptr1,ptr2,sizeof(*ptr2)) into MM.
<

On Tuesday, September 21, 2021 at 5:06:32 PM UTC-5, Michael S wrote:
> On Wednesday, September 22, 2021 at 12:40:19 AM UTC+3, EricP wrote:
> > EricP wrote:
> > > David Brown wrote:
> > >>
> > >> It makes no difference (which is not a surprise, since it is not an
> > >> aliasing issue - the types of s1 and s2 are the same).
> > >
> > > - For MS compiler, no matter what options I tried, the assembler for
> > > DoCopy1, DoCopy2, DoCopy3 are always identical and the same as
> > > foo1 and foo2.
> > >
> > > - For foo1, foo2, DoCopy1, DoCopy2, DoCopy3 it always assumes the
> > > structs are not overlapped and either generates a memcpy as
> > > REP MOVSB
> > > with no direction test, so forward only,
> > > or an inline move sequence using XMM registers,
> > > again ignoring the byte offset overlap.
> > >
> > > - For DoCopy4 it generates a jump to memmove library routine.
> > >
> > > Bottom line: MS VC compiler is ignoring all memory aliasing,
> > > even in a union where it should know that it overlaps,
> > > and generating moves that assume no overlap.
> > >
> > >
> > > I see reference on StackOverflow that MS removed the compile option
> > > Assume No aliasing /Oa in VS 2008 and replaced it with
> > > __declspec(restrict) and __declspec(noalias) directives.
> > >
> > > Optimization best practices
> > > https://docs.microsoft.com/en-us/cpp/build/optimization-best-practices?view=msvc-160
> > >
> > >
> > > It seems they think that it should assume memory aliasing unless
> > > explicitly told not to, though it does not appear to function that way.
> > >
> > > I don't know why this hasn't broken all Win32, kernel and driver code.
> > I tried this on my 10+ year old MSVC compiler and it works the same but
> > without the recent optimizations, that is it uses a forward REP MOVSB
> > except for DoCopy4.
> >
> > I can only assume that everyone who want to do block struct moves
> > uses memmove not *ptr = *ptr.
<
> More likely, nobody uses overlapped structures of the same type.
> Can you give me one reason why anybody would?
<
This is the real crux of the mater.
Allowing it one faces the forces of guaranteeing bad code runs properly.
Disallowing it one faces the forces of "why not let me shoot myself in the foot" ?

On Wednesday, September 22, 2021 at 12:12:42 AM UTC+3, MitchAlsup wrote:
> On Tuesday, September 21, 2021 at 11:52:39 AM UTC-5, Michael S wrote:
> > On Monday, September 20, 2021 at 9:34:00 PM UTC+3, Anton Ertl wrote:
> > > Michael S <already...@yahoo.com> writes:
> > > >IMHO, the main issue is that memcpy instruction can't start until at least two out of three pieces of the puzzle,
> > > >i.e src and count, are in place. In practice, in order to reduce complexity, implementation will likely wait for dst, too.
> > > >On the other hand, load+store based fixed-length copy needs only src to initiate loads.
> > > Given that count is a constant in the cases we are discussing here, it
> > > is available early, so at least in that respect both approaches just
> > > have to wait for src.
> > >
> > > I guess with count available as immediate value, a sufficiently
> > > capable front end can produce a sequence of microinstructuons at least
> > > as good as the compiler you have in mind (but better tuned for the
> > > architecture and microarchitecture). At least in theory; the story of
> > > REP MOVSB has not made me optimisitic about such things.
> > > - anton
> > > --
> > > 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> > > Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>
> > Yes, if count is immediate value, then good HW implementation could be as fast as inlined loads+stores even for small counts and aligned src/dst.
> <
> The integer ADD function unit does not know whether the second operand came from a register
> or came from an immediate. Why should the MM function unit know if the length was constant or
> variable ? It seems to me this makes ZERO difference over at the function unit.

I don't care about what happen at the level of function unit. What's matter is performance observed by SW, which depends on both issue and FU.

Also I don't care if MM function unit that handles immediate value of count shares all HW, part of HW or no HW with MM function unit that handles value of count in register, but I expect that in "take no prisoners" implementation it's either 2nd or 3rd option.
That is, assuming that both options exist at ISA level, which is not the case on x86 and unlikely on ARMv9.

BTW, if you want to know my own preference, I think that ARM *should not* provide immediate value option in their new instructions.
Instead of that, if they think that acceleration of very short memory copies with count, known in compile time is important, they should augment existing "load pair of SIMD registers/store pair of SIMD registers" instructions with ability to specify # of loaded/stored bytes.

> <
> > For non-aligned src/dst there is a potential to make it faster than loads+stores.
> <
> Because one can reutilize the misalignedness of a first LD/STwith the misalignedness
> of a second LD/ST. and save cycles. This is a great reason MM should not be done with
> individual instructions--but places the burden on HW to never screw this up (like many
> seem to have done).
> >
> > But in discussion about we were talking about count in register, like on x86 and, likely, on ARMv9, rather than about count as immediate value, which, if I am not mistaken, is available on S/360 (limited to 256B ?) since very beginning, but I am not aware on it being available on other popular architectures.
> <
> S/360 is memcpy
> We have been arguing that the proper thing to build in HW is memmove and let memcpy have
> lower performance.

Not in this subthread.
And not that everybody agree, anyway.

On 21/09/2021 15:23, EricP wrote:
> David Brown wrote:
>> On 20/09/2021 15:21, EricP wrote:
>>> David Brown wrote:
>>>> On 19/09/2021 19:50, EricP wrote:
>>>>> MitchAlsup wrote:
>>>>>> On Sunday, September 19, 2021 at 7:14:52 AM UTC-5, Anton Ertl wrote:
>>>>>>> Unfortunately, that's not sufficient to get rid of memcpy(); we
>>>>>>> tried, but the library incompatibility problem outlined above
>>>>>>> persisted. IIRC gcc produces calls to memcpy() when you copy some
>>>>>>> structs or somesuch.
>>>>>> <
>>>>>> Yes but !!
>>>>>> <
>>>>>> a structure assignment: s1 = s2; is never in a position where s1 and
>>>>>> s2 overlap; and therefore nobody cares which direction, or how wide,
>>>>>> any
>>>>>> data transfer happens to be.
>>>>> An array of S's can be shifting elements with overlapping slices.
>>>>>
>>>>> If s1 and s2 are structs in different areas of a union and passed
>>>>> to a subroutine void DoCopy (struct S *s1, struct S *s2);
>>>>>
>>>>>   union
>>>>>   {  struct S s1;
>>>>>      struct
>>>>>      {  int  i;
>>>>>         struct S s2;
>>>>>      } foo;
>>>>>   } u;
>>>>>
>>>>> DoCopy cannot prove its args do not overlap so should use memmove.
>>>>>
>>>> I think there is a subtle point here.
>>>>
>>>> I did some testing with:
>>>>
>>>>
>>>> struct S { int a; int b[103]; };
>>>>
>>>>   union
>>>>   {  struct S s1;
>>>>      struct
>>>>      {  int  i;
>>>>         struct S s2;
>>>>      } foo;
>>>>   } u;
>>>>
>>>> void foo1(void) {
>>>>     u.s1 = u.foo.s2;
>>>> }
>>>>
>>>> void foo2(void) {
>>>>     u.foo.s2 = u.s1;
>>>> }
>>>>
>>>> void DoCopy (struct S *s1, struct S *s2) {
>>>>     *s1 = *s2;
>>>> }
>>>>
>>>>
>>>> Testing with gcc and clang shows that they both use memcpy (or a "rep
>>>> movsq" sequence on x86 that I /think/ assumes there is no overlap - but
>>>> I am not an expert on x86 assembly).
>>>>
>>>> It's not often that the gcc folk are wrong about this kind of thing.
>>>> (Overly pedantic or unhelpful, perhaps, but not wrong.)  It's even
>>>> rarer
>>>> for both gcc and clang to be wrong.
>>>>
>>>>
>>>> I believe the answer is in the C standards, 6.5.16.1p3, regarding the
>>>> semantics of the assignment operator:
>>>>
>>>> """
>>>> If the value being stored in an object is read from another object that
>>>> overlaps in any way the storage of the first object, then the overlap
>>>> shall be exact and the two objects shall have qualified or unqualified
>>>> versions of a compatible type; otherwise, the behavior is undefined.
>>>> """
>>>>
>>>> So if you have "s1 = s2", then either they do not overlap at all, or
>>>> they are the same object (in which case memcpy will still be fine in
>>>> practice).
>>> Try it with -fno-strict-aliasing.
>>
>> It makes no difference (which is not a surprise, since it is not an
>> aliasing issue - the types of s1 and s2 are the same).
>>
>>> It should switch to the equivalent of memove which would test
>>> if (source_addr < dest_addr) and CLD/STD clear/set the direction flag
>>> before REP MOVSQ.
>>
>> I'm sorry, my x86 assembly is not advanced enough to have a good
>> understanding of the "rep mov" stuff, so I could easily misinterpret
>> here.  Just use <https://godbolt.org> and pick a different gcc target -
>> for many non-x86 devices, you get a call to memcpy.
>>
>>> From what I've read, Linux and other OS's must compile
>>> with -fno-strict-aliasing to function as programmers intended.
>>>
>>> Most of my applications would too if my MS compiler
>>> didn't assume this anyway.
>>>
>>
>> Are you /sure/ MSVC assumes no strict aliasing?  Is it clearly
>> documented as a feature?  A lot of people assume MSVC has wrapping
>> signed integer arithmetic, and they are wrong about that - it simply has
>> weak optimisation in some areas.  (In the case of signed integer
>> arithmetic, they document that they do /not/ have guaranteed wrapping
>> semantics.)
>
> My MS documentation says similar to K&R (remember, MS is C-89),
> that one can cast between pointers, and between pointer and int.
>

C89 is not K&R C. The C89 standard is based closely on the
specification in the second edition of The C Programming Language, and
is a major improvement on the language described in the first edition,
and generally referred to as "K&R C" or "pre-ANSI C". And MS' compiler
doesn't actually support C89, but a kind of bastardised version of C
that includes most of the parts of C99 that are common to C++.
(Actually, my understanding - without being an MSVC user myself - is
that it now has much better C99 support.)

> And like the C standards it does not say that one can actually use
> a pointer after a cast, but its restrictions are based on alignment
> or size mismatch.

The C standards say you can converted between object pointer types. It
is not just the /use/ of the converted pointer that is dependent on the
alignment - the conversion itself is undefined behaviour if alignment
requirements are not met (section 6.3.2.3p7). There are very few things
the standard defines for converted pointers of incompatible types. You
can convert them back to the original type and the result will compare
equal. You can convert to a character pointer and use that to access
the data. And that's about it, really. See the "strict aliasing"
section in 6.5p7.

Size and alignment of pointers has no bearing on the restrictions on how
you can access data - the aliasing rules are about the object accessed,
not the pointer.

(The section numbering in the C standards has intentionally been kept as
consistent as possible since C99 - my reference here is the C18 draft
N2346, but you can look at it in almost any free draft standard.)

>
> From my Visual Studio documentation:
>
> Conversions to and from Pointer Types:
> "A pointer to one type of value can be converted to a pointer to a
> different type. However, the result may be undefined because of the
> alignment requirements and sizes of different types in storage.
> A pointer to an object can be converted to a pointer to an object
> whose type requires less or equally strict storage alignment,
> and back again without change."

That's fine - it's about conversion of pointers, not aliasing.

It says nothing about accessing data via the pointers. (Complain about
the language being unclear if you like - I'd agree, and I think the C
standard and C compiler documentation could have been a /lot/ clearer here.)

>
> *HOWEVER* the MS documentation has many examples of where casting
> between types is a necessary part of the API design.
> A routine may return a handle which is an unsigned long,
> and document that the handle is actually a pointer to 'blah'.
>

Again, that is fine - all standards-defined behaviour. You can convert
between pointer types and integer types in a range of ways, limited by
sizes and implementation-specific behaviour. It is how you /use/ the
pointers that matters.

> The entire Windows kernel and all drivers depends on being able to cast
> a pointer to one data type into a pointer to a different type, with the
> onus of getting restrictions like alignment correct on the programmer.
>

What Windows does internally does not mean behaviour is guaranteed for
user code. The internal code could be compiled with special flags,
pragmas, code sequences, or anything else. Only the /documented/
behaviour matters for your own code.

> The classic example is from the device driver programming guide,
> on managing linked lists of objects. The C macro
>   CONTAINING_RECORD(Address,Type,Field)
> is passed an Address, a data struct Type contains that address,
> and the Field in that Type that is that address.
> CONTAINING_RECORD subtracts from Address the byte offset of the Field
> and returns a pointer to Type.
>
> I use the same techniques for handling objects that are members
> of multiple linked lists, trees, index tables, etc. when needed.
>
> The following is from the driver programmers guide
> (it's their naming convention, not mine):
>
>   typedef struct {
>     // driver-defined members
>   .
>   .
>   .  SINGLE_LIST_ENTRY SingleListEntry;
>
>   // other driver-defined members
>   .
>   .
>   .
>   } XXX_ENTRY;
>
> We can push an entry onto a single linked list by
>
>   PushEntryList(&ListHead, &myObject->SingleListEntry);
>
> Note that it puts the address of the field in the middle of its struct
> on the list. We can pop that entry back off the list and convert it
> back to a pointer to myObject with:
>
>   PopXxxEntry(PSINGLE_LIST_ENTRY ListHead, PXXX_ENTRY Entry)
>   {
>     PSINGLE_LIST_ENTRY SingleListEntry;
>
>     SingleListEntry = PopEntryList(ListHead);
>     return CONTAINING_RECORD(SingleListEntry, XXX_ENTRY, SingleListEntry);
>   }
>
> The above pops an entry, a pointer to a SINGLE_LIST_ENTRY,
> off a circular single linked list. That pointer is actually the
> address of a field named SingleListEntry in type XXX_ENTRY.
> The macro CONTAINING_RECORD subtracts the field offset to SingleListEntry.
> It returns a pointer to XXX_ENTRY.
>

On 21/09/2021 10:29, Thomas Koenig wrote:
> Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>> Thomas Koenig <tkoenig@netcologne.de> writes:
>>> Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>>> Thomas Koenig <tkoenig@netcologne.de> writes:
>>>>> Just to get the general philosphy behind Linus T's reasoning.
>>>>>
>>>>> A compiler change that introduces a bug in a user program is bad.
>>>>> It does not matter if the program relied on an implementation
>>>>> detail not specified either by the compiler or the standard, or
>>>>> if another compiler or another architecture does this differently.
>>>>>
>>>>> Did I sum that up correctly?
>>>>
>>>> Correct.
>>>>
>>>>> There is no specification that a compiler writer could follow.
>>>>> The respective language standards are irrelevant. Past behavior
>>>>> is all that counts, and pointing out that his program is errnoneous
>>>>> is a no-no.
>>>>
>>>> Correct.
>>>
>>> You subsccibe to "language standards are irrelevant"?
>>
>> In those places where a standard does not specify anything (as the C
>> standard does for undefined behaviour), they certainly are. Isn't
>> that obvious?
>>
>>> Just one question: Is there a clear specification anywhere that,
>>> in your opinion, compiler writers should adhere to, some sort of
>>> additional standard document (approved by whom?) that clearly,
>>> explicitly and unabiguously defines the areas left undefined by
>>> the C standard?
>>
>> Lots of adverbs. Probably more than necessary. Anyway, the document
>> you should read (although it's not quite what you are asking for) is
>>
>> http://www.complang.tuwien.ac.at/papers/ertl17kps.pdf
>
> Has that paper actually been peer-reviewed and published?
> It certainly uses some rather inflammatory terminology,
> which might hinder its acceptance.
>
> However, even if it is published, such a document changes nothing.
>
> If you actually wanted to achieve anything, you would have to
> write up a paper for WG14 for an expanded set of guarantees on C,
> which would then be up for discussion for inclusion in the next
> standard.
>
> If you say "Oh, but it wouldn't be accepted anyway" then you
> already have your answer.
>
>>> Or is it restricted to the bugs that Linus Torvalds and yourself
>>> find irritating?
>>
>> Are you so out of arguments that you have to resort to personal
>> attacks?
>
> Clearly not, because I didn't write a personal attack above.
>

AFAICS, what you wrote is not only not a personal attack, it is an
accurate summary of Anton's position here, described in his paper and
countless posts.

Basically, the C standard (combined with other relevant
implementation-specific documentation such as compiler documentation,
library references, POSIX standards, target ABI's, etc.) give you a set
of contract rules. You, the programmer, promise to follow the
requirements of the language specifications, and the compiler promises
in return to give you code that does what you asked for. If you don't
follow the language rules, you can't expect any particular behaviour as
a result.

The same principle applies to all programming languages, of course, not
just C.

What Anton would like is that the compiler does what he wants the
compiler to do, not what he /tells/ the compiler to do.

When he writes "x = y + z;", he is /telling/ the compiler "Assume that
the mathematically correct sum of y and z falls within the range of
"int". If it is necessary for producing the specified observable
behaviour of the program, assign that value to x, and now you know that
x has the correct mathematical integer sum as its value - a fact that
can be used to optimise or simplify other code". That is the way C is
defined.

What he /means/ might be "generate the assembly instruction for a 32-bit
addition taking the variables y and z, returning the result in x". Or
he might mean "Calculate the two's complement wrapping add operation on
y and z, and assign the result to x".

He has his list in his head of what he wants C to mean, and gets upset
when a compiler writer does not use that same list but follows the C
standards (and other relevant documents). Torvalds is the same - though
his list will be different from Anton's (despite a certain overlap).
There are many others with similar ideas, all varying in detail but all
wanting C compilers to support their own particular set of additional
semantics rather than the documented and agreed set given in the standards.

Now, I think you'll be hard pushed to find an experienced C programmer
who would not like the rules or semantics of C to be changed in some way
- either changing defined behaviour, or adding definitions where the
standards are undefined, or tightening the options for
implementation-defined behaviour. I can think of many cases - some that
I believe would improve C overall, and many more that would suit me
personally though I know others would disagree. But there is a key
difference between /wanting/ a compiler to behave differently, and
/expecting/ it to behave differently.

Note also that compilers often go out of their way to support some of
these semantics, such as gcc's "-fwrapv", "-fno-strict-aliasing" and
"-fno-delete-null-pointer-checks" switches.

And I also think that there is almost endless scope for compilers to be
more helpful to programmers in pointing out their mistakes or
misunderstandings - though this is often in conflict with people who
knowingly and intentionally write code that could be a sign of an error
from a less knowledgeable programmer. (I think "-Wall", and perhaps
"-Wextra", should be the default in C - experts should have to turn off
warnings if they want to write suspicious code.)

(None of this is a personal attack either. I have great respect for the
knowledge and achievements of Anton, Torvalds, and many others. But
blaming compilers for their own bugs is wrong.)

MitchAlsup wrote:
> On Tuesday, September 21, 2021 at 11:52:39 AM UTC-5, Michael S wrote:
>> For non-aligned src/dst there is a potential to make it faster than loads+stores.
> <
> Because one can reutilize the misalignedness of a first LD/STwith the misalignedness
> of a second LD/ST. and save cycles. This is a great reason MM should not be done with
> individual instructions--but places the burden on HW to never screw this up (like many
> seem to have done).
>>
>> But in discussion about we were talking about count in register, like on x86 and, likely, on ARMv9, rather than about count as immediate value, which, if I am not mistaken, is available on S/360 (limited to 256B ?) since very beginning, but I am not aware on it being available on other popular architectures.
> <
> S/360 is memcpy
> We have been arguing that the proper thing to build in HW is memmove and let memcpy have
> lower performance.

Right.

I think it is obvious by now that memmove should be the default behavior
for 99+% of all memory copies, leaving just the single instance of
variable length pattern replication on the table for (forward byte by
byte) memcpy() operations.

This means a memcpy which behaves exactly like the original IBM MVC in
the same situation, so maybe it should have a new and better name, like

replicate(byte *dst, unsigned dstlen,
byte *pattern, unsigned patlen);

It would have to work as if it was implemented by

for (byte *p = pattern, *pend = pattern + patlen; dstlen--;
if (p >= pend) p = pattern) {
*dst++ = *p++;
}

Terje

--
- <Terje.Mathisen at tmsw.no>
"almost all programming can be viewed as an exercise in caching"

Michael S wrote:
> On Wednesday, September 22, 2021 at 12:40:19 AM UTC+3, EricP wrote:
>> EricP wrote:
>>> David Brown wrote:
>>>> It makes no difference (which is not a surprise, since it is not an
>>>> aliasing issue - the types of s1 and s2 are the same).
>>> - For MS compiler, no matter what options I tried, the assembler for
>>> DoCopy1, DoCopy2, DoCopy3 are always identical and the same as
>>> foo1 and foo2.
>>>
>>> - For foo1, foo2, DoCopy1, DoCopy2, DoCopy3 it always assumes the
>>> structs are not overlapped and either generates a memcpy as
>>> REP MOVSB
>>> with no direction test, so forward only,
>>> or an inline move sequence using XMM registers,
>>> again ignoring the byte offset overlap.
>>>
>>> - For DoCopy4 it generates a jump to memmove library routine.
>>>
>>> Bottom line: MS VC compiler is ignoring all memory aliasing,
>>> even in a union where it should know that it overlaps,
>>> and generating moves that assume no overlap.
>>>
>>>
>>> I see reference on StackOverflow that MS removed the compile option
>>> Assume No aliasing /Oa in VS 2008 and replaced it with
>>> __declspec(restrict) and __declspec(noalias) directives.
>>>
>>> Optimization best practices
>>> https://docs.microsoft.com/en-us/cpp/build/optimization-best-practices?view=msvc-160
>>>
>>>
>>> It seems they think that it should assume memory aliasing unless
>>> explicitly told not to, though it does not appear to function that way.
>>>
>>> I don't know why this hasn't broken all Win32, kernel and driver code.
>> I tried this on my 10+ year old MSVC compiler and it works the same but
>> without the recent optimizations, that is it uses a forward REP MOVSB
>> except for DoCopy4.
>>
>> I can only assume that everyone who want to do block struct moves
>> uses memmove not *ptr = *ptr.
>
> More likely, nobody uses overlapped structures of the same type.
> Can you give me one reason why anybody would?

The types in each variant are not important.
If they were int's would that make it more valid?

The problem is not in the union definition.
The problem is that the example copies from one variant to another
and in doing so accesses both variants at once, which is undefined.

It is just a very easy mistake to make.

David Brown wrote:
> On 21/09/2021 15:23, EricP wrote:
>
>> And like the C standards it does not say that one can actually use
>> a pointer after a cast, but its restrictions are based on alignment
>> or size mismatch.
>
> The C standards say you can converted between object pointer types. It
> is not just the /use/ of the converted pointer that is dependent on the
> alignment - the conversion itself is undefined behaviour if alignment
> requirements are not met (section 6.3.2.3p7). There are very few things
> the standard defines for converted pointers of incompatible types. You
> can convert them back to the original type and the result will compare
> equal. You can convert to a character pointer and use that to access
> the data. And that's about it, really. See the "strict aliasing"
> section in 6.5p7.
>
> Size and alignment of pointers has no bearing on the restrictions on how
> you can access data - the aliasing rules are about the object accessed,
> not the pointer.
>
> (The section numbering in the C standards has intentionally been kept as
> consistent as possible since C99 - my reference here is the C18 draft
> N2346, but you can look at it in almost any free draft standard.)

Aliasing is having multiple names, either in whole or in part,
for the same memory location. This creates an ordering to the
reads and writes that interacts with optimizations that reorder.

If a language always translates references in WYSIWYG order then
aliasing would not be an issue (not for a compiler at least).

How multiple names can be created affects alias detection,
which may affect the ability for compilers' reordering optimization.

In C one way to create aliases is by pointer casting (conversion).

This is distinct from whether a language or compiler acknowledges
alias existence and requires the consequences of it on operation
reordering be taken into account.

>> From my Visual Studio documentation:
>>
>> Conversions to and from Pointer Types:
>> "A pointer to one type of value can be converted to a pointer to a
>> different type. However, the result may be undefined because of the
>> alignment requirements and sizes of different types in storage.
>> A pointer to an object can be converted to a pointer to an object
>> whose type requires less or equally strict storage alignment,
>> and back again without change."
>
> That's fine - it's about conversion of pointers, not aliasing.

Conversion to pointers is one way to create aliases.

> It says nothing about accessing data via the pointers. (Complain about
> the language being unclear if you like - I'd agree, and I think the C
> standard and C compiler documentation could have been a /lot/ clearer here.)

Of course. If one doesn't want to use a value as a pointer
then why make that value into a pointer type.

Pointer casting creates aliases, and aliasing may affect the
compilers ability to reorder memory references.

If one doesn't access any data then aliasing isn't an issue.

>> *HOWEVER* the MS documentation has many examples of where casting
>> between types is a necessary part of the API design.
>> A routine may return a handle which is an unsigned long,
>> and document that the handle is actually a pointer to 'blah'.
>>
>
> Again, that is fine - all standards-defined behaviour. You can convert
> between pointer types and integer types in a range of ways, limited by
> sizes and implementation-specific behaviour. It is how you /use/ the
> pointers that matters.

Yes. And people do use them. All the time.

Thus the -fno_strict_aliasing flag.

>> The entire Windows kernel and all drivers depends on being able to cast
>> a pointer to one data type into a pointer to a different type, with the
>> onus of getting restrictions like alignment correct on the programmer.
>>
>
> What Windows does internally does not mean behaviour is guaranteed for
> user code. The internal code could be compiled with special flags,
> pragmas, code sequences, or anything else. Only the /documented/
> behaviour matters for your own code.

It does. Its just not documented under the section marked "How MS C works"
but under names like "Win32 API" or "Examples of code for Windows".

>> The classic example is from the device driver programming guide,
>> on managing linked lists of objects. The C macro
>> CONTAINING_RECORD(Address,Type,Field)
>> is passed an Address, a data struct Type contains that address,
>> and the Field in that Type that is that address.
>> CONTAINING_RECORD subtracts from Address the byte offset of the Field
>> and returns a pointer to Type.
>>
>> I use the same techniques for handling objects that are members
>> of multiple linked lists, trees, index tables, etc. when needed.
>>
>> The following is from the driver programmers guide
>> (it's their naming convention, not mine):
>>
>> typedef struct {
>> // driver-defined members
>> .
>> .
>> . SINGLE_LIST_ENTRY SingleListEntry;
>>
>> // other driver-defined members
>> .
>> .
>> .
>> } XXX_ENTRY;
>>
>> We can push an entry onto a single linked list by
>>
>> PushEntryList(&ListHead, &myObject->SingleListEntry);
>>
>> Note that it puts the address of the field in the middle of its struct
>> on the list. We can pop that entry back off the list and convert it
>> back to a pointer to myObject with:
>>
>> PopXxxEntry(PSINGLE_LIST_ENTRY ListHead, PXXX_ENTRY Entry)
>> {
>> PSINGLE_LIST_ENTRY SingleListEntry;
>>
>> SingleListEntry = PopEntryList(ListHead);
>> return CONTAINING_RECORD(SingleListEntry, XXX_ENTRY, SingleListEntry);
>> }
>>
>> The above pops an entry, a pointer to a SINGLE_LIST_ENTRY,
>> off a circular single linked list. That pointer is actually the
>> address of a field named SingleListEntry in type XXX_ENTRY.
>> The macro CONTAINING_RECORD subtracts the field offset to SingleListEntry.
>> It returns a pointer to XXX_ENTRY.
>>
>
> I don't see any pointer conversions or aliasing here.

CONTAINING_RECORD converts a pointer to a field inside a record
back into a pointer to that record by subtracting the field offset.

That creates an alias for that field of a different data type
(one is a pointer to list-link, one is pointer to parent record).

If the compilers' rules for strict alias detection claims that these are
now distinct, non-overlapping entities by virtue of having different
data types, then there is a possibility for unfortunate reordering.

Thus the -fno_strict_aliasing or equivalent compile option.

>> When combined with other techniques to validate data types,
>> validity check markers on all major data types, it is very reliable
>> (I have never had any programming errors using it).
>>
>> This technique is used throughout the WinNT OS, so if the compiler
>> stopped allowing pointer casts between types, MS won't have a product
>> to sell or any drivers.
>>
>
> Pointer casts are well defined in the C standards - that is not what we
> are talking about.
>
> (Again, you may well ask what is the point of the language letting you
> easily and conveniently cast pointers and then not use the cast pointers
> to access the data. I'm not defending that slightly counter-intuitive
> feature - I'm just trying to explain it.)

The C language punts and says it is defined as undefined.
At that point what MSVC actually does is what matters.

>> ==============
>>
>> The switch from signed ints that wrap to linear they can get away
>> with because their products aren't fundamentally dependent on it.
>> And as far as I know MS never said they wrapped signed ints so
>> suddenly switching so non wrapping wouldn't bother them.
>
> That is correct, AFAIK - they have never documented it. They only
> relatively recently documented that they /don't/ wrap, in connection
> with a blog about new optimisations.
>
> However, you have still not shown any justification for saying that MS
> documents "no strict aliasing" behaviour - i.e., that you can access any
> data using a pointer of any type, not just compatible types or character
> types.
>
>> Besides, they have never shows much hesitancy to abandon customers.
>> After 25 years of actively promoting inline assembler in C,
>> they suddenly dropped it on x64, and it was tough-titties to
>> everyone who had inline assembler.
>>
>>> Anyway, it is not an aliasing matter. And MSVC for ARM generates calls
>>> to memcpy for the test code. (Though for MSVC, the same vendor controls
>>> the compiler and the library - they /could/ have internal documentation
>>> noting that their memcpy implementation works like memmove.)
>> In my original example, the compilers' alias detection should see that the
>> union S changes the offset and understand that those objects MAY overlap.
>> It cannot prove that they do not overlap so it must assume they might
>> (and in this case it should _know_ that they overlap).
>>
>
> Ah, okay - you are talking about overlapping objects, which could be
> called "aliasing" of sorts. But it is unrelated to the "strict
> aliasing" rules, which are concerned with accessing a single object
> using incompatible pointer types.

On Wednesday, September 22, 2021 at 5:31:16 PM UTC+3, EricP wrote:
> Michael S wrote:
> > On Wednesday, September 22, 2021 at 12:40:19 AM UTC+3, EricP wrote:
> >> EricP wrote:
> >>> David Brown wrote:
> >>>> It makes no difference (which is not a surprise, since it is not an
> >>>> aliasing issue - the types of s1 and s2 are the same).
> >>> - For MS compiler, no matter what options I tried, the assembler for
> >>> DoCopy1, DoCopy2, DoCopy3 are always identical and the same as
> >>> foo1 and foo2.
> >>>
> >>> - For foo1, foo2, DoCopy1, DoCopy2, DoCopy3 it always assumes the
> >>> structs are not overlapped and either generates a memcpy as
> >>> REP MOVSB
> >>> with no direction test, so forward only,
> >>> or an inline move sequence using XMM registers,
> >>> again ignoring the byte offset overlap.
> >>>
> >>> - For DoCopy4 it generates a jump to memmove library routine.
> >>>
> >>> Bottom line: MS VC compiler is ignoring all memory aliasing,
> >>> even in a union where it should know that it overlaps,
> >>> and generating moves that assume no overlap.
> >>>
> >>>
> >>> I see reference on StackOverflow that MS removed the compile option
> >>> Assume No aliasing /Oa in VS 2008 and replaced it with
> >>> __declspec(restrict) and __declspec(noalias) directives.
> >>>
> >>> Optimization best practices
> >>> https://docs.microsoft.com/en-us/cpp/build/optimization-best-practices?view=msvc-160
> >>>
> >>>
> >>> It seems they think that it should assume memory aliasing unless
> >>> explicitly told not to, though it does not appear to function that way.
> >>>
> >>> I don't know why this hasn't broken all Win32, kernel and driver code.
> >> I tried this on my 10+ year old MSVC compiler and it works the same but
> >> without the recent optimizations, that is it uses a forward REP MOVSB
> >> except for DoCopy4.
> >>
> >> I can only assume that everyone who want to do block struct moves
> >> uses memmove not *ptr = *ptr.
> >
> > More likely, nobody uses overlapped structures of the same type.
> > Can you give me one reason why anybody would?
> The types in each variant are not important.

What's important is that it's the same type.
Only for the same type you can use language's built-in assignment and thus rely on compiler for choice of semantics of the copy .
When source and destination are of different types you would have to specify explicitly whether you want memcpy or memmove.
Which, IIRC, in Windows DDK is the same thing when there is a call of library routine, but not necessarily the same when inlined.
And for what you had seen to happen, it has to be structure type, because for other assignable "C" types optimizing compiler is not going to generate 'rep movs' or function call.

> If they were int's would that make it more valid?

For int, or for any other type that fits in register, I am pretty sure that compiler will generate 1 load + 1store, i.e. result would be the same as memmove.

>
> The problem is not in the union definition.
> The problem is that the example copies from one variant to another
> and in doing so accesses both variants at once, which is undefined.
>
> It is just a very easy mistake to make.

I think, in real world code, it's not easy, it's pretty much impossible.

On Wednesday, September 22, 2021 at 8:36:37 AM UTC+2, David Brown wrote:
> And MS' compiler
> doesn't actually support C89, but a kind of bastardised version of C
> that includes most of the parts of C99 that are common to C++.
> (Actually, my understanding - without being an MSVC user myself - is
> that it now has much better C99 support.)

Not only does it have much better C99 support now -- it has proper C17 support, which is mostly C99 without the (many) stupid parts.

-Peter

Thomas Koenig [2021-09-21 06:08:05] wrote:
> Stefan Monnier <monnier@iro.umontreal.ca> schrieb:
>>> You wrote "at compile time". Testing the correctnes of a program
>>> written in Turing-complete languge is the equivalent of solving
>>> the halting problem.
>>
>> No it's not. Type checking is the typical example, which lots and lots
>> of compilers perform quite reliably without bumping into the halting
>> problem, even though those languages are Turing complete.
>
> That is only a small sliver of overall correctness.
>
>> All you need to check the correctness of a program is to ask someone to
>> give you a proof of that correctness and then you simply check
>> the proof.
>
> "Simply" is the wrong word here, I think...
>
> Just one minor example: Linear equation solvers are a field where
> there are actually lots of proofs about convergence and so on,
> but these are typically over the field of real, numbers, not
> computer floating-point numbers.

I was talking about correctness of using particular forms of byte
copying, typically related to the possibility (or not) of aliasing
and/or overlap.

AFAIK such proofs never rely on anything sophisticated at all, and in
the rare cases where the argument becomes nontrivial it's easy to change
the code to make the argument trivial again.

Stefan

On 22/09/2021 15:59, EricP wrote:
> David Brown wrote:
>> On 21/09/2021 15:23, EricP wrote:
>>
>>> And like the C standards it does not say that one can actually use
>>> a pointer after a cast, but its restrictions are based on alignment
>>> or size mismatch.
>>
>> The C standards say you can converted between object pointer types.  It
>> is not just the /use/ of the converted pointer that is dependent on the
>> alignment - the conversion itself is undefined behaviour if alignment
>> requirements are not met (section 6.3.2.3p7).  There are very few things
>> the standard defines for converted pointers of incompatible types.  You
>> can convert them back to the original type and the result will compare
>> equal.  You can convert to a character pointer and use that to access
>> the data.  And that's about it, really.  See the "strict aliasing"
>> section in 6.5p7.
>>
>> Size and alignment of pointers has no bearing on the restrictions on how
>> you can access data - the aliasing rules are about the object accessed,
>> not the pointer.
>>
>> (The section numbering in the C standards has intentionally been kept as
>> consistent as possible since C99 - my reference here is the C18 draft
>> N2346, but you can look at it in almost any free draft standard.)
>
> Aliasing is having multiple names, either in whole or in part,
> for the same memory location. This creates an ordering to the
> reads and writes that interacts with optimizations that reorder.

Yes. But that can happen in many different ways. The
"-fno-strict-aliasing" flag is for a specific type, type-based alias
analysis.

The C standards support and allow many kinds of aliasing. They do not,
however, allow aliasing via incompatible pointers except in certain
specific circumstances. (By that I mean such accesses are undefined
behaviour - particular implementations can always choose to support it.)
gcc is one of many compilers that actively use type-based alias
analysis to determine some cases it can be sure that lvalues do not
alias (or that you don't care about the results if they do), and
optimise using that knowledge. Some other compilers, such as MSVC, may
not do such analysis or may not use it to affect code generation.
However, unless they /document/ this as a guaranteed feature, it is not
something you can rely on.

MSVC, to my knowledge, has no documentation guaranteeing it will not do
type-based alias analysis. As far as my research indicates, it does not
/currently/ optimise code based on TBAA, but that is not a guarantee
that it never will.

>
> If a language always translates references in WYSIWYG order then
> aliasing would not be an issue (not for a compiler at least).
>
> How multiple names can be created affects alias detection,
> which may affect the ability for compilers' reordering optimization.
>
> In C one way to create aliases is by pointer casting (conversion).
>

That is one way to create aliases that you cannot use to access data
(without further conversion).

int x = 123;
int * p = &x; // Creates an alias
const int * q = p; // Another alias
char * r = (char *) &x; // A character type pointer alias

*p++; // OK
x += *q; // OK
r[2] = 3; // OK

float * d = (float *) p; // Another alias
*d = 4; // Not okay - breaks TBAA rules
unsigned int * u = (unsigned int *) d; // Convert back
*u = 3; // OK, compatible types

> This is distinct from whether a language or compiler acknowledges
> alias existence and requires the consequences of it on operation
> reordering be taken into account.
>
>>> From my Visual Studio documentation:
>>>
>>> Conversions to and from Pointer Types:
>>> "A pointer to one type of value can be converted to a pointer to a
>>> different type. However, the result may be undefined because of the
>>> alignment requirements and sizes of different types in storage.
>>> A pointer to an object can be converted to a pointer to an object
>>> whose type requires less or equally strict storage alignment,
>>> and back again without change."
>>
>> That's fine - it's about conversion of pointers, not aliasing.
>
> Conversion to pointers is one way to create aliases.
>

The paragraph is about /converting/ pointers, not about using aliases to
data.

>> It says nothing about accessing data via the pointers.  (Complain about
>> the language being unclear if you like - I'd agree, and I think the C
>> standard and C compiler documentation could have been a /lot/ clearer
>> here.)
>
> Of course. If one doesn't want to use a value as a pointer
> then why make that value into a pointer type.
>
> Pointer casting creates aliases, and aliasing may affect the
> compilers ability to reorder memory references.
>

Please stop repeating that. You are mixing different concepts.

There are many ways to have aliases - different lvalues that refer to
the same object. Every pointer, except for the first one used to store
the result of a malloc(), is an alias. You get a new alias by copying
an existing alias, or by taking the address of an object. What you do
with that pointer - convert it, cast it, increment it, store it, etc., -
is irrelevant.

Conversions are change of type in C. Some conversions are done
implicitly, some require an explicit conversion - known in C as a
"cast". Some conversions may change a value (such as converting a float
to an int), others do no conversion. Some are always valid, others may
have undefined behaviour during the conversion. Some result in an
object that can be used immediately, others result in an object with
more limited use - but that does not make the conversion itself invalid.

Alias analysis is the general term for when a compiler considers the
possible allowed and well-defined values a reference (an lvalue, or
pointer) may have, and compares it to what it knows about other
references. It does this to establish if it is sure the two pointers
refer to the same object, or if it is sure they point to different
objects - both of these can let the compiler generate more optimal code.

A good compiler will do many kinds of alias analysis. It knows, for
example, that a pointer that is a parameter to a function cannot refer
to a local variable. Pointers with "restrict" give extra information.
And some compilers use type-based alias analysis - the compiler knows
that pointers to incompatible types cannot refer to the same object,
baring a few specific exceptions.

> If one doesn't access any data then aliasing isn't an issue.
>
>>> *HOWEVER* the MS documentation has many examples of where casting
>>> between types is a necessary part of the API design.
>>> A routine may return a handle which is an unsigned long,
>>> and document that the handle is actually a pointer to 'blah'.
>>>
>>
>> Again, that is fine - all standards-defined behaviour.  You can convert
>> between pointer types and integer types in a range of ways, limited by
>> sizes and implementation-specific behaviour.  It is how you /use/ the
>> pointers that matters.
>
> Yes. And people do use them. All the time.
>

Actually, it is very rare in most code. You see it mainly in some kinds
of low-level code or cases where people try to "hand optimise" their
code instead of using a good compiler. (Sometimes a good compiler is
not available.) You come across it in code that must /implement/
memcpy-type functions, not code that /uses/ such functions.

> Thus the -fno_strict_aliasing flag.

It can be useful for the rare occasions when you really need access to
the same data via different types, but safe methods (unions, memcpy,
character pointers) are inappropriate. It can also be useful when
dealing with older or poorer code that needs such access patterns but
the writer was not aware of the somewhat subtle rules of C.

>
>>> The entire Windows kernel and all drivers depends on being able to cast
>>> a pointer to one data type into a pointer to a different type, with the
>>> onus of getting restrictions like alignment correct on the programmer.
>>>
>>
>> What Windows does internally does not mean behaviour is guaranteed for
>> user code.  The internal code could be compiled with special flags,
>> pragmas, code sequences, or anything else.  Only the /documented/
>> behaviour matters for your own code.
>
> It does. Its just not documented under the section marked "How MS C works"
> but under names like "Win32 API" or "Examples of code for Windows".

No, that is about how the Win API works. It is perfectly reasonable to
have a guarantee that type-based alias analysis does not apply to data
passed across the API boundaries while TBAA based optimisation was
applied within compiled files. It would be astounding for a compiler to
do TBAA across separately compiled libraries and DLL's ! There are also
no strict aliasing violations in taking a pointer to a data type,
converting it to one of the silly Windows API common pointer types or
integer types, and then having the API function convert it back to a
pointer to the original data type.

On 22/09/2021 18:53, Peter Lund wrote:
> On Wednesday, September 22, 2021 at 8:36:37 AM UTC+2, David Brown
> wrote:
>> And MS' compiler doesn't actually support C89, but a kind of
>> bastardised version of C that includes most of the parts of C99
>> that are common to C++. (Actually, my understanding - without being
>> an MSVC user myself - is that it now has much better C99 support.)
>
> Not only does it have much better C99 support now -- it has proper
> C17 support, which is mostly C99 without the (many) stupid parts.
>

C17 is a bug fix (and a nice typography change) to C11. C11 adds a
number of features to C99, most importantly support for multi-threading
and atomics. The only things "removed" were that complex numbers and
VLA's were made optional. It is a pointless change - there are no C11
or C17 compliant compilers that don't also support C99, which require
complex numbers and VLAs.

But it is nice to hear a confirmation that MSVC supports newer C standards.

Types are dim, illusion, types are just something to remind you what
purpose is data. strict aliasing rule is pure stupidity and should be
banned, because it creates lot of bugs because it lets compiler
writers tollerate bugs for sake of "good performance" (which is not
proven).

Greetings, Branimir.
--
7-77-777
\|/
---
/|\

On 2021-09-22, David Brown <david.brown@hesbynett.no> wrote:
> On 22/09/2021 15:59, EricP wrote:
>> David Brown wrote:
>>> On 21/09/2021 15:23, EricP wrote:
>>>
>>>> And like the C standards it does not say that one can actually use
>>>> a pointer after a cast, but its restrictions are based on alignment
>>>> or size mismatch.
>>>
>>> The C standards say you can converted between object pointer types.  It
>>> is not just the /use/ of the converted pointer that is dependent on the
>>> alignment - the conversion itself is undefined behaviour if alignment
>>> requirements are not met (section 6.3.2.3p7).  There are very few things
>>> the standard defines for converted pointers of incompatible types.  You
>>> can convert them back to the original type and the result will compare
>>> equal.  You can convert to a character pointer and use that to access
>>> the data.  And that's about it, really.  See the "strict aliasing"
>>> section in 6.5p7.
>>>
>>> Size and alignment of pointers has no bearing on the restrictions on how
>>> you can access data - the aliasing rules are about the object accessed,
>>> not the pointer.
>>>
>>> (The section numbering in the C standards has intentionally been kept as
>>> consistent as possible since C99 - my reference here is the C18 draft
>>> N2346, but you can look at it in almost any free draft standard.)
>>
>> Aliasing is having multiple names, either in whole or in part,
>> for the same memory location. This creates an ordering to the
>> reads and writes that interacts with optimizations that reorder.
>
> Yes. But that can happen in many different ways. The
> "-fno-strict-aliasing" flag is for a specific type, type-based alias
> analysis.
>
> The C standards support and allow many kinds of aliasing. They do not,
> however, allow aliasing via incompatible pointers except in certain
> specific circumstances. (By that I mean such accesses are undefined
> behaviour - particular implementations can always choose to support it.)
> gcc is one of many compilers that actively use type-based alias
> analysis to determine some cases it can be sure that lvalues do not
> alias (or that you don't care about the results if they do), and
> optimise using that knowledge. Some other compilers, such as MSVC, may
> not do such analysis or may not use it to affect code generation.
> However, unless they /document/ this as a guaranteed feature, it is not
> something you can rely on.
>
> MSVC, to my knowledge, has no documentation guaranteeing it will not do
> type-based alias analysis. As far as my research indicates, it does not
> /currently/ optimise code based on TBAA, but that is not a guarantee
> that it never will.
>
>>
>> If a language always translates references in WYSIWYG order then
>> aliasing would not be an issue (not for a compiler at least).
>>
>> How multiple names can be created affects alias detection,
>> which may affect the ability for compilers' reordering optimization.
>>
>> In C one way to create aliases is by pointer casting (conversion).
>>
>
> That is one way to create aliases that you cannot use to access data
> (without further conversion).
>
> int x = 123;
> int * p = &x; // Creates an alias
> const int * q = p; // Another alias
> char * r = (char *) &x; // A character type pointer alias
>
> *p++; // OK
> x += *q; // OK
> r[2] = 3; // OK
>
> float * d = (float *) p; // Another alias
> *d = 4; // Not okay - breaks TBAA rules
> unsigned int * u = (unsigned int *) d; // Convert back
> *u = 3; // OK, compatible types
>
>
>> This is distinct from whether a language or compiler acknowledges
>> alias existence and requires the consequences of it on operation
>> reordering be taken into account.
>>
>>>> From my Visual Studio documentation:
>>>>
>>>> Conversions to and from Pointer Types:
>>>> "A pointer to one type of value can be converted to a pointer to a
>>>> different type. However, the result may be undefined because of the
>>>> alignment requirements and sizes of different types in storage.
>>>> A pointer to an object can be converted to a pointer to an object
>>>> whose type requires less or equally strict storage alignment,
>>>> and back again without change."
>>>
>>> That's fine - it's about conversion of pointers, not aliasing.
>>
>> Conversion to pointers is one way to create aliases.
>>
>
> The paragraph is about /converting/ pointers, not about using aliases to
> data.
>
>>> It says nothing about accessing data via the pointers.  (Complain about
>>> the language being unclear if you like - I'd agree, and I think the C
>>> standard and C compiler documentation could have been a /lot/ clearer
>>> here.)
>>
>> Of course. If one doesn't want to use a value as a pointer
>> then why make that value into a pointer type.
>>
>> Pointer casting creates aliases, and aliasing may affect the
>> compilers ability to reorder memory references.
>>
>
> Please stop repeating that. You are mixing different concepts.
>
> There are many ways to have aliases - different lvalues that refer to
> the same object. Every pointer, except for the first one used to store
> the result of a malloc(), is an alias. You get a new alias by copying
> an existing alias, or by taking the address of an object. What you do
> with that pointer - convert it, cast it, increment it, store it, etc., -
> is irrelevant.
>
> Conversions are change of type in C. Some conversions are done
> implicitly, some require an explicit conversion - known in C as a
> "cast". Some conversions may change a value (such as converting a float
> to an int), others do no conversion. Some are always valid, others may
> have undefined behaviour during the conversion. Some result in an
> object that can be used immediately, others result in an object with
> more limited use - but that does not make the conversion itself invalid.
>
> Alias analysis is the general term for when a compiler considers the
> possible allowed and well-defined values a reference (an lvalue, or
> pointer) may have, and compares it to what it knows about other
> references. It does this to establish if it is sure the two pointers
> refer to the same object, or if it is sure they point to different
> objects - both of these can let the compiler generate more optimal code.
>
> A good compiler will do many kinds of alias analysis. It knows, for
> example, that a pointer that is a parameter to a function cannot refer
> to a local variable. Pointers with "restrict" give extra information.
> And some compilers use type-based alias analysis - the compiler knows
> that pointers to incompatible types cannot refer to the same object,
> baring a few specific exceptions.
>
>> If one doesn't access any data then aliasing isn't an issue.
>>
>>>> *HOWEVER* the MS documentation has many examples of where casting
>>>> between types is a necessary part of the API design.
>>>> A routine may return a handle which is an unsigned long,
>>>> and document that the handle is actually a pointer to 'blah'.
>>>>
>>>
>>> Again, that is fine - all standards-defined behaviour.  You can convert
>>> between pointer types and integer types in a range of ways, limited by
>>> sizes and implementation-specific behaviour.  It is how you /use/ the
>>> pointers that matters.
>>
>> Yes. And people do use them. All the time.
>>
>
> Actually, it is very rare in most code. You see it mainly in some kinds
> of low-level code or cases where people try to "hand optimise" their
> code instead of using a good compiler. (Sometimes a good compiler is
> not available.) You come across it in code that must /implement/
> memcpy-type functions, not code that /uses/ such functions.
>
>> Thus the -fno_strict_aliasing flag.
>
> It can be useful for the rare occasions when you really need access to
> the same data via different types, but safe methods (unions, memcpy,
> character pointers) are inappropriate. It can also be useful when
> dealing with older or poorer code that needs such access patterns but
> the writer was not aware of the somewhat subtle rules of C.
>
>>
>>>> The entire Windows kernel and all drivers depends on being able to cast
>>>> a pointer to one data type into a pointer to a different type, with the
>>>> onus of getting restrictions like alignment correct on the programmer.
>>>>
>>>
>>> What Windows does internally does not mean behaviour is guaranteed for
>>> user code.  The internal code could be compiled with special flags,
>>> pragmas, code sequences, or anything else.  Only the /documented/
>>> behaviour matters for your own code.
>>
>> It does. Its just not documented under the section marked "How MS C works"
>> but under names like "Win32 API" or "Examples of code for Windows".
>
> No, that is about how the Win API works. It is perfectly reasonable to
> have a guarantee that type-based alias analysis does not apply to data
> passed across the API boundaries while TBAA based optimisation was
> applied within compiled files. It would be astounding for a compiler to
> do TBAA across separately compiled libraries and DLL's ! There are also
> no strict aliasing violations in taking a pointer to a data type,
> converting it to one of the silly Windows API common pointer types or
> integer types, and then having the API function convert it back to a
> pointer to the original data type.
>
> If a compiler has extensions to the C standard with extra semantics that
> it guarantees beyond those required of the language, then it should
> document them /clearly/ and /explicitly/. If it does not document them,
> you have no guarantees. Looking from examples in one case and guessing
> and extrapolating is not the answer.
>
>>>
>>> I don't see any pointer conversions or aliasing here.
>>
>> CONTAINING_RECORD converts a pointer to a field inside a record
>> back into a pointer to that record by subtracting the field offset.
>
> A pointer to a struct may be converted back and forth with a pointer to
> its first element, guaranteed to be at the same address, and you can use
> pointers to a struct type containing another type as a way of accessing
> that type. This is all kosher, according to the C standard's rules on
> type-based aliasing.
>
>>
>> That creates an alias for that field of a different data type
>> (one is a pointer to list-link, one is pointer to parent record).
>>
>> If the compilers' rules for strict alias detection claims that these are
>> now distinct, non-overlapping entities by virtue of having different
>> data types, then there is a possibility for unfortunate reordering.
>>
>> Thus the -fno_strict_aliasing or equivalent compile option.
>
> No, it is not needed here.
>
>>
>>>> When combined with other techniques to validate data types,
>>>> validity check markers on all major data types, it is very reliable
>>>> (I have never had any programming errors using it).
>>>>
>>>> This technique is used throughout the WinNT OS, so if the compiler
>>>> stopped allowing pointer casts between types, MS won't have a product
>>>> to sell or any drivers.
>>>>
>>>
>>> Pointer casts are well defined in the C standards - that is not what we
>>> are talking about.
>>>
>>> (Again, you may well ask what is the point of the language letting you
>>> easily and conveniently cast pointers and then not use the cast pointers
>>> to access the data.  I'm not defending that slightly counter-intuitive
>>> feature - I'm just trying to explain it.)
>>
>> The C language punts and says it is defined as undefined.
>
> The examples you have given are defined by the C standards. You still
> (AFAICS) haven't shown examples where the C standard does not define the
> accesses, but where "gcc -fno-strict-aliasing" /does/ define them.
>
>> At that point what MSVC actually does is what matters.
>
> What MSVC does in some examples does /not/ matter - what MSVC
> /documents/ and /guarantees/ it will do is what matters if you are
> looking for additional semantics.
>
> If you judge purely from examples, x² + x + 41 could be assumed to
> generate primes indefinitely - after all, it looks good for the first 40
> values of x.
>
> I'm trying to understand if MSVC /really/ does not apply type-based
> alias analysis, and guarantees that they will stay that way in the
> future. I've found nothing in what you have written, or in my googling,
> to suggest that they make any such guarantees - while also seeing
> nothing to suggest that they /do/ TBAA. As far as I can see, it is
> simply a case of their optimisers not being that advanced at the moment.
> As they seem to be interested in using more and more from clang and
> llvm in their tools, that could easily change in the future.
>
>>>
>>> Ah, okay - you are talking about overlapping objects, which could be
>>> called "aliasing" of sorts.  But it is unrelated to the "strict
>>> aliasing" rules, which are concerned with accessing a single object
>>> using incompatible pointer types.
>>
>> That is what aliasing is, multiple names for the same memory locations.
>> Different languages acknowledge the existence of this to varying degrees,
>> and use that knowledge to limit the compilers reordering optimizations.
>>
>
> That is true. But your examples of aliasing are unrelated to type-based
> alias analysis and gcc's -fno-strict-aliasing.
>
>

Stefan Monnier <monnier@iro.umontreal.ca> schrieb:
> Thomas Koenig [2021-09-21 06:08:05] wrote:
>> Stefan Monnier <monnier@iro.umontreal.ca> schrieb:
>>>> You wrote "at compile time". Testing the correctnes of a program
>>>> written in Turing-complete languge is the equivalent of solving
>>>> the halting problem.
>>>
>>> No it's not. Type checking is the typical example, which lots and lots
>>> of compilers perform quite reliably without bumping into the halting
>>> problem, even though those languages are Turing complete.
>>
>> That is only a small sliver of overall correctness.
>>
>>> All you need to check the correctness of a program is to ask someone to
>>> give you a proof of that correctness and then you simply check
>>> the proof.
>>
>> "Simply" is the wrong word here, I think...
>>
>> Just one minor example: Linear equation solvers are a field where
>> there are actually lots of proofs about convergence and so on,
>> but these are typically over the field of real, numbers, not
>> computer floating-point numbers.
>
> I was talking about correctness of using particular forms of byte
> copying, typically related to the possibility (or not) of aliasing
> and/or overlap.

Ah, OK.

And I was taking about the general case.

We're in agreement, then.

Just one datapoint on memcpy...
$ cat mem.c
#include <string.h>

void foo (char *a, char *b, int n)
{ memcpy (a, b, n);
} $ cat main.c
#include <stdio.h>
#include <stdlib.h>

void foo (char *a, char *b, int n);
#define N 10

int main()
{ char *a;
a = malloc(N+1);
for (int i=0; i<N; i++)
a[i] = '0' + i;
a[N] = 0;
printf ("%s\n",a);
foo (a, a+1, N-1);
printf ("%s\n",a);
foo (a+1, a, N-1);
} $ gcc -g -fsanitize=address main.c mem.c
$ ./a.out
0123456789
=================================================================
==4416==ERROR: AddressSanitizer: memcpy-param-overlap: memory ranges [0x602000000010,0x602000000019) and [0x602000000011, 0x60200000001a) overlap
#0 0x7f5d68a887af in __interceptor_memcpy ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
#1 0x4012bc in foo /tmp/mem.c:5
#2 0x401251 in main /tmp/main.c:15
#3 0x7f5d6885d0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#4 0x4010bd in _start (/tmp/a.out+0x4010bd)
[allocation location info snipped]

So, this error is certainly detectable.

Detect this error:
#include <stdio.h>
#include <string.h>

char a[] = {0,1,2,3,4,5,6,7,8,9};

int main(void) {
memmove(a,a+5,10);
for (int i = 0; i < 10; ++i){
printf("%d\n",a[i]);
}
}

bmaxa@Branimirs-Air projects % gcc overlap.c
bmaxa@Branimirs-Air projects % ./a.out
5 6
7 8
9 0
0 0
0 0
--
7-77-777
\|/
---
/|\

On 2021-09-23, Thomas Koenig <tkoenig@netcologne.de> wrote:
> Just one datapoint on memcpy...
> $ cat mem.c
> #include <string.h>
>
> void foo (char *a, char *b, int n)
> {
> memcpy (a, b, n);
> }
> $ cat main.c
> #include <stdio.h>
> #include <stdlib.h>
>
> void foo (char *a, char *b, int n);
> #define N 10
>
> int main()
> {
> char *a;
> a = malloc(N+1);
> for (int i=0; i<N; i++)
> a[i] = '0' + i;
> a[N] = 0;
> printf ("%s\n",a);
> foo (a, a+1, N-1);
> printf ("%s\n",a);
> foo (a+1, a, N-1);
> }
> $ gcc -g -fsanitize=address main.c mem.c
> $ ./a.out
> 0123456789
>=================================================================
>==4416==ERROR: AddressSanitizer: memcpy-param-overlap: memory ranges [0x602000000010,0x602000000019) and [0x602000000011, 0x60200000001a) overlap
> #0 0x7f5d68a887af in __interceptor_memcpy ../../../../trunk/libsanitizer/sanitizer_common/sanitizer_common_interceptors.inc:827
> #1 0x4012bc in foo /tmp/mem.c:5
> #2 0x401251 in main /tmp/main.c:15
> #3 0x7f5d6885d0b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> #4 0x4010bd in _start (/tmp/a.out+0x4010bd)
> [allocation location info snipped]
>
> So, this error is certainly detectable.

--
Evil Sinner!

On 23/09/2021 16:16, Branimir Maksimovic wrote:
> Detect this error:
> #include <stdio.h>
> #include <string.h>
>
> char a[] = {0,1,2,3,4,5,6,7,8,9};
>
> int main(void) {
> memmove(a,a+5,10);
> for (int i = 0; i < 10; ++i){
> printf("%d\n",a[i]);
> }
> }
>
> bmaxa@Branimirs-Air projects % gcc overlap.c

I've found the error - the user doesn't know how to work his tools.

If you know how to use your compiler, it will tell you the error
straight away (assuming you have a new enough version of gcc).

gcc -Wall -O2 overlap.c
In file included from /usr/include/string.h:495,
from overlap.c:3:
In function ‘memmove’,
inlined from ‘main’ at overlap.c:8:3:
/usr/include/x86_64-linux-gnu/bits/string_fortified.h:40:10: warning:
‘__builtin_memmove’ forming offset [11, 15] is out of the bounds [0, 10]
of object ‘a’ with type ‘char[10]’ [-Warray-bounds]
40 | return __builtin___memmove_chk (__dest, __src, __len, __bos0
(__dest));
|
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
overlap.c: In function ‘main’:
overlap.c:5:6: note: ‘a’ declared here
5 | char a[] = {0,1,2,3,4,5,6,7,8,9};
| ^

> bmaxa@Branimirs-Air projects % ./a.out
> 5
> 6
> 7
> 8
> 9
> 0
> 0
> 0
> 0
> 0

Sure thing:
bmaxa@Branimirs-Air projects % gcc -Wall overlap.c
bmaxa@Branimirs-Air projects % gcc -Wall -O2 overlap.c
bmaxa@Branimirs-Air projects % ./a.out
5 6
7 8
9 0
0 0
0 0
bmaxa@Branimirs-Air projects % gcc -v
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.5 (clang-1205.0.22.11)
Target: arm64-apple-darwin21.1.0
Thread model: posix

What about this?

Greetings, Branimir.
--
7-77-777
\|/
---
/|\
On 2021-09-23, David Brown <david.brown@hesbynett.no> wrote:
> On 23/09/2021 16:16, Branimir Maksimovic wrote:
>> Detect this error:
>> #include <stdio.h>
>> #include <string.h>
>>
>> char a[] = {0,1,2,3,4,5,6,7,8,9};
>>
>> int main(void) {
>> memmove(a,a+5,10);
>> for (int i = 0; i < 10; ++i){
>> printf("%d\n",a[i]);
>> }
>> }
>>
>> bmaxa@Branimirs-Air projects % gcc overlap.c
>
> I've found the error - the user doesn't know how to work his tools.
>
> If you know how to use your compiler, it will tell you the error
> straight away (assuming you have a new enough version of gcc).
>
>
> gcc -Wall -O2 overlap.c
> In file included from /usr/include/string.h:495,
> from overlap.c:3:
> In function ‘memmove’,
> inlined from ‘main’ at overlap.c:8:3:
> /usr/include/x86_64-linux-gnu/bits/string_fortified.h:40:10: warning:
> ‘__builtin_memmove’ forming offset [11, 15] is out of the bounds [0, 10]
> of object ‘a’ with type ‘char[10]’ [-Warray-bounds]
> 40 | return __builtin___memmove_chk (__dest, __src, __len, __bos0
> (__dest));
> |
> ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> overlap.c: In function ‘main’:
> overlap.c:5:6: note: ‘a’ declared here
> 5 | char a[] = {0,1,2,3,4,5,6,7,8,9};
> | ^
>
>> bmaxa@Branimirs-Air projects % ./a.out
>> 5
>> 6
>> 7
>> 8
>> 9
>> 0
>> 0
>> 0
>> 0
>> 0

--
Evil Sinner!

Branimir Maksimovic <branimir.maksimovic@gmail.com> schrieb:
> Detect this error:
>#include <stdio.h>
>#include <string.h>
>
> char a[] = {0,1,2,3,4,5,6,7,8,9};
>
> int main(void) {
> memmove(a,a+5,10);
> for (int i = 0; i < 10; ++i){
> printf("%d\n",a[i]);
> }
> }

$ gcc -g -fsanitize=address a.c
$ ./a.out
=================================================================
==2202==ERROR: AddressSanitizer: global-buffer-overflow on address 0x557bc06dd02a at pc 0x7f98a23e4f40 bp 0x7fff95db6ad0 sp 0x7fff95db6278
READ of size 10 at 0x557bc06dd02a thread T0
#0 0x7f98a23e4f3f in __interceptor_memmove (/lib/x86_64-linux-gnu/libasan.so.5+0xa0f3f)
#1 0x557bc06da22f in main /tmp/a.c:7
#2 0x7f98a21790b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
#3 0x557bc06da14d in _start (/tmp/a.out+0x114d)
$ gcc --version
gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
Copyright (C) 2019 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

However, from your other post, it seems you use clang. The same
option works threre on Linux at least, try it.

On 23/09/2021 16:53, Branimir Maksimovic wrote:
> Sure thing:
> bmaxa@Branimirs-Air projects % gcc -Wall overlap.c
> bmaxa@Branimirs-Air projects % gcc -Wall -O2 overlap.c
> bmaxa@Branimirs-Air projects % ./a.out
> 5
> 6
> 7
> 8
> 9
> 0
> 0
> 0
> 0
> 0
> bmaxa@Branimirs-Air projects % gcc -v
> Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
> Apple clang version 12.0.5 (clang-1205.0.22.11)
> Target: arm64-apple-darwin21.1.0
> Thread model: posix
>
> What about this?
>
> Greetings, Branimir.

You are not using gcc. (Blame Apple for their idiotic and confusing
renaming of clang to "gcc" - an insult to both compilers and the source
of endless mixups.)

It is slightly surprising to me that clang fails to spot the problem,
but I am not very familiar with that compiler. gcc warns about it from
version 8 onwards, but like many warnings you have to have optimisation
enabled in order to get the more detailed code analysis.

Thanks! Does it works for plain malloc?

@Branimirs-Air projects % gcc -g -fsanitize=address overlap.c
bmaxa@Branimirs-Air projects % gcc -Wall -g -fsanitize=address overlap.c
bmaxa@Branimirs-Air projects % gcc -Wall -g -O2 -fsanitize=address overlap.c
bmaxa@Branimirs-Air projects % ./a.out
=================================================================
==33851==ERROR: AddressSanitizer: global-buffer-overflow on address 0x000102f3c06a at pc 0x0001036a5608 bp 0x00016cecb4d0 sp 0x00016cecac88
READ of size 10 at 0x000102f3c06a thread T0
#0 0x1036a5604 in __asan_memmove+0xd0 (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3d604)
#1 0x102f37d40 in main overlap.c:7
#2 0x1032bd0f0 in start+0x204 (dyld:arm64e+0x50f0)

0x000102f3c06a is located 0 bytes to the right of global variable 'a' defined in 'overlap.c:4:6' (0x102f3c060) of size 10
SUMMARY: AddressSanitizer: global-buffer-overflow (libclang_rt.asan_osx_dynamic.dylib:arm64e+0x3d604) in __asan_memmove+0xd0
Shadow bytes around the buggy address:
0x0070206077b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0070206077c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0070206077d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0070206077e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0070206077f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
=>0x007020607800: 00 00 00 00 00 00 00 00 00 00 00 00 00[02]f9 f9
0x007020607810: f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9 f9
0x007020607820: f9 f9 f9 f9 00 00 00 00 00 00 00 00 00 00 00 00
0x007020607830: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x007020607840: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x007020607850: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==33851==ABORTING
zsh: abort ./a.out

--
7-77-777
\|/
---
/|\

On 2021-09-23, Thomas Koenig <tkoenig@netcologne.de> wrote:
> Branimir Maksimovic <branimir.maksimovic@gmail.com> schrieb:
>> Detect this error:
>>#include <stdio.h>
>>#include <string.h>
>>
>> char a[] = {0,1,2,3,4,5,6,7,8,9};
>>
>> int main(void) {
>> memmove(a,a+5,10);
>> for (int i = 0; i < 10; ++i){
>> printf("%d\n",a[i]);
>> }
>> }
>
> $ gcc -g -fsanitize=address a.c
> $ ./a.out
>=================================================================
>==2202==ERROR: AddressSanitizer: global-buffer-overflow on address 0x557bc06dd02a at pc 0x7f98a23e4f40 bp 0x7fff95db6ad0 sp 0x7fff95db6278
> READ of size 10 at 0x557bc06dd02a thread T0
> #0 0x7f98a23e4f3f in __interceptor_memmove (/lib/x86_64-linux-gnu/libasan.so.5+0xa0f3f)
> #1 0x557bc06da22f in main /tmp/a.c:7
> #2 0x7f98a21790b2 in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x270b2)
> #3 0x557bc06da14d in _start (/tmp/a.out+0x114d)
> $ gcc --version
> gcc (Ubuntu 9.3.0-17ubuntu1~20.04) 9.3.0
> Copyright (C) 2019 Free Software Foundation, Inc.
> This is free software; see the source for copying conditions. There is NO
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
> However, from your other post, it seems you use clang. The same
> option works threre on Linux at least, try it.

--
Evil Sinner!

Again, you are right@Branimirs-Air projects % gcc-11 -Wall -g -O2 overlap.c
In file included from /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/string.h:194,
from overlap.c:2:
overlap.c: In function 'main':
overlap.c:7:3: warning: '__builtin_memmove' forming offset [10, 14] is out of the bounds [0, 10] of object 'a' with type 'char[10]' [-Warray-bounds]
7 | memmove(a,a+5,10);
| ^~~~~~~
overlap.c:4:6: note: 'a' declared here
4 | char a[] = {0,1,2,3,4,5,6,7,8,9};
| ^

--
7-77-777
\|/
---
/|\

On 2021-09-23, David Brown <david.brown@hesbynett.no> wrote:
> On 23/09/2021 16:53, Branimir Maksimovic wrote:
>> Sure thing:
>> bmaxa@Branimirs-Air projects % gcc -Wall overlap.c
>> bmaxa@Branimirs-Air projects % gcc -Wall -O2 overlap.c
>> bmaxa@Branimirs-Air projects % ./a.out
>> 5
>> 6
>> 7
>> 8
>> 9
>> 0
>> 0
>> 0
>> 0
>> 0
>> bmaxa@Branimirs-Air projects % gcc -v
>> Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
>> Apple clang version 12.0.5 (clang-1205.0.22.11)
>> Target: arm64-apple-darwin21.1.0
>> Thread model: posix
>>
>> What about this?
>>
>> Greetings, Branimir.
>
> You are not using gcc. (Blame Apple for their idiotic and confusing
> renaming of clang to "gcc" - an insult to both compilers and the source
> of endless mixups.)
>
> It is slightly surprising to me that clang fails to spot the problem,
> but I am not very familiar with that compiler. gcc warns about it from
> version 8 onwards, but like many warnings you have to have optimisation
> enabled in order to get the more detailed code analysis.
>

--
Evil Sinner!