A "famous security bug":

void f( void )
{ char buffer[ MAX ];
/* . . . */
memset( buffer, 0, sizeof( buffer )); }

. Can you see what the bug is?

(I have already read the answer; I post it as a pastime.)

On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
> A "famous security bug":
>
> void f( void )
> { char buffer[ MAX ];
> /* . . . */
> memset( buffer, 0, sizeof( buffer )); }
>
> . Can you see what the bug is?

I don't know about "the bug", but conditions can be identified under
which that would have a problem executing, like MAX being in excess
of available automatic storage.

If the /*...*/ comment represents the elision of some security sensitive
code, where the memset is intended to obliterate secret information,
of course, that obliteration is not required to work.

After the memset, the buffer has no next use, so the all the assignments
performed by memset to the bytes of buffer are dead assignments that can
be elided.

To securely clear memory, you have to use a function for that purpose
that is not susceptible to optimization.

If you're not doing anything stupid, like link time optimization, an
external function in another translation unit (a function that the
compiler doesn't recognize as being an alias or wrapper for memset)
ought to suffice.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

ram@zedat.fu-berlin.de (Stefan Ram) writes:

> A "famous security bug":
>
> void f( void )
> { char buffer[ MAX ];
> /* . . . */
> memset( buffer, 0, sizeof( buffer )); }
>
> . Can you see what the bug is?
>
> (I have already read the answer; I post it as a pastime.)

The optimizer deletes the memset statement because buffer is not
accessed after it?

On 3/20/2024 11:18 AM, Stefan Ram wrote:
> A "famous security bug":
>
> void f( void )
> { char buffer[ MAX ];
> /* . . . */
> memset( buffer, 0, sizeof( buffer )); }
>
> . Can you see what the bug is?
>
> (I have already read the answer; I post it as a pastime.)

Add in a volatile? ;^)

Kaz Kylheku <433-929-6894@kylheku.com> writes:
>On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>> A "famous security bug":
>>
>> void f( void )
>> { char buffer[ MAX ];
>> /* . . . */
>> memset( buffer, 0, sizeof( buffer )); }
>>
>> . Can you see what the bug is?
>
>I don't know about "the bug", but conditions can be identified under
>which that would have a problem executing, like MAX being in excess
>of available automatic storage.

Perhaps Stephan is under the mistaken assumption that
'buffer' devolves to a type of 'char *' when used
with the sizeof operator.

On 3/20/2024 12:37 PM, Chris M. Thomasson wrote:
> On 3/20/2024 11:18 AM, Stefan Ram wrote:
>>    A "famous security bug":
>>
>> void f( void )
>> { char buffer[ MAX ];
>>    /* . . . */
>>    memset( buffer, 0, sizeof( buffer )); }
>>
>>    . Can you see what the bug is?
>>
>>    (I have already read the answer; I post it as a pastime.)
>
> Add in a volatile? ;^)

Also, show us the definition of MAX...

scott@slp53.sl.home (Scott Lurndal) writes:
> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>>> A "famous security bug":
>>>
>>> void f( void )
>>> { char buffer[ MAX ];
>>> /* . . . */
>>> memset( buffer, 0, sizeof( buffer )); }
>>>
>>> . Can you see what the bug is?
>>
>>I don't know about "the bug", but conditions can be identified under
>>which that would have a problem executing, like MAX being in excess
>>of available automatic storage.
>
> Perhaps Stephan is under the mistaken assumption that
> 'buffer' devolves to a type of 'char *' when used
> with the sizeof operator.

That was my first thought, but I think the idea (not clearly stated) is
that the /* . . . */ code stores sensitive information in buffer, and
the memset call is intended to clobber that information, but may be
elided since buffer is not explicitly used later. A malicious process
with access to the program's memory might be able to read that
information after f() has returned.

C23 adds memset_explicit() for this purpose.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

Keith Thompson <Keith.S.Thompson+u@gmail.com> writes:
> scott@slp53.sl.home (Scott Lurndal) writes:
>> Kaz Kylheku <433-929-6894@kylheku.com> writes:
>>>On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>>>> A "famous security bug":
>>>>
>>>> void f( void )
>>>> { char buffer[ MAX ];
>>>> /* . . . */
>>>> memset( buffer, 0, sizeof( buffer )); }
>>>>
>>>> . Can you see what the bug is?
>>>
>>>I don't know about "the bug", but conditions can be identified under
>>>which that would have a problem executing, like MAX being in excess
>>>of available automatic storage.
>>
>> Perhaps Stephan is under the mistaken assumption that
>> 'buffer' devolves to a type of 'char *' when used
>> with the sizeof operator.
>
> That was my first thought, but I think the idea (not clearly stated) is
> that the /* . . . */ code stores sensitive information in buffer, and
> the memset call is intended to clobber that information, but may be
> elided since buffer is not explicitly used later. A malicious process
> with access to the program's memory might be able to read that
> information after f() has returned.

And I should acknowledge that Kaz mentioned that before I did.

> C23 adds memset_explicit() for this purpose.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 3/20/2024 12:37 PM, Chris M. Thomasson wrote:
> On 3/20/2024 11:18 AM, Stefan Ram wrote:
>>    A "famous security bug":
>>
>> void f( void )
>> { char buffer[ MAX ];
>>    /* . . . */
>>    memset( buffer, 0, sizeof( buffer )); }
>>
>>    . Can you see what the bug is?
>>
>>    (I have already read the answer; I post it as a pastime.)
>
> Add in a volatile? ;^)

Instead of zeroing, what about filling it with random bytes reaped from
a TRNG?

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
> On 3/20/2024 12:37 PM, Chris M. Thomasson wrote:
>> On 3/20/2024 11:18 AM, Stefan Ram wrote:
>>>    A "famous security bug":
>>>
>>> void f( void )
>>> { char buffer[ MAX ];
>>>    /* . . . */
>>>    memset( buffer, 0, sizeof( buffer )); }
>>>
>>>    . Can you see what the bug is?
>>>
>>>    (I have already read the answer; I post it as a pastime.)
>> Add in a volatile? ;^)
>
> Instead of zeroing, what about filling it with random bytes reaped
> from a TRNG?

Why?

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 3/20/2024 4:19 PM, Keith Thompson wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>> On 3/20/2024 12:37 PM, Chris M. Thomasson wrote:
>>> On 3/20/2024 11:18 AM, Stefan Ram wrote:
>>>>    A "famous security bug":
>>>>
>>>> void f( void )
>>>> { char buffer[ MAX ];
>>>>    /* . . . */
>>>>    memset( buffer, 0, sizeof( buffer )); }
>>>>
>>>>    . Can you see what the bug is?
>>>>
>>>>    (I have already read the answer; I post it as a pastime.)
>>> Add in a volatile? ;^)
>>
>> Instead of zeroing, what about filling it with random bytes reaped
>> from a TRNG?
>
> Why?
>

Those zeros might be "targets" for a nefarious program?

On 20/03/2024 19:54, Kaz Kylheku wrote:
> On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>> A "famous security bug":
>>
>> void f( void )
>> { char buffer[ MAX ];
>> /* . . . */
>> memset( buffer, 0, sizeof( buffer )); }
>>
>> . Can you see what the bug is?
>
> I don't know about "the bug", but conditions can be identified under
> which that would have a problem executing, like MAX being in excess
> of available automatic storage.
>
> If the /*...*/ comment represents the elision of some security sensitive
> code, where the memset is intended to obliterate secret information,
> of course, that obliteration is not required to work.
>
> After the memset, the buffer has no next use, so the all the assignments
> performed by memset to the bytes of buffer are dead assignments that can
> be elided.
>
> To securely clear memory, you have to use a function for that purpose
> that is not susceptible to optimization.
>
> If you're not doing anything stupid, like link time optimization, an
> external function in another translation unit (a function that the
> compiler doesn't recognize as being an alias or wrapper for memset)
> ought to suffice.
>

Using LTO is not "stupid". Relying on people /not/ using LTO, or not
using other valid optimisations, is "stupid".

/Really/ dealing with this kind of potential data leakage is a
multi-faceted problem. Does it matter if you zero out this buffer, if
that is just in the cache and the original password data is still in the
ram ready to be read with a Rowhammer attack? Or if there is a copy
somewhere else in code that used it?

Of course it is important to deal with possible security issues at every
practical point, so zeroing out this buffer is part of the solution - as
long as no one thinks that using C23's memset_explicit(), or similar
functions, are somehow complete fixes.

Calling an external function in another translation unit, however, is
not a way to guarantee particular effects. Using volatile accesses is
better (if you don't have a suitable function with the right guarantees
for your target OS and/or C version). There are also compiler-specific
ways, such as adding "__asm__("" : "+m" (buffer));" after the memset.

On 2024-03-21, David Brown <david.brown@hesbynett.no> wrote:
> On 20/03/2024 19:54, Kaz Kylheku wrote:
>> On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>>> A "famous security bug":
>>>
>>> void f( void )
>>> { char buffer[ MAX ];
>>> /* . . . */
>>> memset( buffer, 0, sizeof( buffer )); }
>>>
>>> . Can you see what the bug is?
>>
>> I don't know about "the bug", but conditions can be identified under
>> which that would have a problem executing, like MAX being in excess
>> of available automatic storage.
>>
>> If the /*...*/ comment represents the elision of some security sensitive
>> code, where the memset is intended to obliterate secret information,
>> of course, that obliteration is not required to work.
>>
>> After the memset, the buffer has no next use, so the all the assignments
>> performed by memset to the bytes of buffer are dead assignments that can
>> be elided.
>>
>> To securely clear memory, you have to use a function for that purpose
>> that is not susceptible to optimization.
>>
>> If you're not doing anything stupid, like link time optimization, an
>> external function in another translation unit (a function that the
>> compiler doesn't recognize as being an alias or wrapper for memset)
>> ought to suffice.
>
> Using LTO is not "stupid". Relying on people /not/ using LTO, or not
> using other valid optimisations, is "stupid".

LTO is a nonconforming optimization. It destroys the concept that
when a translation unit is translated, the semantic analysis is
complete, such that the only remaining activity is resolution of
external references (linkage), and that the semantic analysis of one
translation unit deos not use information about another translation
unit.

This has not yet changed in last April's N3096 draft, where
translation phases 7 and 8 are:

7. White-space characters separating tokens are no longer significant.
Each preprocessing token is converted into a token. The resulting
tokens are syntactically and semantically analyzed and translated
as a translation unit.

8. All external object and function references are resolved. Library
components are linked to satisfy external references to functions
and objects not defined in the current translation. All such
translator output is collected into a program image which contains
information needed for execution in its execution environment.

and before that, the Program Structure section says:

The separate translation units of a program communicate by (for
example) calls to functions whose identifiers have external linkage,
manipulation of objects whose identifiers have external linkage, or
manipulation of data files. Translation units may be separately
translated and then later linked to produce an executable program.

LTO deviates from the the model that translation units are separate,
and the conceptual steps of phases 7 and 8.

The translation unit that is prepared for LTO is not fully cooked. You
have no idea what its code will turn into when the interrupted
compilation is resumed during linkage, under the influence of other
tranlation units it is combined with.

So in fact, the language allows us to take it for granted that, given

my_memset(array, 0, sizeof(array)); }

at the end of a function, and my_memset is an external definition
provided by another translation unit, the call may not be elided.

The one who may be acting recklessly is he who turns on nonconforming
optimizations that are not documented as supported by the code base.

Another example would be something like gcc's -ffast-math.
You wouldn't unleash that on numerical code written by experts,
and expect the same correct results.

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Kaz Kylheku to Stefan Ram:

> > > A "famous security bug":
> > >
> > > void f( void )
> > > { char buffer[ MAX ];
> > > /* . . . */
> > > memset( buffer, 0, sizeof( buffer )); }
> > >
> > > . Can you see what the bug is?
>
> I don't know about "the bug", but conditions can be
> identified under which that would have a problem
> executing, like MAX being in excess of available automatic
> storage.
>
> If the /*...*/ comment represents the elision of some
> security sensitive code, where the memset is intended to
> obliterate secret information, of course, that
> obliteration is not required to work.
>
> After the memset, the buffer has no next use, so the all
> the assignments performed by memset to the bytes of buffer
> are dead assignments that can be elided.
>
> To securely clear memory, you have to use a function for
> that purpose that is not susceptible to optimization.

I think this behavior (of a C compiler) rather stupid. In a
low-level imperative language, the compiled program shall
do whatever the programmer commands it to do. If he
commands it to clear the buffer, it shall clear the buffer.
This optimisation is too high-level, too counter-inituitive,
even deceitful. The optimiser is free to perform the task
in the fastest manner possible, but it shall not ignore the
programmer's order to zero-fill the buffer, especially
without emitting a warning about (potentially!) redundant
code, which it is the programmer's reponsibility to confirm
and remove.

Redundant code shall be dealt with in the source, rather than
in the executable.

--
() ascii ribbon campaign -- against html e-mail
/\ www.asciiribbon.org -- against proprietary attachments

On 3/21/2024 10:41 AM, Kaz Kylheku wrote:
> On 2024-03-21, David Brown <david.brown@hesbynett.no> wrote:
>> On 20/03/2024 19:54, Kaz Kylheku wrote:
>>> On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>>>> A "famous security bug":
>>>>
>>>> void f( void )
>>>> { char buffer[ MAX ];
>>>> /* . . . */
>>>> memset( buffer, 0, sizeof( buffer )); }
>>>>
>>>> . Can you see what the bug is?
>>>
>>> I don't know about "the bug", but conditions can be identified under
>>> which that would have a problem executing, like MAX being in excess
>>> of available automatic storage.
>>>
>>> If the /*...*/ comment represents the elision of some security sensitive
>>> code, where the memset is intended to obliterate secret information,
>>> of course, that obliteration is not required to work.
>>>
>>> After the memset, the buffer has no next use, so the all the assignments
>>> performed by memset to the bytes of buffer are dead assignments that can
>>> be elided.
>>>
>>> To securely clear memory, you have to use a function for that purpose
>>> that is not susceptible to optimization.
>>>
>>> If you're not doing anything stupid, like link time optimization, an
>>> external function in another translation unit (a function that the
>>> compiler doesn't recognize as being an alias or wrapper for memset)
>>> ought to suffice.
>>
>> Using LTO is not "stupid". Relying on people /not/ using LTO, or not
>> using other valid optimisations, is "stupid".
>
> LTO is a nonconforming optimization. It destroys the concept that
> when a translation unit is translated, the semantic analysis is
> complete, such that the only remaining activity is resolution of
> external references (linkage), and that the semantic analysis of one
> translation unit deos not use information about another translation
> unit.
>[...]

Side note:

Actually, way back (pre c/c++ 11), I was worried about LTO messing up my
custom, highly sensitive sync code.

https://web.archive.org/web/20070509044340/http://appcore.home.comcast.net/

Notice the externally assembled functions comment?

"All of its “critical-sequences” are contained in externally assembled
functions ( read all ) in order to prevent a rouge C compiler from
reordering anything that would corrupt the data-structure. The queue
allocates its nodes from a three-level cache"

If a damn "rogue" compiler can mess with my custom ASM then things are
going to be broken...

;^)

Anton Shepelev <anton.txt@gmail.moc> writes:
> Kaz Kylheku to Stefan Ram:
>
>> > > A "famous security bug":
>> > >
>> > > void f( void )
>> > > { char buffer[ MAX ];
>> > > /* . . . */
>> > > memset( buffer, 0, sizeof( buffer )); }
>> > >
>> > > . Can you see what the bug is?
>>
>> I don't know about "the bug", but conditions can be
>> identified under which that would have a problem
>> executing, like MAX being in excess of available automatic
>> storage.
>>
>> If the /*...*/ comment represents the elision of some
>> security sensitive code, where the memset is intended to
>> obliterate secret information, of course, that
>> obliteration is not required to work.
>>
>> After the memset, the buffer has no next use, so the all
>> the assignments performed by memset to the bytes of buffer
>> are dead assignments that can be elided.
>>
>> To securely clear memory, you have to use a function for
>> that purpose that is not susceptible to optimization.
>
> I think this behavior (of a C compiler) rather stupid. In a
> low-level imperative language, the compiled program shall
> do whatever the programmer commands it to do. If he
> commands it to clear the buffer, it shall clear the buffer.
> This optimisation is too high-level, too counter-inituitive,
> even deceitful. The optimiser is free to perform the task
> in the fastest manner possible, but it shall not ignore the
> programmer's order to zero-fill the buffer, especially
> without emitting a warning about (potentially!) redundant
> code, which it is the programmer's reponsibility to confirm
> and remove.
>
> Redundant code shall be dealt with in the source, rather than
> in the executable.

Then C is not what you call a "low-level imperative language", and none
of your "shall"s apply to the language defined by the ISO C standard.

C programs define behavior, not generated machine code. If an
implementation can implement the behavior of a C program with a memset()
call without invoking memset(), it's free to do so.

The programmer's intent in the code snippet that started this thread was
for the memset call to erase sensitive data in memory that, after the
function returns, is not part of any object. C (prior to C23) doesn't
provide a way to do that. Any program whose observable behavior differs
depending on whether that memory was cleared has undefined behavior.

Judicious use of "volatile" should avoid the problem.

5.1.2.3p6:

The least requirements on a conforming implementation are:

- Volatile accesses to objects are evaluated strictly according
to the rules of the abstract machine.

- At program termination, all data written into files shall be
identical to the result that execution of the program according
to the abstract semantics would have produced.

- The input and output dynamics of interactive devices shall take
place as specified in 7.23.3. The intent of these requirements
is that unbuffered or line-buffered output appear as soon as
possible, to ensure that prompting messages appear prior to a
program waiting for input.

This is the *observable behavior* of the program.

If you think that's stupid, I won't try to change your mind, but the
meaning is clear.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:

>"All of its “critical-sequences” are contained in externally assembled
>functions ( read all ) in order to prevent a rouge C compiler from

As opposed to a viridian C compiler?

On 2024-03-21, Anton Shepelev <anton.txt@gmail.moc> wrote:
> Kaz Kylheku to Stefan Ram:
>
>> > > A "famous security bug":
>> > >
>> > > void f( void )
>> > > { char buffer[ MAX ];
>> > > /* . . . */
>> > > memset( buffer, 0, sizeof( buffer )); }
>> > >
>> > > . Can you see what the bug is?
>>
>> I don't know about "the bug", but conditions can be
>> identified under which that would have a problem
>> executing, like MAX being in excess of available automatic
>> storage.
>>
>> If the /*...*/ comment represents the elision of some
>> security sensitive code, where the memset is intended to
>> obliterate secret information, of course, that
>> obliteration is not required to work.
>>
>> After the memset, the buffer has no next use, so the all
>> the assignments performed by memset to the bytes of buffer
>> are dead assignments that can be elided.
>>
>> To securely clear memory, you have to use a function for
>> that purpose that is not susceptible to optimization.
>
> I think this behavior (of a C compiler) rather stupid. In a
> low-level imperative language, the compiled program shall
> do whatever the programmer commands it to do. If he
> commands it to clear the buffer, it shall clear the buffer.
> This optimisation is too high-level, too counter-inituitive,
> even deceitful. The optimiser is free to perform the task
> in the fastest manner possible, but it shall not ignore the
> programmer's order to zero-fill the buffer, especially
> without emitting a warning about (potentially!) redundant
> code, which it is the programmer's reponsibility to confirm
> and remove.

If C compilers warned about every piece of dead code that is eliminated,
you'd be up to your ears in diagnostics all day.

Then there is the question of how to silence them on a case-by-case
basis: I did want /that/ code to be eliminated, but not that one.

If you do want the code deleted, that doesn't always mean
you can do it yoruself. What gets eliminated can be target
dependent:

switch (sizeof (long)) {
case 4: ...
case 8: ..
}

Eliminating dead stores is a very basic dataflow-driven optimization.

Because memset is part of the C language, the compiler knows
exactly what effect it has (that it's equivalent to setting
all the bytes to zero, like a sequence of assignments).

If you don't want a call to be optimized away, call your
own function in another translation unit. (And don't turn
on nonconforming cross-translation-unit optimizations.)

--
TXR Programming Language: http://nongnu.org/txr
Cygnal: Cygwin Native Application Library: http://kylheku.com/cygnal
Mastodon: @Kazinator@mstdn.ca

Kaz Kylheku <433-929-6894@kylheku.com> writes:
> On 2024-03-21, David Brown <david.brown@hesbynett.no> wrote:
>> On 20/03/2024 19:54, Kaz Kylheku wrote:
>>> On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>>>> A "famous security bug":
>>>>
>>>> void f( void )
>>>> { char buffer[ MAX ];
>>>> /* . . . */
>>>> memset( buffer, 0, sizeof( buffer )); }
>>>>
>>>> . Can you see what the bug is?
>>>
>>> I don't know about "the bug", but conditions can be identified under
>>> which that would have a problem executing, like MAX being in excess
>>> of available automatic storage.
>>>
>>> If the /*...*/ comment represents the elision of some security sensitive
>>> code, where the memset is intended to obliterate secret information,
>>> of course, that obliteration is not required to work.
>>>
>>> After the memset, the buffer has no next use, so the all the assignments
>>> performed by memset to the bytes of buffer are dead assignments that can
>>> be elided.
>>>
>>> To securely clear memory, you have to use a function for that purpose
>>> that is not susceptible to optimization.
>>>
>>> If you're not doing anything stupid, like link time optimization, an
>>> external function in another translation unit (a function that the
>>> compiler doesn't recognize as being an alias or wrapper for memset)
>>> ought to suffice.
>>
>> Using LTO is not "stupid". Relying on people /not/ using LTO, or not
>> using other valid optimisations, is "stupid".
>
> LTO is a nonconforming optimization. It destroys the concept that
> when a translation unit is translated, the semantic analysis is
> complete, such that the only remaining activity is resolution of
> external references (linkage), and that the semantic analysis of one
> translation unit deos not use information about another translation
> unit.
>
> This has not yet changed in last April's N3096 draft, where
> translation phases 7 and 8 are:
>
> 7. White-space characters separating tokens are no longer significant.
> Each preprocessing token is converted into a token. The resulting
> tokens are syntactically and semantically analyzed and translated
> as a translation unit.
>
> 8. All external object and function references are resolved. Library
> components are linked to satisfy external references to functions
> and objects not defined in the current translation. All such
> translator output is collected into a program image which contains
> information needed for execution in its execution environment.
>
> and before that, the Program Structure section says:
>
> The separate translation units of a program communicate by (for
> example) calls to functions whose identifiers have external linkage,
> manipulation of objects whose identifiers have external linkage, or
> manipulation of data files. Translation units may be separately
> translated and then later linked to produce an executable program.
>
> LTO deviates from the the model that translation units are separate,
> and the conceptual steps of phases 7 and 8.
[...]

Link time optimization is as valid as cross-function optimization *as
long as* it doesn't change the defined behavior of the program.

Say I have a call to foo in main, and the definition of foo is in
another translation unit. In the absence of LTO, the compiler will have
to generate a call to foo. If LTO is able to determine that foo doesn't
do anything, it can remove the code for the function call, and the
resulting behavior of the linked program is unchanged.

--
Keith Thompson (The_Other_Keith) Keith.S.Thompson+u@gmail.com
Working, but not speaking, for Medtronic
void Void(void) { Void(); } /* The recursive call of the void */

On 3/21/2024 1:21 PM, Scott Lurndal wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>
>> "All of its “critical-sequences” are contained in externally assembled
>> functions ( read all ) in order to prevent a rouge C compiler from
>
> As opposed to a viridian C compiler?

I was worried about "overly aggressive" LTO messing around with my ASM.

"Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>On 3/21/2024 1:21 PM, Scott Lurndal wrote:
>> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>>
>>> "All of its “critical-sequences” are contained in externally assembled
>>> functions ( read all ) in order to prevent a rouge C compiler from
>>
>> As opposed to a viridian C compiler?
>
>I was worried about "overly aggressive" LTO messing around with my ASM.

And you missed the oblique reference to the mispelling of 'rogue' as 'rouge'.

On 3/21/2024 4:19 PM, Scott Lurndal wrote:
> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>> On 3/21/2024 1:21 PM, Scott Lurndal wrote:
>>> "Chris M. Thomasson" <chris.m.thomasson.1@gmail.com> writes:
>>>
>>>> "All of its “critical-sequences” are contained in externally assembled
>>>> functions ( read all ) in order to prevent a rouge C compiler from
>>>
>>> As opposed to a viridian C compiler?
>>
>> I was worried about "overly aggressive" LTO messing around with my ASM.
>
> And you missed the oblique reference to the mispelling of 'rogue' as 'rouge'.

Yup! I sure did. I have red on my face!

On 21/03/2024 18:41, Kaz Kylheku wrote:
> On 2024-03-21, David Brown <david.brown@hesbynett.no> wrote:
>> On 20/03/2024 19:54, Kaz Kylheku wrote:
>>> On 2024-03-20, Stefan Ram <ram@zedat.fu-berlin.de> wrote:
>>>> A "famous security bug":
>>>>
>>>> void f( void )
>>>> { char buffer[ MAX ];
>>>> /* . . . */
>>>> memset( buffer, 0, sizeof( buffer )); }
>>>>
>>>> . Can you see what the bug is?
>>>
>>> I don't know about "the bug", but conditions can be identified under
>>> which that would have a problem executing, like MAX being in excess
>>> of available automatic storage.
>>>
>>> If the /*...*/ comment represents the elision of some security sensitive
>>> code, where the memset is intended to obliterate secret information,
>>> of course, that obliteration is not required to work.
>>>
>>> After the memset, the buffer has no next use, so the all the assignments
>>> performed by memset to the bytes of buffer are dead assignments that can
>>> be elided.
>>>
>>> To securely clear memory, you have to use a function for that purpose
>>> that is not susceptible to optimization.
>>>
>>> If you're not doing anything stupid, like link time optimization, an
>>> external function in another translation unit (a function that the
>>> compiler doesn't recognize as being an alias or wrapper for memset)
>>> ought to suffice.
>>
>> Using LTO is not "stupid". Relying on people /not/ using LTO, or not
>> using other valid optimisations, is "stupid".
>
> LTO is a nonconforming optimization.

Really? That is news to me, and I suspect to the folks at gcc and
clang/llvm that developed LTO for these compilers. (I have worked with
embedded compilers that have had LTO-type optimisations for decades, but
these are not often concerned with the minutiae of the standards.)

> It destroys the concept that
> when a translation unit is translated, the semantic analysis is
> complete, such that the only remaining activity is resolution of
> external references (linkage), and that the semantic analysis of one
> translation unit deos not use information about another translation
> unit.

Where is it described in the C standards that semantic information from
one translation unit cannot be used (for optimisation, for static error
checking, for other analysis or any other purposes) in another
translation unit?

What makes you think that LTO, as implemented in compilers like gcc and
clang/llvm, do not generate code according to the "as if" rules? (That
is, they can generate code that is more optimal, but produces the same
observable effects "as if" they were strict dumb translators of the
functioning of the C abstract machine.)

I believe there is very little where the behaviour of a C program is
different if parts of the code are in one translation unit, or if they
are in several. There are utilities that merge multiple C files into
single C files (for easier deployment, or for better optimisation).
They have to take into account renaming static objects and functions to
file-local names, and remove duplicate type definitions, but as long as
certain reasonable rules are followed by the programmer, it all goes
fine. (You could, I suppose, hit complications if you relied on
compatibility of struct or union types across translation units where
the identifiers were different and they are compatible across TU's but
not within TU's, according to the 6.2.7p1 rules. But that would be
unlikely, and I expect LTO compilers to handle those cases.)

>
> This has not yet changed in last April's N3096 draft, where
> translation phases 7 and 8 are:
>
> 7. White-space characters separating tokens are no longer significant.
> Each preprocessing token is converted into a token. The resulting
> tokens are syntactically and semantically analyzed and translated
> as a translation unit.
>
> 8. All external object and function references are resolved. Library
> components are linked to satisfy external references to functions
> and objects not defined in the current translation. All such
> translator output is collected into a program image which contains
> information needed for execution in its execution environment.
>
> and before that, the Program Structure section says:
>
> The separate translation units of a program communicate by (for
> example) calls to functions whose identifiers have external linkage,
> manipulation of objects whose identifiers have external linkage, or
> manipulation of data files. Translation units may be separately
> translated and then later linked to produce an executable program.
>

All of that is irrelevant. It says nothing against sharing other
information.

> LTO deviates from the the model that translation units are separate,
> and the conceptual steps of phases 7 and 8.

No, it does not. These paragraphs are requirements, not limitations.

>
> The translation unit that is prepared for LTO is not fully cooked. You
> have no idea what its code will turn into when the interrupted
> compilation is resumed during linkage, under the influence of other
> tranlation units it is combined with.

You have as much and as little idea of what the generated code will be
as you always do during compilation. Compilers can do all kinds of
manipulations of the source code you write - as long as the observable
behaviour of the program is the same as a dumb translation. They can,
and do, use all kinds of inter-procedural optimisations for inlining
code, outlining it, breaking functions into pieces, cloning them, using
constant propagation, and so on. LTO lets them do this across
translation units.

>
> So in fact, the language allows us to take it for granted that, given
>
> my_memset(array, 0, sizeof(array)); }
>
> at the end of a function, and my_memset is an external definition
> provided by another translation unit, the call may not be elided.
>

No, the C language standards make no such guarantee.

> The one who may be acting recklessly is he who turns on nonconforming
> optimizations that are not documented as supported by the code base.
>
> Another example would be something like gcc's -ffast-math.

That is /completely/ different. That option is clearly documented as
potentially violating some of the rules of the ISO C standards. This is
why it is not enabled by default or by any common optimisation levels
(except "-Ofast", which is also documented as potentially violating
standards).

> You wouldn't unleash that on numerical code written by experts,
> and expect the same correct results.
>

I would not expect identical results to floating point calculations, no.

Depending on the code in question, I would still expect correct results.
I use "-ffast-math" in all my code in order to get correct results a
good deal faster (for my targets, and my type of code) than I would get
without it.

On 21/03/2024 21:21, Kaz Kylheku wrote:

> Eliminating dead stores is a very basic dataflow-driven optimization.
>
> Because memset is part of the C language, the compiler knows
> exactly what effect it has (that it's equivalent to setting
> all the bytes to zero, like a sequence of assignments).
>

Yes.

> If you don't want a call to be optimized away, call your
> own function in another translation unit.

No.

There are several ways that guarantee your code will carry out the
writes here (though none that guarantee the secret data is not also
stored elsewhere). Using a function in a different TU is not one of
these techniques. You do people a disfavour by recommending it.

> (And don't turn
> on nonconforming cross-translation-unit optimizations.)
>

If I knew of any non-conforming cross-translation-unit optimisations in
a compiler, I would avoid using them until the compiler vendor had fixed
the bug in question.

Anton Shepelev <anton.txt@gmail.moc> writes:

> Kaz Kylheku to Stefan Ram:
>
>>>> A "famous security bug":
>>>>
>>>> void f( void )
>>>> { char buffer[ MAX ];
>>>> /* . . . */
>>>> memset( buffer, 0, sizeof( buffer )); }
>>>>
>>>> . Can you see what the bug is?
>>
>> I don't know about "the bug", but conditions can be
>> identified under which that would have a problem
>> executing, like MAX being in excess of available automatic
>> storage.
>>
>> If the /*...*/ comment represents the elision of some
>> security sensitive code, where the memset is intended to
>> obliterate secret information, of course, that
>> obliteration is not required to work.
>>
>> After the memset, the buffer has no next use, so the all
>> the assignments performed by memset to the bytes of buffer
>> are dead assignments that can be elided.
>>
>> To securely clear memory, you have to use a function for
>> that purpose that is not susceptible to optimization.
>
> I think this behavior (of a C compiler) rather stupid. In a
> low-level imperative language, the compiled program shall
> do whatever the programmer commands it to do. If he
> commands it to clear the buffer, it shall clear the buffer.
> This optimisation is too high-level, too counter-inituitive,
> even deceitful. The optimiser is free to perform the task
> in the fastest manner possible, but it shall not ignore the
> programmer's order to zero-fill the buffer, especially
> without emitting a warning about (potentially!) redundant
> code, which it is the programmer's reponsibility to confirm
> and remove.
>
> Redundant code shall be dealt with in the source, rather than
> in the executable.

I have a couple of reactions.

One is that the ship has sailed. Somewhere between 35 and 40
years ago the people who wrote the C standard decided on a
semantic model that allows optimizations like this, and that is
not going to change. Certainly there are people who would prefer
to think of C as being a "low-level imperative language" like
what you describe, but the C community as a whole has accepted
the view taken by the authors of the C standard.

The second reaction is that, to be somewhat blunt, what is being
suggested is naive. I expect you do not yet appreciate the
ramifications of what you are suggesting. It is hard, indeed I
would say very hard, to define a semantic model that faithfully
represents the behaviors you would like to impose. You might
want to look into that, and what has been tried previously along
these lines, before pursuing this advocacy any further.