scott@slp53.sl.home (Scott Lurndal) writes:
>anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>David Brown <david.brown@hesbynett.no> writes:
>>>memmove(), and other standard library functions, are part of the
>>>implementation. They were traditionally implemented in assembly, and
>>>only later in C. But as part of the implementation, they can use
>>>features that are implementation-specific - there was never any need or
>>>intention that such functions should be implementable in portable
>>>standards-compliant C. The language never had any need to support
>>>writing an efficient memmove() in C - it gave you a ready-made memmove()
>>>function so that implementing it yourself is not necessary.
>>
>>For a supposed systems programming language like C, the inability to
>>implement memmove() is a failure.
>
>How so? A simple byte-by-byte loop, reversed if necessary,
>is sufficient. E.g. subtract the two pointers and depending on whether
>the subtraction is negative or positive do a byte-by-byte copy either
>forward or in reverse. Perfectly legal C, if not particuarly performant.

Subtraction of two pointers that do not point to the same object is
undefined behaviour, too.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>scott@slp53.sl.home (Scott Lurndal) writes:
>>anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>David Brown <david.brown@hesbynett.no> writes:
>>>>memmove(), and other standard library functions, are part of the
>>>>implementation. They were traditionally implemented in assembly, and
>>>>only later in C. But as part of the implementation, they can use
>>>>features that are implementation-specific - there was never any need or
>>>>intention that such functions should be implementable in portable
>>>>standards-compliant C. The language never had any need to support
>>>>writing an efficient memmove() in C - it gave you a ready-made memmove()
>>>>function so that implementing it yourself is not necessary.
>>>
>>>For a supposed systems programming language like C, the inability to
>>>implement memmove() is a failure.
>>
>>How so? A simple byte-by-byte loop, reversed if necessary,
>>is sufficient. E.g. subtract the two pointers and depending on whether
>>the subtraction is negative or positive do a byte-by-byte copy either
>>forward or in reverse. Perfectly legal C, if not particuarly performant.
>
>Subtraction of two pointers that do not point to the same object is
>undefined behaviour, too.

An array object that encompasses the entire virtual address space
is easily defined on modern (nonsegmented) processors, in which case
ptrdiff_t is defined for subtraction of any two virtual addresses.

On 10/04/2023 14:36, Anton Ertl wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> On 04/04/2023 07:39, Anton Ertl wrote:
>>> One reason for comparing pointers is if you want to implement
>>> memmove(), as discussed in the article.
>>
>> memmove(), and other standard library functions, are part of the
>> implementation. They were traditionally implemented in assembly, and
>> only later in C. But as part of the implementation, they can use
>> features that are implementation-specific - there was never any need or
>> intention that such functions should be implementable in portable
>> standards-compliant C. The language never had any need to support
>> writing an efficient memmove() in C - it gave you a ready-made memmove()
>> function so that implementing it yourself is not necessary.
>
> For a supposed systems programming language like C, the inability to
> implement memmove() is a failure.

If you want to implement a function "my_memmove" in C that acts like
"memmove" but is written in fully portable C, you can do so :

#include <string.h>

void * my_memmove(void * s1, const void * s2, size_t n) {
return memmove(s1, s2, n);
}

There. Finished.

That was perhaps a bit patronising, but the standard library is part of
C. It is not some extra feature - writing code that calls standard
library functions is as much "programming in C" as writing "for" loops
and using pointers to unsigned chars.

Now, I will certainly agree that it would be /nice/ to have a fully
portable way to implement the standard library function "memmove" in C.
I just don't agree that it is /necessary/ - or that the lack of such a
method is a "failure". As with any language, there are always things
that people would like to do in the language, but can't - the
alternative is that people will complain that the language (and/or
standard library) is too big. (See C++ for evidence of that!)

Writing a mostly-portable memmove() is not difficult - for a great many
implementations, you can convert the pointers to "uintptr_t" and use the
results to choose between forwards or backwards movement. For the
platforms where you can't do that, you're going to have no sensible
comparison of ordinary pointers anyway. But even if you could portably
compare pointers, you are going to have a great deal of difficulty
writing something that is efficient without making it
implementation-specific - the ideal lump size, unrolling thresholds,
alignment handling, etc., is going to be highly dependent on the target.

Thus on good quality compilers, real-life calls to functions like
"memmove" or "memcpy" will often result in direct code generation,
rather than a library call (they may even be omitted altogether).
Likewise, manual code may be turned into calls to memove() or memcpy().
The actual usefulness of being able to implement memmove() is rather
limited.

I do agree that it would be nice for arbitrary pointer comparisons to be
valid in cases where it makes sense, and being able to implement
"memmove" would fall out of that - but I don't think being able to
implement "memmove" is good motivation for such a feature. I'd rank
being able to sort pointers for making binary trees or other data
structures as being more relevant. (And perhaps also for garbage
collection algorithms or other points discussed in this thread - I don't
know enough about the implementations of such algorithms to comment.)

>
>> It's absolutely fine to say that you'd /like/ C to support comparison of
>> arbitrary pointers, and there are other uses that are more useful than
>> replicating existing standard library functions.
>
> "memmove()" more succinct than describing the general case of an
> operation that reads from one memory block and writes to a potentially
> overlapping block in a way that is equivalent to doing all the reading
> before doing any of the writing. We leave it to the intelligence of
> the reader to extrapolate from "memmove()" to the general case. And
> of course it's clear that an advocate like you will try to confuse
> things by pointing out that the special case that was so succinctly
> expressed can be fulfilled with the succinct library call.
>

I am trying to say how things are, to look at what things are important
or merely nice to have, and to look at the implications of insisting on
particular features. I am a believer in specifying things that are
useful to specify, and leaving unspecified or undefined things that are
not - because I find the freedom useful. I am fully aware that
different people would prefer different balances, and that different
languages have different balances from C. I also appreciate that many
people feel that C has design flaws or would prefer it to be somewhat
different - I don't know of anyone, including myself, who is a
significant user of C and who things the language or its standards are
ideal. All languages are compromises, because users disagree about
different things in the language.

The difference between us is not that /you/ think C is broken and /I/
think it is perfect - I don't think it is perfect. It is not even just
a matter of differing in which things we would want to change. It is
that I look at C and see a language which is quite good, but not quite
what I'd like - thus I learn about how it works and decide I can use it
even though it is sub-optimal for /me/. You, apparently, see the
aspects of C that you don't like and declare that the language is broken
and compilers are broken for the way they handle the aspects you dislike.

I am aware that my personal needs and preferences are my own, while you
think your needs and preferences are fundamental and apply to all. Is
that what you mean when you say I am "an advocate" ?

>>> If you
>>> want a language that has a flat address space, use Forth.
>>>
>>
>> There are plenty of systems where you don't have a single flat address
>> space, and it's nice to be able to use C on them. An example would be
>> the extremely popular AVR microcontrollers, where flash and ram IO
>> spaces are entirely separate (with different assembly instructions for
>> accessing them).
>
> There are several Forth systems for AVR microcontrollers. The
> Forth-Gesellschaft even made a whole special edition of it's magazine
> for AVR
> <https://forth-ev.de/wiki/res/lib/exe/fetch.php/vd-archiv:4d2007-sonderheft-avr.pdf>
>
> So a flat address space in the programming language does not prevent
> implementation on the AVR.
>

Correct. But Forth is not C.

>> And then you have segmented address architectures,
>> like x86 - comparing pointers for ordering can be complicated if
>> calculations are needed to determine the real address.
>
> There is no "x86" architecture. If you mean the 8086, there have been
> many Forth systems for that. Some are for the tiny or small memory
> models. If you want one that deals with more memory, look at F-PC (in
> particular, look for "Internals" in
> <https://github.com/uho/F-PC/blob/main/fpc/fpcwords.txt>). There the
> standard Forth words @, !, etc. access 64KB of memory (making F-PC a
> medium-model system in terms of 8086 memory models), but it also has
> words @L, !L, CMOVEL etc. for accessing more memory; the latter words
> did not find the way into the Forth standard, and nobody has ever
> publically missed them. Looking at
> <https://devblogs.microsoft.com/oldnewthing/20200728-00/?p=104012>,
> similar things were used in C, e.g., _fmemcpy() for copying involving
> far pointers (probably a non-standard C type).
>
>> But are you going to allow "p < q" to be valid
>> while "q - p" might not be, due to an overflow in the ptrdiff_t range?
>
> C has this problem even without the 8086 nonsense and with the
> restriction of comparing and subtracting only pointers to the same
> object. I.e., standard C has this problem. Consider a sane system
> like a PDP-11, and you have an array
>
> char c[40000U];
> char *p=c;
> char *q=&c[39999];
>
> The C standard defines what q<p means, but does not define what p-q
> means, does it?
>

The answer is implementation-dependent, but could certainly be "yes".

There are always going to be trade-offs in how much a language can
specify, its efficient use in some cases, and how sensible or useful the
specifications are.

On 10/04/2023 15:30, Terje Mathisen wrote:
> David Brown wrote:
>> On 04/04/2023 07:39, Anton Ertl wrote:
>>> One reason for comparing pointers is if you want to implement
>>> memmove(), as discussed in the article.
>>
>> memmove(), and other standard library functions, are part of the
>> implementation.  They were traditionally implemented in assembly, and
>> only later in C.  But as part of the implementation, they can use
>> features that are implementation-specific - there was never any need
>> or intention that such functions should be implementable in portable
>> standards-compliant C.  The language never had any need to support
>> writing an efficient memmove() in C - it gave you a ready-made
>> memmove() function so that implementing it yourself is not necessary.
>
> Is pointer comparison implementation defined (ID) or UB?

Pointer equality comparison is defined behaviour for any two pointers to
compatible types or void.

Comparison for order (less than or greater than) is valid only if the
two pointers point to something within the same aggregate object (or one
past the end of an array). Otherwise it is undefined behaviour.

>
> If it is ID, then I can write my own version of memmove() (or similar
> ops) on any given compiler, as long as I do it the way that particular
> compiler have defined it, right?

That would be correct. But since the comparison you need is for
ordering, and it is /not/ implementation-defined behaviour, you can't
rely on it for fully portable code.

Of course implementations are free to support comparisons that are not
defined by the C standards - for most targets, a "pointer" will simply
contain an address (possibly a virtual address), and implementations are
free to treat comparisons of arbitrary pointers as simple address
comparisons.

When implementing an arbitrary function that takes two void* pointers,
the implementation will have to assume that they /are/ pointing within
the same array or structure, unless it is able to prove otherwise. So
you can expect code to be generated accordingly. However, the compiler
can handle things differently if it is able to see the source of the
pointers at the same time as interpreting the comparison.

Pointers can also be converted back and forth between integer types, in
an implementation-dependent manner. There's no guarantee that a given
integer type will be big enough to support all pointer values - but if
there is such a type, then "uintptr_t" and "intptr_t" will be big enough
and can be used. So you can write your "memmove" function by casting
the pointers to "uintptr_t" and then doing your comparison. It is not
fully portable, but will work for almost any system except for a few
implementations that use wide pointers which encode access information
in addition to the address. (There's also the hypothetical possibility
that "(uintptr_t)(p + 1)" does not have to equal "(uintptr_t) p +
sizeof(*p)", but that's highly unlikely in practice.)

>
> If it is UB, then that's impossible, you have to step outside of the C
> standard to implement it, and do so in a way that makes it impossible
> for the compiler to discover that you have done so. :-)
>

No, UB does not mean this is impossible - it means it is not guaranteed
to be possible. There's a difference. In practice, almost all modern C
libraries will implement memmove() in C, relying on implementation
features that may or may not be documented but which the library
implementers know are valid for the library code. You are not stepping
outside C, merely outside of fully portable standards-compliant C. The
C standards never intended for the C standard library to be written in
"pure" portable C - the standard library is part of the implementation,
and its own source code can use special features of the implementation.

>> It's absolutely fine to say that you'd /like/ C to support comparison
>> of arbitrary pointers, and there are other uses that are more useful
>> than replicating existing standard library functions.  A lot of
>> implementations will support it - or at the very least, they will have
>> a "uintptr_t" type so that you can cast the pointers to uintptr_t
>> values and compare those.
>
> That's fine: If I can take two arbitrary pointers and convert them to
> uintptr_t, and the resulting values are comparable, at least to the
> point where it is possible to discover if one object could overlap
> another, then I can write my own memmove().
>

Yes - except that "uintptr_t" is not guaranteed to exist. I have never
used a C99+ system that did /not/ support it, but they do exist.

On 4/10/2023 9:34 AM, Scott Lurndal wrote:
> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> David Brown <david.brown@hesbynett.no> writes:
>>> On 04/04/2023 07:39, Anton Ertl wrote:
>>>> One reason for comparing pointers is if you want to implement
>>>> memmove(), as discussed in the article.
>>>
>>> memmove(), and other standard library functions, are part of the
>>> implementation. They were traditionally implemented in assembly, and
>>> only later in C. But as part of the implementation, they can use
>>> features that are implementation-specific - there was never any need or
>>> intention that such functions should be implementable in portable
>>> standards-compliant C. The language never had any need to support
>>> writing an efficient memmove() in C - it gave you a ready-made memmove()
>>> function so that implementing it yourself is not necessary.
>>
>> For a supposed systems programming language like C, the inability to
>> implement memmove() is a failure.
>
> How so? A simple byte-by-byte loop, reversed if necessary,
> is sufficient. E.g. subtract the two pointers and depending on whether
> the subtraction is negative or positive do a byte-by-byte copy either
> forward or in reverse. Perfectly legal C, if not particuarly performant.
>
> In real life, memmove is often implemented in assembler for performance.
>

On many (most?) modern targets, a naive byte-for-byte copy loop is
potentially orders of magnitude slower than using larger units (such as
copying the data 128 bits at a time, and overlapping the loads and
stores in the pipeline).

This was less true of 32-bit x86, where one is limited mostly to moving
32 bits at a time, and the limited register space isn't sufficient to
fully pipeline the copy. I had noted, for example, that on a 20 year old
laptop, this causes memcpy (for small buffers) to peak out at only
around 800 MB/s. This would drop to 200 MB/s if copying a byte at a time.

Granted, this is still faster than what one gets if the buffer doesn't
fit into the L1 cache (seemingly early 2000s RAM wasn't very fast by
modern standards).

My modern desktop PC is faster here on both metrics by over an order of
magnitude.

Granted, it may be possible to do better here, dunno.

Note that in general, laptop is fast enough to run Half-Life and Quake3.
Its GPU seems to be R100 based (fixed-function, OpenGL 1.3 and DX7).

Half-Life 2 sorta works (some graphical limitations and single-digit
frame-rates); pretty much anything newer is no-go.

And, ironically, the ~ 280 MB/s for L1 memcpy I can get on my BJX2 core
is only around 3x slower than said laptop (despite the laptop having a
fairly large clock-speed advantage).

Mostly all due to ability to do fully pipelined 128-bit load/store
sequences.

Meanwhile, what about a naive byte copy loop?...
How about around 4 MB/s ...

Note that in a naive sense, Doom would also be limited in this way,
however one can sorta organize the column drawing loops and similar such
that the inner parts handle multiple pixels at a time in a "closer to
pipelined" fashion, and thus "not suck quite as bad", mostly falling
back to "one pixel at a time" for the last few pixels of the column (*).

....

*: Had tested, and working directly in RGB555 with RGB modulation in
place of the colormap tables didn't really offer any real speed
advantage. However, because of the more accurate luma scaling, it caused
the "brightness drops off slightly with distance" effect to have much
more obvious banding artifacts (with the colormap tables, these add a
little jitter that reduces the effect).

My current port still uses color-map tables, however it has switched to
partially blending the original palette colors with a calculated
RGB-scaled value, slightly increasing luma accuracy but not so much as
to cause the banding to become super obvious (the color map tables and
framebuffer in this case operating using RGB555 / hi-color).

David Brown <david.brown@hesbynett.no> writes:
>On 10/04/2023 14:36, Anton Ertl wrote:
>> David Brown <david.brown@hesbynett.no> writes:
>>> On 04/04/2023 07:39, Anton Ertl wrote:
[...]
>#include <string.h>
>
>void * my_memmove(void * s1, const void * s2, size_t n) {
> return memmove(s1, s2, n);
>}
>
>There. Finished.

Well done. Next:

vadd(double *a, double *b, double c, size_t n)
{ /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
parameters are passed according to C rules, so a and b may or may
not be pointers into the same array */
}

>As with any language, there are always things
>that people would like to do in the language, but can't - the
>alternative is that people will complain that the language (and/or
>standard library) is too big.

C already includes pointer comparison. Making it undefined for
certain cases increases the things a programmer has to remember in
order to use C, i.e., the language the programmer experiences becomes
bigger.

>(See C++ for evidence of that!)

Does C++ define pointer comparison? I don't think so. So C++ bigness
is, at best, an argument for my position.

>The difference between us is not that /you/ think C is broken and /I/
>think it is perfect - I don't think it is perfect. It is not even just
>a matter of differing in which things we would want to change. It is
>that I look at C and see a language which is quite good, but not quite
>what I'd like - thus I learn about how it works and decide I can use it
>even though it is sub-optimal for /me/. You, apparently, see the
>aspects of C that you don't like and declare that the language is broken
>and compilers are broken for the way they handle the aspects you dislike.

No, the difference is that I see production C programs that are
compiled as intended by some compilers, and if another release of the
compiler miscompiles the program, I see this as bug in the compiler.
Some people claim that the program is buggy and has always been, and
base this claim on certain parts of the C standard. Of course, they
prefer to ignore other parts of the C standard, such as the parts
about conforming and strictly conforming C programs.

>>> There are plenty of systems where you don't have a single flat address
>>> space, and it's nice to be able to use C on them. An example would be
>>> the extremely popular AVR microcontrollers, where flash and ram IO
>>> spaces are entirely separate (with different assembly instructions for
>>> accessing them).
>>
>> There are several Forth systems for AVR microcontrollers. The
>> Forth-Gesellschaft even made a whole special edition of it's magazine
>> for AVR
>> <https://forth-ev.de/wiki/res/lib/exe/fetch.php/vd-archiv:4d2007-sonderheft-avr.pdf>
>>
>> So a flat address space in the programming language does not prevent
>> implementation on the AVR.
>>
>
>Correct. But Forth is not C.

And the relevance of that is? If Forth can be implement on the AVR,
surely a flat-address-space C can be implemented on the AVR, too.

>> Consider a sane system
>> like a PDP-11, and you have an array
>>
>> char c[40000U];
>> char *p=c;
>> char *q=&c[39999];
>>
>> The C standard defines what q<p means, but does not define what p-q
>> means, does it?
>>
>
>The answer is implementation-dependent, but could certainly be "yes".

Is this another of your dreamed-up scenarios, one where ptrdiff_t on a
PDP-11 is larger than 16 bits? Or can you name an implementation
where p-q is defined for the case above?

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On Monday, April 10, 2023 at 9:00:45 PM UTC+3, Anton Ertl wrote:
> David Brown <david...@hesbynett.no> writes:
> >On 10/04/2023 14:36, Anton Ertl wrote:
> >> David Brown <david...@hesbynett.no> writes:
> >>> On 04/04/2023 07:39, Anton Ertl wrote:
> [...]
> >#include <string.h>
> >
> >void * my_memmove(void * s1, const void * s2, size_t n) {
> > return memmove(s1, s2, n);
> >}
> >
> >There. Finished.
> Well done. Next:
>
> vadd(double *a, double *b, double c, size_t n)
> {
> /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
> parameters are passed according to C rules, so a and b may or may
> not be pointers into the same array */
> }
>

In Fortran, when a and b are pointers to the same array the result is
undefined. I suppose, it's not what you had in mind.

I suppose, you meant to produce the same result as slow_vadd() but efficiently.
slow_vadd(double *a, double *b, double c, size_t n)
{ double* tmp = malloc(n*sizeof(*tmp));
for (size_t i = 0; i < n; ++i)
tmp[i] = b[i] + c;
memcpy(a, tmp, n*sizeof(*tmp));
}

On Monday, April 10, 2023 at 9:34:37 PM UTC+3, Michael S wrote:
> On Monday, April 10, 2023 at 9:00:45 PM UTC+3, Anton Ertl wrote:
> > David Brown <david...@hesbynett.no> writes:
> > >On 10/04/2023 14:36, Anton Ertl wrote:
> > >> David Brown <david...@hesbynett.no> writes:
> > >>> On 04/04/2023 07:39, Anton Ertl wrote:
> > [...]
> > >#include <string.h>
> > >
> > >void * my_memmove(void * s1, const void * s2, size_t n) {
> > > return memmove(s1, s2, n);
> > >}
> > >
> > >There. Finished.
> > Well done. Next:
> >
> > vadd(double *a, double *b, double c, size_t n)
> > {
> > /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
> > parameters are passed according to C rules, so a and b may or may
> > not be pointers into the same array */
> > }
> >
> In Fortran, when a and b are pointers to the same array the result is
> undefined. I suppose, it's not what you had in mind.
>
> I suppose, you meant to produce the same result as slow_vadd() but efficiently.
> slow_vadd(double *a, double *b, double c, size_t n)
> {
> double* tmp = malloc(n*sizeof(*tmp));
> for (size_t i = 0; i < n; ++i)
> tmp[i] = b[i] + c;
> memcpy(a, tmp, n*sizeof(*tmp));
> }

Ogh, I forgot free(tmp);

On 2023-04-10 21:34, Michael S wrote:
> On Monday, April 10, 2023 at 9:00:45 PM UTC+3, Anton Ertl wrote:
>> David Brown <david...@hesbynett.no> writes:
>>> On 10/04/2023 14:36, Anton Ertl wrote:
>>>> David Brown <david...@hesbynett.no> writes:
>>>>> On 04/04/2023 07:39, Anton Ertl wrote:
>> [...]
>>> #include <string.h>
>>>
>>> void * my_memmove(void * s1, const void * s2, size_t n) {
>>> return memmove(s1, s2, n);
>>> }
>>>
>>> There. Finished.
>> Well done. Next:
>>
>> vadd(double *a, double *b, double c, size_t n)
>> {
>> /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
>> parameters are passed according to C rules, so a and b may or may
>> not be pointers into the same array */
>> }
>>
>
> In Fortran, when a and b are pointers to the same array the result is
> undefined. I suppose, it's not what you had in mind.

Are you sure about that? The F2018 Interpretation Document states, for
an assignment of the form "variable = expr", that:

"The execution of the assignment shall have the same effect as if the
evaluation of expr and the evaluation of all expressions in variable
occurred before any portion of the variable is defined by the assignment."

(https://j3-fortran.org/doc/year/18/18-007r1.pdf, section 10.2.1.2)

So the Fortran assignment should work as if the right-hand expr is
evaluated into a temporary, and then the temporary is assigned (moved)
to the left-hand variable.

Michael S <already5chosen@yahoo.com> writes:
>On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
>> vadd(double *a, double *b, double c, size_t n)=20
>> {=20
>> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
>=20
>> parameters are passed according to C rules, so a and b may or may=20
>> not be pointers into the same array */
>> }=20
>>=20
>
>In Fortran, when a and b are pointers to the same array the result is=20
>undefined.

Thomas Koenig wrote something along the lines that there are standard
Fortran idioms that generate aliases, and that if the compiler does
not know that the operands in an array expression may overlap, the
compiler has to generate code that takes potential overlap into
account. What in particular is undefined IYO?

>I suppose, you meant to produce the same result as slow_vadd() but efficie=
>ntly.
>slow_vadd(double *a, double *b, double c, size_t n)
>{
> double* tmp =3D malloc(n*sizeof(*tmp));
> for (size_t i =3D 0; i < n; ++i)
> tmp[i] =3D b[i] + c;
> memcpy(a, tmp, n*sizeof(*tmp)); =20
>}

And with a guarantee that the malloc() always succeeds.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>Michael S <already5chosen@yahoo.com> writes:
>>On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
>>> vadd(double *a, double *b, double c, size_t n)=20
>>> {=20
>>> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
>>=20
>>> parameters are passed according to C rules, so a and b may or may=20
>>> not be pointers into the same array */
>>> }=20
>>>=20
>>
>>In Fortran, when a and b are pointers to the same array the result is=20
>>undefined.
>
>Thomas Koenig wrote something along the lines that there are standard
>Fortran idioms that generate aliases, and that if the compiler does
>not know that the operands in an array expression may overlap, the
>compiler has to generate code that takes potential overlap into
>account. What in particular is undefined IYO?
>
>>I suppose, you meant to produce the same result as slow_vadd() but efficie=
>>ntly.
>>slow_vadd(double *a, double *b, double c, size_t n)
>>{
>> double* tmp =3D malloc(n*sizeof(*tmp));
>> for (size_t i =3D 0; i < n; ++i)
>> tmp[i] =3D b[i] + c;
>> memcpy(a, tmp, n*sizeof(*tmp)); =20
>>}
>
>And with a guarantee that the malloc() always succeeds.

use alloca(3) instead of malloc(3) and you don't even
need to explicitly free it before slow_vadd returns.

Of course, the stack region would need to be sufficiently
large for 'n' (which is a per-process resource limit on linux, easily
adjusted - particularly on 64-bit systems were large virtual address
spaces are available).

On Monday, April 10, 2023 at 11:55:36 PM UTC+3, Niklas Holsti wrote:
> On 2023-04-10 21:34, Michael S wrote:
> > On Monday, April 10, 2023 at 9:00:45 PM UTC+3, Anton Ertl wrote:
> >> David Brown <david...@hesbynett.no> writes:
> >>> On 10/04/2023 14:36, Anton Ertl wrote:
> >>>> David Brown <david...@hesbynett.no> writes:
> >>>>> On 04/04/2023 07:39, Anton Ertl wrote:
> >> [...]
> >>> #include <string.h>
> >>>
> >>> void * my_memmove(void * s1, const void * s2, size_t n) {
> >>> return memmove(s1, s2, n);
> >>> }
> >>>
> >>> There. Finished.
> >> Well done. Next:
> >>
> >> vadd(double *a, double *b, double c, size_t n)
> >> {
> >> /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
> >> parameters are passed according to C rules, so a and b may or may
> >> not be pointers into the same array */
> >> }
> >>
> >
> > In Fortran, when a and b are pointers to the same array the result is
> > undefined. I suppose, it's not what you had in mind.
> Are you sure about that?

No, I don't.
I just assumed that rules for subroutines apply to "implicit subrotines" like above.

> The F2018 Interpretation Document states, for
> an assignment of the form "variable = expr", that:
>
> "The execution of the assignment shall have the same effect as if the
> evaluation of expr and the evaluation of all expressions in variable
> occurred before any portion of the variable is defined by the assignment."
>
> (https://j3-fortran.org/doc/year/18/18-007r1.pdf, section 10.2.1.2)
>
> So the Fortran assignment should work as if the right-hand expr is
> evaluated into a temporary, and then the temporary is assigned (moved)
> to the left-hand variable.

If above applies to vector portion of the language then I am wrong.

The question is - how could we possibly cause a and b to refer to different
overlapping portions of the same array?
I can think about two ways - common block in the same compilation module,
in which case compiler knows exactly about relationships, and passing to
subroutine, in which case UB happens before we got to our line of interest.
But my knowledge of Fortran is really weak, possibly there are other ways.

On Tuesday, April 11, 2023 at 12:13:31 AM UTC+3, Scott Lurndal wrote:
> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >Michael S <already...@yahoo.com> writes:
> >>On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
> >>> vadd(double *a, double *b, double c, size_t n)=20
> >>> {=20
> >>> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
> >>=20
> >>> parameters are passed according to C rules, so a and b may or may=20
> >>> not be pointers into the same array */
> >>> }=20
> >>>=20
> >>
> >>In Fortran, when a and b are pointers to the same array the result is=20
> >>undefined.
> >
> >Thomas Koenig wrote something along the lines that there are standard
> >Fortran idioms that generate aliases, and that if the compiler does
> >not know that the operands in an array expression may overlap, the
> >compiler has to generate code that takes potential overlap into
> >account. What in particular is undefined IYO?
> >
> >>I suppose, you meant to produce the same result as slow_vadd() but efficie=
> >>ntly.
> >>slow_vadd(double *a, double *b, double c, size_t n)
> >>{
> >> double* tmp =3D malloc(n*sizeof(*tmp));
> >> for (size_t i =3D 0; i < n; ++i)
> >> tmp[i] =3D b[i] + c;
> >> memcpy(a, tmp, n*sizeof(*tmp)); =20
> >>}
> >
> >And with a guarantee that the malloc() always succeeds.
> use alloca(3) instead of malloc(3) and you don't even
> need to explicitly free it before slow_vadd returns.
>
> Of course, the stack region would need to be sufficiently
> large for 'n' (which is a per-process resource limit on linux, easily
> adjusted - particularly on 64-bit systems were large virtual address
> spaces are available).

With C99 we don't even have to use alloca() which IIRC is not the part of
the standard, although widely available in practice.
We can express ourselves cleaner with VLA.
However the topic is about C23 and in C23 VLA support is optional.
Anyway, that's off topic of off topic.

On Tuesday, April 11, 2023 at 12:10:14 AM UTC+3, Anton Ertl wrote:
> Michael S <already...@yahoo.com> writes:
> >On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
> >> vadd(double *a, double *b, double c, size_t n)=20
> >> {=20
> >> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
> >=20
> >> parameters are passed according to C rules, so a and b may or may=20
> >> not be pointers into the same array */
> >> }=20
> >>=20
> >
> >In Fortran, when a and b are pointers to the same array the result is=20
> >undefined.
>
> Thomas Koenig wrote something along the lines that there are standard
> Fortran idioms that generate aliases, and that if the compiler does
> not know that the operands in an array expression may overlap, the
> compiler has to generate code that takes potential overlap into
> account. What in particular is undefined IYO?
>

I am pretty sure that it is opposite way around, at least for most common
form of Fortran parameters passing, i.e. passing array - compiler assumes
no aliasing unless it knows for sure that aliasing presents.
For a newer less common form of parameter passing, i.e. pointers, the rules
are similar to 'C'. But I don't know if syntax like in your example is legal for
both for arrays and for pointers.

As discussed above, your example is not a subroutine call, so let's say that
I am no more sure in anything.

May be, both of us should not make statements about Fortran, because both
of us lack the knowledge. Otherwise we look too much like few other c.a. posters
that like to discuss Ada without ever reading a single Ada reference book about it
or ever writing a single Ada program.

> >I suppose, you meant to produce the same result as slow_vadd() but efficie> >ntly.
> >slow_vadd(double *a, double *b, double c, size_t n)
> >{
> > double* tmp =3D malloc(n*sizeof(*tmp));
> > for (size_t i =3D 0; i < n; ++i)
> > tmp[i] =3D b[i] + c;
> > memcpy(a, tmp, n*sizeof(*tmp)); =20
> >}
>
> And with a guarantee that the malloc() always succeeds.
> - anton
> --
> 'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
> Mitch Alsup, <c17fcd89-f024-40e7...@googlegroups.com>

On 4/10/2023 4:13 PM, Scott Lurndal wrote:
> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>> Michael S <already5chosen@yahoo.com> writes:
>>> On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
>>>> vadd(double *a, double *b, double c, size_t n)=20
>>>> {=20
>>>> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
>>> =20
>>>> parameters are passed according to C rules, so a and b may or may=20
>>>> not be pointers into the same array */
>>>> }=20
>>>> =20
>>>
>>> In Fortran, when a and b are pointers to the same array the result is=20
>>> undefined.
>>
>> Thomas Koenig wrote something along the lines that there are standard
>> Fortran idioms that generate aliases, and that if the compiler does
>> not know that the operands in an array expression may overlap, the
>> compiler has to generate code that takes potential overlap into
>> account. What in particular is undefined IYO?
>>
>>> I suppose, you meant to produce the same result as slow_vadd() but efficie=
>>> ntly.
>>> slow_vadd(double *a, double *b, double c, size_t n)
>>> {
>>> double* tmp =3D malloc(n*sizeof(*tmp));
>>> for (size_t i =3D 0; i < n; ++i)
>>> tmp[i] =3D b[i] + c;
>>> memcpy(a, tmp, n*sizeof(*tmp)); =20
>>> }
>>
>> And with a guarantee that the malloc() always succeeds.
>
> use alloca(3) instead of malloc(3) and you don't even
> need to explicitly free it before slow_vadd returns.
>
> Of course, the stack region would need to be sufficiently
> large for 'n' (which is a per-process resource limit on linux, easily
> adjusted - particularly on 64-bit systems were large virtual address
> spaces are available).

FWIW, some of us implemented "alloca()" as a wrapper on top of
"malloc()" with an "automatically free when function returns" mechanism
(alloca adds the object to an invisible linked list, and the compiler
then inserts a runtime call to a function that frees everything in the
list).

This mechanism was also used for VLAs and for arrays exceeding a certain
maximum cut-off size, ...

This being partly because:
Variable size stack frames are evil;
If one excludes large allocations from the stack (including large
arrays), one may find that 128k of stack space is sufficient for most
programs.

Though, this is more relevant for NOMMU systems, as with an MMU, burning
1MB (or several MB) of virtual address space on a stack is less of an issue.

But, it is maybe a waste to spend 1MB on the stack for a program that
can easily fit into 64k or 128k.

However, it is a tradeoff, because if too many of the arrays that might
have otherwise been put onto the stack are put into the heap, this may
have an adverse effect on performance. So, the limit might still end up
being in kB territory (and as such, only a few levels with big arrays,
or a modestly recursive program with large stack frames, may still end
up blowing out the stack).

....

Niklas Holsti <niklas.holsti@tidorum.invalid> schrieb:
> On 2023-04-10 21:34, Michael S wrote:
>> On Monday, April 10, 2023 at 9:00:45 PM UTC+3, Anton Ertl wrote:
>>> David Brown <david...@hesbynett.no> writes:
>>>> On 10/04/2023 14:36, Anton Ertl wrote:
>>>>> David Brown <david...@hesbynett.no> writes:
>>>>>> On 04/04/2023 07:39, Anton Ertl wrote:
>>> [...]
>>>> #include <string.h>
>>>>
>>>> void * my_memmove(void * s1, const void * s2, size_t n) {
>>>> return memmove(s1, s2, n);
>>>> }
>>>>
>>>> There. Finished.
>>> Well done. Next:
>>>
>>> vadd(double *a, double *b, double c, size_t n)
>>> {
>>> /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
>>> parameters are passed according to C rules, so a and b may or may
>>> not be pointers into the same array */
>>> }

The exact equivalent is impossible to write, because Fortran does not
support pointer arithmetic.

What you wrote is what f2c or nagfor (which compile Fortran to C) would generate
for

SUBROUTINE VADD (A, B, C, N)
INTEGER N
DOUBLE PRECISION A(N), B(N), C

but this is an implementation detail, other argument passing conventions
are possible.

And if they overlap, the program is not valid Fortran, which is
I assume what MichaelS meant when he wrote

>> In Fortran, when a and b are pointers to the same array the result is
>> undefined. I suppose, it's not what you had in mind.

so the task above is actually ill-formed, and does not constitute an
argument for or against C's suitability as a systems level language.

> Are you sure about that? The F2018 Interpretation Document states, for
> an assignment of the form "variable = expr", that:
>
> "The execution of the assignment shall have the same effect as if the
> evaluation of expr and the evaluation of all expressions in variable
> occurred before any portion of the variable is defined by the assignment."
>
> (https://j3-fortran.org/doc/year/18/18-007r1.pdf, section 10.2.1.2)

That is correct. If arguments are passed separately, like in
Anton's example, then the compiler can determine that there is
no overlap, and procede with a straightforward loop.

> So the Fortran assignment should work as if the right-hand expr is
> evaluated into a temporary, and then the temporary is assigned (moved)
> to the left-hand variable.

Correct. And this is what compilers actually do. For example,
the code

subroutine foo(a,n,i,j,m)
real, dimension(n) :: a
a(i:i+m) = a (j:j+m) + 42.
end subroutine foo

which realizes such an overlap is actually translated to (part of
gfortran's -fdump-tree-original output)

D.3912 = (void * restrict) __builtin_malloc (D.3910 ? 1 : MAX_EXPR <(unsigned long) (D.3911 * 4), 1>);
D.3913 = D.3912;
atmp.2.data = D.3913;
atmp.2.offset = 0;
D.3914 = NON_LVALUE_EXPR <D.3882>;
D.3915 = NON_LVALUE_EXPR <D.3884>;
{
integer(kind=8) S.3;

S.3 = 0;
while (1)
{
if (S.3 > D.3886) goto L.1;
(*(real(kind=4)[0:] * restrict) atmp.2.data)[S.3] = (*a)[(S.3 + D.3914) + -1] + 4.2e+1;
S.3 = S.3 + 1;
}
L.1:;
S.3 = 0;
while (1)
{
if (S.3 > D.3886) goto L.2;
(*a)[(S.3 + D.3915) + -1] = (*(real(kind=4)[0:] * restrict) atmp.2.data)[S.3];
S.3 = S.3 + 1;
}
L.2:;
}
__builtin_free ((void *) atmp.2.data);
}

so this uses a temporary (same with nagfor). This particular case
is a known missed optimization because conditional loop reversal
would work here, see https://gcc.gnu.org/bugzilla/PR108665 .

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> Michael S <already5chosen@yahoo.com> writes:
>>On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
>>> vadd(double *a, double *b, double c, size_t n)=20
>>> {=20
>>> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
>>=20
>>> parameters are passed according to C rules, so a and b may or may=20
>>> not be pointers into the same array */
>>> }=20
>>>=20
>>
>>In Fortran, when a and b are pointers to the same array the result is=20
>>undefined.
>
> Thomas Koenig wrote something along the lines that there are standard
> Fortran idioms that generate aliases, and that if the compiler does
> not know that the operands in an array expression may overlap, the
> compiler has to generate code that takes potential overlap into
> account. What in particular is undefined IYO?

It's not related to normal argument passing.

What you may be referring to is when arrays have to be packed.
Assume

call (a(1:n:2),size(a(1:n:2))

subroutine foo(a,n)
real :: a(n)

where two different passing conventions are mixed. Here, foo has to
receive a contiguous argument, and is passed a non-contiguous one.
For this, packing (and unpacking, because INTENT is not specified)
is necessary.

On Monday, April 10, 2023 at 11:18:47 PM UTC-4, BGB wrote:
> On 4/10/2023 4:13 PM, Scott Lurndal wrote:
> > an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >> Michael S <already...@yahoo.com> writes:
> >>> On Monday, April 10, 2023 at 9:00:45=E2=80=AFPM UTC+3, Anton Ertl wrote:
> >>>> vadd(double *a, double *b, double c, size_t n)=20
> >>>> {=20
> >>>> /* write the equivalent of the Fortran a(1:n)=3Db(1:n)+c; note that the=
> >>> =20
> >>>> parameters are passed according to C rules, so a and b may or may=20
> >>>> not be pointers into the same array */
> >>>> }=20
> >>>> =20
> >>>
> >>> In Fortran, when a and b are pointers to the same array the result is=20
> >>> undefined.
> >>
> >> Thomas Koenig wrote something along the lines that there are standard
> >> Fortran idioms that generate aliases, and that if the compiler does
> >> not know that the operands in an array expression may overlap, the
> >> compiler has to generate code that takes potential overlap into
> >> account. What in particular is undefined IYO?
> >>
> >>> I suppose, you meant to produce the same result as slow_vadd() but efficie=
> >>> ntly.
> >>> slow_vadd(double *a, double *b, double c, size_t n)
> >>> {
> >>> double* tmp =3D malloc(n*sizeof(*tmp));
> >>> for (size_t i =3D 0; i < n; ++i)
> >>> tmp[i] =3D b[i] + c;
> >>> memcpy(a, tmp, n*sizeof(*tmp)); =20
> >>> }
> >>
> >> And with a guarantee that the malloc() always succeeds.
> >
> > use alloca(3) instead of malloc(3) and you don't even
> > need to explicitly free it before slow_vadd returns.
> >
> > Of course, the stack region would need to be sufficiently
> > large for 'n' (which is a per-process resource limit on linux, easily
> > adjusted - particularly on 64-bit systems were large virtual address
> > spaces are available).
> FWIW, some of us implemented "alloca()" as a wrapper on top of
> "malloc()" with an "automatically free when function returns" mechanism
> (alloca adds the object to an invisible linked list, and the compiler
> then inserts a runtime call to a function that frees everything in the
> list).
>
> This mechanism was also used for VLAs and for arrays exceeding a certain
> maximum cut-off size, ...
>
>
> This being partly because:
> Variable size stack frames are evil;
> If one excludes large allocations from the stack (including large
> arrays), one may find that 128k of stack space is sufficient for most
> programs.

>
> Though, this is more relevant for NOMMU systems, as with an MMU, burning
> 1MB (or several MB) of virtual address space on a stack is less of an issue.
>
> But, it is maybe a waste to spend 1MB on the stack for a program that
> can easily fit into 64k or 128k.
>
>
> However, it is a tradeoff, because if too many of the arrays that might
> have otherwise been put onto the stack are put into the heap, this may
> have an adverse effect on performance. So, the limit might still end up
> being in kB territory (and as such, only a few levels with big arrays,
> or a modestly recursive program with large stack frames, may still end
> up blowing out the stack).
>
> ...
I agree.

I have seen a very large piece of software using only a few kB of stack.
It went 68 levels deep on stack.
A16kB memory page may be enough.

On 10/04/2023 17:54, Anton Ertl wrote:
> David Brown <david.brown@hesbynett.no> writes:
>> On 10/04/2023 14:36, Anton Ertl wrote:
>>> David Brown <david.brown@hesbynett.no> writes:
>>>> On 04/04/2023 07:39, Anton Ertl wrote:
> [...]
>> #include <string.h>
>>
>> void * my_memmove(void * s1, const void * s2, size_t n) {
>> return memmove(s1, s2, n);
>> }
>>
>> There. Finished.
>
> Well done. Next:
>
> vadd(double *a, double *b, double c, size_t n)
> {
> /* write the equivalent of the Fortran a(1:n)=b(1:n)+c; note that the
> parameters are passed according to C rules, so a and b may or may
> not be pointers into the same array */
> }

I don't know much about Fortran, and therefore don't know the details of
the semantics. I believe in Fortran there is always the assumption that
such pointers are completely independent (that's one of the reasons why
some things can be implemented more efficiently in Fortran than C, at
least until "restrict" was added in C99). But I assume you mean it is
quite possible for a(1:n) and b(1:n) to overlap, otherwise your question
would be pointless.

The answer is just to write non-portable code - cast your pointers to
uintptr_t and do the comparison there. (I'd first question the
requirements of the function, however.)

>
>> As with any language, there are always things
>> that people would like to do in the language, but can't - the
>> alternative is that people will complain that the language (and/or
>> standard library) is too big.
>
> C already includes pointer comparison.

It does, yes - but like everything else in the language (and in all
other well-specified and standardised languages), it works according to
the definition in the standard. It does not work according to whatever
idea pops into some programmer's head - no matter how rational or useful
that idea may seem.

> Making it undefined for
> certain cases increases the things a programmer has to remember in
> order to use C, i.e., the language the programmer experiences becomes
> bigger.

It's hardly rocket science. If you want to use a programming language,
you have to learn it. All programming languages let you express things
that don't make sense. Some try harder than others to force you to
write correct code - others, like C, trust the programmer to know what
he/she is doing.

>
>> (See C++ for evidence of that!)
>
> Does C++ define pointer comparison? I don't think so. So C++ bigness
> is, at best, an argument for my position.

The point is that if you keep adding features to a language, people will
complain that there are too many features.

>
>> The difference between us is not that /you/ think C is broken and /I/
>> think it is perfect - I don't think it is perfect. It is not even just
>> a matter of differing in which things we would want to change. It is
>> that I look at C and see a language which is quite good, but not quite
>> what I'd like - thus I learn about how it works and decide I can use it
>> even though it is sub-optimal for /me/. You, apparently, see the
>> aspects of C that you don't like and declare that the language is broken
>> and compilers are broken for the way they handle the aspects you dislike.
>
> No, the difference is that I see production C programs that are
> compiled as intended by some compilers, and if another release of the
> compiler miscompiles the program, I see this as bug in the compiler.

So if one program works by luck rather than design, all future compilers
and language standards have to enshrine that luck as though it were a
rule? What a totally bizarre concept. Seriously - what you write is
absolute piddle. And I don't think you really mean it. I think you
mean that /some/ such code should be supported for ever more - such as
the code /you/ write. You probably also think it should apply to some
of the software you use - but not code /you/ deem to be badly written or
not particularly important. Compiler writers should read /your/ mind
and understand /your/ personal, arbitrary and vague idea of where to
draw the line between "production C programs" and "bad C code", and
support these "production C programs" for all eternity.

Honestly, I /do/ understand wanting such as coding utopia. What I
completely fail to understand is how anyone would think it is remotely
realistic - and how you can fail to understand it is a matter of one
person believing that his own personal ideas are somehow absolute and
objective, and should apply to everyone.

Programming is based on contracts. The programmer promises to write
code according to the rules of the language. The compiler promises to
implement any code written according to the rules. If the programmer
breaks their side of the bargain, the compiler is under no obligation to
read their mind and generate object code to match the programmer's
unwritten intentions.

It is /irrelevant/ what the programmer thinks the rules should be - even
if they are good, useful and practical ideas. It is only the rules
specified in the standards and any other documentation (such as compiler
manuals) that are relevant.

Of course real-world tools go way beyond this minimum. They work hard
to provide help to developers - such as by static analysis, run-time
checkers, and other tools to help you get things right. Some compilers
give extra guarantees or features, making the programmer's life easier
at the expense of non-portable code. (There's nothing wrong with
writing non-portable code, if that's appropriate for the task in hand.)

But in the end, regardless of the features of the tool, if the
programmer does not know the relevant parts of the language they are
writing, that is /their/ problem, not the compiler's. If they know the
language details and still write incorrect code, that is /their/ problem.

> Some people claim that the program is buggy and has always been, and
> base this claim on certain parts of the C standard. Of course, they
> prefer to ignore other parts of the C standard, such as the parts
> about conforming and strictly conforming C programs.
>

>
>>> Consider a sane system
>>> like a PDP-11, and you have an array
>>>
>>> char c[40000U];
>>> char *p=c;
>>> char *q=&c[39999];
>>>
>>> The C standard defines what q<p means, but does not define what p-q
>>> means, does it?
>>>
>>
>> The answer is implementation-dependent, but could certainly be "yes".
>
> Is this another of your dreamed-up scenarios, one where ptrdiff_t on a
> PDP-11 is larger than 16 bits? Or can you name an implementation
> where p-q is defined for the case above?
>

From C99 until C23, ptrdiff_t was minimum 17 bits in size. (Prior to
C99, the minimum size was not specified.) Maybe the PDP-11 never had a
compliant C99 compiler - it's not a system I have ever used personally.

BGB <cr88192@gmail.com> writes:
>On 4/10/2023 4:13 PM, Scott Lurndal wrote:
>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>> Michael S <already5chosen@yahoo.com> writes:

>This being partly because:
>Variable size stack frames are evil;
>If one excludes large allocations from the stack (including large
>arrays), one may find that 128k of stack space is sufficient for most
>programs.

It's not 1989 anymore.

On Tuesday, April 11, 2023 at 9:32:07 AM UTC-4, Scott Lurndal wrote:
> BGB <cr8...@gmail.com> writes:
> >On 4/10/2023 4:13 PM, Scott Lurndal wrote:
> >> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >>> Michael S <already...@yahoo.com> writes:
>
> >This being partly because:
> >Variable size stack frames are evil;
> >If one excludes large allocations from the stack (including large
> >arrays), one may find that 128k of stack space is sufficient for most
> >programs.
> It's not 1989 anymore.

I may be a bit out of date here, but, I think some rules still apply.
Passing large objects like arrays as parameters on the stack is not
generally a good idea. It may lead to copying lots of memory around
uselessly. Objects can be allocated on the stack but usually they are
allocated on the heap especially if they are large, so they can be
managed. Allocating them on the stack forces them to be deallocated
at method return.
I usually try and statically allocate any buffers that are known to be
large and can be of a fixed size. This causes the program to fail right
away at startup, instead of in the middle after processing tons of
records. It also frees the program from doing memory management
work while it is running, and moves the work to startup and shutdown.
But a lot depends on the purpose and type of application.
Having said my two cents, I do use a lot more stack space in programs
that I used to, simply because the space is available. It can reduce
development time, not thinking through every algorithm. Development
time cost. I used to do a lot of high-speed development with
“throw-away” apps.

On 11/04/2023 16:36, robf...@gmail.com wrote:
> On Tuesday, April 11, 2023 at 9:32:07 AM UTC-4, Scott Lurndal wrote:
>> BGB <cr8...@gmail.com> writes:
>>> On 4/10/2023 4:13 PM, Scott Lurndal wrote:
>>>> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>>> Michael S <already...@yahoo.com> writes:
>>
>>> This being partly because:
>>> Variable size stack frames are evil;
>>> If one excludes large allocations from the stack (including large
>>> arrays), one may find that 128k of stack space is sufficient for most
>>> programs.
>> It's not 1989 anymore.
>
> I may be a bit out of date here, but, I think some rules still apply.
> Passing large objects like arrays as parameters on the stack is not
> generally a good idea. It may lead to copying lots of memory around
> uselessly.

This is jumbling several different concepts which may or may not be an
issue.

Useless copying of lots of memory is never good for efficiency, but
that's completely orthogonal to putting things on the stack rather than
the heap (or statically allocating them). You can put large arrays on
the stack without ever making unnecessary copies of them.

Variable stack frames are also orthogonal to the size of the stack
frames. Variable size stack frames are harder to analyse to be sure of
correctness, but can be more efficient (in memory use, and especially
cache use) than larger fixed size frames.

Stacks are sometimes limited in size, depending on the target system and
OS, and sometimes they grow automatically (limited only by actual memory
available). They are less flexible for allocation than the heap as you
have to deallocate in strict reverse order, but they can be much faster
to use.

> Objects can be allocated on the stack but usually they are
> allocated on the heap especially if they are large, so they can be
> managed. Allocating them on the stack forces them to be deallocated
> at method return.

In my world, small-systems embedded programming, the heap should be
avoided unless there is no other way to handle the task in hand.
Putting objects - even big ones and variable sized ones - on the stack
is fine, as long as you are sure of the maximum sizes and the amount of
stack you have available. Requirements vary for different kinds of
programming.

> I usually try and statically allocate any buffers that are known to be
> large and can be of a fixed size.

I do too.

> This causes the program to fail right
> away at startup, instead of in the middle after processing tons of
> records. It also frees the program from doing memory management
> work while it is running, and moves the work to startup and shutdown.
> But a lot depends on the purpose and type of application.

MitchAlsup <MitchAlsup@aol.com> writes:
>On Tuesday, April 11, 2023 at 8:32:07=E2=80=AFAM UTC-5, Scott Lurndal wrote=
>:
>> BGB <cr8...@gmail.com> writes:=20
>> >On 4/10/2023 4:13 PM, Scott Lurndal wrote:=20
>> >> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:=20
>> >>> Michael S <already...@yahoo.com> writes:=20
>>=20
>> >This being partly because:=20
>> >Variable size stack frames are evil;=20
>> >If one excludes large allocations from the stack (including large=20
>> >arrays), one may find that 128k of stack space is sufficient for most=20
>> >programs.
><
>> It's not 1989 anymore.
><
>Algol60 had no where other than the stack to put dynamically allocated=20
>arrays.......

The burroughs systems were stack based architectures. That said, the
array "descriptor" was on the stack. The array itself would be allocated
elsewhere in memory.

On 4/11/2023 8:32 AM, Scott Lurndal wrote:
> BGB <cr88192@gmail.com> writes:
>> On 4/10/2023 4:13 PM, Scott Lurndal wrote:
>>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>> Michael S <already5chosen@yahoo.com> writes:
>
>> This being partly because:
>> Variable size stack frames are evil;
>> If one excludes large allocations from the stack (including large
>> arrays), one may find that 128k of stack space is sufficient for most
>> programs.
>
> It's not 1989 anymore.
>

?...

AFAIK, back then stack sizes then were usually smaller, say, 8K or 16K?...

Where, say:
CS, program;
DS = SS, global variables and stack.
Also heap for "small" or "tiny" model.
ES, stomped for sake of "far" pointers.

Comparably, 128K is massive...

I have tested, and a lot of stuff does not work with a 16K or 32K stack,
whereas 64K is borderline (say, Doom works, but Quake overflows the stack).

Meanwhile, 128K usually works, and 256K if one "wants to be safe".

GLQuake didn't work at first, but this was because they had put a large
array (256x256 32-bit) the stack for upscaling non-power-of-2 textures,
which immediately blew out the stack.
Had turned this into something like:
static int *texbuf=NULL;
if(!texbuf)
texbuf=malloc(256*256*4);

In this case, doing an "alloca()" for every call to
"GL_UploadTexture32()" (or something to this effect) would be not ideal.
This array is also "a little on the large side" for ".bss" as well.

Windows typically uses 1MB for the stack, but this seems overkill IME,
and also Windows always operates with virtual memory enabled.

But, then again, as I have noted, in some of the programs I have tested
there are also non-zero amounts of code using K&R style declaration
syntax, eg:
foo(p, x, y) int x, y; char *p; {
...
}

Thus far, my TestKern thing is often operating with the MMU disabled
(unless a pagefile is detected). In this case, only physical RAM is
available, which leaves things a little more cramped.

Say, 128MB of physical-mapped RAM doesn't go very far if one is
allocating 1MB for stack buffers and similar.

Note that my PBO ABI was designed for use in NOMMU configurations (along
with putting multiple logical processes in a shared virtual address
space), so works OK here.

I wasn't using the pagefile in the "actual hardware" tests thus far, as
it seems there is an issue with my SDcard code and writing data to the
SDcard did not seem to be working correctly.

Had tried correcting it (apparently my code for writing to the SDcard
wasn't quite right), but seems to not have worked...

Though, I am getting conflicting information on whether the CMD24 "Write
Block" operation needs a CRC. A PDF from SanDisk mentions a CRC
following the data for a CMD24, it is possible I may need to figure out
this CRC for data to "actually get written" to the SDcard? ...

A lot of code online seemingly just sends two 0xFF bytes (following the
main 512 bytes of block data), but, as noted, this did not seem to be
working in my case (with the SDcard effectively behaving almost like it
was Read-Only).

Everything works in simulation, but as noted, my SDcard mockup doesn't
check for or generate CRCs.

....

On Tuesday, April 11, 2023 at 12:36:39 PM UTC-5, Scott Lurndal wrote:
> BGB <cr8...@gmail.com> writes:
> >On 4/11/2023 8:32 AM, Scott Lurndal wrote:
> >> BGB <cr8...@gmail.com> writes:
> >>> On 4/10/2023 4:13 PM, Scott Lurndal wrote:
> >>>> an...@mips.complang.tuwien.ac.at (Anton Ertl) writes:
> >>>>> Michael S <already...@yahoo.com> writes:
> >>
> >>> This being partly because:
> >>> Variable size stack frames are evil;
> >>> If one excludes large allocations from the stack (including large
> >>> arrays), one may find that 128k of stack space is sufficient for most
> >>> programs.
> >>
> >> It's not 1989 anymore.
> >>
> >
> >?...
> >
> >AFAIK, back then stack sizes then were usually smaller, say, 8K or 16K?....
> The VAX had a 32-bit flat address space in 1978[*]. The i386 had one by 1985.
>
> Not to mention the dozens of mainframe and minicomputers available with
> large memory spaces from the last 1960s.
<
CDC 6600, 7600, CRAY 1, 1S, XMP, YMP,
IBM 360s, 370s, 30x0s,
Univac 1108,
S.E.L., InterData 32,
PDP 6, PDP 10, PDP 11, VAX, Alpha,
All RISC architectures,
:: all had flat address spaces.
>
> [*] Although in 1979, 4MB was a large amount of physical memory, VMS
> had working sets and paging, as did the 370 successors and most other
> mainframe lines. Even the PDP-10 had 256k words (36-bit) of magnetic
> core memory.
<
When I bought my first computer in 1995, It came with 4 MB, and after using
it for 2 days, I went out and bought 16MB more. Since windows 3.1 did not
use more than 16 MB, I configured the other 4MB as the disk cache, and
turned off the swapper. My 33 MHz 486 outran my wife's 100 MHz Pentium.
<
The real question is why anyone would want something other than a large flat
address space ??