On 11/10/2021 20:04, Victor Yodaiken wrote:
> On Sunday, October 10, 2021 at 5:48:01 AM UTC-5, David Brown wrote:
>
>> The idea of C being a "high-level assembler" is right there in
>> section 1, "Introduction".
>>
>
> There is something about high level assembler in the introduction -
> it is a QUOTE from the C Rationale. If you honestly interpreted it as
> a claim that C was supposed to be a complete replacement for all
> assembly languages, then maybe you need new glasses.
>

I know that what you wrote was a quotation from a C Rationale document.
Clearly I did not interpret it as you suggest - you cannot complain
that I have exaggerated your writings and then do /exactly/ the same
thing yourself! The difference, however, is that I was doing so as a
criticism of your argument - you appear to be doing so as some kind of
revenge.

But do I take it that you agree with me that C is not a "high-level
assembly"?

A vital characteristic of assembly, as a programming language, is that
the generated code does /exactly/ what you order, no more and no less,
without re-ordering, optimising, removing or adding anything. C is not
a language remotely like that, never has been, never will be.

Another vital characteristic of assembly is that it is defined entirely
in terms of the underlying machine - giving you access to maximum
efficiency at the cost of minimal portability. C is intended to give a
balance here - it has many aspects that are implementation-defined in
some of their details, so that you can have maximal (or close to
maximal) efficiency. But the majority of the language is more abstract
and defined entirely in terms of an abstract machine. Thus it is a
high-level language, but a lower-level high-level language than most
other high-level languages.

And an important use of assembly is to get detailed control of a system
outside of the usual "calculation" type coding. This is needed for
low-level work - device drivers, OS's, etc. Key requirements are things
like precise control, including ordering, of memory accesses that are
outside of the knowledge of the language and the compiler (such as
memory-mapped device registers, or accessing data for other programs. C
gives you that in an /almost/ portable manner via "volatile". (C17
improved on the semantics for volatile accesses.) Assembly is also used
to access hardware or processor features that simply have no
correspondence in "normal" programming languages - special registers,
special instructions, etc. Standard C does not help here - but it was
always intended that C compilers can have additional features such as
intrinsics, extensions, inline assembly, etc., that solve this.

C is /not/ an assembly language of any sort, but it is intended to
replace the need to write so much code in assembly.

>> Yes, I exaggerated somewhat, and clearly I was parodying and note
>
> Exactly. You didn't respond to the argument made, you just
> caricatured it, refuted the caricature and then congratulated
> yourself on being so smart.
>

You didn't make much of an argument, to be honest. And it has been
refuted many times before.

>> quoting. However, that /is/ the impression I got from the paper.
>> I'm sorry, but you appear to have some serious misunderstandings
>> about the C language, its tools, how it was originally intended to
>> be used, how it developed as a language, how its tools developed,
>> and how its use developed. The very title, "How ISO C became
>> unusable for operating systems development" shows that - C was
>> /never/ intended or designed to let you write operating systems
>> purely in standard portable C.
>
> Brilliant! And where did I write that C was designed to let one write
> operating systems purely in standard portable C ? Nowhere. That's
> something you made up by adding "purely in standard portable C" to
> what I wrote, changing the meaning completely. Of course some
> assembler has always been necessary. Nobody said otherwise - except
> for you.

Um, ISO C /is/ portable C. That's the point - the ISO Standards
describe the fully portable standardised parts of the language and
standard library, along with the minimal requirements for the
implementation-dependent parts. Any features a compiler provides beyond
that are non-portable and non-standard.

"Portable C" does /not/ mean the code will have an identical effect on
any system, as the final effect always depends on system features
outside the control and scope of the C standards. It means the /C/ code
has the same semantics. So you can write "printf" in portable C code,
knowing that the output will go somewhere appropriate.

In this post, you seem to be backing away from your paper entirely. Was
it just click-bait? It is claiming that C was invented for writing
operating systems, and that now ISO C is unusable for operating system
development. You justify your argument by showing mistakes people have
made in their C coding. Are you now saying that ISO C was /never/
intended for writing OS's ? Are you saying you are unclear on the
difference between the original historic C language and tools used for
early UNIX, and ISO C?

>
>> The intention - which was achieved, and is still the case today -
>> was that you could write /most/ of the OS in at least reasonably
>> portable C, relying somewhat on implementation-dependent behaviour
>> in parts.
>
> And the point of the article, was exactly that you cannot do that any
> more because implementation dependent C has been largely displaced by
> "undefined behavior" C.

Eh, no. If that was the point of the article, it's wrong. Plain and
simple. This is easily demonstrated by the fact that pretty much all
modern operating systems /are/ written almost entirely in C. And pretty
much all of the code they are written in is /portable/ C, relying on
nothing more than standards-defined behaviour and a few extra
assumptions about the target (such as POSIX requirements of an "int" of
at least 32-bit, consistent endianness, 8-bit char, existence of
uintptr_t, etc.). The parts that need more than this are, of course,
essential - and they always have been, from day one of C, long before
standardisation.

OS's generally have certain basic requirements that limit portability to
a particular class of target. For the more specific details,
well-written OS's will keep the target-specific parts to a minimum and
will keep them modularised rather than scattered throughout the source.
Some will require particular compilers with particular extensions in
order to make the code simpler, clearer, or more effiicent, others will
try minimise the dependency on compilers. Both options are perfectly
valid in C, have always been valid, and have always been intended use of
C. This is nothing new.

> This is why Linux relies on a growing number
> of gcc specific flags that turn off different compiler behaviors
> permitted by, encouraged by, the ISO standard.

The ISO standard does not "encourage" any particular compiler behaviour,
beyond a few footnotes. It describes the semantics of the language.
Linux does not have a growing number of gcc-specific flags - it has a
only a few flags that are required for correctness (and many for
optimisation, efficiency, and warnings). Some - such as
"-fno-delete-null-pointer-checks", are clearly to work around bugs in
the code that have not yet been fixed. Others, such as
"-fno-strict-aliasing", are for the convenience of the programmers in
writing efficient and neat code. There are /no/ situations where code
has to be written in a way that relies on that flag. But there /are/
situations where relying on that flag is a clearer and simpler way to
get efficient results than portable alternatives.

Don't make the mistake - as you did in the paper - of thinking that
because one particular OS requires such flags, that /all/ OS's written
in C require them. Linux is a wonderful project and achievement, but it
is certainly not a perfect model of what can be done in C, nor is it a
good example of how to write an OS. It is an organic development over
several decades and thousands of developers, and it has its own way of
doing things - coding in portable or standards-defined C was never an aim.

> Can you compare
> pointers? In K&R C you can, if the hardware allows.

You can compare them for equality (if they are of compatible types, or
one is a null pointer), not for relation, unless they are part of the
same object. This makes sense, because C does not require a single
linear address space - and there are real targets (with real OS's,
written in C) that have independent address spaces.

(C90, the first ISO standardisation of C, was basically a slightly more
formal version of the appendix from K&R version 2.)

> In ISO-C you
> cannot unless they are both within the same object or (possible, one
> past).

> Eh, no. If that was the point of the article, it's wrong. Plain and
> simple. This is easily demonstrated by the fact that pretty much all
> modern operating systems /are/ written almost entirely in C.

Well, maybe a language closely related to C, but not really C.
That's why they need special flags passed to the compiler before the
result is usable.

> You can compare them for equality (if they are of compatible types, or
> one is a null pointer), not for relation, unless they are part of the
> same object. This makes sense, because C does not require a single
> linear address space - and there are real targets (with real OS's,
> written in C) that have independent address spaces.

And the problem is that as soon as you use `<` on pointers to different
objects you run the risk of firing rockets, even though `<` is supposed
to be a pure function returning a boolean.

I'm fine with a C standard that says that which boolean is returned is
undefined (and I'm even willing to let it be not-quite-pure and not
always return the same value when called with the same two pointers),
but firing rockets is a bit excessive here, don't you think?
There should also be a "quality of implementation" note stating that on
systems where it makes sense, `<` should be pure and obey the expect
rules like anti-symmetry etc...

Stefan

On Tuesday, October 12, 2021 at 8:39:58 AM UTC-5, David Brown wrote:
> On 11/10/2021 20:04, Victor Yodaiken wrote:
> > On Sunday, October 10, 2021 at 5:48:01 AM UTC-5, David Brown wrote:
> >
> >> The idea of C being a "high-level assembler" is right there in
> >> section 1, "Introduction".
> >>
> >
> > There is something about high level assembler in the introduction -
> > it is a QUOTE from the C Rationale. If you honestly interpreted it as
> > a claim that C was supposed to be a complete replacement for all
> > assembly languages, then maybe you need new glasses.
> >
> I know that what you wrote was a quotation from a C Rationale document.
> Clearly I did not interpret it as you suggest - you cannot complain
> that I have exaggerated your writings and then do /exactly/ the same
> thing yourself! The difference, however, is that I was doing so as a
> criticism of your argument - you appear to be doing so as some kind of
> revenge.
>
>
> But do I take it that you agree with me that C is not a "high-level
> assembly"?
>
> A vital characteristic of assembly, as a programming language, is that
> the generated code does /exactly/ what you order, no more and no less,
> without re-ordering, optimising, removing or adding anything. C is not
> a language remotely like that, never has been, never will be.
<
You have obviously never used a good macro assembler............
>
> Another vital characteristic of assembly is that it is defined entirely
> in terms of the underlying machine - giving you access to maximum
> efficiency at the cost of minimal portability. C is intended to give a
> balance here - it has many aspects that are implementation-defined in
> some of their details, so that you can have maximal (or close to
> maximal) efficiency. But the majority of the language is more abstract
> and defined entirely in terms of an abstract machine. Thus it is a
> high-level language, but a lower-level high-level language than most
> other high-level languages.
>
> And an important use of assembly is to get detailed control of a system
> outside of the usual "calculation" type coding. This is needed for
> low-level work - device drivers, OS's, etc. Key requirements are things
> like precise control, including ordering, of memory accesses that are
> outside of the knowledge of the language and the compiler (such as
> memory-mapped device registers, or accessing data for other programs. C
> gives you that in an /almost/ portable manner via "volatile". (C17
> improved on the semantics for volatile accesses.) Assembly is also used
> to access hardware or processor features that simply have no
> correspondence in "normal" programming languages - special registers,
> special instructions, etc. Standard C does not help here - but it was
> always intended that C compilers can have additional features such as
> intrinsics, extensions, inline assembly, etc., that solve this.
>
> C is /not/ an assembly language of any sort, but it is intended to
> replace the need to write so much code in assembly.
> >> Yes, I exaggerated somewhat, and clearly I was parodying and note
> >
> > Exactly. You didn't respond to the argument made, you just
> > caricatured it, refuted the caricature and then congratulated
> > yourself on being so smart.
> >
> You didn't make much of an argument, to be honest. And it has been
> refuted many times before.
> >> quoting. However, that /is/ the impression I got from the paper.
> >> I'm sorry, but you appear to have some serious misunderstandings
> >> about the C language, its tools, how it was originally intended to
> >> be used, how it developed as a language, how its tools developed,
> >> and how its use developed. The very title, "How ISO C became
> >> unusable for operating systems development" shows that - C was
> >> /never/ intended or designed to let you write operating systems
> >> purely in standard portable C.
> >
> > Brilliant! And where did I write that C was designed to let one write
> > operating systems purely in standard portable C ? Nowhere. That's
> > something you made up by adding "purely in standard portable C" to
> > what I wrote, changing the meaning completely. Of course some
> > assembler has always been necessary. Nobody said otherwise - except
> > for you.
> Um, ISO C /is/ portable C. That's the point - the ISO Standards
> describe the fully portable standardised parts of the language and
> standard library, along with the minimal requirements for the
> implementation-dependent parts. Any features a compiler provides beyond
> that are non-portable and non-standard.
>
> "Portable C" does /not/ mean the code will have an identical effect on
> any system, as the final effect always depends on system features
> outside the control and scope of the C standards. It means the /C/ code
> has the same semantics. So you can write "printf" in portable C code,
> knowing that the output will go somewhere appropriate.
>
>
> In this post, you seem to be backing away from your paper entirely. Was
> it just click-bait? It is claiming that C was invented for writing
> operating systems, and that now ISO C is unusable for operating system
> development. You justify your argument by showing mistakes people have
> made in their C coding. Are you now saying that ISO C was /never/
> intended for writing OS's ? Are you saying you are unclear on the
> difference between the original historic C language and tools used for
> early UNIX, and ISO C?
> >
> >> The intention - which was achieved, and is still the case today -
> >> was that you could write /most/ of the OS in at least reasonably
> >> portable C, relying somewhat on implementation-dependent behaviour
> >> in parts.
> >
> > And the point of the article, was exactly that you cannot do that any
> > more because implementation dependent C has been largely displaced by
> > "undefined behavior" C.
> Eh, no. If that was the point of the article, it's wrong. Plain and
> simple. This is easily demonstrated by the fact that pretty much all
> modern operating systems /are/ written almost entirely in C. And pretty
> much all of the code they are written in is /portable/ C, relying on
> nothing more than standards-defined behaviour and a few extra
> assumptions about the target (such as POSIX requirements of an "int" of
> at least 32-bit, consistent endianness, 8-bit char, existence of
> uintptr_t, etc.). The parts that need more than this are, of course,
> essential - and they always have been, from day one of C, long before
> standardisation.
>
> OS's generally have certain basic requirements that limit portability to
> a particular class of target. For the more specific details,
> well-written OS's will keep the target-specific parts to a minimum and
> will keep them modularised rather than scattered throughout the source.
> Some will require particular compilers with particular extensions in
> order to make the code simpler, clearer, or more effiicent, others will
> try minimise the dependency on compilers. Both options are perfectly
> valid in C, have always been valid, and have always been intended use of
> C. This is nothing new.
> > This is why Linux relies on a growing number
> > of gcc specific flags that turn off different compiler behaviors
> > permitted by, encouraged by, the ISO standard.
> The ISO standard does not "encourage" any particular compiler behaviour,
> beyond a few footnotes. It describes the semantics of the language.
> Linux does not have a growing number of gcc-specific flags - it has a
> only a few flags that are required for correctness (and many for
> optimisation, efficiency, and warnings). Some - such as
> "-fno-delete-null-pointer-checks", are clearly to work around bugs in
> the code that have not yet been fixed. Others, such as
> "-fno-strict-aliasing", are for the convenience of the programmers in
> writing efficient and neat code. There are /no/ situations where code
> has to be written in a way that relies on that flag. But there /are/
> situations where relying on that flag is a clearer and simpler way to
> get efficient results than portable alternatives.
>
> Don't make the mistake - as you did in the paper - of thinking that
> because one particular OS requires such flags, that /all/ OS's written
> in C require them. Linux is a wonderful project and achievement, but it
> is certainly not a perfect model of what can be done in C, nor is it a
> good example of how to write an OS. It is an organic development over
> several decades and thousands of developers, and it has its own way of
> doing things - coding in portable or standards-defined C was never an aim.
> > Can you compare
> > pointers? In K&R C you can, if the hardware allows.
> You can compare them for equality (if they are of compatible types, or
> one is a null pointer), not for relation, unless they are part of the
> same object. This makes sense, because C does not require a single
> linear address space - and there are real targets (with real OS's,
> written in C) that have independent address spaces.
>
> (C90, the first ISO standardisation of C, was basically a slightly more
> formal version of the appendix from K&R version 2.)
> > In ISO-C you
> > cannot unless they are both within the same object or (possible, one
> > past).
> It is the same in pre-ISO C, once it moved beyond the initial PDP
> implementation and began to be a portable language.
> > So the malloc/free implementation in K&R2, if you read the
> > paper, you would know, might or might not work depending on
> > optimization level and compiler version even when the hardware (like
> > 99% of modern processors) allows you to make the comparison.
> There was never a guarantee, or even an intention, that the standard
> library functions could be written in portable C. There was the
> intention (and it has /always/ been the practice) that almost all of it
> could be written in C, but that it might require implementation-specific
> features or extensions, or may even require assembly code.
>
> The implementations for things like malloc/free in K&R were never
> intended to be /the/ implementations for the functions - they were never
> claimed to be portable, efficient, or to have all the features one might
> want. They were /examples/.
> > Can you
> > cast a pointer to a different type? Sure.
> /Not/ "sure". You can often do so on many systems, but not on all.
> There are C systems where character pointers are a different size from
> int pointers, or where casting that results in a mis-aligned pointer
> gives a trap. The standards give an implementation leeway to make some
> aspects implementation-defined or undefined behaviour, precisely because
> this is required to make an efficient C implementation on such target
> systems. (And the section on such conversions is one of the few places
> where the standard encourages particular implementation behaviour, with
> a note saying that conversions between pointers and integers should
> attempt to follow the addressing structure of the target.)
> > Can you dereference it?
> > Maybe, maybe not, depending on the optimizer level and how the
> > compiler is interpreting type rules today.
> Correct - and that has always been the case. There has never been a
> portable C that said you can mess around with pointer types and get
> particular results. (Note that this is generally not dependent on
> optimisation levels - it's simply that the effects of code without
> defined behaviour can be different depending on luck and details of code
> generation.)
> >
> >> And if you look at OS's today, that's what you see. Whether you
> >> look at massive OS's like BSD or Linux, or small ones like
> >> FreeRTOS, you'll see the great majority is written in plain
> >> standards-defined C. Some parts
> >
> > And if you had read the article you would know that what you just
> > wrote is wrong.
> Sorry, no. They may be /compiled/ with particular flags to give extra
> semantics beyond those of the C standards (such as handling of
> dereferencing converted pointers). But that does not mean the majority
> of the code depends on such semantics. Most does not.
> >
> >> are unavoidably written in assembly (either inline or stand-alone),
> >> some parts are dependent on implementation details for the targets
> >> supported, and sometimes the writers take advantage of compiler
> >> extensions or features in order to write code that is more
> >> efficient, clearer, or otherwise better than standard C alone.
> >>
> >> Some details have changed since C's inception, but the principles
> >> have not.
> >
> >
> > Charitably, you are just 25 years behind on what gcc/clang do. Catch
> > up.
> >
> I write low-level code all the time. I write it correctly, and I use
> gcc to do so (without "-fno-strict-aliasing", and certainly without
> cop-outs like "-fno-delete-null-pointer-checks"). When I look back at
> the low-level C code I wrote 25 years ago, I don't see any such issues
> with that code either. (Oh, and I was using compilers that optimised
> heavily - including using type-based alias analysis, signed integer
> overflow analysis, and other "undefined behaviour optimisations", 25
> years ago.)
>
> I suppose it is conceivable that I am a genius and have somehow managed
> to work with C correctly while most others failed. I think, however,
> that is unlikely. But it is clear that some people view the past with
> rose-tinted spectacles, and prefer to blame others for changing the
> language rather than accept that they made mistakes or misunderstandings
> themselves.

George Neuner <gneuner2@comcast.net> writes:

[...]

> GCs for uncooperative languages (like C) do check potential
> pointers - i.e. properly aligned values - against the heap limits
> as a first step. But what the GC really needs to know is if a
> pointer refers to an actual allocated block. Determining /that/
> is a lot harder because the value must be range checked against
> every allocation, not just against the heap limits.

It does take more development effort. It need not incur a
significantly larger run-time cost.

> There are various data structures that facilitate interval / range
> searching, but maintaining them is extra overhead in the allocator
> and collector, and potentially is heap space unavailable to the
> program.

One: the space overhead is close to negligible.

Two: the time overhead (on an allocation) is mostly subsumable
in what needs to be done to do the allocation in the first place.

Three: very few programs need to use 100% of available memory. More
to the point, these days very few programs need to use 25% of
available memory. Certainly some programs suffer from memory bloat
(ahem, web browsers), but that is not due to inefficiencies in the
memory allocator, even those that makes use of data structures like
the ones mentioned above.

> It's a lot more effective to require that the compiler retain
> original pointers and always work on copies of them.

Probably so in some cases, probably not in others. But that option
isn't always available.

Thomas Koenig <tkoenig@netcologne.de> writes:

> Victor Yodaiken <victor.yodaiken@gmail.com> schrieb:
>
>> Can you cast a pointer to a different type? Sure.
>>
>> Can you dereference it?
>
> No.

This statement is wrong, or at best much overstated. There are
plenty of situations where a C pointer value converted to a
different type may be safely and portably dereferenced.
Certainly that isn't always true, but it isn't always false
either.

Stefan Monnier <monnier@iro.umontreal.ca> writes:

[..are operating systems written in C..]

> Well, maybe a language closely related to C, but not really C.

Yes, really C. The ISO C standard unequivocally gives conforming
implementations the freedom to assign whatever meaning they
choose to any situation that falls into the category of undefined
behavior.

> That's why they need special flags passed to the compiler before
> the result is usable.

AFAIAA none of flags used in compiling programs like the Linux
kernel render the implementation non-conforming, and if the
implementation is conforming then by definition the language
being compiled is C.

(Btw, if you choose to respond to this posting and quote some of
what I have said but do not give an attributing citation, then
you're an asshole.)

Stefan Monnier <monnier@iro.umontreal.ca> writes:
> (Btw, if you choose to respond to this posting and quote some of
> what I have said but do not give an attributing citation, then
> you're an asshole.)

Not sure what I was thinking when I wrote that, sorry,

Stefan

On 10/12/2021 10:31 AM, Tim Rentsch wrote:
> George Neuner <gneuner2@comcast.net> writes:
>
> [...]
>
>> GCs for uncooperative languages (like C) do check potential
>> pointers - i.e. properly aligned values - against the heap limits
>> as a first step. But what the GC really needs to know is if a
>> pointer refers to an actual allocated block. Determining /that/
>> is a lot harder because the value must be range checked against
>> every allocation, not just against the heap limits.
>
> It does take more development effort. It need not incur a
> significantly larger run-time cost.
>

Average-case cost for many GC designs is fairly low.

One downside is the creation of delays or stalls.

For example, if one has a mark/sweep collector, there is a visible (and
potentially fairly obvious) stall whenever the GC runs. In a
soft-real-time application (like a 3D game), it can get really annoying
to have a game which periodically stalls.

In harder real-time uses, these sorts of delays are unacceptable.

There are ways to avoid delays (relative to mark/sweep), but many have
their own drawbacks (and higher complexity), so naive mark/sweep remains
popular. One of the major (not "stop the world" varieties of GC) is
basically to run a mark-sweep collector in parallel with the main
application (in its own thread), any then use trickery to try to keep
the application and GC from stepping on each other.

In these cases, one might also use the GC state as a counter, meaning it
may require 2 or 3 GC passes to reclaim a given garbage object. This
allows a reference to be "missed" for several passes before it is
finally removed. But, is generally enough to deal with the "inexactness"
of running the program thread and GC thread in parallel.

Say, 2 bit GC state:
00, White (sweep removes it now);
01, Light Gray (likely to be reclaimed next cycle);
10, Dark Gray (missed last GC cycle);
11, Black (hit by most recent GC cycle).

There is reference counting, which can avoid any big delays, at the cost
of significantly worse performance and needing a lot more assistance
from the compiler (every time a reference is
copied/assigned/removed/...) it needs to be counted. It also can't
reclaim cyclic structures.

Some languages / VMs, such as Python, combine reference counting with
mark/sweep.

>> There are various data structures that facilitate interval / range
>> searching, but maintaining them is extra overhead in the allocator
>> and collector, and potentially is heap space unavailable to the
>> program.
>
> One: the space overhead is close to negligible.
>
> Two: the time overhead (on an allocation) is mostly subsumable
> in what needs to be done to do the allocation in the first place.
>
> Three: very few programs need to use 100% of available memory. More
> to the point, these days very few programs need to use 25% of
> available memory. Certainly some programs suffer from memory bloat
> (ahem, web browsers), but that is not due to inefficiencies in the
> memory allocator, even those that makes use of data structures like
> the ones mentioned above.
>

Yep.

As noted, in my case, I mentioned a preference for binary-searched arrays.

This does mean that the arrays need to be sized for basically the
maximum number of objects of a given size class. But, this is only a
tiny fraction of the memory that would actually be required to allocate
these objects.

Say, one has an array of 16B "headers" to manage a 4K minimum object
size, this is an overhead of ~ 0.4% ...

A cell allocator might eat a 3% or 6% space overhead for the memory it
manages, but for small objects, this is cheaper than alternatives.

>> It's a lot more effective to require that the compiler retain
>> original pointers and always work on copies of them.
>
> Probably so in some cases, probably not in others. But that option
> isn't always available.
>

If one does lookup by binary search, it isn't too much of an issue to
allow for interior pointers. This all depends though.

One design tradeoff in an MM is whether 'free()' can be done directly
(as an O(1) operation), or needs a lookup.

The lookup strategy, while slower, does offer some added protection in
that 'free()' can do something sensible if the program tries to give it
a bad pointer (such as raise a fault), rather than behave in a way that
is basically unpredictable (blindly stomping memory with no idea what it
points at).

An intermediate option is to check magic values and raise a fault, which
does also allow for detecting bad pointers and buffer overruns.

....

Stefan Monnier <monnier@iro.umontreal.ca> writes:
[Someone wrote the following; Did you delete the name to protect the culprit?]
>> Eh, no. If that was the point of the article, it's wrong. Plain and
>> simple. This is easily demonstrated by the fact that pretty much all
>> modern operating systems /are/ written almost entirely in C.
>
>Well, maybe a language closely related to C, but not really C.

That's a question of what you mean by C. In my book the language that
the operating systems and pretty much all other software is written in
is C.

As far as the C standard is concerned, they are not strictly
conformant programs (but there are practically no production programs
that are strictly conformat). They are conformant programs, however.

Of course, fans of undefined behaviour claim that despite that, these
programs are not written in C. Why should you or I care about their
opinion?

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

> Of course, fans of undefined behaviour claim that despite that, these
> programs are not written in C. Why should you or I care about their
> opinion?

I don't think they're wrong. I think the two views are basically the
result of the current failure to find acceptable descriptions beyond
"undefined behavior".

I can very much see the benefit of giving some freedom to the compiler
and I can very much see the drawbacks of being at the mercy of the
compiler's freedom.

I think the solution involves going carefully over every case of UB
and refining it (on a case-by-case basis) so it still provides the
freedom the compilers need while giving sufficient guarantees for the
programmers when they know the properties of their underlying system
(e.g. so they can write code which depends on a flat address space
without fearing the compiler).

This is hard work and I'm not volunteering.

Stefan

On Tue, 12 Oct 2021 01:24:33 GMT, Branimir Maksimovic
<branimir.maksimovic@icloud.com> wrote:

>On 2021-10-11, George Neuner <gneuner2@comcast.net> wrote:
>> On Sat, 09 Oct 2021 22:22:45 GMT, Branimir Maksimovic
>><branimir.maksimovic@icloud.com> wrote:
>>
>>>Boehm's collector scans heap/stack/bss, pointing anywhere within
>>>allocated area, no problem with that.
>>
>> The problem isn't finding pointers that refer to the heap, the problem
>> is finding pointers that refer to particular allocated blocks IN the
>> heap.
>\Well, I thouht this was easy task as it is simple pointer
>arithemtic, but now I think you are right, heap can be
>fragmented and that's a problem.

Right. With an uncooperative language you can't relocate blocks to
compact the heap because you can't be certain which program values
actually are pointers to them.

>> Even with programmer assistance, BDW slows down VERY quickly as the
>> live data set grows.
>
>Well, surelly less than when you move allocated area then have to
>update ll references in profram or to apply membar on every memory
>access through pointers :p

Actually, many semispace (moving) collectors are /much/ faster than
BDW which is a mark/sweep system.

Compilers for GC'd languages make sure that pointers can be located
definitively, and that head pointers to heap blocks are never lost or
obscured. In a well designed system, that means never having to look
at any non-pointer values.

In a semispace collector, scanning can be done /while/ moving the
object, and a collection only ever looks at live objects once. In
contrast a mark/compact system has to look at the whole set of live
objects at least twice: once to mark them, and once to move them.
A mark/sweep system has to look at live objects to mark them, but at
ALL objects - both live and dead - to sweep up the garbage.

>GC is costly no question in that :P

But you can pay the cost in different ways.

Baker's Treadmill is the fastest GC implementation known. It is a
/logical/ semispace collector that doesn't physically move blocks (so
collection can be done in parallel with the mutator and no need to
update pointers), and ... when implemented well ... all of it's
operations are simple and completely deterministic.

Unfortunately, Treadmill achieves its blinding speed at the expense of
space ... it has the highest memory overhead of all implementations.

YMMV,
George

BGB <cr88192@gmail.com> writes:
>There was a proposal to add lambdas (with C++ style syntax) to C23,
>though with slightly restricted semantics.
>
>Though, granted, lambdas are sort of their own thing, as are GCC style
>nested functions, ...

That's interesting. Just two weeks ago I presented a paper on a
simpler alternative to lexical scoping: Two-stage parameter passing.
This could be used for a lambda-like feature. I used C as example
(and thought up some syntax that I expect can be added in a
backwards-compatible way; it's not my ideal syntax).

Slides: https://www.complang.tuwien.ac.at/anton/tmp/slides-kps21.pdf
Paper: https://www.complang.tuwien.ac.at/anton/tmp/ertl21kps.pdf

The talk was better than the paper, but of course on the slides you
miss the audio track (and you may not understand the German), and may
have to fill in from the paper. It appeared in:

https://www.uni-kiel.de/journals/receive/jportal_jparticle_00000382

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 10/12/2021 10:25 AM, Stefan Monnier wrote:
>> Of course, fans of undefined behaviour claim that despite that, these
>> programs are not written in C. Why should you or I care about their
>> opinion?
>
> I don't think they're wrong. I think the two views are basically the
> result of the current failure to find acceptable descriptions beyond
> "undefined behavior".
>
> I can very much see the benefit of giving some freedom to the compiler
> and I can very much see the drawbacks of being at the mercy of the
> compiler's freedom.
>
> I think the solution involves going carefully over every case of UB
> and refining it (on a case-by-case basis) so it still provides the
> freedom the compilers need while giving sufficient guarantees for the
> programmers when they know the properties of their underlying system
> (e.g. so they can write code which depends on a flat address space
> without fearing the compiler).

Are you suggesting that be done in the standard, or in each
implementation of the standard (i.e. each compiler's documentation)?

>
> This is hard work and I'm not volunteering.

Can be. I think the work varies across the various cases. Even fixing
the simple to document ones is progress.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

On 10/12/2021 10:42 AM, Tim Rentsch wrote:
> Thomas Koenig <tkoenig@netcologne.de> writes:
>
>> Victor Yodaiken <victor.yodaiken@gmail.com> schrieb:
>>
>>> Can you cast a pointer to a different type? Sure.
>>>
>>> Can you dereference it?
>>
>> No.
>
> This statement is wrong, or at best much overstated. There are
> plenty of situations where a C pointer value converted to a
> different type may be safely and portably dereferenced.
> Certainly that isn't always true, but it isn't always false
> either.
>

Yeah. For some types of code it is desirable to be able to cast and
dereference a pointer as whatever type matches what one wants to do with it.

Requiring something like, say, using "memcpy()" to mash bits from one
format to another and hoping the compiler is smart and reliable enough
to make this "not suck" is a crap solution, IMHO.

Other options are:
Just deref it, and hope TBAA doesn't ruin ones day;
Stick 'volatile' on all the pointers in an attempt to defeat TBAA;
Maybe works, but still technically UB;
...

By the time the solution to make something like this work semi-reliable
and with good performance starts looking like:
#ifdef _MSC_VER
... //use pointer deref (memcpy sucks, pointer deref works)
#endif
#ifdef __GNUC__
... //use memcpy (memcpy is fast, pointer deref may break)
#endif
...

Stuff is kinda broken...

For performance-sensitive code, one may often end up with a bunch of
compiler and target sensitive boilerplate followed by "generic" fallback
cases.

This is tolerable, but still kinda sucks.

I guess, if I were to define a "common behavior subset", it is possible
that TBAA rules might be allowed in a restricted form, but maybe define
it that 'volatile' can effectively circumvent it (by also being able to
alias with anything).

This would then allow using it on only the operation which violates
TBAA, rather than all of the pointers in the general vicinity of the
logic which violates TBAA (which may in turn cross multiple function
call levels if the compiler does automatic inlining, ...).

One can use 'char *' or 'unsigned char *' operations, but these are also
playing roulette with "may work well" (compiler infers away the byte
operations) or "performance may be garbage" (compiler actually emits
byte operations).

....

On 10/12/2021 1:18 PM, BGB wrote:
> On 10/12/2021 10:42 AM, Tim Rentsch wrote:
>> Thomas Koenig <tkoenig@netcologne.de> writes:
>>
>>> Victor Yodaiken <victor.yodaiken@gmail.com> schrieb:
>>>
>>>> Can you cast a pointer to a different type?  Sure.
>>>>
>>>> Can you dereference it?
>>>
>>> No.
>>
>> This statement is wrong, or at best much overstated.  There are
>> plenty of situations where a C pointer value converted to a
>> different type may be safely and portably dereferenced.
>> Certainly that isn't always true, but it isn't always false
>> either.
>>
>
> Yeah. For some types of code it is desirable to be able to cast and
> dereference a pointer as whatever type matches what one wants to do with
> it.
>
> Requiring something like, say, using "memcpy()" to mash bits from one
> format to another and hoping the compiler is smart and reliable enough
> to make this "not suck" is a crap solution, IMHO.
>
> Other options are:
>   Just deref it, and hope TBAA doesn't ruin ones day;
>   Stick 'volatile' on all the pointers in an attempt to defeat TBAA;
>     Maybe works, but still technically UB;
>   ...
>
>
> By the time the solution to make something like this work semi-reliable
> and with good performance starts looking like:
> #ifdef _MSC_VER
>   ...  //use pointer deref (memcpy sucks, pointer deref works)
> #endif
> #ifdef __GNUC__
>   ...  //use memcpy (memcpy is fast, pointer deref may break)
> #endif
>   ...
>
> Stuff is kinda broken...
>
>
> For performance-sensitive code, one may often end up with a bunch of
> compiler and target sensitive boilerplate followed by "generic" fallback
> cases.
>
> This is tolerable, but still kinda sucks.
>
>
> I guess, if I were to define a "common behavior subset", it is possible
> that TBAA rules might be allowed in a restricted form, but maybe define
> it that 'volatile' can effectively circumvent it (by also being able to
> alias with anything).
>
> This would then allow using it on only the operation which violates
> TBAA, rather than all of the pointers in the general vicinity of the
> logic which violates TBAA (which may in turn cross multiple function
> call levels if the compiler does automatic inlining, ...).
>

I will also add that in such a subset, any construct of the form
'*(foo_t *)bar' would also be assumed to potentially alias with anything
else in its immediate vicinity.

So:
i=*(int *)p; //assumed to alias with anything.
*(float *)q=i; //likewise...

With this particular construct having a relatively low probability of
occurring in any sequence of code which is not already breaking strict
aliasing rules.

>
> One can use 'char *' or 'unsigned char *' operations, but these are also
> playing roulette with "may work well" (compiler infers away the byte
> operations) or "performance may be garbage" (compiler actually emits
> byte operations).
>
>
> ...

On 10/12/2021 12:31 PM, George Neuner wrote:
> On Tue, 12 Oct 2021 01:24:33 GMT, Branimir Maksimovic
> <branimir.maksimovic@icloud.com> wrote:
>
>> On 2021-10-11, George Neuner <gneuner2@comcast.net> wrote:
>>> On Sat, 09 Oct 2021 22:22:45 GMT, Branimir Maksimovic
>>> <branimir.maksimovic@icloud.com> wrote:
>>>
>>>> Boehm's collector scans heap/stack/bss, pointing anywhere within
>>>> allocated area, no problem with that.
>>>
>>> The problem isn't finding pointers that refer to the heap, the problem
>>> is finding pointers that refer to particular allocated blocks IN the
>>> heap.
>> \Well, I thouht this was easy task as it is simple pointer
>> arithemtic, but now I think you are right, heap can be
>> fragmented and that's a problem.
>
> Right. With an uncooperative language you can't relocate blocks to
> compact the heap because you can't be certain which program values
> actually are pointers to them.
>

Generally true, though in some sense it is partly more of
"implementation" than "language".

In premise, a C implementation could be made where a subset of the
language was amendable to precise garbage collection and compacting
collectors. Whether or not it would be viable to do so without relying
on language extensions is another matter.

>
>>> Even with programmer assistance, BDW slows down VERY quickly as the
>>> live data set grows.
>>
>> Well, surelly less than when you move allocated area then have to
>> update ll references in profram or to apply membar on every memory
>> access through pointers :p
>
> Actually, many semispace (moving) collectors are /much/ faster than
> BDW which is a mark/sweep system.
>

It depends. On targets which suffer from poor memory bandwidth, then
mark/sweep is generally preferable.

Compacting mostly makes more sense on machines with fast memory but slow
computation (uncommon), rather than machines with fast computation but
slower memory (more common).

Either way, still generally want precise GC rather than conservative GC,
which means the compiler needs to be involved in the process.

Personally I don't feel that precise GC is entirely excluded from being
doable in C, though whether or not doing so would be "worthwhile" is a
bigger issue.

> Compilers for GC'd languages make sure that pointers can be located
> definitively, and that head pointers to heap blocks are never lost or
> obscured. In a well designed system, that means never having to look
> at any non-pointer values.
>

Yeah, this is a major advantage to precise GC.

A semi-precise strategy can also work, where in many locations pointers
are marked by the compiler, object layouts specified, ...

But, then the system falls back to conservative collection for cases
which fall outside the scope of the precise collector.

This approach would likely make more sense for gluing a GC onto C, as
opposed to pure conservative (slow), or pure precise (doesn't play well
with C).

Absent language extensions, will require the compiler to perform some
level of "arcane trickery" with malloc calls and similar, and the GC
would effectively fall apart for user-managed allocators.

Indirectly, this means trusting that the C library provided "malloc()"
doesn't suck, but this issue (hit or miss quality, and by itself lacking
features often needed by the applications) is why so many custom-rolled
allocators exist in the first place.

So, it would mean likely needing to define an extended set of "malloc()"
related API calls, to make it more viable for many of these cases:
Optional GC support;
Ability to query whether or not GC is supported/enabled;
Ability to get object sizes (both MSVCRT and GNU libc have this);
Ability to get/set type dynamic tags;
Ability to associate a heap object with a compile-time layout (*1);
...

Though, while one as at it, may as well add support for:
Dynamically typed values;
Associative dictionary style objects (like in JavaScript);
...

Yeah, errm, I may or may not already have a lot of this glued on as
language extensions in BGBCC...

Well, along with other misc stuff, like:
char *bigstr="""
Well now...
How about a big ol' blob of text...
""";

Though, this is a bit of a stretch to expect any sort of standardized
adoption though.

*1:
There is a type-signature system, but also things like function
signatures, struct layouts, ... can be crammed down into ASCII strings:
int i; // => "i"
void foo(int a, int b); // => "(ii)v"
struct foo_s { int a; int b; }; // => "{a:i;b:i}"
...

> In a semispace collector, scanning can be done /while/ moving the
> object, and a collection only ever looks at live objects once. In
> contrast a mark/compact system has to look at the whole set of live
> objects at least twice: once to mark them, and once to move them.
> A mark/sweep system has to look at live objects to mark them, but at
> ALL objects - both live and dead - to sweep up the garbage.
>

IME, the sweep process tends to be much faster than the mark process
though, since it may only need to touch a tiny fraction as much memory.

One may find that a moving collector effectively needs to touch more
than twice as many cache lines as a pure mark phase.

>
>> GC is costly no question in that :P
>
> But you can pay the cost in different ways.
>
> Baker's Treadmill is the fastest GC implementation known. It is a
> /logical/ semispace collector that doesn't physically move blocks (so
> collection can be done in parallel with the mutator and no need to
> update pointers), and ... when implemented well ... all of it's
> operations are simple and completely deterministic.
>
> Unfortunately, Treadmill achieves its blinding speed at the expense of
> space ... it has the highest memory overhead of all implementations.
>
> YMMV,
> George
>

On 10/12/2021 12:38 PM, Anton Ertl wrote:
> BGB <cr88192@gmail.com> writes:
>> There was a proposal to add lambdas (with C++ style syntax) to C23,
>> though with slightly restricted semantics.
>>
>> Though, granted, lambdas are sort of their own thing, as are GCC style
>> nested functions, ...
>
> That's interesting. Just two weeks ago I presented a paper on a
> simpler alternative to lexical scoping: Two-stage parameter passing.
> This could be used for a lambda-like feature. I used C as example
> (and thought up some syntax that I expect can be added in a
> backwards-compatible way; it's not my ideal syntax).
>

Yeah, though they were not capture-by-reference, merely capture-by-value
only.

I added them to BGBCC in C since they aligned with my own homebrew
language (BS2), and since much of BS2's feature-set has ended up
effectively back-ported to C in this case.

There is a slight funkiness in that there are two slightly different
forms of syntax:
The C++ / C23 syntax, which more or less matches the C23 proposal:
[x, y]()->int { return(x+y); }
Capture by value only (unlike C++);
Automatic lifetime only (if variables are captured).
BGBCC syntax:
__var():int { return(x+y); } //implicit capture by value
__function[x, y]():int { return(x+y); } //explicit capture
...
Differs in that it can also support unbounded lifetime;
Capture by-value by-default.

Both cases differ from Scheme style lambdas though, which were capture
by reference, which when combined with unbounded lifespan, requires
folding out the captured bindings into a heap-allocated structure (this
situation isn't currently supported by BGBCC, which in-effect treat
capture-by-reference and unbounded lifespan as mutually exclusive).

Eg:
__function[&x, &y]():int { return(x+y); } //Invalid (unbounded)
__function![&x, &y]():int { return(x+y); } //OK (automatic)

Both lifespan cases are handled by creating special "read/write/execute"
memory objects, though for automatic lifespan, they are auto-freed when
the parent function returns (same mechanism as alloca and VLAs).

The main difference between __var and __function is that __function
supports capture lists, whereas __var[...] tries to create a "variant
array", where: "__var[1,2,3]" is basically analogous to "[1,2,3]" in
JavaScript or BS2 (basically the same situation with "__var{x:3,y:4}" vs
"{x:3,y:4}").

Well, and also:
__var x;
__variant y;

Look very similar, and in this case do the same thing, but are not
strictly the same (the former creates a variable which merely defaults
to variant, whereas the latter creates a variable of type variant).

Where variant in this context means a dynamically typed value (as a
type-tagged pointer, as opposed to a union, so very much not to be
confused with "std::variant" or similar).

....

> Slides: https://www.complang.tuwien.ac.at/anton/tmp/slides-kps21.pdf
> Paper: https://www.complang.tuwien.ac.at/anton/tmp/ertl21kps.pdf
>
> The talk was better than the paper, but of course on the slides you
> miss the audio track (and you may not understand the German), and may
> have to fill in from the paper. It appeared in:
>
> https://www.uni-kiel.de/journals/receive/jportal_jparticle_00000382
>

OK, yeah, I only really speak English (more or less).

Stephen Fuld [2021-10-12 11:14:10] wrote:
> Are you suggesting that be done in the standard, or in each
> implementation of the standard (i.e. each compiler's documentation)?

I'd expect this to be a compiler-agnostic effort that eventually makes
its way into the standard, but it would inevitably involve
participation of people who have first hand experience of the
gcc/llvm code.

>> This is hard work and I'm not volunteering.
> Can be. I think the work varies across the various cases.
> Even fixing the simple to document ones is progress.

Maybe there are low hanging fruits, indeed. But devil is in the
details, and I don't know enough of the details to guess.

Stefan

On 2021-10-12, MitchAlsup <MitchAlsup@aol.com> wrote:
> On Tuesday, October 12, 2021 at 8:39:58 AM UTC-5, David Brown wrote:
>> On 11/10/2021 20:04, Victor Yodaiken wrote:
>> > On Sunday, October 10, 2021 at 5:48:01 AM UTC-5, David Brown wrote:
>> >
>> >> The idea of C being a "high-level assembler" is right there in
>> >> section 1, "Introduction".
>> >>
>> >
>> > There is something about high level assembler in the introduction -
>> > it is a QUOTE from the C Rationale. If you honestly interpreted it as
>> > a claim that C was supposed to be a complete replacement for all
>> > assembly languages, then maybe you need new glasses.
>> >
>> I know that what you wrote was a quotation from a C Rationale document.
>> Clearly I did not interpret it as you suggest - you cannot complain
>> that I have exaggerated your writings and then do /exactly/ the same
>> thing yourself! The difference, however, is that I was doing so as a
>> criticism of your argument - you appear to be doing so as some kind of
>> revenge.
>>
>>
>> But do I take it that you agree with me that C is not a "high-level
>> assembly"?
>>
>> A vital characteristic of assembly, as a programming language, is that
>> the generated code does /exactly/ what you order, no more and no less,
>> without re-ordering, optimising, removing or adding anything. C is not
>> a language remotely like that, never has been, never will be.
><
> You have obviously never used a good macro assembler............
>>

Macros in assembler are different, you just define macro, that's all :P
No surprises...

--

7-77-777
Evil Sinner!
with software, you repeat same experiment, expecting different results...

On Tuesday, October 12, 2021 at 8:39:58 AM UTC-5, David Brown wrote:

> But do I take it that you agree with me that C is not a "high-level
> assembly"?

The first paragraph of the paper
----
The C programming language [33] is the first, and, so far,only widely successful programming language that provides operating system developers with a high-level language alternative to assembler (compare to [42]).
C’s success was predicated on its design: a small language, close to the machine yet with a great deal of flexibility for experienced programmers.
The Rationale for the C standard [9] cited C’s capability to function as a "high-level assembler" and explained that"many operations are defined to be how the target machine’s hardware does it rather than by a general abstract rule"
but C also has traditional attributes of an ALGOL style programming language.
-----

Note, among other phrases:
1) a high-level language alternative to assembler
2) C also has traditional attributes of an ALGOL style programming language.

Read better.

On Tuesday, October 12, 2021 at 1:18:42 PM UTC-5, BGB wrote:
> On 10/12/2021 10:42 AM, Tim Rentsch wrote:
> > Thomas Koenig <tko...@netcologne.de> writes:
> >
> >> Victor Yodaiken <victor....@gmail.com> schrieb:
> >>
> >>> Can you cast a pointer to a different type? Sure.
> >>>
> >>> Can you dereference it?
> >>
> >> No.
> >
> > This statement is wrong, or at best much overstated. There are
> > plenty of situations where a C pointer value converted to a
> > different type may be safely and portably dereferenced.
> > Certainly that isn't always true, but it isn't always false
> > either.
> >
> Yeah. For some types of code it is desirable to be able to cast and
> dereference a pointer as whatever type matches what one wants to do with it.
<
During my career I went to so many meetings about memory referencing
and how this/that machine was good/bad, I stabilized my thinking about
referencing memory into the camp of::
a) memory is Inherently misaligned,
b) you can't do anything about making that go away,
c) you can make the HW capable of delivering near zero overhead to misalignment;
THAT: you should simply design the machine as inherently misaligned.
<
And at this point, If you create/cast a pointer to something, you CAN
dereference it. Whether it is profitable or not is up to the programmer.
>
> Requiring something like, say, using "memcpy()" to mash bits from one
> format to another and hoping the compiler is smart and reliable enough
> to make this "not suck" is a crap solution, IMHO.
<
Requiring a trip through memory to get from FP-register to int-register
is similarly stupid !! {Note: my restrained hand, here.}
<
SIMD-register to int-register or FP-register probably remains necessary--
because SIMD is not the same (or similar size) as int or FP.
>
> Other options are:
> Just deref it, and hope TBAA doesn't ruin ones day;
> Stick 'volatile' on all the pointers in an attempt to defeat TBAA;
> Maybe works, but still technically UB;
> ...
>
>
> By the time the solution to make something like this work semi-reliable
> and with good performance starts looking like:
> #ifdef _MSC_VER
> ... //use pointer deref (memcpy sucks, pointer deref works)
> #endif
> #ifdef __GNUC__
> ... //use memcpy (memcpy is fast, pointer deref may break)
> #endif
> ...
>
> Stuff is kinda broken...
>
>
> For performance-sensitive code, one may often end up with a bunch of
> compiler and target sensitive boilerplate followed by "generic" fallback
> cases.
>
> This is tolerable, but still kinda sucks.
>
>
> I guess, if I were to define a "common behavior subset", it is possible
> that TBAA rules might be allowed in a restricted form, but maybe define
> it that 'volatile' can effectively circumvent it (by also being able to
> alias with anything).
>
> This would then allow using it on only the operation which violates
> TBAA, rather than all of the pointers in the general vicinity of the
> logic which violates TBAA (which may in turn cross multiple function
> call levels if the compiler does automatic inlining, ...).
>
>
> One can use 'char *' or 'unsigned char *' operations, but these are also
> playing roulette with "may work well" (compiler infers away the byte
> operations) or "performance may be garbage" (compiler actually emits
> byte operations).
>
>
> ...

On Tuesday, October 12, 2021 at 4:03:29 PM UTC-5, Branimir Maksimovic wrote:
> On 2021-10-12, MitchAlsup <Mitch...@aol.com> wrote:
> > On Tuesday, October 12, 2021 at 8:39:58 AM UTC-5, David Brown wrote:
> >> On 11/10/2021 20:04, Victor Yodaiken wrote:
> >> > On Sunday, October 10, 2021 at 5:48:01 AM UTC-5, David Brown wrote:
> >> >
> >> >> The idea of C being a "high-level assembler" is right there in
> >> >> section 1, "Introduction".
> >> >>
> >> >
> >> > There is something about high level assembler in the introduction -
> >> > it is a QUOTE from the C Rationale. If you honestly interpreted it as
> >> > a claim that C was supposed to be a complete replacement for all
> >> > assembly languages, then maybe you need new glasses.
> >> >
> >> I know that what you wrote was a quotation from a C Rationale document.
> >> Clearly I did not interpret it as you suggest - you cannot complain
> >> that I have exaggerated your writings and then do /exactly/ the same
> >> thing yourself! The difference, however, is that I was doing so as a
> >> criticism of your argument - you appear to be doing so as some kind of
> >> revenge.
> >>
> >>
> >> But do I take it that you agree with me that C is not a "high-level
> >> assembly"?
> >>
> >> A vital characteristic of assembly, as a programming language, is that
> >> the generated code does /exactly/ what you order, no more and no less,
> >> without re-ordering, optimising, removing or adding anything. C is not
> >> a language remotely like that, never has been, never will be.
> ><
> > You have obviously never used a good macro assembler............
> >>
> Macros in assembler are different, you just define macro, that's all :P
> No surprises...
<
Macros in assembler can perform a variety of tests to determine the sequence
of code emitted. In the DEC assemblers, when you "call" an assembly macro,
it pops the macro off the symbol table list. This enables an ADD macro to
look at its parameters, and when acceptable, emit an ADD instruction.
<
After this phase, the assembler performs code scheduling and sometimes
peephhole optimizations.
<
We used to use the DEC macro facility to choose between 18-bit offsets
and loading from constant pools on the PDP-10, and what some might
call some "heinous" stuff in the PDP-11 macro facility. I had several macros
that were more than 100 lines of ASM long, typically producing exactly
1 instruction.
> --
>
> 7-77-777
> Evil Sinner!
> with software, you repeat same experiment, expecting different results...

On Tuesday, October 12, 2021 at 4:19:34 PM UTC-5, victor....@gmail.com wrote:
> On Tuesday, October 12, 2021 at 8:39:58 AM UTC-5, David Brown wrote:
>
> > But do I take it that you agree with me that C is not a "high-level
> > assembly"?
> The first paragraph of the paper
> ----
> The C programming language [33] is the first, and, so far,only widely successful programming language that provides operating system developers with a high-level language alternative to assembler (compare to [42]).
> C’s success was predicated on its design: a small language, close to the machine yet with a great deal of flexibility for experienced programmers.
> The Rationale for the C standard [9] cited C’s capability to function as a "high-level assembler" and explained that"many operations are defined to be how the target machine’s hardware does it rather than by a general abstract rule"
> but C also has traditional attributes of an ALGOL style programming language.
> -----
>
> Note, among other phrases:
> 1) a high-level language alternative to assembler
> 2) C also has traditional attributes of an ALGOL style programming language.
2.1) without nesting of {blocks, subroutines, data}
2.2) without strict typing
2.3) without allocating dynamic arrays on the stack
2.4) without do-end
......
>
> Read better.
<
C only smells a bit like Algol, I don't even consider it in the Algol family
unlike {Pascal, ADA, Jovial, Simula }
<
There is an ADA operating system within USA-DOD entirely written in ADA,
with handfulls of ASM (similar to C<->UNIX)

On 10/12/2021 4:46 PM, MitchAlsup wrote:
> On Tuesday, October 12, 2021 at 1:18:42 PM UTC-5, BGB wrote:
>> On 10/12/2021 10:42 AM, Tim Rentsch wrote:
>>> Thomas Koenig <tko...@netcologne.de> writes:
>>>
>>>> Victor Yodaiken <victor....@gmail.com> schrieb:
>>>>
>>>>> Can you cast a pointer to a different type? Sure.
>>>>>
>>>>> Can you dereference it?
>>>>
>>>> No.
>>>
>>> This statement is wrong, or at best much overstated. There are
>>> plenty of situations where a C pointer value converted to a
>>> different type may be safely and portably dereferenced.
>>> Certainly that isn't always true, but it isn't always false
>>> either.
>>>
>> Yeah. For some types of code it is desirable to be able to cast and
>> dereference a pointer as whatever type matches what one wants to do with it.
> <
> During my career I went to so many meetings about memory referencing
> and how this/that machine was good/bad, I stabilized my thinking about
> referencing memory into the camp of::
> a) memory is Inherently misaligned,
> b) you can't do anything about making that go away,
> c) you can make the HW capable of delivering near zero overhead to misalignment;
> THAT: you should simply design the machine as inherently misaligned.
> <
> And at this point, If you create/cast a pointer to something, you CAN
> dereference it. Whether it is profitable or not is up to the programmer.

Granted.

Nearly all access in BJX2 is misaligned-safe by default, with a few
partial exceptions which don't significantly effect code generation for
the most part (mostly effecting the handling of constant displacements
and MOV.X and similar).

>>
>> Requiring something like, say, using "memcpy()" to mash bits from one
>> format to another and hoping the compiler is smart and reliable enough
>> to make this "not suck" is a crap solution, IMHO.
> <
> Requiring a trip through memory to get from FP-register to int-register
> is similarly stupid !! {Note: my restrained hand, here.}
> <
> SIMD-register to int-register or FP-register probably remains necessary--
> because SIMD is not the same (or similar size) as int or FP.

Granted, but these is no way to express this in (standard) C.
There are compiler intrinsics for this case in BGBCC.

In effect, it is mostly how much extra overhead one wants to impose on
transferring a value from one register to another.

Meanwhile, the ASM programmer can make use of both integer and FP
operations using GPRs.

This was also a partial incentive for building dynamic typing into
BGBCC, besides it being needed for my BS2 language: the compiler can do
all this a lot more efficiently than leaving it up to user code.

In theory, it could also allows for a fairly lightweight JavaScript FFI
if C code can more directly access JS data and environments without
needing an awkward and convoluted wrapper API.

Even if, yeah, one probably doesn't want to use dynamic typing when
avoidable for normal code, in my tests, the performance overhead of
using dynamic typing is actually surprisingly modest though (as-in,
"doesn't ruin performance quite as bad as one might intuitively expect
given it turns pretty much every operator into a runtime call").

Full hardware support is unlikely, but it is possible that a few special
cases can be useful, such as a combined "Morton shuffle and Logical
Shift Right" operator and similar, or a "Table-Switch with Unsigned
Saturate".

Say:
ADD R4, -64, R3
BRUSS.L 180, R3
BRA4B .L_case_64
BRA4B .L_case_65
...
BRA4B .L_case_242
BRA4B .L_case_243
BRA4B .L_case_default

Where, any value outside the range of 0..179 goes to 180, while
otherwise branching to (PC+Rn*4).

Mostly this is because dynamically-typed runtime-support code internally
tends to make fairly heavy use of "switch()".

At present, this would be done more like:
ADD R4, -64, R3
CMPHI 179, R3
BT .L_case_default
BRA.L R3
BRA4B .L_case_64
BRA4B .L_case_65
...
BRA4B .L_case_242
BRA4B .L_case_243

The Morton-Shuffle and Shift-Right operation is more needed for
dispatching binary operators based on type-tag.

....

>>
>> Other options are:
>> Just deref it, and hope TBAA doesn't ruin ones day;
>> Stick 'volatile' on all the pointers in an attempt to defeat TBAA;
>> Maybe works, but still technically UB;
>> ...
>>
>>
>> By the time the solution to make something like this work semi-reliable
>> and with good performance starts looking like:
>> #ifdef _MSC_VER
>> ... //use pointer deref (memcpy sucks, pointer deref works)
>> #endif
>> #ifdef __GNUC__
>> ... //use memcpy (memcpy is fast, pointer deref may break)
>> #endif
>> ...
>>
>> Stuff is kinda broken...
>>
>>
>> For performance-sensitive code, one may often end up with a bunch of
>> compiler and target sensitive boilerplate followed by "generic" fallback
>> cases.
>>
>> This is tolerable, but still kinda sucks.
>>
>>
>> I guess, if I were to define a "common behavior subset", it is possible
>> that TBAA rules might be allowed in a restricted form, but maybe define
>> it that 'volatile' can effectively circumvent it (by also being able to
>> alias with anything).
>>
>> This would then allow using it on only the operation which violates
>> TBAA, rather than all of the pointers in the general vicinity of the
>> logic which violates TBAA (which may in turn cross multiple function
>> call levels if the compiler does automatic inlining, ...).
>>
>>
>> One can use 'char *' or 'unsigned char *' operations, but these are also
>> playing roulette with "may work well" (compiler infers away the byte
>> operations) or "performance may be garbage" (compiler actually emits
>> byte operations).
>>
>>
>> ...

According to Branimir Maksimovic <branimir.maksimovic@icloud.com>:
>>> A vital characteristic of assembly, as a programming language, is that
>>> the generated code does /exactly/ what you order, no more and no less,
>>> without re-ordering, optimising, removing or adding anything. C is not
>>> a language remotely like that, never has been, never will be.
>><
>> You have obviously never used a good macro assembler............
>
>Macros in assembler are different, you just define macro, that's all :P
>No surprises...

I'm not getting the impression you've not done a lot of programming in assemblers
with strong macro languages. Someone else mentioned PDP-10 MACRO, but take a
look at HLASM (the oddly named High Level Assembler) for IBM mainframes. They
have macros that with one parameter generate a block of arguments to a call, with
a different parameter generate the call that uses the arguments, and by default
generates both.

https://www-01.ibm.com/servers/resourcelink/svc00100.nsf/pages/zOSV2R3sc264940/$file/asmr1023.pdf

--
Regards,
John Levine, johnl@taugh.com, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. https://jl.ly