On 10/27/2021 11:15 AM, MitchAlsup wrote:
> On Wednesday, October 27, 2021 at 12:04:50 PM UTC-5, Niklas Holsti wrote:
>> On 2021-10-27 19:48, MitchAlsup wrote:
>>> On Wednesday, October 27, 2021 at 11:32:02 AM UTC-5, Stephen Fuld wrote:
>>>> On 10/26/2021 11:38 PM, David Brown wrote:
>>>>> On 27/10/2021 03:28, Tim Rentsch wrote:
>>>>>> Niklas Holsti <niklas...@tidorum.invalid> writes:
>>>>>>
>>>>>>>> [.. volatile ..]
>> [ snip ]
>>>>> That does sound like a simpler mechanism for the same effect. But
>>>>> perhaps we can go further and simpler:
>>>>>
>>>>> x = 1;
>>>>> _Execution_barrier();
>>>>> foo();
>>>>> _Execution_barrier();
>>>>> y = 1;
>>>>>
>>>>> _Execution_barrier() would be an implementation-dependent macro that
>>>>> imposes a sequence point in the actual machine, in the manner described
>>>>> by Tim above. I see no advantage to making it a qualifier or tying it
>>>>> to a particular lvalue access, since it acts on the entire machine.
>>>>>
>>>>> It would still be a big challenge (I believe) to get clear and precise
>>>>> definitions for how this would work, but I think it would be easier to
>>>>> define, understand and implement than "volatile functions" or a new
>>>>> "_Volatile" qualifier.
>>> <
>>>> I think there are two separable questions here. One is what the
>>>> mechanism does. The second is whether the mechanism should be a
>>>> property of the function or of the invocation of that function. Or
>>>> equivalently, should the syntax (whatever it is) be required for each
>>>> call, or should it be invoked automatically (without additional syntax)
>>>> every time the function is invoked.
>>> <
>>> None of this would be necessary if foo() was simply compiled separately
>>> all the way down to code so the compiler could not have any knowledge
>>> of what it does or does not do.
> <
>> Except if link-time optimization is used. And the _Execution_barrier
>> mechanism could be used for any code, not just for function calls.
> <
> You can prevent link-time optimization by erasing the information
> about the function excepting the entry point names. {In any event,
> there does need to be some way of preventing link-time optimization.}

OK, but you certainly want this to be on a per function basis, not
globally. And at that point, you might just as well add some annotation
to the function definition.

Another potential advantage of doing it in the function is that, if some
instruction is required to implement the functionality, then you avoid
the extra instructions in the code stream at each function invocation.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> schrieb:
> On 10/26/2021 3:50 AM, Thomas Koenig wrote:
>> Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>> Terje Mathisen <terje.mathisen@tmsw.no> writes:
>>>> Anton Ertl wrote:
>>>>> Quadibloc <jsavard@ecn.ab.ca> writes:
>>>>>> That isn't a solution. If the problem is to satisfy both sides,
>>>>>> killing the branch of the language that satisfies one of them
>>>>>> constitutes failure; one side is not satisfied.
>>>>>>
>>>>>> However, reality is more complex than that. The reality is
>>>>>> that C99 and gcc are viewed by the dissatisfied side as
>>>>>> just that "solution" as having happened already...
>>>>>
>>>>> Actually, it seems to me that C++ is the solution for the nasal demon
>>>>> worshippers. And once C compilers started to be written by C++
>>>>> programmers, they applied the C++ view of how to deal with undefined
>>>>> behaviour also to C.
>>>>
>>>> So why aren't more people switching to Rust?
>>>
>>> More generally, why aren't people switching to other programming
>>> languages? Because they have a legacy of C code.
>>
>> Or because they like the language, or do not feel like being
>> constrained, or the language is not available on their
>> favorite platform, or...
>>
>> In general, I am in favor of languages which lets me do things I
>> cannot do in other languages.
>
> So you prefer assembler over C, as the you can write instruction
> sequences or even instructions that the compiler would never generate?

I think there is a misunderstanding here, or that I worded that
a bit poorly :-)

Any of the languages we are discussing is Turing complete, so any
language can do anything (modulo limitations in speed and memory).

What I meant is a language that allows me to do things easily
that would otherwise require a lot of programming effort.

A language might, for example, allow multi-dimensional arrays,
or might force me to do it by hand. It might allow strings as
subscripts to arrays, or not. It might allow me to extend types,
or not. It might allow partitioned global address spaces and
parallel execution out of the box, or might require me to
use MPI (ugh). And so on...

C is actually pretty low on that scale.

On 10/27/2021 10:26 AM, Thomas Koenig wrote:
> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> schrieb:
>> On 10/26/2021 3:47 AM, Thomas Koenig wrote:
>>> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> schrieb:
>>>> On 10/24/2021 12:42 AM, Thomas Koenig wrote:
>>>>> Ivan Godard <ivan@millcomputing.com> schrieb:
>>>>>
>>>>>> The lit history is full of efforts to put units into the type systems.
>>>>>> They sure look plausible but don't work. Think of a NR sqrt routine.
>>>>>
>>>>> MathCad has got this right. It is not a compiled language,
>>>>> but it certainly works. I use it a lot in my day job for simple
>>>>> engineering calculations: You see the formulas, you immediately
>>>>> see the output, you do input in units (like kg/h), you can convert
>>>>> the output to units you want.
>>>>>
>>>>> And, of course, it complains if you try to add or subtract
>>>>> dimensions that do not match.
>>>>
>>>> What happens if you multiply two variables with dimension "length"
>>>> together? Does it understand that the result is an "area"?
>>>
>>> It understands that the result is a length squared (and you can
>>> put in an area measurement if you feel like it).
>>
>> I guess the question is, given a variable L of dimension "Length", and a
>> variable A of dimension "Area", does it complain at a statement like
>>
>> A = A + (L*L)
>
> That is accepted, because both are Length^2.

So it "knows" that area is length squared. Neat! But then the question
is how much more does it know about the relationships among names of
dimensions. There must be some limit. You already mentioned work
versus torque issue. I am sure there are others. The question in all
of this is how far to go? I expect that for MathCAD, it is sufficient
for almost all of what its users need. But for a more general
programming language, I think it can get pretty messy.

--
- Stephen Fuld
(e-mail address disguised to prevent spam)

Stephen Fuld <sfuld@alumni.cmu.edu.invalid> schrieb:
> On 10/27/2021 10:26 AM, Thomas Koenig wrote:
>> Stephen Fuld <sfuld@alumni.cmu.edu.invalid> schrieb:

>>> I guess the question is, given a variable L of dimension "Length", and a
>>> variable A of dimension "Area", does it complain at a statement like
>>>
>>> A = A + (L*L)
>>
>> That is accepted, because both are Length^2.
>
> So it "knows" that area is length squared. Neat! But then the question
> is how much more does it know about the relationships among names of
> dimensions. There must be some limit.

It has the standard SI units, and you can define new ones if you
so desire. I often find myself defining ppm as a "unit" (simply
a constant of 1e-6) because I need it in my work.

> You already mentioned work
> versus torque issue. I am sure there are others.

There is the issue of degrees Celsius, which has an offset of
273.15K vs. the SI unit. Solved in the old version with a
function for converting back and forth, I think the new version
has it better.

When looking at atomic units of mass and energy, SI units are pretty
far away from 1, and there could be cases of overflow or underflow.

Apart from that... it works pretty well.

> The question in all
> of this is how far to go? I expect that for MathCAD, it is sufficient
> for almost all of what its users need. But for a more general
> programming language, I think it can get pretty messy.

I think a programming language would have to cater to physicists as
well, so defining another (consisten!) base system might be crucial.

Thomas Koenig <tkoenig@netcologne.de> writes:
>Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>> David Brown <david.brown@hesbynett.no> writes:
>>>C compilers are almost always better at optimising code written in clear
>>>and standard ways - they compiler can get more information from such
>>>code, and the compiler developers put most effort into working with
>>>common code constructs. When people "manually optimise" their C code,
>>>such as with these labels-as-value arrays and hand-written jump tables,
>>>it's difficult for a compiler to see what is happening, and difficult to
>>>optimise - normal optimisation passes may regress performance.
>>
>> Jon Bentley wrote "Writing Efficient Programs" in 1982, and he gave an
>> example in Pascal. I transliterated them to C IIRC in 1999 and you
>> can find the results at
>><http://www.complang.tuwien.ac.at/anton/lvas/effizienz/tsp.html>.
>>
>> Last I looked, the manual optimization steps that Jon Bentley
>> described in 1982 still produced a speedup factor of >2.7,
>
>So, which of these are actually permitted according to the C
>semantics?

Judge for yourself, the code is accessible through this link. The
step from tsp6.c to tsp8.c certainly changes the semantics that the C
compiler sees (it changes the output).

But wrt to the claims made by David Brown, tsp1.c certainly looks
pretty clear to me, and I think it's what David Brown considers
standard, someone who believes him might come to the expectation that
tsp1.c would perform best when compiled with optimization with a
modern compiler. I doesn't.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

There was the short lived effort developing the "Fortress" language.
https://en.wikipedia.org/wiki/Fortress_(programming_language)
and
http://www.ccs.neu.edu/home/samth/fortress-spec.pdf

Early versions (March 2007) of the spec had:
"Numeric types in Fortress can be annotated with physical units and dimensions.
For example, the following function declares that its parameter is a tuple
represented in the units kg and m/s , respectively:
kineticEnergy(m : R kg, v : R m/s) : R kg m 2 /s 2 = (m v 2 )/2"

and
"Measurements in the same unit can be compared, added, subtracted, multiplied
and divided. Measurements in different units can be multiplied and divided."

but
all the units stuff appears to have disappeared in the March 2088 spec:
"To synchronize the specification with the implementation, we temporarily
dropped the following features from the specification:
....
– Dimensions and units"

David Brown <david.brown@hesbynett.no> writes:
>On 27/10/2021 19:34, Anton Ertl wrote:
>> I put up the challenge before <2017Aug18.152854@mips.complang.tuwien.ac.at>
>>
>> |Write a proof-of-concept Forth interpreter in the language you
>> |advocate that runs at least one of bubble-sort, matrix-mult or sieve
>> |from bench/forth in
>> |<http://www.complang.tuwien.ac.at/forth/bench.zip>
>>
>> We can then check whether the language you advocate can compete with a
>> low-level language by benchmarking your interpreter against Gforth.
>>
>>> But to me, that's just a small step from
>>> programming in assembly,
>>
>> That's the point.
>>
>> - anton
>>
>
>I am highly unlikely to have the time to work on your challenge

Yes, writing a proof-of-concept for that takes a few days even if you
already have a plan. What you can do instead is to look at one of the
existing implementations (lots of people have written proof-of-concept
Forth implementations in various languages over the last
half-century), and see how that fares. E.g., pforth claims to be
written in ANSI C.

>I appreciate that Gforth is a rather special case, both in what it needs
>to do and how it is written, which does not fit well with typical C
>coding patterns and typical C compiler techniques.
>
>What I wonder in all this, is whether C is actually the appropriate
>language for Gforth? C is not the right answer to every programming
>need - not even for every low-level high efficiency programming need.

In the beginning, in 1992, GNU C was perfect. We wanted to support
many architectures, wanted to write a threaded-code implementation
(nicely supported by GNU C's labels-as-values). Also, gforth needed
integer types that are as long as a pointer and twice as long; GNU C
provided long long, documented to be twice as long as long int.

Another nice feature of GNU C was (and is) explicit register
variables, which helped work around not-so-great register allocation.

GNU C had it all, and Gforth was nicely portable.

The first difficulty came with the C mainstream for 64-bit systems
made long long 64 bits long, and GNU C followed the mainstream rather
than its own documentation (they changed the documentation after we
filed a bug report). We then implemented the now-missing operations
ourselves, problem solved.

Gcc worked nicely for Gforth from about gcc-2.2 to gcc-2.95.

gcc-3.2 deduplicated the indirect branches, which resulted in severe
slowdowns of threaded code on CPUs of the day like the Pentium III,
the Athlon, or the Pentium 4 (numbers in
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=8092#c6>). This
deduplication sometimes disappeared and sometimes reappeared

At around that time we implemented dynamic native code generation
based on [rossi&sivalingam96] and [piumarta&riccardi98], and we found
a way to automate finding which primitives can be copied (are
relocatable) and which cannot be copied; also worked nicely with
gcc-2.7 .. gcc-2.95. It requires a little bit of originally
non-portable code for synchronising the I-cache with the D-cache; in
theory this could now be completely portable (within gcc) with
__builtin___clear_cache(), but in practice there is Bug 93811.

We eventually found a way to make dynamic native code generation work
with the deduplication and workarounds for the aggressive block
reordering that also appeared in 3.2 or thereabouts. And because the
deduplication slowed down plain threaded code, it became all the more
important to be able to use dynamic native-code generation.

Eventually we got pretty much all the kinks out and the code worked
remarkably reliably across various gcc versions in the 2010s, with the
only hickup due to aggressive code duplication starting with gcc-7 or
so, but I found a workaround for that.

>Are there alternatives that are better? A C-assembly hybrid approach,
>perhaps, with C for the less-critical parts and a fall-back for
>portability while assembly is used on key sections for major targets?

I don't see a benefit here.

>Maybe it should be written in Forth, with the C version just for
>bootstrapping?

Much of Gforth is written in Forth, but being based on an interpreter
means that it needs a base language.

Future plans are to switch to native-code compilation, which is being
helped by the fact that many architectures have died out, so targeting
is not that big of an effort.

>Perhaps llvm code or an internal gcc language as a
>"portable assembly" ?

And the advantage of that would be? The same people who mark a GNU C
bug report as "invalid" would mark an llvm bug report as invalid, and
for the gcc internal representations it's probably even worse.

>Much of what I hear from you is about fighting the C language and
>fighting the gcc compiler - not working /with/ them but working
>/against/ them and trying to use them in ways that neither were designed
>for.

That may be your impression, but it is wrong. As mentioned above, we
were pretty happy with GNU C and the hurdles along the way were either
fixed or could be worked around.

The relationship soured because of the way the bug reports were
treated, e.g. the utter cluelessness exhibited in
<https://gcc.gnu.org/bugzilla/show_bug.cgi?id=25285#c3>. And the
advocates of undefined behaviour saw and see such things as ways to
push their agenda.

@TechReport{rossi&sivalingam96,
author = {Markku Rossi and Kengatharan Sivalingam},
title = {A Survey of Instruction Dispatch Techniques for
Byte-Code Interpreters},
institution = {Faculty of Information Technology, Helsinki
University of Technology},
year = {1996},
number = {TKO-C79},
month = may,
url = {http://www.cs.hut.fi/~cessu/papers/dispatch.ps},
annote = {Describes a number of interpreter dispatch
techniques and compares five dispatch techniques
empirically on five different machines (each with a
different architecture); unfortunately, the
benchmark is hardly described. The most remarkable
part about the paper is that it is the first to
propose the memcpy method for optimizing dispatch
(aka selective inlining \cite{piumarta&riccardi98}
or dynamic superinstructions); however, it does not
explain how to perform control flow in that method
(and their code example looks as if one cannot do
it); it mentions problems with non-relocatable code,
but does not provide a solution; it also mentions
dealing with immediate arguments using a data
pointer, but does not explain it in detail. The idea
is attributed to Kenneth Oksanen.}
}

@InProceedings{piumarta&riccardi98,
author = {Ian Piumarta and Fabio Riccardi},
title = {Optimizing Direct Threaded Code by Selective
Inlining},
crossref = {sigplan98},
pages = {291--300},
url = {ftp://ftp.inria.fr/INRIA/Projects/SOR/papers/1998/ODCSI_pldi98.ps.gz},
annote = {They reduce the overhead of a direct threaded
interpreter by combining all the VM instructions in
a basic block into a single virtual machine
instruction. This is done by simply concatenating
the machine code for the virtual machine
instructions (except for the Next code). Multiple
instances of the same sequence are just translated
once, and then reused. They evaluated this technique
empirically in the context of a fine-grained
RISC-like VM, and in an Objective Caml
interpreter. The speedup over plain direct threaded
code for the RISC-like VM is a factor
1.33--2.06. For the Caml VM the speedups varied
with benchmark and processor, from 1 to 2.2. The
code expansion ranges from 2.2--4 for the Sparc,
with larger benchmarks having less expansion (due to
more reuse of sequences). Implementing this
technique on the Caml VM took only one day.}
}

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 2021-10-28 12:18, David Brown wrote:
> On 27/10/2021 18:31, Stephen Fuld wrote:
>> On 10/26/2021 11:38 PM, David Brown wrote:
>>> On 27/10/2021 03:28, Tim Rentsch wrote:
>>>> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>>>>
>>>>>> [.. volatile ..]
>>>>>
>>>>> These discussions of volatile, and execution order wrt timing,
>>>>> gave me an idea:  perhaps C (and other languages) should allow
>>>>> marking functions (subprograms) as "volatile", with the meaning
>>>>> that all of the effects of a call of that function (including use
>>>>> of processor time) should be ordered as volatile accesses are
>>>>> ordered, with respect to other volatile accesses.
>>>>>
>>>>> For example, if x and y are volatile variables, and foo is a
>>>>> volatile function, then in this code
>>>>>
>>>>>     x = 1;
>>>>>     foo ();
>>>>>     y = 1;
>>>>>
>>>>> we would be sure that all effects and all dynamic resource usage
>>>>> of the foo() call would occur between the assignments to x and to
>>>>> y.
>>>>>
>>>>> A more flexible approach would be to mark selected function calls
>>>>> as volatile, in the same way that C allows in-line use of
>>>>> pointer-to-volatile to force a volatile access to an object that
>>>>> is not itself marked volatile.  Something like:
>>>>>
>>>>>     x = 1;
>>>>>     (volatile) foo ();
>>>>>     y = 1;
>>>>>
>>>>> Are volatile functions and/or volatile function calls a good idea?
>>>>
>>>> Let me propose a simpler mechanism that I believe does a better
>>>> job of what (I think) it is you want to do.  By way of example:
>>>>
>>>>      * (_Volatile int *) &x = 1;
>>>>      foo();
>>>>      * (_Volatile int *) &y = 1;
>>>>
>>>> The semantics of the new _Volatile qualifier, speaking
>>>> informally, is that it imposes a sequence point in the actual
>>>> machine, not just in the abstract machine.  So all logically
>>>> previous evaluations must be finished before a volatile access,
>>>> and after a volatile access all logically subsequent evaluations
>>>> must not yet be started.  To say that another way, no expression
>>>> evaluation (including side effects) may be "moved across" a
>>>> read or write to a _Volatile object.
>>>>
>>>> Note that foo() is a call to an ordinary function, and expressions
>>>> in foo() may be re-ordered in all the usual ways, except that they
>>>> must not be "moved across" the assignment to x or the assignment
>>>> to y.
>>>>
>>>
>>> That does sound like a simpler mechanism for the same effect.  But
>>> perhaps we can go further and simpler:
>>>
>>>     x = 1;
>>>     _Execution_barrier();
>>>     foo();
>>>     _Execution_barrier();
>>>     y = 1;
>>>
>>> _Execution_barrier() would be an implementation-dependent macro that
>>> imposes a sequence point in the actual machine, in the manner described
>>> by Tim above.  I see no advantage to making it a qualifier or tying it
>>> to a particular lvalue access, since it acts on the entire machine.
>>>
>>> It would still be a big challenge (I believe) to get clear and precise
>>> definitions for how this would work, but I think it would be easier to
>>> define, understand and implement than "volatile functions" or a new
>>> "_Volatile" qualifier.
>>
>> I think there are two separable questions here.  One is what the
>> mechanism does.  The second is whether the mechanism should be a
>> property of the function or of the invocation of that function.  Or
>> equivalently, should the syntax (whatever it is) be required for each
>> call, or should it be invoked automatically (without additional syntax)
>> every time the function is invoked.
>>
>
> Agreed.
>
> For the first question, I think there is still a lot of thinking to do.
> I suspect some people will be of the opinion that they already know the
> answer, along the lines of "it means the code should do what the
> programmer wrote", but that is too vague to be used as a specification.
>
> For the second question, my thought was that _Execution_barrier() would
> give you all the options. You can use it stand-alone in code, or
> surrounding a function call. And if the function is supposed to be a
> "volatile function" that is always handled this way, put barriers as the
> first and last statements of the function. But I think that in the end,
> the capabilities of all the suggestions here are the same - it's just a
> question of where you put the new feature in the source code.

Mainly so, but I now realize that the _Execution_barrier method goes
beyond my suggestion of "volatile function". Both methods prevent the
code of a (possibly in-lined) foo() call from being moved past the
barriers, but only the _Execution_barrier method prevents moving other
code into the same barrier-delimited region where the foo() call (or
in-lined code) lies.

In the "volatile function" method, to prevent such motion of other code,
that other code would also have to be defined as a volatile function, or
a volatile block. That could of course be done (and rather easily if a
volatile block is used) but it would be more verbose than the
_Execution_barrier method.

> However, I have now thought of a /big/ problem with all this. At the
> moment, when code calls "foo()" (whose source is unknown to the
> compiler), the compiler can must assume that foo might access any bit of
> memory, or use volatiles, etc., and this puts certain restrictions on
> the optimiser. It can't move "global" reads and writes across the
> function call, for example. But it can also assume that foo does not
> have any undefined behaviour - it can assume foo does not "cheat". So
> it can assume that "foo" follows the ABI with respect to registers it
> preserves, how it accesses the stack, how it interacts with static
> variables that don't escape, and so on. If there is a possibility that
> the external function has such "do as I say despite the UB" behaviour
> that the compiler can't keep these assumptions, then the compiler will
> be crippled and the language efficiency at least halved. It won't
> matter if this is due to an _Execution_barrier, volatile function, or
> _Volatile qualifier.

I don't understand that at all. I don't see any connection between
undefined behaviour and the _Execution_barrier or volatile function
proposals. These proposals bring no new semantics in terms of the C
virtual machine; they only change the pragmatical, non-functional
properties of the code, such as ordering and timing. In other words,
they try to define, to some practical extent, behaviour that is not
specified in the C standard (specifically, the order of computations),
but where every possible behaviour is equally valid from the C
standard's point of view, and equivalent in terms of the observable
behaviour of the C program (as currently defined, that is, without
considering the order and timing of computations as observable effects).

I also don't see why limiting the compiler's code-reordering ability
would tempt the compiler into violating the ABI.

On 10/28/2021 9:18 AM, EricP wrote:
> MitchAlsup wrote:
>> On Tuesday, October 26, 2021 at 7:19:12 PM UTC-5, Bakul Shah wrote:
>>> MitchAlsup wrote:
>>>> On Tuesday, October 26, 2021 at 11:57:44 AM UTC-5, Ivan Godard wrote:
>>>>> On 10/26/2021 9:36 AM, MitchAlsup wrote:
>>>>>> But that function needs to be provided by the compiler as part of
>>>>>> its own run time (support library). It is not something that is
>>>>>> willy-nilly generated by a semi-concerned programmer using a few
>>>>>> compiler switches. < Also, how does one unwind the stack when one
>>>>>> has no access to the parts of the stack that are protected from
>>>>>> the user code ?
>>>> <
>>>>> Good question. Our answer is NYF.
>>>> < Seems to me you have 2 choices:: some HW instruction that performs
>>>> job at hand faking access rights {sort of what I do in HW context
>>>> switches...........} Or a cheap context switch to a address space
>>>> with permissions to do the job in SW. < It seems painfully obvious
>>>> that the entirety of the mechanism is to prevent user (unprivileged
>>>> code) from accessing those locations {security, RoP,...}
>>> First, note that this is a separate issue from a language not having
>>> the primitives to examining program stack. This is a protection
>>> domain issue.
>> <
>> Yes, indeed, this is a pure protection issue--one designed to totally
>> eliminate RoP style of attacks. One can find the RET instructions,
>> but since one cannot see the return address, nor the preserved registers
>> one cannot string together a meaningful sequence of existing instruction
>> {beneficially or malevolently.}
>> <
>>>                                              One can look at
>>> capability OSes such as KeyKOS for such issues. This is what Norm
>>> Hardy says in http://www.cap-lore.com/CapTheory/KK/Debug.html
>>> In classic programming environments I am accustomed to a debugger
>>> that displays the stack and understands the name spaces defined by
>>> each of the stack frames. This pattern does not work for the Keykos
>>> application that is distributed in many Keykos objects running in
>>> separate address spaces and with no conceptual stack known to the
>>> system or debugger. A further problem in Keykos is that the
>>> programmer who is debugging code may not be authorized to see the
>>> implementation of other objects involved with the current state where
>>> the problem is manifested. OO languages typically assume that
>>> encapsulation is merely for limiting access by programs and that a
>>> programmer can ignore such barriers while debugging the code. In
>>> Keykos we took the opposite view and assumed that encapsulation was
>>> really to keep secrets, even from programmers, debuggers and DDT
>>> (gdb). We took this attitude with some trepidation, but business
>>> plans allowed no alternative.
>>> Presumably the debugger would have to be given the necessary keys
>>> (capabilities) to be able to access stack segment in a different domain.
>> <
>> To bring you up to speed, Ivan and I are talking about a stack
>> environment
>> where there are 2 stacks, one where visible data is stored, and one
>> where invisible data is stored. The user does not have read/write/execute
>> access to the second store, while the HW does have access, and indeed
>> looks to see that the pages being used are RWE=000--which means only
>> HW is allowed access.
>> <
>> Now debuggers are not HW, and thus the only means they have to look into
>> that stack area is by being a different task with a different virtual
>> address
>> space where the hidden store is visible. But given the debugger knows
>> which
>> task it is debugging, there is a straightforward means to access the
>> suspended
>> task's header obtaining the root pointer, and then read/write thread
>> registers,
>> or read from and writ to memory in the foreign address space.
>
> There are two other ways to do this kind of stuff which I quite
> like so I'll mention them (but I don't know if or how this could
> fit with your approaches).
>
> (1) First method is for threads to support what I call
> Thread Interrupt Procedures (TIP's),
> VMS calls them Asynchronous System Traps (AST's) and
> WinNT calls them Asynchronous Procedure Calls (APC's).
> It is all the same basic concept - a software interrupt to a thread.
>
> A TIP is a kernel packet that contains a pointer to a routine to call
> and 3 or 4 arguments to pass to that routine, and links for a list.
> Each thread has a list of pending TIPs. When a TIP is posted to a
> thread, if that thread is in a Wait state it wakes up to a Ready state.
> When the thread next runs, the first pending TIP is delivered in kernel
> mode and its routine is called passing the args, interrupting whatever
> the thread was doing, and blocking TIP delivery until it returns.
>
> Note that when the thread executes the TIP routine,
> it is in the address space of the thread's process.
>
> One thing TIP/AST/APC's are used for is IO completion notifications.
> When an IO driver is all finished it posts a TIP to the thread to wake
> it up and perform final IO clean up and actions like set event flag.
>
> One way to force threads to do things is to allocate some memory and
> package up some code with a TIP header and post it to the thread.
> One of the args is a pointer to the TIP packet so the packet so
> when the code finishes executing it can free itself.
>
> (2) On WinNT kernel mode threads can detach from their process and
> attach to a different one, in effect switching address spaces.
> It can then execute as though is was a thread of that process,
> and later revert back to its original parent process.
>
> Conceptually a thread has a pointer to its parent process which contains
> the address space so this looks like just switching parent pointers.
> Under the hood it is more complicated because, for example,
> those TIP's mentioned above should only be delivered when attached
> to the base process. Also when a thread reverts all IO issued while
> attached must be complete.
>
> This threads' process-attach ability can be used to create functionality
> that allows a suitably privileged thread to create a memory section
> in another process, or map a DLL, or open an end of a pipe and
> put the pipe handle into that process handle table.
> Service threads can roam around, attaching to processes and
> performing housekeeping work in that address space.
>
>

Does the migrating thread retain visibility into its down-stack state,
even though that state is not visible to the process it attaches to?
That is, is it migrating an actual thread or just a protection domain
and an entry point?

On Thursday, October 28, 2021 at 11:19:19 AM UTC-5, EricP wrote:
> MitchAlsup wrote:
> > On Tuesday, October 26, 2021 at 7:19:12 PM UTC-5, Bakul Shah wrote:
> >> MitchAlsup wrote:
> >>> On Tuesday, October 26, 2021 at 11:57:44 AM UTC-5, Ivan Godard wrote:
> >>>> On 10/26/2021 9:36 AM, MitchAlsup wrote:
> >>>>> But that function needs to be provided by the compiler as part of its own run
> >>>>> time (support library). It is not something that is willy-nilly generated by a
> >>>>> semi-concerned programmer using a few compiler switches.
> >>>>> <
> >>>>> Also, how does one unwind the stack when one has no access to the parts
> >>>>> of the stack that are protected from the user code ?
> >>> <
> >>>> Good question. Our answer is NYF.
> >>> <
> >>> Seems to me you have 2 choices:: some HW instruction that performs job at hand
> >>> faking access rights {sort of what I do in HW context switches...........}
> >>> Or a cheap context switch to a address space with permissions to do the job in SW.
> >>> <
> >>> It seems painfully obvious that the entirety of the mechanism is to prevent user
> >>> (unprivileged code) from accessing those locations {security, RoP,...}
> >>>
> >> First, note that this is a separate issue from a language not
> >> having the primitives to examining program stack. This is a
> >> protection domain issue.
> > <
> > Yes, indeed, this is a pure protection issue--one designed to totally
> > eliminate RoP style of attacks. One can find the RET instructions,
> > but since one cannot see the return address, nor the preserved registers
> > one cannot string together a meaningful sequence of existing instruction
> > {beneficially or malevolently.}
> > <
> >> One can look at capability OSes
> >> such as KeyKOS for such issues. This is what Norm Hardy says in
> >> http://www.cap-lore.com/CapTheory/KK/Debug.html
> >>
> >> In classic programming environments I am accustomed to a
> >> debugger that displays the stack and understands the name
> >> spaces defined by each of the stack frames. This pattern
> >> does not work for the Keykos application that is
> >> distributed in many Keykos objects running in separate
> >> address spaces and with no conceptual stack known to the
> >> system or debugger. A further problem in Keykos is that the
> >> programmer who is debugging code may not be authorized to
> >> see the implementation of other objects involved with the
> >> current state where the problem is manifested. OO languages
> >> typically assume that encapsulation is merely for limiting
> >> access by programs and that a programmer can ignore such
> >> barriers while debugging the code. In Keykos we took the
> >> opposite view and assumed that encapsulation was really to
> >> keep secrets, even from programmers, debuggers and DDT
> >> (gdb). We took this attitude with some trepidation, but
> >> business plans allowed no alternative.
> >>
> >> Presumably the debugger would have to be given the necessary
> >> keys (capabilities) to be able to access stack segment in a
> >> different domain.
> > <
> > To bring you up to speed, Ivan and I are talking about a stack environment
> > where there are 2 stacks, one where visible data is stored, and one where
> > invisible data is stored. The user does not have read/write/execute
> > access to the second store, while the HW does have access, and indeed
> > looks to see that the pages being used are RWE=000--which means only
> > HW is allowed access.
> > <
> > Now debuggers are not HW, and thus the only means they have to look into
> > that stack area is by being a different task with a different virtual address
> > space where the hidden store is visible. But given the debugger knows which
> > task it is debugging, there is a straightforward means to access the suspended
> > task's header obtaining the root pointer, and then read/write thread registers,
> > or read from and writ to memory in the foreign address space.
>
> There are two other ways to do this kind of stuff which I quite
> like so I'll mention them (but I don't know if or how this could
> fit with your approaches).
>
> (1) First method is for threads to support what I call
> Thread Interrupt Procedures (TIP's),
> VMS calls them Asynchronous System Traps (AST's) and
> WinNT calls them Asynchronous Procedure Calls (APC's).
> It is all the same basic concept - a software interrupt to a thread.
>
> A TIP is a kernel packet that contains a pointer to a routine to call
> and 3 or 4 arguments to pass to that routine, and links for a list.
> Each thread has a list of pending TIPs. When a TIP is posted to a
> thread, if that thread is in a Wait state it wakes up to a Ready state.
> When the thread next runs, the first pending TIP is delivered in kernel
> mode and its routine is called passing the args, interrupting whatever
> the thread was doing, and blocking TIP delivery until it returns.
<
So TIPs are serially reuseable (to get rid of a variety of problems) and
if a TIP is currently running other pending TIPs are dropped on the 'list' !?
>
> Note that when the thread executes the TIP routine,
> it is in the address space of the thread's process.
>
> One thing TIP/AST/APC's are used for is IO completion notifications.
> When an IO driver is all finished it posts a TIP to the thread to wake
> it up and perform final IO clean up and actions like set event flag.
<
So, if the thread is still in the run state--the TIP is held on its list until
the thread reaches a wait-state ?!? and then when the thread goes into
a wait-state the TIP is dispatched !! does the wait-state vanish or does
running of TIP have to cause the wait-state to be changed back to
run-state ? imperatively ?
<
Does the completion of 1 TIP immediately cause the next TIP on the
thread-list to run ? or does this mechanism knock 1 off per wait-state ?
>
> One way to force threads to do things is to allocate some memory and
> package up some code with a TIP header and post it to the thread.
> One of the args is a pointer to the TIP packet so the packet so
> when the code finishes executing it can free itself.
>
> (2) On WinNT kernel mode threads can detach from their process and
> attach to a different one, in effect switching address spaces.
> It can then execute as though is was a thread of that process,
> and later revert back to its original parent process.
>
> Conceptually a thread has a pointer to its parent process which contains
> the address space so this looks like just switching parent pointers.
> Under the hood it is more complicated because, for example,
> those TIP's mentioned above should only be delivered when attached
> to the base process. Also when a thread reverts all IO issued while
> attached must be complete.
>
> This threads' process-attach ability can be used to create functionality
> that allows a suitably privileged thread to create a memory section
> in another process, or map a DLL, or open an end of a pipe and
> put the pipe handle into that process handle table.
> Service threads can roam around, attaching to processes and
> performing housekeeping work in that address space.

On Thursday, October 28, 2021 at 7:19:19 PM UTC+3, EricP wrote:
> MitchAlsup wrote:
> > On Tuesday, October 26, 2021 at 7:19:12 PM UTC-5, Bakul Shah wrote:
> >> MitchAlsup wrote:
> >>> On Tuesday, October 26, 2021 at 11:57:44 AM UTC-5, Ivan Godard wrote:
> >>>> On 10/26/2021 9:36 AM, MitchAlsup wrote:
> >>>>> But that function needs to be provided by the compiler as part of its own run
> >>>>> time (support library). It is not something that is willy-nilly generated by a
> >>>>> semi-concerned programmer using a few compiler switches.
> >>>>> <
> >>>>> Also, how does one unwind the stack when one has no access to the parts
> >>>>> of the stack that are protected from the user code ?
> >>> <
> >>>> Good question. Our answer is NYF.
> >>> <
> >>> Seems to me you have 2 choices:: some HW instruction that performs job at hand
> >>> faking access rights {sort of what I do in HW context switches...........}
> >>> Or a cheap context switch to a address space with permissions to do the job in SW.
> >>> <
> >>> It seems painfully obvious that the entirety of the mechanism is to prevent user
> >>> (unprivileged code) from accessing those locations {security, RoP,...}
> >>>
> >> First, note that this is a separate issue from a language not
> >> having the primitives to examining program stack. This is a
> >> protection domain issue.
> > <
> > Yes, indeed, this is a pure protection issue--one designed to totally
> > eliminate RoP style of attacks. One can find the RET instructions,
> > but since one cannot see the return address, nor the preserved registers
> > one cannot string together a meaningful sequence of existing instruction
> > {beneficially or malevolently.}
> > <
> >> One can look at capability OSes
> >> such as KeyKOS for such issues. This is what Norm Hardy says in
> >> http://www.cap-lore.com/CapTheory/KK/Debug.html
> >>
> >> In classic programming environments I am accustomed to a
> >> debugger that displays the stack and understands the name
> >> spaces defined by each of the stack frames. This pattern
> >> does not work for the Keykos application that is
> >> distributed in many Keykos objects running in separate
> >> address spaces and with no conceptual stack known to the
> >> system or debugger. A further problem in Keykos is that the
> >> programmer who is debugging code may not be authorized to
> >> see the implementation of other objects involved with the
> >> current state where the problem is manifested. OO languages
> >> typically assume that encapsulation is merely for limiting
> >> access by programs and that a programmer can ignore such
> >> barriers while debugging the code. In Keykos we took the
> >> opposite view and assumed that encapsulation was really to
> >> keep secrets, even from programmers, debuggers and DDT
> >> (gdb). We took this attitude with some trepidation, but
> >> business plans allowed no alternative.
> >>
> >> Presumably the debugger would have to be given the necessary
> >> keys (capabilities) to be able to access stack segment in a
> >> different domain.
> > <
> > To bring you up to speed, Ivan and I are talking about a stack environment
> > where there are 2 stacks, one where visible data is stored, and one where
> > invisible data is stored. The user does not have read/write/execute
> > access to the second store, while the HW does have access, and indeed
> > looks to see that the pages being used are RWE=000--which means only
> > HW is allowed access.
> > <
> > Now debuggers are not HW, and thus the only means they have to look into
> > that stack area is by being a different task with a different virtual address
> > space where the hidden store is visible. But given the debugger knows which
> > task it is debugging, there is a straightforward means to access the suspended
> > task's header obtaining the root pointer, and then read/write thread registers,
> > or read from and writ to memory in the foreign address space.
>
> There are two other ways to do this kind of stuff which I quite
> like so I'll mention them (but I don't know if or how this could
> fit with your approaches).
>
> (1) First method is for threads to support what I call
> Thread Interrupt Procedures (TIP's),
> VMS calls them Asynchronous System Traps (AST's) and
> WinNT calls them Asynchronous Procedure Calls (APC's).
> It is all the same basic concept - a software interrupt to a thread.
>
> A TIP is a kernel packet that contains a pointer to a routine to call
> and 3 or 4 arguments to pass to that routine, and links for a list.
> Each thread has a list of pending TIPs. When a TIP is posted to a
> thread, if that thread is in a Wait state it wakes up to a Ready state.
> When the thread next runs, the first pending TIP is delivered in kernel
> mode and its routine is called passing the args, interrupting whatever
> the thread was doing,

I heard that AST on VMS works as you described. But APC on WNT does not.
Despite the name, WNT ASP is asynchronous only in a sense that QueueUserAPC() call is non-blocking.
But execution at destination tread is synchronous and happens only after the target thread entered an alertable
state by SleepEx(), WaitForSingleObjectEx(), WaitForMultipleObjectsEx() and couple of other similar syscalls.

> and blocking TIP delivery until it returns.
>
> Note that when the thread executes the TIP routine,
> it is in the address space of the thread's process.
>
> One thing TIP/AST/APC's are used for is IO completion notifications.
> When an IO driver is all finished it posts a TIP to the thread to wake
> it up and perform final IO clean up and actions like set event flag.
>
> One way to force threads to do things is to allocate some memory and
> package up some code with a TIP header and post it to the thread.
> One of the args is a pointer to the TIP packet so the packet so
> when the code finishes executing it can free itself.
>
> (2) On WinNT kernel mode threads can detach from their process and
> attach to a different one, in effect switching address spaces.
> It can then execute as though is was a thread of that process,
> and later revert back to its original parent process.
>
> Conceptually a thread has a pointer to its parent process which contains
> the address space so this looks like just switching parent pointers.
> Under the hood it is more complicated because, for example,
> those TIP's mentioned above should only be delivered when attached
> to the base process. Also when a thread reverts all IO issued while
> attached must be complete.
>
> This threads' process-attach ability can be used to create functionality
> that allows a suitably privileged thread to create a memory section
> in another process, or map a DLL, or open an end of a pipe and
> put the pipe handle into that process handle table.
> Service threads can roam around, attaching to processes and
> performing housekeeping work in that address space.

Michael S wrote:
> On Thursday, October 28, 2021 at 7:19:19 PM UTC+3, EricP wrote:
>> There are two other ways to do this kind of stuff which I quite
>> like so I'll mention them (but I don't know if or how this could
>> fit with your approaches).
>>
>> (1) First method is for threads to support what I call
>> Thread Interrupt Procedures (TIP's),
>> VMS calls them Asynchronous System Traps (AST's) and
>> WinNT calls them Asynchronous Procedure Calls (APC's).
>> It is all the same basic concept - a software interrupt to a thread.
>>
>> A TIP is a kernel packet that contains a pointer to a routine to call
>> and 3 or 4 arguments to pass to that routine, and links for a list.
>> Each thread has a list of pending TIPs. When a TIP is posted to a
>> thread, if that thread is in a Wait state it wakes up to a Ready state.
>> When the thread next runs, the first pending TIP is delivered in kernel
>> mode and its routine is called passing the args, interrupting whatever
>> the thread was doing,
>> and blocking TIP delivery until it returns.
>
> I heard that AST on VMS works as you described. But APC on WNT does not.
> Despite the name, WNT ASP is asynchronous only in a sense that QueueUserAPC() call is non-blocking.
> But execution at destination tread is synchronous and happens only after the target thread entered an alertable
> state by SleepEx(), WaitForSingleObjectEx(), WaitForMultipleObjectsEx() and couple of other similar syscalls.

WinNT kernel mode APCs do work that way, as thread interrupts.
Yes, the the _documented_ user mode "APCs" are actually synchronous,
delivered only at certain points and only on request.

There are undocumented user mode APCs that are asynchronous and delivered
at the next user-kernel transition (which you can force so they behave
like interrupts) but these are only available from within kernel mode
and have been intentionally lobotomized so they can only deliver
a program counter on MS approved list.

See section "nt!PsExitSpecialApc and The Special User APC"
http://www.opening-windows.com/techart_windows_vista_apc_internals2.htm#_Toc229652501

I don't know why MS lobotomized them that way.
I suspect it is because Win32 didn't sort out reentrancy properly in its
design and they are stuck with a bunch of non interrupt reentrant code.
VMS user mode AST's are general and work just fine.

I'm guessing MS uses the secret user mode APCs are used for things like
when a thread maps a new DLL then all the threads in the process must be
forced to run the DLL init routine, because DLL init routines do things
like allocate TLS storage.
Also these might be used to implement Posix signals properly
(as opposed to the botched way they did signals the first time).

Ivan Godard wrote:
> On 10/28/2021 9:18 AM, EricP wrote:
>> MitchAlsup wrote:
>>> On Tuesday, October 26, 2021 at 7:19:12 PM UTC-5, Bakul Shah wrote:
>>>> MitchAlsup wrote:
>>>>> On Tuesday, October 26, 2021 at 11:57:44 AM UTC-5, Ivan Godard wrote:
>>>>>> On 10/26/2021 9:36 AM, MitchAlsup wrote:
>>>>>>> But that function needs to be provided by the compiler as part of
>>>>>>> its own run time (support library). It is not something that is
>>>>>>> willy-nilly generated by a semi-concerned programmer using a few
>>>>>>> compiler switches. < Also, how does one unwind the stack when one
>>>>>>> has no access to the parts of the stack that are protected from
>>>>>>> the user code ?
>>>>> <
>>>>>> Good question. Our answer is NYF.
>>>>> < Seems to me you have 2 choices:: some HW instruction that
>>>>> performs job at hand faking access rights {sort of what I do in HW
>>>>> context switches...........} Or a cheap context switch to a address
>>>>> space with permissions to do the job in SW. < It seems painfully
>>>>> obvious that the entirety of the mechanism is to prevent user
>>>>> (unprivileged code) from accessing those locations {security, RoP,...}
>>>> First, note that this is a separate issue from a language not having
>>>> the primitives to examining program stack. This is a protection
>>>> domain issue.
>>> <
>>> Yes, indeed, this is a pure protection issue--one designed to totally
>>> eliminate RoP style of attacks. One can find the RET instructions,
>>> but since one cannot see the return address, nor the preserved registers
>>> one cannot string together a meaningful sequence of existing instruction
>>> {beneficially or malevolently.}
>>> <
>>>> One can look at
>>>> capability OSes such as KeyKOS for such issues. This is what Norm
>>>> Hardy says in http://www.cap-lore.com/CapTheory/KK/Debug.html
>>>> In classic programming environments I am accustomed to a debugger
>>>> that displays the stack and understands the name spaces defined by
>>>> each of the stack frames. This pattern does not work for the Keykos
>>>> application that is distributed in many Keykos objects running in
>>>> separate address spaces and with no conceptual stack known to the
>>>> system or debugger. A further problem in Keykos is that the
>>>> programmer who is debugging code may not be authorized to see the
>>>> implementation of other objects involved with the current state
>>>> where the problem is manifested. OO languages typically assume that
>>>> encapsulation is merely for limiting access by programs and that a
>>>> programmer can ignore such barriers while debugging the code. In
>>>> Keykos we took the opposite view and assumed that encapsulation was
>>>> really to keep secrets, even from programmers, debuggers and DDT
>>>> (gdb). We took this attitude with some trepidation, but business
>>>> plans allowed no alternative.
>>>> Presumably the debugger would have to be given the necessary keys
>>>> (capabilities) to be able to access stack segment in a different
>>>> domain.
>>> <
>>> To bring you up to speed, Ivan and I are talking about a stack
>>> environment
>>> where there are 2 stacks, one where visible data is stored, and one
>>> where invisible data is stored. The user does not have
>>> read/write/execute
>>> access to the second store, while the HW does have access, and indeed
>>> looks to see that the pages being used are RWE=000--which means only
>>> HW is allowed access.
>>> <
>>> Now debuggers are not HW, and thus the only means they have to look into
>>> that stack area is by being a different task with a different virtual
>>> address
>>> space where the hidden store is visible. But given the debugger knows
>>> which
>>> task it is debugging, there is a straightforward means to access the
>>> suspended
>>> task's header obtaining the root pointer, and then read/write thread
>>> registers,
>>> or read from and writ to memory in the foreign address space.
>>
>> There are two other ways to do this kind of stuff which I quite
>> like so I'll mention them (but I don't know if or how this could
>> fit with your approaches).
>>
>> (1) First method is for threads to support what I call
>> Thread Interrupt Procedures (TIP's),
>> VMS calls them Asynchronous System Traps (AST's) and
>> WinNT calls them Asynchronous Procedure Calls (APC's).
>> It is all the same basic concept - a software interrupt to a thread.
>>
>> A TIP is a kernel packet that contains a pointer to a routine to call
>> and 3 or 4 arguments to pass to that routine, and links for a list.
>> Each thread has a list of pending TIPs. When a TIP is posted to a
>> thread, if that thread is in a Wait state it wakes up to a Ready state.
>> When the thread next runs, the first pending TIP is delivered in kernel
>> mode and its routine is called passing the args, interrupting whatever
>> the thread was doing, and blocking TIP delivery until it returns.
>>
>> Note that when the thread executes the TIP routine,
>> it is in the address space of the thread's process.
>>
>> One thing TIP/AST/APC's are used for is IO completion notifications.
>> When an IO driver is all finished it posts a TIP to the thread to wake
>> it up and perform final IO clean up and actions like set event flag.
>>
>> One way to force threads to do things is to allocate some memory and
>> package up some code with a TIP header and post it to the thread.
>> One of the args is a pointer to the TIP packet so the packet so
>> when the code finishes executing it can free itself.
>>
>> (2) On WinNT kernel mode threads can detach from their process and
>> attach to a different one, in effect switching address spaces.
>> It can then execute as though is was a thread of that process,
>> and later revert back to its original parent process.
>>
>> Conceptually a thread has a pointer to its parent process which contains
>> the address space so this looks like just switching parent pointers.
>> Under the hood it is more complicated because, for example,
>> those TIP's mentioned above should only be delivered when attached
>> to the base process. Also when a thread reverts all IO issued while
>> attached must be complete.
>>
>> This threads' process-attach ability can be used to create functionality
>> that allows a suitably privileged thread to create a memory section
>> in another process, or map a DLL, or open an end of a pipe and
>> put the pipe handle into that process handle table.
>> Service threads can roam around, attaching to processes and
>> performing housekeeping work in that address space.
>>
>>
>
> Does the migrating thread retain visibility into its down-stack state,
> even though that state is not visible to the process it attaches to?
> That is, is it migrating an actual thread or just a protection domain
> and an entry point?

I interpret your question to ask does the thread retain access
to its original user mode context and stack from the prior process
while in the new process, and the answer is no.

However the kernel thread context and its kernel stack are retained as
they are in system space which is the high address range of all processes.

To move data from one process to another you copy it from process address
space to system address space, switch processes, then copy it out.

The process virtual space and its supported management data structures
which reside inside the process address space would switch.
These could be the process page table, process working set table,
process virtual space table, and process handle table.

Once inside the new process the thread can modify its address space,
create or modify memory sections, page fault pages in, etc.

MitchAlsup wrote:
> On Thursday, October 28, 2021 at 11:19:19 AM UTC-5, EricP wrote:
>> MitchAlsup wrote:
>>> On Tuesday, October 26, 2021 at 7:19:12 PM UTC-5, Bakul Shah wrote:
>>>> MitchAlsup wrote:
>>>>> On Tuesday, October 26, 2021 at 11:57:44 AM UTC-5, Ivan Godard wrote:
>>>>>> On 10/26/2021 9:36 AM, MitchAlsup wrote:
>>>>>>> But that function needs to be provided by the compiler as part of its own run
>>>>>>> time (support library). It is not something that is willy-nilly generated by a
>>>>>>> semi-concerned programmer using a few compiler switches.
>>>>>>> <
>>>>>>> Also, how does one unwind the stack when one has no access to the parts
>>>>>>> of the stack that are protected from the user code ?
>>>>> <
>>>>>> Good question. Our answer is NYF.
>>>>> <
>>>>> Seems to me you have 2 choices:: some HW instruction that performs job at hand
>>>>> faking access rights {sort of what I do in HW context switches...........}
>>>>> Or a cheap context switch to a address space with permissions to do the job in SW.
>>>>> <
>>>>> It seems painfully obvious that the entirety of the mechanism is to prevent user
>>>>> (unprivileged code) from accessing those locations {security, RoP,...}
>>>>>
>>>> First, note that this is a separate issue from a language not
>>>> having the primitives to examining program stack. This is a
>>>> protection domain issue.
>>> <
>>> Yes, indeed, this is a pure protection issue--one designed to totally
>>> eliminate RoP style of attacks. One can find the RET instructions,
>>> but since one cannot see the return address, nor the preserved registers
>>> one cannot string together a meaningful sequence of existing instruction
>>> {beneficially or malevolently.}
>>> <
>>>> One can look at capability OSes
>>>> such as KeyKOS for such issues. This is what Norm Hardy says in
>>>> http://www.cap-lore.com/CapTheory/KK/Debug.html
>>>>
>>>> In classic programming environments I am accustomed to a
>>>> debugger that displays the stack and understands the name
>>>> spaces defined by each of the stack frames. This pattern
>>>> does not work for the Keykos application that is
>>>> distributed in many Keykos objects running in separate
>>>> address spaces and with no conceptual stack known to the
>>>> system or debugger. A further problem in Keykos is that the
>>>> programmer who is debugging code may not be authorized to
>>>> see the implementation of other objects involved with the
>>>> current state where the problem is manifested. OO languages
>>>> typically assume that encapsulation is merely for limiting
>>>> access by programs and that a programmer can ignore such
>>>> barriers while debugging the code. In Keykos we took the
>>>> opposite view and assumed that encapsulation was really to
>>>> keep secrets, even from programmers, debuggers and DDT
>>>> (gdb). We took this attitude with some trepidation, but
>>>> business plans allowed no alternative.
>>>>
>>>> Presumably the debugger would have to be given the necessary
>>>> keys (capabilities) to be able to access stack segment in a
>>>> different domain.
>>> <
>>> To bring you up to speed, Ivan and I are talking about a stack environment
>>> where there are 2 stacks, one where visible data is stored, and one where
>>> invisible data is stored. The user does not have read/write/execute
>>> access to the second store, while the HW does have access, and indeed
>>> looks to see that the pages being used are RWE=000--which means only
>>> HW is allowed access.
>>> <
>>> Now debuggers are not HW, and thus the only means they have to look into
>>> that stack area is by being a different task with a different virtual address
>>> space where the hidden store is visible. But given the debugger knows which
>>> task it is debugging, there is a straightforward means to access the suspended
>>> task's header obtaining the root pointer, and then read/write thread registers,
>>> or read from and writ to memory in the foreign address space.
>> There are two other ways to do this kind of stuff which I quite
>> like so I'll mention them (but I don't know if or how this could
>> fit with your approaches).
>>
>> (1) First method is for threads to support what I call
>> Thread Interrupt Procedures (TIP's),
>> VMS calls them Asynchronous System Traps (AST's) and
>> WinNT calls them Asynchronous Procedure Calls (APC's).
>> It is all the same basic concept - a software interrupt to a thread.
>>
>> A TIP is a kernel packet that contains a pointer to a routine to call
>> and 3 or 4 arguments to pass to that routine, and links for a list.
>> Each thread has a list of pending TIPs. When a TIP is posted to a
>> thread, if that thread is in a Wait state it wakes up to a Ready state.
>> When the thread next runs, the first pending TIP is delivered in kernel
>> mode and its routine is called passing the args, interrupting whatever
>> the thread was doing, and blocking TIP delivery until it returns.
> <
> So TIPs are serially reuseable (to get rid of a variety of problems) and

Yes, the TIP packet is immediately reusable after delivery.
So for example you can reuse it to post a reply back to the sender.

> if a TIP is currently running other pending TIPs are dropped on the 'list' !?

Yes. Like priority interrupts, the current TIP blocks new deliveries
at the same priority.

>> Note that when the thread executes the TIP routine,
>> it is in the address space of the thread's process.
>>
>> One thing TIP/AST/APC's are used for is IO completion notifications.
>> When an IO driver is all finished it posts a TIP to the thread to wake
>> it up and perform final IO clean up and actions like set event flag.
> <
> So, if the thread is still in the run state--the TIP is held on its list until
> the thread reaches a wait-state ?!? and then when the thread goes into
> a wait-state the TIP is dispatched !! does the wait-state vanish or does
> running of TIP have to cause the wait-state to be changed back to
> run-state ? imperatively ?

No. If the thread is in a Wait state it transitions to a Ready state
where the scheduler will eventually Run it.
If already in a Ready or Run state it remains there.

Additionally if a TIP is posted to a thread that is currently running
on a different core, the poster sends an inter-processor interrupt to
cause that other core to transition into an IPI interrupt handler.
Return From Interrupt checks for pending unmasked TIPS and re-routes
through the TIP delivery, thereby ensuring interrupt delivery semantics.

> <
> Does the completion of 1 TIP immediately cause the next TIP on the
> thread-list to run ? or does this mechanism knock 1 off per wait-state ?

TIP delivery would loop draining the TIP queue until all were gone
as long as TIPs were unmasked, similar to having multiple devices
connected to a single priority interrupt line.

The way I do this is that if the thread was originally waiting for,
say, an IO to complete when a TIP arrives then it wakes up the
thread so it can run and process the TIP.

When all TIPs are flushed it returns to what it was previously doing
and checks if the thread wait state completed while it was away.
If not then it waits again.
If yes then it continues on its way.

On 28 Oct 2021, George Neuner wrote
(in article<tanknght82p32476bocsneeaa985mgersh@4ax.com>):

> On Mon, 25 Oct 2021 08:53:31 +0200, Bernd Linsel
> <bl1-removethis@gmx.com> wrote:
>
> > On 24.10.2021 19:38, MitchAlsup wrote:
> > > On Sunday, October 24, 2021 at 11:31:59 AM UTC-5, Stefan Monnier wrote:
> > > <
> > > PL/1 had areas. If you deallocated an area, everything you allocated in
> > > the area
> > > "went away".
> >
> > Original Pascal: Mark(b); ... New(p); ... Dispose(b);
>
> Umm. My (perhaps faulty) recollection was that it was mark/release,
> and that it was not in Wirth's Pascal originally, but rather was an
> extension introduced by UCSD and then copied by others ... notably
> Turbo.

Correct.

But truly original Pascal had "class" types, which were locally
declarable areas within which new allocated storage on demand.
Pointers were bound to classes, not to the types pointed-to.
A class was completely deallocated on exit from its scope.
So, somewhat like PL/1 areas.

--
Bill Findlay

On 28/10/2021 20:11, Niklas Holsti wrote:
> On 2021-10-28 12:18, David Brown wrote:
>> On 27/10/2021 18:31, Stephen Fuld wrote:
>>> On 10/26/2021 11:38 PM, David Brown wrote:
>>>> On 27/10/2021 03:28, Tim Rentsch wrote:
>>>>> Niklas Holsti <niklas.holsti@tidorum.invalid> writes:
>>>>>
>>>>>>> [.. volatile ..]
>>>>>>
>>>>>> These discussions of volatile, and execution order wrt timing,
>>>>>> gave me an idea:  perhaps C (and other languages) should allow
>>>>>> marking functions (subprograms) as "volatile", with the meaning
>>>>>> that all of the effects of a call of that function (including use
>>>>>> of processor time) should be ordered as volatile accesses are
>>>>>> ordered, with respect to other volatile accesses.
>>>>>>
>>>>>> For example, if x and y are volatile variables, and foo is a
>>>>>> volatile function, then in this code
>>>>>>
>>>>>>      x = 1;
>>>>>>      foo ();
>>>>>>      y = 1;
>>>>>>
>>>>>> we would be sure that all effects and all dynamic resource usage
>>>>>> of the foo() call would occur between the assignments to x and to
>>>>>> y.
>>>>>>
>>>>>> A more flexible approach would be to mark selected function calls
>>>>>> as volatile, in the same way that C allows in-line use of
>>>>>> pointer-to-volatile to force a volatile access to an object that
>>>>>> is not itself marked volatile.  Something like:
>>>>>>
>>>>>>      x = 1;
>>>>>>      (volatile) foo ();
>>>>>>      y = 1;
>>>>>>
>>>>>> Are volatile functions and/or volatile function calls a good idea?
>>>>>
>>>>> Let me propose a simpler mechanism that I believe does a better
>>>>> job of what (I think) it is you want to do.  By way of example:
>>>>>
>>>>>       * (_Volatile int *) &x = 1;
>>>>>       foo();
>>>>>       * (_Volatile int *) &y = 1;
>>>>>
>>>>> The semantics of the new _Volatile qualifier, speaking
>>>>> informally, is that it imposes a sequence point in the actual
>>>>> machine, not just in the abstract machine.  So all logically
>>>>> previous evaluations must be finished before a volatile access,
>>>>> and after a volatile access all logically subsequent evaluations
>>>>> must not yet be started.  To say that another way, no expression
>>>>> evaluation (including side effects) may be "moved across" a
>>>>> read or write to a _Volatile object.
>>>>>
>>>>> Note that foo() is a call to an ordinary function, and expressions
>>>>> in foo() may be re-ordered in all the usual ways, except that they
>>>>> must not be "moved across" the assignment to x or the assignment
>>>>> to y.
>>>>>
>>>>
>>>> That does sound like a simpler mechanism for the same effect.  But
>>>> perhaps we can go further and simpler:
>>>>
>>>>      x = 1;
>>>>      _Execution_barrier();
>>>>      foo();
>>>>      _Execution_barrier();
>>>>      y = 1;
>>>>
>>>> _Execution_barrier() would be an implementation-dependent macro that
>>>> imposes a sequence point in the actual machine, in the manner described
>>>> by Tim above.  I see no advantage to making it a qualifier or tying it
>>>> to a particular lvalue access, since it acts on the entire machine.
>>>>
>>>> It would still be a big challenge (I believe) to get clear and precise
>>>> definitions for how this would work, but I think it would be easier to
>>>> define, understand and implement than "volatile functions" or a new
>>>> "_Volatile" qualifier.
>>>
>>> I think there are two separable questions here.  One is what the
>>> mechanism does.  The second is whether the mechanism should be a
>>> property of the function or of the invocation of that function.  Or
>>> equivalently, should the syntax (whatever it is) be required for each
>>> call, or should it be invoked automatically (without additional syntax)
>>> every time the function is invoked.
>>>
>>
>> Agreed.
>>
>> For the first question, I think there is still a lot of thinking to do.
>>   I suspect some people will be of the opinion that they already know the
>> answer, along the lines of "it means the code should do what the
>> programmer wrote", but that is too vague to be used as a specification.
>>
>> For the second question, my thought was that _Execution_barrier() would
>> give you all the options.  You can use it stand-alone in code, or
>> surrounding a function call.  And if the function is supposed to be a
>> "volatile function" that is always handled this way, put barriers as the
>> first and last statements of the function.  But I think that in the end,
>> the capabilities of all the suggestions here are the same - it's just a
>> question of where you put the new feature in the source code.
>
>
> Mainly so, but I now realize that the _Execution_barrier method goes
> beyond my suggestion of "volatile function". Both methods prevent the
> code of a (possibly in-lined) foo() call from being moved past the
> barriers, but only the _Execution_barrier method prevents moving other
> code into the same barrier-delimited region where the foo() call (or
> in-lined code) lies.
>
> In the "volatile function" method, to prevent such motion of other code,
> that other code would also have to be defined as a volatile function, or
> a volatile block. That could of course be done (and rather easily if a
> volatile block is used) but it would be more verbose than the
> _Execution_barrier method.
>
>
>> However, I have now thought of a /big/ problem with all this.  At the
>> moment, when code calls "foo()" (whose source is unknown to the
>> compiler), the compiler can must assume that foo might access any bit of
>> memory, or use volatiles, etc., and this puts certain restrictions on
>> the optimiser.  It can't move "global" reads and writes across the
>> function call, for example.  But it can also assume that foo does not
>> have any undefined behaviour - it can assume foo does not "cheat".  So
>> it can assume that "foo" follows the ABI with respect to registers it
>> preserves, how it accesses the stack, how it interacts with static
>> variables that don't escape, and so on.  If there is a possibility that
>> the external function has such "do as I say despite the UB" behaviour
>> that the compiler can't keep these assumptions, then the compiler will
>> be crippled and the language efficiency at least halved.  It won't
>> matter if this is due to an _Execution_barrier, volatile function, or
>> _Volatile qualifier.
>
>
> I don't understand that at all. I don't see any connection between
> undefined behaviour and the _Execution_barrier or volatile function
> proposals. These proposals bring no new semantics in terms of the C
> virtual machine; they only change the pragmatical, non-functional
> properties of the code, such as ordering and timing. In other words,
> they try to define, to some practical extent, behaviour that is not
> specified in the C standard (specifically, the order of computations),
> but where every possible behaviour is equally valid from the C
> standard's point of view, and equivalent in terms of the observable
> behaviour of the C program (as currently defined, that is, without
> considering the order and timing of computations as observable effects).
>
> I also don't see why limiting the compiler's code-reordering ability
> would tempt the compiler into violating the ABI.

I may have been mixing up branches in this unwieldy and overgrown
thread. But there have been a lot of calls for the compiler to stop
assuming that UB does not occur, and have some way to force it to "do
exactly what the programmer writes" in some sense. I am pointing out
some of the problems with that concept - showing that there is no way
(and certainly no practical and efficient way) for a compiler to /avoid/
"optimising on the assumption that UB does not occur". The best that
can be done, for those that dislike some aspects of C, is to give
specific behaviours to specific types of UB.

Bill Findlay <findlaybill@blueyonder.co.uk> writes:
>But truly original Pascal had "class" types, which were locally
>declarable areas within which new allocated storage on demand.
>Pointers were bound to classes, not to the types pointed-to.
>A class was completely deallocated on exit from its scope.
>So, somewhat like PL/1 areas.

Interesting. Makes me wonder why per-object explicit memory
allocation (new/dispose in Pascal, malloc()/free() in C) won out over
region-based memory allocation.

- anton
--
'Anyone trying for "industrial quality" ISA should avoid undefined behavior.'
Mitch Alsup, <c17fcd89-f024-40e7-a594-88a85ac10d20o@googlegroups.com>

On 27/10/2021 23:11, Anton Ertl wrote:
> Thomas Koenig <tkoenig@netcologne.de> writes:
>> Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>> David Brown <david.brown@hesbynett.no> writes:
>>>> C compilers are almost always better at optimising code written in clear
>>>> and standard ways - they compiler can get more information from such
>>>> code, and the compiler developers put most effort into working with
>>>> common code constructs. When people "manually optimise" their C code,
>>>> such as with these labels-as-value arrays and hand-written jump tables,
>>>> it's difficult for a compiler to see what is happening, and difficult to
>>>> optimise - normal optimisation passes may regress performance.
>>>
>>> Jon Bentley wrote "Writing Efficient Programs" in 1982, and he gave an
>>> example in Pascal. I transliterated them to C IIRC in 1999 and you
>>> can find the results at
>>> <http://www.complang.tuwien.ac.at/anton/lvas/effizienz/tsp.html>.
>>>
>>> Last I looked, the manual optimization steps that Jon Bentley
>>> described in 1982 still produced a speedup factor of >2.7,
>>
>> So, which of these are actually permitted according to the C
>> semantics?
>
> Judge for yourself, the code is accessible through this link. The
> step from tsp6.c to tsp8.c certainly changes the semantics that the C
> compiler sees (it changes the output).
>
> But wrt to the claims made by David Brown, tsp1.c certainly looks
> pretty clear to me, and I think it's what David Brown considers
> standard, someone who believes him might come to the expectation that
> tsp1.c would perform best when compiled with optimization with a
> modern compiler. I doesn't.
>

That's an unusual misinterpretation of what I have written.

Look at what I wrote: "C compilers are almost always better at
optimising code written in clear and standard ways". That is true.
What some guy long ago did with a Pascal program, and how you might have
copied it into C, is completely irrelevant.

One thing compilers do not do (and should not do) is change the results
of your code. That puts limits on how they can rearrange things. They
are also not (yet!) smart enough to do much in the way of algorithmic
rearrangements, except specific cases programmed by their creators. So
a compiler cannot be expected to figure out that you can work with the
square of the distances instead of the distances.

However, you can write code clearly with separate functions, and let the
compiler do the inlining - manual inlining is not needed. If you don't
need full IEEE compatibility (and who does? Only a tiny fraction of
programs actually need it) and enable "-ffast-math", then you don't need
to do manual re-arrangement of floating point code - the compiler will
do it.

The whole point of using a compiler that does the donkey work of
re-arrangements and optimisations is so that the programmer can
concentrate on the high-level algorithmic work - and these often make a
big difference.

On 28/10/2021 11:41, David Brown wrote:
> On 27/10/2021 19:34, Anton Ertl wrote:

>> Interestingly, the differences exist more for the more optimized
>> programs. Read all about it in Section 4.2 of
>> <https://www.complang.tuwien.ac.at/kps2015/proceedings/KPS_2015_submission_29.pdf>.
>>

>
> I am highly unlikely to have the time to work on your challenge, but I
> will have a read of the links you sent when I get the chance. It might
> be a while before I do, since they will take time to do them justice -
> thus I am posting to thank you for them in advance.
>

OK, I have now looked at that paper. Was it supposed to be some kind of
academic paper? It's a load of opinionated, angry, subjective,
unjustified and invalid nonsense with a few interesting and valid points
hidden under the chaff. Sorry, but that is my opinionated and objective
view of it. You've written a paper that pretends to be looking at
real-world challenges and problems in C compilers but starts by calling
major compiler developers "clueless", "adversarial", and interested only
in the performance of their tools on SPEC CPU and related benchmarks.

Don't you know the basics of writing such papers? How about presenting
some facts and data? Defining terms objectively, rather than with
confusing and deliberately provocative terms for your imagined
variations of the C language? Perhaps looking at the alternative
viewpoints - collecting the information /before/ you reach your conclusions?

Really, it doesn't even matter if you are right or wrong - when you you
act like a fanatic, people ignore you. The appropriate response from
the people who could actually make a difference in the C world, such as
the gcc and clang developers and the C standards committee, would be to
stop reading the paper after the first couple of pages and throw it out.

The paper has references taken out of context or designed to give the
incorrect impression. For example, on page 4 we have:

"""
The GCC maintainers subsequently disabled this optimization for the case
occuring in SPEC, demonstrating that, inofficially, GCC supports Cbench,
not “C”.
"""

The associated footnote refers to a gcc bug report which you presumably
hope readers of your paper will not bother checking. If they do, they
will see that the reporter of the "bug" thanks the gcc developers for
helping him find the error in his code, and that report is marked as
"invalid" - not a bug in the compiler.

Eventually, I gave up reading the rest of the paper too. My critique
here should not be considered complete.

Let's look at some of the /real/ conclusions that can be drawn from all
this:

People make mistakes when writing C code. It happens - they make
mistakes for /all/ programming languages, and C is certainly not the
easiest language to write. Writing C that is reasonably portable and
highly efficient is particularly hard.

Strangely written C code with unusual and specialised needs can be a
poor fit for compilers and optimisations targeting more general code.
Making such code work well with a variety of C compilers is hard.

C compilers are written primarily to implement the C language described
by the international C standards. They are not designed to implement
C*, C², "C", C-ertl, or whatever confusing new concept you are inventing
next.

Most optimisations in C compilers only help in a small number of cases,
and some will occasionally make code slower. It's the sum overall that
has an effect.

There is scope for making a standardised way to make certain particular
aspects of C that are currently undefined, into defined behaviour for
the programmers and programs that might benefit from them. Other
programmers and programs can keep the benefits of the current
interpretations, however minor those might be.

There is always scope for compilers to be better at providing
information to help programmers find their mistakes.

Writing aggressive, provocative, accusing and fanatic complaints is a
good way to ensure that the people who can implement the changes you
want, will ignore you.

Am I missing anything important?

Anton Ertl wrote:
> Bill Findlay <findlaybill@blueyonder.co.uk> writes:
>> But truly original Pascal had "class" types, which were locally
>> declarable areas within which new allocated storage on demand.
>> Pointers were bound to classes, not to the types pointed-to.
>> A class was completely deallocated on exit from its scope.
>> So, somewhat like PL/1 areas.
>
> Interesting. Makes me wonder why per-object explicit memory
> allocation (new/dispose in Pascal, malloc()/free() in C) won out over
> region-based memory allocation.
>
> - anton

If it only recovered the whole heap when you leave the class/heap
declaration scope then if the scope is whole-program it would never
automatically recover and you have to manually free anyway.

On 29 Oct 2021, Anton Ertl wrote
(in article<2021Oct29.115542@mips.complang.tuwien.ac.at>):

> Bill Findlay<findlaybill@blueyonder.co.uk> writes:
> > But truly original Pascal had "class" types, which were locally
> > declarable areas within which new allocated storage on demand.
> > Pointers were bound to classes, not to the types pointed-to.
> > A class was completely deallocated on exit from its scope.
> > So, somewhat like PL/1 areas.
>
> Interesting. Makes me wonder why per-object explicit memory
> allocation (new/dispose in Pascal, malloc()/free() in C) won out over
> region-based memory allocation.
>
> - anton

Flexibility?

--
Bill Findlay

On Friday, October 29, 2021 at 6:29:34 PM UTC-5, Bill Findlay wrote:
> On 29 Oct 2021, Anton Ertl wrote
> (in article<2021Oct2...@mips.complang.tuwien.ac.at>):
> > Bill Findlay<findl...@blueyonder.co.uk> writes:
> > > But truly original Pascal had "class" types, which were locally
> > > declarable areas within which new allocated storage on demand.
> > > Pointers were bound to classes, not to the types pointed-to.
> > > A class was completely deallocated on exit from its scope.
> > > So, somewhat like PL/1 areas.
> >
> > Interesting. Makes me wonder why per-object explicit memory
> > allocation (new/dispose in Pascal, malloc()/free() in C) won out over
> > region-based memory allocation.
<
In the DPD-11 time frame, memory was so small, the std way of allocating
and deallocating memory was to fork a task and pipe back the stuff that
you wanted to keep. Letting cleanup happen because of exit();
<
It is still a good model of a LOT of stuff.
> >
> > - anton
> Flexibility?
>
> --
> Bill Findlay

On Friday, October 29, 2021 at 6:51:29 PM UTC-5, MitchAlsup wrote:
> On Friday, October 29, 2021 at 6:29:34 PM UTC-5, Bill Findlay wrote:
> > On 29 Oct 2021, Anton Ertl wrote
> > (in article<2021Oct2...@mips.complang.tuwien.ac.at>):
> > > Bill Findlay<findl...@blueyonder.co.uk> writes:
> > > > But truly original Pascal had "class" types, which were locally
> > > > declarable areas within which new allocated storage on demand.
> > > > Pointers were bound to classes, not to the types pointed-to.
> > > > A class was completely deallocated on exit from its scope.
> > > > So, somewhat like PL/1 areas.
> > >
> > > Interesting. Makes me wonder why per-object explicit memory
> > > allocation (new/dispose in Pascal, malloc()/free() in C) won out over
> > > region-based memory allocation.
> <
> In the DPD-11 time frame, memory was so small, the std way of allocating
> and deallocating memory was to fork a task and pipe back the stuff that
> you wanted to keep. Letting cleanup happen because of exit();
> <
> It is still a good model of a LOT of stuff.
> > >
> > > - anton
> > Flexibility?
> >
> > --
> > Bill Findlay

|>in the DPD-11 time frame memory was small
In Fortran one could have a large array for work space and pass portions of it to a subroutine by subscripting
The subroutine saw several arrays of appropriate sizes and on return they were automatically "deallocated".
Ugh, one needed to be comfortable with lengthy subscript expressions.

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> Thomas Koenig <tkoenig@netcologne.de> writes:
>>Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>> Thomas Koenig <tkoenig@netcologne.de> writes:
>>>>-fno-reorder-blocks puts the compiler back before the very first
>>>>FORTRAN compiler, which invented basic blocks and moving them
>>>>around for optimization.
> ...
>>> In any case, your implied claim that
>>> -fno-reorder-blocks generally hurts performance is debunked.
>>
>>You have a habit of reading things into my posts that I did not
>>write, and then refuting them.
>
> Sure, you never wanted to give the impression that -fno-reorder-blocks
> generally hurts performance when you wrote:

>>>>-fno-reorder-blocks puts the compiler back before the very first
>>>>FORTRAN compiler, which invented basic blocks and moving them
>>>>around for optimization.
>

Please check what I wrote above.

I never wrote that it does so in the case of gforth, or in every
specific case.

In your arguments, there were two steps

From my remark, you extrapolated to something general I did not
say or write. And to see if that actually is a correct
description of what I meant, we have to agree what "generally"
means - if you meant "in each in every case", that would be silly.
If you mean "quite often, but with significant exceptions", then I
am fine with that.

And then, you claimed that what you inferred from my writing was
refuted from a single test case.

I have news for you: You can only refute an absolute statement
by a single example.

There's a name for that: Straw man argument. I am not sure if
you did it intentionally or not. If it was intentionally, well,
OK - then you are that kind of dishonest debater.

If it was unintentionally, you really should consider reading
what people actually write and replying to that, instead of
what you think they might have in the back of their heads, and
then turning that into nonsense and refuting that nonsense.

>>Additionally, your grasp of compiler
>>optimzation seems rather weak.
>
> This statement says more about you than me.

What it says about me is that I now understand you much better
where you come from. Your insistence above that any compiler
optimzation should help in each and every case is ridiculous,
as you should know.

>>I can understand, however, that "goto variable" can be faster
>>because of inefficiencies in code generation for the switch
>>statement. That can be tricky, and I have submitted at least
>>one still open bug report about that.
>>
>>If this were my project, my solution would be to develop standard
>>C constructs as an alternate, identify bottlenecks in speed and
>>submit missed-optimization PRs for them.
>>
>>So, a proposal: Do you have a piece of code which works appreciably
>>faster using "goto variable" rather than using switch and goto?
>
> Sure, Gforth. And it not only works faster because of the goto part
> of labels-as-values, but also because of the dynamic native-code
> generation which is also based on labels-as-values.

Not a viable test case, it needs to be smaller.

>
>>If you have it, I can submit a PR for it, if you want to.
>
> I could not care less.

Fine by me. If you do not want to have the problem fixed, at
least not long-term, but would prefer to rail about evil compiler
writers who threaten your favourite toy, knock yourself out.

Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:

> A better approach would be to go with tail calls (see the
> "continuation-passing style" section in
><https://www.complang.tuwien.ac.at/forth/threaded-code.html#portability>),

Now that has broken other people's non-standard, ABI-violating code in
the past, so so by your standard we can't have that, can we?

These people were polite, explained their problem very well (they
even did a lot of debugging which pinpointed the problem exactly),
and a solution was developed so that the broken code in the old
versions could continue to function until the new one was deployed.

Spot the difference.