Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
> Thomas Koenig <tkoenig@netcologne.de> writes:
>>Anton Ertl <anton@mips.complang.tuwien.ac.at> schrieb:
>>
>>> My position is basically that a responsible maintainer for a
>>> widely-used compiler does not break existing source code
>>
>>Question: You mentioned tail call optimization in one of
>>your posts.
>>
>>Should compilers be mandated to copy arguments for
>>
>>int bar (int a, int b, int c);
>>
>>int foo(int a, int b, int c)
>>{
>> if (a > 0)
>> return bar(a,b,c);
>> else
>> return -1;
>>}
>
> I don't see why. I also don't see the connection to tail-call
> optimization. On AMD64 gcc -O does not copy arguments, whether you
> enable tail-call optimization in this case (and in this case it
> happens to work) with -foptimize-sibling-calls or not.

Now, add a few arguments to they have to be passed on the stack.

Add code which (in an inter-language setting) lies about the
presence of an argument by not passing it because it is never
accessed.

In other words, the callee expects that there is an argument
on the stack, which the caller never supplied, in violation
of the callee's ABI.

The callee then does a sibling call, based on the assumption that
that argument is there and that the stack space for that argument
is there.

Also assume that the callee language actually had its ABI wrong
because it assumed that it was called via varargs, which it was not.

This is usually harmless on x86_64, because all that means is that
an extra register is set, but on other platforms, this is grave
(like subroutines with too many arguments not working).

However, all the world's not a VAX, and the ABI bug was fixed,
and caller and callee in that particular language finally agreed,
both are now like non-varargs C calls.

Everything great, right? Well, except that the people who had
violated the ABI all along on x86_64 suddenly started seeing
failures because of sibling calls.

So, should the ABI bug not have been fixed, in your opinion?

Thomas Koenig wrote:
> MitchAlsup <MitchAlsup@aol.com> schrieb:
>
>> From the comp.arch layer of understanding::
>> <
>> What is it that ISAs can do to minimize the space of UB ?
>
> We have discussed part of this before, but here is a bit
> of an expansion (and some repetition).
>
> There are tools to find undefined behavior (or erroneous programs),
> ranging from sanitizers (which add a low factor to execution time,
> maybe two or so) to really heavy-handed tools like valgrind which
> find a _lot_ of errors while making the code run abysmally slow.
>
> Think of ways that an ISA could be designed to speed up those tests.
> Zero overhead is almost certainly unreachable, but even reducing
> the speed overhead of checked code would help a lot.
>
> Going through the CERT coding standards, look for what can
> be implemented either efficiently or at zero cost. An example:
>
> 5.1 INT30-C. Ensure that unsigned integer operations do not wrap.
>
> Integer values must not be allowed to wrap, especially if they
> are used in any of the following ways:
>
> * Integer operands of any pointer arithmetic, including array
> indexing

The scaled index address mode [rBase + rIndex*scale + imm]
must not trap-on-wrap by default. As an _optional_ check it is fine.

For languages with non base-0 arrays, using a biased-base array
pointer saves continuously subtracting the lower array index.
The initial base bias is allowed to wrap because it will wrap back again
when the array is indexed, and the index should have been range checked.

So in this case address arithmetic is modulo.
If the array index is a signed integer, it is sign extended to size
intptr_t, changed to unsptr_t, scaled and added modulo (unchecked).

If it was going to do checked [rBase + rIndex*scale + imm] address
arithmetic, then the scaling of rIndex register should check that
no significant digits are shifted out, which requires knowing if
rIndex is signed or unsigned.
To check the sum we need to know if addresses are signed or unsigned.
Personally I consider addresses unsigned because there is no reason
a buffer cannot straddle the 7FFF... 8000... boundary.
However others consider address space as signed.

So that would be 5 address mode variants, 1 unchecked and up to 4 checked,
times all the other address mode variants.

I'd say it's better to just check the rIndex range and
leave address calculations out of it.

On Friday, November 5, 2021 at 12:31:07 PM UTC-5, EricP wrote:
> Thomas Koenig wrote:
> > MitchAlsup <Mitch...@aol.com> schrieb:
> >
> >> From the comp.arch layer of understanding::
> >> <
> >> What is it that ISAs can do to minimize the space of UB ?
> >
> > We have discussed part of this before, but here is a bit
> > of an expansion (and some repetition).
> >
> > There are tools to find undefined behavior (or erroneous programs),
> > ranging from sanitizers (which add a low factor to execution time,
> > maybe two or so) to really heavy-handed tools like valgrind which
> > find a _lot_ of errors while making the code run abysmally slow.
> >
> > Think of ways that an ISA could be designed to speed up those tests.
> > Zero overhead is almost certainly unreachable, but even reducing
> > the speed overhead of checked code would help a lot.
> >
> > Going through the CERT coding standards, look for what can
> > be implemented either efficiently or at zero cost. An example:
> >
> > 5.1 INT30-C. Ensure that unsigned integer operations do not wrap.
> >
> > Integer values must not be allowed to wrap, especially if they
> > are used in any of the following ways:
> >
> > * Integer operands of any pointer arithmetic, including array
> > indexing
<
> The scaled index address mode [rBase + rIndex*scale + imm]
> must not trap-on-wrap by default. As an _optional_ check it is fine.
>
> For languages with non base-0 arrays, using a biased-base array
> pointer saves continuously subtracting the lower array index.
> The initial base bias is allowed to wrap because it will wrap back again
> when the array is indexed, and the index should have been range checked.
<
So, what about the case where the virtual address space is 63-bits
(not 64-bits) and one does NOT want AGEN to change the HoB of
Rbase+immed when adding Rindex<<scale to it ? An array for an
application is not allowed to span the 2^63 boundary.
<
That is, the HoB of the AGENed address tells the HW about what address
space this access accesses. You do not want AGEN to change the virtual
address space of access.
<
Now, should you actually WANT to change the VaS, this is easily done
with ADD, OR, XOR outside of AGEN influences; generally of the unsigned
variety.
<
>
> So in this case address arithmetic is modulo.
<
I would argue that parts of AGEN are modulo and parts are not.
<
> If the array index is a signed integer, it is sign extended to size
> intptr_t, changed to unsptr_t, scaled and added modulo (unchecked).
<
But should address arithmetic be able to change the virtual address space
the access accesses ??
>
> If it was going to do checked [rBase + rIndex*scale + imm] address
> arithmetic, then the scaling of rIndex register should check that
> no significant digits are shifted out, which requires knowing if
> rIndex is signed or unsigned.
<
That seems fair. How about including VaS[63] too ?
<
> To check the sum we need to know if addresses are signed or unsigned.
> Personally I consider addresses unsigned because there is no reason
> a buffer cannot straddle the 7FFF... 8000... boundary.
> However others consider address space as signed.
<
There are arguments on each side.
7FFF-8000 is fine
7FFFFFFF-80000000 is fine
3FFFFFFFFFFFFFFF-4000000000000000 is fine
7FFFFFFFFFFFFFFF-8000000000000000 is questionable.
<
However, I do not think that an array or buffer should be able to span
0X7FFFFFFFFFFFFFFF to 0x8000000000000000 especially if/when
the later address is in a fundamentally different VaS than the former.
In many circumstances the OS is up in 0x8000000000000000 while
the application is down in x07FFFFFFFFFFFFFFF.
>
> So that would be 5 address mode variants, 1 unchecked and up to 4 checked,
> times all the other address mode variants.
<
Rbase+immed should determine the HoB of the address this unit of
arithmetic wraps. Rindex<<scale should not cause Rbase+immed
to wrap (or alter VaS[63]).
<
>
> I'd say it's better to just check the rIndex range and
> leave address calculations out of it.
<
We agree, there.

MitchAlsup <MitchAlsup@aol.com> writes:

> For a new ISA, why would anyone go with I32LP64 why not ILP64 ?

If 'int' is 64 bits, then a 32-bit unsigned type will end up
using signed arithmetic for +, -, etc (unless there are casts or
some other decorations to sidestep the problem).

I would rather 'int' be only 32 bits, which gives a natural
32-bit unsigned type that acts like unsigned types ought
to act. The alternative is ugly and more error-prone code.

We might blame the original C standardization group for deciding
to endorse the so-called "value preserving" promotion semantics,
but whoever is to blame we are stuck with it now. To me 32 bits
is a natural width for 'int' and 'unsigned int'; if a wider type
is wanted then 'long' and 'unsigned long' being 64 bits can
provide that.

Niklas Holsti <niklas.holsti@tidorum.invalid> writes:

> On 2021-11-04 19:58, Tim Rentsch wrote:
>
>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>
>>> Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
>>>
>>>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>>
>>>>> Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
>>>>>
>>>>>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>>>>
>>>>>>> Standard C is such a programming language; it only specifies
>>>>>>> how "strictly conforming programs" should behave, [...]
>>>>>>
>>>>>> That statement is patently false.
>>>>>
>>>>> Please tell us where the C standard specifies how, e.g., the
>>>>> following program should behave:
>>>>>
>>>>> int main() {
>>>>> return 0;
>>>>> }
>>>>
>>>> First let me ask a question. Do you think there's a difference
>>>> between that program and this one:
>>>>
>>>> int main(void) {
>>>> return 0;
>>>> }
>>>
>>> I would not expect a difference, but in any case, the arguments of
>>> main() are not the point of this example. Choose whatever arguments
>>> you want to provide an answer for. The result is not a "strictly
>>> conforming program" in any case, because it terminates.
>>
>> What makes you think strictly conforming programs cannot
>> terminate?
>
> Anton has not yet answered that, but I suspect that the reason
> is that a [he considers implementation-defined return status
> prevents a program from being strictly conforming].

Check. Sometimes I have my own suspicions, but I try to keep
them as just suspicions and not turn them into assumptions.

> However, IMO, returning to the host environment is not "producing
> output", so IMO it cannot violate the strict conformance rules.
> [...]

You're in good company.

MitchAlsup wrote:
> On Friday, November 5, 2021 at 12:31:07 PM UTC-5, EricP wrote:
>> Thomas Koenig wrote:
>>> MitchAlsup <Mitch...@aol.com> schrieb:
>>>
>>>> From the comp.arch layer of understanding::
>>>> <
>>>> What is it that ISAs can do to minimize the space of UB ?
>>> We have discussed part of this before, but here is a bit
>>> of an expansion (and some repetition).
>>>
>>> There are tools to find undefined behavior (or erroneous programs),
>>> ranging from sanitizers (which add a low factor to execution time,
>>> maybe two or so) to really heavy-handed tools like valgrind which
>>> find a _lot_ of errors while making the code run abysmally slow.
>>>
>>> Think of ways that an ISA could be designed to speed up those tests.
>>> Zero overhead is almost certainly unreachable, but even reducing
>>> the speed overhead of checked code would help a lot.
>>>
>>> Going through the CERT coding standards, look for what can
>>> be implemented either efficiently or at zero cost. An example:
>>>
>>> 5.1 INT30-C. Ensure that unsigned integer operations do not wrap.
>>>
>>> Integer values must not be allowed to wrap, especially if they
>>> are used in any of the following ways:
>>>
>>> * Integer operands of any pointer arithmetic, including array
>>> indexing
> <
>> The scaled index address mode [rBase + rIndex*scale + imm]
>> must not trap-on-wrap by default. As an _optional_ check it is fine.
>>
>> For languages with non base-0 arrays, using a biased-base array
>> pointer saves continuously subtracting the lower array index.
>> The initial base bias is allowed to wrap because it will wrap back again
>> when the array is indexed, and the index should have been range checked.
> <
> So, what about the case where the virtual address space is 63-bits
> (not 64-bits) and one does NOT want AGEN to change the HoB of
> Rbase+immed when adding Rindex<<scale to it ? An array for an
> application is not allowed to span the 2^63 boundary.

That is why using detection of a transition across some arbitrary
address boundary is not a good indicator for errors.
Better to do this with actual accesses and the MMU.

> <
> That is, the HoB of the AGENed address tells the HW about what address
> space this access accesses. You do not want AGEN to change the virtual
> address space of access.

Right but don't define a _change_ to a bit as the error detector.
What if code used immediate constant as an absolute address?
Then there is no address calculation, no bit changes,
but the access is just as legal or illegal.

The MMU is the right place for all of this.

> <
> Now, should you actually WANT to change the VaS, this is easily done
> with ADD, OR, XOR outside of AGEN influences; generally of the unsigned
> variety.
> <
>> So in this case address arithmetic is modulo.
> <
> I would argue that parts of AGEN are modulo and parts are not.

It only matters if one wants to check address calculations and I don't.

> <
>> If the array index is a signed integer, it is sign extended to size
>> intptr_t, changed to unsptr_t, scaled and added modulo (unchecked).
> <
> But should address arithmetic be able to change the virtual address space
> the access accesses ??

The virtual address space is dynamically defined by the program and OS.
The ISA should not make assumptions about its specifics.
The MMU can change from model to model.

>> If it was going to do checked [rBase + rIndex*scale + imm] address
>> arithmetic, then the scaling of rIndex register should check that
>> no significant digits are shifted out, which requires knowing if
>> rIndex is signed or unsigned.
> <
> That seems fair. How about including VaS[63] too ?

I _dont_ want to check address calculations for wrap.
I was just walking through my reasons why,
what trying to do so would do to an ISA.

> <
>> To check the sum we need to know if addresses are signed or unsigned.
>> Personally I consider addresses unsigned because there is no reason
>> a buffer cannot straddle the 7FFF... 8000... boundary.
>> However others consider address space as signed.
> <
> There are arguments on each side.
> 7FFF-8000 is fine
> 7FFFFFFF-80000000 is fine
> 3FFFFFFFFFFFFFFF-4000000000000000 is fine
> 7FFFFFFFFFFFFFFF-8000000000000000 is questionable.
> <
> However, I do not think that an array or buffer should be able to span
> 0X7FFFFFFFFFFFFFFF to 0x8000000000000000 especially if/when
> the later address is in a fundamentally different VaS than the former.
> In many circumstances the OS is up in 0x8000000000000000 while
> the application is down in x07FFFFFFFFFFFFFFF.

Note that on a 64-bit ISA the OS will occupy a tiny fraction of the
unsigned high virtual address range so there is nothing special about
the 7FFFFFFFFFFFFFFF-8000000000000000 address boundary,
just what is configured in the MMU.

>> So that would be 5 address mode variants, 1 unchecked and up to 4 checked,
>> times all the other address mode variants.
> <
> Rbase+immed should determine the HoB of the address this unit of
> arithmetic wraps. Rindex<<scale should not cause Rbase+immed
> to wrap (or alter VaS[63]).
> <
>> I'd say it's better to just check the rIndex range and
>> leave address calculations out of it.
> <
> We agree, there.

On Saturday, November 6, 2021 at 10:02:11 AM UTC-5, EricP wrote:
> MitchAlsup wrote:
> > On Friday, November 5, 2021 at 12:31:07 PM UTC-5, EricP wrote:
> >> Thomas Koenig wrote:
> >>> MitchAlsup <Mitch...@aol.com> schrieb:
> >>>
> >>>> From the comp.arch layer of understanding::
> >>>> <
> >>>> What is it that ISAs can do to minimize the space of UB ?
> >>> We have discussed part of this before, but here is a bit
> >>> of an expansion (and some repetition).
> >>>
> >>> There are tools to find undefined behavior (or erroneous programs),
> >>> ranging from sanitizers (which add a low factor to execution time,
> >>> maybe two or so) to really heavy-handed tools like valgrind which
> >>> find a _lot_ of errors while making the code run abysmally slow.
> >>>
> >>> Think of ways that an ISA could be designed to speed up those tests.
> >>> Zero overhead is almost certainly unreachable, but even reducing
> >>> the speed overhead of checked code would help a lot.
> >>>
> >>> Going through the CERT coding standards, look for what can
> >>> be implemented either efficiently or at zero cost. An example:
> >>>
> >>> 5.1 INT30-C. Ensure that unsigned integer operations do not wrap.
> >>>
> >>> Integer values must not be allowed to wrap, especially if they
> >>> are used in any of the following ways:
> >>>
> >>> * Integer operands of any pointer arithmetic, including array
> >>> indexing
> > <
> >> The scaled index address mode [rBase + rIndex*scale + imm]
> >> must not trap-on-wrap by default. As an _optional_ check it is fine.
> >>
> >> For languages with non base-0 arrays, using a biased-base array
> >> pointer saves continuously subtracting the lower array index.
> >> The initial base bias is allowed to wrap because it will wrap back again
> >> when the array is indexed, and the index should have been range checked.
> > <
> > So, what about the case where the virtual address space is 63-bits
> > (not 64-bits) and one does NOT want AGEN to change the HoB of
> > Rbase+immed when adding Rindex<<scale to it ? An array for an
> > application is not allowed to span the 2^63 boundary.
<
> That is why using detection of a transition across some arbitrary
> address boundary is not a good indicator for errors.
> Better to do this with actual accesses and the MMU.
<
But the damage has already been done by the time the VA gets to the MMU.
> > <
> > That is, the HoB of the AGENed address tells the HW about what address
> > space this access accesses. You do not want AGEN to change the virtual
> > address space of access.
<
> Right but don't define a _change_ to a bit as the error detector.
> What if code used immediate constant as an absolute address?
<
That is why I phrased my responses in the form of Rbase+imm
That one can change the HoBs. Adding Rindex<<scale is not
allowed to change HOBs.
<
> Then there is no address calculation, no bit changes,
> but the access is just as legal or illegal.
<
Yes, but you went out of your way to get there.
>
> The MMU is the right place for all of this.
<
Once again, the MMU only sees the VA, by this tmie the damage has
already been done.
<
> > <
> > Now, should you actually WANT to change the VaS, this is easily done
> > with ADD, OR, XOR outside of AGEN influences; generally of the unsigned
> > variety.
> > <
> >> So in this case address arithmetic is modulo.
> > <
> > I would argue that parts of AGEN are modulo and parts are not.
<
> It only matters if one wants to check address calculations and I don't.
> > <
> >> If the array index is a signed integer, it is sign extended to size
> >> intptr_t, changed to unsptr_t, scaled and added modulo (unchecked).
> > <
> > But should address arithmetic be able to change the virtual address space
> > the access accesses ??
<
> The virtual address space is dynamically defined by the program and OS.
> The ISA should not make assumptions about its specifics.
> The MMU can change from model to model.
<
There are a number of 63-bit virtual address space all addressable from
a thread. When using VA[62]==VA[63] you are accessing your won threads
VaS. When VA[62] !=VA[63] you are accessing a foreign address space
through a root pointer associated with a different thread.
<
It is my considered opinion that an array cannot span an address space.
{Nor can a struct or union or any base type.}
<
> >> If it was going to do checked [rBase + rIndex*scale + imm] address
> >> arithmetic, then the scaling of rIndex register should check that
> >> no significant digits are shifted out, which requires knowing if
> >> rIndex is signed or unsigned.
> > <
> > That seems fair. How about including VaS[63] too ?
<
> I _dont_ want to check address calculations for wrap.
> I was just walking through my reasons why,
> what trying to do so would do to an ISA.
> > <
> >> To check the sum we need to know if addresses are signed or unsigned.
> >> Personally I consider addresses unsigned because there is no reason
> >> a buffer cannot straddle the 7FFF... 8000... boundary.
> >> However others consider address space as signed.
> > <
> > There are arguments on each side.
> > 7FFF-8000 is fine
> > 7FFFFFFF-80000000 is fine
> > 3FFFFFFFFFFFFFFF-4000000000000000 is fine
> > 7FFFFFFFFFFFFFFF-8000000000000000 is questionable.
> > <
> > However, I do not think that an array or buffer should be able to span
> > 0X7FFFFFFFFFFFFFFF to 0x8000000000000000 especially if/when
> > the later address is in a fundamentally different VaS than the former.
> > In many circumstances the OS is up in 0x8000000000000000 while
> > the application is down in x07FFFFFFFFFFFFFFF.
> Note that on a 64-bit ISA the OS will occupy a tiny fraction of the
> unsigned high virtual address range so there is nothing special about
> the 7FFFFFFFFFFFFFFF-8000000000000000 address boundary,
<
Neither the user nor the OS span that boundary. The user stays "down below"
and the OS stays "up above".
<
> just what is configured in the MMU.
> >> So that would be 5 address mode variants, 1 unchecked and up to 4 checked,
> >> times all the other address mode variants.
> > <
> > Rbase+immed should determine the HoB of the address this unit of
> > arithmetic wraps. Rindex<<scale should not cause Rbase+immed
> > to wrap (or alter VaS[63]).
> > <
> >> I'd say it's better to just check the rIndex range and
> >> leave address calculations out of it.
> > <
> > We agree, there.

anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:

> Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
>
>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>
>>> Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
>>>
>>>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>>
>>>>> Tim Rentsch <tr.17687@z991.linuxsc.com> writes:
>>>>>
>>>>>> anton@mips.complang.tuwien.ac.at (Anton Ertl) writes:
>>>>>>
>>>>>>> Standard C is such a programming language; it only specifies
>>>>>>> how "strictly conforming programs" should behave, [...]
>>>>>>
>>>>>> That statement is patently false.
>>>>>
>>>>> Please tell us where the C standard specifies how, e.g., the
>>>>> following program should behave:
>>>>>
>>>>> int main() {
>>>>> return 0;
>>>>> }
>>>>
>>>> First let me ask a question. Do you think there's a difference
>>>> between that program and this one:
>>>>
>>>> int main(void) {
>>>> return 0;
>>>> }
>>>
>>> I would not expect a difference, but in any case, the arguments of
>>> main() are not the point of this example. Choose whatever arguments
>>> you want to provide an answer for. The result is not a "strictly
>>> conforming program" in any case, because it terminates.
>>
>> What makes you think strictly conforming programs cannot
>> terminate?
>
> 7.22.4.4 in n1570 says:
>
> |Finally, control is returned to the host environment. If the value
> |of status is zero or EXIT_SUCCESS, an implementation-defined form
> |of the status successful termination is returned. If the value of
> |status is EXIT_FAILURE, an implementation-defined form of the
> |status unsuccessful termination is returned. Otherwise the status
> |returned is implementation-defined.
>
> All other ways to terminate C programs eventually do this.

This question has been discussed at length in comp.std.c. No one
thinks the ISO C standard means to count "return status" as being
an "output" as part of deciding whether a program is strictly
conforming. The word "output" does not appear anywhere in the
section for <stdlib.h>, where exit() et al are described. The term
"return status" is not part of a program's observable behavior,
as defined in section 5.1.2.3. By contrast, 5.1.2.3 does mention
"files", which are defined in the section for <stdio.h>. That
section is titled, rather unsurprisingly, "Input/output <stdio.h>".

Besides all that, it is simply a matter of plausibility. The
people who worked on the C standardization effort are, at least
in many cases, some pretty bright people. It seems highly
unlikely that they would go to the trouble of defining "strictly
conforming program" and then write other parts of the C standard
so it cannot ever apply. A much more likely assumption is that
"return status" is not among the "outputs" of a program, and so
has no bearing on whether any given program is or is not strictly
conforming.

If these observations don't convince you, feel free to submit a
defect report (or whatever they are called these days) to WG14,
putting the question directly to the one group who can give you
an authoritative answer on the matter.

> WRT my original statement, this means that the C standard does not
> specify how the program behaves.

Is it now your assertion that the ISO C standard does not specify
any behavior for C programs that are not strictly conforming?

> You claimed that my statement is "patently false". Please support
> your position.

Do you consider the comments you have made to be supporting your
position? Or is it only my side of the discussion that has a
burden of supporting its position?

> It seems to me that my statement is a trivial consequence of the
> definition of a "strictly conforming program",

I haven't seen any reasoning supporting this inference.

> and is therefore
> patently true, but if you have a good argument to the contrary,
> please elucidate us.

Apparently you think it's my job to convince you of something you
think is not true. It isn't.

To the best of my knowledge, every other person with whom I have
discussed the ISO C standard (and there have been a fair number of
those) agrees with my position that the C standard does specify
the behavior of C programs, both strictly conforming programs and
C programs that are not strictly conforming. That group includes
probably somewhere between one and two dozen who have served on
the WG14 committee.

Furthermore, the C standard itself directly contradicts the idea
that it does not specify the behavior of C programs. Section 1,
paragraph 1 says this in part:

This International Standard specifies the form and
establishes the interpretation of programs written in the C
programming language. It specifies

[...]

* the semantic rules for interpreting C programs;

[...]

Section 5.1.2.3, paragraph 1, says this in part:

The semantic descriptions in this International Standard
describe the behavior of an abstract machine [...]

Section 5.1.2.3, paragraph 4, says this in part:

In the abstract machine, all expressions are evaluated as
specified by the semantics.

Is it your position that you understand what the ISO C standard
does or does not specify better than do the people who wrote it?
If not, then what is your position exactly? The C standard
itself seems to say pretty directly that it does specify the
behavior of C programs. I'm at a loss to see how that can be
consistent with what you have been saying.

On Friday, November 5, 2021 at 7:18:16 PM UTC+2, Thomas Koenig wrote:
> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
> > Thomas Koenig <tko...@netcologne.de> writes:
> >>Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
> >>
> >>> My position is basically that a responsible maintainer for a
> >>> widely-used compiler does not break existing source code
> >>
> >>Question: You mentioned tail call optimization in one of
> >>your posts.
> >>
> >>Should compilers be mandated to copy arguments for
> >>
> >>int bar (int a, int b, int c);
> >>
> >>int foo(int a, int b, int c)
> >>{
> >> if (a > 0)
> >> return bar(a,b,c);
> >> else
> >> return -1;
> >>}
> >
> > I don't see why. I also don't see the connection to tail-call
> > optimization. On AMD64 gcc -O does not copy arguments, whether you
> > enable tail-call optimization in this case (and in this case it
> > happens to work) with -foptimize-sibling-calls or not.
> Now, add a few arguments to they have to be passed on the stack.
>
> Add code which (in an inter-language setting) lies about the
> presence of an argument by not passing it because it is never
> accessed.
>
> In other words, the callee expects that there is an argument
> on the stack, which the caller never supplied, in violation
> of the callee's ABI.
>
> The callee then does a sibling call, based on the assumption that
> that argument is there and that the stack space for that argument
> is there.
>
> Also assume that the callee language actually had its ABI wrong
> because it assumed that it was called via varargs, which it was not.
>
> This is usually harmless on x86_64, because all that means is that
> an extra register is set, but on other platforms, this is grave
> (like subroutines with too many arguments not working).
>
> However, all the world's not a VAX, and the ABI bug was fixed,
> and caller and callee in that particular language finally agreed,
> both are now like non-varargs C calls.
>
> Everything great, right? Well, except that the people who had
> violated the ABI all along on x86_64 suddenly started seeing
> failures because of sibling calls.
>
> So, should the ABI bug not have been fixed, in your opinion?

I don't understand.
Can you give more examples?
Including pseudo-asm of platforms where it mattered.

Michael S <already5chosen@yahoo.com> schrieb:
> On Friday, November 5, 2021 at 7:18:16 PM UTC+2, Thomas Koenig wrote:
>> Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
>> > Thomas Koenig <tko...@netcologne.de> writes:
>> >>Anton Ertl <an...@mips.complang.tuwien.ac.at> schrieb:
>> >>
>> >>> My position is basically that a responsible maintainer for a
>> >>> widely-used compiler does not break existing source code
>> >>
>> >>Question: You mentioned tail call optimization in one of
>> >>your posts.
>> >>
>> >>Should compilers be mandated to copy arguments for
>> >>
>> >>int bar (int a, int b, int c);
>> >>
>> >>int foo(int a, int b, int c)
>> >>{
>> >> if (a > 0)
>> >> return bar(a,b,c);
>> >> else
>> >> return -1;
>> >>}
>> >
>> > I don't see why. I also don't see the connection to tail-call
>> > optimization. On AMD64 gcc -O does not copy arguments, whether you
>> > enable tail-call optimization in this case (and in this case it
>> > happens to work) with -foptimize-sibling-calls or not.
>> Now, add a few arguments to they have to be passed on the stack.
>>
>> Add code which (in an inter-language setting) lies about the
>> presence of an argument by not passing it because it is never
>> accessed.
>>
>> In other words, the callee expects that there is an argument
>> on the stack, which the caller never supplied, in violation
>> of the callee's ABI.
>>
>> The callee then does a sibling call, based on the assumption that
>> that argument is there and that the stack space for that argument
>> is there.
>>
>> Also assume that the callee language actually had its ABI wrong
>> because it assumed that it was called via varargs, which it was not.
>>
>> This is usually harmless on x86_64, because all that means is that
>> an extra register is set, but on other platforms, this is grave
>> (like subroutines with too many arguments not working).
>>
>> However, all the world's not a VAX, and the ABI bug was fixed,
>> and caller and callee in that particular language finally agreed,
>> both are now like non-varargs C calls.
>>
>> Everything great, right? Well, except that the people who had
>> violated the ABI all along on x86_64 suddenly started seeing
>> failures because of sibling calls.
>>
>> So, should the ABI bug not have been fixed, in your opinion?
>
> I don't understand.
> Can you give more examples?
> Including pseudo-asm of platforms where it mattered.

Here are a few links with non-pseudo-asm (and x86_64
was affected):

https://gcc.gnu.org/legacy-ml/fortran/2019-05/msg00021.html

https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90329

https://developer.r-project.org/Blog/public/2019/05/15/gfortran-issues-with-lapack/

https://developer.r-project.org/Blog/public/2019/09/25/gfortran-issues-with-lapack-ii/

On Wednesday, November 3, 2021 at 3:30:53 PM UTC-6, MitchAlsup wrote:
> On Wednesday, November 3, 2021 at 3:19:19 PM UTC-5, Quadibloc wrote:

> > Since the other side has already won, C99 being an official standard
> > and all that, I think that forking gcc is the only reasonable measure
> > available at this late date.

> But does K&R C get the name 'C' or does modern C get to use that name ?

Modern C will be the official standard.

Since C isn't a trademark or otherwise controlled by someone, the legacy
C can still call itself C... but it will have to include something else too in
order to avoid confusion in that case.

John Savard

On Thursday, November 4, 2021 at 12:00:45 PM UTC-6, Anton Ertl wrote:

> Providing a saner alternative to gcc and clang may indeed be the
> answer. However, gcc (and clang) are complex and by now nasal demons
> have grown deep and intricate roots in the vast source code of these
> projects, so basing the alternative of gcc or clang is probably not
> the most effective approach.

Isn't the source code to earlier versions of gcc and clang also archived?

Yes, I certainly agree that it would be difficult to unwind the changes to
gcc manually, but there is still another alternative to doing everything
from scratch.

I would accept that gcc in its present state is frustrating to programmers
with your needs. But I don't think that the C you need can really call itself
a "saner" alternative to gcc and clang. Since these compilers in their present
state conform to the current C specification, it's not really reasonable to
call them "insane", any more than a Pascal compiler is insane.

But a Pascal compiler can't do some of the jobs that a K&R C compiler is
good for... and now gcc and clang have basically joined it. Whether this is
due to popular demand, or the possibly mistaken tastes and preferences of
the developers... may make the change regrettable. But it's not a change
from sanity to insanity, merely a change from one kind of language to another
kind of language.

John Savard

On Thursday, November 4, 2021 at 4:01:10 AM UTC-6, David Brown wrote:

> If there were people on your "side" who were interested in doing
> something more than just preaching on street corners, there would
> already be a fork of gcc.

_His_ side. While I have some sympathy towards his complaints,
and don't view them as entirely unreasonable, I personally don't
have a need for a K&R-conformant C compiler. For the kinds of tasks
that I might wish to program, a "modern" language like Pascal or
Fortran would do just fine.

John Savard

Quadibloc <jsavard@ecn.ab.ca> schrieb:
> On Thursday, November 4, 2021 at 12:00:45 PM UTC-6, Anton Ertl wrote:
>
>> Providing a saner alternative to gcc and clang may indeed be the
>> answer. However, gcc (and clang) are complex and by now nasal demons
>> have grown deep and intricate roots in the vast source code of these
>> projects, so basing the alternative of gcc or clang is probably not
>> the most effective approach.
>
> Isn't the source code to earlier versions of gcc and clang also archived?

http://ftp.gnu.org/gnu/gcc/ and http://ftp.gnu.org/old-gnu/gcc/ are
your friend.

> Yes, I certainly agree that it would be difficult to unwind the changes to
> gcc manually, but there is still another alternative to doing everything
> from scratch.

LLVM has done that, but with a different aim and a different
reason.

On 07.11.2021 07:31, Quadibloc wrote:
> On Wednesday, November 3, 2021 at 3:30:53 PM UTC-6, MitchAlsup wrote:
>
>
>
> Modern C will be the official standard.
>
> Since C isn't a trademark or otherwise controlled by someone, the legacy
> C can still call itself C... but it will have to include something else too in
> order to avoid confusion in that case.
>
> John Savard
>
How about
(C) © U+00A9 ?
SCNR

MitchAlsup wrote:
> On Saturday, November 6, 2021 at 10:02:11 AM UTC-5, EricP wrote:
>> MitchAlsup wrote:
>>> On Friday, November 5, 2021 at 12:31:07 PM UTC-5, EricP wrote:
>>>> Thomas Koenig wrote:
>>>>>
>>>>> 5.1 INT30-C. Ensure that unsigned integer operations do not wrap.
>>>>>
>>>>> Integer values must not be allowed to wrap, especially if they
>>>>> are used in any of the following ways:
>>>>>
>>>>> * Integer operands of any pointer arithmetic, including array
>>>>> indexing
>>> <
>>>> The scaled index address mode [rBase + rIndex*scale + imm]
>>>> must not trap-on-wrap by default. As an _optional_ check it is fine.
>>>>
>>>> For languages with non base-0 arrays, using a biased-base array
>>>> pointer saves continuously subtracting the lower array index.
>>>> The initial base bias is allowed to wrap because it will wrap back again
>>>> when the array is indexed, and the index should have been range checked.
>>> <
>>> So, what about the case where the virtual address space is 63-bits
>>> (not 64-bits) and one does NOT want AGEN to change the HoB of
>>> Rbase+immed when adding Rindex<<scale to it ? An array for an
>>> application is not allowed to span the 2^63 boundary.
> <
>> That is why using detection of a transition across some arbitrary
>> address boundary is not a good indicator for errors.
>> Better to do this with actual accesses and the MMU.
> <
> But the damage has already been done by the time the VA gets to the MMU.

What damage? If the access is illegal the MMU catches it.
Wrapping is "a bad thing" for calculations that assume integers are linear.
As I have illustrated, wrapping an address is actually necessary
in some situations. As your 2^63 example shows, the choice boundaries
between areas can be arbitrary.

>>> <
>>> That is, the HoB of the AGENed address tells the HW about what address
>>> space this access accesses. You do not want AGEN to change the virtual
>>> address space of access.
> <
>> Right but don't define a _change_ to a bit as the error detector.
>> What if code used immediate constant as an absolute address?
> <
> That is why I phrased my responses in the form of Rbase+imm
> That one can change the HoBs. Adding Rindex<<scale is not
> allowed to change HOBs.

But it is your definition that 2^63 is a boundary is not anyone elses.
Why hardwiring it into an ISA if it is unnecessary to do so?

> <
>> Then there is no address calculation, no bit changes,
>> but the access is just as legal or illegal.
> <
> Yes, but you went out of your way to get there.

I wouldn't call an immediate absolute address "out of the way".
It is a standard linkage mechanism.

>> The MMU is the right place for all of this.
> <
> Once again, the MMU only sees the VA, by this tmie the damage has
> already been done.

What damage!!!???

> <
>>> <
>>> Now, should you actually WANT to change the VaS, this is easily done
>>> with ADD, OR, XOR outside of AGEN influences; generally of the unsigned
>>> variety.
>>> <
>>>> So in this case address arithmetic is modulo.
>>> <
>>> I would argue that parts of AGEN are modulo and parts are not.
> <
>> It only matters if one wants to check address calculations and I don't.
>>> <
>>>> If the array index is a signed integer, it is sign extended to size
>>>> intptr_t, changed to unsptr_t, scaled and added modulo (unchecked).
>>> <
>>> But should address arithmetic be able to change the virtual address space
>>> the access accesses ??
> <
>> The virtual address space is dynamically defined by the program and OS.
>> The ISA should not make assumptions about its specifics.
>> The MMU can change from model to model.
> <
> There are a number of 63-bit virtual address space all addressable from
> a thread. When using VA[62]==VA[63] you are accessing your won threads
> VaS. When VA[62] !=VA[63] you are accessing a foreign address space
> through a root pointer associated with a different thread.
> <
> It is my considered opinion that an array cannot span an address space.
> {Nor can a struct or union or any base type.}

That is fine, but my point is the choice of VA[63:62] as a boundary
is arbitrary and is confined to the MMU as set up by the OS.
This distinction doesn't need to show up in the ISA.

Or to put it another way, the road you are headed down implies either
two sets of integer arithmetic, one for numbers and one for addresses,
or separate address registers a-la 68000 with their own arithmetic.
I don't think either of those is appealing as I'm sure you don't,
which leaves the MMU to be the decider of legality.

> <
>>>> If it was going to do checked [rBase + rIndex*scale + imm] address
>>>> arithmetic, then the scaling of rIndex register should check that
>>>> no significant digits are shifted out, which requires knowing if
>>>> rIndex is signed or unsigned.
>>> <
>>> That seems fair. How about including VaS[63] too ?
> <
>> I _dont_ want to check address calculations for wrap.
>> I was just walking through my reasons why,
>> what trying to do so would do to an ISA.
>>> <
>>>> To check the sum we need to know if addresses are signed or unsigned.
>>>> Personally I consider addresses unsigned because there is no reason
>>>> a buffer cannot straddle the 7FFF... 8000... boundary.
>>>> However others consider address space as signed.
>>> <
>>> There are arguments on each side.
>>> 7FFF-8000 is fine
>>> 7FFFFFFF-80000000 is fine
>>> 3FFFFFFFFFFFFFFF-4000000000000000 is fine
>>> 7FFFFFFFFFFFFFFF-8000000000000000 is questionable.
>>> <
>>> However, I do not think that an array or buffer should be able to span
>>> 0X7FFFFFFFFFFFFFFF to 0x8000000000000000 especially if/when
>>> the later address is in a fundamentally different VaS than the former.
>>> In many circumstances the OS is up in 0x8000000000000000 while
>>> the application is down in x07FFFFFFFFFFFFFFF.
>> Note that on a 64-bit ISA the OS will occupy a tiny fraction of the
>> unsigned high virtual address range so there is nothing special about
>> the 7FFFFFFFFFFFFFFF-8000000000000000 address boundary,
> <
> Neither the user nor the OS span that boundary. The user stays "down below"
> and the OS stays "up above".

In one definition - one that it dependent on a particular
configuration of the MMU tables.

If an OS only needs 4 GB of virtual space for itself then why
should an ISA assume it reserves the whole 2^63 upper half?
Especially since this is determined by how the page table
is set up and how that table is edited during a process switch.

On Sunday, November 7, 2021 at 11:04:17 AM UTC-6, EricP wrote:
> MitchAlsup wrote:
> > On Saturday, November 6, 2021 at 10:02:11 AM UTC-5, EricP wrote:
> >> MitchAlsup wrote:
> >>> On Friday, November 5, 2021 at 12:31:07 PM UTC-5, EricP wrote:
> >>>> Thomas Koenig wrote:
> >>>>>
> >>>>> 5.1 INT30-C. Ensure that unsigned integer operations do not wrap.
> >>>>>
> >>>>> Integer values must not be allowed to wrap, especially if they
> >>>>> are used in any of the following ways:
> >>>>>
> >>>>> * Integer operands of any pointer arithmetic, including array
> >>>>> indexing
> >>> <
> >>>> The scaled index address mode [rBase + rIndex*scale + imm]
> >>>> must not trap-on-wrap by default. As an _optional_ check it is fine.
> >>>>
> >>>> For languages with non base-0 arrays, using a biased-base array
> >>>> pointer saves continuously subtracting the lower array index.
> >>>> The initial base bias is allowed to wrap because it will wrap back again
> >>>> when the array is indexed, and the index should have been range checked.
> >>> <
> >>> So, what about the case where the virtual address space is 63-bits
> >>> (not 64-bits) and one does NOT want AGEN to change the HoB of
> >>> Rbase+immed when adding Rindex<<scale to it ? An array for an
> >>> application is not allowed to span the 2^63 boundary.
> > <
> >> That is why using detection of a transition across some arbitrary
> >> address boundary is not a good indicator for errors.
> >> Better to do this with actual accesses and the MMU.
> > <
> > But the damage has already been done by the time the VA gets to the MMU.
> What damage?
<
The Address has already crossed a VaS boundary. As if a citizen of USA
went and grabbed an apple owned by a Canadian.
<
After AGEN has created its bit pattern the fact of changing VaSs has been lost.
<
> If the access is illegal the MMU catches it.
<
The MMU simultaneously supports multiple VaSs.
<
> Wrapping is "a bad thing" for calculations that assume integers are linear.
<
This is not wrapping, this is changing of the HoB such that you are accessing
a different VaS.
<
Can we at least agree that an "array" must be within a single VaS ?
<
> As I have illustrated, wrapping an address is actually necessary
> in some situations.
<
You showed an example where the OS is mapped into the address space
of the user. This is not a necessary condition in my architecture, all that
is necessary is KNOWING that the address space is foreign.
<
> As your 2^63 example shows, the choice boundaries
> between areas can be arbitrary.
<
2^63 HAS to be a better choice than 2^16 :: do you not agree ?
> >>> <
> >>> That is, the HoB of the AGENed address tells the HW about what address
> >>> space this access accesses. You do not want AGEN to change the virtual
> >>> address space of access.
> > <
> >> Right but don't define a _change_ to a bit as the error detector.
> >> What if code used immediate constant as an absolute address?
> > <
> > That is why I phrased my responses in the form of Rbase+imm
> > That one can change the HoBs. Adding Rindex<<scale is not
> > allowed to change HOBs.
> But it is your definition that 2^63 is a boundary is not anyone elses.
> Why hardwiring it into an ISA if it is unnecessary to do so?
<
OK, how do you allow for accessing other VaSs without creating a whole
new slew of memory reference instructions ?
<
And it is not just 1 new VaS it could be potentially thousands of different
VaSs. Basically, this is a system where anyone can provide the kind of
information to anyone else that allows them to access your VaS. None
of these accesses require any privilege or use special instructions.
> > <
> >> Then there is no address calculation, no bit changes,
> >> but the access is just as legal or illegal.
> > <
> > Yes, but you went out of your way to get there.
> I wouldn't call an immediate absolute address "out of the way".
> It is a standard linkage mechanism.
<
Immediate relative address is the std in My 66000. As long as you (linker)
know where you are, they both work out the same.
<
> >> The MMU is the right place for all of this.
> > <
> > Once again, the MMU only sees the VA, by this tmie the damage has
> > already been done.
> What damage!!!???
<
You have changed the VaS ! an array a[i] must exist entirely within a single
VaS.
<
> > <
> >>> <
> >>> Now, should you actually WANT to change the VaS, this is easily done
> >>> with ADD, OR, XOR outside of AGEN influences; generally of the unsigned
> >>> variety.
> >>> <
> >>>> So in this case address arithmetic is modulo.
> >>> <
> >>> I would argue that parts of AGEN are modulo and parts are not.
> > <
> >> It only matters if one wants to check address calculations and I don't.
> >>> <
> >>>> If the array index is a signed integer, it is sign extended to size
> >>>> intptr_t, changed to unsptr_t, scaled and added modulo (unchecked).
> >>> <
> >>> But should address arithmetic be able to change the virtual address space
> >>> the access accesses ??
> > <
> >> The virtual address space is dynamically defined by the program and OS.
> >> The ISA should not make assumptions about its specifics.
> >> The MMU can change from model to model.
> > <
> > There are a number of 63-bit virtual address space all addressable from
> > a thread. When using VA[62]==VA[63] you are accessing your won threads
> > VaS. When VA[62] !=VA[63] you are accessing a foreign address space
> > through a root pointer associated with a different thread.
> > <
> > It is my considered opinion that an array cannot span an address space.
> > {Nor can a struct or union or any base type.}
> That is fine, but my point is the choice of VA[63:62] as a boundary
> is arbitrary and is confined to the MMU as set up by the OS.
<
Once again, how can the MMU tell that a change in VaS has been calculated ?
<
> This distinction doesn't need to show up in the ISA.
>
> Or to put it another way, the road you are headed down implies either
> two sets of integer arithmetic, one for numbers and one for addresses,
> or separate address registers a-la 68000 with their own arithmetic.
> I don't think either of those is appealing as I'm sure you don't,
> which leaves the MMU to be the decider of legality.
> > <
> >>>> If it was going to do checked [rBase + rIndex*scale + imm] address
> >>>> arithmetic, then the scaling of rIndex register should check that
> >>>> no significant digits are shifted out, which requires knowing if
> >>>> rIndex is signed or unsigned.
> >>> <
> >>> That seems fair. How about including VaS[63] too ?
> > <
> >> I _dont_ want to check address calculations for wrap.
> >> I was just walking through my reasons why,
> >> what trying to do so would do to an ISA.
> >>> <
> >>>> To check the sum we need to know if addresses are signed or unsigned.
> >>>> Personally I consider addresses unsigned because there is no reason
> >>>> a buffer cannot straddle the 7FFF... 8000... boundary.
> >>>> However others consider address space as signed.
> >>> <
> >>> There are arguments on each side.
> >>> 7FFF-8000 is fine
> >>> 7FFFFFFF-80000000 is fine
> >>> 3FFFFFFFFFFFFFFF-4000000000000000 is fine
> >>> 7FFFFFFFFFFFFFFF-8000000000000000 is questionable.
> >>> <
> >>> However, I do not think that an array or buffer should be able to span
> >>> 0X7FFFFFFFFFFFFFFF to 0x8000000000000000 especially if/when
> >>> the later address is in a fundamentally different VaS than the former.
> >>> In many circumstances the OS is up in 0x8000000000000000 while
> >>> the application is down in x07FFFFFFFFFFFFFFF.
> >> Note that on a 64-bit ISA the OS will occupy a tiny fraction of the
> >> unsigned high virtual address range so there is nothing special about
> >> the 7FFFFFFFFFFFFFFF-8000000000000000 address boundary,
> > <
> > Neither the user nor the OS span that boundary. The user stays "down below"
> > and the OS stays "up above".
> In one definition - one that it dependent on a particular
> configuration of the MMU tables.
>
> If an OS only needs 4 GB of virtual space for itself then why
> should an ISA assume it reserves the whole 2^63 upper half?
<
If the OS exists within a 4GB VaS, it can be placed on any MMU boundary !
it does not need the HoB set/clr in any recognizable way. That 4GB OS can
still manage 2^63 VaS applications using foreign VaS access.
<
This is the beauty of not lobbing this into the MMU.
<
The only other alternative is segment registers (ala Power)
<
> Especially since this is determined by how the page table
> is set up and how that table is edited during a process switch.
<
The page tables should be set up to manage the size of the VaS of the
task at hand--whether that task be HV, OS, or application.

On 11/7/2021 9:36 AM, MitchAlsup wrote:
> On Sunday, November 7, 2021 at 11:04:17 AM UTC-6, EricP wrote:

<snip>

>> Especially since this is determined by how the page table
>> is set up and how that table is edited during a process switch.
> <
> The page tables should be set up to manage the size of the VaS of the
> task at hand--whether that task be HV, OS, or application.
>

Mitch, you have confused me just as much as Eric. Just what is a VaS as
you use the term?

To me, and I think to Eric, a VaS is the set of addresses that can be
stated by the machine instructions. Only some of those addresses can be
reached of course - accessibility is a distinct issue. Thus if the EA
computed by the instruction can be any bit pattern from zero to 2^64-1
then the address space is 64 bits. The value 2^67+12345 is not in the
address space, but any computable bit pattern is.

I think (am not sure) that you have in mind a HW-centric vieu in which
there is a pre-step in the VA-to-PA translation, wherein the 2^64 bit
patterns are partitioned into two subsets which use different MMUs (or
maybe just different underlying tables) and you are calling each of
those subsets a "VaS". If so then you need to introduce a new term to
distinguish the superset from the subset(s).

If you let "VaS" apply to the full 2^64, as I and apparently Eric and
the general practice would have it, then please coin a term for your
subsets - perhaps "sub-VaS". If you want to continue to use "VaS" for
the subset(s) - at whatever risk of confusing your audience - then you
need a term for what comes out of the address adder; perhaps "virtual
domain"?

Names of things matter, or things descent into confusion and pointless
wrangling, c.f. "conforming" on this board.

On Sunday, November 7, 2021 at 1:55:56 PM UTC-6, Ivan Godard wrote:
> On 11/7/2021 9:36 AM, MitchAlsup wrote:
> > On Sunday, November 7, 2021 at 11:04:17 AM UTC-6, EricP wrote:
> <snip>
> >> Especially since this is determined by how the page table
> >> is set up and how that table is edited during a process switch.
> > <
> > The page tables should be set up to manage the size of the VaS of the
> > task at hand--whether that task be HV, OS, or application.
> >
> Mitch, you have confused me just as much as Eric. Just what is a VaS as
> you use the term?
>
> To me, and I think to Eric, a VaS is the set of addresses that can be
> stated by the machine instructions. Only some of those addresses can be
> reached of course - accessibility is a distinct issue. Thus if the EA
> computed by the instruction can be any bit pattern from zero to 2^64-1
> then the address space is 64 bits. The value 2^67+12345 is not in the
> address space, but any computable bit pattern is.
<
A Virtual Address Space (VaS) is an area of memory mapped by a single
set of page tables found from a Root Pointer. In My 66000 one can configure
a root pointer to support a 23-bit, 33-bit, 43-bit, 53-bit, and 63-bit VaS.
You cannot support a 2^64-bit space from a single root pointer (PageFault)
You CAN support one 2^63 bit VaS, and at least 1024 additional PortHole
VaSs.
<
I arranged this so that negative addresses -2^63..-1 and positive addresses
0..2^63-1 give the OS/HV the illusion they seem to desire. The -2^64..-2^63+1
and +263+1..2^64-1 access foreign VaSs which are base and bounds protected
along with being access rights protected and mapped.
<
A thread may accress 1 or more VaSs. In addition, VaSs can be passed
around from thread to thread, where the recipient is then granted access
to the VaS handed to him by the caller. In certain transistions (tap, exception,
accept) the root pointer of the caller is squirreled away (in an easy to find
place) and can be read and then used as a foreign VaS by recipient.
<
This is sort-of-like how PDP-11/45 and /70 access application threads
(read from and store to) except I have generalized the notion so one can
access many thread VaSs (should the need arise). This is how I am supporting
ADA accepts, the accept routine can access the root pointer of the caller,
and access the callers IN arguments (register and stack) and deliver OUT
results to the caller upon return. INOUT are simply read when needed and
written when needed. The caller and the acceptor do not have to be in the
same VaS for this rendezvous to work.
<
Using this notion, the OS does not have to be in the same VaS as the
application it is providing services too. Neither does the interrupt
handlers, I/O completion handlers, or even the file system, network system,
.... The OS can be a collection of thing-a-ma-bobs each in their own VaS
unable to cause damage to the other VaSs (when SW is written properly).
This gives the OS the kind of safety for itself that it imposes upon applications.
<
Over on the I/O side, I/O accesses are translated and protected by the same
mechanisms, so a 32-bit device can write to the entire 64-bit address space
even though it cannot form an address of that size. But (properly managed)
said I/O device cannot write to a page it does not currently have permission.
,...
<
>
> I think (am not sure) that you have in mind a HW-centric vieu in which
> there is a pre-step in the VA-to-PA translation, wherein the 2^64 bit
> patterns are partitioned into two subsets which use different MMUs (or
> maybe just different underlying tables) and you are calling each of
> those subsets a "VaS". If so then you need to introduce a new term to
> distinguish the superset from the subset(s).
<
Yes, at the flat-space model, there are normal accesses from -11xxxxxxxx
to +00xxxxxxxxx coverd by a single root pointer. Between -10xxxxxxxxx
and +01xxxxxxxxx there is another layer of indirection where these VaSs
are addressed by a PortHole VaS descriptor {Root pointer + rights + Base
+ Bounds}
<
>
> If you let "VaS" apply to the full 2^64, as I and apparently Eric and
> the general practice would have it, then please coin a term for your
> subsets - perhaps "sub-VaS". If you want to continue to use "VaS" for
> the subset(s) - at whatever risk of confusing your audience - then you
> need a term for what comes out of the address adder; perhaps "virtual
> domain"?
<
It seems to me that a VaS is an area of accessible memory and 2^63
is "big enough". I am reserving the other ½ of addresses that can be
computed to access other VaSs through PortHoles.
<
And in particular, while I am perfectly happy allowing a[i] have 2^63 of
size, I remain adamant that this array not SPAN a 2^63 boundary. Thus,
I allow Rbase+immed to point anywhere, but I restrict +Rindex<<scale
term from spanning across that 2^63 boundary.
<
It is not a sub-VaS because you are not allowed to "run over the boundary".
<
And, it can be programmed up so the foreign VaS can be "over the network"
on another machine with no common address spaces, except for which the
network manager seems fit to provide. This allows for (say) and ADA accept
in the Flight control computer of a F22 to accept a call from the tail managment
computer, with no change in code compared to the abstraction where both
the FCC and TMC are on the same chip and within the same actual memory.
No change in code is required to go from actually local, to absolutely remote
(only the latency addition and inter-mediation SW is required).
>
> Names of things matter, or things descent into confusion and pointless
> wrangling, c.f. "conforming" on this board.
<
It seems to me that I an using the word VaS as an area of virtual memory that is
mapped by a single resource (root pointer or PortHole).
<
I agree that this is slightly different from the notion that a VaS is every address
that could be generated by AGEN address arithmetic. It is also safer.

On 11/7/2021 2:29 PM, MitchAlsup wrote:
> On Sunday, November 7, 2021 at 1:55:56 PM UTC-6, Ivan Godard wrote:

<snip>

>> Names of things matter, or things descent into confusion and pointless
>> wrangling, c.f. "conforming" on this board.
> <
> It seems to me that I an using the word VaS as an area of virtual memory that is
> mapped by a single resource (root pointer or PortHole).
> <
> I agree that this is slightly different from the notion that a VaS is every address
> that could be generated by AGEN address arithmetic. It is also safer.
>

OK, it was as I had suspected. It's not clear to me how a choice of
terms can be "safer" in any sense (as opposed to what the terms name,
for which "safe" may, but need not, be a valid measure).

I still recommend that you coin a new term, or at minimum provide
heavily qualified definition before using "VaS" in text.

On Sunday, November 7, 2021 at 4:45:04 PM UTC-6, Ivan Godard wrote:
> On 11/7/2021 2:29 PM, MitchAlsup wrote:
> > On Sunday, November 7, 2021 at 1:55:56 PM UTC-6, Ivan Godard wrote:
> <snip>
> >> Names of things matter, or things descent into confusion and pointless
> >> wrangling, c.f. "conforming" on this board.
> > <
> > It seems to me that I an using the word VaS as an area of virtual memory that is
> > mapped by a single resource (root pointer or PortHole).
> > <
> > I agree that this is slightly different from the notion that a VaS is every address
> > that could be generated by AGEN address arithmetic. It is also safer.
> >
> OK, it was as I had suspected. It's not clear to me how a choice of
> terms can be "safer" in any sense (as opposed to what the terms name,
> for which "safe" may, but need not, be a valid measure).
<
It is not the terms that make it safer, it is the prevention of Rindex<<scale
from changing the HoBs that make it safer, while allowing Rbase+immed
to still create any bit pattern appropriate. You cannot index over the
boundary of the VaS you are attempting to access. You can still point
anywhere !
>
> I still recommend that you coin a new term, or at minimum provide
> heavily qualified definition before using "VaS" in text.
<
The LvL (level) field in the Root Pointer defines the size of the VaS.
So a VaS is not associated with how big the AGEN adder can produce.
<
Others have been using the concept that a VaS is as big as the AGEN
adder can produce and you get to access it (or not) based on the MMU
tables.

On 11/7/2021 3:21 PM, MitchAlsup wrote:
> On Sunday, November 7, 2021 at 4:45:04 PM UTC-6, Ivan Godard wrote:
>> On 11/7/2021 2:29 PM, MitchAlsup wrote:
>>> On Sunday, November 7, 2021 at 1:55:56 PM UTC-6, Ivan Godard wrote:
>> <snip>
>>>> Names of things matter, or things descent into confusion and pointless
>>>> wrangling, c.f. "conforming" on this board.
>>> <
>>> It seems to me that I an using the word VaS as an area of virtual memory that is
>>> mapped by a single resource (root pointer or PortHole).
>>> <
>>> I agree that this is slightly different from the notion that a VaS is every address
>>> that could be generated by AGEN address arithmetic. It is also safer.
>>>
>> OK, it was as I had suspected. It's not clear to me how a choice of
>> terms can be "safer" in any sense (as opposed to what the terms name,
>> for which "safe" may, but need not, be a valid measure).
> <
> It is not the terms that make it safer, it is the prevention of Rindex<<scale
> from changing the HoBs that make it safer, while allowing Rbase+immed
> to still create any bit pattern appropriate. You cannot index over the
> boundary of the VaS you are attempting to access. You can still point
> anywhere !

Yes, it lets AGEN catch a particular kind of bug, which is always
helpful even though it doesn't prevent manually calculating the address.
And I don't think it tromps on any of our patent work in this area. But
the granularity is a bit large. You could set a more constraining
maximum object size (say 50 bits, or make it dynamic) and then fault any
AGEN that rolls over that no matter where in the object was in the 64
bit space. Shouldn't be any more hardware.

>>
>> I still recommend that you coin a new term, or at minimum provide
>> heavily qualified definition before using "VaS" in text.
> <
> The LvL (level) field in the Root Pointer defines the size of the VaS.
> So a VaS is not associated with how big the AGEN adder can produce.
> <
> Others have been using the concept that a VaS is as big as the AGEN
> adder can produce and you get to access it (or not) based on the MMU
> tables.

And that's what I and it seems others here expect. Your 66 has a lot of
novel notions; if you will take some advice from one who has a lot of
experience introducing novelty: if your idea is different enough to brag
about (much less patent) then it needs a new name.

In article <8fe79286-c374-4129-b2a2-cb93099e9448n@googlegroups.com>,
MitchAlsup <MitchAlsup@aol.com> wrote:
>It is not the terms that make it safer, it is the prevention of Rindex<<scale
>from changing the HoBs that make it safer, while allowing Rbase+immed
>to still create any bit pattern appropriate. You cannot index over the
>boundary of the VaS you are attempting to access. You can still point
>anywhere !

If I understand what you are saying, for a load/store using addressing
of the form:

[Rbase + Rindex << scale + imm]

You calculate this in two parts. First, Rbase+imm is calculated and the
high-order bits are looked at. These bits determine which page tables to
use. Then, "Rindex << scale" is added in. Adding in Rindex must not change
the high-order bits of the VA, and if they do, you will trap (or something).

I'm going to suggest this is not a good idea. Let instructions form the
VA any way they want, then look at the high-order bits of the final VA
result and use that for any purpose you want.

A complex enough pointer calculation may not have a clear "base" register,
and if the compiler chooses incorrectly, the resulting code will not work.
Maybe the compiler has a reason to transform:

a[i] = 0;

which in C is the same as:

*(a + i) = 0;

STR zero,[i + (a << 0) + 0)]

User code could be dumbly written, where they've casted pointers to an
integer type, and in forming a pointer again, the code casts some offset to
be a pointer, and adds in the real pointer as an integer. This could be
happening in the compiler as well for jump tables, or for dynamic library
linking, etc.

But with your checking that Rbase must have the proper high-order bits, this
will not always work. And, it adds complexity to code generation for what
I suspect is minimal hardware savings. In fact, it may be more complex to
rely on the Rbase+imm to calculate the high-order bits, since then you have
to do the AGEN in a certain way to get that answer.

Kent

In article <sm0050$109j$1@gioia.aioe.org>,
Terje Mathisen <terje.mathisen@tmsw.no> wrote:
[ snip ]
>Yes, compilers do know a lot more about the nooks & crannies of cpu
>performance than the vast majority of programmers, but OTOH, the type of
>optimizations you have to employ to handle micro-architectural balance
>shifts can very often be far larger than anything a compiler would be
>allowed to do.
>
>A case in point: I have done several conference presentations and a
>university guest lecture on a single program/algorithm (unix wc - word
>count) which I've optimized for most generations of x86 cpus from 8088
>to Pentium and PentiumPro.
>
>The type of optimizations needed consisted of total rewrites, using new
>algorithms, but still achieving the same end result.

I am curious what wc algorithms you used, and I wasn't able to find your
presentation online with a little bit of Googling. If it is available,
can you point me to it?

I wanted a way to test the speed of some algorithms (and SSD
performance), and wc seemed like a good choice, where counting spaces
and lines would give me a "checksum" of sorts as well. But wc on a Mac
is just stupidly slow, so I wrote:

while(1) {
len = reliable_read(in_fd, &(g_inbuf[0]), INBUF_SIZE);
if(len == 0) {
break;
}
inptr = &g_inbuf[0];
for(i = 0; i < len; i++) {
c = *inptr++;
type = g_chartype[c];
dcount_lines += (type & 2);
is_new_nonspace = last_space & type;
dcount_words += is_new_nonspace;
last_space = (~type) & 1;
}
dcount_chars += len;
}

where dcount_lines count is 2x the proper value. (Yes, this code will
fail on 2^63 newlines). reliable_read is just read() which always gets
the required size unless EOF is hit. The definition of "words" in wc
was not easy for me to understand, so I have probably done it
incorrectly.

Kent

On Thursday, November 11, 2021 at 9:34:01 AM UTC-8, Kent Dickey wrote:
> In article <sm0050$109j$1...@gioia.aioe.org>,
> Terje Mathisen <terje.m...@tmsw.no> wrote:
> [ snip ]
> >Yes, compilers do know a lot more about the nooks & crannies of cpu
> >performance than the vast majority of programmers, but OTOH, the type of
> >optimizations you have to employ to handle micro-architectural balance
> >shifts can very often be far larger than anything a compiler would be
> >allowed to do.
> >
> >A case in point: I have done several conference presentations and a
> >university guest lecture on a single program/algorithm (unix wc - word
> >count) which I've optimized for most generations of x86 cpus from 8088
> >to Pentium and PentiumPro.
> >
> >The type of optimizations needed consisted of total rewrites, using new
> >algorithms, but still achieving the same end result.
>
> I am curious what wc algorithms you used, and I wasn't able to find your
> presentation online with a little bit of Googling. If it is available,
> can you point me to it?
>
> I wanted a way to test the speed of some algorithms (and SSD
> performance), and wc seemed like a good choice, where counting spaces
> and lines would give me a "checksum" of sorts as well. But wc on a Mac
> is just stupidly slow, so I wrote:
>
> while(1) {
> len = reliable_read(in_fd, &(g_inbuf[0]), INBUF_SIZE);
> if(len == 0) {
> break;
> }
> inptr = &g_inbuf[0];
> for(i = 0; i < len; i++) {
> c = *inptr++;
> type = g_chartype[c];
> dcount_lines += (type & 2);
> is_new_nonspace = last_space & type;
> dcount_words += is_new_nonspace;
> last_space = (~type) & 1;
> }
> dcount_chars += len;
> }
>
> where dcount_lines count is 2x the proper value. (Yes, this code will
> fail on 2^63 newlines). reliable_read is just read() which always gets
> the required size unless EOF is hit. The definition of "words" in wc
> was not easy for me to understand, so I have probably done it
> incorrectly.
>
> Kent

You may find this to be of interest regarding the optimizations to wc:
http://downloads.gamedev.net/pdf/gpbb/gpbb13.pdf