On Saturday, July 22, 2023 at 10:26:27 AM UTC-4, Arne Vajhøj wrote:
> On 7/20/2023 8:49 AM, Simon Clubley wrote:
> > I get the impression that you have had to spend a lot of effort to
> > duplicate low-level behaviour in a way that simply isn't a concern
> > with other operating systems due to the need, especially, to support
> > Macro-32 and ensure that code written at this low-level, and which
> > uses the various idioms as a result, continues to work.
> I believe Macro-32 is a relative simple compiler. And when they got
> that then they can compile the existing Macro-32 code.
>
Well, I just spit my wine out reading that.

Simple? Try to deal with condition codes. Try to deal with routines
that jump between each other. Try to make the stack "look like a VAX"
on platforms that say otherwise. :)

There are three main parts of the Macro compiler.
-The parser (written in Macro-32) is 99% target independent.
- The flow analyzer (written in C) looks for all those condition codes,
loops, etc. It has some knowledge about the calling standard (how
many arguments in registers, etc.)
- The code generator that maps internal Macro operators that feel
like "instructions". Opcodes, mem-operands, reg-operands, etc.
Each of those had to be recoded to pick the correct x86 instruction.
For x86, we call the LLVM MC interface to emit x86 instructions.
The way LLVM enumerates the x86 opcodes and registers is a
little funky and needed some abstractions we didn't need on Alpha
or Itanium.

> I believe that VSI is mostly reusing existing Macro-32 code
> and writing new code in C.
>
Actually there was a good chunk of Macro-32 that was rewritten into
C. We redid much of the memory management code for example. It
needed to change to deal with the more complex page table structures
on x86, etc. Decided to rewrite into C.

>
> It means that changing from 32 bit pointers to 64 bit pointers is
> way easier in C than in Macro-32.
>
> But based on what we know about the port, then VSI has not changed
> the VMS kernel from 32 bit pointers to 64 bit pointers (aka moving
> everything from S0 and S1 to S2).
>

Actually lots more stuff is now in S2 space and several internal pointers
were lengthened from 32-bits to 64-bits.

On 7/22/2023 4:57 PM, John Reagan wrote:
> On Saturday, July 22, 2023 at 10:26:27 AM UTC-4, Arne Vajhøj wrote:
>> On 7/20/2023 8:49 AM, Simon Clubley wrote:
>>> I get the impression that you have had to spend a lot of effort to
>>> duplicate low-level behaviour in a way that simply isn't a concern
>>> with other operating systems due to the need, especially, to support
>>> Macro-32 and ensure that code written at this low-level, and which
>>> uses the various idioms as a result, continues to work.
>> I believe Macro-32 is a relative simple compiler. And when they got
>> that then they can compile the existing Macro-32 code.
>>
> Well, I just spit my wine out reading that.

:-)

Ya know, I was sort of expecting that ...

You've mentioned in the past some of what's written below.

If I had to guess, I'd consider the various Macro-32 compilers to be some of the
most difficult software introduced on VMS Alpha and itanic, and now x86.

> Simple? Try to deal with condition codes. Try to deal with routines
> that jump between each other. Try to make the stack "look like a VAX"
> on platforms that say otherwise. :)
>
> There are three main parts of the Macro compiler.
> -The parser (written in Macro-32) is 99% target independent.
> - The flow analyzer (written in C) looks for all those condition codes,
> loops, etc. It has some knowledge about the calling standard (how
> many arguments in registers, etc.)
> - The code generator that maps internal Macro operators that feel
> like "instructions". Opcodes, mem-operands, reg-operands, etc.
> Each of those had to be recoded to pick the correct x86 instruction.
> For x86, we call the LLVM MC interface to emit x86 instructions.
> The way LLVM enumerates the x86 opcodes and registers is a
> little funky and needed some abstractions we didn't need on Alpha
> or Itanium.
>
>
>> I believe that VSI is mostly reusing existing Macro-32 code
>> and writing new code in C.
>>
> Actually there was a good chunk of Macro-32 that was rewritten into
> C. We redid much of the memory management code for example. It
> needed to change to deal with the more complex page table structures
> on x86, etc. Decided to rewrite into C.
>
>
>>
>> It means that changing from 32 bit pointers to 64 bit pointers is
>> way easier in C than in Macro-32.
>>
>> But based on what we know about the port, then VSI has not changed
>> the VMS kernel from 32 bit pointers to 64 bit pointers (aka moving
>> everything from S0 and S1 to S2).
>>
>
> Actually lots more stuff is now in S2 space and several internal pointers
> were lengthened from 32-bits to 64-bits.
>

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 22/07/2023 21:57, John Reagan wrote:
> On Saturday, July 22, 2023 at 10:26:27 AM UTC-4, Arne Vajhøj wrote:
>> On 7/20/2023 8:49 AM, Simon Clubley wrote:
>>> I get the impression that you have had to spend a lot of effort to
>>> duplicate low-level behaviour in a way that simply isn't a concern
>>> with other operating systems due to the need, especially, to support
>>> Macro-32 and ensure that code written at this low-level, and which
>>> uses the various idioms as a result, continues to work.
>> I believe Macro-32 is a relative simple compiler. And when they got
>> that then they can compile the existing Macro-32 code.
>>
> Well, I just spit my wine out reading that.
>
> Simple? Try to deal with condition codes. Try to deal with routines
> that jump between each other. Try to make the stack "look like a VAX"
> on platforms that say otherwise. :)
>
> There are three main parts of the Macro compiler.
> -The parser (written in Macro-32) is 99% target independent.
> - The flow analyzer (written in C) looks for all those condition codes,
> loops, etc. It has some knowledge about the calling standard (how
> many arguments in registers, etc.)
> - The code generator that maps internal Macro operators that feel
> like "instructions". Opcodes, mem-operands, reg-operands, etc.
> Each of those had to be recoded to pick the correct x86 instruction.
> For x86, we call the LLVM MC interface to emit x86 instructions.
> The way LLVM enumerates the x86 opcodes and registers is a
> little funky and needed some abstractions we didn't need on Alpha
> or Itanium.
>
>
>> I believe that VSI is mostly reusing existing Macro-32 code
>> and writing new code in C.
>>
> Actually there was a good chunk of Macro-32 that was rewritten into
> C. We redid much of the memory management code for example. It
> needed to change to deal with the more complex page table structures
> on x86, etc. Decided to rewrite into C.
>
>
>>
>> It means that changing from 32 bit pointers to 64 bit pointers is
>> way easier in C than in Macro-32.
>>
>> But based on what we know about the port, then VSI has not changed
>> the VMS kernel from 32 bit pointers to 64 bit pointers (aka moving
>> everything from S0 and S1 to S2).
>>
>
> Actually lots more stuff is now in S2 space and several internal pointers
> were lengthened from 32-bits to 64-bits.

I was expecting some sort of answer like this. Shame we are not on a
modern forum - that would have got a 'like; from me!

--
Chris

On 7/22/2023 4:57 PM, John Reagan wrote:
> On Saturday, July 22, 2023 at 10:26:27 AM UTC-4, Arne Vajhøj wrote:
>> On 7/20/2023 8:49 AM, Simon Clubley wrote:
>>> I get the impression that you have had to spend a lot of effort to
>>> duplicate low-level behaviour in a way that simply isn't a concern
>>> with other operating systems due to the need, especially, to support
>>> Macro-32 and ensure that code written at this low-level, and which
>>> uses the various idioms as a result, continues to work.
>> I believe Macro-32 is a relative simple compiler. And when they got
>> that then they can compile the existing Macro-32 code.
>>
> Well, I just spit my wine out reading that.
>
> Simple? Try to deal with condition codes. Try to deal with routines
> that jump between each other. Try to make the stack "look like a VAX"
> on platforms that say otherwise. :)

OK. I was wrong. Happens all the time. :-)

>> I believe that VSI is mostly reusing existing Macro-32 code
>> and writing new code in C.
>>
> Actually there was a good chunk of Macro-32 that was rewritten into
> C. We redid much of the memory management code for example. It
> needed to change to deal with the more complex page table structures
> on x86, etc. Decided to rewrite into C.

If the old Macro-32 code did not work, then doing the new code
in C instead of writing new Macro-32 code makes sense.

>> It means that changing from 32 bit pointers to 64 bit pointers is
>> way easier in C than in Macro-32.
>>
>> But based on what we know about the port, then VSI has not changed
>> the VMS kernel from 32 bit pointers to 64 bit pointers (aka moving
>> everything from S0 and S1 to S2).
>
> Actually lots more stuff is now in S2 space and several internal pointers
> were lengthened from 32-bits to 64-bits.

Interesting.

I thought there were still room in S1.

Are you at liberty to disclose what has been moved to S2?

Arne

On 2023-07-22 20:57:03 +0000, John Reagan said:
>
> Simple? Try to deal with condition codes. Try to deal with routines
> that jump between each other. Try to make the stack "look like a VAX"
> on platforms that say otherwise. :)

Kinda wonder what radare2 would do with that stress test, err, with
that xMacro32 executable.

Semi-related is XTRAN, and which has been discussed before:
http://www.xtran-llc.com/vxmcee.html
http://www.xtran-llc.com/xtran.html#reengrg

--
Pure Personal Opinion | HoffmanLabs LLC

On Saturday, July 22, 2023 at 7:22:35 PM UTC-4, Arne Vajhøj wrote:

>
> I thought there were still room in S1.
>
> Are you at liberty to disclose what has been moved to S2?
>
> Arne

I think things like large buffers for NICs, page tables, etc. I'm not sure about
exactly what but I had to help with some S2 pool code that needs to do
atomic updates a 64-bit flink/blink pointer in some data structures.

On Saturday, July 22, 2023 at 8:52:28 PM UTC-4, Stephen Hoffman wrote:
> On 2023-07-22 20:57:03 +0000, John Reagan said:
> >
> > Simple? Try to deal with condition codes. Try to deal with routines
> > that jump between each other. Try to make the stack "look like a VAX"
> > on platforms that say otherwise. :)
> Kinda wonder what radare2 would do with that stress test, err, with
> that xMacro32 executable.
>
> Semi-related is XTRAN, and which has been discussed before:
> http://www.xtran-llc.com/vxmcee.html
> http://www.xtran-llc.com/xtran.html#reengrg
>
> --
> Pure Personal Opinion | HoffmanLabs LLC

The real ugly Macro-32 sources include LAD driver and the shadow driver.
When we did the Itanium compiler, I had the compilers print out some
internal counters into the .LIS file. We were dealing with routines that
enter in one places with one set of arguments and then jumped to another
routine and returned from another routine's epilogue. There are issues dealing
with NaTs and such. From what I can tell, EVERY .CALL_ENTRY and .JSB_ENTRY
in the shadow driver can exist from EVERY possible RET. Of course, in real life
I'm sure the data doesn't do that, but Macro had to worry about it. I'm not sure
I could prepare myself for the XTRAN version of the shadow driver.

On 7/23/2023 3:45 PM, John Reagan wrote:
> From what I can tell, EVERY .CALL_ENTRY and .JSB_ENTRY
> in the shadow driver can exist from EVERY possible RET. Of course, in real life
> I'm sure the data doesn't do that, but Macro had to worry about it.

Because of the home-grown threading package that shadowing uses, yeah, there is
an insane amount of jumping all over the place.

--

--- Rob

In article <e9738d50-3a9d-489e-b59b-c472eadaf618n@googlegroups.com>,
xyzzy1959@gmail.com (John Reagan) wrote:

> The real ugly Macro-32 sources include LAD driver and the shadow
> driver.

I've been reading the Macro compiler manual. I have never done any VAX
assembler, but I've done a lot on other platforms, and reported a lot of
compiler bugs on IA-64, x86-64, and ARM64.

It looks to me as if cross-compiling Macro-32 isn't so hard if you just
want it to run, but making it run /fast/ on x86-64 was probably hard.
There aren't quite enough registers to map the VAX registers 1:1 and
putting them in memory makes for fun with calling conventions, caching
and so on.

John

On 7/23/2023 6:03 PM, John Dallman wrote:
> In article <e9738d50-3a9d-489e-b59b-c472eadaf618n@googlegroups.com>,
> xyzzy1959@gmail.com (John Reagan) wrote:
>
>> The real ugly Macro-32 sources include LAD driver and the shadow
>> driver.
>
> I've been reading the Macro compiler manual. I have never done any VAX
> assembler, but I've done a lot on other platforms, and reported a lot of
> compiler bugs on IA-64, x86-64, and ARM64.
>
> It looks to me as if cross-compiling Macro-32 isn't so hard if you just
> want it to run, but making it run /fast/ on x86-64 was probably hard.
> There aren't quite enough registers to map the VAX registers 1:1 and
> putting them in memory makes for fun with calling conventions, caching
> and so on.

The Macro-32 compiler supposedly does some optimization.

<quote>
$ help macro /opt

MACRO

/OPTIMIZE

/OPTIMIZE[=(option[,...])]
/NOOPTIMIZE

Enables or disables optimization options. All options are enabled
by default except VAXREGS.

The options are:

Option Description

[NO]PEEPHOLE Peephole optimization
[NO]SCHEDULE Code scheduling
[NO]ADDRESSES Common base address loading
[NO]REFERENCES Common data referencing
ALL All optimizations
</quote>

But difficult to check the difference between /OPT and /NOOPT,
because for some unknown reason /LIST/MACH does not list the
generated code.

Anyway I am not so sure that the optimization is critical. My thinking
is that a typical piece of Macro-32 probably goes back to the VAX 780
days, the code must have performed acceptable back then or it would
have been changed, the VUPS.COM's seems to return around 750 on x86-64
PC's, so even if the optimized Macro-32 run at only 20% of max speed,
then the code will be approx. 150 times faster than on a VAX 780, which
is probably more than fine.

Arne

On Monday, July 24, 2023 at 1:31:46 AM UTC+2, Arne Vajhøj wrote:
> But difficult to check the difference between /OPT and /NOOPT,
> because for some unknown reason /LIST/MACH does not list the
> generated code.

MACRO-32 on x86? You can always (except you compile with /NOOBJECT :-) get the machine code listing with ANALYZE/OBJECT/DISASSEMBLE.

On Saturday, July 22, 2023 at 9:57:05 PM UTC+1, John Reagan wrote:
> On Saturday, July 22, 2023 at 10:26:27 AM UTC-4, Arne Vajhøj wrote:
> > On 7/20/2023 8:49 AM, Simon Clubley wrote:
> > > I get the impression that you have had to spend a lot of effort to
> > > duplicate low-level behaviour in a way that simply isn't a concern
> > > with other operating systems due to the need, especially, to support
> > > Macro-32 and ensure that code written at this low-level, and which
> > > uses the various idioms as a result, continues to work.
> > I believe Macro-32 is a relative simple compiler. And when they got
> > that then they can compile the existing Macro-32 code.
> >
> Well, I just spit my wine out reading that.
>
> Simple? Try to deal with condition codes. Try to deal with routines
> that jump between each other. Try to make the stack "look like a VAX"
> on platforms that say otherwise. :)
>
> There are three main parts of the Macro compiler.
> -The parser (written in Macro-32) is 99% target independent.
> - The flow analyzer (written in C) looks for all those condition codes,
> loops, etc. It has some knowledge about the calling standard (how
> many arguments in registers, etc.)
> - The code generator that maps internal Macro operators that feel
> like "instructions". Opcodes, mem-operands, reg-operands, etc.
> Each of those had to be recoded to pick the correct x86 instruction.
> For x86, we call the LLVM MC interface to emit x86 instructions.
> The way LLVM enumerates the x86 opcodes and registers is a
> little funky and needed some abstractions we didn't need on Alpha
> or Itanium.
>
>
> > I believe that VSI is mostly reusing existing Macro-32 code
> > and writing new code in C.
> >
> Actually there was a good chunk of Macro-32 that was rewritten into
> C. We redid much of the memory management code for example. It
> needed to change to deal with the more complex page table structures
> on x86, etc. Decided to rewrite into C.
>
>
> >
> > It means that changing from 32 bit pointers to 64 bit pointers is
> > way easier in C than in Macro-32.
> >
> > But based on what we know about the port, then VSI has not changed
> > the VMS kernel from 32 bit pointers to 64 bit pointers (aka moving
> > everything from S0 and S1 to S2).
> >
>
> Actually lots more stuff is now in S2 space and several internal pointers
> were lengthened from 32-bits to 64-bits.

I look forward to seeing a OpenVMS x86 internals document at some point particularly on memory management.

On 2023-07-22, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 7/22/2023 4:57 PM, John Reagan wrote:
>> On Saturday, July 22, 2023 at 10:26:27?AM UTC-4, Arne Vajhøj wrote:
>>> On 7/20/2023 8:49 AM, Simon Clubley wrote:
>>>> I get the impression that you have had to spend a lot of effort to
>>>> duplicate low-level behaviour in a way that simply isn't a concern
>>>> with other operating systems due to the need, especially, to support
>>>> Macro-32 and ensure that code written at this low-level, and which
>>>> uses the various idioms as a result, continues to work.
>>> I believe Macro-32 is a relative simple compiler. And when they got
>>> that then they can compile the existing Macro-32 code.
>>>
>> Well, I just spit my wine out reading that.
>>
>> Simple? Try to deal with condition codes. Try to deal with routines
>> that jump between each other. Try to make the stack "look like a VAX"
>> on platforms that say otherwise. :)
>
> OK. I was wrong. Happens all the time. :-)
>

And _this_ Arne, is exactly why I keep saying the kinds of things I do.

VSI are having to expend serious engineering effort on trying to
duplicate the behaviour of an assembly language (with all its built-in
idioms) on a completely different architecture.

This is simply a non-issue on operating systems where the lowest
supported language is C instead of assembly language. The stuff John
is having to deal with is just an implementation detail in the backend
code generator on Linux and elsewhere that is hidden from the source code.

In Macro-32 however, it's directly visible in the source code and you have
to go to great efforts to duplicate that behaviour on other architectures.

On Linux, and other operating systems written in C or above, that's an
issue you simply don't have to worry about.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-07-23, John Reagan <xyzzy1959@gmail.com> wrote:
> On Saturday, July 22, 2023 at 8:52:28?PM UTC-4, Stephen Hoffman wrote:
>> On 2023-07-22 20:57:03 +0000, John Reagan said:
>> >
>> > Simple? Try to deal with condition codes. Try to deal with routines
>> > that jump between each other. Try to make the stack "look like a VAX"
>> > on platforms that say otherwise. :)
>> Kinda wonder what radare2 would do with that stress test, err, with
>> that xMacro32 executable.
>>
>> Semi-related is XTRAN, and which has been discussed before:
>> http://www.xtran-llc.com/vxmcee.html

Bloody hell Stephen, that auto-translated C code is even more ugly than
the Macro-32 version!!! :-)

>> http://www.xtran-llc.com/xtran.html#reengrg
>
> The real ugly Macro-32 sources include LAD driver and the shadow driver.
> When we did the Itanium compiler, I had the compilers print out some
> internal counters into the .LIS file. We were dealing with routines that
> enter in one places with one set of arguments and then jumped to another
> routine and returned from another routine's epilogue. There are issues dealing
> with NaTs and such. From what I can tell, EVERY .CALL_ENTRY and .JSB_ENTRY
> in the shadow driver can exist from EVERY possible RET. Of course, in real life
> I'm sure the data doesn't do that, but Macro had to worry about it. I'm not sure
> I could prepare myself for the XTRAN version of the shadow driver.

Errr, YUCK!!! :-)

You could always rewrite the shadow driver in C... :-)

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-07-24, Ian Miller <gxys@uk2.net> wrote:
>
> I look forward to seeing a OpenVMS x86 internals document at some point particularly on memory management.

I don't think that is going to happen. I've asked about a version of
the I&DS for x86-64 and was told there isn't the market for it.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2023-07-22, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 7/22/2023 11:06 AM, Single Stage to Orbit wrote:
>> On Sat, 2023-07-22 at 10:48 -0400, Arne Vajhøj wrote:
>>> If DEC had been able to see into the future then they
>>> may have gone with C.
>>
>> IF they had, VMS would have had a shitload of exploits against it.
>> Thankfully they didn't and avoided a lot of issues that plagues most
>> operating systems written in C.
>
> Maybe.
>
> C is very unsafe compared to almost all other high level
> languages.
>

Yes, very much so. I believe I have expressed alternative preferences
in the past. :-)

> But I don't think it is more unsafe than Macro-32.
>
>:-)

:-)

Even with C, the compiler can find problems in your code that simply
are not possible in assembly language. (A pointer which references
the wrong structure type, unused variables, dodgy type conversions, etc).

>
> Anyway: Windows and Linux has not failed commercially
> due to the buffer overflows.
>
>:-)
>

And Linux has security enhancements such as SELinux, ASLR, etc that
simply do not have any comparable functionality in the VMS world.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 7/24/2023 8:33 AM, Simon Clubley wrote:
> On 2023-07-22, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> On 7/22/2023 4:57 PM, John Reagan wrote:
>>> On Saturday, July 22, 2023 at 10:26:27?AM UTC-4, Arne Vajhøj wrote:
>>>> On 7/20/2023 8:49 AM, Simon Clubley wrote:
>>>>> I get the impression that you have had to spend a lot of effort to
>>>>> duplicate low-level behaviour in a way that simply isn't a concern
>>>>> with other operating systems due to the need, especially, to support
>>>>> Macro-32 and ensure that code written at this low-level, and which
>>>>> uses the various idioms as a result, continues to work.
>>>> I believe Macro-32 is a relative simple compiler. And when they got
>>>> that then they can compile the existing Macro-32 code.
>>>>
>>> Well, I just spit my wine out reading that.
>>>
>>> Simple? Try to deal with condition codes. Try to deal with routines
>>> that jump between each other. Try to make the stack "look like a VAX"
>>> on platforms that say otherwise. :)
>>
>> OK. I was wrong. Happens all the time. :-)
>>
>
> And _this_ Arne, is exactly why I keep saying the kinds of things I do.
>
> VSI are having to expend serious engineering effort on trying to
> duplicate the behaviour of an assembly language (with all its built-in
> idioms) on a completely different architecture.
>
> This is simply a non-issue on operating systems where the lowest
> supported language is C instead of assembly language. The stuff John
> is having to deal with is just an implementation detail in the backend
> code generator on Linux and elsewhere that is hidden from the source code.
>
> In Macro-32 however, it's directly visible in the source code and you have
> to go to great efforts to duplicate that behaviour on other architectures.
>
> On Linux, and other operating systems written in C or above, that's an
> issue you simply don't have to worry about.
>
> Simon.
>

Ok, how about this perspective?

If x86 VMS does not support existing VMS applications, then what's the point of
x86 VMS? So, yeah, some tough work, but, it supports the customers.

Easy for Simon to tell users to re-write their applications. It's not his time
and money.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 7/24/2023 8:39 AM, Simon Clubley wrote:
> On 2023-07-23, John Reagan <xyzzy1959@gmail.com> wrote:
>> On Saturday, July 22, 2023 at 8:52:28?PM UTC-4, Stephen Hoffman wrote:
>>> On 2023-07-22 20:57:03 +0000, John Reagan said:
>>>>
>>>> Simple? Try to deal with condition codes. Try to deal with routines
>>>> that jump between each other. Try to make the stack "look like a VAX"
>>>> on platforms that say otherwise. :)
>>> Kinda wonder what radare2 would do with that stress test, err, with
>>> that xMacro32 executable.
>>>
>>> Semi-related is XTRAN, and which has been discussed before:
>>> http://www.xtran-llc.com/vxmcee.html
>
> Bloody hell Stephen, that auto-translated C code is even more ugly than
> the Macro-32 version!!! :-)
>
>>> http://www.xtran-llc.com/xtran.html#reengrg
>>
>> The real ugly Macro-32 sources include LAD driver and the shadow driver.
>> When we did the Itanium compiler, I had the compilers print out some
>> internal counters into the .LIS file. We were dealing with routines that
>> enter in one places with one set of arguments and then jumped to another
>> routine and returned from another routine's epilogue. There are issues dealing
>> with NaTs and such. From what I can tell, EVERY .CALL_ENTRY and .JSB_ENTRY
>> in the shadow driver can exist from EVERY possible RET. Of course, in real life
>> I'm sure the data doesn't do that, but Macro had to worry about it. I'm not sure
>> I could prepare myself for the XTRAN version of the shadow driver.
>
> Errr, YUCK!!! :-)
>
> You could always rewrite the shadow driver in C... :-)
>
> Simon.
>

Simon is consistent. Offering other people's time and money.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 2023-07-24, Dave Froble <davef@tsoft-inc.com> wrote:
> On 7/24/2023 8:33 AM, Simon Clubley wrote:
>>
>> VSI are having to expend serious engineering effort on trying to
>> duplicate the behaviour of an assembly language (with all its built-in
>> idioms) on a completely different architecture.
>>
>> This is simply a non-issue on operating systems where the lowest
>> supported language is C instead of assembly language. The stuff John
>> is having to deal with is just an implementation detail in the backend
>> code generator on Linux and elsewhere that is hidden from the source code.
>>
>> In Macro-32 however, it's directly visible in the source code and you have
>> to go to great efforts to duplicate that behaviour on other architectures.
>>
>> On Linux, and other operating systems written in C or above, that's an
>> issue you simply don't have to worry about.
>>
>
> Ok, how about this perspective?
>
> If x86 VMS does not support existing VMS applications, then what's the point of
> x86 VMS? So, yeah, some tough work, but, it supports the customers.
>
> Easy for Simon to tell users to re-write their applications. It's not his time
> and money.
>

We are not talking about people rewriting their applications.

We are talking about structural and design issues within VMS that is
making this porting process a lot more involved than it should be.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 7/24/2023 1:12 PM, Simon Clubley wrote:
> On 2023-07-24, Dave Froble <davef@tsoft-inc.com> wrote:
>> On 7/24/2023 8:33 AM, Simon Clubley wrote:
>>>
>>> VSI are having to expend serious engineering effort on trying to
>>> duplicate the behaviour of an assembly language (with all its built-in
>>> idioms) on a completely different architecture.
>>>
>>> This is simply a non-issue on operating systems where the lowest
>>> supported language is C instead of assembly language. The stuff John
>>> is having to deal with is just an implementation detail in the backend
>>> code generator on Linux and elsewhere that is hidden from the source code.
>>>
>>> In Macro-32 however, it's directly visible in the source code and you have
>>> to go to great efforts to duplicate that behaviour on other architectures.
>>>
>>> On Linux, and other operating systems written in C or above, that's an
>>> issue you simply don't have to worry about.
>>>
>>
>> Ok, how about this perspective?
>>
>> If x86 VMS does not support existing VMS applications, then what's the point of
>> x86 VMS? So, yeah, some tough work, but, it supports the customers.
>>
>> Easy for Simon to tell users to re-write their applications. It's not his time
>> and money.
>>
>
> We are not talking about people rewriting their applications.
>
> We are talking about structural and design issues within VMS that is
> making this porting process a lot more involved than it should be.
>
> Simon.
>

You have been down on Macro-32, but, there are customers that will be very happy
there is a Macro-32 compiler on x86 VMS. It's not just for the OS.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On Monday, July 24, 2023 at 1:12:45 PM UTC-4, Simon Clubley wrote:
> We are talking about structural and design issues within VMS that is
> making this porting process a lot more involved than it should be.

They need to be emulated or mapped to new platform functions
unless you want to just call the operating system VMS but have it
be substantially different "under the hood" . That may work for some
things where both sides of an interface are internal to the operating
system, but (for example) not having supervisor mode is going to
have repercussions for some user code as well as require a re-
working of DCL.

The "low-hanging fruit" has already been dealt with in the VAX ->
Alpha -> Itanic progression.

For things that have been replaced with C code, performance tuning
may be needed once things are complete and deemed functionally
correct. This can take the form of in-line x86 assembly code or the
replacement of an entire C source file with an optimized assembly
version.

I have a Very Large Program that I have been dragging around with
me, starting on 4.2BSD/VAX and currently on FreeBSD/x86-64. It
does a more-than-reasonable amount of rummaging around inside
supposedly-internal operating system structures. So I have some
idea of the issues involved.

On 7/24/2023 8:33 AM, Simon Clubley wrote:
> On 2023-07-22, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> On 7/22/2023 4:57 PM, John Reagan wrote:
>>> On Saturday, July 22, 2023 at 10:26:27?AM UTC-4, Arne Vajhøj wrote:
>>>> On 7/20/2023 8:49 AM, Simon Clubley wrote:
>>>>> I get the impression that you have had to spend a lot of effort to
>>>>> duplicate low-level behaviour in a way that simply isn't a concern
>>>>> with other operating systems due to the need, especially, to support
>>>>> Macro-32 and ensure that code written at this low-level, and which
>>>>> uses the various idioms as a result, continues to work.
>>>> I believe Macro-32 is a relative simple compiler. And when they got
>>>> that then they can compile the existing Macro-32 code.
>>>>
>>> Well, I just spit my wine out reading that.
>>>
>>> Simple? Try to deal with condition codes. Try to deal with routines
>>> that jump between each other. Try to make the stack "look like a VAX"
>>> on platforms that say otherwise. :)

> VSI are having to expend serious engineering effort on trying to
> duplicate the behaviour of an assembly language (with all its built-in
> idioms) on a completely different architecture.
>
> This is simply a non-issue on operating systems where the lowest
> supported language is C instead of assembly language. The stuff John
> is having to deal with is just an implementation detail in the backend
> code generator on Linux and elsewhere that is hidden from the source code.
>
> In Macro-32 however, it's directly visible in the source code and you have
> to go to great efforts to duplicate that behaviour on other architectures.
>
> On Linux, and other operating systems written in C or above, that's an
> issue you simply don't have to worry about.

We were discussing what was causing the relative long time to
get VMS x86-64 out the door.

You suggested that it was due to Macro-32.

VSI said that was not the case. They should know.

And it seems very unlikely that Macro-32 should be the
reason:
- even when it is one sneaky compiler then it it is just one
compiler out of many
- the compiler had to be done anyway due to customers
also using Macro-32 and they expect something working
- the porting strategy seems to be that new functionality
and functionality that has to rewritten is done in C while
Macro-32 code is getting compiled with the new compiler

Arne

On 7/24/2023 1:12 PM, Simon Clubley wrote:
> We are talking about structural and design issues within VMS that is
> making this porting process a lot more involved than it should be.

There are two very different questions:

#1: Did DEC's decisions in 1977 make sense given 1977 knowledge?

#2: Did DEC's decisions in 1977 make sense given 2023 knowledge?

I believe the answer to #1 is YES. Their decisions was similar to how
most OS was designed at the time.

The answer to #2 is probably NO, because they should have gone
with a HLL, they should have prepared for future migrations to other
ISA, they should have prepared for 64 bit, they should have
focused more on security etc.etc..

But trying to evaluate decisions of the past using todays
knowledge is a totally futile exercise.

If you plan on inventing a time machine and travel back in
time to tell them to do things differently, then maybe. But
otherwise then accept that decisions was made based on
available information at the time for the decisions. And
complaining about such decisions does not change a thing.

There is a term for it "Monday morning quarterback".

Arne

On Monday, July 24, 2023 at 5:50:56 PM UTC-7, Arne Vajhøj wrote:

(snip)
> But trying to evaluate decisions of the past using todays
> knowledge is a totally futile exercise.
> If you plan on inventing a time machine and travel back in
> time to tell them to do things differently, then maybe. But
> otherwise then accept that decisions was made based on
> available information at the time for the decisions. And
> complaining about such decisions does not change a thing.

Not that I don't like VAX, but they pretty much did everything wrong.

The 512 byte page size was likely already too small, but definitely
too small a few years later.

A CISC instruction set designed to make it easy for assembly
programmers, just at the time that systems programming was
moving to high level languages.

And an instruction set not at all well designed for pipelined
processing.

Consider how hard it is to figure out how long an instruction is!

It would have been, maybe just a little bit, better to put all the
address mode bytes immediately after the opcode.

On 7/24/2023 4:38 AM, hb@end.of.inter.net wrote:
> On Monday, July 24, 2023 at 1:31:46 AM UTC+2, Arne Vajhøj wrote:
>> But difficult to check the difference between /OPT and /NOOPT,
>> because for some unknown reason /LIST/MACH does not list the
>> generated code.
>
> MACRO-32 on x86?

Yes.

> You can always (except you compile with /NOOBJECT :-) get the machine code listing with ANALYZE/OBJECT/DISASSEMBLE.

I did not know that.

But also weird.

I do not see a difference between /OPT and /NOOPT at all.

Arne