On 2022-05-21, Jake Hamby <jake.hamby@gmail.com> wrote:
>
> There's a good argument for Itanium for that purpose (perhaps the only
> good argument left for Itanium these days). The DCL vulnerability reported
> in early 2018 was patched for VAX, Alpha, and Itanium, but the Itanium
> patch was to protect any Alpha nodes on the same cluster: Itanium alone
> wasn't vulnerable, even though this was specifically a VMS attack and they
> tried to attack it. The register stack engine and unusual instruction
> encoding seem to thwart the common exploits.
>

Hello Jake,

I'm the person who found the DCL vulnerability.

There was never a patch developed for VAX that I am aware of, unless
some consultant did it privately. There was a workaround, which was
to disable CDU access for normal users. This may or may not have been
viable for a specific site.

I never probed Itanium myself because there is no such thing as an
Itanium full system emulator and I have no desire to run a stonking
big and noisy Itanium at home... :-) Those things are rather expensive
as well...

VSI tested my research on Itanium and confirmed that it caused a crash
on Itanium. Itanium was patched because it was still a buffer overflow
on Itanium. It was just that it couldn't be exploited using the specific
exploit I developed.

I consider it an open question whether someone with more detailed DCL
internals knowledge than I have could have developed another attack
that would have worked on Itanium by overwriting something else within DCL.

You should also be aware that the patch merely closed off this specific
DCL buffer overflow. It did nothing to fix VMS's dirty secret, which is
that shells running in supervisor mode have access to the privileges of
the programs they are running.

Until something is done about that, if someone finds another way back
into DCL from user mode that allows them to get code they control running
within the context of DCL itself, then this could start all over again.

BTW, I call it a dirty secret because when I found the buffer overflow
I initially didn't know what to do with it. There had been rumours
and suggestions floating around for years that once in supervisor mode,
you could do "things", but no-one who knew was saying anything, even
after the patch was released by VSI.

The timeline on this was not early 2018 BTW. I sent it to VSI in mid-2017,
they created a patch that was released a month or so later, and then in
November 2017, I decided to see if I could find what the big secret was,
which I succeeded in doing.

I was going to release the details in accordance with standard disclosure
procedures, given that the patch had been available for a couple of months
at that point, but I got very strong feedback that people needed more time
to patch their systems, so I held off until early 2018.

That big secret, as we now all know, is that DCL has access to the
privileges of the programs it runs. I still don't believe that one. :-)

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2022-05-23 13:25:36 +0000, Simon Clubley said:

> That big secret, as we now all know, is that DCL has access to the
> privileges of the programs it runs.

Arbitrary code executing in supervisor mode in the context of DCL can
gain full system access.

--
Pure Personal Opinion | HoffmanLabs LLC

On Monday, May 23, 2022 at 11:44:20 AM UTC-7, Stephen Hoffman wrote:
> On 2022-05-23 13:25:36 +0000, Simon Clubley said:
>
> > That big secret, as we now all know, is that DCL has access to the
> > privileges of the programs it runs.
> Arbitrary code executing in supervisor mode in the context of DCL can
> gain full system access.

The VMS security model is very interesting / surprising to someone used to UNIX. Instead of "sudo", I have a user account with SETPRV as an "authorized" privilege, but any DCL script or process can see that I have that privilege and can take over the entire system. It's the same as malware seeing that "sudo" is in the path with no password.

VMS also reminds me of z/OS, where there are different "address keys", some that everyone can read, others only for certain system users. It's just that in VMS, the rings are concentric, there are fewer of them, and less hardware assistance for them. In the meanframe world, third parties and individual developers can and do install SVC calls to do whatever, and in that sense it's similar to VMS letting sufficiently authorized users install any image they want with privileges.

DCL is really a weak link, for sure. Code executing in supervisor mode in DCL context shouldn't be able to gain full system access. I'm sure you know of multiple ways by which it could, but those should be patched. I'm probably very foolish by giving my normal account SETPRV, but I try to only ever run code that I've built myself, or trust was built by someone unlikely to put anything malicious in.

Regards,
Jake

> Hello Jake,
>
> I'm the person who found the DCL vulnerability.

Thank you so much for introducing yourself and for the extra information.

> BTW, I call it a dirty secret because when I found the buffer overflow
> I initially didn't know what to do with it. There had been rumours
> and suggestions floating around for years that once in supervisor mode,
> you could do "things", but no-one who knew was saying anything, even
> after the patch was released by VSI.
>
> That big secret, as we now all know, is that DCL has access to the
> privileges of the programs it runs. I still don't believe that one. :-)

Here's what I'm curious about. VMS has the four-ring VAX model at heart, but as it got ported to different platforms, the relevance of the model became less important. What's unusual/interesting about Itanium is that it was designed for the same "privileged page that when you jump to it, the code can raise its privilege level" model that VMS uses, despite being one of the later OSs ported to that unfortunate architecture. It's much more expensive on Itanium to perform an exception, so Linux uses its shared kernel vdso module to provide "gates" to replace the usual inline trap/exception call.

I need to rewatch the long "Re-architecting SWIS for X86-64 (OpenVMS Boot Camp 2017)" talk by Camiel Vanderhoeven because now I'm very curious how SWIS will look on x86-64. Since it's now fully emulating/virtualizing the original VAX model, perhaps it would be feasible for VSI to make it so that DCL in supervisor mode did *not* have access to the privs of the programs it can run. It would be one check among many that had to be done in a low layer of software anyway. It's not hidden away in PALcode that's difficult to change. It's all software, even on Itanium.

On 2022-05-27, Jake Hamby <jake.hamby@gmail.com> wrote:
>> Hello Jake,
>>
>> I'm the person who found the DCL vulnerability.
>
> Thank you so much for introducing yourself and for the extra information.
>
>> BTW, I call it a dirty secret because when I found the buffer overflow
>> I initially didn't know what to do with it. There had been rumours
>> and suggestions floating around for years that once in supervisor mode,
>> you could do "things", but no-one who knew was saying anything, even
>> after the patch was released by VSI.
>>
>> That big secret, as we now all know, is that DCL has access to the
>> privileges of the programs it runs. I still don't believe that one. :-)
>
> Here's what I'm curious about. VMS has the four-ring VAX model at heart, but as it got ported to different platforms, the relevance of the model became less important. What's unusual/interesting about Itanium is that it was designed for the same "privileged page that when you jump to it, the code can raise its privilege level" model that VMS uses, despite being one of the later OSs ported to that unfortunate architecture. It's much more expensive on Itanium to perform an exception, so Linux uses its shared kernel vdso module to provide "gates" to replace the usual inline trap/exception call.
>
> I need to rewatch the long "Re-architecting SWIS for X86-64 (OpenVMS Boot Camp 2017)" talk by Camiel Vanderhoeven because now I'm very curious how SWIS will look on x86-64. Since it's now fully emulating/virtualizing the original VAX model, perhaps it would be feasible for VSI to make it so that DCL in supervisor mode did *not* have access to the privs of the programs it can run. It would be one check among many that had to be done in a low layer of software anyway. It's not hidden away in PALcode that's difficult to change. It's all software, even on Itanium.

One of the problems is with (for example) Ctrl-Y.

Ctrl-Y would need to pass through a formal kernel to supervisor mode
barrier that stripped the extra privileges so supervisor mode never
saw them, and the CONTINUE command (if the user typed that), would
need to pass back through the barrier in the other direction so that
the kernel restored those extra privileges before handing control back
to the user program.

To extend the above, basically all interactions between the kernel and
supervisor mode would need to pass through such a barrier so supervisor
mode never, ever, saw those elevated privileges at any point during the
lifecycle of a privileged image.

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2022-05-27 04:56:20 +0000, Jake Hamby said:

> Here's what I'm curious about. VMS has the four-ring VAX model at
> heart, but as it got ported to different platforms, the relevance of
> the model became less important.

The four-mode / four-ring model from VAX/VMS sought to isolate software
failures, where user and supervisor failures don't directly effect the
rest of the system, exec failures might or might not, and while kernel
failures tend to be catastrophic.

There were exec-mode bugchecks in RMS around Stream Linefeed file
mis-formatting early on in the availability of that format, and those
deleted the process and left a nice entry in the error log, for
instance. There are other exec-mode failures that'll take out the
system.

This multi-user memory management design is also the foundation of
system security.

> What's unusual/interesting about Itanium is that it was designed for
> the same "privileged page that when you jump to it, the code can raise
> its privilege level" model that VMS uses, despite being one of the
> later OSs ported to that unfortunate architecture. It's much more
> expensive on Itanium to perform an exception, so Linux uses its shared
> kernel vdso module to provide "gates" to replace the usual inline
> trap/exception call.

The security model of most modern multi-user systems based on the
memory protections, and the VAX change-mode instructions (e.g. CMKRNL)
and the Intel x86 call gates are the controlling checkpoints between
code executing in differently-protected memory.

Those memory modes are central to being able to prevent access. With
OpenVMS, those modes are also used to run-down / flush memory contents
no longer required. Remember that OpenVMS processes can routinely run
multiple executables on OpenVMS, and can stash symbols and logical
names and other data where it can be used across image startup and
image run-down, and which is somewhat different from how Unix processes
work around app startup and app exit.

In some ways, the Unix root user and sudo is closer to the RSX-11M/M+
system UIC. This allows passage through the checkpoints. This system
UIC (UIC groups octal 10 and lower) is carried through into OpenVMS,
and is the origin of what OpenVMS refers to as SYSPRV system privilege.
This is also how an OpenVMS user with UIC 2,7 and no privileges
authorized or granted is still a fully-privileged user. That [2,7] user
has system-level access through the checkpoints. But I digress.

> I need to rewatch the long "Re-architecting SWIS for X86-64 (OpenVMS
> Boot Camp 2017)" talk by Camiel Vanderhoeven because now I'm very
> curious how SWIS will look on x86-64. Since it's now fully
> emulating/virtualizing the original VAX model, perhaps it would be
> feasible for VSI to make it so that DCL in supervisor mode did *not*
> have access to the privs of the programs it can run. It would be one
> check among many that had to be done in a low layer of software anyway.
> It's not hidden away in PALcode that's difficult to change. It's all
> software, even on Itanium.

Marketing aside, OpenVMS effectively has two modes / two rings. One is
the familiar user mode, and one is divided up into three parts. The
three parts isolates some failures as might separate processes in other
designs, but they're all necessarily trusted code; all part of the
Trusted Computing Base as it's sometimes known.

How does supervisor mode fit into the OpenVMS user experience? To be
able to perform a user-mode run-down / flush, the code performing the
run-down / flush must inherently be in a different mode, and must be
more trusted. The symbol and logical name storage that exists for the
lifetime of the process is also protected against user corruptions, as
apps must use supervisor-mode APIs and code to access the
supervisor-protected memory involved. (This behavior also ties back to
things like SPAWN needing access to a command interpreter, as SPAWN
copies symbols and logical names into the newly-created subprocess.
This particular part of OpenVMS process handling around spawn is
somewhat more like Unix process handling too, as those symbol and
logical name values are not copied back when the subprocess exits.)
There are other activities of the command line interpreter—or something
acting in its stead—that are also necessarily trusted.

Looking at the requirements differently...

Another way these requirements can architecturally all fit together is
an operating system design with fast interprocess communications, and
with processes as be part of the isolation. Fewer modes / rings are
needed, and usually two modes / rings. This with a thinner layer of
kernel-mode code mediating communications and process creation and
exit. Much of what would be inner-mode code on OpenVMS will reside in
operating-system-provided processes in these microkernel designs. This
approach is intended to have less inner-mode code—OpenVMS has a lot of
that—and with processes that can come and go. A user login providing
services akin to those expected on OpenVMS might have the kernel shim,
a process running the command line interface (in user mode) and a
second process that runs the application code. Run-down / flush then
runs down / flushes the process, rather than running down / flushing
the memory mode / ring as done with OpenVMS. Designs that are closer to
this approach include Erlang OTP and the L4 family. (OpenVMS was ported
to Mach as a research project, too. There's a DEC whitepaper around
named Usenix_VMS-on-Mach.ps.)

Going the other way with the design and all into one big hunk of app
and data and usually all in the same address space, there's VAXELN and
other unikernels. VAXELN was standalone, and some of these now operate
as dedicated guests in hypervisors. One failure takes out the whole
thing with this design.

errata: PALcode is SWIS for Alpha.

errata: no, macOS is not a microkernel design, though it does contain
Mach-like features.

--
Pure Personal Opinion | HoffmanLabs LLC

Thanks for the explanation of how it all fits together, including the historical part about UIC groups under 10 octal having special powers. I did get the impression that the usefulness of having supervisor and executive is much more to catch accidental coding errors than to provide any serious security against malicious users beyond user and "trusted".

I'm very curious about both Erlang OTP and the L4 family, but I've never had a chance to use either of them in a project. I notice RabbitMQ is popular and has been ported to VMS, so I might play around with that as my introduction to Erlang.

On 5/30/2022 8:46 PM, Jake Hamby wrote:
> I'm very curious about both Erlang OTP and the L4 family, but I've
> never had a chance to use either of them in a project. I notice
> RabbitMQ is popular and has been ported to VMS, so I might play
> around with that as my introduction to Erlang.

RabbitMQ is written in Erlang, but most access of RabbitMQ is
done from other languages. Unless you want to modify RabbitMQ
itself, then you will not really see any Erlang.

Arne