Java and log4j / log4shell (CVE-2021-44228) remote command execution
vulnerability

intro: https://www.lawfareblog.com/whats-deal-log4shell-security-nightmare

Base OpenVMS itself does not include Java, though add-on apps and
layered products can have dependencies and can install Java.

If you have Java installed anywhere on OpenVMS (try DIRECTORY
ddcu:[*...]JAVA*, etc), you will need to evaluate your configuration in
more detail.

You'll want to evaluate other components and services around your
servers, as well.

List of possibly-effected services and apps:
https://gist.github.com/SwitHak/b66db3a06c2955a9cb71a8718970c592#file-20211210-tlp-white_log4j-md

Reportedly, all versions of Java are vulnerable when log4i is present
and reachable, and exploits are active and under development.

It appears there are efforts underway to create worms using this
vulnerability, as well.

--
Pure Personal Opinion | HoffmanLabs LLC

On 12/13/2021 4:12 PM, Stephen Hoffman wrote:
> Java and log4j / log4shell (CVE-2021-44228) remote command execution
> vulnerability
>
> intro: https://www.lawfareblog.com/whats-deal-log4shell-security-nightmare
>
> Base OpenVMS itself does not include Java, though add-on apps and
> layered products can have dependencies and can install Java.
>
> If you have Java installed anywhere on OpenVMS (try DIRECTORY
> ddcu:[*...]JAVA*, etc), you will need to evaluate your configuration in
> more detail.
>
> You'll want to evaluate other components and services around your
> servers, as well.
>
> List of possibly-effected services and apps:
> https://gist.github.com/SwitHak/b66db3a06c2955a9cb71a8718970c592#file-20211210-tlp-white_log4j-md
>
> Reportedly, all versions of Java are vulnerable when log4i is present
> and reachable, and exploits are active and under development.
>
> It appears there are efforts underway to create worms using this
> vulnerability, as well.

"all versions of Java are vulnerable when log4j is present and reachable"

is a funny description.

It is a vulnerability for all running Java applications using
log4j 2.0 - 2.14.1 that logs user input.

That is serious. A very large portion of Java server applications
(think 50% magnitude!) use log4j and it seems likely that most of
them have potential for logging user input (user input is important
when troubleshooting).

The version of Java does not impact a bug in log4j more than
the version of C compiler impact a buffer overrun in a C library.

Note that log4j 2.x does not run on VMS Alpha due to too old Java
version (2.0 - 2.3 requires Java 6, 2.4 - 2.12.1 requires Java 7 and
2.13 and newer requires Java 8).

And log4j 1.x is not vulnerable to this bug. But it is out of
support and has other vulnerabilities, so it is not a good
version to be on.

But anybody running a Java application on Itanium that uses
log4j 2.x better upgrade to 2.15 or newer (latest as of today
is 2.16).

To check:

$ dir [whereever...]log4j-core-2.*.jar

should reveal any log4j 2.x present ion that tree.

Every system manager would (hopefully) know whether Java
is installed or not - but very few will know offhand
which applications use log4j, so you better check!!

Arne

On 12/13/2021 8:51 PM, Arne Vajhøj wrote:
> To check:
>
> $ dir [whereever...]log4j-core-2.*.jar
>
> should reveal any log4j 2.x present ion that tree.

Maybe better:

> $ dir [whereever...]log4j-core-2*.jar

just in case it is installed on an ODS-2 disk with _ instead of
period.

Also look out for fatjars which could hide its presence.

Arne

> (think 50% magnitude!)

Optimists are so cute.

De

On 12/13/21 7:51 PM, Arne Vajhøj wrote:
> On 12/13/2021 4:12 PM, Stephen Hoffman wrote:

> The version of Java does not impact a bug in log4j more than
> the version of C compiler impact a buffer overrun in a C library.

Not true. From <https://www.openwall.com/lists/oss-security/2021/12/10/1>:

---
Java 8u121 (see
https://www.oracle.com/java/technologies/javase/8u121-relnotes.html)
protects against remote code execution by defaulting
"com.sun.jndi.rmi.object.trustURLCodebase" and
"com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
---

So an up-to-date Java (currently I think 1.8.0_312-b07 for Java 8)
mitigates (some of) the mayhem that can be caused by the vulnerability.
Unless I missed one, the latest release from VSI is 1.8.0_222-b05, so
people using Java 8 on VMS should consider getting that if they are
using any older release of Java 8, including, IIRC, any release from HPE.

But also note per this:

<https://www.openwall.com/lists/oss-security/2021/12/10/2>

that an updated Java protects against only one of several remote code
execution vectors, so it's far from a complete fix, but the Java version
certainly does impact the severity of the bug.

On 12/14/2021 8:33 AM, Craig A. Berry wrote:
> On 12/13/21 7:51 PM, Arne Vajhøj wrote:
>> The version of Java does not impact a bug in log4j more than
>> the version of C compiler impact a buffer overrun in a C library.
>
> Not true. From <https://www.openwall.com/lists/oss-security/2021/12/10/1>:
>
> ---
> Java 8u121 (see
> https://www.oracle.com/java/technologies/javase/8u121-relnotes.html)
> protects against remote code execution by defaulting
> "com.sun.jndi.rmi.object.trustURLCodebase" and
> "com.sun.jndi.cosnaming.object.trustURLCodebase" to "false".
> ---
>
> So an up-to-date Java (currently I think 1.8.0_312-b07 for Java 8)
> mitigates (some of) the mayhem that can be caused by the vulnerability.
> Unless I missed one, the latest release from VSI is 1.8.0_222-b05, so
> people using Java 8 on VMS should consider getting that if they are
> using any older release of Java 8, including, IIRC, any release from HPE.
>
> But also note per this:
>
> <https://www.openwall.com/lists/oss-security/2021/12/10/2>
>
> that an updated Java protects against only one of several remote code
> execution vectors, so it's far from a complete fix, but the Java version
> certainly does impact the severity of the bug.

Ah. I stand corrected.

I actually did know that 8u121 did get some of the 9
deserialization protection features backported.

But I did not realize that it would help with this one.

That is good. Most Java 8 users should be way above u121.

Arne

On 12/13/21 8:51 PM, Arne Vajhøj wrote:
>

....

> The version of Java does not impact a bug in log4j more than
> the version of C compiler impact a buffer overrun in a C library.

We keep hearing this mantra over and over. If it really is
such a problem why has no one ever bothered to write a new
library keeping the original APIs while internally removing
the overrun problem? Oh wait, someone did. Back in the early
80's. On the PDP-11. For all the DEC OSes and Ultrix-11
and Version 7 Unix. And there was even a version for the VAX.
How did that work out?

bill

On 2021-12-14 13:33:40 +0000, Craig A. Berry said:

> So an up-to-date Java (currently I think 1.8.0_312-b07 for Java 8)
> mitigates (some of) the mayhem that can be caused by the vulnerability.
> Unless I missed one, the latest release from VSI is 1.8.0_222-b05, so
> people using Java 8 on VMS should consider getting that if they are
> using any older release of Java 8, including, IIRC, any release from
> HPE.

Reports that all versions of Java are vulnerable to exploitation when
log4j is accessible. Early reports that Java 8 and newer were not
vulnerable were later found incorrect.

There are mitigations posted, though some of those are seemingly now
getting bypassed.

There's ransomware active now, too.

HPE has indicated that 3PAR and some other products are vulnerable to
this mess, and has posted a list of not-vulnerable products.

--
Pure Personal Opinion | HoffmanLabs LLC

On 12/14/2021 11:39 AM, Stephen Hoffman wrote:
> On 2021-12-14 13:33:40 +0000, Craig A. Berry said:
>
>> So an up-to-date Java (currently I think 1.8.0_312-b07 for Java 8)
>> mitigates (some of) the mayhem that can be caused by the
>> vulnerability. Unless I missed one, the latest release from VSI is
>> 1.8.0_222-b05, so people using Java 8 on VMS should consider getting
>> that if they are using any older release of Java 8, including, IIRC,
>> any release from HPE.
>
> Reports that all versions of Java are vulnerable to exploitation when
> log4j is accessible.

"accessible"

Why not just say "used".

> Early reports that Java 8 and newer were not
> vulnerable were later found incorrect.

So you are saying that the deserialization protections done in 9 and
backported to 8u121 are not enough to prevent this vulnerability?

Arne

On 12/14/2021 9:11 AM, Bill Gunshannon wrote:
> On 12/13/21 8:51 PM, Arne Vajhøj wrote:
>>
>
> ...
>
>> The version of Java does not impact a bug in log4j more than
>> the version of C compiler impact a buffer overrun in a C library.
>
> We keep hearing this mantra over and over. If it really is
> such a problem why has no one ever bothered to write a new
> library keeping the original APIs while internally removing
> the overrun problem? Oh wait, someone did. Back in the early
> 80's. On the PDP-11. For all the DEC OSes and Ultrix-11
> and Version 7 Unix. And there was even a version for the VAX.
> How did that work out?
>
> bill

Way too many people don't really care about security ... until it bites them on
the ass. Then they expect a law to be passed that will protect them. As if the
hackers really care about laws.

One moment I'll never forget. I was telling a customer that it would be a very
bad idea for them to store their customer's data, bank account, credit card
info, and such in plain text on a IIS server. The response was "why not,
everyone else does". And they ignored my warning and did just that. Don't know
the result, the business relationship didn't last much longer.

--
David Froble Tel: 724-529-0450
Dave Froble Enterprises, Inc. E-Mail: davef@tsoft-inc.com
DFE Ultralights, Inc.
170 Grimplin Road
Vanderbilt, PA 15486

On 12/14/21 12:31 PM, Dave Froble wrote:
> On 12/14/2021 9:11 AM, Bill Gunshannon wrote:
>> On 12/13/21 8:51 PM, Arne Vajhøj wrote:
>>>
>>
>> ...
>>
>>> The version of Java does not impact a bug in log4j more than
>>> the version of C compiler impact a buffer overrun in a C library.
>>
>> We keep hearing this mantra over and over.  If it really is
>> such a problem why has no one ever bothered to write a new
>> library keeping the original APIs while internally removing
>> the overrun problem?  Oh wait, someone did. Back in the early
>> 80's.  On the PDP-11.  For all the DEC OSes and Ultrix-11
>> and Version 7 Unix.  And there was even a version for the VAX.
>> How did that work out?
>>
>> bill
>
> Way too many people don't really care about security ... until it bites
> them on the ass.  Then they expect a law to be passed that will protect
> them.  As if the hackers really care about laws.
>
> One moment I'll never forget.  I was telling a customer that it would be
> a very bad idea for them to store their customer's data, bank account,
> credit card info, and such in plain text on a IIS server.  The response
> was "why not, everyone else does".  And they ignored my warning and did
> just that.  Don't know the result, the business relationship didn't last
> much longer.
>

And then you have the cloud. Take all your data and place it in the
hands of someone you have no reason to trust.

bill

On 12/14/21 10:51 AM, Arne Vajhøj wrote:
> On 12/14/2021 11:39 AM, Stephen Hoffman wrote:

>> Early reports that Java 8 and newer were
>> not vulnerable were later found incorrect.
>
> So you are saying that the deserialization protections done in 9 and
> backported to 8u121 are not enough to prevent this vulnerability?

Those protections block one of several deserialization mechanisms. It
helps a little for what I understand to be the easiest way to execute
remote code. But we're way beyond what's easiest for the state actors
and others who are investing a lot of resources into exploiting this in
the wild. So do upgrade Java. But don't stop there.

On 12/14/2021 12:43 PM, Craig A. Berry wrote:
> On 12/14/21 10:51 AM, Arne Vajhøj wrote:
>> On 12/14/2021 11:39 AM, Stephen Hoffman wrote:
>>>  Early reports that Java 8 and newer were not vulnerable were later
>>> found incorrect.
>>
>> So you are saying that the deserialization protections done in 9 and
>> backported to 8u121 are not enough to prevent this vulnerability?
>
> Those protections block one of several deserialization mechanisms.  It
> helps a little for what I understand to be the easiest way to execute
> remote code.  But we're way beyond what's easiest for the state actors
> and others who are investing a lot of resources into exploiting this in
> the wild.  So do upgrade Java. But don't stop there.

The best obviously is to upgrade log4j.

Nobody needs that feature causing the vulnerability (obviously
except whoever introduced it).

Arne

On 12/14/2021 11:39 AM, Stephen Hoffman wrote:
> HPE has indicated that 3PAR and some other products are vulnerable to
> this mess,

Which is a pretty good indication of how big the problem is.

3PAR is not exactly what one assocoate with a Java server.

Some security guy was out in the medias today and say that they expect
it to take 2 years before everyone has patched.

Arne

On 12/14/2021 11:39 AM, Stephen Hoffman wrote:
> HPE has indicated that 3PAR and some other products are vulnerable to

Seems HPE now reporting that the 3PAR StorServ is not vulnerable.

https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00120086en_us

On 2021-12-14, Arne Vajhøj <arne@vajhoej.dk> wrote:
>
> The best obviously is to upgrade log4j.
>
> Nobody needs that feature causing the vulnerability (obviously
> except whoever introduced it).
>

I wonder how that feature got past a design review ?

I wonder if there were too many layers involved for someone to be
able to connect the dots ?

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 2021-12-14, Arne Vajhøj <arne@vajhoej.dk> wrote:
> On 12/14/2021 11:39 AM, Stephen Hoffman wrote:
>> HPE has indicated that 3PAR and some other products are vulnerable to
>> this mess,
>
> Which is a pretty good indication of how big the problem is.
>
> 3PAR is not exactly what one assocoate with a Java server.
>
> Some security guy was out in the medias today and say that they expect
> it to take 2 years before everyone has patched.
>

Based on previous major events, that would not surprise me.

Perhaps now we might actually move away from needing to use 5 zillion
layers to be able to write a modern version of a Hello World application...

Simon.

--
Simon Clubley, clubley@remove_me.eisner.decus.org-Earth.UFP
Walking destinations on a map are further away than they appear.

On 12/14/21 3:12 PM, Simon Clubley wrote:
> On 2021-12-14, Arne Vajhøj <arne@vajhoej.dk> wrote:
>>
>> The best obviously is to upgrade log4j.
>>
>> Nobody needs that feature causing the vulnerability (obviously
>> except whoever introduced it).
>>
>
> I wonder how that feature got past a design review ?
>

People do design reviews?

> I wonder if there were too many layers involved for someone to be
> able to connect the dots ?

bill

On 2021-12-14 19:56:24 +0000, Jim said:

> On 12/14/2021 11:39 AM, Stephen Hoffman wrote:
>> HPE has indicated that 3PAR and some other products are vulnerable to
>
> Seems HPE now reporting that the 3PAR StorServ is not vulnerable.
>
> https://support.hpe.com/hpesc/public/docDisplay?docId=emr_na-a00120086en_us

You're looking at the "not vulnerable" list from HPE.

You'll also want to review the "vulnerable" list from HPE, for some
problematic 3PAR and XP apps.

https://support.hpe.com/hpesc/public/docDisplay?docLocale=en_US&docId=hpesbgn04215en_us

As for why log4j has this particular feature, there was a recent report
that the maintainers tried to remove these misfeatures, and ran afoul
of compatibility requirements.

The same-origin logic in this same neighborhood of code makes for an
interesting read, too—it's somewhere between un-robust and un-reliable.

There's what seems a robust workaround for the jndi flaw included with
the second of the two CVEs for the log4j code; with CVE-2021-45046.
This if you can't upgrade to the latest log4j.

zip -q -d log4j-core-*.jar
org/apache/logging/log4j/core/lookup/JndiLookup.class

--
Pure Personal Opinion | HoffmanLabs LLC

On 12/14/2021 3:12 PM, Simon Clubley wrote:
> On 2021-12-14, Arne Vajhøj <arne@vajhoej.dk> wrote:
>> The best obviously is to upgrade log4j.
>>
>> Nobody needs that feature causing the vulnerability (obviously
>> except whoever introduced it).
>
> I wonder how that feature got past a design review ?

Good question. It should not have.

> I wonder if there were too many layers involved for someone to be
> able to connect the dots ?

There are some code in loggers, but the architecture is pretty simple:

|--multiple formatters
core--|
|--multiple appenders

Arne

So the vulnerability is there. How would one take advantage of it anyway?
Wouldn't you need a privileged account to even get into the server to
start taking advantage of the flaws?
Assuming one is pretty careful about access to the network, and the
directory permissions are controlled, wouldn't it be hard to get to do
any damage?

Curious as I cannot find much information outside of the warning. I
guess no one wants to give hackers the keys to the rolls....

DT

>>>> The version of Java does not impact a bug in log4j more than
>>>> the version of C compiler impact a buffer overrun in a C library.
>>>
>>> We keep hearing this mantra over and over.  If it really is
>>> such a problem why has no one ever bothered to write a new
>>> library keeping the original APIs while internally removing
>>> the overrun problem?  Oh wait, someone did. Back in the early
>>> 80's.  On the PDP-11.  For all the DEC OSes and Ultrix-11
>>> and Version 7 Unix.  And there was even a version for the VAX.
>>> How did that work out?
>>>
>>> bill
>>
>> Way too many people don't really care about security ... until it
>> bites them on the ass.  Then they expect a law to be passed that will
>> protect them.  As if the hackers really care about laws.
>>
>> One moment I'll never forget.  I was telling a customer that it would
>> be a very bad idea for them to store their customer's data, bank
>> account, credit card info, and such in plain text on a IIS server. 
>> The response was "why not, everyone else does". And they ignored my
>> warning and did just that.  Don't know the result, the business
>> relationship didn't last much longer.
>>
>
> And then you have the cloud.  Take all your data and place it in the
> hands of someone you have no reason to trust.
>
> bill
>
So

On 2021-12-15 03:06:43 +0000, David Turner said:

> So the vulnerability is there. How would one take advantage of it anyway?
> Wouldn't you need a privileged account to even get into the server to
> start taking advantage of the flaws?
> Assuming one is pretty careful about access to the network, and the
> directory permissions are controlled, wouldn't it be hard to get to do
> any damage?
>
> Curious as I cannot find much information outside of the warning. I
> guess no one wants to give hackers the keys to the rolls....

It's a full remote command execution flaw (RCE), meaning that pretty
much anything that the Java app has access to is also exposed to the
attacker, if the attacker can get the exploit text string to the
logging software.

Access to the vulnerable logger can be via host name string, by HTTP
headers, or by embedding the text into other data streams, depending on
the app involved. It varies. Widely.

What can you do with the vulnerability? There are reportedly already
cryptocurrency miners and ransomware efforts underway using the
vulnerability, and I expect we'll see a long tail of more...

An intro to the log4j vulnerability from the Swiss government:
https://www.govcert.ch/blog/zero-day-exploit-targeting-popular-java-library-log4j/

List of affected products (re-post) :
https://gist.github.com/SwitHak/b66db3a06c2955a9cb71a8718970c592

One of the security vendors with some info:
https://www.lunasec.io/docs/blog/log4j-zero-day-mitigation-guide/

There's a Python detection tool at that last URL, though I've not
checked to see if that might be reasonably portable to OpenVMS.

Been working with some OpenVMS folks on this, and y'all with Java
installed and Java apps in use will want to take a look at what's in
those jars. Or if you have Java installed and don't need it, best
remove it.

--
Pure Personal Opinion | HoffmanLabs LLC

On 12/14/2021 10:06 PM, David Turner wrote:
> So the vulnerability is there. How would one take advantage of it anyway?
> Wouldn't you need a privileged account to even get into the server to
> start taking advantage of the flaws?
> Assuming one is pretty careful about access to the network, and the
> directory permissions are controlled, wouldn't it be hard to get to do
> any damage?
>
> Curious as I cannot find much information outside of the warning. I
> guess no one wants to give hackers the keys to the rolls....

The vulnerability is in the processing of log messages in
a logging framework.

To exploit the attackers need to get a specific message
logged.

And that is usually pretty easy. The primary purpose of logging
is to help troubleshooting problems and to do that it makes sense
to log user input.

It does not matter how the user input comes into the system.

The two most common ways are probably:
* web service calls coming in from the internet
* web form post coming in from internet

But it could also be an insider instead of internet.

And it could be plain socket or a message queue or a file
instead of HTTP(S).

Because the problem is in the data content, then traditional
isolation often doesn't help.

--firewall--Java web service with log4j

--firewall--Apache httpd proxy--firewall--Java web service with log4j

--firewall--Apache httdd proxy--firewall--PHP web service--message
queue--Java backend with log4j

--firewall--Apache httdd proxy--firewall--PHP web
service--database--Python job--message queue--Java backend with log4j

does not make a difference if that Java code calls log4j with
user input with malicious content then bad things can happen.

To move on with the exploit the Java code need to be able to
reach a server controlled by the attacker, but many places
only have strict control of inbound traffic not outbound traffic.

And the Java code need to have access to something. But if the
code is intended to do something on behalf of the user then it
obviously need to have access to user data. That it may not have
access to modify OS does not stop bad things from happening.

Arne

I wonder if there is anything that can be done in the .htaccess file

I have most countries outside of our business realm blocked so they
cannot even connect to the website
Since these attempts are typically made by a few malicious players in a
few countries, blocking access via country and ip has really helped us

On 12/15/2021 10:22 AM, Arne Vajhøj wrote:
> On 12/14/2021 10:06 PM, David Turner wrote:
>> So the vulnerability is there. How would one take advantage of it
>> anyway?
>> Wouldn't you need a privileged account to even get into the server to
>> start taking advantage of the flaws?
>> Assuming one is pretty careful about access to the network, and the
>> directory permissions are controlled, wouldn't it be hard to get to
>> do any damage?
>>
>> Curious as I cannot find much information outside of the warning. I
>> guess no one wants to give hackers the keys to the rolls....
>
> The vulnerability is in the processing of log messages in
> a logging framework.
>
> To exploit the attackers need to get a specific message
> logged.
>
> And that is usually pretty easy. The primary purpose of logging
> is to help troubleshooting problems and to do that it makes sense
> to log user input.
>
> It does not matter how the user input comes into the system.
>
> The two most common ways are probably:
> * web service calls coming in from the internet
> * web form post coming in from internet
>
> But it could also be an insider instead of internet.
>
> And it could be plain socket or a message queue or a file
> instead of HTTP(S).
>
> Because the problem is in the data content, then traditional
> isolation often doesn't help.
>

> --firewall--Java web service with log4j
>
> --firewall--Apache httpd proxy--firewall--Java web service with log4j
>
> --firewall--Apache httdd proxy--firewall--PHP web service--message
> queue--Java backend with log4j
>
> --firewall--Apache httdd proxy--firewall--PHP web
> service--database--Python job--message queue--Java backend with log4j
>
> does not make a difference if that Java code calls log4j with
> user input with malicious content then bad things can happen.
>
> To move on with the exploit the Java code need to be able to
> reach a server controlled by the attacker, but many places
> only have strict control of inbound traffic not outbound traffic.
>
> And the Java code need to have access to something. But if the
> code is intended to do something on behalf of the user then it
> obviously need to have access to user data. That it may not have
> access to modify OS does not stop bad things from happening.
>
> Arne
>
>
>
>
>
>
>
>
>
>

On 12/15/2021 12:51 PM, David Turner wrote:
> I wonder if there is anything that can be done in the .htaccess file

I don't think so.

It must be possible to come up with blocking rules for mod_security though.

> I have most countries outside of our business realm blocked so they
> cannot even connect to the website
> Since these attempts are typically made by a few malicious players in a
> few countries, blocking access via country and ip has really helped us

That can help against amateur hackers.

It does not help against the more professional hackers.

This vulnerability is supposedly already being exploited by what
is known as "state actors".

Arne