>We'd like to be able to leverage 2fa for some services (admins) and some
>services (ssh logins) but not have to pump a 2fa code into, say, our mail
>applications. Is there a way to make the acquisition of a TGT (for GSSAPI
>authentication) vs Password Authentication require 2fa?

Yes (I'll explain more below).

>That's complication number one.
>
>Complication number 2 is something like "SecurID is *expensive* for a
>fairly small (<10) admin team."

Yeah, tell me about it.

>Is there any reasonable support for off-the-shelf TOTP or HOTP
>authenticators, i.e. google authenticator or whatnot? If so, is there
>support to have a user have *multiple* available authenticators, such that
>one can be expired and others not?

There is not such a thing yet, AFAIK. It's something on my list to
implement TOTP support (but, it's kinda far down). It's complicated
(again, see below).

>Googling this all gets me a bunch of (some older, some newer articles
>about the varying states of SPAKE and the like), and...a whole bunch of
>ads now being shown for startups that want to do it differently but I'm
>SURE no way to integrate with this.
>
>The final problem, of course, is that if I make all my KDC's 2fa-aware on
>their own, there's no communication of double-use of a token, unless I
>centralize things, which breaks the purpose of having geo-diverse KDC's.
>I don't suppose the kerberos db replication mechanism has anything that
>can also share this state?
>
>This is all pie-in-the-sky stuff, but practical answers "just an FAQ" are
>hard to find.

So, I think I speak with some authority on this subject; I've been involved
with some kind of 2FA with Kerberos for a ... couple of decades? Yikes,
I feel old. Anyway, the super short summary is:

- You can do it, but depending on exactly WHAT you want to do may
involve writing some code. You've heard the expression, "With a big
enough engine, you can make a barn door fly"? That applies here.
- There are some architectural issues that makes this all complicated
to deploy. I'll try to touch on those.

The first question you need to answer is: what are your clients? This
is important because which specific 2FA protocol you use depends on
what clients you have. The 3 main protocols I am aware of are

- SAM2 - this is the old protocol that we originally used waay back
when we started (and we still use today for some factors). You can
pretty much use any factor with it; we used SecurID, CryptoCARD,
and today use YubiKey. The major disadvantage is it is only supported
by MIT-based Kerberos clients. I did a small amount of the protocol
work on SAM2 but for various complicated reasons it never resulted
in a published RFC.

- FAST. This is a newer protocol that provides a generic encrypted
tunnel to send preauthentication messages over (these are called "FAST
factors"). It's a more complicated protocol, but superior to SAM2;
if it was around when we started we would have used it. It is supported
by MIT and Heimdal; it may be supported by Windows, I am not sure.
There is already a FAST factor defined for One-Time Passwords, called
"OTP". I am not sure of the client coverage of the OTP FAST
factor, though. I will admit that I do not know as much about FAST as
the other mechanisms, so if people chime in and correct me you should
believe them over me.

- PKINIT. This uses an existing PKI to completely replace a password. I
mention this as 2FA since in our deployment we use client certificates
on smartcards which require PINs to unlock, hence the two factors.
This is supported by all 3 major Kerberos implementations (well, maybe
not the smartcard piece, but MIT Kerberos does, except on Windows).
If you can leverage an existing PKI then it's not a bad choice, but
it is a complicated mess to get going; there are SO MANY ways for
it to go wrong.

So, which protocol should you use? Well, even though I hold a soft spot
for SAM2, I can't really recommend using it for new implementations;
ours is really just due to legacy code. So that's either FAST OTP or
PKINIT. However ... FAST does require a special "FAST armour key"
(which is typically a host key) or support for anonymous PKINIT.
Either of those requires a bit of magic at kinit time, which I haven't
completely sussed out all yet. I'm not actually sure if FAST OTP has
been deployed anywhere; it may be! If so, no one talks about it.
One advantage of SAM2 is that it "just works" when you enable it
(assuming everything else is set up correctly).

There is an existing FAST OTP implementation for MIT Kerberos; the
documentation is here:

https://web.mit.edu/kerberos/www/krb5-latest/doc/admin/otp.html

The implementation requires you to verify the token using a RADIUS
server. If you do not like that, you'd have to write your own
implementation (and, sadly, that requires Kerberos internals; sigh, I
wish that interface was better defined).

In terms of using it for access to certain hosts, you have a couple
of options. The preauth mechanism can set what are known as "authentication
indicators" in the ticket; you can either configure a host principal
to only allow the acquisition of a service ticket if it has a particular
indicator, or you can modify the server software so it checks for
a particular indicator. We used a variation of the second approach;
there is a Kerberos ticket flag called HW_AUTH, and our application server
code checks for that on the subset of hosts that require it (this was
before authentication indicators were developed as a protocol extension).

Now, in terms of "challenges" ... here's my understanding of the
practical issues you will face:

- As you note, talking to an external server kills the KDC redundancy.
And a lot of OTP mechanisms require this. In the larger deployment
I have worked on, we supported three OTP hardware tokens: SecurID,
CRYPTOCard, and YubiKey. With CRYPTOCard, I wrote that mechanism and
put the "secret" directly in the KDC. That worked, but it had an
unfortunate wart that if you talked to a replica KDC there was no way
of propagating the "used" token back to the primary (since 99% of the
requests went to the primary KDC, it never ended up being a problem in
practice, but I knew it was an issue and never figured out a reasonable
solution). With SecurID and YubiKey, the KDC ends up talking to external
verification servers. For various dumb reasons the SecurID server
implementation never worked well, but the YubiKey one seems to work
pretty well. It does have the obvious problem that if the YubiKey
server goes down then you can't kinit if you're using YubiKey (and
that has happened to us).

- There's not a way to OPTIONALLY say, "I'd like to use OTP now". You can
set it on a per-user basis, but kinit (and the underlying API) doesn't
have the ability to say, "Please let me use OTP for this request".
We have some local hacks to enable this functionality. In discussions
on krbdev I believe MIT was open to this functionality, but the
actual WORK hasn't been done. I was the obvious one to do this work,
but I just submitted a pullup request for a very simple change that I
said I'd do back in May, so ... I'm behind. Discussion on this is
here:

http://mailman.mit.edu/pipermail/krbdev/2021-April/013448.html

Now, strangely enough, even though PKINIT is a giant pain in the ass
it solves a lot of these issues (admittedly, by foisting them off onto
a PKI, but hey ...). So if you want to use PKINIT, that might be a
way to go. In terms of second factors, have you looked at the YubiKey
tokens? The latest ones can act as a PIV token, and you should be able
to get PKINIT working with them (I will admit I've played around with
a YubiKey token and I see it shows up as a PIV token, but I didn't get
a chance to go farther). In theory you could implement the YubiKey
verification directly in the KDC, but that has all of the issues with
the CRYPTOCard support I mentioned.

I hope it doesn't sound like I'm crapping on people here; a lot of
very smart people did a lot of very good work in terms of the protocol
and the code framework for implementing multi-factor authentication in
Kerberos. It is not an easy problem! But the documentation on it all
is a bit ... diffused.

Anyway, I hope that helps.

--Ken