Gentoo NFSv4 and MIT Kerberos

I was unable to find any guides which accurately described setting up a NFSv4 client with Kerberos on Gentoo. There are guides for setting things up on other distros, but I have run into numerous issues which were directly related to using Gentoo. Therefore, I am going to use this guide to document some of those problems. Please note that the NFS server is running Ubuntu 10.04, so there are some parts of this guide which won’t apply to Gentoo.

Setting up the Kerberos server is fairly straightforward, however, there is a difference in the way things are compiled on Gentoo. The OpenAFS guide on the wiki is mostly correct. I’ll reiterate the correct steps here.

Installation

First, you need to install the Kerberos server.

emerge -av mit-krb5

Copy the /etc/krb5.conf.example file that is included over to /etc/krb5.conf and edit it according to your needs.

cp /etc/krb5.conf.example /etc/krb5.conf

The edited file will look similar to this

[libdefaults]
        default_realm = EXAMPLE.COM
        forwardable = true
        renew_lifetime = 7days

[realms]
        EXAMPLE.COM = {
                kdc = krb.example.com
                admin_server = krb.example.com
        }

[domain_realm]
        .example.com = EXAMPLE.COM
        example.com = EXAMPLE.COM

You will need to replace “EXAMPLE.COM”, “example.com”, and “krb.example.com” with appropriate values for your environment. Note that realm names are always uppercase. The name of your KDC (krb.example.com in the example) is arbitrary.

Setting up the primary KDC

This is where the OpenAFS guide is confusing. The kdc.conf file should reside at /var/lib/krb5kdc/kdc.conf, not /etc/kdc.conf. So, go ahead and copy /var/lib/krb5kdc/kdc.conf.example and create a new file. Here are what the contents should look like.

[kdcdefaults]
        kdc_ports = 750,88

[realms]
        EXAMPLE.COM = {
                database_name = /var/lib/krb5kdc/principal
                admin_keytab = FILE:/var/lib/krb5kdc/kadm5.keytab
                acl_file = /var/lib/krb5kdc/kadm5.acl
                key_stash_file = /var/lib/krb5kdc/.k5.EXAMPLE.COM
                kdc_ports = 750,88
                max_life = 10h 0m 0s
                max_renewable_life = 7d 0h 0m 0s
                default_principal_flags = +preauth
        }

[logging]
        kdc = FILE:/var/log/kerberos/kdc.log
        admin_server = FILE:/var/log/kerberos/kadmin.log

Replace “EXAMPLE.COM” with your own realm name. Also note that some of the options above are changed from their default values. I have added a logging section at the end and changed the directory where things reside.

An important difference is that the default_principal_flags has been set to +preauth. The reason for this is that without it, Kerberos is vulnerable to offline dictionary attacks. If you are going to have your KDC publicly accessible, then you definitely want to consider enabling preauthentication. In my opinion, you probably want this even if the KDC is not publicly accessible, but that’s because I trust no one.

After modifying /var/lib/krb5kdc/kadm5.acl to your liking, you can go ahead and create the database.

cd /var/lib/krb5kdc
kdb5_util create -r EXAMPLE.COM -s

As usual, make sure you use your realm name.

Principal Creation

I’ll leave this as an exercise for the reader. I generally create varying policies for services and users and those won’t be entirely useful for most. For a really good guide on creating/using policies, see http://techpubs.spinlocksolutions.com/dklar/kerberos.html#id2500817.

Start Kerberos Server

To start the kdc and kadmind servers, run the following.

/etc/init.d/mit-krb5kadmind start
/etc/init.d/mit-krb5kdc start

Add them to the default runlevel so that they start up after a reboot

rc-update add mit-krb5kadmind default
rc-update add mit-krb5kdc default

Installing NFSv4 client

First install the nfs client utilities

emerge -av nfs-utils

You will want to make sure you have both the kerberos and the nfsv4 USE flags enabled.

Configuring the kernel

You will need to configure the kernel with the appropriate relevant options. I won’t bother going through that entire process. Rather, I’ll point out some things that went wrong for me, but weren’t immediately obvious.

The kernel needs to have the rpcsec_gss_krb5 option configured as a module. I spent quite a while debugging this problem. I had the option compiled directly into the kernel. Looking in the nfs client’s syslog, I also found this obscure error message.

gss_create: Pseudoflavor 390003 not found!
RPC: Couldn't create auth handle (flavor 390003)

Whatever the hell that means. Surprisingly, there are very few references to this error. One of them I found suggested recompiling the kernel with the rpcsec_gss_krb5 module and simple loading it after boot. Surprisingly, this actually worked.

Adding nfs principals

Both the nfs server and the nfs client need nfs principals added to their krb5.keytab. Since my nfs server was running an older kernel (Ubuntu 10.04), I needed to do a couple things to get this to work.

First, you need to add an nfs principal for both the client and the server. In my case, the server needed an encryption type which isn’t generated by default on a Gentoo Kerberos server. Therefore, I generated the principal like this.

addprinc -policy service -randkey -e "des-cbc-crc:normal" nfs/nfsserver

Since I had a service policy defined, this created the nfs/www.siriad.com principal with the “des-cbc-crc” encryption type. This is necessary for the older version of nfs that is available for Ubuntu 10.04. You then need to login to the nfs server, run kadmin, and do the following.

kadmin:  ktadd -e des-cbc-crc:normal nfs/nfsserver

This will add the entry to your nfs server’s host keytab. Using this encryption type is extremely important. If you don’t, you will probably end up with very cryptic errors like the ones I had.

rpc.svcgssd: ERROR: prepare_krb5_rfc_cfx_buffer: not implemented
rpc.svcgssd: ERROR: failed serializing krb5 context for kernel
rpc.svcgssd: WARNING: handle_nullreq: serialize_context_for_kernel failed

This indicates that the NFS server has not implemented the encryption types being used in your keytab.

Now you just need to add an nfs principal for your client. In this case, Gentoo had support for the more recent encryption types, so I didn’t need to do anything special. I just created the principal.

addprinc -policy service -randkey nfs/nfsclient

then added it to the client’s host keytab using kadmin on the client

kadmin:  ktadd nfs/nfsclient

Lastly, you need to make sure you allow for weak encryption types in the /etc/krb5.conf file. Add the following to the [libdefaults] section.

allow_weak_crypto = true

Setting up the NFS server

First, you need to allow for weak encryption types on the NFS server. You can do this by modifying the /etc/krb5.conf file. You will need to add the following two lines in the [libdefaults] section.

allow_weak_crypto = true
permitted_enctypes = "des-cbc-crc arcfour-hmac des3-cbc-sha1 aes128-cts-hmac-sha1-96 aes256-cts-hmac-sha1-96"

Note that the values listed as permitted are those generated by default on my Kerberos server. Please DO NOT set the default encryption type to the weak encryption. I see far too many howtos that tell you to do this and it is NOT a good idea. If you can use the stronger encryption for things other than NFS, there is no reason not to.

On the NFS server, you also need to make sure that rpc.svcgssd is set to start alongside NFS. On Ubuntu, you can do this by editing your /etc/default/nfs-kernel-server file and editing/modifying the following line.

NEED_SVCGSSD=yes

You will also need to edit the following line in the /etc/default/nfs-common file.

NEED_IDMAPD=yes

Edit the /etc/idmapd.conf file and set the Domain line to the appropriate value for your environment. Make sure you restart rpc.idmapd if necessary.

Lastly, you need to modify /etc/exports with the appropriate values. My export looks something like this.

/export/dir        gss/krb5(rw,fsid=0,insecure,no_subtree_check)

You can then restart the nfs-kernel-server service and your NFS server should be ready to go.

Setting up the NFS client

You need to first make sure that rpc.idmapd and rpc.gssd are set to start with nfs. Edit your /etc/conf.d/nfs file and modify the following line.

NFS_NEEDED_SERVICES=”rpc.idmapd rpc.gssd”

You will need to edit /etc/idmapd.conf with the same information from the NFS server. Then you can /etc/init.d/nfs restart and test your NFS mount.

Testing your NFS mount

You can now test your nfs mount with the following command

 mount -vvv -t nfs4 -o sec=krb5 nfsserver:/ test/

This should work successfully and you should be able to see the appropriate requests coming through in your KDC logs.