I just spent a frustrating morning configuring our servers to talk NFS to each other properly. So we have:
1) NFS servers (ceph-mon1 and so on) running CentOS 7.
2) NFS clients (gridj1 and so on) running Ubuntu 14.04.
The first problem: RBD mounting and NFS startup were not configured on the servers. I fixed that by adding entries in
/etc/ceph/rbdmap and enabling the rbdmap and nfs-server services using
systemctl enable. I also used e2label to label the ext4 filesystems in the RBDs and then used these labels in
/etc/fstab instead of device names. And used the _netdev option in the mount options because these devices are network devices.
The second problem: I had to add the insecure option to the exports in
/etc/exports. This is because the mount request comes from a port higher than 1024, a so-called insecure port. And then exportfs -r to resync everything.
And the third problem: Ubuntu autofs makes a NFS4 mount request by default (even though I had specified nfsvers=3 in the mount options), and I haven’t configured NFS4’s authenticated mounts, so I was getting authenticated mount request from 192.168.6.71:862 messages in /var/log.messages on the NFS server. I switched the NFS server to not do NFS4 by adding
--no-nfs-version 4 to the RPCNFSDARGS variable in
/etc/sysconfig/nfs on the server, restarted the NFS server (systemctl restart nfs-server) and the mounts finally worked.
Finally, documented this here for posterity…