Shared File Storage and NFS in the Cloud

First, Check Whether You Really Need It

Shared file storage is the answer to one specific requirement: several machines need to read and write the same files, through ordinary filesystem calls, at the same time. If that is genuinely your situation, read on. If it is not, a shared filesystem will cost you money and introduce a failure mode for no benefit.

The requests that lead here fall into two groups. The legitimate ones include content management systems whose plugin architecture writes to disk, legacy applications that cannot be modified, media pipelines where several workers process the same large files, and shared home directories for a team of users.

The illegitimate ones are more common. User uploads on a horizontally scaled web application belong in object storage, not on a share. Sessions belong in a cache or a database. Logs belong in a log pipeline. Build artefacts belong in a registry or an artefact store. In each case the shared filesystem is being used to avoid changing the application, which is a real reason, but it should be a conscious one.

The distinction between the two storage models is laid out in block storage versus object storage, and it is worth settling before adding a third model to the stack.

What a Network Filesystem Actually Guarantees

NFS presents a remote directory tree as though it were local. Applications open, read, write and lock files with the same system calls they would use on an attached disk, which is exactly why it solves the legacy problem so neatly.

The guarantees are weaker than a local filesystem's, and the differences cause the confusing bugs.

Caching makes consistency approximate. Clients cache attributes and data for a short interval, so a file written on one host may not be visible on another immediately. Mount options control the window, and tightening them costs performance.

Locking is a separate protocol with real edge cases. NFSv4 improved this considerably over v3 by integrating locking and handling client recovery, which is one reason to prefer v4 for anything that depends on locks.

Latency is per operation, not per byte. Every stat, every directory listing, every open is a network round trip. A workload that touches thousands of small files feels dramatically slower than the same work on a local disk, even when throughput looks fine. This single fact explains most disappointment with shared storage.

Identity mapping matters. Ownership travels as numeric identifiers unless you configure otherwise, so the same user needs the same uid across every client or permissions will behave unpredictably.

Three Ways to Get It

The managed services are the mainstream answer where they exist. Amazon EFS presents an elastic NFSv4 filesystem across availability zones and is described in the EFS documentation. Azure Files offers both SMB and NFS shares, covered in the Azure Files introduction. Google Cloud Filestore and OVHcloud's managed NAS fill the same role on their platforms.

The developer clouds mostly do not offer a managed NFS product, which surprises people arriving from a hyperscaler. There the options are running your own server or using a storage appliance such as a Hetzner Storage Box, which speaks SFTP, Samba and WebDAV rather than NFS but covers many of the same needs.

The third row deserves a warning. Mounting a bucket with a FUSE driver looks like a shared filesystem and is not one. It is excellent for read-mostly access to large files and genuinely dangerous under concurrent writes, because a rename is a copy and a delete, and locking does not exist. Use it knowingly or not at all.

Mounting It Properly

Whether the export comes from a managed service or your own server, the client side is the same and a few choices matter.

Mount over the private network, never over a public address. NFS has no meaningful transport security in common configurations, so network isolation is the security boundary. The setup is covered in private networking between cloud servers.

A reasonable entry in /etc/fstab specifies the version explicitly and sets sensible timeouts:

10.0.1.50:/exports/shared /mnt/shared nfs4 rw,hard,timeo=600,retrans=2,noatime,_netdev 0 0

Four of those deserve explanation. hard makes a client retry indefinitely when the server is unreachable rather than returning errors that most applications handle badly; it means processes block during an outage, which is usually the lesser evil. _netdev tells the system this mount needs the network, preventing a boot that hangs waiting for it. noatime removes a write for every read. And naming nfs4 avoids negotiating down to an older version by accident.

On the server side, keep the export list narrow. Export to specific client addresses rather than a subnet where practical, and think carefully before using no_root_squash, which lets root on a client act as root on the exported files.

Then treat the share as a dependency in your monitoring. A hung mount presents as an application that stops responding with no obvious cause, so an explicit check that reads a file from the share is worth more than any server-side metric.

Making It Fast Enough

Most complaints about shared storage performance come down to the round trip cost described earlier. The fixes follow from that.

• Keep clients and server in the same zone where availability allows. Crossing zones adds latency to every single operation, and on some providers it adds metered traffic as well.
• Increase the read and write sizes. The rsize and wsize options control how much data moves per request; larger values mean fewer round trips for sequential work.
• Avoid deep directory trees and huge directories. Listing a directory with a hundred thousand entries over the network is painful, and applications do it more often than their authors realise.
• Cache aggressively where correctness permits. Static assets read from a share should be cached by the web server or served through a CDN so the share is touched once, not per request.
• Put the hot path on local storage. Databases, build scratch space and anything latency-sensitive belong on an attached volume, as the NVMe storage article explains.

Never run a database on NFS. The locking and consistency semantics are not what the engine assumes, and the failure mode is corruption rather than slowness.

Questions About Shared Storage