File Storage #

In cloud compute architecture, File Storage (often called Shared File Systems) is a managed storage service presenting data as a standard hierarchy of directories and files. Unlike block storage, which mounts as an exclusive raw disk device for one virtual machine, or object storage, which is accessed flatly via HTTP APIs, file storage is specifically designed to support concurrent access from many virtual instances simultaneously (multi-instance concurrent access). This service follows standard filesystem semantics (like POSIX), so our applications can read, write, create folders, and lock files concurrently using standard network protocols like NFS (Network File System) for Linux or SMB (Server Message Block) for Windows. This article deeply dissects the working mechanisms, protocols, performance comparisons, and practical guidance for choosing and optimizing file storage in the cloud.

How File Storage Works and Its Architecture #

Physically, file storage in cloud environments is implemented as a distributed storage cluster fully managed by the cloud provider (managed NAS/Network Attached Storage). This cluster exposes one or more network access points (mount targets) inside our Virtual Private Cloud (VPC). Our Virtual Machine (VM) instances connect to those mount targets using network filesystem protocols.

flowchart TD
    subgraph VPC ["Virtual Private Cloud (VPC)"]
        VM1["App Server Instance 1<br>(Linux /mount/shared)"]
        VM2["App Server Instance 2<br>(Linux /mount/shared)"]
        VM3["App Server Instance 3<br>(Linux /mount/shared)"]
        
        MountTarget["Network Mount Target<br>(Private IP: 10.0.1.50)"]
        
        VM1 -->|"NFS v4 (TCP Port 2049)"| MountTarget
        VM2 -->|"NFS v4 (TCP Port 2049)"| MountTarget
        VM3 -->|"NFS v4 (TCP Port 2049)"| MountTarget
    end
    
    subgraph StorageService ["Cloud Managed File Storage Service"]
        Controller["Storage Controller Layer<br>(Metadata & Lock Management)"]
        PhysicalDisks["Physical Storage Pool<br>(Redundant SSD/HDD Array)"]
        
        MountTarget --> Controller
        Controller --> PhysicalDisks
    end

Synchronization and Concurrent Access Mechanisms #

When Instance 1 writes a new file to the /mount/shared/data.txt directory, the following operation sequence happens:

1. OS Translation: The application on Instance 1 calls a standard filesystem API (like write() on Linux). The operating system’s kernel driver converts this operation into an NFS/SMB network protocol command.
2. Network Delivery: The command is sent over the private network TCP connection to the Mount Target.
3. Redundant Storage: The storage controller receives the data, updates filesystem metadata (like inodes, file sizes, and timestamps), writes the data blocks to replicated physical disks, and sends a response back.
4. Instant Visibility: When Instance 2 makes a read() call on the same file location milliseconds later, the command goes to the same access target and immediately gets the latest data written by Instance 1.

This capability is called read-after-write consistency across all connected instances. It’s crucial for distributed web application architectures where one user uploads an image through Server 1, and another user must immediately download that image from Server 2.

Main Protocols: NFS and SMB/CIFS #

Cloud providers usually separate their file storage services based on the target operating system and network protocol used. Choosing the wrong protocol can badly impact feature compatibility and application performance.

1. NFS (Network File System) #

NFS is the de facto standard protocol for file sharing in Unix and Linux environments. The newest versions commonly used in the cloud are NFS v4 (NFSv4.1 or NFSv4.2).

• Advantages: Fully supports POSIX features including detailed file access permissions (user/group permissions), symbolic links, file locking, and stateful client connection detection.
• Security: Access is controlled at the network level using VPC firewall rules (security groups) and internal cloud authorization policies (like IAM policies).
• Linux Mount Options: For optimal performance and stability during network disruptions, we must carefully set the mount options in the /etc/fstab file.

# Example of an optimized NFSv4 mount configuration on Linux (/etc/fstab)
# CORRECT: Using optimization options for performance and network failure tolerance
fs-xxxxxx.efs.ap-southeast-1.amazonaws.com:/ /mnt/shared nfs4 defaults,rsize=1048576,wsize=1048576,hard,timeo=600,retrans=2,noresvport,_netdev 0 0

# Explanation of the options above:
# - rsize & wsize: Set the maximum data block size (1 MB) for read/write operations to reduce network overhead.
# - hard: Ensures the OS keeps retrying if the connection to storage drops, preventing app crashes from lost disk writes.
# - timeo=600: Waits 60 seconds before retrying after a timeout.
# - retrans=2: Makes a maximum of 2 retry attempts before considering a timeout.
# - _netdev: Tells the OS to defer mounting this disk during boot until the network card is active.

2. SMB (Server Message Block) / CIFS #

SMB is the native protocol developed by Microsoft for Windows. In the cloud, services like Azure Files or AWS FSx for Windows File Server provide fully managed SMB implementations.

• Active Directory Integration: SMB supports full integration with Microsoft Active Directory (AD). This means we can apply very specific folder access permissions based on users and groups in our company domain using Windows’ built-in NTFS Access Control Lists (ACLs).
• Advanced Features: Supports in-transit encryption natively starting from SMB 3.0, plus SMB Multichannel optimization techniques combining multiple network connections to multiply throughput.

# Example PowerShell command to mount an SMB share on Windows Server
# ✓ CORRECT: Using SMB 3.0 encryption and storing credentials securely
New-SmbMapping -LocalPath "Z:" -RemotePath "\\fs-xxxx.file.core.windows.net\sharedfolder" -UserName "AD\admin_user" -Password "SecurePassword123"

Comprehensive Comparison: Object vs Block vs File Storage #

To help us choose the right storage technology, let’s compare the fundamental characteristics of all three storage types in the following table:

(Note: Some cloud providers offer Multi-Attach Block Storage, but it requires special distributed filesystems like GFS2 or OCFS2 at the application level to avoid data corruption).

When to Choose File Storage (Ideal Scenarios) #

File storage is ideal when our applications need the following characteristics:

1. Legacy Application Migration (Lift-and-Shift) #

Many old enterprise applications (legacy apps) designed to run on local servers assume they have access to a local filesystem. These applications often use code operations like opening files at /var/app/uploads/data.csv, writing to the middle of files, or using OS-level file locks (POSIX file locks).

• The Solution: Rewriting that application code to use Object Storage APIs (like the AWS S3 SDK) would cost enormous development time and money. By mounting shared cloud file storage to the same directory on all new VM instances, those applications run directly with zero code changes.

2. Shared Data for Web Server Clusters (Shared Content) #

The classic scenario where we run a group of web servers behind a Load Balancer (like a WordPress or Drupal cluster).

• Need: When an admin uploads a new theme or a user uploads image files, those files must be instantly readable by all other web server instances.
• Implementation: By mounting the wp-content/uploads directory on network file storage, all web servers always see a consistent, real-time synchronized image list.

flowchart TD
    LB["Load Balancer"] --> Web1["Web Server 1"]
    LB --> Web2["Web Server 2"]
    
    subgraph Storage ["Shared Storage"]
        SharedFS["Shared File Storage<br>(NFS /mnt/uploads)"]
    end
    
    Web1 -->|"Write New Image"| SharedFS
    Web2 -->|"Read New Image"| SharedFS

3. Data Processing & Machine Learning Pipelines #

In some scientific data pipelines or machine learning (ML) training runs, we have large GPU compute instance clusters that need to read the same dataset for parallel analysis processing.

• Advantage: The giant dataset sits on one central file storage. Each worker node reads its own part of the dataset randomly and writes its analysis results to a shared output folder.

When to Avoid File Storage (Anti-Patterns) #

Although flexible, file storage has significant architectural limitations. Using file storage for unsuitable workloads leads to poor performance and cost bloat.

1. Don’t Use It for OLTP Databases (Transactional Databases) #

It’s very tempting to put our MySQL, PostgreSQL, or Oracle database data files on file storage, reasoning that data becomes easier to back up or the database can be easily switched to a standby server.

• ANTI-PATTERN: OLTP transactional databases perform small random write operations (e.g., 8 KB or 16 KB data pages) at very high continuous frequency. Because file storage is on the network, every database write incurs additional network round-trip latency. This triggers long I/O queues (write queue stagnation), drastic transaction performance drops, and file corruption risk from lock synchronization failures (lock contention).
• CORRECT: For databases, always use Block Storage (like AWS EBS gp3/io2 or Azure Premium SSD) attached directly to one database server instance. If data replication is needed, do it at the database level (like master-slave replication or active-passive database clustering architectures), not the storage level.

// ANTI-PATTERN: MySQL database data directory configuration on an NFS mount
[mysqld]
datadir=/mnt/shared-nfs/mysql/data  // DON'T: Triggers high network latency & data corruption!

// CORRECT: Data directory configuration on local block storage
[mysqld]
datadir=/var/lib/mysql              // ✓ CORRECT: Sub-millisecond latency with SSD block device

2. Don’t Use It for Inactive Data (Cold Data & Backups) #

Storing system backups (backup images), financial report archives from 5 years ago, or system logs on file storage is a huge budget waste.

• Cost Problem: File storage’s per-GB price is roughly 3 to 5 times more expensive than standard object storage, and can reach 10 to 20 times more when compared to archive-tier object storage (like Glacier Deep Archive).
• Solution: Write backup scripts that directly upload backup files to object storage using the cloud provider’s CLI, instead of keeping them in a mounted NFS folder.

3. Workloads with Millions of Small Files (Small File I/O Workloads) #

Workloads like Node.js dependency installation processes (npm install, which creates node_modules folders with millions of small javascript files) or large-scale C++ source compilation.

• Why Slow?: Every file operation (like checking file existence, opening files, reading metadata) requires a network round-trip. If a process must sequentially read 10,000 files of 1 KB each, the 5ms network latency per file accumulates to: $$10,000 \times 5\text{ ms} = 50\text{ seconds}$$ Whereas on local SSD block storage, the same operation takes less than 1 second.
• Solution: Run build and compilation processes in the VM’s local directory (e.g., /tmp using local ephemeral block storage), then compress the result into a single tarball (.tar.gz) and send that compressed file to shared file storage.

Performance Management: Throughput Modes and File Locks #

File storage performance in the cloud is governed by throughput determination rules and distributed file locking mechanisms.

1. Throughput Mode Characteristics #

Cloud providers usually offer several performance determination models for file storage:

• Bursting Mode: Default throughput scales up dynamically in direct proportion to the data capacity we store. The more data we store, the higher the base throughput we get. If we need high throughput for small data sizes, we’re given “burst credits” that can run out if used continuously.
• Provisioned Mode: We pay extra to lock a specific throughput (e.g., 100 MB/s) regardless of how little data we store. This mode is highly recommended for production web applications with stable traffic patterns.
• Elastic Mode (Auto-scaling): The service measures performance in real-time and automatically adjusts throughput allocation based on our application workload, with dynamic billing based on transaction volume.

2. Distributed File Locking Mechanisms #

When hundreds of VM instances write to the same filesystem, file locking is essential to prevent data overlap leading to file corruption. NFS v4 manages this using lease-based stateful locks.

• Advisory Locking: By default, locking is optional. Applications must actively request locks before writing files (e.g., using the flock() function in Python or Linux system calls).
• Lock Bottlenecks: If many instances try to write the same file simultaneously, they experience lock contention queues. The metadata controller gets busy managing the lock queue list, reducing our overall filesystem throughput.
• Architecture Recommendation: Design systems so different instances write to files with unique names (e.g., including instance ID and timestamp: data_worker01_1718910000.json), then use a separate background consolidation worker process to merge data if it needs to be unified.

Summary #

• File storage provides shared file systems mountable by thousands of virtual instances simultaneously using standard NFSv4 or SMB3 protocols.
• The service’s main niche is POSIX compatibility and instant data visibility consistently across distributed web application servers.
• Highly suitable for Lift-and-Shift migrations of enterprise applications dependent on folder structures and traditional OS file operations without code rewrites.
• It’s an anti-pattern for OLTP databases because high network round-trip latency ruins random database write performance and risks data corruption.
• Avoid sequential I/O of millions of small files (like code dependencies or compilation) because per-operation network protocol overhead drastically slows processes.
• Use optimized mount options on Linux (like large rsize/wsize settings and hard retry options) to ensure systems tolerate minor network disruptions.

← Previous: Block Storage   Next: Durability vs Availability →