Memory-Mapped Files in Operating Systems

Description
A memory-mapped file is a technique that maps part or all of a disk file's content directly into a process's virtual address space. After mapping, the process can read and write file data as if accessing ordinary memory (via pointers), without the need to invoke system calls like read/write. The operating system automatically handles data loading (from disk to memory) and flushing (from memory to disk) in the background through the virtual memory mechanism. This technology is widely used in file processing, inter-process shared memory, program loading, and other scenarios.

Key Points and Steps

1. Basic Concepts and Advantages

   • Core Idea: Map a file into the process's virtual address space, converting file I/O operations into memory accesses.
   • Advantages:
     • Reduced data copying: Avoids copying between kernel buffers and user buffers.
     • Simplified programming: Directly manipulate file data using pointers.
     • Efficient sharing: When multiple processes map the same file, they share physical memory pages (for inter-process communication).

2. Implementation Principles

   • Mapping Establishment:
     1. The process invokes a system call (e.g., Linux's mmap), specifying the file descriptor, mapping region size, and access permissions (read/write).
     2. The operating system allocates a contiguous virtual memory area (VMA region) in the process's virtual address space and establishes a mapping relationship with the file's disk blocks, without actually loading the file data at this stage.
     • Note: The virtual address space only records the mapping relationship; actual physical memory pages are allocated on-demand during page faults.
   • Data Loading (On-Demand Paging):
     1. When the process first accesses a virtual address within the mapped region, a page fault is triggered.
     2. The operating system checks whether the faulting address is within the mapped region:
        • If yes, it allocates a physical page frame, reads the corresponding file data block into that frame, and establishes a page table entry mapping the virtual address to the physical page.
        • If no, it handles the page fault conventionally (e.g., by allocating an anonymous page).
   • Data Write-Back:
     • After the process modifies data in memory, the operating system periodically (or via manual invocation of msync) writes dirty pages back to disk.
     • When unmapping (e.g., via munmap), dirty pages are automatically written back to ensure data consistency.

3. Application Pattern Examples

   • Scenario 1: Efficient Reading of Large Files
     • Traditional approach: Requires multiple read calls, with data copied from kernel buffers to user space.
     • Memory-mapped approach: Only requires a single mapping operation; subsequent accesses are direct memory accesses, reducing system call and copying overhead.
   • Scenario 2: Inter-Process Shared Memory
     • Multiple processes map the same file (e.g., files under Linux's /dev/shm), making modifications immediately visible to each other.
     • Compared to other IPC methods (e.g., pipes): No serialization or deserialization is required, resulting in faster speeds.

4. Considerations

   • File Size Limitation: The mapped region cannot exceed the actual file size (the file can be extended via ftruncate).
   • Synchronization Issues: When multiple processes modify the same mapped region, synchronization mechanisms (e.g., mutex locks) are needed to protect the data.
   • Performance Trade-offs: For small files or frequent random accesses, the overhead of mapping may be higher than that of traditional I/O.

Summary
Memory-mapped files convert file operations into memory accesses via the virtual memory mechanism, combining the persistence of file I/O with the efficiency of memory operations. They are an important tool for operating systems to optimize I/O-intensive applications and implement efficient IPC.