python 36 lines · 7 steps

How atomic file writes work in Python

Write to a temp file, fsync it, then rename into place so readers never see a half-written file.

Explained by highlit
Advanced atomicity filesystem durability error-handling fsync
1import os
2import tempfile
3from pathlib import Path
4from typing import Union
5 
6 
7def atomic_write(path: Union[str, Path], data: Union[str, bytes], *, encoding: str = "utf-8") -> None:
8 path = Path(path)
9 directory = path.parent
10 
11 if isinstance(data, str):
12 mode = "w"
13 kwargs = {"encoding": encoding}
14 else:
15 mode = "wb"
16 kwargs = {}
17 
18 fd, tmp_name = tempfile.mkstemp(dir=directory, prefix=f".{path.name}.", suffix=".tmp")
19 tmp_path = Path(tmp_name)
20 
21 try:
22 with os.fdopen(fd, mode, **kwargs) as f:
23 f.write(data)
24 f.flush()
25 os.fsync(f.fileno())
26 
27 os.replace(tmp_path, path)
28 
29 dir_fd = os.open(directory, os.O_RDONLY)
30 try:
31 os.fsync(dir_fd)
32 finally:
33 os.close(dir_fd)
34 except BaseException:
35 tmp_path.unlink(missing_ok=True)
36 raise
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1A rename on the same filesystem is atomic, so writing to a temp file and replacing avoids ever exposing partial data.
  2. 2Durability needs fsync on both the file and its parent directory, since the rename itself must be flushed to survive a crash.
  3. 3Cleaning up the temp file on any exception keeps failures from littering the directory with stale partial files.

Related explainers

python 
import argparse
import sys
from pathlib import Path
 

Building a subcommand CLI with argparse

cli argparse subcommands
Intermediate 6 steps
go 
package main
 
import (
	"errors"

Parsing and validating CLI flags in Go

cli-parsing validation error-handling
Intermediate 8 steps
java 
public class ThumbnailProcessor {
 
    private static final int MAX_CONCURRENCY = 4;
 

Bounded parallel thumbnail rendering in Java

concurrency thread-pool futures
Intermediate 7 steps
python 
from collections.abc import Mapping
from typing import Any, Iterator
 
 

Flattening nested config into dotted keys

recursion generators tree-traversal
Intermediate 7 steps
python 
import csv
import io
from datetime import datetime
 

Streaming a CSV export in Flask

streaming generators csv
Intermediate 9 steps
python 
import time
from collections import defaultdict
from threading import Lock
 

Sliding-window login rate limiting in Flask

rate-limiting sliding-window thread-safety
Intermediate 7 steps

Share this explainer

Here's the card — post it anywhere.

How atomic file writes work in Python — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code