python 36 lines · 7 steps

How atomic file writes work in Python

Write to a temp file, fsync it, then rename into place so readers never see a half-written file.

Explained by highlit
Advanced atomicity filesystem durability error-handling fsync
1import os
2import tempfile
3from pathlib import Path
4from typing import Union
5 
6 
7def atomic_write(path: Union[str, Path], data: Union[str, bytes], *, encoding: str = "utf-8") -> None:
8 path = Path(path)
9 directory = path.parent
10 
11 if isinstance(data, str):
12 mode = "w"
13 kwargs = {"encoding": encoding}
14 else:
15 mode = "wb"
16 kwargs = {}
17 
18 fd, tmp_name = tempfile.mkstemp(dir=directory, prefix=f".{path.name}.", suffix=".tmp")
19 tmp_path = Path(tmp_name)
20 
21 try:
22 with os.fdopen(fd, mode, **kwargs) as f:
23 f.write(data)
24 f.flush()
25 os.fsync(f.fileno())
26 
27 os.replace(tmp_path, path)
28 
29 dir_fd = os.open(directory, os.O_RDONLY)
30 try:
31 os.fsync(dir_fd)
32 finally:
33 os.close(dir_fd)
34 except BaseException:
35 tmp_path.unlink(missing_ok=True)
36 raise
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1A rename on the same filesystem is atomic, so writing to a temp file and replacing avoids ever exposing partial data.
  2. 2Durability needs fsync on both the file and its parent directory, since the rename itself must be flushed to survive a crash.
  3. 3Cleaning up the temp file on any exception keeps failures from littering the directory with stale partial files.

Related explainers

python 
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, status
from pydantic import BaseModel, EmailStr
from sqlalchemy.orm import Session
 

Building a signup endpoint in FastAPI

dependency-injection request-validation background-tasks
Intermediate 8 steps
ruby 
class TemplateInterpolator
  PLACEHOLDER = /\{\{\s*([\w.]+)\s*\}\}/
 
  def initialize(strict: false)

Interpolating templates with dotted keys in Ruby

regex string-interpolation hash-traversal
Intermediate 6 steps
python 
from django import forms
from django.utils import timezone
 
from .models import Reservation

Multi-field validation in a Django ModelForm

form validation cross-field validation modelform
Intermediate 7 steps
python 
from django import template
from django.urls import reverse, NoReverseMatch
from django.utils.html import format_html
 

Active nav-link template tags in Django

template tags url routing active state
Intermediate 7 steps
go 
package config
 
import (
	"fmt"

A thread-safe config singleton in Go

singleton concurrency environment-variables
Intermediate 7 steps
python 
from functools import wraps
import asyncio
 
from fastapi import APIRouter, FastAPI, Request

Per-route request timeouts in FastAPI

decorators async timeouts
Intermediate 6 steps

Share this explainer

Here's the card — post it anywhere.

How atomic file writes work in Python — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code