python 42 lines · 7 steps

Redacting secrets in Python log records

A logging.Filter that rewrites log messages to strip out tokens, keys, passwords, and card numbers before they hit disk.

Explained by highlit
Intermediate logging regex redaction security filters
1import logging
2import re
3 
4 
5class SecretRedactingFilter(logging.Filter):
6 DEFAULT_PATTERNS = [
7 re.compile(r"(?i)(authorization:\s*bearer\s+)([a-z0-9._\-]+)"),
8 re.compile(r"(?i)(api[_-]?key\"?\s*[:=]\s*\"?)([a-z0-9\-]{8,})"),
9 re.compile(r"(?i)(password\"?\s*[:=]\s*\"?)([^\s\"&]+)"),
10 re.compile(r"\b([A-Za-z0-9+/]{40,})={0,2}\b"),
11 re.compile(r"\b(\d[ -]?){13,16}\b"),
12 ]
13 
14 def __init__(self, patterns=None, placeholder="[REDACTED]"):
15 super().__init__()
16 self.patterns = patterns or self.DEFAULT_PATTERNS
17 self.placeholder = placeholder
18 
19 def _scrub(self, value):
20 if not isinstance(value, str):
21 return value
22 for pattern in self.patterns:
23 if pattern.groups >= 2:
24 value = pattern.sub(lambda m: m.group(1) + self.placeholder, value)
25 else:
26 value = pattern.sub(self.placeholder, value)
27 return value
28 
29 def filter(self, record):
30 record.msg = self._scrub(record.getMessage())
31 record.args = ()
32 if isinstance(record.args, tuple):
33 record.args = tuple(self._scrub(a) for a in record.args)
34 return True
35 
36 
37def install_redaction(logger=None):
38 target = logger or logging.getLogger()
39 redactor = SecretRedactingFilter()
40 for handler in target.handlers:
41 handler.addFilter(redactor)
42 return redactor
01 / 01
STEP 01

Walkthrough

Space play ←→ step click any line
Three takeaways
  1. 1Subclassing logging.Filter lets you mutate records in-flight before any handler formats them.
  2. 2Regex capture groups let you keep the label but replace only the secret that follows it.
  3. 3Attaching a filter to handlers scrubs output no matter where the log call originated.

Related explainers

python 
from fastapi import FastAPI, WebSocket, WebSocketDisconnect
 
app = FastAPI()
 

Building a WebSocket chat with FastAPI

websockets broadcast connection-management
Intermediate 9 steps
ruby 
class UserAgentParser
  BROWSERS = [
    [/Edg\/([\d.]+)/, "Edge"],
    [/OPR\/([\d.]+)/, "Opera"],

Parsing user-agent strings in Ruby

regex pattern-matching lookup-tables
Intermediate 8 steps
python 
import time
import uuid
 
from django.utils.deprecation import MiddlewareMixin

Attaching per-request context in Django

middleware request lifecycle multi-tenancy
Intermediate 7 steps
python 
import random
from typing import Iterator, List
 
 

How reservoir sampling picks k items

reservoir-sampling streaming randomness
Intermediate 5 steps
python 
import secrets
 
from django.contrib.auth import authenticate, login
from django.core.cache import cache

Two-factor login with OTP in Django

two-factor-auth one-time-passwords caching
Intermediate 9 steps
php 
<?php
 
final class RotatingFileLogger
{

How a rotating file logger works in PHP

logging file-rotation io
Intermediate 9 steps

Share this explainer

Here's the card — post it anywhere.

Redacting secrets in Python log records — share card
Made with highlit — turn any snippet into a walkthrough like this in about a minute.
Explain your code