Code Explainers

Code explainers tagged #deduplication

python 
from django.db.models import Q
 
from .models import Order
 

DISTINCT ON queries in Django

orm querysets postgresql
Intermediate 7 steps
rust 
use std::collections::BTreeSet;
use std::fs::File;
use std::io::{self, BufRead, BufReader, BufWriter, Write};
use std::path::Path;

Deduplicating and sorting words in Rust

file-io btreeset string-processing
Intermediate 6 steps
java 
public class RequestCoalescer<K, V> {
 
    private final ConcurrentHashMap<K, CompletableFuture<V>> inFlight = new ConcurrentHashMap<>();
    private final Function<K, V> loader;

Coalescing duplicate requests in Java

concurrency caching completablefuture
Advanced 6 steps
go 
package email
 
import (
	"errors"

Normalizing and deduping email addresses in Go

validation normalization deduplication
Intermediate 8 steps
python 
from pathlib import Path
from collections import defaultdict
 
from PIL import Image

Finding near-duplicate images by perceptual hash

perceptual-hashing clustering hamming-distance
Intermediate 9 steps
go 
package handlers
 
import (
	"net/http"

Custom query validation in Gin

validation query-binding deduplication
Intermediate 8 steps
rust 
use axum::{
    extract::State,
    http::StatusCode,
    Json,

Idempotent webhook handling in Axum

deduplication idempotency shared-state
Intermediate 8 steps
ruby 
require "digest"
 
class UploadDeduplicator
  CHUNK_SIZE = 64 * 1024

Deduplicating uploads by content hash

hashing deduplication streaming-io
Intermediate 7 steps
ruby 
class ContactImporter
  def initialize(rows)
    @rows = rows
  end

Deduplicating contacts with Enumerable#uniq

deduplication normalization enumerable
Intermediate 4 steps
rust 
use std::collections::HashSet;
use std::io::{self, BufRead, BufWriter, Write};
 
fn main() -> io::Result<()> {

Filtering duplicate lines in Rust

stdin hashset buffering
Beginner 5 steps
ruby 
class BulkImporter
  BATCH_SIZE = 500
 
  def initialize(records)

Batching bulk inserts in Rails

batching bulk-insert deduplication
Intermediate 5 steps
typescript 
type Fetcher<T> = (key: string) => Promise<T>;
 
export class RequestDeduplicator<T> {
  private inFlight = new Map<string, Promise<T>>();

Deduplicating in-flight requests in TypeScript

deduplication promises caching
Intermediate 7 steps
javascript 
const RESERVED_SLUGS = new Set(['new', 'edit', 'admin', 'api']);
 
function slugify(title) {
  return title

Generating URL-safe unique slugs

string-normalization regex deduplication
Intermediate 9 steps
python 
from collections import OrderedDict
from typing import Callable, Hashable, Iterable, Iterator, TypeVar
 
T = TypeVar("T")

Two ways to dedupe while keeping order

deduplication generators ordered-data
Intermediate 7 steps
ruby 
class PaymentsController < ApplicationController
  before_action :require_idempotency_key
 
  def create

Idempotent payment creation in Rails

idempotency transactions race-conditions
Advanced 10 steps
go 
package collection
 
func Deduplicate[T comparable](items []T) []T {
	seen := make(map[T]struct{}, len(items))

Generic deduplication in Go

generics deduplication maps
Intermediate 5 steps
java 
import java.util.ArrayList;
import java.util.Comparator;
import java.util.HashSet;
import java.util.List;

Three ways to deduplicate a list by key in Java

streams deduplication collectors
Intermediate 6 steps
python 
def two_sum_sorted(numbers, target):
    """Find indices of two values that sum to target in a sorted array."""
    left, right = 0, len(numbers) - 1
    while left < right:

Two-pointer search on sorted arrays

two-pointers arrays sorting
Intermediate 9 steps