Django Under Hood 08: Django’s Static Files - From collectstatic to Browser Cache

2026-06-04

Django Development

Part 8 of the “Django Under the Hood ” series — deep dives into Django’s internals, edge cases, and the mechanics that separate production-grade applications from tutorial code.

~] python manage.py collectstatic

This command copies files. That’s what most developers think.

It actually does much more: finds files across multiple locations, resolves naming conflicts, optionally hashes filenames for cache busting, compresses assets, and builds a manifest mapping original names to processed names.

Then there’s serving. Development uses Django’s static view. Production uses… what exactly? Nginx? WhiteNoise? A CDN? Each has different performance characteristics, different caching behaviors, different security implications.

Most Django applications have misconfigured static files. They work, but they’re slow, or they break on deploy, or they don’t cache properly. Understanding the staticfiles system prevents these issues.

Let’s trace a static file from your static/ directory to a browser's cache.

The Static Files App

# settings.py
INSTALLED_APPS = [
    'django.contrib.staticfiles',
    # ...
]

This app provides:

The {% static %} template tag
The collectstatic management command
Finders that locate static files
Storage backends that process and serve files

Configuration:

# Where collectstatic puts files
STATIC_ROOT = BASE_DIR / 'staticfiles'

# URL prefix for static files
STATIC_URL = '/static/'

# Additional directories to search (beyond app static/ folders)
STATICFILES_DIRS = [
    BASE_DIR / 'static',
]

# How files are found
STATICFILES_FINDERS = [
    'django.contrib.staticfiles.finders.FileSystemFinder',
    'django.contrib.staticfiles.finders.AppDirectoriesFinder',
]

Finders: Locating Static Files

When Django needs a static file, it asks finders:

from django.contrib.staticfiles import finders

# Find a file
path = finders.find('js/app.js')
# Returns: '/path/to/project/static/js/app.js'

# Find all matches (if file exists in multiple locations)
paths = finders.find('js/app.js', all=True)
# Returns: ['/path/to/project/static/js/app.js', '/path/to/app/static/js/app.js']

FileSystemFinder

Searches STATICFILES_DIRS:

# django/contrib/staticfiles/finders.py
class FileSystemFinder(BaseFinder):
    def __init__(self, app_names=None, *args, **kwargs):
        self.locations = []
        for root in settings.STATICFILES_DIRS:
            if isinstance(root, (list, tuple)):
                prefix, root = root
            else:
                prefix = ''
            self.locations.append((prefix, root))
    
    def find(self, path, all=False):
        matches = []
        for prefix, root in self.locations:
            if prefix:
                if not path.startswith(prefix):
                    continue
                path = path[len(prefix):]
            
            full_path = os.path.join(root, path)
            if os.path.exists(full_path):
                if not all:
                    return full_path
                matches.append(full_path)
        
        return matches

AppDirectoriesFinder

Searches static/ subdirectories in each installed app:

class AppDirectoriesFinder(BaseFinder):
    def __init__(self, app_names=None, *args, **kwargs):
        self.apps = []
        for app_config in apps.get_app_configs():
            static_dir = os.path.join(app_config.path, 'static')
            if os.path.isdir(static_dir):
                self.apps.append(app_config.name)
                self.storages[app_config.name] = FileSystemStorage(location=static_dir)

Order matters: First finder to return a path wins. If you have project/static/js/app.js and myapp/static/js/app.js, FileSystemFinder (listed first) returns the project-level file.

collectstatic: The Build Step

~] python manage.py collectstatic

What actually happens:

# django/contrib/staticfiles/management/commands/collectstatic.py
class Command(BaseCommand):
    def handle(self, **options):
        self.storage = staticfiles_storage  # Usually StaticFilesStorage
        
        for finder in get_finders():
            for path, storage in finder.list(ignore_patterns):
                # Get source path
                source_path = storage.path(path)
                
                # Destination in STATIC_ROOT
                dest_path = self.storage.path(path)
                
                # Copy (or process)
                self.copy_file(source_path, dest_path)

File Processing Pipeline

With ManifestStaticFilesStorage:

STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'

The pipeline:

Collect: Copy files to STATIC_ROOT
Hash: Calculate content hash for each file
Rename: Copy to hashed filename (app.js → app.abc123.js)
Rewrite: Update references in CSS/JS files
Manifest: Write staticfiles.json mapping original → hashed names

# django/contrib/staticfiles/storage.py
class ManifestStaticFilesStorage(ManifestFilesMixin, StaticFilesStorage):
    manifest_name = 'staticfiles.json'
    
    def post_process(self, paths, dry_run=False, **options):
        # Process files in dependency order
        for name, hashed_name, processed in self._post_process(paths, ...):
            yield name, hashed_name, processed
    
    def hashed_name(self, name, content=None):
        # Calculate hash from content
        if content is None:
            content = self.open(name)
        
        md5 = hashlib.md5()
        for chunk in content.chunks():
            md5.update(chunk)
        
        file_hash = md5.hexdigest()[:12]
        
        # Insert hash before extension
        root, ext = os.path.splitext(name)
        return f'{root}.{file_hash}{ext}'

The Manifest File

// staticfiles/staticfiles.json
{
    "paths": {
        "js/app.js": "js/app.d41d8cd98f00.js",
        "css/style.css": "css/style.5eb63bbbe01e.css",
        "img/logo.png": "img/logo.098f6bcd4621.png"
    },
    "version": "1.0"
}

CSS URL Rewriting

References in CSS files get updated:

/* Before: style.css */
.logo {
    background: url('../img/logo.png');
}

/* After: style.5eb63bbbe01e.css */
.logo {
    background: url('../img/logo.098f6bcd4621.png');
}

class HashedFilesMixin:
    patterns = (
        ("*.css", (
            r"url\((['\"]?)(.+?)\1\)",  # url('...')
            r"@import\s+(['\"])(.+?)\1",  # @import '...'
        )),
    )
    
    def url_converter(self, name, hashed_files, template=None):
        def converter(matchobj):
            url = matchobj.group(2)
            # Resolve relative URL
            url_path = posixpath.join(posixpath.dirname(name), url)
            # Look up hashed name
            hashed_url = hashed_files.get(url_path, url)
            return template % hashed_url
        return converter

The static Template Tag

{% load static %}
<script src="{% static 'js/app.js' %}"></script>

What it returns depends on storage:

# django/templatetags/static.py
@register.simple_tag
def static(path):
    return staticfiles_storage.url(path)

Without Manifest (Development)

class StaticFilesStorage(FileSystemStorage):
    def url(self, name):
        return urljoin(self.base_url, name)
        # Returns: '/static/js/app.js'

With Manifest (Production)

class ManifestStaticFilesStorage:
    def url(self, name):
        # Look up in manifest
        hashed_name = self.stored_name(name)
        return urljoin(self.base_url, hashed_name)
        # Returns: '/static/js/app.d41d8cd98f00.js'
    
    def stored_name(self, name):
        # Load manifest if not cached
        if self.manifest is None:
            self.load_manifest()
        
        return self.manifest.get(name, name)

Key insight: The template tag doesn’t change, but output changes based on storage backend.

Development: Django’s Static View

# urls.py
from django.conf import settings
from django.conf.urls.static import static

urlpatterns = [
    # ...
] + static(settings.STATIC_URL, document_root=settings.STATIC_ROOT)

But that only serves from STATIC_ROOT. During development, files aren't collected yet. Django provides a different view:

# With DEBUG=True, staticfiles app adds this automatically
from django.contrib.staticfiles.views import serve

def serve(request, path, insecure=False, **kwargs):
    # Use finders to locate the file
    normalized_path = posixpath.normpath(path).lstrip('/')
    absolute_path = finders.find(normalized_path)
    
    if not absolute_path:
        raise Http404
    
    # Serve directly
    return FileResponse(open(absolute_path, 'rb'))

Important: This bypasses STATIC_ROOT entirely. Files are served directly from their source locations using finders.

Production: The Options

Option 1: Reverse Proxy (Nginx)

server {
    location /static/ {
        alias /path/to/staticfiles/;
        expires 1y;
        add_header Cache-Control "public, immutable";
    }
    
    location / {
        proxy_pass http://django:8000;
    }
}

Pros: Fast, battle-tested, handles range requests Cons: Separate process, configuration outside Django

Option 2: WhiteNoise

~] pip install whitenoise

# settings.py
MIDDLEWARE = [
    'django.middleware.security.SecurityMiddleware',
    'whitenoise.middleware.WhiteNoiseMiddleware',  # After SecurityMiddleware
    # ...
]

STATICFILES_STORAGE = 'whitenoise.storage.CompressedManifestStaticFilesStorage'

WhiteNoise serves static files directly from Django.

Option 3: CDN

STATIC_URL = 'https://cdn.example.com/static/'

Files are still collected locally, but URLs point to CDN. You sync STATIC_ROOT to CDN during deploy.

WhiteNoise: Deep Dive

WhiteNoise is fascinating because it makes Python competitive with Nginx for static file serving. How?

Middleware Architecture

# whitenoise/middleware.py
class WhiteNoiseMiddleware:
    def __init__(self, get_response):
        self.get_response = get_response
        self.static_prefix = settings.STATIC_URL
        
        # Pre-load all static files at startup
        self.files = {}
        self.load_static_files()
    
    def __call__(self, request):
        if self.is_static_file(request.path):
            return self.serve_static(request)
        return self.get_response(request)
    
    def load_static_files(self):
        # Walk STATIC_ROOT, load file metadata
        for root, dirs, files in os.walk(settings.STATIC_ROOT):
            for filename in files:
                path = os.path.join(root, filename)
                url_path = os.path.relpath(path, settings.STATIC_ROOT)
                self.files[url_path] = StaticFile(path)

Startup loading: All file metadata is loaded when Django starts. No filesystem calls during requests.

StaticFile: Precomputed Everything

class StaticFile:
    def __init__(self, path):
        self.path = path
        
        # Read file once
        with open(path, 'rb') as f:
            self.content = f.read()
        
        # Precompute headers
        self.stat = os.stat(path)
        self.etag = self.compute_etag()
        self.last_modified = http_date(self.stat.st_mtime)
        self.content_length = len(self.content)
        
        # Precompute compressed versions
        self.gzip_content = self.compress_gzip()
        self.brotli_content = self.compress_brotli()
    
    def compute_etag(self):
        return hashlib.md5(self.content).hexdigest()

Serving a Request

def serve_static(self, request):
    static_file = self.files.get(request.path.lstrip('/'))
    
    if not static_file:
        return self.get_response(request)  # Pass to Django
    
    # Check If-None-Match (ETag)
    if request.headers.get('If-None-Match') == static_file.etag:
        return HttpResponse(status=304)  # Not Modified
    
    # Check If-Modified-Since
    if request.headers.get('If-Modified-Since') == static_file.last_modified:
        return HttpResponse(status=304)
    
    # Select content based on Accept-Encoding
    if 'br' in request.headers.get('Accept-Encoding', ''):
        content = static_file.brotli_content
        encoding = 'br'
    elif 'gzip' in request.headers.get('Accept-Encoding', ''):
        content = static_file.gzip_content
        encoding = 'gzip'
    else:
        content = static_file.content
        encoding = None
    
    response = HttpResponse(content)
    response['Content-Length'] = len(content)
    response['ETag'] = static_file.etag
    response['Cache-Control'] = 'public, max-age=31536000, immutable'
    
    if encoding:
        response['Content-Encoding'] = encoding
    
    return response

Why WhiteNoise Is Fast

No filesystem I/O: Files loaded at startup, served from memory
Precomputed headers: ETag, Last-Modified calculated once
Pre-compressed: Gzip/Brotli done at startup, not per-request
304 responses: Conditional requests skip body entirely
Immutable caching: Hashed filenames enable infinite caching

CompressedManifestStaticFilesStorage

# whitenoise/storage.py
class CompressedManifestStaticFilesStorage(
    CompressedFilesMixin,
    ManifestStaticFilesStorage
):
    pass

class CompressedFilesMixin:
    def post_process(self, paths, **options):
        # Run parent post_process (hashing)
        for name, hashed_name, processed in super().post_process(paths, **options):
            yield name, hashed_name, processed
        
        # Compress each file
        for path in paths:
            full_path = self.path(path)
            
            # Create .gz version
            with open(full_path, 'rb') as f_in:
                with gzip.open(f'{full_path}.gz', 'wb') as f_out:
                    f_out.write(f_in.read())
            
            # Create .br version (if brotli installed)
            if brotli:
                with open(full_path, 'rb') as f_in:
                    with open(f'{full_path}.br', 'wb') as f_out:
                        f_out.write(brotli.compress(f_in.read()))

After collectstatic:

staticfiles/
├── js/
│   ├── app.d41d8cd98f00.js
│   ├── app.d41d8cd98f00.js.gz
│   └── app.d41d8cd98f00.js.br
├── css/
│   ├── style.5eb63bbbe01e.css
│   ├── style.5eb63bbbe01e.css.gz
│   └── style.5eb63bbbe01e.css.br

Cache Headers: Getting Them Right

Without Manifest (Bad)

Cache-Control: max-age=0

Every page load re-fetches every static file.

With Manifest (Good)

Cache-Control: public, max-age=31536000, immutable

public: CDNs can cache
max-age=31536000: Cache for 1 year
immutable: Don't even revalidate

This works because filenames change when content changes. app.abc123.js is a different URL than app.def456.js.

Versioned URLs Alternative

If you can’t use manifest storage:

# settings.py
import time
STATIC_VERSION = str(int(time.time()))

# template
<script src="{% static 'js/app.js' %}?v={{ STATIC_VERSION }}"></script>

Less elegant, but works.

Common Issues

Issue 1: Static Files 404 in Production

Cause: collectstatic not run, or STATIC_ROOT not served.

# Run collectstatic
~] python manage.py collectstatic --noinput

# Verify files exist
~] ls -la staticfiles/

Issue 2: Old Files Cached After Deploy

Cause: Not using manifest storage, or CDN not purged.

Fix: Use ManifestStaticFilesStorage — new deploys get new hashes.

Issue 3: CSS url() References Broken

Cause: Relative paths in CSS don’t match after hashing.

Fix: Manifest storage rewrites URLs. Ensure paths are correct relative to CSS file location.

Issue 4: collectstatic Slow

Cause: Processing thousands of files.

# Only collect changed files
~] python manage.py collectstatic --noinput --ignore=*.map

# Use parallel processing (Django 4.2+)
~] python manage.py collectstatic --noinput

Issue 5: ManifestStaticFilesStorage Error on Missing File

ValueError: Missing staticfiles manifest entry for 'js/missing.js'

Cause: Template references file that wasn’t collected.

Fix: Ensure file exists, or use ManifestStaticFilesStorage with manifest_strict = False:

class ForgivingManifestStorage(ManifestStaticFilesStorage):
    manifest_strict = False

Performance Comparison

WhiteNoise is ~95% as fast as Nginx while being much simpler to configure.

What’s Next

This was static files — from finders to browser cache.

Next in the series: Migration System Deep Dive — how Django detects model changes, generates migration operations, and applies them to databases.

Series: Django Under the Hood

Request Lifecycle
ORM Query Compiler
Connection Management
Signal Dispatch
Template Engine
Form Pipeline
Authentication Chain
Static Files and WhiteNoise Internals ← You are here
Migration System
Test Client

django

« Previous article

comments powered by Disqus

Django Under Hood 08: Django’s Static Files - From collectstatic to Browser Cache

The Static Files App

Finders: Locating Static Files

FileSystemFinder

AppDirectoriesFinder

collectstatic: The Build Step

File Processing Pipeline

The Manifest File

CSS URL Rewriting

The static Template Tag

Without Manifest (Development)

With Manifest (Production)

Development: Django’s Static View

Production: The Options

Option 1: Reverse Proxy (Nginx)

Option 2: WhiteNoise

Option 3: CDN

WhiteNoise: Deep Dive

Middleware Architecture

StaticFile: Precomputed Everything

Serving a Request

Why WhiteNoise Is Fast

CompressedManifestStaticFilesStorage

Cache Headers: Getting Them Right

Without Manifest (Bad)

With Manifest (Good)

Versioned URLs Alternative

Common Issues

Issue 1: Static Files 404 in Production

Issue 2: Old Files Cached After Deploy

Issue 3: CSS url() References Broken

Issue 4: collectstatic Slow

Issue 5: ManifestStaticFilesStorage Error on Missing File

Performance Comparison

What’s Next

Series: Django Under the Hood

Architecture

Bash

Databases

Debian

Development

Django

Hardware

Hugo

Linux

Linux Utilities

Monitoring

Networking

Python

Security

Virtualization

Web Servers

Windows

SUBSCRIBE FOR NEW ARTICLES