Distributed Task Scheduling: Celery Beat & Lock Contention

1. Deep Architecture

Celery Beat triggers scheduled tasks periodically. In multi-server deploys, running multiple Beat instances can trigger duplicate tasks. We implement distributed locks using Redis SETNX (Set if Not Exists) to ensure only one worker executes the scheduled task.

2. The Feynman Gatekeeper

[KNOWLEDGE CHECK] What is lock contention, and how does SETNX prevent multiple workers from running the same scheduled task simultaneously?

3. The Code

from django.core.cache import cache

def execute_unique_task():
    # Attempt to acquire a lock for 10 minutes
    lock_acquired = cache.add("lock:cleanup_temp_files", "locked", timeout=600)
    if lock_acquired:
        try:
            # Perform clean-up logic
            pass
        finally:
            cache.delete("lock:cleanup_temp_files")

4. The Funnel

Stat Level-Up: Lock Guardian (Lvl 1).
Sanjaya Integration: Run daily cleanups on temporary video assets without duplicate deletion conflicts.