Celery & Redis: Message Broker Protocols & Queue Architecture

1. Deep Architecture

Redis serves as a fast in-memory store. When configured as a Celery broker, it uses Redis lists (via LPUSH and BRPOP) to queue messages. Workers listen using open socket connections. Task payloads are serialized (JSON or Msgpack) into binary data frames.

2. The Feynman Gatekeeper

[KNOWLEDGE CHECK] Explain what a visibility timeout is in Celery, and what happens when a task takes longer to execute than this value.

3. The Code

# config/celery.py
import os
from celery import Celery

os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'config.settings')
app = Celery('sanjaya')
app.config_from_object('django.conf:settings', namespace='CELERY')
app.conf.broker_transport_options = {'visibility_timeout': 3600}

4. The Funnel

Stat Level-Up: Queue Commander (Lvl 1).
Sanjaya Integration: Offload video processing tasks to background workers, returning response status codes immediately.