Developer Documentation

Build & Deploy Sub Search

Clone the repo, configure your environment, and start contributing to the subreddit directory. Sub Search is built with Django 5.2, Celery for async tasks, and Redis as the message broker. These docs assume you're comfortable with a terminal.

1

Clone & Install

  1. 1. Install Python 3.11+ and git
  2. 2.
    Clone and set up virtual environment:
    git clone https://github.com/ericrosenberg1/reddit-sub-analyzer.git
    cd reddit-sub-analyzer
    python3 -m venv .venv
    source .venv/bin/activate
    pip install -r requirements.txt
  3. 3.
    Set up PostgreSQL and Redis:
    # PostgreSQL
    createdb subsearch
    
    # Redis (via Docker)
    docker run -d --name redis -p 6379:6379 redis:alpine
  4. 4.
    Run migrations and start the server:
    python manage.py migrate
    python manage.py runserver  # http://localhost:8000
  5. 5.
    Start Celery workers (separate terminal):
    celery -A reddit_analyzer worker --loglevel=info
2

Configure Your .env

Copy .env.example to .env and fill in these values:

Reddit API
  • REDDIT_CLIENT_ID
  • REDDIT_CLIENT_SECRET
  • REDDIT_USERNAME
  • REDDIT_PASSWORD
  • REDDIT_USER_AGENT
Django
  • SECRET_KEY
  • DEBUG
  • ALLOWED_HOSTS
Database
  • DATABASE_URL (PostgreSQL)
Redis
  • REDIS_URL
  • CELERY_BROKER_URL
3

Deploy to Production

For production, use Gunicorn with your preferred process manager:

pip install gunicorn
gunicorn reddit_analyzer.wsgi:application --bind 0.0.0.0:8000 --workers 4

Use systemd, supervisor, or Docker to manage the Django app and Celery workers.

4

Celery Task Architecture

Sub Search uses a priority-based task queue. Start both worker and beat scheduler:

# Worker (handles task execution)
celery -A reddit_analyzer worker --loglevel=info

# Beat (schedules periodic tasks)
celery -A reddit_analyzer beat --loglevel=info
Priority 0 (User)

run_sub_search - User-submitted searches always run first

Priority 5 (Retry)

retry_errored_searches - Re-queue failed searches every 10 minutes

Priority 9 (Auto)

run_random_search and run_auto_ingest - Background discovery

5

Stream Data with PHONE_HOME

  1. 1. Set PHONE_HOME=true in your .env
  2. 2. Point PHONE_HOME_ENDPOINT at the main collector
  3. 3. Add PHONE_HOME_TOKEN if you have one (Bearer auth)
  4. 4. Set PHONE_HOME_SOURCE to label your node
  5. 5. Run Sub Search as usual - data syncs automatically!
6

Volunteer a Node

Nodes add extra throughput by lending your computer + Reddit account:

  • • Open the Nodes page and submit the "Add a node" form
  • • Use the private link emailed to you to update availability or pause the node
  • • Mark a node as "broken" if it goes offline - automatic cleanup after 7 days

Report & Discuss

Open an Issue

Share bugs, feature requests, or architectural ideas.

GitHub Issues

Contribute Code

Fork the Repo

Fork the repository, open pull requests, and help expand the toolset.

View Repository