Developer Documentation
Build & Deploy Sub Search
Clone the repo, configure your environment, and start contributing to the subreddit directory. Sub Search is built with Django 5.2, Celery for async tasks, and Redis as the message broker. These docs assume you're comfortable with a terminal.
Clone & Install
- 1. Install Python 3.11+ and git
-
2.
Clone and set up virtual environment:
git clone https://github.com/ericrosenberg1/reddit-sub-analyzer.git cd reddit-sub-analyzer python3 -m venv .venv source .venv/bin/activate pip install -r requirements.txt -
3.
Set up PostgreSQL and Redis:
# PostgreSQL createdb subsearch # Redis (via Docker) docker run -d --name redis -p 6379:6379 redis:alpine -
4.
Run migrations and start the server:
python manage.py migrate python manage.py runserver # http://localhost:8000 -
5.
Start Celery workers (separate terminal):
celery -A reddit_analyzer worker --loglevel=info
Configure Your .env
Copy .env.example to .env and fill in these values:
Reddit API
REDDIT_CLIENT_IDREDDIT_CLIENT_SECRETREDDIT_USERNAMEREDDIT_PASSWORDREDDIT_USER_AGENT
Django
SECRET_KEYDEBUGALLOWED_HOSTS
Database
DATABASE_URL(PostgreSQL)
Redis
REDIS_URLCELERY_BROKER_URL
Deploy to Production
For production, use Gunicorn with your preferred process manager:
pip install gunicorn
gunicorn reddit_analyzer.wsgi:application --bind 0.0.0.0:8000 --workers 4
Use systemd, supervisor, or Docker to manage the Django app and Celery workers.
Celery Task Architecture
Sub Search uses a priority-based task queue. Start both worker and beat scheduler:
# Worker (handles task execution)
celery -A reddit_analyzer worker --loglevel=info
# Beat (schedules periodic tasks)
celery -A reddit_analyzer beat --loglevel=info
Priority 0 (User)
run_sub_search - User-submitted searches always run first
Priority 5 (Retry)
retry_errored_searches - Re-queue failed searches every 10 minutes
Priority 9 (Auto)
run_random_search and run_auto_ingest - Background discovery
Stream Data with PHONE_HOME
-
1.
Set
PHONE_HOME=truein your.env -
2.
Point
PHONE_HOME_ENDPOINTat the main collector -
3.
Add
PHONE_HOME_TOKENif you have one (Bearer auth) -
4.
Set
PHONE_HOME_SOURCEto label your node - 5. Run Sub Search as usual - data syncs automatically!
Volunteer a Node
Nodes add extra throughput by lending your computer + Reddit account:
- • Open the Nodes page and submit the "Add a node" form
- • Use the private link emailed to you to update availability or pause the node
- • Mark a node as "broken" if it goes offline - automatic cleanup after 7 days
Contribute Code
Fork the Repo
Fork the repository, open pull requests, and help expand the toolset.
View Repository