Rails Ascent Logo Rails Ascent

Solid Queue: Background Tasks Without Redis


Welcome to the third article in the Solid Triumvirate series in Rails 8! Today we're talking about Solid Queue. The SOLUTION you'll wonder how you ever lived without for creating background tasks with ease.


  1. Send an automatic email? ✅ Check.
  2. As soon as a user signs up, Solid Queue takes over to send a welcome email without slowing down the HTTP response.
  3. Generate a PDF in the background? ✅ Check.
  4. Need a report or an invoice? The job is launched in the background, and the file is ready when the user returns.
  5. Synchronize data with an external API? ✅ Check.
  6. A profile update triggers a job that sends the information to a CRM or marketing tool.
  7. Clean up old sessions or temporary files? ✅ Check.
  8. A nightly scheduled job keeps your database clean and lean.
  9. Import a large CSV file? ✅ Check.
  10. The user uploads a file? Solid Queue processes it line by line without overloading the server.
  11. Schedule reminders or follow-ups? ✅ Check.
  12. Thanks to job scheduling, you can send a reminder 24 hours before an event.


You can do so many things that it can be dizzying! From installation to configuration to its use. Solid Queue will have no secrets left for you!



Don't hesitate to check out my other articles on the Solid Triumvirate topic—these will reveal all its secrets to you!


  1. "Solid Cache": No longer depending on Redis for Rails caching
  2. "Solid Cable": WebSockets and real-time features, simply and hassle-free
  3. "Solid Queue": Background tasks Rails-style (you're here)
  4. The Triumvirate applied to a complete project


Buckle up—it's going to be solid :)

Focus on Solid Queue

What is a Background Job?


Background jobs are tasks that take time and we don't want to execute during the user's HTTP request. Otherwise we would block the application or get a timeout error telling us that the server we queried took too long to respond.

  1. Classic examples:
  2. Send a welcome email after signup
  3. Generate a complex PDF report
  4. Process an uploaded image
  5. Synchronize data with an external API


Clean up old data

Utilisateur clique "S'inscrire"
↓
Serveur crée le compte (100ms)
↓
Serveur envoie l'email (2000ms) ← L'utilisateur attend...
↓
Réponse "Inscription réussie !" (2100ms total)


Without background jobs:

Utilisateur clique "S'inscrire"
↓
Serveur crée le compte (100ms)
↓
Serveur enqueue le job email (10ms)
↓
Réponse "Inscription réussie !" (110ms total)
↓
(Plus tard) Le job envoie l'email en fond

With background jobs:

Solid Queue: How does it work?


Solid Queue is a job management system based on your database, using Active Job (Rails' standard interface for background jobs).

Architecture

# Tables principales
solid_queue_jobs # Jobs en attente
solid_queue_ready_executions # Jobs prêts à être exécutés
solid_queue_claimed_executions # Jobs en cours d'exécution
solid_queue_blocked_executions # Jobs bloqués (concurrence)
solid_queue_failed_executions # Jobs échoués
solid_queue_scheduled_executions # Jobs schedulés pour plus tard
solid_queue_recurring_tasks # Tâches récurrentes (cron-like)

Solid Queue creates several tables in your database:

Key components

1. Workers (Workers) The workers are the processes that actually execute your jobs. They poll the solid_queue_ready_executions table regularly to find work.

2. Dispatchers (Dispatchers) The dispatchers move jobs from solid_queue_scheduled_executions to solid_queue_ready_executions when their execution time arrives.

3. Scheduler (Scheduler) The scheduler handles recurring tasks (like cron jobs).

The magic: FOR UPDATE SKIP LOCKED

-- Requête simplifiée d'un worker
SELECT * FROM solid_queue_ready_executions
WHERE queue_name = 'default'
ORDER BY priority DESC, created_at ASC
LIMIT 1
FOR UPDATE SKIP LOCKED;


Solid Queue uses an advanced SQL feature called FOR UPDATE SKIP LOCKED which allows multiple workers to read the same table concurrently without blocking each other.

  1. What happens:
  2. Worker 1 takes job A and locks it
  3. Worker 2 tries to take a job
  4. Instead of waiting for Worker 1 to finish (LOCKED), it skips job A (SKIP LOCKED) and takes job B

Result: zero blocking, maximum performance!

Installation and Configuration

On Rails 8


# Vérifier que tout est en place
rails db:migrate

Solid Queue is enabled by default! Just check your configuration.

default: &default
dispatchers:
- polling_interval: 1 # Vérifier les jobs schedulés chaque seconde
batch_size: 500 # Traiter 500 jobs à la fois
workers:
- queues: "*" # Traiter toutes les queues
threads: 3 # 3 threads par processus
processes: 1 # Nombre de processus
polling_interval: 0.1 # Vérifier les nouveaux jobs toutes les 0.1s
production:
<<: *default
workers:
# Queue critique : haute priorité
- queues: critical
threads: 5
processes: 2
polling_interval: 0.1
# Queue par défaut : ressources modérées
- queues: default
threads: 3
processes: 3
polling_interval: 1
# Queue emails : beaucoup de threads (I/O bound)
- queues: mailers
threads: 10
processes: 2
polling_interval: 2
# Queue rapports : peu de threads (CPU intensive)
- queues: reports
threads: 2
processes: 1
polling_interval: 5
development:
<<: *default

Configuration in config/solid_queue.yml

Practical Examples

# app/jobs/welcome_email_job.rb
class WelcomeEmailJob < ApplicationJob
queue_as :mailers # Utiliser la queue "mailers"
def perform(user_id)
user = User.find(user_id)
UserMailer.welcome_email(user).deliver_now
end
end


# app/controllers/users_controller.rb
class UsersController < ApplicationController
def create
@user = User.new(user_params)
if @user.save
# Enqueuer le job pour envoi immédiat
WelcomeEmailJob.perform_later(@user.id)
redirect_to root_path, notice: "Compte créé ! Vérifiez vos emails."
else
render :new
end
end
end

Example 1: Simple Job - Send an Email

# Envoyer dans 1 heure
ReminderEmailJob.set(wait: 1.hour).perform_later(user.id)
# Envoyer à une heure précise
ReportJob.set(wait_until: Date.tomorrow.noon).perform_later
# Avec une priorité spécifique (plus le nombre est bas, plus c'est prioritaire)
UrgentJob.set(priority: 0).perform_later(data)

Example 2: Scheduled Job - Send Later

# config/solid_queue.yml
production:
recurring_tasks:
# Nettoyer les données tous les jours à 2h du matin
cleanup:
class: CleanupJob
schedule: "0 2 * * *" # Format cron
# Envoyer une newsletter tous les lundis à 9h
weekly_newsletter:
class: NewsletterJob
args: ["weekly"]
schedule: "0 9 * * 1" # 9h tous les lundis
# Générer des rapports toutes les heures
hourly_reports:
class: GenerateReportsJob
schedule: "0 * * * *" # Toutes les heures

Example 3: Recurring Tasks (Cron Jobs)

# app/jobs/api_sync_job.rb
class ApiSyncJob < ApplicationJob
queue_as :default
# Réessayer jusqu'à 5 fois avec délai exponentiel
retry_on StandardError, wait: :exponentially_longer, attempts: 5
# Mais ne pas réessayer si c'est une erreur d'authentification
discard_on AuthenticationError
def perform(resource_id)
resource = Resource.find(resource_id)
# Appel API qui peut échouer
ExternalApiService.sync(resource)
# Log du succès
Rails.logger.info "Resource #{resource_id} synced successfully"
rescue => e
# Log de l'erreur
Rails.logger.error "Failed to sync resource #{resource_id}: #{e.message}"
raise # Re-raise pour que le retry fonctionne
end
end

Example 4: Job with Error Handling

Example 5: Concurrency Control

# app/jobs/report_generator_job.rb
class ReportGeneratorJob < ApplicationJob
queue_as :reports
# Maximum 3 jobs de génération de rapport en même temps
limits_concurrency to: 3, key: -> { "report_generation" }
def perform(report_id)
report = Report.find(report_id)
# Génération coûteuse en CPU
report.generate_pdf
report.generate_excel
report.generate_charts
report.mark_as_complete!
end
end


Solid Queue allows limiting how many jobs of a certain type can run concurrently.

Use case: If you have 10 reports to generate, only 3 will run in parallel. The other 7 will wait for a slot to free up.

# app/jobs/bulk_import_job.rb
class BulkImportJob < ApplicationJob
queue_as :default
def perform(file_path, user_id)
user = User.find(user_id)
lines = File.readlines(file_path)
total = lines.count
lines.each_with_index do |line, index|
# Importer la ligne
import_line(line)
# Mettre à jour la progression
progress = ((index + 1).to_f / total * 100).round(2)
# Notifier l'utilisateur via WebSocket
ActionCable.server.broadcast(
"import_#{user_id}",
{ progress: progress, completed: index + 1, total: total }
)
end
end
private
def import_line(line)
# Logique d'import
end
end

Example 6: Job with Progress

Starting the Worker

In development

# config/puma.rb
plugin :solid_queue if ENV["SOLID_QUEUE_IN_PUMA"] || Rails.env.development?

Rails 8 uses Puma with a Solid Queue plugin:

With bin/dev, everything starts automatically!


In production

# Démarrer le worker Solid Queue en plus de votre application
bin/jobs


Option 1: Separate process

# config/deploy.yml
servers:
web:
- 192.168.1.100
job:
hosts:
- 192.168.1.101
cmd: bin/jobs # Serveur dédié aux jobs


Option 2: With Kamal/Docker

# /etc/systemd/system/solid-queue.service
[Unit]
Description=Solid Queue Worker
After=network.target
[Service]
Type=simple
User=deploy
WorkingDirectory=/var/www/mon_app
ExecStart=/usr/local/bin/bundle exec bin/jobs
Restart=always
[Install]
WantedBy=multi-user.target

Option 3: Systemd

Mission Control: The Administration Interface

Rails 8 can use Mission Control - Jobs, a web interface for monitoring your jobs. It's incredibly handy, so I strongly recommend installing it if you use Solid Queue.

# Gemfile
gem "mission_control-jobs"
bundle install
rails mission_control:jobs:install

Installation

# config/routes.rb
Rails.application.routes.draw do
mount MissionControl::Jobs::Engine, at: "/jobs"
end


Configuration

mission_control:
http_basic_auth_user: dev
http_basic_auth_password: secret


Mission Control comes with basic HTTP authentication. Configuration is done via Rails credentials:

# config/routes.rb
Rails.application.routes.draw do
authenticate :user, ->(u) { u.admin? } do
mount MissionControl::Jobs::Engine, at: '/jobs'
end
end


But if you use an authentication system like Devise, it's best to nest the Mission Control route behind, for example, an administrator role:

class User < ApplicationRecord
def admin?
admin
end
end


Of course, your user model should have a boolean column "admin" and an admin? method.

  1. Visit http://localhost:3000/jobs to see:
  2. Running jobs
  3. Failed jobs with stack traces
  4. Job history
  5. Performance statistics

Ability to retry failed jobs

Monitoring and Debugging

# Console Rails
rails c
# Voir tous les jobs en attente
SolidQueue::Job.count
# Voir les jobs par queue
SolidQueue::Job.group(:queue_name).count
# => {"default"=>5, "mailers"=>12, "reports"=>2}
# Voir les jobs échoués
SolidQueue::FailedExecution.count
# Voir le dernier job échoué
failed = SolidQueue::FailedExecution.last
failed.error # Message d'erreur
failed.exception_executions # Stack trace
# Réessayer un job échoué
failed.retry
# Purger les vieux jobs terminés (gardés 7 jours par défaut)
SolidQueue::Job.finished_before(7.days.ago).delete_all

Check jobs in the console

# config/environments/production.rb
config.active_job.logger = Logger.new(Rails.root.join('log', 'jobs.log'))

Useful logs

Solid Queue vs Other Solutions
Solution
Speed
Features
Setup
Dependencies
Solid Queue
Good
Rich
Very simple
None
Sidekiq
Excellent
Very rich
Simple
Redis
GoodJob
Good
Rich
Simple
None
Resque
Average
Basic
Simple
Redis
DelayedJob
Slow
Basic
Very simple


None

  1. Use Solid Queue if:
  2. You are starting with Rails 8
  3. You want zero external dependencies
  4. Your needs are standard (80% of apps)

You have a few thousand jobs per hour

  1. Switch to Sidekiq if:
  2. You have tens of thousands of jobs per hour
  3. You need advanced features, such as async jobs

Performance is critical

  1. Consider GoodJob if:
  2. You want the benefits of Solid Queue


But with an even more advanced admin interface


Conclusion: Solid Queue, simplicity in the service of performance

Solid Queue marks a natural evolution in the Rails ecosystem: a solution for managing asynchronous jobs that focuses on simplicity, robustness, and native integration. No more external dependencies like Redis or Sidekiq: with an SQL database and minimalist configuration, Solid Queue lets you manage background tasks with elegance and efficiency.


Whether it's sending emails, generating reports, or orchestrating complex workflows, Solid Queue proves to be a reliable ally for Rails developers. And with Rails 8 integrating it by default, it's time to rethink how we design asynchronous processing.