— Commercial License Changes and the Return to Second-Generation Queue Management

Something interesting happened in our industry recently. Redis, the in-memory data store that became synonymous with modern web architecture, suddenly feels less inevitable than it once did. The licensing changes and operational costs have prompted many e-commerce platforms to reconsider what was once an obvious choice. In the Rails community especially, the "Redis or nothing" mentality is giving way to something more nuanced.

Still Powerful, But Redis Dominance Is Waning

Key Point: Redis still excels in many use cases like caching and pub/sub. However, the reality in 2025 is that more workloads can conclude "persistent layer in DB, queues in DB too" is sufficient.

Rethinking Infrastructure Costs and Maintenance Load

Physical Costs:
Being in-memory means data volume equals memory requirements. As application count and transactions increase, costs scale accordingly.

Operational Costs:
Building and maintaining Redis Sentinel/Cluster configurations, handling failures, and managing version upgrades requires specialized knowledge separate from RDB operations. Even cloud services require ongoing management.

Database Performance Improvements:
Compared to ten years ago, PostgreSQL 15/16 series feature advanced shared buffer management and JIT optimization, making them less likely to bottleneck even at tens of thousands of TPS.

Considering these factors, more projects are concluding that "databases are sufficient for queues."

How PostgreSQL's FOR UPDATE SKIP LOCKED Changed Everything

This feature, available since PostgreSQL 9.5, transformed how we think about queue management. For example, when retrieving tasks from a background processing queue, you execute SQL like this:

SELECT * FROM jobs
WHERE status = 'pending'
ORDER BY id
LIMIT 100
FOR UPDATE SKIP LOCKED;

Retrieving from queue tables this way enables the following capabilities:

• Safe parallel processing by multiple workers:
  • Other processes skip rows locked by others, avoiding conflicts
  • Eliminates risk of different workers processing the same job twice
• Transactional consistency:
  • Data and queue management within the same database makes maintaining data consistency easier
• Simplified infrastructure:
  • Queue management becomes possible without Redis, simplifying system architecture

Second-Generation Queue Management gem — Solid Queue

At Flagship, we've been migrating from Redis-based Sidekiq to Solid Queue throughout 2024. Solid Queue is built by the Rails core team and represents a "second-generation" queue gem that leverages FOR UPDATE SKIP LOCKED for safe job retrieval.

Real-World Case: Migration in Shopify Integration Applications

Specifically, we migrated two Shopify integration Rails applications from Redis/Sidekiq to Solid Queue, gaining significant benefits in infrastructure simplification and operational reliability. Our applications primarily handle background processing for time-sensitive e-commerce operations like periodic order exports and fulfillment processing that run every 15 minutes.

By switching to PostgreSQL-backed Solid Queue, we eliminated Redis dependency while maintaining all scheduled job functionality. Solid Queue's RecurringTask feature made this migration particularly elegant—we could migrate CRON-style scheduled jobs using simple YAML configuration. We maintained existing ActiveJob classes with minimal changes, preserving business logic while changing only the underlying queue infrastructure.

This consolidation simplified monitoring to a single database technology, reduced infrastructure costs on Heroku, and eliminated an entire category of potential failure points. What proved especially valuable was no longer needing to manage separate connection pools and monitoring systems for a second data store. For Shopify integration applications where webhook processing and order fulfillment are critical paths, having these jobs backed by the same transactional database that stores application data improved data consistency and eliminated concerns about cross-datastore synchronization.

Implementation Results

History Repeats — delayed_job's Creator is Shopify's CEO

Before Redis, the Rails community's standard database-backed job gem was delayed_job. Its creator? Current Shopify CEO, Tobias Lütke. This represents a notable example of technical leadership at the executive level.

The progression from first-generation DB (delayed_job) → Redis dominance (Sidekiq) → second-generation DB return (Solid Queue) serves as an excellent example of open source and infrastructure cost dynamics.

Summary: Now That Options Have Expanded, It's About "Right Tool for the Job"

• Redis remains powerful, but we've entered a phase of reevaluating its applications.
• PostgreSQL's evolution and the emergence of FOR UPDATE SKIP LOCKED have made returning job queues to the database a practical option.
• Technology selection balances performance × cost × operations × licensing risk. The commercialization risk of OSS projects is particularly unavoidable.
• Redis or DB—it's not either/or, but rather asking "what best fits current requirements?" That's 2025's best practice. Redis continues to serve valuable functions across many of our applications.