Rails 5 with sidekiq adapter and redis queues

Sidekiq

Sidekiq is a thread based efficient background processing solution for ruby. Read more about it here.

Let’s start configuring sidekiq for our application. First thing we need is to sidekiq as a gem. Sidekiq installs redis as a dependency.

# in your Gemfile
gem install sidekiq

By default sidekiq tries to connect to Redis at localhost:6379 . You can define your redis configuration via initializer.

# config/initializers/sidekiq.rb
Sidekiq.configure_server do |config|
  config.redis = { url: 'redis://redis.example.com:7372/0' }
end

Sidekiq.configure_client do |config|
  config.redis = { url: 'redis://redis.example.com:7372/0' }
end

More options to play around are available here.

After your sidekiq and redis is in place, now it’s time to define sidekiq configuration. First we define the queue adapter as sidekiq in the application.

# config/application.rb
config.active_job.queue_adapter = :sidekiq

Now let’s create one worker job

# app/jobs/hello_world_job.rb
# frozen_string_literal: true
class HelloWorldJob < ApplicationJob
  queue_as :low_priority
def perform
    puts 'Hello World!'
  end
end

This is a simple hello world job which queues the job to low_priority queue. To run this job just start the sidekiq handler with this queue.

bundle exec sidekiq -q low_priority

This will run jobs enqueued to low_priority queue. This is also an example of reserved queues — meaning this queue will handle only low_priority jobs.

But what if you want to run multiple queues? Passing them as arguments is allowed but can be a headache. This is where sidekiq configuration comes into place. In this configuration, you can define multiple queues, their priorities, timeout, concurrency, etc.

By default sidekiq looks for the configuration file in config/sidekiq.yml and if the file is present the workers start with defined configuration. You can also define the custom path to your configuration and run it with bundle exec sidekiq -c config/your_sidekiq_config_path/config_name.yml

# config/sidekiq.yml
---
:verbose: false
:concurrency: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
:timeout: 30
:queues:
  - [low_priority, 1]
  - [high_priority, 2] # high priority

There are 2 queues defined with different priorities in the above configuration. The high_priority queue lookup is twice the lookup of low_priority queue. So processing of jobs is like 22122122… (where 2 priority jobs are executed and then one low priority.)

You can define concurrency based on the machine configuration you are running your workers on. By default one sidekiq process starts 10 threads. Read more about concurrency here.

Configuring Sidekiq Dashboard

Sidekiq provides a dashboard to manage jobs. You can enable it by defining a route in your routes.rb .

require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'

Let’s say your application is running on localhost:3000, you can access the dashboard on localhost:3000/sidekiq . Setting up authentication around the dashboard can be found here.

The dashboard will look something like this

Sidekiq Dashboard (Source https://github.com/mperham/sidekiq)

That’s all. It’s easy to configure sidekiq. The performance of sidekiq vs Resque can be seen here. The official sidekiq wiki is available here.

Learn how to use cron schedulers with sidekiq in another blog post.