Active Job
Use Anway as the Active Job adapter in Rails, and run the server and workers with Puma.
With the Anway adapter, your existing Active Job classes run on Anway workers. You keep perform_later, retry_on, and perform_all_later, and you get the Anway console, retries, and autoscale.
config.active_job.queue_adapter = :anwayclass PublishProductJob < ApplicationJob
queue_as :default
def perform(product)
product.update!(status: 'processing')
product.update!(status: 'published', published_at: Time.current)
end
endPublishProductJob.perform_later(product)Set up
Install the gem with the adapter
bundle add anway
bin/rails generate anway:install --active-jobWith --active-job, the installer adds config.active_job.queue_adapter = :anway to config/application.rb. It also writes config/anway.yml and bin/anway, and adds plugin :anway to config/puma.rb.
Start the app
bin/devHow Active Job jobs run
- Each job runs in the Rails reloader.
- The job goes to the queue of
queue_as. - The console shows the name of the job class.
set(wait:)andset(wait_until:)send a scheduled job.- A job with no
retry_onuses the retries of the server. Refer to Retries. - A job with
retry_onkeeps the Active Job attempts. Its final error sends the job to the Dead set.
Active Job classes and Anway::Worker classes can run in the same worker process.
Transactions
The gem has a Railtie. When Active Record loads, a push inside a database transaction waits for the commit, and a rollback cancels it. This applies to perform_later, perform_async, perform_bulk, Anway.push, and Anway.publish.
ActiveRecord::Base.transaction do
product = Product.create!(name: 'Lamp')
PublishProductJob.perform_later(product)
endThe job goes to the server only after the commit, so the worker always finds the product.
Push many jobs
ActiveJob.perform_all_later sends one PUSHB for each 1,000 jobs, for jobs of all classes and queues:
jobs = Product.draft.map { |product| PublishProductJob.new(product) }
ActiveJob.perform_all_later(jobs)- A job with
set(wait:)orset(wait_until:)keeps its time. - Each job gets
successfully_enqueued. - A job that the server refused gets an
ActiveJob::EnqueueErrorinenqueue_error.
Refer to Bulk push.
What the Railtie does
- It registers the
:anwayadapter. - It makes pushes wait for the transaction commit, as above.
- It sends pushes from the web process to the same server as the workers: the embedded one, or the url in
config/anway.ymlorANWAY_URL. A url that you set in an initializer wins. - It uses
Rails.loggerfor the gem logs. - It runs each job in the Rails reloader.
- It reports job errors to
Rails.error.
Run the server and workers in development
plugin :anway in config/puma.rb runs bin/anway next to Puma, and stops it with Puma. In development, bin/dev starts Puma, so it also starts the embedded server and the workers.
plugin :anwayANWAY_IN_PUMAturns the plugin on or off, asSOLID_QUEUE_IN_PUMAdoes for Solid Queue. It takestrueorfalse.- Without
ANWAY_IN_PUMA, the plugin runs only in development. - If
bin/anwaystops by itself, Puma stops too, so a broken worker is not hidden.
To run the process without Puma, use bin/anway. For a server that runs apart, use bin/anway worker.
Code reloading in the worker
With Solid Cable 4.1, a code reload in one job thread can deadlock with a broadcast from another job thread. To prevent it, turn off reloading in the worker process in development:
config.enable_reloading = ENV['ANWAY_WORKER'].blank?Web requests still reload as usual. After a change to a job, run bin/rails restart. It restarts Puma and the worker with the new code.
Database pool
Each job thread can hold an Active Record connection. If the worker runs more threads than its pool, jobs fail with ActiveRecord::ConnectionTimeoutError. The gem protects you:
- At start, the worker logs a warning when the pool is smaller than
concurrency. - The worker sends its pool size to the server as its limit. The server refuses a thread count or an autoscale
maxabove the limit. - An autoscale
maxinconfig/anway.ymlabove the pool goes down to the pool size, and the worker logs a warning at start.
Make the pool of the worker process as large as concurrency, or as autoscale max. The worker process sets ANWAY_WORKER=1 before Rails boots, so the web process keeps its own pool:
default: &default
adapter: sqlite3
max_connections: <%= ENV.fetch("RAILS_MAX_THREADS") { ENV["ANWAY_WORKER"] ? 40 : 5 } %>Rails 8.1 calls the key max_connections. Before Rails 8.1, it is pool.
Check the pool that the worker gets:
ANWAY_WORKER=1 bin/rails runner 'puts ActiveRecord::Base.connection_pool.size'The database server must also accept the connections. For PostgreSQL, max_connections must hold the pool of each process.
Mount the console
Anway::Web serves the Anway web console inside your app. Mount it only in a route that checks the user, because the console can kill and delete jobs:
authenticate :user, ->(user) { user.admin? } do
mount Anway::Web => '/anway'
endThe console needs a console binary. Set console.path in config/anway.yml, or ANWAY_CONSOLE_PATH. Then bin/anway starts the console next to the server, on 127.0.0.1:7421. Refer to Configuration and Console.
Move from Solid Queue
Tasks in recurring.yml and jobs that wait in Solid Queue do not move automatically.
- Drain the old queue first.
- Move the recurring tasks to the cron of the server. Refer to Server configuration.
- Set
config.active_job.queue_adapter = :anway.