Anwaydocsv0.5.3

Node.js workers

Run jobs in Node.js or Bun with Anway.Worker, worker.on, and worker.run.

A worker fetches jobs from the server and runs the handler for each job. You declare one handler for each event, and the worker runs many jobs at the same time in the event loop.

worker.ts
import Anway from '@anway/client';

const worker = new Anway.Worker({ queues: ['critical', 'default'], concurrency: 10 });

worker.on<[string]>('email.send', async (job, { signal }) => {
  const [address] = job.args;

  await sendEmail(address, { signal });
});

await worker.run();

Run the file as a normal process: node worker.ts or bun worker.ts.

new Anway.Worker(options) creates a worker. The url option, or ANWAY_URL, sets the server, as for a client. worker.run() connects, fetches, and runs jobs until a stop.

Declare handlers

worker.on(event, handler, ...options) declares the handler of an event. The worker runs each job whose event is this event. on returns the worker, so you can chain calls.

worker
  .on('report.build', buildReport, { queue: 'reports' })
  .on('report.mail', mailReport, { queue: 'reports' });
  • A worker has one handler for each event. A second on for the same event throws an Error.
  • on uses only queue, retry, reserveFor, and topics. The keys for a push, such as at, uniqueFor, and custom, have no effect in on.
  • When a declaration has queue or topics, the worker also fetches from that queue.
  • retry and reserveFor go to the topic subscriptions of the declaration. See topics.

on takes more than one option set. The sets merge in order:

  • A single value, such as queue or retry: the last set that has it wins.
  • topics: the topics of all sets add up, in order, with no duplicates.
  • custom: the objects merge, and a later key wins.
const realtime: Anway.Options = { queue: 'realtime', retry: 3 };

worker.on('chat.broadcast', broadcast, realtime, { topics: ['chat.message'] });

The worker fetches from queues (default ['default']), and then from the queues that on adds.

The handler

A handler gets the job and the job context:

type Handler<Args extends unknown[] = unknown[]> = (job: Anway.Job<Args>, context: Anway.JobContext) => unknown;

The type argument of on sets the type of job.args:

type Product = { id: string; name: string; price: number };

worker.on<[Product]>('product.create', async job => {
  const [product] = job.args;

  await saveProduct(product);
});

The job has these fields:

FieldMeaning
jidThe job id.
typeThe event of the job.
argsThe args of the job.
queueThe queue of the job.
retry, reserveFor, backtraceThe options of the job.
at, createdAt, enqueuedAtThe times of the job, as ISO 8601 strings.
customThe custom hash of the job.
failureThe last failure: retry_count, failed_at, next_at, message, errtype, and backtrace. It is not set at the first run.

The job context has one field, signal, an AbortSignal. It aborts at a shutdown, when the shutdown timeout ends. Pass it to every call that can wait, such as fetch and database queries.

Middleware

worker.use(middleware) wraps all handlers. The first middleware that you add is the outermost.

worker.use(async (job, next) => {
  const started = Date.now();

  const result = await next();

  console.log(job.type, Date.now() - started);

  return result;
});

Errors and retries

A handler that resolves acknowledges the job, and the server removes it. A thrown error or a rejected promise fails the job.

  • The worker sends the name of the error as the error type, its message, and the first 30 lines of its stack.
  • A thrown value that is not an Error goes out with the type Error and the value as text.
  • A job with no handler for its event fails with an UnknownJobError.

The server retries a failed job. It waits longer after each failure, until no retries remain. Then the job goes to the Dead set. The retry option of the job sets the number of retries. See retries.

Delivery is at least once. A worker can get the same job again, for example after a crash. Make each handler idempotent.

When the worker cannot report a result, it logs a warning. The server then gives the job again when its reservation (reserveFor) expires.

Concurrency

Node.js and Bun run JavaScript on one thread. A worker does not start threads. It starts job slots. A slot is one fetch-and-run loop in the event loop: it fetches a job, runs it, and then fetches the next job. concurrency (default 10) is the number of slots, so it is the number of jobs in progress at the same time.

  • More slots help jobs that wait on I/O, such as HTTP calls or database queries.
  • More slots do not help jobs that use the CPU. Synchronous CPU work blocks the event loop, and this also stops the heartbeat. Move heavy CPU work to worker_threads, or run more processes.
const worker = new Anway.Worker({ concurrency: 20, maxConcurrency: 40 });
  • The connection pool of the worker has one connection for each slot, plus one.
  • maxConcurrency is the most slots that the worker can run, such as the size of its database pool. The worker never runs more. A concurrency above maxConcurrency throws a RangeError.

Each slot can hold a database connection. Set maxConcurrency to the size of your database pool, or set the autoscale max at or below it. If not, jobs can fail with a pool timeout.

Autoscale

The web UI shows the slots as threads. The server can change the number of slots while the worker runs: from the web UI, from the HTTP API, or with autoscale. autoscale starts the worker with autoscale on.

const worker = new Anway.Worker({
  concurrency: 5,
  hostname: 'worker.1',
  tag: 'shop',
  autoscale: { min: 5, max: 40 },
  maxConcurrency: 40,
});
  • autoscale is { max } or { min, max }. Without min, the server uses concurrency as the min. The server lowers a max above maxConcurrency to it.
  • When the server asks for more slots, the worker starts them at once. When it asks for fewer, a slot stops after its job. The worker never stops a running job to scale down.
  • When the server asks for more slots while a stop is pending, the worker cancels the stop first.
  • The connection pool grows with the slots.
  • After a change, the worker sends a heartbeat with the new count at once.
  • After quiet, the worker does not change its slots.

The server saves a change from the web UI or the API for the worker name: hostname and tag. A new process with that name starts with the saved setting. See autoscale.

Worker identity

The web UI shows each worker by its hostname and tag.

OptionDefaultMeaning
hostnameos.hostname()The name of the worker.
tagNoneA short name for the app or the role of the process, such as shop.
labels[]More labels. The worker always sends the label js first.

The worker also sends its process id, and a new random worker id at each run.

A container often gets a random hostname at each deploy. Set a stable hostname, such as worker.1, so that a saved thread setting stays across deploys.

Heartbeat

The worker sends a heartbeat (BEAT) every beatInterval (default 15 seconds). The first heartbeat goes out when run starts.

  • The heartbeat tells the server that the worker is alive, its state, and the number of slots that run now.
  • The reply can ask for a new number of slots, a state change (quiet or terminate), or both.
  • When a heartbeat fails, the worker logs a warning and continues. It tries again at the next interval.

run rejects when the first heartbeat fails, for example when the server is not available.

Graceful shutdown

The worker stops in these cases:

  • The process gets SIGTERM or SIGINT.
  • The signal that you give to run aborts.
  • A heartbeat reply has terminate.

At a stop, the worker does these steps:

  1. It stops fetching. Running jobs get shutdownTimeout (25 seconds) to finish.
  2. It aborts context.signal. Jobs get cancelGrace (5 seconds) to return.
  3. It fails the jobs that still run, and run resolves. A late result from these handlers is ignored.

JavaScript cannot stop a promise. A handler that ignores signal continues to run until the process ends. Set the stop timeout of your container above 30 seconds.

To handle signals yourself, turn off the signal handlers and give an AbortSignal:

const stop = new AbortController();

process.once('SIGTERM', () => {
  stop.abort();
});

await worker.run({ signal: stop.signal, handleSignals: false });

A heartbeat reply with quiet stops fetching. The worker stays alive, continues to send heartbeats, and finishes its running jobs. It stops at terminate, at a signal, or when the signal of run aborts.

Server restarts

The server does not send quiet or terminate when it shuts down. A worker reconnects and fetches again when the server comes back.

Worker options

All options are optional.

OptionTypeDefaultMeaning
urlstringANWAY_URLThe server address.
queuesstring[]['default']The queues to fetch, in order. on can add more.
concurrencynumber10The number of job slots when the worker starts.
maxConcurrencynumberNoneThe most slots that the worker can run.
autoscale{ min?, max }OffStarts the worker with autoscale on.
hostnamestringos.hostname()The name of the worker in the web UI.
tagstringNoneA short name for the app or the role of the process.
shutdownTimeoutnumber25000Milliseconds for running jobs to finish at a stop.
cancelGracenumber5000Milliseconds for canceled jobs to return.
beatIntervalnumber15000Milliseconds between heartbeats.
labelsstring[][]More labels. The worker also sends js.
timeoutnumber10000Milliseconds for each server call.
tlstls.ConnectionOptionsNonenode:tls options for a tcp+tls:// URL.
logger{ warn }consoleThe logger for warnings, such as a failed fetch.

worker.run(options) takes two options:

OptionTypeDefaultMeaning
signalAbortSignalNoneThe worker stops when it aborts.
handleSignalsbooleantrueStop at SIGTERM and SIGINT.

On this page