Go topics
Publish one event to many queues, and subscribe Go workers to topics.
A topic sends one event to many queues. client.Publish makes one job for each subscription of the topic, in one transaction. Each copy is a normal job in the queue of its subscriber, with its own retries. For the model, see topics.
Publish one chat.message event:
published, err := client.Publish(ctx, "chat.message", []any{map[string]any{"room": "general", "body": "hi"}}, anway.PublishOptions{
ID: "msg-42",
})
if err != nil {
return err
}
log.Printf("event %s made %d jobs", published.Event, published.Jobs)Three apps subscribe to chat.message, each with its own event and queue. Each app declares its subscription in its worker. In this short example, one worker has all three:
worker := anway.NewWorker("", anway.WorkerOptions{})
topics := []string{"chat.message"}
worker.On("chat.broadcast", broadcast, anway.Options{Queue: "realtime", Topics: topics})
worker.On("chat.store", store, anway.Options{Queue: "cassandra", Topics: topics})
worker.On("chat.notify", notify, anway.Options{Queue: "push", Retry: anway.Int(5), Topics: topics})
return worker.Run(ctx)The publish above then makes three jobs: one in each of the realtime, cassandra, and push queues.
Subscribe in a worker
Put the topics in the Topics option of worker.On. For each topic, the worker records one subscription with these values:
| Subscription field | Value |
|---|---|
| Topic | The topic name. |
| Event | The event of the declaration. Each copy is a job with this event. |
| Queue | The Queue of the declaration, or default. |
| Retry | The Retry of the declaration. |
| ReserveFor | The ReserveFor of the declaration. |
RunsendsSUBSCRIBEfor each subscription before its first fetch.Runreturns an error when a subscription fails.- The worker fetches from the queue of each declaration that has topics.
Runsends the subscriptions again at each start, so the worker code sets them. A secondSUBSCRIBEwith the same topic, queue, and event keeps one subscription, with the newRetryandReserveFor.- The server saves subscriptions on disk. A copy goes to the queue also when the subscriber app is down. The copy waits in the queue until a worker fetches it.
- To send one topic to two flows in one app, declare two events, each with the topic. A worker has one handler for each event.
An event that is published before a subscription exists does not reach that subscription.
Publish
client.Publish(ctx, topic, args, options) sends an event to every subscription of the topic. Every copy gets the same args.
window := 10 * time.Minute
published, err := client.Publish(ctx, "product.created", []any{item}, anway.PublishOptions{
ID: item.ID,
UniqueFor: &window,
Custom: map[string]any{"source": "catalog"},
})
if err != nil {
return err
}
if published.Duplicate {
log.Printf("product %s was already published", item.ID)
}All fields of anway.PublishOptions are optional.
| Field | Type | Default | Meaning |
|---|---|---|---|
ID | string | The server makes one | The event id. |
UniqueFor | *time.Duration | One day (server) | With an ID, a second event with the same topic and id in this time makes no job. A zero value turns the check off. The server maximum is 30 days. |
Custom | map[string]any | None | Keys that go to each copy. The server adds topic and event to them. |
Publish returns an anway.Published:
| Field | Meaning |
|---|---|
Event | The event id: your ID, or the id that the server made. |
Jobs | The number of copies, one for each subscription. |
Duplicate | true when the same topic and id were published in UniqueFor. Then no job was made. |
Use a stable id, such as the id of the record, when the same publish can run twice, for example after a network error. Without an ID, the server does no duplicate check.
An event with no subscription is not lost. It goes to the Dead set as one job, and Jobs is 0. A requeue of that dead job publishes the event again to the subscriptions of that moment.
Publish in bulk
client.PublishBulk(ctx, events) sends up to 1,000 events (anway.MaxBulk) with one PUBLISHB command. The server writes them in one write. It needs server v0.5.2 or later.
events := make([]anway.Event, 0, len(items))
for _, item := range items {
events = append(events, anway.Event{Topic: "product.created", Args: []any{item}, Options: anway.PublishOptions{ID: item.ID}})
}
results, err := client.PublishBulk(ctx, events)
if err != nil {
return err
}
for _, result := range results {
if result.Error != "" {
log.Printf("event %s refused: %s", result.Event, result.Error)
}
}PublishBulk returns one anway.PublishResult for each event, in the same order. It has the fields of Published, and Error for a refused event. A refused event does not stop the others. More than 1,000 events return an error before the client sends them. See Bulk publish.
Idempotent handlers
Each copy is delivered at least once. The server adds the event id to the custom hash of each copy, so use it as the key for idempotence:
worker.On("chat.store", func(ctx context.Context, job *anway.Job) error {
eventID, _ := job.Custom["event"].(string)
return store(ctx, eventID, job.Args)
}, anway.Options{Queue: "cassandra", Topics: []string{"chat.message"}})Subscribe from any app
client.Subscribe and client.Unsubscribe change subscriptions without a worker.
err := client.Subscribe(ctx, anway.Subscription{
Topic: "chat.message",
Queue: "audit",
Event: "chat.audit",
Retry: anway.Int(3),
ReserveFor: 60,
})Subscribeis idempotent. A second call with the same topic, queue, and event sets the newRetryandReserveFor.ReserveForin aSubscriptionis anintin seconds, not atime.Duration.
Unsubscribe
client.Unsubscribe(ctx, topic, queue, event) removes one subscription. A missing subscription is not an error.
if err := client.Unsubscribe(ctx, "chat.message", "audit", "chat.audit"); err != nil {
return err
}The server keeps a subscription until you remove it. When you remove Topics from a declaration, the worker stops sending SUBSCRIBE, but the old subscription stays. Call Unsubscribe to remove it.