Webhooks

Sequin’s webhook consumer is simple but fast and efficient. You can filter the data you want a consumer to handle by specifying syncs, collections, and event types. The consumer is configured with a sensible retry and backoff policy to minimize the chances of your system missing a webhook.

​

Overview

Your webhook consumer can consume records from the records stream or the events from the events stream.

You’ll create an HTTP endpoint for Sequin to send webhooks to. You can have the consumer consume everything in your stream or filter by specific syncs, collections, or event types.

A single consumer can send webhooks to a single HTTP URL. If you want to send webhooks to multiple endpoints, you’ll need to create multiple consumers. So for simplicity, most customers create a single HTTP endpoint that receives all webhooks and then routes them internally.

You can choose to initialize your webhook consumer at the beginning or at the end of the stream.

After setting up a webhook consumer, Sequin will send webhooks to your endpoint in real-time. If your endpoint is unreachable, Sequin will retry sending the event for up to 24 hours. If your endpoint is still unreachable after 24 hours, Sequin will stop retrying and send you an email alert.

You can add or remove syncs, collections, and event types from a consumer at any time.

Sequin’s webhook consumer is located in AWS’ us-west-2.

​

Setup

​

Add your webhook endpoint to Sequin (console)

Click on “Targets” in the Sequin console and then click on “Add new → Webhook endpoint.”

You’ll be prompted for details like your endpoint’s base URL. You can provide Sequin with credentials that the consumer can use to authenticate with your URL.

​

Add your webhook endpoint to Sequin (API)

Alternatively, you can add webhook endpoints to Sequin via our Management API.

​

Setup the consumer

The consumer is the worker cluster that will pull events off your event stream and send them to your webhook endpoint. You can have one consumer pull data from multiple collections across multiple syncs writing to one webhook target.

In the Sequin console, click on “Consumers” and then click on “Add new → Webhooks.” That will bring up Sequin’s webhook consumer setup flow:

1. Select target

Select your webhook endpoint as the target.

2. Select syncs

Select which syncs you want this consumer to pull data from. You can change this list of syncs at any time in the future.

You can have one webhook consumer pull data from one sync or thousands of syncs.

3. Select collections

Select which collections you want this consumer to consume, for example Salesforce contacts or Stripe subscriptions.

4. Select event types

Select which event types you want this consumer to consume, for example created or updated.

5. Select start position

Select whether you want this consumer to start consuming events from the beginning of the event stream or from the end of the event stream.

Starting at the beginning can be useful as it will “seed” your webhook endpoint with events. You can, for example, setup logging in your receiver to get a feel for what your events look like.

6. Save

After saving, Sequin’s webhooks consumer will begin sending events right away.

​

When to use webhooks

Webhooks are a great way to get started with consuming data. They leverage existing infrastructure like application servers and load balancers. And they’re often familiar.

However, they have some notable limitations. Webhooks are a “push-based” event model vs a “pull-based” one (like Kafka or polling), which means:

Local development is harder: Each developer on your team can easily spin up their own Sequin webhooks consumer and send events to their local machine. But if you want to send events to a local machine, you’ll need to setup a tunneling service like ngrok.

Contrast to pull-based solutions like the Stream APIs or Kafka: no tunneling needed, development machines can simply pull events from the event stream.

No back pressure: If your application is overloaded and can’t handle the volume of events Sequin is sending, there is no easy way for you to apply back-pressure to Sequin.

In a pull-based solution, you’ll simply fall a little behind in processing events. When your acute load is alleviated, your application will catch up.

No control over backoffs and retries: Related, Sequin’s webhook consumer is configured with a sensible backoff and retry policy. However, this can mean that Sequin throttles sending events to your application for a period of time. You might deploy a fix, yet still need to wait for the retry interval to elapse before Sequin starts sending events again.

In a pull-based solution, on your next deploy your consumer can simply start pulling events again from where it left off.