Getting started

Overview

This guide will show you how to:

Create your first stream
Create a sync to an API provider in the stream
Create a consumer and read data from the stream
Write data back to the API provider

Then, you’ll see how to do all this programmatically using Sequin’s Management API.

Create a stream

Streams are at the center of Sequin. The data from your syncs is stored in a stream. Your consumers read from a stream, keeping track of which messages you have processed and which still need to be delivered.

To get started, you’ll need a Sequin account. If you don’t have one, you can sign up for free. Once you’re logged in, you’ll be prompted to create your first stream:

Click “Create stream” and enter a name. I chose dev as this stream is just for development, not yet production.

Create a sync

That’s it! Our first stream is created. Now let’s add a sync.

In Sequin, an instance of a connection to an API is called a sync. Sequin uses a credential (e.g. an access token) to access the API.

Here are some examples of things you can sync:

A Salesforce account
A HubSpot sandbox
A Shopify store
A Stripe account

You can connect to your API account or to your customers’ API accounts. This guide will focus first on connecting to an account you own, but later we’ll show you how to programmatically connect to your customers’ accounts.

On our stream page, you will see no syncs yet and a prompt to create one.

Click “Create sync”.

Our first option is which provider to connect to. I chose HubSpot, but feel free to choose any provider that you have a login or an API key for.

Once selected, you are prompted to create to connect to HubSpot. This will authorize Sequin to access your HubSpot account through a credential:

Create a credential

In the “Credential” section, click + Add New then Connect Sequin to HubSpot. Follow the prompts in HubSpot to authorize Sequin.

Select collections to sync

Under “Collections,” select the data you want to sync. A collection is a type of API object, like a Salesforce Contact or a Shopify Order.

Start your sync

Click Start syncing.

Introspect

After starting your sync, introspect the shape of objects flowing from the API. Click the “Records” tab to see the list of records Sequin has extracted so far:

You can click on any record to see an expanded view. You can make changes to records in the API and see the corresponding records update in real-time!

Create a consumer to read data

Now that you are syncing data to your stream, let’s add add a consumer to read data into your application.

Sequin’s Consume API is an HTTP interface for processing all your API data. It provides the same capabilities as modern messaging systems, like Kafka or NATS. You can process data in parallel, at high-throughput, and with strong guarantees about message delivery.

Navigate to your dev stream page and click the Consumers tab. Click Create consumer:

There are a few configuration options available when creating a consumer:

Deliver Policy

string

Where in the stream the consumer starts.

all (default): the consumer will start from the beginning of the stream.
new: the consumer will start from the most recent message.

Ack Wait

integer

How long to wait for a message to be acknowledged, in seconds. Defaults to 60 seconds.

If a worker doesn’t acknowledge a message within this time, the message will be redelivered (i.e. reappear for other workers to process).

Max Deliveries

integer

Maximum number of times a message can be redelivered to workers. For example, if a message causes your workers to crash, it will be redelivered up to max_deliver times before being discarded.

Defaults to -1, which means there is no maximum.

For now, the defaults will work. To learn more about configuring your consumer, see the consume docs.

Click “Create”. You now have a consumer and are ready to ingest data from your stream.

On the consumer page you can see:

The configuration details you selected
The number of messages that are pending delivery (num pending)
The number of messages that are pending acknowledgement (num ack pending)
Code snippets to consume records from this consumer

At this point, if you have not already, you should create an API Token. Navigate to Account -> API Tokens and click “Create API Token”.

The code snippets automatically have your API Token so they are ready to run. Try copying the the /next snippet into your terminal to request your first batch of records!

curl --request POST \
  --url https://api.sequin.io/v1/http-consumers/{id}/next \
  --header 'Authorization: Bearer {api token}' \
  --header 'Content-Type: application/json' \
  -d '{ "batch_size": 10 }'

200 OK

[
  {
    "ack_token": "NjA1",
    "record": {
      "sync_id": "my-sync-id",
      "collection_id" : "stripe:subscription",
      "upstream_id": "sub_tognkns00nj",
      "updated_idx": 2745327037,
      "upstream_created_at": "2024-11-10T18:38:00.070453Z",
      "upstream_updated_at": "2024-11-10T18:38:00.070453Z",
      "data": {
        "id": "sub_tognkns00nj",
        "object": "subscription",
        "billing": "charge_automatically",
        // …
      }
    }
  },
  // ...
]

Consumers are the best way to build a sync to postgres or trigger workflows in your application when API data changes.

Mutate data

On top of reading data, your integration probably needs to write data back to APIs as well. Sequin has a mutation API that allows you to create, update, and delete API records.

With the mutation API, mutations look the same regardless of the API. The data payload for API records matches the same shape used everywhere in Sequin.

Mutations are linked to syncs. When you create a mutation, you’ll specify the sync ID of the sync you want to mutate. Sequin will route the mutation to the right place.

Here’s an example of creating a Salesforce Contact:

curl -X POST \
  https://api.sequin.io/v1/mutations/run \
  -H 'content-type: application/json' \
  -d '{
        "kind": "create",
        "sync_id": "my-sync-id",
        "collection_id": "salesforce:contact",
        "data": [
          {
            "FirstName": "Paul",
            "LastName": "Atreides",
            "Email": "paul@arrakis.org"
          }
        ]
      }'

To learn more about mutations, check out our docs on the mutation API.

Management API

Everything you can do in the Sequin console you can do via Sequin’s Management API. This empowers you to write code that starts syncs, manages consumers, or anything else.

You’ll use the Management API if you’re programmatically connecting to your customers’ API accounts. For example, if you’re building a product that syncs data from your customers’ Salesforce accounts to your app, you’ll use the Management API to capture credentials and spin up syncs for each customer.

You can also use the Management API to integrate Sequin into your development and CI/CD workflows.

There’s a lot you can do with the Management API. Here’s a sampling of some common tasks:

List syncs

List all the syncs in your account:

curl --request GET \
  --url https://api.sequin.io/v1/syncs \
  --header 'Authorization: Bearer <YOUR_API_KEY>'

Add a credential

Add a credential to your account:

curl --request POST \
  --url https://api.sequin.io/v1/credentials \
  --header 'Content-Type: application/json' \
           'Authorization: Bearer <YOUR_API_KEY>' \
  --data '{
    "kind": "stripe_oauth2",
    "access_token": "sk_live_7eC39HqLyjWDarjtT1zdp7dc",
    "oauth_app_id": "8ff58ef0-d376-4ae8-b2e2-9f0206aa65b8",
    "metadata": { "custom_property": 42 }
  }'

Create a sync

Create a new sync using the credential you just created:

curl --request POST
  --url https://api.sequin.io/v1/syncs
  --header 'Content-Type: application/json' \
           'Authorization: Bearer <YOUR_API_KEY>' \
  --data '{
    "provider": "stripe",
    "name": "Choam Corp - Stripe Production",
    "collection_ids": ["stripe:customer", "stripe:invoice"],
    "rate_limit": {
      "allowed_requests": 100,
      "interval": 60,
      "max_concurrent_requests": 0
    },
    "credential_id": "8ff58ef0-d376-4ae8-b2e2-9f0206aa65b8",
    "metadata": { "custom_property": 42 }
  }'

Questions and support

If you have any issues or questions, don’t hesitate to reach out. We’d love to hear what you’re building.

Overall

Auth

Streams

Mutations

How to

Getting started

Overview

Create a stream

Create a sync

Introspect

Create a consumer to read data

Mutate data

Management API

List syncs

Add a credential

Create a sync

Questions and support

Overall

Auth

Streams

Mutations

How to

​Overview

​Create a stream

​Create a sync

​Introspect

​Create a consumer to read data

​Mutate data

​Management API

​List syncs

​Add a credential

​Create a sync

​Questions and support

Overview

Create a stream

Create a sync

Introspect

Create a consumer to read data

Mutate data

Management API

List syncs

Add a credential

Create a sync

Questions and support