Skip to content

Area: User guide

How pipelines work

A pipeline (Data Platform › Pipelines) is an outbound stream. It takes the events or profile changes that Releva records for your shop and forwards them, one by one, to a destination you own: a webhook, an Amazon S3 bucket or a Google Cloud Storage bucket. Use it to feed a data warehouse, a CRM or your own services with what Releva sees.

Pipelines send data out of Releva. To get customer data in, use the CSV import on Data Platform › Profiles (Import profiles from a CSV file) or the Profiles API. For products, read How your catalog gets into Releva.

Feature Sends When Best for
Pipeline Every matching event or profile change, as it happens Continuously while switched on Mirroring Releva’s event stream or profile changes elsewhere
Webhook (Marketing Channels) One request per profile, with the product recommendations of a campaign or workflow step When a campaign or workflow reaches the step Driving your own channel from a Releva audience
Export A file with results for a date range Once, or on a schedule Reports

A running pipeline processes every record of your shop in this order:

  1. Type. An Event pipeline receives events; a Profile pipeline receives every profile create and update. The type is fixed when the pipeline is created.
  2. Event list (Event pipelines only). With Events action set to Include, only the events listed under Events pass. With Exclude, every event passes except the listed ones. The Events picker leaves out high-volume events (page views, page engagement, product views, searches, UTM session starts, campaign entries, and message sends and deliveries); an Exclude pipeline with an empty list forwards all of them. An Include pipeline with an empty list forwards nothing, although its card reads “Events including all”.
  3. Filter. Your JavaScript function (payload) => boolean from the Advanced filter tab. A falsy result drops the record. The default (payload) => { return payload ? true : false; } keeps everything.
  4. Transform. Your function (payload) => anything from the Transform tab. Its return value is what the destination receives. The default returns the payload unchanged.
  5. Output. A POST to the webhook, or a line in the next file written to the bucket.

Filter and transform run in a sandbox with no network access and no access to Releva’s data beyond the payload. The transform can also use URL (reading hostname, pathname, search and the other properties), btoa and atob; the filter cannot. A function that throws counts the record as Failed. The code is not checked when you save, so test it on a copy of a real payload before switching the pipeline on.

Changes take effect within one minute: the worker reloads the list of running pipelines every 60 seconds. Saving the form never starts or stops a pipeline; only the switch on the list does. A new pipeline is always created switched off.

The payload is the event or profile record as Releva stores it, with these fields normalised:

Field Meaning
type event or profile
profileId Your customer id, without Releva’s internal prefix
action Events only: the event key, such as cartAdd or cartCheckoutSuccess
timestamp ISO 8601 in UTC, for example 2026-10-21T10:20:30.123Z
userAgent Parsed into an object (browser, os, source, …); {} when absent
customString, customNumeric, customDate Arrays of { key, values[] }; dates as ISO strings
products Events only: the products on the event, [] when none
previous Profile pipelines only: the previous values of the fields this write changed, or null for a brand-new profile. Compare it with the current values to react to a change, for example a new subscribedAt.
domainId Your shop id

For the event keys and what each event records, see the Events API overview.

A webhook pipeline sends one POST per record to the Output URL with the transform result as a JSON body and these headers:

Header Value
Your Headers rows As configured, one header per row
Content-Type application/json
X-Pipeline-ID The pipeline’s id
X-Message-ID Internal message id of the batch the record arrived in
X-Releva-Delivery-Id One UUID per record, the same on every retry. De-duplicate on it.
X-Releva-Attempt 1, 2, …
Authorization Bearer <secret> when an Auth Secret is selected. It replaces any Authorization row under Headers.

Any 2xx answer is a success. Delivery is at least once: a retry can repeat a request whose answer was lost, so make your endpoint idempotent on X-Releva-Delivery-Id.

What is retried. Timeouts, connection errors (ECONNREFUSED, ECONNRESET, ETIMEDOUT, temporary DNS failures) and the statuses 408, 429, 500, 502, 503 and 504. Any other status (400, 401, 403, 404, 422, …) and a hostname that does not resolve are treated as permanent: the record is dropped and counted as Failed.

How. The first attempt waits up to 30 seconds. Two quick retries follow after 250 ms and 1 s with a 5-second timeout each. A record that still fails is parked and retried in the background every 30 seconds with a doubling delay (30 s, 1 min, 2 min, … up to 1 hour), with a 30-second timeout. After 8 attempts in total the record is dead-lettered and dropped; dead records are kept for 30 days for support and then deleted. At most 2 000 parked records are kept per pipeline; beyond that, new failures are dropped. Parked records are retried only while the pipeline is switched on and still a webhook pipeline. Deleting the pipeline deletes them.

Auth secrets. Secrets live under Settings › Shop & Integration › Secrets. A static secret is sent as it is; a token-refresh secret is renewed before each call. A secret that is in use by a pipeline cannot be deleted; revoke it instead. Deliveries of a pipeline whose secret was revoked or cannot be refreshed fail and are dead-lettered after 8 attempts; Releva never sends them unauthenticated.

An Amazon S3 Bucket or Google Cloud Storage Bucket pipeline does not call you. It buffers records as JSON Lines (one JSON object per line), compresses them with gzip and writes a file to the bucket under your Output URL prefix (s3://bucket/prefix or gs://bucket/prefix):

<prefix>/<UTC midnight of the day, in epoch milliseconds>/<uuid>.jsonl.gz

A file is written every 30 seconds when the buffer is not empty, or earlier when it reaches about 60 MB. Each object carries the metadata message-count and pipeline-id (S3) or messageCount and pipelineId (GCS) and the content type application/gzip. A failed upload drops that file; there is no retry for bucket outputs.

Bucket outputs use the shop’s own cloud credentials: the AWS account configured in Settings for S3 (Output Region sets the S3 region, for example eu-west-1) and the Google service account for GCS. If the shop has no credentials for the chosen provider, nothing is written and no error is shown; the card still counts the records as Processed. Configure the account before switching the pipeline on.

Each card on the Pipelines list shows four numbers for the selected period:

Counter Counts Note
Sent Records of the shop offered to the pipeline, before the type, event list and filter A profile pipeline’s Sent also counts events, and the other way round, so it is a measure of the shop’s traffic, not of the pipeline’s matches.
Captured Records that passed the event list and the filter Shown with its share of Sent
Processed Records transformed and handed to the output: posted to the webhook, or added to the bucket buffer For bucket outputs this does not mean the file was uploaded.
Failed Captured records that were not processed: the filter or transform threw, or the webhook delivery failed or was parked for retry Shown with its share of Captured
  • Edit. Open a card to change anything but Type. Saving returns you to the list and keeps the switch as it was.
  • Make a copy. Opens a new pipeline prefilled from the current one, named “Copy of …”, switched off; nothing is saved until you select Save. With several shops you pick a Target Domain first. A copy into another shop keeps the Auth Secret reference, which that shop does not have, so clear it before saving.
  • Delete. Asks Are you sure? and removes the pipeline and its parked deliveries.
  • Export on the list creates a one-time export for the selected period under Analytics › Exports (see How exports work).
  • Names are unique per shop. Pipelines are a paid add-on: locked buttons show Unlock feature.

There is no public API for pipelines; create and manage them in the admin.