Sync operations

Syncs, retries, and backfills should be visible before reports trust the data.

Review SIRJ scheduled syncs, manual syncs, webhook-triggered runs, default backfills, retry timing, heartbeat processing, stale-source detection, and sync-run history.

Review connector health Follow data routing Webhook ingestion Usage controls Setup and permissions Data validation

Buyer answers

Explain source freshness before the numbers are challenged.

Reviewers need a readable path for first syncs, historical imports, scheduled due runs, manual retries, webhook ingestion, heartbeat processing, failed runs, and stale-source warnings.

Sync answer

What does sync observability include?

Sync observability includes connector sync mode, cadence, first-sync backfill settings, manual and scheduled run reasons, webhook-triggered runs, resource-listing attempts, raw payload counts, normalized event counts, inserted and updated rows, duration, completion time, errors, last successful sync, last failed sync, next scheduled sync, and consecutive failure count.

A buyer can tell whether a source is fresh, late, setup-blocked, retrying, backfilling, or failing before a dashboard, weekly email, report, answer, or export treats the source as trustworthy.

Review the supporting path

Sync answer

How does SIRJ handle first syncs and backfills?

A first sync can apply connector default backfill settings so provider fetchers request the appropriate starting window before normal cadence takes over. The backfill window stays tied to the connector, selected resources, and sync state instead of being hidden as a background import.

The initial data load has a reviewable boundary, which helps teams understand whether a report is built from a short setup window, a deeper historical pull, or a source that still needs more evidence.

Review the supporting path

Sync answer

How are scheduled, manual, and webhook-triggered runs separated?

Sync run records preserve the run reason and status so a manual operator retry, a scheduled due sync, a webhook ingestion run, and a resource-listing pass can be reviewed as separate operational events even when they update the same connection.

Teams can distinguish an intentional retry from a scheduled pipeline, an incoming provider event, or a setup-resource refresh before using the result in client-facing reporting.

Review the supporting path

Sync answer

What happens when a sync fails or gets skipped?

Failed and skipped runs keep status, reason, error, duration, payload counts, normalized-event counts, completion time, failure count, and next retry timing visible. Connector health can then mark the source as setup required, stale, failing, disconnected, or needing reauthorization.

A quiet chart does not have to be interpreted as quiet performance. The product can show whether the source itself was late, blocked, skipped, or broken.

Review the supporting path

Run workflow

Every source run should say why it happened.

Sync observability connects selected resources, run reason, supported normalization, run history, and downstream freshness controls before a report treats the source as current.

Start with selected resources Apply the right run reason Fetch and normalize supported records Update sync state and run history Protect downstream output

Run types

Manual, scheduled, webhook, and heartbeat work need separate evidence.

The page makes the operational reason visible so teams can tell whether the source was refreshed by setup, schedule, event delivery, retry, or system processing.

Run type	Trigger	Reviewer sees
First sync with backfill	A newly connected source runs with default backfill settings before routine cadence is trusted for current reporting.	Backfill start settings, selected resources, imported raw payloads, normalized events, inserted rows, updated rows, last success, and next scheduled sync stay tied to the connection.
Manual sync	An operator asks a connection to run now after setup, resource selection, a provider outage, or a disputed reporting review.	The run reason, status, counts, duration, completion time, error, and next sync timing stay visible instead of being blended into the scheduled cadence.
Scheduled due sync	A connection reaches its nextSyncAt time and the due-sync route processes eligible sources for the active tenant scope.	Freshness moves forward only when the run succeeds, while failures preserve last failure time, failure count, and next retry timing for health review.
Webhook-triggered run	A verified provider event arrives through a scoped callback and creates a sync-run record for the ingestion path.	Incoming activity can be reviewed by verifier result, resource match, raw payload creation, normalized events, inserted or updated rows, and sync status.
Resource listing	A setup flow refreshes accounts, forms, pipelines, companies, or other provider resources before a connection can be treated as ready.	Resource-listing attempts and errors can be separated from record ingestion so setup gaps do not masquerade as failed performance data.
System heartbeat	The heartbeat route processes due connection work across tenant scopes when the configured secret is present.	Due sync work runs only for eligible tenant scopes, while skipped subscriptions and missing heartbeat configuration are kept out of normal reporting freshness.

Backfill controls

Historical imports need source-specific boundaries.

Backfills are only useful when the import window, provider filters, selected resources, sync state, and output eligibility are visible beside the first report.

Control	Why it matters	SIRJ path
Connector default backfill window	A first dashboard can look incomplete if the initial import only covers the current day when the client expects historical spend, lead, invoice, or payment context.	Connector definitions and integration API setup carry default backfill settings into provider fetch parameters before routine sync cadence takes over.
Provider cursor and date filters	Ad, CRM, form, payment, and operations APIs often use different cursor names or date filters, so backfill behavior needs source-specific handling.	Provider fetchers read cursor, updated-after, start-date, and backfill-start settings before mapping raw provider rows into normalized events.
Selected-resource boundary	A historical import is dangerous when it pulls the wrong ad account, form, company, location, pipeline, or payment account into a client workspace.	Resource selection and setup checks run before backfilled records can be treated as usable reporting evidence for the client workspace.
Sync state update	Backfill output should not reset freshness as healthy unless the run actually succeeded and produced a defensible state for the connection.	The sync state stores last success, last failure, failure count, retry timing, and next scheduled sync after orchestration decides whether the run succeeded or failed.
Output eligibility	A partially backfilled source can produce attractive charts while still missing the evidence needed for attribution, weekly summaries, or report exports.	Data validation, metric definitions, connector health, and report surfaces keep stale, partial, or unsupported source states visible before output is treated as client-ready.

Failure states

A failed or stale source should not look like normal performance.

Setup requirements, stale data, provider failures, skipped runs, and partial backfills need explicit states before a client-facing surface turns them into confident output.

State	Reviewer sees	Protected surface
Setup required	The source is not treated as ready when credentials, selected resources, webhook activation, pipeline mapping, or required setup checks are still missing.	Dashboards, reports, weekly insights, Ask Data answers, and exports can avoid presenting a connected-but-unusable source as complete proof.
Stale source	The runtime compares last successful sync with connector cadence and current time before marking the connection stale.	Client-facing metrics can show freshness risk instead of turning old provider rows into current performance claims.
Provider failure	A failed run keeps escaped provider error text, duration, status, reason, counts, completion time, failure count, and retry schedule.	Operators can fix the source problem before a client review treats missing leads, revenue, or spend as real business performance.
Skipped due sync	A scheduled pass can be skipped when tenant scope, subscription eligibility, setup, connection state, or timing does not allow safe processing.	The system avoids turning an ineligible or unsafe run into a false success state that would make the source appear fresher than it is.
Partial backfill	Imported rows can be reviewed alongside raw payload count, normalized event count, inserted and updated rows, selected resources, and unsupported payload behavior.	Reports and exports can stay honest about the evidence window instead of making a first import look like a complete historical record.

Workflow

The sync path should be plain enough to audit.

Workflow

Start with selected resources

A sync should begin from a connection that already belongs to the right account and client workspace, with the provider resource selected before records are fetched or accepted.

Setup and permissions pages show the selected account, form, pipeline, company, or payment source that controls what a sync can safely import.

Inspect this path

Workflow

Apply the right run reason

Manual, scheduled, webhook, resource-listing, setup-required, and backfill runs keep separate reasons so operators can understand why a source changed or did not change.

Run history can show the difference between a user-triggered refresh, a scheduled due sync, and an event delivered through a webhook callback.

Inspect this path

Workflow

Fetch and normalize supported records

Provider fetchers and webhook handlers create raw payload evidence, reject unsupported payload types, normalize supported records, and update event rows under tenant scope.

Data routing shows where raw provider activity becomes normalized events before cards, answers, reports, or exports use the records.

Inspect this path

Workflow

Update sync state and run history

The orchestration path records success or failure, last success, last failure, failure count, next sync timing, retry delay, counts, duration, and any escaped error.

Connector health can show fresh, stale, failing, disconnected, setup-required, or reconnect-needed states without hiding the operational evidence.

Inspect this path

Workflow

Protect downstream output

Dashboards, Ask Data, weekly insights, reports, and exports should inherit freshness and evidence boundaries from the sync state underneath them.

Validation and report pages keep stale, partial, unsupported, or setup-blocked data from becoming confident client-ready claims.

Inspect this path

Standards

Keep source freshness explicit, scoped, and hard to misread.

Standard

Do not make first import look complete by default

A first sync can create a polished dashboard before the historical window, selected resources, unsupported fields, or provider limits have been reviewed.

Default backfill settings, provider date filters, source coverage, and output eligibility keep the first import understandable before it is used in a client report.

Standard

Do not merge run reasons

Manual retries, scheduled syncs, webhook events, and setup-resource refreshes answer different operational questions and can mislead reviewers if blended together.

Sync run records keep reason, status, counts, duration, completion time, error, and connection scope visible for each run type.

Standard

Do not treat stale data as normal silence

A late sync can look like weak lead volume, missing revenue, no booked work, or a quiet client period when the real issue is operational freshness.

Connector health compares last success with expected cadence and escalates stale, failing, setup-required, disconnected, or reconnect-needed states before reports rely on the source.

Standard

Do not let system jobs cross tenant scope

A heartbeat or due-sync processor that ignores account and client boundaries could mix source evidence or process subscriptions that should not run.

System heartbeat and due-sync processing build tenant scopes explicitly, require the configured secret, and skip unusable subscriptions before running connection work.

FAQ

Sync observability and backfill questions.

What is a sync run in SIRJ?

A sync run is an operational record for a connector attempt. It can represent manual sync, scheduled due sync, webhook ingestion, resource listing, setup-required checks, or retry behavior, with status, reason, counts, duration, completion time, and errors kept visible.

Does SIRJ backfill historical source data?

SIRJ can apply connector default backfill settings during first sync so provider fetchers request a supported historical window. The exact evidence still depends on connector support, selected resources, provider limits, and output eligibility.

How does SIRJ decide a source is stale?

Connector health compares the connector cadence with last successful sync and current time, while also checking setup requirements, connection status, reauthorization needs, errors, and recent failure history.

Why does heartbeat processing matter?

Heartbeat processing lets due sync work run across eligible tenant scopes, but it must require the configured secret and skip unusable subscriptions so system work does not create misleading freshness states.

Freshness review

Before a report trusts a source, confirm the run history behind it.

Review connector health, then follow the data path through selected resources, backfills, sync reasons, run status, stale-source controls, and supported reporting surfaces.

Review connector health Follow data routing

Sync operations

Syncs, retries, and backfills should be visible before reports trust the data.

Review SIRJ scheduled syncs, manual syncs, webhook-triggered runs, default backfills, retry timing, heartbeat processing, stale-source detection, and sync-run history.

Review connector health Follow data routing Webhook ingestion Usage controls Setup and permissions Data validation

Buyer answers

Explain source freshness before the numbers are challenged.

Reviewers need a readable path for first syncs, historical imports, scheduled due runs, manual retries, webhook ingestion, heartbeat processing, failed runs, and stale-source warnings.

Sync answer

What does sync observability include?

A buyer can tell whether a source is fresh, late, setup-blocked, retrying, backfilling, or failing before a dashboard, weekly email, report, answer, or export treats the source as trustworthy.

Review the supporting path

Sync answer

How does SIRJ handle first syncs and backfills?

Review the supporting path

Sync answer

How are scheduled, manual, and webhook-triggered runs separated?

Teams can distinguish an intentional retry from a scheduled pipeline, an incoming provider event, or a setup-resource refresh before using the result in client-facing reporting.

Review the supporting path

Sync answer

What happens when a sync fails or gets skipped?

A quiet chart does not have to be interpreted as quiet performance. The product can show whether the source itself was late, blocked, skipped, or broken.

Review the supporting path

Run workflow

Every source run should say why it happened.

Sync observability connects selected resources, run reason, supported normalization, run history, and downstream freshness controls before a report treats the source as current.

Start with selected resources Apply the right run reason Fetch and normalize supported records Update sync state and run history Protect downstream output

Run types

Manual, scheduled, webhook, and heartbeat work need separate evidence.

The page makes the operational reason visible so teams can tell whether the source was refreshed by setup, schedule, event delivery, retry, or system processing.

Run type	Trigger	Reviewer sees
First sync with backfill	A newly connected source runs with default backfill settings before routine cadence is trusted for current reporting.	Backfill start settings, selected resources, imported raw payloads, normalized events, inserted rows, updated rows, last success, and next scheduled sync stay tied to the connection.
Manual sync	An operator asks a connection to run now after setup, resource selection, a provider outage, or a disputed reporting review.	The run reason, status, counts, duration, completion time, error, and next sync timing stay visible instead of being blended into the scheduled cadence.
Scheduled due sync	A connection reaches its nextSyncAt time and the due-sync route processes eligible sources for the active tenant scope.	Freshness moves forward only when the run succeeds, while failures preserve last failure time, failure count, and next retry timing for health review.
Webhook-triggered run	A verified provider event arrives through a scoped callback and creates a sync-run record for the ingestion path.	Incoming activity can be reviewed by verifier result, resource match, raw payload creation, normalized events, inserted or updated rows, and sync status.
Resource listing	A setup flow refreshes accounts, forms, pipelines, companies, or other provider resources before a connection can be treated as ready.	Resource-listing attempts and errors can be separated from record ingestion so setup gaps do not masquerade as failed performance data.
System heartbeat	The heartbeat route processes due connection work across tenant scopes when the configured secret is present.	Due sync work runs only for eligible tenant scopes, while skipped subscriptions and missing heartbeat configuration are kept out of normal reporting freshness.

Backfill controls

Historical imports need source-specific boundaries.

Backfills are only useful when the import window, provider filters, selected resources, sync state, and output eligibility are visible beside the first report.

Control	Why it matters	SIRJ path
Connector default backfill window	A first dashboard can look incomplete if the initial import only covers the current day when the client expects historical spend, lead, invoice, or payment context.	Connector definitions and integration API setup carry default backfill settings into provider fetch parameters before routine sync cadence takes over.
Provider cursor and date filters	Ad, CRM, form, payment, and operations APIs often use different cursor names or date filters, so backfill behavior needs source-specific handling.	Provider fetchers read cursor, updated-after, start-date, and backfill-start settings before mapping raw provider rows into normalized events.
Selected-resource boundary	A historical import is dangerous when it pulls the wrong ad account, form, company, location, pipeline, or payment account into a client workspace.	Resource selection and setup checks run before backfilled records can be treated as usable reporting evidence for the client workspace.
Sync state update	Backfill output should not reset freshness as healthy unless the run actually succeeded and produced a defensible state for the connection.	The sync state stores last success, last failure, failure count, retry timing, and next scheduled sync after orchestration decides whether the run succeeded or failed.
Output eligibility	A partially backfilled source can produce attractive charts while still missing the evidence needed for attribution, weekly summaries, or report exports.	Data validation, metric definitions, connector health, and report surfaces keep stale, partial, or unsupported source states visible before output is treated as client-ready.

Failure states

A failed or stale source should not look like normal performance.

Setup requirements, stale data, provider failures, skipped runs, and partial backfills need explicit states before a client-facing surface turns them into confident output.

State	Reviewer sees	Protected surface
Setup required	The source is not treated as ready when credentials, selected resources, webhook activation, pipeline mapping, or required setup checks are still missing.	Dashboards, reports, weekly insights, Ask Data answers, and exports can avoid presenting a connected-but-unusable source as complete proof.
Stale source	The runtime compares last successful sync with connector cadence and current time before marking the connection stale.	Client-facing metrics can show freshness risk instead of turning old provider rows into current performance claims.
Provider failure	A failed run keeps escaped provider error text, duration, status, reason, counts, completion time, failure count, and retry schedule.	Operators can fix the source problem before a client review treats missing leads, revenue, or spend as real business performance.
Skipped due sync	A scheduled pass can be skipped when tenant scope, subscription eligibility, setup, connection state, or timing does not allow safe processing.	The system avoids turning an ineligible or unsafe run into a false success state that would make the source appear fresher than it is.
Partial backfill	Imported rows can be reviewed alongside raw payload count, normalized event count, inserted and updated rows, selected resources, and unsupported payload behavior.	Reports and exports can stay honest about the evidence window instead of making a first import look like a complete historical record.

Workflow

The sync path should be plain enough to audit.

Workflow

Start with selected resources

A sync should begin from a connection that already belongs to the right account and client workspace, with the provider resource selected before records are fetched or accepted.

Setup and permissions pages show the selected account, form, pipeline, company, or payment source that controls what a sync can safely import.

Inspect this path

Workflow

Apply the right run reason

Manual, scheduled, webhook, resource-listing, setup-required, and backfill runs keep separate reasons so operators can understand why a source changed or did not change.

Run history can show the difference between a user-triggered refresh, a scheduled due sync, and an event delivered through a webhook callback.

Inspect this path

Workflow

Fetch and normalize supported records

Provider fetchers and webhook handlers create raw payload evidence, reject unsupported payload types, normalize supported records, and update event rows under tenant scope.

Data routing shows where raw provider activity becomes normalized events before cards, answers, reports, or exports use the records.

Inspect this path

Workflow

Update sync state and run history

The orchestration path records success or failure, last success, last failure, failure count, next sync timing, retry delay, counts, duration, and any escaped error.

Connector health can show fresh, stale, failing, disconnected, setup-required, or reconnect-needed states without hiding the operational evidence.

Inspect this path

Workflow

Protect downstream output

Dashboards, Ask Data, weekly insights, reports, and exports should inherit freshness and evidence boundaries from the sync state underneath them.

Validation and report pages keep stale, partial, unsupported, or setup-blocked data from becoming confident client-ready claims.

Inspect this path

Standards

Keep source freshness explicit, scoped, and hard to misread.

Standard

Do not make first import look complete by default

A first sync can create a polished dashboard before the historical window, selected resources, unsupported fields, or provider limits have been reviewed.

Default backfill settings, provider date filters, source coverage, and output eligibility keep the first import understandable before it is used in a client report.

Standard

Do not merge run reasons

Manual retries, scheduled syncs, webhook events, and setup-resource refreshes answer different operational questions and can mislead reviewers if blended together.

Sync run records keep reason, status, counts, duration, completion time, error, and connection scope visible for each run type.

Standard

Do not treat stale data as normal silence

A late sync can look like weak lead volume, missing revenue, no booked work, or a quiet client period when the real issue is operational freshness.

Connector health compares last success with expected cadence and escalates stale, failing, setup-required, disconnected, or reconnect-needed states before reports rely on the source.

Standard

Do not let system jobs cross tenant scope

A heartbeat or due-sync processor that ignores account and client boundaries could mix source evidence or process subscriptions that should not run.

System heartbeat and due-sync processing build tenant scopes explicitly, require the configured secret, and skip unusable subscriptions before running connection work.

FAQ

Sync observability and backfill questions.

What is a sync run in SIRJ?

Does SIRJ backfill historical source data?

How does SIRJ decide a source is stale?

Why does heartbeat processing matter?

Freshness review

Before a report trusts a source, confirm the run history behind it.

Review connector health, then follow the data path through selected resources, backfills, sync reasons, run status, stale-source controls, and supported reporting surfaces.

Review connector health Follow data routing

Syncs, retries, and backfills should be visible before reports trust the data.

Explain source freshness before the numbers are challenged.

What does sync observability include?

How does SIRJ handle first syncs and backfills?

How are scheduled, manual, and webhook-triggered runs separated?

What happens when a sync fails or gets skipped?

Every source run should say why it happened.

Manual, scheduled, webhook, and heartbeat work need separate evidence.

Historical imports need source-specific boundaries.

A failed or stale source should not look like normal performance.

The sync path should be plain enough to audit.

Start with selected resources

Apply the right run reason

Fetch and normalize supported records

Update sync state and run history

Protect downstream output

Keep source freshness explicit, scoped, and hard to misread.

Do not make first import look complete by default

Do not merge run reasons

Do not treat stale data as normal silence

Do not let system jobs cross tenant scope

Sync observability and backfill questions.

What is a sync run in SIRJ?

Does SIRJ backfill historical source data?

How does SIRJ decide a source is stale?

Why does heartbeat processing matter?

Before a report trusts a source, confirm the run history behind it.

Preparing the source route.

Syncs, retries, and backfills should be visible before reports trust the data.

Explain source freshness before the numbers are challenged.

What does sync observability include?

How does SIRJ handle first syncs and backfills?

How are scheduled, manual, and webhook-triggered runs separated?

What happens when a sync fails or gets skipped?

Every source run should say why it happened.

Manual, scheduled, webhook, and heartbeat work need separate evidence.

Historical imports need source-specific boundaries.

A failed or stale source should not look like normal performance.

The sync path should be plain enough to audit.

Start with selected resources

Apply the right run reason

Fetch and normalize supported records

Update sync state and run history

Protect downstream output

Keep source freshness explicit, scoped, and hard to misread.

Do not make first import look complete by default

Do not merge run reasons

Do not treat stale data as normal silence

Do not let system jobs cross tenant scope

Sync observability and backfill questions.

What is a sync run in SIRJ?

Does SIRJ backfill historical source data?

How does SIRJ decide a source is stale?

Why does heartbeat processing matter?

Before a report trusts a source, confirm the run history behind it.