Customer profiles

Customer profiles are an opt-in, Topline-hosted projection of a customer's PestRoutes account. A missing customer_profile_products row and a row with status = 'disabled' mean the same thing: no profile reads, backfills, incremental work, schedules, or model calls are started.

Product modes

• on_demand activates the product without starting work. The first refresh/backfill for a customer creates its baseline.
• portfolio queues an initial portfolio backfill and provisions two tenant-bound schedules: a five-minute capture poll and a separate synthesis schedule.
• Capture uses the server-owned */5 * * * * cron. Daily synthesis uses 2 2 * * *; priority synthesis uses 2,7,12,17,22,27,32,37,42,47,52,57 * * * *. The two-minute offset gives capture the first chance to commit and the active-run guard prevents overlapping work.
• Activation accepts synthesisCadence = daily | five_minutes and an IANA timezone. Arbitrary cron expressions are rejected because capture fidelity and cost are product policy, not per-tenant scheduler input.
• Pause disables both schedules before changing product state. Resume enables both and queues an immediate incremental synthesis. History and immutable profile versions remain available while paused.
• deterministic_only is the default generation mode. vertex_flex is available only when workload identity, token rates, a completed and approved evaluation for the current profile schema, and a per-generation provider-cost cap are all present. Every response must report ON_DEMAND_FLEX; an ordinary-priority response is rejected and there is no standard-priority fallback.

Customer Profiles is never enabled by default. An authorized Topline operator must select one PestRoutes source, configure the effective customer price or non-billable pilot state, acknowledge the estimate, and explicitly enable the product. Enabling portfolio mode starts the initial backfill; a price edit by itself starts no work.

Capture and synthesis flow

Capture and synthesis are intentionally separate:

1. The existing PestRoutes connector mirrors source rows into the tenant's reporting database. Customer Profiles does not bypass that connector or make a second direct vendor API claim.
2. Every five minutes, the capture run compares supported mirrored rows with the latest committed customer-history state. Changed customer IDs are discovered without rereading every customer.
3. For each changed customer, capture appends idempotent created, updated, hard_deleted, or reactivated history events. Capture does not rebuild the profile.
4. Synthesis selects only customers whose newest history event is beyond the profile's currentEventId. It reads the customer's current model-safe projection inputs and advances the profile cursor after a successful projection.
5. API, chat, and MCP consumers reuse the stored profile plus a bounded activity window. They do not reprocess all PestRoutes rows for each generation.

The five-minute value is Topline observation fidelity, not database CDC and not a guarantee that every intermediate vendor state is preserved. If a row changes twice between mirror refreshes or between capture polls, only the state Topline actually observes can be recorded. Connector lag therefore remains visible operational evidence, and lastSuccessfulCaptureAt is a poll watermark rather than proof that PestRoutes itself was current.

The initial backfill establishes the currently observed baseline. Capture and synthesis share a tenant-level active-run guard, so a long backfill may defer scheduled capture; a later capture catches the latest observed state but cannot reconstruct unobserved intermediate states. Activation and run monitoring must surface this limitation rather than claiming complete pre-activation or vendor-level history.

Admin commercial activation

The target operator surface is a Customer Profiles card on each customer's Topline Admin detail page. The card may present one concise form, but entitlement, processing policy, and pricing are independent controls:

1. Product entitlement: disabled, on-demand, or portfolio. Saving pricing must not start processing. Enabling processing must be a separate confirmed action.
2. Generation policy: deterministic-only or Vertex Flex. Standard-priority fallback is a separate opt-in and cannot be inferred from Flex selection.
3. Customer price policy: one effective-dated price per active PestRoutes customer per month. It records currency, unit price, pricing version, effective date, and approving operator. V1 has no tiers, included allowance, minimum, overage, or customer-facing model-usage markup; immutable customer-day snapshots provide calendar-day proration.
4. Operational state: source readiness, worker readiness, initial estimate, backfill state, last successful sync, last error, and billing reconciliation state.

The recommended Admin presentation is:

• A top-level Product mode control with Off, On demand, and Portfolio choices.
• A Customer price summary beside it, shown as $X per active customer / month, with an Edit pricing action for the versioned rate details.
• A Profile refresh selector with Daily and a separately priced Every five minutes option. Both portfolio choices capture observed changes every five minutes; the selector controls how quickly those events are synthesized into the reusable profile.
• A Generation selector showing Deterministic or Vertex Flex, plus a separately confirmed standard fallback option when supported.
• A preview panel showing estimated active-customer count, unit price, estimated monthly customer charge, current pricing version, internal estimated/unknown costs, schedule, and initial-backfill impact.
• A final Enable and start backfill confirmation. Price edits alone never invoke this action.

An Admin enable action must fail closed unless exactly one active PestRoutes source is selected, the customer price policy is approved and effective (or the customer is explicitly recorded as a non-billable pilot), the requested provider route is ready, required queues/schedules are healthy, and the estimate has been acknowledged. Price changes create a new effective-dated version; they do not retroactively reprice earlier usage. Pause stops new profile work without deleting history or changing historical billing records.

The initial commercial model applies to portfolio mode and deliberately has no allowance, limit, tier, minimum, or overage. On the first healthy five-minute capture tick of each UTC day, Topline records one immutable snapshot of distinct PestRoutes customers where status = 1 and _source_hard_deleted_at is null; later ticks are no-ops. That hosted maintenance step also applies the configured raw-history retention policy. The snapshot retains source/profile watermarks, source-health check time, unit price, pricing version, and actor/source evidence. Month close requires every calendar day; a missing or stale daily snapshot keeps billing pending instead of inventing zero usage. The exact charge is the sum of each day's active_customer_count × unit_price / calendar_days_in_month, rounded once at month close. A mid-month effective price change therefore prorates by day without rewriting earlier evidence. on_demand remains a pilot/operational mode until it receives a separate commercial policy.

This add-on sits beside Topline's flat platform fee, AWS pass-through, and future Chat Member/Build seat tiers. AWS allocated into internal feature cost must not be billed twice, and the flat fee must not silently include Customer Profiles unless an effective-dated tier says so. When a tier does include the add-on, the invoice snapshot records that inclusion and suppresses the separate profile line.

Enabling Vertex Flex

Vertex Flex cannot be enabled merely by selecting it in Admin. It becomes selectable only after all of the following are present and verified:

1. Application Default Credentials resolve through the deployed workload-identity path; no GCP service-account key is accepted or stored.
2. CUSTOMER_PROFILE_VERTEX_PROJECT_ID, CUSTOMER_PROFILE_VERTEX_LOCATION=global, and the approved input/output token rates are configured. Flex currently requires Google's global endpoint. The hosted worker sends both the shared and flex request headers plus a ten-minute server/client timeout, and performs at most three exponentially delayed attempts for the documented transient 408/429/5xx capacity responses.
3. The customer pricing policy has an effective per-active-customer monthly price and pricing version.
4. Internal model usage records tenant, profile run/customer, provider, model, service tier, routing type, tokens, provider cost, pricing version, and reconciliation status. These costs inform Topline margin but do not create a separate customer overage in V1.
5. A bounded evaluation records quality score, sample size, completion rate, maximum provider cost, evaluator, and evidence. An operator must explicitly approve that evaluation, and the product cost cap must be at least its observed maximum.
6. The Admin preview and confirmation flow is available, and separate deployment/customer-rollout authorization has been granted.

Until those gates pass, the API fails closed with the missing configuration or evaluation gate. At runtime, a response not attributed by Vertex to Flex traffic is rejected. It must never silently fall back to standard-priority inference.

History contract

The durable history is baseline plus observed changes, not a claim that PestRoutes exposes every historical version that existed before activation.

1. Initial backfill writes one baseline event for every currently observed source row.
2. Later reads compare canonical row hashes and append created, updated, hard_deleted, or reactivated events.
3. The event identity includes observed_at, so repeated state cycles such as A → B → A are retained while a retry of the same observation remains idempotent.
4. Every supported materialized PestRoutes table receives _source_row_hash; this supplies change detection when dateUpdated is absent or unreliable.
5. If a mirrored record moves from one customer relationship to another, capture writes a Topline-observed hard_deleted association event for the former customer and a created event for the new customer. This prevents the old profile from silently retaining the moved object.
6. Full source payloads remain in customer_history_events. Profile sections contain only allowlisted model-safe fields. Full event payload reads require the separately granted customer_profiles.history.payload.read capability.

Portfolio capture discovers changed customer IDs in Postgres by comparing each mirrored row with the latest committed history event. Only those customer IDs are dispatched. Synthesis then selects customers with captured events beyond their profile cursor. Each run records afterEventId and throughEventId; a truly unchanged capture is a guarded no-op with no new history event or profile version.

The current official customerFlag contract has no dateUpdated and no stable flag-row identifier. Topline does not fabricate either field. Incremental refresh performs a full customer-ID scan, aggregates all returned flag code/value pairs into one deterministic row keyed by customerID, and relies on the canonical payload hash plus _source_fetched_at to capture observed changes. Malformed rows and conflicting duplicate flag values now fail the sync instead of being silently discarded or overwritten.

Offline contract, materialization, duplicate, removal, and profile-history tests pass. A bounded authenticated production response remains a rollout gate until a Topline operator uses Admin's Verify customerFlag live response action or calls POST /api/internal/customer-profile-product/customer-flags/certification with exact sourceId, matching confirmSourceId, and maxCustomers from 1 through 25. The internal endpoint enqueues the manual certify-pestroutes-customer-flags task. The task revokes any prior proof before performing one complete live ID search and two live bounded hydrations of the same customer sample; fixture, cache, stale, failed, and unstable responses cannot leave an older proof active. It records only counts and SHA-256 digests in source_data_resources.metadata and never returns flag values or customer IDs. Product status remains official_contract_verified_live_response_pending until current-contract evidence is stored, then reports official_contract_and_live_response_verified. Evidence expires after 90 days or immediately when the checked-in contract digest changes. A later or concurrent materialization preserves the certification evidence.

generic_flag_assignments is included for customer (CUST), subscription (SUBS), and appointment (APPT) relationships and is joined to generic_flags. Both tables expose dateUpdated. The catalog's EQAS relationship has no verified customer join, so source.generic_flag_assignments_eqas is explicitly unavailable rather than guessed. appointment_reminders uses stable reminderID identity and its documented dateUpdated, joins through appointments, and retains cancellation, deletion/recreation, non-monotonic edits, and moves between customer parents as separate observed events. Raw destinations, reminder text, and response text are omitted from profile sections, while channel, status, timing, and response-presence facts remain available.

Profile contract

Each profile is a versioned JSON document with appendable sections:

{
  "schemaVersion": 1,
  "identity": {
    "tenantId": "tenant-id",
    "sourceId": "source-uuid",
    "externalCustomerId": "1001"
  },
  "status": "ready",
  "historyCoverage": {
    "mode": "baseline_plus_observed_changes",
    "historyStartedAt": "2026-08-29T10:00:00.000Z",
    "completeObservedChangesSince": "2026-08-29T10:00:00.000Z"
  },
  "coverage": {
    "included": ["customers", "appointments", "customer_flags"],
    "unavailable": []
  },
  "sections": {
    "source.customers": {},
    "source.appointments": {},
    "synthesis.communication_history": {},
    "synthesis.flags": {},
    "synthesis.summary": {}
  },
  "asOf": "2026-08-29T12:00:00.000Z",
  "capturedAt": "2026-08-29T11:59:00.000Z",
  "generatedAt": "2026-08-29T12:00:00.000Z"
}

Source sections use the stable source.<table> namespace. New source or synthesis sections can be appended without changing existing keys. Each section stores its own schema version, source-table provenance, input digest, evidence, source maximum dateUpdated, generated time, and event cursor. A profile version advances only when model-safe section content changes; routine source fetch timestamps do not invalidate it. Deterministic facts remain authoritative; an approved Flex narrative is additive and independently attributable. The complete immutable snapshots live in customer_profile_versions, while customer_profile_sections supports targeted regeneration, rebuild, and reads.

The reusable profile never embeds an ever-growing raw-data appendix. GET .../context and MCP compose the canonical versioned profile with a separate, bounded recent-activity window (24 hours by default). This preserves prompt stability while still giving generations exact recent evidence. Durable observed history remains paginated in customer_history_events and can rebuild every profile section.

Customer API

All routes require a tenant-bound browser session and the named capability. Work-triggering actions and pricing writes require an Idempotency-Key header. Daily snapshots and month close are naturally idempotent by tenant/date or tenant/month; reconciliation accepts an exact retry and rejects conflicting evidence.

Method	Path	Capability	Purpose
GET	/api/customer-profile-product	customer_profiles.read	Read disabled/active product state and schedule policy.
GET	/api/customer-profile-product/status	customer_profiles.read	Read source/profile freshness, pricing, latest runs, generation gate, and billing state.
POST	/api/customer-profile-product/pricing	customer_profiles.manage	Create an effective-dated pilot or billable per-active-customer-month price. Does not start work.
POST	/api/customer-profile-product/generation-evaluations	customer_profiles.manage	Record a completed Vertex Flex evaluation draft.
POST	/api/customer-profile-product/generation-evaluations/{id}/approve	customer_profiles.manage	Approve a complete evaluation before Flex enablement.
POST	/api/customer-profile-product/billing/daily-snapshot	customer_profiles.manage	Write or idempotently verify today's immutable active-customer snapshot; arbitrary historical dates are rejected.
POST	/api/customer-profile-product/billing/month-close	customer_profiles.manage	Close a complete month from daily snapshots. Missing days fail closed.
POST	/api/customer-profile-product/billing/reconcile	customer_profiles.manage	Attach provider/infrastructure cost evidence without changing customer charge or AWS pass-through.
GET	/api/customer-profile-product/economics	feature_economics.view	Read current-month or bounded-range tenant totals and per-customer usage/cost coverage. This internal Topline optimization view is not granted to customer admins by default and is not an invoice.
POST	/api/customer-profile-product/estimate	customer_profiles.manage	Inspect customer count and pricing availability before activation; unknown storage/compute and customer charges remain null.
POST	/api/customer-profile-product/actions/enable	customer_profiles.manage	Explicitly enable on-demand or portfolio mode.
POST	/api/customer-profile-product/actions/pause	customer_profiles.manage	Pause the product and recurring schedule.
POST	/api/customer-profile-product/actions/resume	customer_profiles.manage	Resume and trigger a portfolio incremental run.
POST	/api/customer-profile-backfills	customer_profiles.manage	Start one customer baseline or a portfolio backfill.
GET	/api/customer-profile-backfills/{runId}	customer_profiles.read	Read backfill/run state and bounded counters/cost fields.
POST	/api/customer-profile-runs/{runId}/actions/cancel	customer_profiles.manage	Request cooperative cancellation; cursor and heartbeat remain available for a recovery run.
POST	/api/customer-profiles/{customerId}/refresh	customer_profiles.refresh	Start initial baseline or incremental refresh for one customer.
GET	/api/customer-profiles/{customerId}	customer_profiles.read	Read the current profile with ETag/304 support.
GET	/api/customer-profiles/{customerId}/context	customer_profiles.read	Read the reusable profile, freshness watermarks, and bounded model-safe recent activity. Accepts recentHours (1–168, default 24) and limit (1–100, default 50), with ETag/304 support.
GET	/api/customer-profiles/{customerId}/activity	customer_profiles.history.read	Read only the bounded model-safe activity window with the same recentHours and limit bounds.
GET	/api/customer-profiles/{customerId}/sections	customer_profiles.read	List current sections.
GET	/api/customer-profiles/{customerId}/sections/{sectionKey}	customer_profiles.read	Read one section.
GET	/api/customer-profiles/{customerId}/versions	customer_profiles.read	List immutable profile versions.
GET	/api/customer-profiles/{customerId}/history	customer_profiles.history.read	Read a stable, signed-cursor history page.

History payloads are excluded by default. includePayload=true additionally requires customer_profiles.history.payload.read. History pagination fixes a through event ID in the signed cursor so concurrent syncs cannot reorder a traversal.

The context and activity endpoints are optimized for generation, not bulk export. Their activity window ends at the latest captured event so an unchanged response and ETag remain stable as wall-clock time advances. Results are newest-first, limited to seven days and 100 events, and each event contains allowlisted facts capped at 4 KiB. Sensitive reminder destination, text, and response fields are not returned. Use the paginated history endpoint for a complete Topline-observed traversal; use its payload option only for explicitly authorized investigations.

An enable request has this policy shape:

{
  "sourceId": "pestroutes-source-uuid",
  "mode": "portfolio",
  "generationMode": "deterministic_only",
  "synthesisCadence": "daily",
  "syncTimezone": "America/Denver",
  "sourceFreshnessEnforced": true,
  "sourceFreshnessMaxMinutes": 15,
  "recentActivityHours": 24,
  "rawHistoryRetentionDays": 400,
  "maxConcurrentCustomers": 4
}

syncCron is accepted only when it exactly matches the server-derived cadence and should normally be omitted. This keeps the Admin toggle and hosted API on the same supported policy.

Workspace MCP

An enabled tenant can grant the Workspace MCP scope customer_profiles:read. Tool discovery then exposes:

• get_customer_profile: requires the operator capability customer_profiles.read and returns the reusable profile, explicit freshness watermarks, and bounded recent activity.
• get_customer_activity: requires customer_profiles.history.read and returns the bounded model-safe activity window without raw source payloads.
• get_customer_profile_status: requires customer_profiles.read and returns read-only product, source-health, pricing, run, and generation-gate status. It never enables, pauses, backfills, or changes pricing.

The OAuth scope never creates entitlement and never widens the signed-in operator's tenant capabilities. If the product is disabled, the customer is unknown, the scope is absent, or the operator lacks the capability, the tool fails closed. Full event payloads are not exposed through these MCP tools.

Storage and cost accounting

• customer_profile_products owns tenant activation, source binding, mode, both schedule links, synthesis cadence, capture/synthesis watermarks, and generation policy.
• customer_profile_runs is the auditable job ledger. Capture, synthesis, and manual/backfill work record separate delta rows, projection rows, customers discovered, and customers processed. Provider cost and customer charge are distinct nullable fields; unknown values are not represented as zero.
• feature_economics_events records capture/synthesis discovery, attempted customer work, guarded no-ops, duration, rows/events/sections, model activity, and nullable cost evidence. Empty polls are measurable without creating per-customer metric dimensions. Portfolio work is attributable both to the Topline client and, when customer work occurs, to the source customer that caused it.
• customer_history_events is append-only.
• customer_profiles, customer_profile_sections, and customer_profile_versions are reusable projections for future generations.
• customer_signals.inferences contains only a small compatibility facade pointing to the latest profile; Topline Postgres remains the source of truth.
• Fetching a selected customer context resolves that facade to the ready stored profile (bounded to 32,000 characters), so future generations reuse the projection instead of re-reading PestRoutes rows.

Fleet metrics use Environment, Feature, Operation, and Outcome only. Tenant and customer IDs stay outside CloudWatch dimensions and are queried from the durable ledger. Infrastructure and total Topline costs initially remain unpriced until an approved allocation/reconciliation process supplies them; deterministic model cost can independently be known as zero. See Feature economics for the shared contract intended for Customer Profiles, main chat, and connectors.

Customer-profile operations additionally emit low-cardinality Topline/CustomerProfiles metrics for source freshness/unavailability, run outcome and p99 duration, cancellation, completed backfills, provider cost/cap violations, and guarded stuck-run heartbeats. Terraform defines a dedicated dashboard and SNS-backed alarms; source integration alone does not apply them.

Retention, deletion, and recovery

• Raw observed history defaults to 400 days. Retention purges only older superseded events and always preserves the latest event for every tenant/source/customer/resource/record identity so incremental comparison remains correct.
• settings.legalHold = true blocks retention. Payload access remains separately capability-gated and all profile/history queries remain tenant-bound.
• Deleting a profile cascades its sections and immutable versions but does not silently delete the source mirror or history ledger. A privacy deletion workflow must identify the tenant, source customer, billing evidence retention requirement, and legal-hold state before removing history.
• A failed or cancelled portfolio run retains its last customer cursor, event cursor, heartbeat, counters, and sanitized failure. A new idempotent run resumes from explicit evidence; an operator must not rewrite the old run to look successful.
• If the source mirror is stale or missing, capture, synthesis, activation, and daily billing snapshots fail closed. Existing ready profiles remain identifiable with separate sourceDataThrough, sourceFreshnessCheckedAt, profileCapturedAt, and profileGeneratedAt timestamps.

Hosted Admin and rollout boundary

Topline Admin exposes the card under the customer's Tools page. The shortest supported flow is: verify source health and active count, save a pilot or billable price version, review the estimated monthly add-on and cadence, acknowledge the initial backfill, then explicitly enable. Pricing edits never start work. Pause/resume are reversible; starting another backfill requires a separate confirmation. Admin calls a bearer-protected internal customer-runtime management API and records the operator action without exposing that internal bearer to the browser.

Migration 0189_customer_profile_productization.sql adds the pricing, daily snapshot, monthly statement, generation-evaluation, freshness, heartbeat/cancellation, concurrency, cost-cap, and captured/generated-time contracts. Applying that migration, deploying API/worker/Admin changes, enabling any tenant, approving pricing, and running a customer backfill are separate authorized operations.

This guide and code availability do not activate the product. Each tenant remains disabled until an authorized product action succeeds, and deployment, commercial availability, tenant enablement, and customer rollout are separately controlled operations.