Skip to content

Marketing Broadcast Feature

Overview

The Marketing module supports WhatsApp and Email broadcast capabilities, allowing marketing team members to send targeted messages to customer segments. Broadcasts are integrated into the existing campaign/activity workflow and require approval before sending.

WhatsApp broadcasts support two providers: Twilio (legacy, via Content API) and Meta (direct Cloud API via Python service). The provider is configurable per-broadcast or globally.

Activity Types

  • WHATSAPP_BROADCAST — Send WhatsApp messages via Twilio Content API or Meta Cloud API
  • EMAIL_BROADCAST — Send HTML emails via SMTP with configurable templates

Broadcast Lifecycle

Activity created (type = WHATSAPP_BROADCAST or EMAIL_BROADCAST)
    ↓ (auto-creates broadcast record)
DRAFT → configure template, message body, media
    ↓ (add one or more Recipient Lists — see recipient-lists.md)
READY → recipients populated (append + dedupe), opt-outs flagged
    ↓ (activity must be APPROVED first, then trigger send)
SENDING → async send in progress with rate limiting
    ↓
SENT / FAILED → delivery complete, stats updated via callbacks

Recipient lists are now a first-class entity managed outside the activity (see Recipient Lists). The legacy per-activity CRM-resolve path (broadcast/build-recipients) has been removed; activities populate recipients by copying from one or more recipient lists at compose time.

Provider Branching (WhatsApp)

When sendBroadcast() is triggered, the BroadcastDispatcher abstraction routes to the correct provider:

MarketingFacade.sendBroadcast()
    ↓
BroadcastDispatcherFactory.get(provider)
    ├── "TWILIO" → TwilioBroadcastDispatcher → BroadcastService (Java, existing)
    └── "META"   → MetaBroadcastDispatcher   → Python /api/broadcast/dispatch

When a WhatsApp broadcast is created, the provider is set from the broadcast.wa.provider config property (default TWILIO). At send time, BroadcastDispatcherFactory reads the stored provider field to select the correct dispatcher.

WhatsApp Provider Selection

Twilio (legacy)

  • Sends via Twilio Content API with Content SID templates
  • Status callbacks via Twilio webhooks (broadcast/webhook/twilio-status)
  • Rate limiting handled in Java BroadcastService (configurable delay)

Meta (direct Cloud API)

  • Sends via Python FastAPI service using Meta Marketing Messages API
  • Java dispatches to Python via HTTP POST; Python handles the send loop
  • Status tracking via Meta webhooks → Python → TQPro internal API
  • Template rendering uses component_spec from tqwa.whatsapp_templates table
  • Supports 131049 retry scheduling (per-user marketing throttle)

See doc/features/marketing/whatsapp-meta-integration.md for full Meta integration details.

Database Schema

Migrations

Migration Description
0058-marketing-broadcast.sql Core broadcast tables
0059-broadcast-media-url.sql Media URL support
0060-broadcast-card-template.sql Card template type
0061-broadcast-templatetype-text-to-media.sql Rename TEXT → MEDIA
0062-broadcast-meta-support.sql Widen messagesid to VARCHAR(200), add provider column
0063-whatsapp-meta-tables.sql Create tqwa schema for Python service
0064-ai-usage-log-csr-support.sql AI usage log shared with CSR
0066-message-template.sql Unified mkt_message_template table, templateid FK on broadcast
0067-backfill-broadcast-templates.sql Create templates from existing broadcasts, link by CSID/name

Tables (Java-owned, nts schema)

  • nts.mkt_message_template — Unified template entity for Twilio and Meta. Stores provider-specific fields (Content SID, Meta name, component_spec), variable mapping, button config, and language code. Referenced by broadcasts via FK.
  • nts.mkt_broadcast — One record per broadcast activity. Stores channel, templateId FK, message body, media, delivery stats, status, and provider (TWILIO/META). Also retains inline template fields for backward compatibility.
  • nts.mkt_broadcast_recipient — One record per resolved recipient. Tracks customer ID, contact info, message SID (Twilio SMxxx or Meta wamid.xxx), delivery status, and error details.
  • nts.mkt_broadcast_optout — Opt-out register keyed by (channel, contactvalue).
  • nts.mkt_recipient_list / nts.mkt_recipient_list_member — System-wide recipient lists and their members (TQ-113, migration 0071). See Recipient Lists.

The legacy nts.mkt_audience and nts.mkt_activityaudience tables are superseded by recipient lists and no longer referenced; scheduled for drop in a follow-up migration.

Tables (Python-owned, tqwa schema)

  • tqwa.whatsapp_templates — Meta template component_spec for payload construction and language_code (must match the template's language in Meta Business Manager, e.g., en)
  • tqwa.campaign_messages — WAMID-to-broadcast tracking for webhook correlation
  • tqwa.conversation_state — AI conversation history, lead qualification, routing mode
  • tqwa.processed_webhook_messages — Webhook idempotency (deduplicate Meta retries)
  • tqwa.wa_media_events — Media message metadata log (never file content)

Architecture

Backend

  • CBroadcast / CBroadcastRecipient / CBroadcastOptout — Canonical entities in com.perun.tlinq.entity.marketing
  • MarketingFacade — Broadcast CRUD, recipient building, send triggering, opt-out management
  • BroadcastDispatcher — Interface for provider-specific dispatch
  • BroadcastDispatcherFactory — Resolves dispatcher by provider name (TWILIO, META)
  • TwilioBroadcastDispatcher — Delegates to BroadcastService (existing Java path)
  • MetaBroadcastDispatcher — HTTP POST to Python service
  • BroadcastService — Singleton async orchestrator for Twilio sends

Send Flow (Twilio)

  1. MarketingFacade.sendBroadcast() validates activity is APPROVED and broadcast is READY
  2. BroadcastDispatcherFactory resolves to TwilioBroadcastDispatcher
  3. Delegates to BroadcastService.executeBroadcast() which runs async
  4. Iterates PENDING recipients with rate limiting (default 75ms)
  5. Routes to MEDIA or CARD send path, calls MessageUtil.sendWhatsappMessage() via Twilio
  6. Updates recipient status and broadcast aggregate stats

Send Flow (Meta)

  1. MarketingFacade.sendBroadcast() validates as above
  2. BroadcastDispatcherFactory resolves to MetaBroadcastDispatcher
  3. HTTP POST to Python /api/broadcast/dispatch (fire-and-forget)
  4. Python reads dispatch data via TQPro /internal/broadcast/dispatch-data
  5. Async send loop: resolves variables, builds Meta components[], sends via Cloud API
  6. Batch status updates via TQPro /internal/recipient/status
  7. Broadcast stats and status updated via TQPro internal API

Message Template Management

Templates are managed as a unified entity (CMessageTemplate / nts.mkt_message_template) supporting both Twilio and Meta providers. The admin UI at template-mgmt.html provides full CRUD management — no more manual CSID entry or SQL inserts.

Template entity fields:

Field Description
name Display name
channel WHATSAPP or EMAIL
provider TWILIO or META
twilioContentSid Twilio Content SID (HXxxx)
metaTemplateName Meta template name (as registered in Business Manager)
languageCode Language code for Meta (e.g., en, en_US, ar)
templateType MEDIA, CARD, or TEXT
variableMapping JSON variable mapping
buttonConfig JSON button configuration
componentSpec JSON Meta component_spec for payload construction

Broadcast → Template link: Broadcasts reference a template by templateId FK. When a template is selected in the broadcast UI, the facade auto-populates the broadcast's inline fields (templateCsid, templateName, templateType, variableMapping, buttonConfig, provider) from the template record. This ensures the Twilio send path (BroadcastService) works unchanged.

Python integration: The dispatch-data internal API response includes the full template entity when templateId is set. The Python sender reads componentSpec and languageCode directly from this response — no separate DB query needed.

WhatsApp Template Types

Two template types are supported:

  • MEDIA (default/legacy): Hardcoded 4-variable mapping ({{1}}=media path, {{2}}=name, {{3}}=body, {{4}}=shortlink). Media is attached via Twilio's setMediaUrl() parameter.
  • CARD (whatsapp/card): Configurable variable mapping stored as JSON. Supports header media + body text + CTA buttons in a single structured message.

Card Template Variable Mapping

Each template stores a variableMapping JSON that maps template variable positions to data sources:

{"1": "media_url", "2": "recipient_name", "3": "message_body", "4": "button_1_url"}

Available source identifiers:

Source Value
media_url Full CDN URL of broadcast media
recipient_name Recipient's display name
message_body Broadcast message body
shortlink WhatsApp shortlink from config
button_N_url URL suffix from button config at index N
static:value Literal string

Button Configuration

[{"index": 1, "urlSuffix": "summer2026"}, {"index": 2, "urlSuffix": "contact"}]

Delivery Tracking

  • WhatsApp (Twilio): Status callbacks to broadcast/webhook/twilio-status. Progression: QUEUED → SENT → DELIVERED → READ (or FAILED/UNDELIVERED).
  • WhatsApp (Meta): Meta webhooks → Python service → TQPro /internal/recipient/status. Same status progression. WAMID stored in messagesid column (widened to VARCHAR(200) for Meta format).
  • Email: SMTP send success sets status to SENT.

Meta Error Handling

Code Meaning Action
131049 Per-user marketing throttle Schedule retry (2h → 12h → 24h)
131050 User opted out Add to opt-out register, never retry
131026 Not on WhatsApp / invalid number Mark failed, skip
130429 Throughput exceeded Back off 5s, reduce rate
131042 Payment method issue Alert ops, pause campaign

AI CSR "Leila"

When AI_CSR_ENABLED=true, ALL inbound WhatsApp messages route to the AI agent (AI-first mode). The dedicated WhatsApp number is fully AI-managed — no human operator.

  • Persona: Leila, travel specialist at BookMyHoliday
  • Product Search: Recognizes staycation/cruise requests, fetches live product data from TQPro APIs, sends options as separate messages with approximate pricing
  • Handoff: Two-step flow (collect name + preference), then owner notified via lead_notification template
  • Languages: EN, AR, RU, HI, SR (auto-detected, follows switches, English translations stored)
  • Conversation Thread: All messages stored in tqwa.conversation_messages (encrypted, with WAMID for quote-reply resolution)
  • Returning Customers: Post-handoff messages route back to AI with previous conversation context
  • Offer Context: Broadcast replies include structured package data (hotel, dates, meal plan) in AI system prompt

See doc/features/marketing/whatsapp-meta-integration.md for the full specification.

Dedicated Number (AI-First)

The business WhatsApp number is registered to Meta Cloud API only (no WhatsApp Business App). All conversations are AI-managed. The owner is notified of qualified leads via template messages and follows up by calling back on a personal number.

Opt-Out Handling

WhatsApp (Twilio)

  • Custom STOP handler via broadcast/webhook/twilio-inbound endpoint
  • Parses STOP/UNSUBSCRIBE/OPTOUT keywords from inbound WhatsApp messages

WhatsApp (Meta)

  • STOP keyword detection at Priority 2 in Python inbound handler (before AI routing)
  • Meta 131050 error auto-adds contact to opt-out register
  • Both paths call TQPro /internal/optout/add or MarketingFacade.addOptout()

Email

  • RFC 8058 List-Unsubscribe header in broadcast emails
  • Links to broadcast/optout/unsubscribe GET endpoint

API Endpoints

All under /tlinq-api/marketing/broadcast/. See doc/api/marketing.md for full specification.

Endpoint Auth Description
broadcast/get agent,admin Get broadcast by activityId
broadcast/save agent,admin Update broadcast config
broadcast/compose agent,admin Create/update broadcast and copy recipients from one or more lists (listIds)
broadcast/send admin only Trigger async send
broadcast/recipients agent,admin Paginated recipient list
list/copy-to-broadcast agent,admin Append members from a recipient list to a broadcast (dedupe + opt-out flag). See Recipient Lists.
broadcast/optout/add agent,admin Add opt-out
broadcast/optout/remove admin only Remove opt-out
broadcast/webhook/twilio-status public Twilio delivery status callback
broadcast/webhook/twilio-inbound public Twilio inbound STOP handler
broadcast/optout/unsubscribe public (GET) Email unsubscribe

Removed in TQ-113: broadcast/preview-audience, broadcast/build-recipients, and the marketing/audience/* / marketing/activityaudience/* endpoint groups.

Internal API Endpoints (Python service)

Endpoint Auth Description
marketing/internal/broadcast/dispatch-data internal Broadcast + recipients + optouts for dispatch
marketing/internal/recipient/status internal Batch recipient status updates
marketing/internal/broadcast/stats internal Recalculate broadcast aggregate stats
marketing/internal/broadcast/status internal Update broadcast-level status
marketing/internal/recipient/lookup internal Find recipient by messageSid
marketing/internal/optout/add internal Add opt-out entry
marketing/internal/ai/config internal AI configuration (API key, model)
marketing/internal/ai/usage internal Log AI usage for cost tracking

Configuration

Properties in config/properties.d/messaging.properties:

# Core broadcast settings
broadcast.enabled=true
broadcast.whatsapp.enabled=true
broadcast.email.enabled=true
broadcast.send.delay.ms=75
broadcast.email.send.delay.ms=50
broadcast.phone.prefixes=0,00971,+971
broadcast.phone.default.cc=971
broadcast.max.recipients=10000

# WhatsApp provider: TWILIO (default) or META
broadcast.wa.provider=TWILIO

# Twilio settings
broadcast.wa.template.csid=
broadcast.wa.shortlink=https://wa.me/yourshortlink
broadcast.wa.media.cdn.prefix=https://media.perunapps.com/
broadcast.twilio.status.callback.url=

# Python WhatsApp service (Meta provider)
whatsapp.python.service.url=http://localhost:8001
whatsapp.python.service.api-key=

Email Templates

File-based HTML templates in config/email-templates/marketing/. MailUtil placeholder format: %VARIABLE%.

Available placeholders: %SUBJECT%, %BODY%, %RECIPIENT_NAME%, %UNSUBSCRIBE_LINK%.

Frontend

The broadcast panel is integrated into the Marketing Planning page (mktplan.html). It appears conditionally when viewing a broadcast activity, showing template configuration, recipient management, delivery statistics, and send controls.

Prerequisites

  1. Twilio path: Register a marketing template with Twilio/Meta. Configure broadcast.wa.template.csid.
  2. Meta path: Deploy Python service, register templates in Meta Business Manager, populate tqwa.whatsapp_templates with component_spec.
  3. Public API URL: Twilio callbacks need broadcast.twilio.status.callback.url; Meta webhooks need the Python service's public URL registered in Meta App Dashboard.
  4. SMTP Configuration: Email sending requires valid SMTP config.
  5. WhatsApp Business Verification: Meta-approved marketing templates require a verified WhatsApp Business Account.