Real-time CDR processing at carrier scale: an architecture

Call Detail Records are the bloodstream of a telecom operation: every call, message, and session becomes a record that feeds billing, fraud detection, regulatory reporting, and network analytics. When the pipeline that moves them is slow, lossy, or can't be replayed, everything downstream inherits the problem — you bill wrong, you spot fraud a day late, and you can't answer the regulator. This is the architecture we build when CDR processing has to be real-time, carrier-scale, and provably lossless.

It sits under several of the use cases in our overview of AI in telecom — none of which work without a trustworthy event pipeline first.

The shape: ingest, dedupe, enrich, store

Every stage exists to defend a specific guarantee. Skip one and you trade a guarantee you needed for latency you didn't.

• Ingest — accept SIP/Diameter/mediation events from many sources at peak without dropping any. A durable log (Kafka) is the backbone; producers never block on downstream.
• Dedupe — the same event will arrive more than once. Idempotent keys make double-delivery a no-op instead of a double charge.
• Enrich — join the raw record against subscriber, tariff, roaming, and network-element data so downstream systems get a complete row, not a lookup problem.
• Store — a columnar store (ClickHouse) for sub-second analytical queries, plus a durable archive for replay and audit.

Idempotency is not optional

At carrier scale, exactly-once delivery is a fiction you should not design around. Networks retry, mediation layers replay, consumers restart mid-batch. The realistic contract is at-least-once delivery plus idempotent processing — which means every record carries a stable key, and every write is a no-op if that key has already been seen. Get this wrong and a retry becomes a double-billed call; get it right and duplicates are free.

-- Idempotent ingest key: a duplicate SIP event collapses to one row.
-- ReplacingMergeTree keeps the latest version per key at merge time.
CREATE TABLE cdr_events (
  event_key   String,        -- switch_id + call_id + leg + seq (stable)
  occurred_at DateTime64(3),
  msisdn      String,
  cell_id     String,
  duration_ms UInt32,
  ingested_at DateTime64(3)
) ENGINE = ReplacingMergeTree(ingested_at)
ORDER BY (event_key);

-- Downstream billing reads a deduplicated view, never the raw stream.

From SIP to dashboard in under a second

Sub-second end-to-end means the enrichment joins cannot be per-record database lookups — that is the classic pipeline killer. The reference data (subscribers, tariffs, network elements) is held in memory in the stream processor and refreshed on change, so enrichment is a hash join at wire speed, not a round trip. The columnar store then answers analytical queries — volume, deflection, fraud signals — in tens of milliseconds because it never scans what it doesn't need.

Replay and backfill are features, not recovery tools

The question every operator eventually asks is: 'a tariff was misconfigured for six hours last Tuesday — can you reprocess those calls?' If the answer is no, the pipeline is a liability. Because the durable log retains raw events and every stage is idempotent, replay is a first-class operation: re-run a window through corrected enrichment logic and the deduplicated store converges to the right answer without double-counting. Backfill (loading historical data into a new schema) is the same mechanism pointed at the archive.

Migrating off legacy without a gap

Nobody gets to switch off the old billing pipeline and turn on the new one at midnight. The migration that works is dual-run: the new pipeline consumes the same event stream in parallel, and you reconcile its output against the legacy system record-for-record until the deltas are explainable and zero. Only then does billing cut over. We have replaced four legacy systems this way for a carrier-scale operator — 10M+ records a day, sub-second latency, zero data loss — precisely because the new system proved itself against the old one before anything depended on it.

// Reconciliation harness: the migration's actual acceptance test.
// Run until the delta is zero or fully explained, then cut over.
async function reconcileWindow(from: string, to: string) {
  const legacy = await legacyBilling.aggregate(from, to);
  const next   = await newPipeline.aggregate(from, to);
  const deltas = diffByKey(legacy, next);           // per-tariff, per-hour
  report(deltas);                                    // explain every non-zero
  return deltas.filter((d) => Math.abs(d.amount) > 0);
}

Observability the regulator would accept

For a system that feeds billing and compliance, observability is not dashboards for engineers — it is evidence. You need per-stage record counts that balance (what came in equals what was stored plus what was rejected, with reasons), lag metrics on every consumer, and an audit trail that can answer 'where did this charge come from' for any single record. If you can't trace one CDR end to end, you can't defend the bill.

“At carrier scale you don't design for exactly-once. You design for at-least-once and make duplicates free — then replay stops being an emergency and becomes a button.”

When you need this

Not every operator needs sub-second CDR processing — if billing runs nightly and volumes are modest, a batch pipeline is cheaper and simpler. You need this architecture when real-time fraud detection, live network analytics, or usage-based products depend on the data being current, and when the cost of a lost or double-counted record is measured in revenue and regulatory exposure. That is the line where our telecom solution work starts.