Engineering & Architecture: Build Decisions This Week
2026‑09‑18
The tech landscape in late September is defined by three interlocking themes that every CTO—whether operating a global portfolio or a niche market—must grapple with: agentic software factories at scale, zero‑downtime migrations for critical services, and the need for intelligent observability around LLM‑powered agents. Below, I distill actionable take‑aways, trade‑offs, and concrete build decisions that should surface in your backlog this week.
---
OpenAI’s deep dive into its “agentic software factory” reveals a production model built on an unlimited token budget that lets engineers, researchers, and product teams iterate at unprecedented speed. The key architectural shift is moving from single‑purpose inference to continuous, multi‑tool agent loops where prompts trigger tool calls, stateful reasoning, and nested sub‑agents.
| Benefit | Cost |
|---------|------|
| Rapid prototyping of internal bots (e.g., PR reviewers, support chat) | Persistent compute cost per token; higher latency for large models |
| Centralized logging via the factory’s built‑in tracing | Data privacy concerns—token contents may contain PII or proprietary logic |
| Easier compliance with POPIA Act 4 of 2013 (SA) and UK GDPR: data can be filtered before exposure to external APIs | Requires tight control over outbound traffic; risk of inadvertently leaking sensitive state |
Decision Point:
Do you build your own agentic factory on an open‑source LLM, or do you subscribe to a managed service that exposes the same abstraction? A managed path gives instant scale and lower developer overhead but locks you into vendor pricing and data residency constraints. Building in‑house offers full control of data flow and compliance but demands investment in GPU infrastructure and specialist ML ops talent—a significant CAPEX hit for most SA or UK teams.
---
The ByteByteGo case study on migrating an online shopping platform’s database underscores the reality that high‑traffic services cannot afford outages. Their solution—double‑writing to a new store while gradually shifting traffic—balances performance with risk.
| Strategy | Pros | Cons |
|----------|------|------|
| Synchronous replication + feature flags | Immediate rollback path; minimal data loss | High write amplification; potential lag under peak load |
| Event sourcing & change‑data capture | Decouples read/write concerns; natural audit trail | Requires re‑engineering of domain models; more complex deployment pipeline |
Decision Point:
If your product is a payment or logistics platform in the UK/EU, data residency under GDPR and AI Act (for any downstream ML pipelines) makes event sourcing attractive because it keeps every state change auditable. However, for a simpler SaaS offering with moderate traffic, double‑writing with feature flags provides a quicker win with lower engineering cost.
---
The “How LLMs Can Find a Needle in a Haystack” workshop highlighted Sentry Agent Tracing as a powerful way to capture prompt, tool call, and response metadata, including token spend and latency. For teams experimenting with multi‑agent workflows, this stack turns otherwise opaque interactions into actionable telemetry.
| Observation | Benefit | Overhead |
|-------------|---------|----------|
| Prompt & tool call logs | Detect malformed inputs; enable SLA monitoring | Increased storage cost |
| Token‑level spend metrics | Budgeting for GPT‑4 usage; detect runaway agents | Additional API calls to LLM provider |
| Latency traces per agent step | Pinpoint bottlenecks in orchestration | Potential latency impact on user experience |
Decision Point:
Should you instrument every agent or limit tracing to production traffic only? Full instrumentation gives the most insight but doubles telemetry data and can surface sensitive content. A hybrid approach—sampling high‑volume agents at 1 % frequency while fully tracing critical paths—balances observability with privacy and cost.
---
Deploy a lightweight wrapper around OpenAI’s Codex API or an open‑source LLM (e.g., GPT‑4o) in your dev environment. Measure token cost vs latency, then decide whether to continue with a managed service or build your own.
For the next major schema change, map out both double‑write + feature flag and event‑sourcing approaches. Run a cost‑benefit analysis that includes audit‑log storage under POPIA/UK GDPR compliance.
Add tracing to one production LLM agent per platform (e.g., customer support bot). Collect baseline metrics, then iterate on sampling rate and alerting thresholds to avoid overwhelming your monitoring stack.
---
Bottom Line:
This week’s engineering decisions hinge on how much latency you can tolerate, which compliance regime dominates your data flow, and the scale at which your agents will operate. By carefully weighing these trade‑offs, you can build systems that stay fast, remain compliant, and are observable—exactly the triad any CTO needs in 2026.
---
The recommendations around agentic factories assume that your organization has or can obtain sufficient GPU capacity and that you have compliance experts to audit token‑level data handling. Validation against actual infrastructure budgets (e.g., SA’s ZAR‑based compute pricing vs UK cloud rates) is needed before committing to a managed vs in‑house LLM stack. Additionally, the decision matrix for zero‑downtime migrations presumes existing CDC pipelines; if those are absent, the cost of building them may outweigh short‑term migration benefits.