Engineering & Architecture: Build Decisions This Week
2026‑11‑21
---
The week has brought a mix of geopolitical ripples and practical engineering take‑aways. A national strike in Portugal has exposed the fragility of public‑sector services, while industry thought‑leaders have sharpened the lens on AI infrastructure, code‑review economics, and agent‑centric governance. For CTOs juggling the South African and UK/European landscapes, the message is clear: build decisions that balance cost, control, and risk are more critical than ever.
The Inference engineering for software engineers deep‑dive illustrates how Uber, Pinterest, and AT&T have slashed per‑token costs by 50 %–90 % by moving from frontier‑lab APIs to open models. The author presents the classic “bare‑metal vs. cloud GPU” decision matrix:
| Option | CAPEX | OPEX | Control | Latency | Ecosystem |
|--------|-------|------|---------|---------|-----------|
| Bare‑metal GPUs (on‑prem or colocation) | High upfront, predictable | Low once the hardware is in place | Full control over hardware and software stack | Lowest end‑to‑end latency | Requires in‑house expertise |
| Cloud GPU (e.g., AWS, GCP) | Zero upfront, variable | Pay‑as‑you‑go, often higher per‑token | Limited control over GPU firmware | Slightly higher latency due to network hop | Easier scaling, integrated monitoring |
In South Africa, where broadband costs can be volatile, the trade‑off leans toward bare‑metal if you have a stable, low‑latency connection to a colocation facility. In the UK or EU, where regulatory compliance and data residency can be stricter, a cloud‑first approach may reduce the burden of managing physical hardware and help satisfy GDPR‑style requirements.
Actionable take‑away: Run a cost‑benefit analysis that incorporates your average inference volume, peak‑hour traffic, and the expected lifespan of the model. Factor in the opportunity cost of hiring GPU‑operations talent versus using a managed cloud provider.
---
Harjot Gill’s interview on Software factories and code reviews with Harjot Gill shows how a small team can ship over 5,000 pull requests in a month by investing heavily in automated review pipelines. He notes that “human code review is dead” in favour of AI‑driven triage, yet that such systems can cost upwards of $250,000 a month if not properly scoped.
Key trade‑offs:
| Factor | Manual Review | AI‑Assisted Review |
|--------|---------------|--------------------|
| Speed | Slower, depends on reviewer availability | Faster, near‑real‑time feedback |
| Cost | Low tooling cost, high labor cost | High tooling cost, lower labor cost |
| Quality | Consistent human judgment | Variable, depends on model training |
| Security | Can catch subtle policy violations | May miss nuanced context |
For teams that already run CI/CD pipelines on Kubernetes or Docker‑based workloads, integrating an AI review step can reduce cycle time by 20 %–30 %. However, if your codebase includes sensitive regulatory logic (e.g., compliance‑centric microservices), a hybrid approach—AI triage followed by a mandatory human gate on high‑impact changes—often provides the best balance.
Actionable take‑away: Pilot an AI code‑review tool on a single repository, measure time to merge and defect density, and compare against the baseline of manual reviews.
---
The Evals for AI Engineers article discusses the challenges of deploying AI agents at enterprise scale, citing GlobalFoundries’ use of Redpanda to isolate data streams and enforce freshness constraints. The talk emphasizes that agents must be evaluated against real‑world data, with strict controls on the data they can access.
Critical design choices:
| Choice | Pros | Cons |
|--------|------|------|
| Centralised evaluation server | Easier governance, single point of truth | Bottleneck, potential latency |
| Distributed evaluation with event brokers (e.g., Redpanda) | High throughput, decoupled services | Requires operational expertise |
| On‑prem sandboxed agents | Strong isolation, no external dependencies | Hard to scale, difficult to patch |
For South African developers dealing with intermittent connectivity, a distributed broker can buffer agent calls and allow offline processing. In the EU, where GDPR mandates strict data residency, keeping the broker within the same jurisdiction can satisfy compliance.
Actionable take‑away: Build a lightweight evaluation harness that mocks critical external APIs, runs nightly test suites, and feeds metrics into a monitoring stack. Use that data to gate promotion of agents into production.
---
---
Sources
Review Note