Engineering & Architecture: Build Decisions This Week
2026‑11‑19
The tech landscape is still in flux after last week’s policy shifts in South Africa and the UK, but the core questions for an engineering leader remain the same: how to keep costs under control, how to architect for flexibility, and how to lock in a secure, auditable pipeline. Below are three concrete build actions that align with the latest industry thinking from the Pragmatic Engineer and ByteByteGo newsletters, and that can be applied in both South African and European contexts.
---
The Inference engineering for software engineers article shows that large firms like Uber and Pinterest have already slashed per‑token costs by 50 %–90 % by moving from frontier‑lab APIs to open‑source models. The choice boils down to bare‑metal vs. cloud GPU services.
| Option | CAPEX | OPEX | Control | Latency | Compliance |
|--------|-------|------|---------|---------|------------|
| Bare‑metal GPUs (on‑prem or colocated) | High upfront, predictable | Low, only maintenance | Full control of KV cache, network isolation | Very low, hardware‑bound | Simplifies POPIA/GDPR data residency |
| Cloud GPU services (AWS/GCP) | None | Pay‑as‑you‑go, higher for sustained use | Managed, easier scaling | Slightly higher, multi‑tenant | Requires careful data‑egress controls |
If your products touch regulated domains, keeping inference in‑house reduces data‑egress exposure and eases audit trails under the POPIA Act 4 of 2013 or GDPR. However, the capital outlay and ops overhead are non‑trivial; you’ll need a robust supply‑chain for NVIDIA hardware, a seasoned infra team, and a well‑defined fail‑over strategy. For bursty workloads, a hybrid approach—local inference for latency‑critical paths and cloud for seasonal spikes—offers a middle ground.
Action for this week: Conduct a quick cost‑benefit analysis of moving 10 % of your inference traffic to the cloud, including data‑egress costs and compliance implications.
---
Harjot Gill’s interview on Software factories and code reviews with Harjot Gill describes a scenario where a CEO is spending $250 k/month on AI tooling, shipping over 5 000 PRs a month, and believes human code review is becoming obsolete. The trade‑off is between speed and the risk of subtle bugs slipping through automated checks.
| Approach | Cost | Velocity | Assurance |
|----------|------|----------|-----------|
| Human review | Low tooling cost, high manpower cost | Moderate | High, if reviewers are experienced |
| AI‑assisted review | Medium tooling cost, low manpower | High | Medium, depends on model fidelity |
| Fully automated | Medium tooling cost, minimal manpower | Very high | Low, requires extensive testing |
The key takeaway from the episode is that AI can surface patterns and flag violations faster than humans, but it is not a silver bullet. A practical middle ground is to adopt an AI assistant that surfaces the most likely problematic lines, and then let a human reviewer focus on high‑impact areas—effectively a “human‑in‑the‑loop” approach.
Action for this week: Pilot an AI code‑reviewer on a low‑stakes microservice and measure the reduction in review time versus the number of regressions found in production.
---
Evals for AI Engineers outlines the challenges of production‑grade AI agents: data freshness, scope of reach, and keeping the agent “in bounds”. The case study from GlobalFoundries shows how Redpanda was used to stream agent interactions and maintain an audit trail.
A robust evaluation framework should include:
In South Africa, where internet latency can vary, a local instance of the agent that mirrors the production dataset ensures that compliance checks (POPIA) are performed within the country, avoiding cross‑border data transfer concerns.
Action for this week: Design a simple eval harness for one of your current AI agents, using Redpanda as the event bus, and run a simulated load to uncover any security or data‑freshness gaps.
---
These three actions give a clear roadmap for engineering leaders to address cost, security, and scalability in a manner that is immediately actionable and aligned with the latest thought leadership.
---
Review Note:
**
Sources: