The Problem With Assumed Security
As systems scale, security shifts from a one-time setup to an ongoing architectural responsibility that needs continuous attention as infrastructure evolves and teams change. A key risk in this environment is the gap between assumed and actual security posture, where controls are taken for granted rather than actively verified, and unintended traffic paths emerge by default. Over time, these small gaps can accumulate, making it important to regularly reassess and reinforce how security is implemented in practice.
This blog is about how we took a hard look at our edge security architecture, asked some uncomfortable questions about what was actually being enforced versus what we were taking for granted, and built something more robust as a result.
Starting With First Principles: What Are We Actually Protecting?
Before making any changes, we stepped back and asked a basic question: what does our intended traffic flow look like, and where are the points where that intent could break down?
Our design intent was clear: all inbound traffic should pass through our edge security layer- the component responsible for request filtering, rate limiting, bot mitigation, and request validation before reaching backend services.
The real question was: was this actually enforced everywhere, or just assumed?
This distinction matters a lot. An assumed control is one that works under normal conditions but hasn't been explicitly hardened against someone who is actively trying to circumvent it. An enforced control is one that works regardless of how traffic arrives.
When we mapped out our ingress paths end-to-end, we found that the answer wasn't as clean as we'd hoped. There were paths through which traffic could reach backend services without going through the edge layer at all. Not because of any failure in the edge itself, but because the architecture didn't explicitly close off alternatives.
This is an architectural enforcement gap. It's not a bug in any single component. It's a gap in how the components were wired together. And it's surprisingly common at scale, because infrastructure grows faster than the intent behind it.
Our intended request flow was simple:
Public GLB IPs let attackers skip WAF entirely:
Why Network-Level Controls Aren't Enough
The first instinct in situations like this is often to reach for IP whitelisting, restrict inbound traffic to known source ranges and call it done.
We considered this, and deliberately chose not to rely on it as the primary control. Here's why:
- IP ranges change. Maintaining an accurate, up-to-date whitelist across a large, dynamic infrastructure is operationally expensive and error-prone.
- Network location isn't identity. Knowing that a request came from a particular IP range tells you something about where it came from but it doesn't verify that it actually traversed the security controls you intended.
- Misconfiguration risk grows with scale. The more entries in a whitelist, the higher the probability that something is wrong, stale, or overlapping.
So we looked for a way to verify the request itself, rather than relying only on where it appeared to come from. What we wanted instead was verifiable proof, a signal that a request had actually passed through our edge layer, not just that it appeared to come from the right place on the network.
The approach we settled on was header-based origin verification. Our edge layer injects a header into every request it forwards. The load balancer validates the header before routing traffic to backend services. Requests that arrive without this header, regardless of source IP are rejected before they reach any backend.
This is a meaningful shift in how trust is established: from network location to request-level verification. Even if someone knows the addresses of our backend infrastructure, they can't replicate the header that only our edge layer produces, because it travels over an encrypted channel end-to-end.
Closing the Gap: Domain Migration and Enforcement at Scale
Closing the enforcement gap required more than adding a verification mechanism, it meant ensuring that every domain we operated was actually routed through the edge layer. At scale, this is rarely straightforward. Over time, DNS tends to accumulate complexity: outdated records, wildcard entries that obscure what’s truly active, and root domains that require special handling due to DNS constraints.
We approached this systematically by classifying domains based on their role and risk profile. User-facing services and backend endpoints reachable via the load balancer were prioritized for edge protection, while static assets which serve largely immutable content without dynamic logic were handled differently. Externally managed domains were also treated separately. This allowed us to focus enforcement where it mattered most, instead of applying a one-size-fits-all approach.
The migration itself was done in stages. We began with validation in pre-production, followed by running policies in alert mode in production where traffic was observed and flagged, but not blocked. This gave us time to tune the system and build confidence before moving to full enforcement. Only after a period of stable behavior were domains brought under enforced policies, ensuring a smooth transition without impacting live traffic.
Certain edge cases required additional handling. For example, root domains cannot directly point to dynamic hostnames due to DNS limitations. We addressed this by redirecting them to their corresponding subdomains using dedicated infrastructure, keeping the logic isolated while ensuring all traffic continued to pass through the edge layer.
For domains not yet fully onboarded, we routed unverified traffic to a logging-only backend. This provided visibility into unexpected traffic patterns without forwarding requests to live services, allowing us to close gaps safely and iteratively.
Honest Limitations
Any security write-up that doesn’t acknowledge limitations should be read with skepticism. Here’s ours:
The header-based verification approach we implemented depends on correct configuration of the encrypted channel between the edge and the load balancer. If that configuration drifts, the trust signal can be weakened.
This approach also doesn’t provide per-request cryptographic signing. It’s a strong operational control, not a cryptographic proof and that distinction matters in threat modelling.
Not all parts of the system require the same level of enforcement. For example, static asset backends are lower risk because they primarily serve immutable content without dynamic logic behind them. This allows us to apply controls more selectively based on context.
What this approach does provide is a meaningful reduction in unintended ingress paths, with lower operational complexity than many alternatives. For our context, this trade-off has worked well, though different systems may require different choices.
What We Took Away From This
A few things became clearer through this process:
- Security controls need to be enforced, not assumed. The gap between intent and reality is where most issues emerge.
- Verify the request, not just the source. Network location alone isn’t a reliable signal of trust.
- Alert before you enforce. Observe and build confidence before blocking traffic in production.
- DNS hygiene matters. Keeping domain mappings clean is essential for consistent enforcement.Staged rollouts help reduce risk. Phased migrations make it easier to validate and course-correct early.
Closing Thoughts
The work described here wasn't a single fix. It was a series of deliberate architectural decisions made in sequence, each one building on the last.
The core shift was moving from a model where security was assumed to flow through the right path, to one where it's explicitly verified at each step. That's a harder system to build, but a much more trustworthy one to operate.
At scale, security is less about any individual control and more about consistently enforcing trust boundaries and making sure that traffic can only flow through the paths you explicitly intend.
