Secure AI Starts with Solid Compute Foundations

No ratings

Presented at No Hat 2025 by

When your favorite AI companion responds to your prompts, an intricate software ecosystem is activated. Although all of this seems straightforward, the reality is very different: a single API call to a cloud service endpoint triggers a cascade of events that ultimately translates into machine code executing on power-efficient CPUs. Fast data transfers run through metal wires and gates, sitting in a metal server chassis, hosted in enormous datacenters that drain megawatts of power. All of this should be fast, efficient, and secure. This keynote starts here, in the deepest layers of the modern cloud computing stacks. After an overview of a custom software and hardware system that we designed and called Nitro, we will dive into how specialized security teams operate from the silicon level up to the hypervisor by defining security requirements, performing design reviews, conducting security testing, and investing in research & tooling to continue innovating. We will explore how we continue to engineer security into the architecture of every system our team builds, to create the secure foundations for modern AI applications.