AI offers tremendous opportunities to build new and useful tools. It also offers tremendous opportunities to shoot ourselves in the foot, via malfunctions, malicious use, security vulnerabilities, and even complex societal effects. To prevent this, we need to design, build, and test our systems with those failure modes in mind. But what does it mean to do that? How can we do that efficiently and effectively, especially in a world where the technology is changing this rapidly? The answer can be found in going back to the very basics of engineering technique, and understanding how to apply it to this new world.