Security co-pilots, chatbots and automation that leverage large language models are rampant in Security Operations with the intent of boosting analyst productivity and outcome quality. While there is a lot of focus on implementing GenAI use cases for the SOC, **there is little focus on understanding the effects of introducing GenAI tooling** before and after implementation in an analyst workflow leading to a counter-productive "AI in the human loop" scenario. This session covers 1. Results from **A/B testing** different types of AI models with different levels of tooling and workflow integration and what it means for a security practitioner 2. Insights gained around friction points in integrating and obtaining alignment with GenAI in SecOps