We will be presenting a new approach to evaluating web application firewall capabilities that is suitable to the real world use case. Our methodology touches on issues like False Positive / False Negative rates, evasion techniques and white listing / black listing balance. We will demonstrate a tool that can be used by organizations to implement the methodology either when choosing an application protection solution or after deployment.