CryptoAPI-Bench: A Comprehensive Benchmark on Java Cryptographic API Misuses

No ratings

Presented at IEEESecDev 2019 by

Sharmin Afrose

Sazzadur Rahaman

Danfeng Yao

Several studies showed that misuses of cryptographic APIs are common in real-world code (e.g., Apacheprojects and Android apps). There exist several open-sourcedand commercial security tools that automatically screen Javaprograms to detect misuses. In order to compare their accuracyand security guarantees, we develop a comprehensive benchmark named CryptoAPI-Bench. CryptoAPI-Bench consists of171 unit test cases that cover basic cases, as well as complexcases, including interprocedural, field sensitive, multiple classtest cases, and path sensitive data flow of misuse cases. Thebenchmark also includes correct cases for testing false positiverates. We evaluate CryptoAPI-Bench on four tools, namely,SpotBugs, CryptoGuard, CrySL, and Coverity and present theirperformance and comparative analysis. Our benchmark is usefulfor advancing state-of-the-art solutions in the space of misusedetection.