Automated Evidence Generation for Continuous Certification
No ratings
Presented at High Confidence Software and Systems Conference 2022 by
This talk will introduce the methodological and tooling foundations of an automated evidence generation workbench devised to support continuous assurance case-based certification and Authority-to-Operate (ATO). We will discuss approaches and challenges associated with the systematization and automation of diverse evidence generation techniques, including static analysis, testing, and Formal Methods. Furthermore, we will present the distinction between information for assurance assessment versus developer feedback. In contraposition to activities that require substantial human judgment, such as requirements capture or risk assessment, evidence generation offers substantial opportunities for automation and hence for accelerating certification. However, the current practice in evidence generation is still largely manual, producing results that vary widely in content and format across tools and users. Consequently, assessing the relevance and validity of evidence artifacts is difficult. In many cases, assurance engineers fail to consistently record the intent, assumptions, justifications, and context of the evidence. Further, shaping the inputs that feed evidence generation, especially in the context of Formal Methods tools, still requires significant human involvement. We propose reusing and recomposing evidence in conjunction with automated input transformations to improve efficiency and speed in continuous assurance regimes that assume incremental or partial system changes. In our talk, we will describe three essential elements of our approach in detail: A model of incremental, continuous system evolution and the assurance information supporting continuous certification/ATO. Evidential Assurance Case Fragments (EACFs), as packages of composable and reusable evidence, including their computational representation. The Evidence Generation Language (EGL) — a Domain-specific Language devised to systematize and automate diverse evidence generation techniques, including Formal Methods. Finally, we will conclude the presentation by covering several examples of techniques implemented in our evidence generation workbench, along with results.