CMU SEI Research Review 2024

Dr. Tom Longstaff will provide an introduction to kick of the Software Engineering Institute Research Review 2024.

​Message from the CTO​​​

Modern data analytic methods and tools, including Artificial Intelligence (AI) and Machine Learning (ML) models, depend on correlations; however, such approaches fail to account for confounding in the data, which prevents accurate modeling of cause and effect and often leads to bias. Edge cases, drift in data/concept, and emerging phenomena undermine the significance of correlations relied upon by AI. New test and evaluation methods are therefore needed for ongoing evaluation. Carnegie Mellon University Software Engineering Institute (CMU SEI) has developed a new AI Robustness (AIR) tool that allows users to gauge AI and ML classifier performance with data-based confidence.

​Learn More about AIR​​​

Hardware inefficiencies pose major limitations to U.S. Department of Defense (DoD) applications; current processors simply cannot keep up with the large and complex machine learning (ML) workloads needed to perform their missions. Utilizing co-design (concurrent design of electronic system hardware and software components), this project aims to provide tools to overcome these limitations and yield practical results that increase battery life, reduce weight warfighters must carry, increase maneuverability of forces, and reduce time from sensor to shooter.

​Learn More​​​

Detecting malicious code is a challenge, particularly when it’s implanted in otherwise legitimate software. If undetected, malware injected into legitimate software can result in costly compromises of computer systems, such as in the SolarWinds incident of 2020. This project aims to produce a tool to detect such malicious code to prevent system compromises. Our goal is to enable analysts throughout the DoD to detect malicious code before it can compromise DoD systems.

​Learn More​​​

Deploying critical software with certification and frequent updates is a major challenge. Exhaustive testing of safety-critical systems is not possible due to the exponential growth of test cases; extensive but non-exhaustive testing is potentially unsafe; and while formal methods (FM) can provide full coverage, many FM tools have been found to be defective, producing the wrong output and leaving software practitioners understandably hesitant to trust FM tools.

​Learn More​​​

This project focuses on creating a foundational model for all container reproducibility efforts (open-source, commercial, U.S. Department of Defense (DoD), etc.). The lack of build reproducibility is a fundamental problem that affects most software built today. While some tools exist to aid in promoting build reproducibility, the research and developer communities lack a holistic model of container build reproducibility and its failure scenarios. This project aims to develop this model as well as tools for verifying reproducibility in container builds.

​Learn More about Vessel​​​

Assuring evolving large-scale systems (i.e., systems with multiple subsystems) can be a bottleneck in deploying capabilities with the speed and confidence needed for current Department of Defense (DoD) missions. There is no easy way to automatically integrate the complex web of interacting assurance techniques (i.e., for control stability, timing, security, logical correctness, and more) from multiple interacting subsystems in a system. In addition, without awareness of assurance interdependencies and effective reuse of prior assurance, the cost and time required for re-assurance can skyrocket.

​Learn More about Large-Scale Assurance​​​