Use Cases
While many of Nemesys Insights’ clients and the work we do for them cannot be disclosed for proprietary or security sensitivity reasons, we provide below several examples of prior work that we are able to publicly share:
AI CBRN Safety Evaluations and Mitigations
There is significant concern that rapidly improving large language models (LLMs) will enable malicious actors to acquire and use chemical, biological, radiological or nuclear weapons. Nemesys has worked with leading technology companies to evaluate and mitigate the safety risks that their models might pose in this area. Nemesys Insights has designed scientifically rigorous, state-of-the-art evaluation approaches, engaged in large-scale Red Teaming exercises and implemented supervised fine tuning and unlearning mitigation efforts for their clients. Descriptions of some of our work with one of our clients, Amazon Inc., can be found in the document located here.
Red Teaming Training for the U.S. State Department
Nemesys provided in depth training on how to apply Red Teaming as an effective decision support tool for improving operations at the State Department’s Bureau of Conflict and Stabilization Operations by identifying key uncertainties and reducing strategic surprises. Training was provided in the application of four different techniques to real-world case scenarios: Pre-Mortem Analysis, Key Assumptions Check, What If Analysis, and Alternative Futures Analysis.
Development of Biothreat Benchmarks and a Benchmark Generation Framework for Frontier Models
As part of work for the AI Safety Fund, Nemesys developed the Bacterial Biothreat Benchmark Generation (BBG) Framework to improve and advance the ability of biothreat benchmarks to provide safety assessments of AI models. This included developing a conceptual architecture for biothreats and using three different approaches – web-mediated prompt generation, extraction from existing corpora, and asynchronous dynamics red teaming – to develop a set of benchmarks that are both aligned to the biosecurity threat chain and diagnostic in the sense of providing uplift over traditional search tools.
Global Survey on the Origins and Implications of the COVID-19 Pandemic
Nemesys conducted the first global survey of experts addressing a highly controversial topic: the origins and implications of the COVID-19 pandemic. A rigorous survey of epidemiologists, virologists and other scientific experts was carried out across 47 countries, with the results being analyzed to extract lessons for future pandemics. The corresponding report was widely cited and discussed, both among experts and the news media.
