This is an idea proposed in 2022 as a Cambridge Computer Science PhD topic, and has been completed by Zahra Tarkhani. It was supervised by Anil Madhavapeddy as part of my Information Flow for Trusted Execution project.
This PhD project proposes novel approaches and mechanisms for application compartmentalization and isolation to reduce their ever-growing attack surfaces.
Our approach is motivated by the key observation that while hardware vendors compete to provide security features (notably memory safety and privilege separation) existing systems software like commodity OSs fail to utilize such features to improve application security and privacy properly.
We propose a novel principled approach to privilege separation and isolation, enabling application security to be designed and enforced within and across different isolation boundaries, and yet remain flexible in the face of diverse threats and changing hardware requirements.
Specifically, we design dispersed compartments as a building block for applications that can encapsulate arbitrary isolation boundaries across privilege levels. Dispersed compartments provide a unified model for extensible and auditable compartmentalization. To enable such system-wide privilege separation, we introduce two key concepts; first, dispersed monitoring to check extensible security policies. Secondly, dispersed enforcement to enforce isolation and security policies across various privilege boundaries while reducing the trusted computing base (TCB) through deprivileging the host kernel on-demand.
See Zahra Tarkhani's completed PhD thesis on the subject for more details!