Self-Governing Systems

Jeremy Pitt^1*Asimina Mertzani¹Josiah Ober²

• ¹Imperial College London, London, United Kingdom
• ²Stanford University, Stanford, United States

A key driver in the digital transformation of commercial, educational, organisational and social systems is the increasing footprint of Artificial Intelligence (AI). This is producing a different kind of hybrid socio-technical system, which consists of interacting human and artificial 'components'. However, beyond the realisation of basic Agentic AI, AI components are likely to be taking over more advisory, supervisory and administrative roles, especially with respect to human components, and potentially without oversight from some external authority. This raises fundamental questions of self-governance -i.e. both operational and constitutional decisions concerning the selection, modification, application and enforcement of social arrangementsas a co-production of meaningful interaction between human and artificial intelligences. Using examples, this paper scopes out the identifying features of such self-governing systems, which raise several critical political questions about the kind of human rights that could reasonably be expected in such systems. This includes agency, voluntary association, empowerment, innovation and metrication, as they relate to this profound shift in our understanding of "humancomputer interaction" and "human-machine teamwork". Finally, given that self-governing social systems don't tend to persist if they can't adapt to a changing environment or resist entropic decay, we consider the idea of continuous self-improvement as a right in itself, from the perspectives of human factors and user experience, and what this implies for human flourishing and the right to human rights in this new techno-political ecology.