Seminar: Value Forks

Value forks arise when we want artificial agents to realize different moral values than human agents in otherwise identical situations. In a dilemma situation, for instance, we might prefer a human agent to act in accordance with certain deontological maxims, yet hold that an AI agent should maximize overall utility. Value forks thus pose a serious challenge for value alignment of AI: If values are agent-type-sensitive, which values should we align AI to?

The presentation has three parts. First, I will explore on moral-philosophical grounds whether value forks could, in principle, arise. Here the question is whether it is normatively appropriate to think that AI agents must sometimes pursue different values than human agents in otherwise identical situations. Second, I will present data from a suite of empirical experiments which suggests that value forks do arise in folk morality. That is, according to the evidence, laypeople want to see different values realized across agent-types (human v. AI). Naturally, morality (part 1) and social morality (part 2) do not need to converge. However, when they come apart this can itself give rise to interesting moral challenges (see e.g. Danaher’s (2016) work on the “retribution gap”). Third, I will trace out the implications of value forks from the perspective of philosophy on the one hand, and responsible AI governance on the other.

Markus Kneer (BA: Oxford, PhD: ENS Paris) is Professor of Ethics of Artificial Intelligence at the University of Graz, Austria and the Director of the Guilty Minds Lab. He works on a variety of topics in ethics of AI, HCI, philosophy of language and experimental philosophy. His new research project concerning norms of assertion in linguistic human-AI interaction just started in March 2025.

Alternatively, join online:

Microsoft Teams

Meeting ID: 359 476 699 597 1

Passcode: cR3hk7xT