To discuss conflicting visions in AI personalisation in this article, personalisation is defined as a suite of technologies designed to filter out irrelevant information and provide recommendations adapted to a person’s tastes and needs. But distinct from personalisation used on commercial platforms such as Facebook, Google, and Amazon, which is increasingly viewed through the lens of engineering optimisation and control. Large corporate platforms approach personalisation through a self-interested, neoliberal economic logic of competition, catalysed by technological imperatives of innovation and creative destruction. Personalisation includes segmentation, hyper-personalisation or individualisation does not.
Therefore applying engineering formalism and economic theory to your typical ecommerce website, raises ethical issues around personal autonomy and human self-determination, some of which are now being translated into formal legislation in places such as the European Union with the GDPR, and the recently proposed AI Act and Digital Services and Markets Act.
Economic Justifications for AI Hyper-personalisation
In economic theory, platforms are conceived as multi-sided markets aimed at the commodification of user activities and content. Interested third parties, particularly advertisers, benefit from “thick markets” of human attention provided by the platform’s algorithmic standardisation and governance structures. Specialisation serves mutual advantage through trade. Ecommerce retailers contract out the complex work of machine-learning-driven market individualisation of product selection to specialist sources like an advanced autonomous SaaS. These offer top-tier data scientists, computational power, and massive volumes of implicit behavioural data, enabling them to utilise their scarce resources in AI personalisation messages, and product recommendations of such accuracy that the ROI is ever likely to be beaten.
Personalisation is thus the technical realisation of this market-driven view of economic freedom in which persons can get what they prefer — no matter the genealogy or legitimacy of those preferences. Personalisation, so viewed, promotes freedom because it is based on voluntary, and thus fair, exchange: rational users are assumed to voluntarily interact with the platform/stack and leave behavioural trace data which can then be processed based on users’ revealing their wants and needs.
Conflicting Visions of Personalisation
To better make sense of this tension between humanistic and economic perspectives relevant to personalisation, we’d like to adopt a distinction made by Thomas Sowell in his insightful book A Conflict of Visions.
Sowell, an economist by training, distinguishes between two conflicting political and moral visions in Western thought. These visions are all-encompassing worldviews that not only bias one’s ethical and political theories but also one’s understanding of the nature and scope of scientific knowledge. Yet these two visions are mutually incompatible. In effect, where one sees a duck, the other sees a rabbit. For instance, the standard economic thinking behind the notion of consumer sovereignty implies that because we desire something, it must be good. Yet Kant claims the opposite: something is good, therefore we must desire it (insofar as we are rational beings).
We surmise the emergence of the field of AI ethics is a manifestation of this conflict of visions. As such, the rapid growth of work and interest in AI ethics should be interpreted as expressing dissatisfaction that personalisation technology has up to now neglected essential elements of the unconstrained vision. We can roughly associate these unconstrained elements with what Habermas calls our hermeneutical and emancipatory interests in achieving mutual understanding and freedom from our baser, animalistic nature.
The Constrained Vision
Up till now, personalisation technology has largely been developed in pursuit of the constrained vision. The constrained vision follows an intellectual thread first articulated by Scottish Enlightenment thinkers such as Adam Smith and David Hume. It prioritises correlations over causation, elevates intuitive, empirical knowledge implicit in habits and traditions over explicit reason, regards observable consequences over unobservable intentions, and weighs ideals against the costs required to achieve them. The perfect should not be the enemy of the good. Scarcity and finitude are viewed as fundamental aspects of human existence, and thus questions about the relative efficiency of various ways of distributing such scarce resources are prioritised. The constrained vision accepts tradeoffs as an inevitable fact of life.
From an ethical perspective, the constrained view relies heavily on consequentialist, instrumentalist reasoning about right action and values, and takes a “system-level” utilitarian view favouring properties of aggregates over individuals. The constrained vision shares a philosophy of science similar to both behaviourism and (logical) positivism in that it looks for universally unchanging law-like explanations of human behaviour by reference to observable, verifiable empirical features of the environment, avoiding dubious metaphysical claims of unverifiable, unobservable internal causes. As with neo-classical economics, the constrained vision seems to suffer from a clear case of physics envy, whereby the materialist science of physics embodies the methodological and epistemological ideal of human investigative inquiry.
The Unconstrained Vision
Personalisation technologies have largely been developed independently of considerations of the unconstrained vision, whether for practical or ideological reasons. But this may slowly change as the field of AI ethics grows.
The unconstrained vision descends from French and German Enlightenment-era ideas developed by Condorcet and Immanuel Kant and elevates reason and conscious deliberation above intuition, habit, and tradition, highlights abstract ideals over the actual sacrifices needed to achieve them, and marvels at the power of the cultivated human mind to arrive at self-knowledge and universal truths. Presupposing that we credit each individual with consciously having personal preferences let alone knowing what they are.
The unconstrained vision views certain actions and states of affairs as intrinsically good, owing to their objective properties as seen by ideal impartial observers, or as the result of an idealised procedure of deliberation or universalisation. The instrumental or consequentialist reasoning of the constrained vision is generally viewed as inferior because it tends to uncritically take ends as given (but by whom?) and can justify treating persons as objects, mere means towards someone else’s end.
In its political, ethical, and legal forms, the unconstrained vision often endorses placing strict rules on what can be done to individuals in pursuit of the common good, often employing the concept of rights as “trumps” to delineate this sphere of individual sanctity that cannot be crossed no matter the utility or consequences.
In terms of a philosophy of science, the rationalism of the unconstrained vision goes beyond observable correlations and explains human behaviour by reference to internal causes of action, which involve unobservable intentions, and whose contents may only be fully accessible and intelligible to those participating in a shared form of life or culture. The unconstrained vision endorses a realist or even transcendental realist metaphysics.
