In Summary
Over the course of human history, the source of authority in decision-making has shifted from the gods to humanism, and now to dataism. The shift from humanism to dataism represents a significant change in the source of authority in decision-making. While the use of algorithms in decision-making has its advantages, it raises significant concerns about their potential to perpetuate discrimination and bias. To mitigate these concerns, we must ensure that algorithms are transparent, explainable, and accountable and that decision-making remains human well-being as the centerpiece.
___________________________________________________________________________
For most of human history, the source of authority came from the heavens. Human beings in ancient societies looked to the sky for guidance on how to live and make decisions of all kinds. It was not uncommon, for instance, for ancient humans to consult with the gods before making decisions about matters such as who to marry, whether to go to war, dealing with problematic neighbors, or tax matters. In ancient Greece, decisions about public policy and war were often made after consulting oracles, who were believed to have a direct line of communication with the gods. See, the logic was that the gods were all-knowing, all-seeing, and omnipresent, and could offer insights and wisdom beyond what was available through human reason and experience. The will of the gods was believed to be the ultimate authority.
This changed, starting in the Renaissance period in Europe, between the 14th and 17th centuries. During this time, there was a renewed interest in the works of the ancient Greeks and Romans, which emphasized the importance of human reason and experience in understanding the world. This ushered in the rise of humanism, which placed a high premium on individual achievement, creativity, and the importance of the human perspective. Human beings began to look to themselves, rather than the gods, as the source of authority in decision-making. The emphasis became on the power of the individual to shape their own destiny through hard work, determination, and self-belief. More importantly, it emphasized human agency in decision-making – you could be anything you set your mind to, and, the customer is always right. This is the world you and I exist in today. Until now, at least.
In recent years, Ai algorithms are increasingly used to make or support decisions previously made by human decision-makers. Specifically, Ai algorithms are used to process large volumes of data in order to make statistically accurate decisions, which are then applied and implemented in the real world – often engaging human rights impacts for individuals.
On the lower end of the spectrum, algorithms are used every day to make and support decisions such as the best route to take to a destination, which emails are spam, which Netflix show to watch, or what product to buy next. On the higher end of the spectrum, Ai algorithms are being used to make and support decisions with a legal impact for individuals, such as in criminal justice and policing decisions, social welfare, credit, and employment decision, global finance, warfare, surveillance, education, and the distribution of public services.[1]
Why? Well, it’s because Ai is perceived to be better than human beings at making decisions. Ai benefits from a perceived sense of efficiency,[3] rationality, and flawlessness[4] because of its capacity to analyze large volumes of wide-ranging, problem-relevant data, and find patterns in these large data sets in order to make statistically accurate decisions, predictions, or recommendations to the user.[5] If you’ve used ChatGPT or any other recent system, you know that this can happen at incredible speed, saving time and resources. Secondly, Ai algorithms are perceived to be more “objective” and “neutral” than humans, free from the biases and errors that animate human decision-making. The shift towards dataism, then, is based on the belief that data and algorithms that process it can provide insights and wisdom beyond what is available through human reason and experience; the algorithm is all-knowing, all-seeing and the data is omnipresent. Sounds familiar?
At first, glance, relying on algorithms for decision-making seems like an efficient solution. In a world where humans produce an overwhelming amount of data every day, algorithms can sift through this data and identify patterns that are impossible for humans to detect. However, complete reliance on algorithms can severely limit our ability to think critically and understand the world around us.
Further, algorithms lack context, meaning, and flexibility. For instance, algorithms are unable to account for the nuances and complexities of human decision-making (like social and cultural factors), and for the value-based, ethical considerations that often arise in these contexts. Ai algorithms can also perpetuate biases and discrimination because they are only as objective as the data on which they are trained. If the data contains biases, then the algorithms will also be biased, potentially leading to discriminatory outcomes.
Dataism emphasizes data and algorithms and suggests that the universe is made up of information, with data collection and processing being the most important things, rather than human experience or consciousness. This raises deep philosophical questions about the nature of consciousness and the role of humans in the universe. If the universe is merely a giant data processing system, what does that mean for the value of human experience and consciousness? What does it mean to be human?
These questions require us to think critically about the role of data in our lives and to actively shape the way data is collected, processed, and used.
However, algorithms are tools, and like any tool, they can be used for good or ill. To address the concerns raised by the use of algorithms in decision-making, we need to incorporate a human-centered approach to algorithmic decision-making, recognizing the value of human experience, emotions, values, and relationships in decision-making.
It’s advisable to approach algorithms with caution and to recognize their limitations. While they can help us make better decisions, they should not replace human judgment and decision-making entirely. The best approach is one that values both objective data and subjective human experience. This requires critical thinking about the role of data in our lives and actively shaping the way data is collected, processed, and used. At TECHila Law, we employ a human-centered approach to algorithmic governance, balancing both efficiency and humanity. Our mission is to help you do the same in your organisation.
Authors: Keketso Kgomosotho and Kyle Cloete are co-Founders of TECHila Law, a consulting form working at the intersection of law, human rights and emerging technology.
[1] Tshilidzi Marwala, Closing the Gap: The Fourth Industrial Revolution in Africa, Pan Macmillan South Africa, October 2020, ISBN 9781770107878; Adams, R., Pienaar, G., Olorunju, N., Gaffley, M., Gastrow, M., Thipanyane, T., Ramkissoon, Y., Van der Berg, S. & Adams, F., Human rights and the fourth industrial revolution in South Africa (2021) Cape Town: HSRC Press; Adrian Cozgarea, Artificial Intelligence Applications In The Financial Sector, Academy of Economic Studies, Bucharest, Available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.473.6463&rep=rep1&type=pdf#page=58
[3] Defined as “a situation where each good is produced at the minimum cost and where individual people and firms get the maximum benefit from the use of the resources.”
[4] Gillian Tett, Mapping Crime – or Stirring Hate? When Chicago Police Ran Their Predictive Statistics, There Was a Strong Racial Imbalance, Financial Times, available at https://www.ft.com/content/200bebee-28b9-11e4-8bda-00144feabdcO
[5] B. and Spiess (n 13).; European Parliament. Directorate General for Parliamentary Research Services., Understanding Algorithmic Decision-Making: Opportunities and Challenges. (Publications Office 2019) <https://data.europa.eu/doi/10.2861/536131> accessed 2 August 2022.
