Power and inequality: is tech neutral?
By Sanne Stevens, in collaboration with Commons Network
One could argue that institutions in the Netherlands seem heavily focused on hunting down people who 'cheat' the welfare system. While white collar fraude, tax evasion, Double Irish and Dutch Sandwich tactics are all much more expensive for the taxpayer (check out page 15 of this report by SOMO for more on those tactics). Yet new technology is implemented to make the hunting for welfare cheaters more efficient, more automated.
Power structures and ingrained inequalities are reproduced and often re-inforced by new technologies and the way our society implements them. But technology, isn’t that neutral and objective?
A simple example: in the Netherlands we are taught that social security fraude is wrong, and so it goes. One could argue that our institutions are heavily focused on hunting down people who 'cheat' the welfare system. The burden of the practices by the wealthy – white collar fraude, tax evasion, 'Double Irish' and 'Dutch Sandwhich' tactics – on our annual budget outweigh the cost of social security fraude exceedingly. Yet new technology is implemented to make the hunting of welfare cheaters more efficient, more automated. Leading to more and more unjust situations for the most vulnerable, who are left fighting systems that are now automated and anonymous, run by computers who simply say no.
Mathematician and algorithm researcher Cathy O'Neil: ‘Those who develop the code have the power, and those who have power, apply the code to people who do not have the power.’
An inspiring way of shifting narratives, looking at this problem from another angle, is the Protective Optimization Technologies, a framework proposed by Seda Gürses, Bogdan Kulynych, Rebekah Overdorf and Carmela Troncoso. Their research adresses that the focus on algorithms’ inputs and outputs misses harms that arise from systems interacting with the world. They emphasize interventions that can correct, shift, or expose harms that systems impose on populations and their environments.
'Algorithmic fairness aims to address the economic, moral, social, and political impact that digital systems have on populations through solutions that can be applied by service providers. Fairness frameworks do so, in part, by mapping these problems to a narrow definition and assuming the service providers can be trusted to deploy counter measures. Not surprisingly, these decisions limit fairness frameworks’ ability to capture a variety of harms caused by systems. […] We show that the focus on algorithms’ inputs and outputs misses harms that arise from systems interacting with the world; that the focus on bias and discrimination omits broader harms on populations and their environments; and that relying on service providers excludes scenarios where they are not cooperative or intentionally adversarial.’
Instead the researchers propose Protective Optimization Technologies. ‘POTs provide means for affected parties to address the negative impacts of systems in the environment, expanding avenues for political contestation. POTs intervene from outside the system, do not require service providers to cooperate, and can serve to correct, shift, or expose harms that systems impose on populations and their environments. We illustrate the potential and limitations of POTs in two case studies: countering road congestion caused by traffic-beating applications, and recalibrating credit scoring for loan applicants.'
Watch their presentation on The Conference on Fairness, Accountability, and Transparency in 2020
Technology in plural
Another interesting new initiative is The New Public a collective of designers, thinkers and technologists that considers digital technology human centered and plural. 'We believe that for spaces to be equitable, they must work actively to design for difference and remediate the effects of racism, economic domination and willful manipulation.
We believe that public spaces should not be frictionless. Friction and under-optimization lead to the serendipitous, incidental and generative human interactions where we encounter, discover and negotiate difference.
We believe that our best solutions already exist and are hiding in public — which is why finding common cause with other organizations and forging unlikely alliances is a centrally important task.'