Even if democracy manages to adapt and survive, people might become the victims of new kinds of oppression and discrimination. Already today more and more banks, corporations and institutions are using algorithms to analyse data and make decisions about us. When you apply to your bank for a loan, it is likely that your application is processed by an algorithm rather than by a human. The algorithm analyses lots of data about you and statistics about millions of other people, and decides whether you are reliable enough to give you a loan. Often, the algorithm does a better job than a human banker. But the problem is that if the algorithm discriminates against some people unjustly, it is difficult to know that. If the bank refuses to give you a loan, and you ask 'Why?', the bank replies, 'The algorithm said no.' You ask, 'Why did the algorithm say no? What's wrong with me?', and the bank replies, 'We don't know. No human understands this algorithm, because it is based on advanced machine learning. But we trust our algorithm, so we won't give you a loan.'
When discrimination is directed against entire groups, such as women or black people, these groups can organise and protest against their collective discrimination. But now an algorithm might discriminate against you personally, and you have no idea why. Maybe the algorithm found something in your DNA, your personal history or your Facebook account that it does not like. The algorithm discriminates against you not because you are a woman, or an African American — but because you are you. There is something specific about you that the algorithm does not like. You don't know what it is, and even if you knew, you cannot organise with other people to protest, because there are no other people suffering the exact same prejudice. It is just you. Instead of just collective discrimination, in the twenty-first century we might face a growing problem of individual discrimination.
