At first glance, there isn’t much that connects Joseph Marie Jacquard, a 16th-century French weaver and merchant, a documentary about Anthony Bourdain, the late chef, writer and TV personality, and the revelations of whistleblower Frances Haugen on the extent to which Facebook is aware of and is causing serious social and political damage. But it was Jacquard’s success that brought Bourdain back to life. Coupled with what we now know about Facebook and the government’s encroachment on individual rights, this act of transcendent necromancy should frighten us all.
Haugen’s revelations highlight three fundamental points. First, those who run social media are not inherently evil-minded. But their moral ambivalence towards the consequences of their products and the agnosticism that is embedded in the design of the algorithms made it so that they could be. Take, for example, Instagram’s effect on teenage mental health, or the role WhatsApp and Facebook have played in promoting ethnic violence in places as diverse as Myanmar, parts of the country. Africa and India. Haugen has provided documents that show the company that runs the three apps was well aware of these consequences – and yet, they did little to stop them.
Second, there is no “right” way, no market-based solution that offers a plausible outcome. Apps are so closely tied to the way we live and work (just look at the crippling effect of a seven-hour shutdown of all Facebook apps on October 5) that a competitor is likely to fill the gap. space left vacant by a company.
Finally, it is naïve to believe in any substantial form of self-regulation. Simply put, the whole social media architecture is based on maximizing screen time and the data collected in this way. What the algorithm does is find what will keep people hooked the most, and for the longest time – the actual content doesn’t matter. Expecting social media giants to regulate the very thing their profits are based on is like asking drug dealers to prioritize drug rehab clinics.
If there is no longer self-regulation, is government regulation the solution? Unfortunately, the very actions of democratically elected governments often inspire little confidence. Take just two recent examples: the Pegasus spy scandal and the Arsenal Consulting findings. In either case, it seems clear that for many governments, including ours, the use of technology to violate individual rights is not ancillary to a larger purpose – as in the case of social media companies. – but an intrinsic part of their functioning.
Forget the fact that the Indian government appears to be the only national government that has not been shaken by the Pegasus scandal. What is more important is that governments can now deploy “zero click” spyware that can easily bypass security mechanisms. And that such capabilities have been deployed against journalists, friends and political opponents, defense personnel, businessmen – citizens with an inalienable right to privacy and dignity. As Subhasis Banerjee wrote (“Guardrails of privacy”, IE, July 27): “… what if an attempt at malware injection and surveillance as sophisticated as Pegasus bypassed the data protection architecture entirely? and regulatory oversight? … Pegasus was apparently also designed to self-destruct on detection attempts, although, according to Amnesty’s report, it was not entirely successful and left its mark. While the possibilities have always been theoretically understood, the fact that such James Bond-like tools actually exist and are used by governments is certainly a revelation. “
Sadly, the Pegasus scandal is just the tip of the iceberg.
In July 2021, a New York Times documentary, Roadrunner: A film about Anthony Bourdain, received almost universal critical acclaim. But director Morgan Neville’s revelation that the film contains snippets of dialogue in Bourdain’s voice, created using AI after his death, has left a lot of unease. In essence, it showed us just how close we are to resurrecting the dead: between voice cloning technology, the advanced robotics that’s about to create human-like androids, and the sheer amount of our personalities. which has been poured into sites and applications, it will soon be possible to create a sham of deceased relatives. But fears about how technological advancements can fundamentally change the human condition don’t just concern esoteric questions of life and death.
Imagine spyware as sophisticated as Pegasus that can evade detection, combined with the ability to create authentic voice and facial features. The dangers pointed out by the Arsenal Consulting revelations – which evidence has likely been lodged on the computers of academics, lawyers and activists in the Bhima Koregaon case – become all the more frightening. What if a forged video was used to jail activists? Or to institute scans of “anti-national”, “seditious” slogans?
Since governments have at least as much interest in retaining power as corporations have in making profits, they can hardly be expected to be impartial arbiters of the limits of technology.
Maybe there aren’t really any solutions to the challenges of the internet age. The apple has been eaten, and the sin is so systemic that there is nothing to be done. Or, perhaps, the answer to the power of software lies in the early battles against it.
In 1804, Joseph Marie Jacquard invented the Jacquard loom, which simplified and, to a certain extent, automated the weaving of complex fabrics. It was arguably the first software ever made, the ancestor of Facebook, Pegasus, and vocal recreation AI. While those in power at the time – profit makers and politicians – welcomed the loom, others saw it as an evil, like the cutting of jobs and agency. These opponents threw their clogs (“sabots” in French) on the infernal machine. From clogs were born the first saboteurs and acts of sabotage. But they were also acts of affirmation against a changing world – at least for them – for the worse.
Today, it is impossible to throw a shoe on the ephemeral global network. But perhaps, because of its ability to undermine democracy, agency, sanity, privacy, and individuality, we need saboteurs more than ever.
This column first appeared in the print edition on October 13, 2021 under the title “Why We Need Saboteurs”. [email protected]