#107 X Marks the Spot: Analysing Deepfake Regulation in the Musk Era
Today, Rohan Pai looks at deepfake regulation in light of Donald Trump’s presidency and Elon Musk’s appointment as the head of the ‘Department of Government Efficiency’. Rijesh Panicker follows, with a piece on the importance of open source. Lastly, Lokendra Sharma explores what being a responsible actor means in the context of cyberspace in this week’s curated note.
Technology has become important not just in our everyday lives, but has also become an arena for contestation among major powers including India. The Takshashila Institution has designed the 'Technopolitik: A Technology Geopolitics Survey' to understand and assess what people think about how India should navigate high-tech geopolitics. We are sure you are going to love the questions! Please take this 5-minute survey at the following link: https://bit.ly/technopolitik_survey
Technopolitik: A Deep-dive into Deepfake Regulation
— Rohan Pai
Following close on the heels of Trump’s election victory, the soon-to-be head of the latest ‘Department of Government Efficiency’ made headlines last week for an entirely unrelated matter. Elon Musk, better known as the owner of the microblogging platform X, has filed a lawsuit in federal court suing the State of California for implementing a new law that concerns the regulation of deepfake content online. This law, formally known as AB 2655 or the Defending Democracy from Deepfake Deception Act of 2024, was drafted in response to the abundance of election misinformation that sprouted up during the last few months of campaigning. Although the legislation merely states that ‘materially deceptive’ content referring to elections must be labelled as such to prevent suspension, Musk’s newest lawsuit claims that it would violate freedom of speech.
Ever since its enactment in 1791, the First Amendment’s protection of free speech has captured the American imagination and continues to be something of a cultural touchstone in U.S. policymaking. Against such a backdrop, any attempts by government bodies at policing the exchange of thoughts and ideas, even if born out of noble intentions, are quick to ruffle feathers. The US election season, after all, has long been characterised by satirical commentary and ad hominem attacks between candidates. It was none other than Musk, after all, who reposted a doctored campaign ad which used an audio deepfake of Kamala Harris’ voice seeming to agree with Republican talking points. While the original creator of the ad, who goes by the name of ‘Mr Reagan’ had specified it as a parody, Musk failed to do the same upon reposting it thereby being complicit in the spread of misinformation.
This is by no means the first time Musk has come under fire for this reason. In fact, the years since Musk’s acquisition of Twitter has witnessed a mass exodus of important stakeholders from the platform, such as Apple, Disney and IBM, due to their advertising being displayed alongside a flurry of antisemitic and racist posts. Most recently, the CCHD or the Center for Countering Digital Hate, made their grand exit from X after months of accusations levied against Musk for allowing hate speech to propagate unhindered. Specifically, the non-profit took issue with X’s updated terms of service that required all further legal disputes to be heard only by the US District Court for the Northern District of Texas, thus creating a possible bias in favour of Musk. The allegations are further bolstered by Musk’s disbanding of the Trust and Safety Council in December 2022. This was an advisory group made up of human rights organisations whose sole function was making the platform a safer space devoid of issues like child exploitation, hate speech and incitement of violence.
At the end of the day, it must be questioned whether California’s hardline approach to deepfake misinformation is the most effective route. Beyond perhaps coming in the way of free speech, stricter rules also risk pigeonholing deepfake technology as an inherently malicious usage of artificial intelligence, which is far from the truth. Whether by breathing life into historical figures, enabling influential speakers to communicate in multiple languages, or even providing anonymity to journalists reporting from war-torn regions, deepfakes boast a multitude of benefits in the arena of social welfare. The law AB 2655 has been scheduled to take effect in California from January 1st, so it remains to be seen whether Musk’s lawsuit will present enough evidence to halt or at least delay it further.
Technomachy: At the Crossroads: Will Geopolitics reshape Open Source?
— Rijesh Panicker
Marc Andreessen’s 2011 prediction that “software is eating the world” has become a reality, with software now central to society and open-source[1] at its core. Today, 80-90% of all software uses open-source code, forming the foundation of our critical infrastructure from cloud computing to encryption. A 2021 EU report highlighted the economic impact, showing a EUR 1 billion investment in open source generated EUR 65-95 billion in economic value while emerging technologies like AI and IoT continue to build on open source foundations.
The open source landscape is now dominated by tech giants, with Microsoft, Google, and Red Hat leading contributions to Linux, while individual volunteers account for only 15% of Linux code. Major foundations like Linux Foundation and Apache Foundation oversee project maintenance and funding, while Microsoft's ownership of Github, the world's largest code repository, has raised concerns about the movement shifting from openness to corporate interests.
Despite the increasing interest from large tech firms in funding, contributing and owning open source assets, incidents like XZUtils and Log4Shell show the risk to large parts of our public digital infrastructure from attacks on or errors in poorly maintained yet critical open source projects. This has led to increased government scrutiny and regulatory moves by the U.S., EU and China. Since governments have very little say in how open source communities develop and govern their projects or in how private companies and individuals may choose to use these projects, state action has focused on cyber security and funding for maintenance.
Broadly, we see three approaches to the open source movement across governments. The U.S. has primarily approached it from a cybersecurity perspective by ensuring the integrity and sustainability of these projects. These include efforts to create a Software Bill of Materials (SBOM), which allows for better identification of dependencies in the supply chain of any project and efforts at improving funding for critical projects. Alongside this, there is an increasing perception that risk in open source can also come from malicious efforts by adversaries, which has led to direct interference with these projects. The most significant examples of these coming out of the U.S. include the decision to remove Russian developers from the Linux kernel maintainers list for compliance reasons, the banning of Tornado Cash from Github (temporarily)in 2002 over concerns that the cryptocurrency mixer was being used for money laundering by North Korea and the sanctions against Huawei since 2018, which locked it out of the Android ecosystem and has forced it to come up with alternatives.
The Chinese approach has been twofold. Since the early 2010s, China has increasingly participated in and contributed to open source projects, with some estimates showing the Chinese contributors to GitHub went up 4x from 2012-2018, while companies like Huawei, Alibaba and Tencent are major contributors to open source projects. This has also been the case in emerging fields like generative AI where China regularly publishes open-source models that are competitive with the best in the field, or in RISC-V, where China has created a China Alliance of research institutes and companies.
Alongside this, China has also increasingly looked to create its own national open-source ecosystem. Open Source has been identified as a national strategic priority in their 14th Five-Year Plan (2021-2025) to create open source communities with international influence by 2025. Gitee, the Chinese version of Github, is the de facto repository for Chinese open-source projects, receiving contributions from Chinese and global contributors. There are also ongoing attempts to replace Western versions of critical software, such as operating systems, with locally built ones, e.g. OpenKylin, as a replacement for Windows and HarmonyOS to replace Android (albeit due to U.S. sanctions).
The European approach looks to combine the ambitions of European digital sovereignty with the preservation of a digital commons. The EU approach seeks to maintain a decentralised structure of projects driven by EU member states to preserve autonomy from foreign laws. A central focus of this has been an effort to create a European Foundation for Digital Commons, which would organise financial support for projects, lead efforts to secure and audit open source components and promote innovation in the ecosystem. Some efforts are already underway in the EU around audits (FOSSA, FOSSEP) and funding (German Sovereign Fund and NGI - Next Generation Internet), which may be subsumed under the broader foundation.
Will Geopolitics eat Open Source?
Governments have understood the critical importance of open source and are increasingly treating it as a strategic resource. As their involvement in this area increases, we must wonder about the effects on the greater open-source ecosystem. On the one hand, we will likely see increased funding for critical projects and better risk detection and vulnerability auditing mechanisms such as the Software Bill of Materials (SBOM) to be developed. On the other hand, as geopolitical competition intensifies between the U.S. and its competitors like China and Russia, we should expect to see increased state interference in the functioning of open-source projects and the use of state power to regulate access to the underlying tools and components. This will ultimately lead to a shift from a broadly global, decentralised and collaborative open-source ecosystem to a more fragmented and nationalised set of ecosystems. Such a shift will likely lead to duplication of efforts and a decreased pace of innovation without any real reduction in the risk.
For countries like India that primarily consume open-source software, the European model of digital commons and sovereignty may offer the best path forward. While global convergence on security standards seems likely, maintaining a decentralised, autonomous ecosystem remains crucial.
[1] Open source code here refers to a way of developing software and does not refer to specific licensing methods. This differs from Free and Open Source (FOSS), which requires certain freedoms to be provided. All FOSS is open source but not all open source code is FOSS.
If you like the newsletter, you will love to read our in-depth research and analysis at https://takshashila.org.in/high-tech-geopolitics.
All things cyber: Reckless, responsible and everything in between
— Lokendra Sharma
If one were to closely follow Western media outlets and academic publications, China would often come out to be a reckless and irresponsible actor in cyberspace. While there is little doubt that China’s cyber capabilities are formidable, is it always the bad actor? Not according to Global Times, a Chinese Communist Party’s newspaper that deals with global developments. Last month, on 14 October, the Global Times published an exclusive story based on a report released by China's National Computer Virus Emergency Response Center. Turning the tables, the Global Times piece brought into focus the ‘irresponsible’ behaviour exhibited by the US in cyberspace, embodied by the latter’s ‘Defend Forward’ strategy. The story also goes on to claim that the US has developed a secret toolkit called ‘Marble’ that helps mount ‘false flag’ operations ‘to mislead investigators and defame China, Russia, North Korea, Iran, and Arab countries.’
Amid this clash of narratives, who is reckless and who is responsible when it comes to employing cyber weapons? The following 2018 book by David E Sanger may have some answers:
Sanger, D. E. (2018). The Perfect Weapon: War, Sabotage, and Fear in the Cyber Age. Crown.
Most of the information surrounding cyberattacks, cyberweapons and cyberwarfare is difficult to access for researchers and non-researchers alike owing to the secrecy states maintain around their cyber programmes and operations. While some journal articles do discuss some cyber developments and topics in great detail, they do so mostly relying on public information. But the book by Sanger explains and discusses all things cyber in much more detail with numerous examples from the US to Russia, from China to North Korea, and from Iran to Europe. This is due to Sanger's position as senior journalist at the New York Times, which gives him access to the national security elite in the US, Europe and elsewhere.
The main highlight of the book, in addition to the aforementioned point, is Sanger's argument that the US should first disclose its capabilities and actions in cyberspace to begin a process of norm-setting. Unless the US does so, it cannot expect other countries to fall in line and be responsible in the cyber domain. Sanger states: ‘Most important, just as the United States must show other nations there is a price to pay for truly serious cyberattacks, we must also show that some things are off-limits. And until America discusses publicly—at the presidential level—what we will not do in cyberspace, we have no hope of getting other countries to limit themselves as well.’
Therefore, if Sanger’s recommendation is to be followed, then before the US calls out states such as Russia, North Korea, China and Iran for mounting cyberattacks and reckless behaviour in cyberspace, the US should first consider disclosing its own actions in cyberspace.