Embedded GenAI on Social Media: Platform Law Meets AI law
by Paddy Leerssen, postdoctoral researcher at the University of Amsterdam
6 August 2024
Social media platforms are integrating generative AI features into their services. This post discusses how these features trigger overlapping obligations under the AI Act and the Digital Services Act.
On 26 September 2024, Instagram and Facebook rolled out a new AI feature called Imagine. It allows users to generate images based on the text of their posts, integrated directly into the social media interface. Available in 22 countries, though not yet in Europe, it’s the latest and splashiest example of a broader trend: social media platforms are gradually integrating generative AI features into more and more of their services, first for advertisers and now end-users. Your personal Shrimp Jesus Generator is only one click away.
This embedding of generative AI into social media services is, in legal terms, quite a big deal. Our legal frameworks were designed to treat these services separately: for platforms, the Digital Services Act (DSA) and for Generative AI, the AI Act (AIA). Only last year this may have seemed like a workable distinction, but it is rapidly becoming obsolete. In this post I discuss some key implications for liability, risk management and content labeling.
Imagine™ the liabilities!
The social media business model hinges on one basic legal privilege: limited liability for user-generated content. Also known as the ‘safe harbour’, some variant of this rule is present in almost all major jurisdictions. In the EU, it was first enshrined in Article 14 of the e-Commerce Directive of 2000, and later reaffirmed in Article 6 of the Digital Services Act (DSA). This safe harbour rule ensures that platforms cannot be held liable for unlawful user content, unless they gain actual knowledge of its illegality. The safe harbour applies to ‘hosting providers’, which are defined as online services that consist in the storage of information provided by the user.
When Generative AI systems like ChatGPT and Midjourney produce content, their outputs are quite clearly ineligible for safe harbour protection since they are not ‘provided by the user’. (They might still act as hosting provider for user prompts and file uploads could technically still constitute hosting, however.) This fact alone doesn’t mean that GenAI services would necessarily be liable for their models’ outputs; that still depends on national law. In theory the proposed AI Liability Directive might also become relevant, but its current draft does not touch on GenAI specifically. But it’s clear that generative AI services don’t benefit from the same privileged position as platforms do for user-generated content.
What does this mean for embedded AI products like Meta’s Imagine With AI? The further that platform services involve themselves in the generation of user content, as opposed to its mere distribution, the more they risk venturing outside their safe harbour. Existing case law already indicates that platforms can lose safe harbour protection if their involvement is considered to go beyond ‘a merely technical and automatic processing of data’. For posts generated through embedded GenAI, is the content still ‘provided by the user’? Is this user-generated content, AI-generated content, or both?
There is some ambiguity here. On the other hand, the images produced through tools like Imagine are in a technical, literal sense not information provided by the user; it originates from the platform and its embedded generative AI models. On the other hand, its publication via the platform does still depend on a decision by the user, making the final decision whether or not to post the content. On balance it seems like the safe harbour could still apply in many cases, including most content produced through Meta’s Imagine feature, but outcomes may vary on a case-by case basis. It exceeds the scope of this short post to offer a full analysis. To offer just one example of the possible edge-cases, consider, a future where embedded AI tools are introduced in real-time streaming; here, the end-user may not have the same opportunity to control AI-generated outputs before their publication.
New risk management, same as the old risk management?
The DSA and AI Act also lay down two different risk management frameworks. The DSA requires large platforms with 45+ million users to periodically assess and mitigate systemic risks stemming from the use of their service, related to various policy objectives including combating illegal content, respecting fundamental rights and protecting (e.g.) civic discourse, election integrity, and public order public health. The AI Act’s concept of systemic risks, defined in Article 3(65) is similar in wording but not identical. Both frameworks are also enforced by different entities within the European Commission.
Where does platform-embedded AI land? The AIA makes clear that “large generative AI models” are typical examples of general-purpose AI: “given that they allow for flexible generation of content, such as in the form of text, audio, images or video, that can readily accommodate a wide range of distinctive tasks.” (Recital 99). These generative models are subject to risk management obligations if (1) the model has ‘high impact capabilities’, or if the Commission deems it to have equivalent capabilities, these qualify as ‘high risk’ systems subject to risk management obligations, or (2) if the cumulative amount of computation exceeds a specified numeric threshold (floating point operations greater than 10^25).
While this framework quite clearly covers large generative models like ChatGPT and Midjourney, the integrated offerings of Meta’s Imagine are a slightly harder cases. Since Imagine with AI can only be used to post on Instagram and Facebook, it is arguably not of the same ‘general purpose’ as ChatGPT or Midjourney. On the other hand, Instagram and Facebook are such large and popular services that the potential use cases for Imagine are still incredibly diverse and impactful.
Assuming that Imagine With AI is in fact captured by the AIA’s risk management framework, these obligations could still be largely overridden by the DSA’s. Recital 118 of the AIA creates a presumption of compliance for embedded models subject to the DSA’s risk management obligations:
“To the extent that such systems or models are embedded into designated very large online platforms or very large online search engines, they are subject to the risk-management framework provided for in Regulation (EU) 2022/2065. Consequently, the corresponding obligations of this Regulation should be presumed to be fulfilled, unless significant systemic risks not covered by Regulation (EU) 2022/2065 emerge and are identified in such models.”
So for embedded models like Meta’s Imagine, the AI Act’s risk framework is at most supplementary to the DSA’s. If Meta can demonstrate adequate risk management under its DSA obligations as a VLOP, they benefit from a presumption of compliance for their AI Act obligations. The onus will then be on the AI Office to demonstrate that risks not covered by the DSA have emerged and are identified in the model.
Given how similar Imagine’s functionalities are to genAI like MidJourney, or like Meta.AI’s own standalone service, this could lead to some curious double standards in practice, with very similar AI services being regulated through entirely different frameworks. When the first DSA risk assessments are (finally]) published later this year, it will be interesting to see whether Imagine has been addressed at all in this process. And if it isn’t, should that be a reason for the AI Office to drop their presumption of compliance? How much work does a VLOP need to show?
To make matters even more complicated, bear in mind that the DSA’s risk management framework only applies to Very Large Online Platforms and Search Engines (VLOPs and VLOSEs). So if you’re a small platform, any embedded AI application subject to risk management duties won’t benefit from the presumption of compliance. It looks like a win for the Very Large services, which might be meeting two risk management requirements for the price of one.
Content Labeling
The AI Act has some specific transparency rules for generative AI, which require providers to ensure that content generated by their AI model is recognizable as such (Article 50(2) AIA). What this looks like in practice is probably some form of label, watermark or disclaimer attached to the content. The design requirements are somewhat flexible, and take into account “the specificities and limitations of various types of content, the costs of implementation and the generally acknowledged state of the art” (50(2) AIA).
This labeling rule is not part of the risk management framework, so in principle VLOPs still need to comply with this obligation even for their embedded AI. However, there is an important carveout to keep in mind: the labeling requirement does not apply “to the extent the AI systems perform an assistive function for standard editing or do not substantially alter the input data provided by the deployer or the semantics thereof, or where authorised by law to detect, prevent, investigate or prosecute criminal offences”.
Again, embedded AI raises some difficult edge-cases. Between different features such as selfie filters, grammar tools and text or image generators, quite a range of outcomes might be possible. Generally speaking, it seems that embedded AI tools integrated into other platform services are more likely to satisfy the exception than standalone counterparts, since they are integrated into a more general purpose communications tool. For text-to-image tools like Imagine, however, which generate content in an entirely new format, it seems hard to deny that the information is in fact ‘substantially altered’. Whether it performs an ‘assistive function for standard editing’ is also doubtful.
It’s also interesting to note that Meta already has an AI labeling system in place for third party AI content. As shown in the screenshot below, their interface now discloses whether an image is known to be AI-generated. It’s not foolproof; detection relies on self-reporting by the uploader or metadata analysis, so savvy users can still game the system. The design has also been criticized for not being prominent enough and easily escaping notice, especially since a recent update made the label harder to find.
Image credit: Meta
This labeling of third party content by platforms is not regulated directly by the AI Act. Here Meta is implicated as a hosting provider and platform, not as model producer, bringing us back to the DSA. The DSA doesn’t have any specific rules about AI content labeling. However, might they become part of its risk management framework? It is not inconceivable that the risks posed by misleading AI content or otherwise unlawful or harmful AI content could justify such a reading. In some cases, one might even argue that non-labeled AI content constitutes illegal content, since the AI Act also contains obligations for model deployers to label AI-generated content as such (this obligation applies where the content is a ‘deep fake’ or when the content is “published with the purpose of informing the public on matters of public interest” (Article 50(4)). Whether it’s embedding its own models or not, the platform remains an important gatekeeper for the enforcement of generative AI rules.
Conclusion
This post has highlighted some curious interactions between the AI Act and DSA that occur when platforms embed generative AI into their own services. On liability, we see platforms potentially leaving their familiar safe harbours – though outcomes may still vary on a case-by-case basis. As for risk management, we might see platforms’ generative AI systems largely escaping the AIA framework as they would already be covered by the DSA’s framework, with entirely distinct norms and processes. And on content labeling, disclosure rules overlap in unclear ways: the AIA act can require platforms to label content from their own AI-generated tools, whilst the DSA can potentially also require them to label AI-generated content from third parties.
It’s still too early to tell whether platform-embedded AI will actually take off. Indeed, the entire AI industry is still on shaky ground; much points to it being an overfunded, overhyped, environmentally-ruinous bubble. But if platform-embedded AI is here to stay, it will pose a complex set of legal compliance and content moderation challenges for platforms and regulators alike.
Acknowledgements: The author is grateful to Michael Veale, Joris van Hoboken, João Quintais and Ljubiša Metikoš for their helpful input on earlier drafts of this text.