6.1.1. The notion of ‘deep fake’
(113) Article 3(60) AI Act defines ‘deep fakes’ as ‘AI-generated or manipulated image, audio or video content that resembles existing persons, objects, places, entities or events and would falsely appear to a person to be authentic or truthful’. The difference between manipulation and generation has been clarified in Section 4.1 above. For the purpose of clarifying the notion of deep fake, four cumulative criteria from that definition are examined below: (i) resemblance of (ii) existing (iii) persons, objects, places, entities or events (iv) that would falsely appear to a person to be authentic or truthful.
i. Resemblance: Article 3(60) AI Act specifies that AI-generated or manipulated image, audio or video content must ‘resemble’ existing subjects to constitute a deep fake. Recital 134 AI Act adds that such resemblance should be ‘appreciable’. AI-generated or manipulated image, audio or video content appreciably resembles a subject if there is a high level of similarity between the deep fake content and the subject (including any of its recognisable elements) being simulated by the deep fake. The content does not need to be identical to the subject. Whether the level of resemblance is appreciable is a case-by-case assessment to be done by the deployer based on an objective comparison between the simulated subject and the deep fake content, based on, among others, the extent to which characteristic or distinctive features are represented by the deep fake.
ii. Existing: Article 3(60) AI Act specifies that AI-generated or manipulated image, audio or video content must resemble ‘existing’ subject(s) to constitute a deep fake. To effectively fulfil the purpose of Article 50(4) AI Act to reduce the risks of impersonation, deception, misinformation, manipulation and fraud, the characteristic
of existence implies that the AI-generated or manipulated image, audio or video content should resemble realistic subjects (e.g., persons, objects, places). Therefore, it is sufficient for simulated persons, objects, places, entities or events to resemble someone or something that exists, can plausibly exist or could have plausibly existed in reality to be considered a deep fake, since such content can be a major source of deception, manipulation and disinformation. By contrast, simulated persons, objects, places, entities or events that, for example, defy the laws of nature or physics or depict lifeforms that are not commonly accepted in biology (such as e.g. humans flying without mechanical aids, dragons, or elephants driving cars) and have no potential to mislead are considered unrealistic and therefore fall outside the scope of the transparency obligation.
iii. Persons, objects, places, entities or events: Article 3(60) AI Act specifies that AI- generated or manipulated image, audio or video content should resemble existing ‘persons, objects, places, entities or events’ to constitute a deep fake. ‘Persons’ is to be understood as realistic, human beings (including digital replicas of real persons, realistic AI-generated human avatars or personas, and personal characteristics or expressions, such as image, voice, behaviour, performances etc.). ‘Objects’ is to be understood as realistic, inanimate material items, including buildings, artworks, machinery, consumer goods etc. ‘Places’ is to be understood as realistic locations. ‘Entities’ is to be understood as realistic, non-human but animate beings including animals or other biological lifeforms. ‘Events’ is to be understood as realistic scenes or situations that can involve persons, objects, places and entities (e.g. evoking historical events or the depiction of professional or consumer services).
iv. False appearance to a person to be authentic or truthful: Fourth, Article 3(60) AI Act specifies that the AI-generated or manipulated image, audio or video content resembling existing persons, objects, places, entities or events ’would falsely appear to a person to be authentic or truthful’. This criterion relates to the essential characteristic of deep fake content and its capacity to potentially deceive or mislead a person regarding the content’s authenticity or truthfulness (32).Content authenticity refers to whether the content is genuinely what it purports to be in terms of its source or creation process (including e.g. the involvement of real human beings or animals and their actual behaviour or actions, the actual appearance or use of objects or the delivery of services, the accurate unfolding or course of an event). Truthfulness pertains to the veracity of the content (e.g. factual correctness of the representations in the deep fake).
(114) Whether content ‘falsely appears to a person to be authentic or truthful’ should be assessed as a whole, taking into account the level of resemblance, the potential substantive message of the content, the intended and foreseeable deployment contexts, the environment in which the content is presented, and the intended and reasonably foreseeable audience composition and their expectations. However, this assessment is objective and does not require the intention of the deployer to deceive or mislead the natural persons exposed to the content for it to constitute a deepfake. As regards the deployment context and audience expectations, this means that if the content is used in specific deployment contexts where the intended and reasonably foreseeable audience does not expect content to be authentic or truthful, the AI-generated or manipulated
(32) Therefore, this criterion must be understood independently and distinctly from the concept of the deception as mentioned by Directive 2005/29/EC concerning unfair business-to-consumer commercial practices in the internal market (Unfair Commercial Practices Directive), https://eur-lex.europa.eu/eli/dir/2005/29/oj/eng
content may not falsely appear to be authentic or truthful even if it is, in fact, non- authentic or untruthful. In such cases, the fourth criterion of the deep fake definition will not be fulfilled. For example, the AI-generation or manipulation of background scenes, special effects, or technical pre- and post-processing as part of standard movie production processes are not likely to make content falsely appear to the audience to be authentic or truthful. Conversely, the use of AI for generating or manipulating essential elements that impact audience perception is likely to do so (such as fully AI-generated actors, digital replicas of real or deceased actors, de-aging, simulated performances including parts thereof, or non-authentic or untruthful representation of persons, objects, places or events in documentaries). For image and video content, it is acknowledged that a high degree of photorealism renders it more likely that such content should be considered a deep fake (as it likely resembles existing subjects), but photorealism alone is not determinative for the assessment. What should still be assessed, is whether such content is capable of deceiving or misleading a person regarding the content’s authenticity or truthfulness while taking into account, among others, the intended and foreseeable deployment context.
(115) As opposed to Article 50(1) AI Act, the assessment of false appearance to a person to be authentic or truthful should not be based on a hypothetical “average” person expected to be exposed to the content (see Section 3.2.1), but should duly take into account the possible diverse composition of the reasonably foreseeable audience that may be exposed to the deep fake content. Due consideration of audience composition is especially important if it is reasonably foreseeable that the deep fake content may, at some point, be perceived by children, the elderly, or other groups of persons with lower digital and AI literacy or general knowledge levels, since they may be more easily deceived or misled regarding the content’s authenticity or truthfulness. In such cases, false appearance of the content to be authentic or truthful to that part of the foreseeable audience may suffice to qualify content as a deep fake. However, deployers are not expected to consider potential further dissemination of the deep fake content by third parties beyond the reasonably foreseeable audience by the deployer, taking into account the intended distribution channels (e.g. if deep fake content is shown solely on a subscriber-only part of a website or as part of a corporate newsletter, this does not imply that the deployer should consider broad public accessibility by default).
(116) In a similar vein, AI-supported manipulation of insignificant substantive or technical aspects of pre-existing content may be of minor relevance for a person’s assessment of the authenticity or truthfulness of the content, not rendering the resulting content to become a deep fake. This could include, for example, editing background details (e.g. removing passerby), lighting adjustments, adapting audio parameters, colour correction, noise reduction or removal, improving accessibility or file compression, cosmetic adjustments and enhancements. Whether the manipulation will fall into this category depends on the context and the impact on the persons’ perception of the content’s authenticity or truthfulness in the specific case. For example, AI-powered colour correction, background extensions of existing content, adjustments or replacements of backgrounds for clearly aesthetic purposes, compositions and arrangements of existing products, or re-scaling of images applied in product advertisements or packaging is likely to have only a minor impact on a person’s perception of the authenticity and truthfulness of the advertisement and the product. Conversely, substantial AI-powered editing of background details of journalistic images beyond standard technical, editorial practices may negatively affect a person’s perception of the content’s authenticity and truthfulness.
Examples of deep fakes under Article 3(60) AI Act
- AI-manipulated image of a scene involving two real professional footballers in front of a building resembling a football stadium.
- AI-generated audio involving voice cloning of a newspaper podcast’s regular presenters and a guest interlocutor discussing some news events. -AI-generated video of an individual resembling a politician, holding a speech in front of an audience.
- AI-generated video featuring an AI-generated depiction of a celebrity influencer in an advertising or promotional context.
- AI-generated video featuring a realistic synthetic avatar of a company CEO congratulating employees with their work and the corporate results of the past year.
- An AI-generated image of a product in advertisement or packaging that can affect the audience’s perception and mislead as to the actual product appearance, characteristics or use (e.g. making the product appear not identical to the real product, more appealing or with improved quality than in real life). Examples that do not constitute deep fakes under Article 3(60) AI Act
- AI-generated image of a scene involving a sphinx flying over the Eiffel Tower.
- AI-generated video of mice arguing in human language over the best type of cheese as part of an advertisement campaign for a cheese manufacturer.
- AI voice replication for fictional characters (e.g., for audiobooks, games or animation) when there is no deception as to the identity of the narrators.
- AI-manipulated radio broadcast adjusting technical audio parameters (e.g. normalising volume levels, noise reduction, audio compression) without altering the actual words spoken by speakers or their way of speaking.
- AI-generated cartoon of a pre-existing image depicting a historical event.
- Video of a presenter of a scientific TV-programme who walks across a mountainous region while addressing the retreat of glaciers which is illustrated by an AI-manipulated background animation that shows how the glaciers retreated from their historical levels to their current levels.
- AI-generated fictitious environments (e.g. fictitious forests and castles) in video games.
- Movies where real existing actors (not manipulated by AI) are playing against an AI-generated background (e.g. depicting a fictitious stretch of prairie, historical buildings of ancient cities etc.).
- A real product (e.g., a car) shown in an advertisement against an AI-generated background and surrounding environment as long as the ad is not likely to mislead the audience about the product’s actual representation and its characteristics and use.
Link a questa unità: https://fwks.it/news/normativa/testi/linee-guida-art50?u=g-6-1-1