Europe Tightens Data Protection Rules for AI Models—And It’s a Big Deal for Healthcare and Life Sciences

Imagine you’re a researcher developing an AI model, such as an algorithm or other computer process, that can predict early-stage cancer using data from thousands of patient records. You’ve been careful—anonymizing datasets, following ethical guidelines, and ensuring patient confidentiality. But the release of the European Data Protection Board (EDPB)’s Opinion 28/2024 on Dec. 17, 2024, may have shifted the ground beneath your feet.

---

The EDPB is an independent body established under the European General Data Protection Regulation (GDPR) that ensures consistent application of data protection rules across the European Economic Area (EEA). That said, it’s crucial to understand what this Opinion does—and doesn’t—mean. EDPB opinions are not guidelines; they are narrowly focused on specific questions raised by one of its members. In this case, the Opinion on AI Models should not be interpreted as a general data protection framework for AI. Instead, it provides clarity on two key issues: how the concept of personal data applies to AI models, and whether legitimate interest can serve as a valid legal basis for AI processing.

This opinion marks a watershed moment for the intersection of AI and data protection, particularly in healthcare and bioscience innovation. While not legally binding, the Opinion signals that the GDPR’s broader scope will likely extend to AI applications. This development paves the way for stricter GDPR enforcement actions, shapes compliance expectations, and influences how AI systems handle personal data responsibly. European Data Protection Authorities (DPAs) will likely follow it, shaping how the GDPR is interpreted and driving future enforcement actions.

What Is Opinion 28/2024 and Why Should You Care?

The Opinion addresses key issues surrounding the processing of personal data in AI models,  answering four pressing questions:

1. When and how can AI models be considered anonymous?
2. Can legitimate interest justify AI data processing during development?
3. Can legitimate interest justify AI data processing during deployment?
4. What happens if personal data is unlawfully processed during AI development?

Anonymization means altering or removing personal data so that individuals can no longer be identified, even if the data is combined with other information. Legitimate interest, under the GDPR, allows organizations to process personal data without explicit consent if they can demonstrate that the processing is necessary for a legitimate purpose, does not override the fundamental rights and freedoms of the individual, and is proportionate to the intended outcome. In other words—for AI to rely on legitimate interest, organizations must prove the benefits outweigh the risks and that users’ rights are protected throughout the process.

Anonymization in AI: Why It’s Trickier Than You Think

In response to question 1, the EDPB’s Opinion raises a red flag: Can AI models trained on personal data ever be truly anonymous? The answer isn’t simple. Removing identifiers like names and IDs might seem enough, but it isn’t. Even when data is aggregated or masked, advanced attacks like model inversion can re-identify individuals. A 2023 study by Milad Nasr and colleagues drove this point home, showing how adversaries can extract training data from advanced AI models.

But anonymization isn’t a free pass. The EDPB emphasizes that DPAs should assess AI models case-by-case to ensure re-identification risks are “not reasonably likely.” Factors like training data, model use, and evolving technology impact this risk. The standard “not reasonably likely” suggests that some risk may be tolerated if minimal. The approach implies zero risk isn’t required, but rather that the likelihood of re-identification must be so low that it is not a realistic concern under current technological and contextual conditions.

To prove anonymity, organizations need solid proof by conducting Data Protection Impact Assessments (DPIAs), applying privacy-enhancing measures like differential privacy, and documenting safeguards. AI models are rarely anonymous by default, requiring ongoing testing and monitoring to ensure privacy and compliance.

Can Legitimate Interest Justify AI Data Processing? The EDPB Weighs In

Regarding questions 2 and 3, many AI developers have so far leaned heavily on legitimate interest as the legal basis for training models, given the challenges of obtaining consent. The EDPB outlines a three-step test to assess its validity, using examples like chatbots and AI tools that improve cybersecurity. These can benefit users, but organizations must prove processing is strictly necessary and respects rights.

Relying on legitimate interest to process personal data isn’t a golden ticket to bypass GDPR rules. For it to hold up, three cumulative conditions must be met: a) Identify a concrete legitimate interest, b) Assess if personal data is truly essential, and c) Balance the risks by prioritizing individual rights and freedoms over interests. If AI processing poses harm, mitigating measures like technical solutions, clearer user rights, or transparency can reduce its impact.

Unlawfully Processed Data: A Roadblock That Could Derail AI Models

Addressing question 4, one of the key aspects of the Opinion concerns the handling of data unlawfully processed during AI development. The EDPB considers various scenarios, emphasizing the need for case-by-case assessments. This could mean retraining models from scratch, deleting entire datasets, or halting deployments—potentially leading to significant costs and jeopardizing the entire project. Due diligence is crucial to ensure the model’s legality, and achieving effective anonymization before deployment can help safeguard future processing from being impacted by prior violations.

A Wake-Up Call for AI Models in Healthcare and Life Science

AI is revolutionizing healthcare and life science, offering new possibilities for diagnosis, treatment, and patient care. But with that power comes responsibility. The EDPB’s Opinion raises the bar, reinforcing that GDPR compliance isn’t a one-size-fits-all approach. Here are the three key takeaways you need to know:

• Anonymization isn’t a box to check—it’s an ongoing process. The EDPB adopts a narrow interpretation, meaning anonymity claims will face rigorous scrutiny. For AI models to comply, privacy must be baked in from the start. Organizations can’t just assume their AI models are secure. In genetic data or rare disease groups, this risk is even higher—where patient data often forms the backbone of AI training sets and unique patterns make re-identification easier. Anonymization means they need to test for vulnerabilities and prove their models can withstand real-world attacks.
• Legitimate interest isn’t a fast track to bypass the GDPR rules. Organizations must clearly define it and navigate the delicate balance between AI innovation and patients’ rights. The EDPB makes it clear—developing an AI model (training and refining the model) and deploying it (using the model in real-world applications) are two distinct phases, and each requires its own legal review. A legal basis for training does not guarantee the same for deployment.
• If an AI model is developed with unlawfully processed personal data, its deployment—and any downstream system built in—could also be unlawful. This poses a major risk to the entire AI value chain, as one non-compliant model can undermine entire ecosystems. Privacy-by-design is essential to prevent legal and operational risks in AI development.

---

About the author

Marcelo Corrales Compagnucci is an Inter-CeBIL Research Affiliate, and Associate Professor and Associate Director at the Center for Advanced Studies in Bioscience Innovation Law; Faculty of Law at the University of Copenhagen.