Scraping Web Data for AI: EDPB 03/2026 GDPR Rules for SMEs

Under EDPB Guidelines 03/2026, European companies cannot treat publicly accessible web data as free fuel for generative AI models. Ingesting online personal data for training, fine-tuning, or retrieval-augmented generation (RAG) requires a documented three-part Legitimate Interest Assessment under GDPR Article 6(1)(f), strict pre-ingestion filtering of Article 9 sensitive data, and respect for machine-readable opt-outs.

Many engineering teams assume public web data is fair game. They point scrapers at forums, business directories, and social platforms, extract millions of records, and feed them into vector databases. European data protection authorities have dismantled this assumption. The European Data Protection Board adopted Guidelines 03/2026 to define how the GDPR applies whenever private companies harvest web data for generative AI.

Here is what European founders, CTOs, and compliance teams must document to keep their AI training pipelines defensible.

Who is actually in scope?

Guidelines 03/2026 apply to any commercial organization that harvests personal data from public online sources to train, fine-tune, or evaluate generative AI models.

The guidance establishes that controllership attaches across three common engineering setups:

• In-house web crawlers: Running proprietary spiders or scripts against public websites.
• Commissioned collection: Hiring external scraping agencies or contractors to assemble datasets.
• Purchased web corpora: Buying pre-scraped commercial datasets or downloading open-web snapshots.

Buying an off-the-shelf dataset does not shield you from liability. Under GDPR Article 4(7), your company acts as a data controller for every personal data point ingested into your pipeline or model weights. If a third-party vendor scraped personal data unlawfully, training your model on that corpus inherits the compliance defect.

What documentation is required for legitimate interest?

Obtaining direct consent from millions of internet users is technically impossible for open-web harvesting. Most European AI builders rely on Article 6(1)(f) (legitimate interests).

Informal assertions will not satisfy European DPAs. Regulators demand an upfront, written Legitimate Interest Assessment (LIA) covering three cumulative tests before scrapers launch:

1. The Purpose Test: You must articulate a real, lawful, and sufficiently specific commercial or research objective. Vague claims like "improving AI capabilities" will fail regulatory scrutiny.
2. The Necessity Test: You must demonstrate that your AI system cannot achieve its objective using synthetic data, publicly licensed content, or narrower non-personal datasets. If your model can work with anonymized benchmarks, scraping personal profiles fails the necessity hurdle.
3. The Balancing Test: You must prove that your commercial interests do not override the fundamental rights, freedoms, and reasonable expectations of affected web users.

Every operational data harvest must be logged in your internal GDPR ROPA Guide. If your AI pipeline processes personal data on a large scale or evaluates individuals systematically, you must also complete a formal DPIA Step-by-Step Guide under Article 35 before scraping begins.

How must SMEs handle Article 9 and opt-out signals?

Public websites contain sensitive personal data. Discussion boards, comment sections, and social channels routinely hold information revealing political opinions, religious beliefs, sexual orientation, or health conditions.

Under GDPR Article 9, processing special category data is strictly prohibited without explicit consent or narrow statutory exemptions. The EDPB clarified that posting personal views online does not mean an individual has "manifestly made public" their sensitive data under Article 9(2)(e) for commercial AI ingestion.

To keep pipelines lawful, SME engineering teams must deploy automated pre-ingestion safeguards:

• Automated content filters: Implement regex and NLP classification pipelines to detect and purge Article 9 attributes before text reaches embedding models.
• Exclusion lists: Blacklist patient support forums, political discussion boards, and sensitive review sites from crawler targets.
• Machine-readable opt-outs: Configure web scrapers to respect robots.txt disallow rules and HTTP X-Robots-Tag noindex/noarchive headers.
• Article 21 objection workflows: Establish a public, accessible mechanism where individuals can object to their data being used in training iterations.

What are the penalties for non-compliant AI scraping?

European regulators are moving aggressively against unauthorized AI data ingestion. Supervisory authorities across France (CNIL), Italy (Garante), Spain (AEPD), and Ireland (DPC) have already investigated leading AI model providers and imposed significant sanctions.

Under GDPR Article 83(5), violations of core processing principles, lack of lawful basis, or mishandling of Article 9 data trigger administrative fines of up to €20,000,000 or 4% of total worldwide annual turnover.

Beyond monetary fines, DPAs possess corrective powers under Article 58(2). Regulators can issue temporary or permanent processing bans, order the immediate deletion of harvested datasets, or demand algorithmic disgorgement—forcing companies to destroy trained model weights derived from unlawful data.

If your AI application is integrated into HR tech, biometric systems, or credit scoring, exposure escalates under the EU AI Act Guide. Deploying unverified training data into systems evaluated by the High-Risk AI Classifier can trigger concurrent investigations from both data protection authorities and national AI market surveillance bodies.

What should SMEs do first?

European tech companies should take five immediate steps to audit their AI data intake:

1. Map existing training inputs: Trace every dataset, scraped text corpus, and third-party benchmark currently stored in your vector database or fine-tuning pipelines.
2. Draft a written LIA: Document the three-part legitimate interest test for each active AI training or RAG workflow.
3. Audit crawler configurations: Verify that your web harvesting infrastructure strictly checks and obeys robots.txt directives on target domains.
4. Deploy Article 9 scrubbing: Build automated validation scripts to verify that sensitive attributes are stripped before embeddings are generated.
5. Evaluate regulatory overlap: Use the Regulation Checker to verify whether your AI workloads trigger secondary obligations under the AI Act, NIS2, or sectoral EU laws.

Disclaimer: EuroComply provides compliance readiness tools and evidence drafts. This guide is for informational purposes and does not constitute legal, regulatory, or compliance advice. Have technical documentation and assessments reviewed by qualified counsel.