Skip to content
KS

Paper breakdown

The DCR Delusion: Measuring the Privacy Risk of Synthetic Data

One paper, read closely.

All paper breakdowns

Zexi Yao, Nataša Krčo, Georgi Ganev, Yves-Alexandre de Montjoye · 2026-10-18 · arXiv preprint · 2025

Must read30 min

The most popular privacy test for synthetic data passes datasets that membership inference attacks show are leaking.

Scores

  • Novelty3/5
  • Rigor4/5
  • Reproducibility3/5
  • Practical impact5/5

01Problem

Many teams share synthetic data instead of real data. To check it is safe, they often run one quick test called Distance to Closest Record (DCR). If the fake rows are no closer to the real training rows than some other real rows are, the data counts as private. This paper asks whether that test actually measures privacy, by putting it next to a real attack

02Method

The authors ran the same synthetic data through two checks and compared what each one said

One dataset, two verdicts
Real training data

Adult, Census, Bank, Berka

Generator

IndHist, Baynet, CTGAN, TabDDPM, ClavaDDPM

Synthetic dataset

same size as the training data

Proxy test

Holdout real rows
DCR, NNDR, IMS
Verdict: private

Attack

Membership inference
Verdict: leaking
Result

The proxy test called the data safe. The attack found people it could identify.

How the distance test decides
For each synthetic row, find the closest real training row
Measure the distance: numeric columns by Euclidean distance, categorical columns by counting mismatches
Do the same for holdout rows, which are real rows the generator never saw
Take the 5th percentile of both sets of distances
If the synthetic rows are at least as far away as the holdout rows, the data passes as private

NNDR is the nearest distance divided by the second nearest distance. IMS is the number of synthetic rows that are exact copies of real rows. The strict test requires passing all three.

Every column counts equally, so one rare value hardly changes the distance.

How the attack finds a person
Pick a target record (the 100 most vulnerable, ranked by the Achilles score)
Train 1,000 shadow generators on similar data, with the target included in half of them
Generate synthetic data from each shadow model
Count how many synthetic rows match the target on random groups of columns
Train a classifier to tell "trained with the target" from "trained without it"
Test it on 1,000 fresh models and get an AUC score for that one person

Scale: 0.5 = guessing | 0.6 = leaking | 0.8 = serious leak

For diffusion models the authors used a different attack (MIDST) that looks at the model's loss and reports true positives at zero false positives.

03Key results

ClaimResult
Classical generators, 9 setupsIn 7, every synthetic dataset passed. Attacks still hit AUC 0.8 or more on some records
Pass vs failAttacks worked just as well on datasets that passed as on those that failed
Average distance scoreNo link to how vulnerable a record was
Diffusion modelsEvery dataset passed the distance test. Attacks still identified members
Changing the thresholdAnywhere from 0 to 10%, every dataset still passed
All 1,000 records, not just outliers54 seriously exposed, 200 leaking
One person from Holland-Netherlands
Target record

the only person in the Adult dataset from Holland-Netherlands

CTGAN trained with them

92% of synthetic datasets contain that country

CTGAN trained without them

0% contain it

Attack

AUC 0.94, the highest of any record

Distance test

rows with that value are not even the closest to the target

Verdict

Passed as private

Anyone looking at the data would spot this in seconds. The distance test could not.

04What I'd question

It is still a preprint, so the results have not been through peer review yet. All datasets are public census and banking data, not health records, so I want to see the same test on clinical data. The main experiment attacks only the 100 most vulnerable looking records per setup, and the diffusion results rest on 10 models each. None of this changes the main point, but it shapes how far I would push the numbers

05How I'd use it

I used to think that if synthetic data looks different from the real data, it is safe. This paper showed me that looking different and being private are not the same thing. The distance test only catches near copies, while a real leak can hide in one rare detail, like a country only one person comes from. Adding a rule for rare values would not fix it either, because leaks can also come from combinations of ordinary values. The better question is not "how similar is this data" but "could an attacker find someone in it". This is the starting point of my MSc thesis on agentic privacy auditing of synthetic health data, where I test whether an LLM agent can reason like an attacker and catch leaks that simple metrics miss

06Reproduce it

I have not run this yet. The authors build on public tools: reprosyn for the classical generators and MIDSTModels for the diffusion models