Most ML pipelines are designed to consume data, not protect it. You get a raw CSV from the data team, load it into a training notebook, and figure out the compliance question later. That works until someone in legal reads your architecture diagram and asks where the patient IDs go.
The hard version of "no PII in training" isn't scrubbing identifiers before the model sees data. It's designing the pipeline so that real records never enter the compute environment at all. There is a meaningful architectural difference between those two things, and it affects what you can build, how fast, and with how much legal exposure.
This post walks through the architecture we settled on at Synthpylon after working through this problem ourselves. The core idea is a three-tier boundary: real data stays in the source environment, schema and statistical properties cross into an intermediate generation tier, and only fully synthetic output reaches the ML environment. No raw records cross any boundary at any point.
The Boundary Problem
When a data team moves records from a production database into a training environment, something legally significant happens: PII crosses an organizational boundary or a system-trust boundary. Even if both environments are inside the same cloud account, they typically have different access controls, different logging levels, and different data retention policies. A HIPAA compliance officer would call this "unnecessary data movement" and it creates a paper trail you may not want.
The boundary problem is also a practical engineering problem. Sensitive data moving between systems means you need network policies, encryption in transit, audit logging on both sides, and deletion procedures at end of life. Every one of those steps is a place where something goes wrong.
The cleaner architecture inverts the question: instead of asking "how do we move data safely," ask "what is the minimum information that needs to cross the boundary to produce useful synthetic data." The answer is the statistical structure of the data, not the records themselves.
Tier One: Schema and Statistics at the Source
The source environment is wherever your real data lives. This could be a production Postgres database, a warehouse like Redshift or BigQuery, or a HIPAA-scoped data lake. The key constraint: nothing leaves this environment except metadata and aggregate statistics.
What specifically crosses the boundary from Tier 1 to Tier 2:
- Column names, data types, and nullable constraints
- Foreign key relationships and cardinality
- Per-column marginal distributions (histogram buckets for numerics, frequency tables for categoricals)
- Pairwise correlation structure (a correlation matrix or equivalent)
- Business constraints that the data must satisfy (age must be positive, transaction amount must not exceed account balance, etc.)
None of these contain real individual records. A histogram over a "diagnosis_code" column tells you how often each code appears; it doesn't tell you which patient has which code. A correlation between "age" and "loan_amount" tells you the relationship direction and magnitude; it doesn't expose anyone's age or loan amount.
This is the extraction step we call schema ingestion. It runs inside the source environment, produces a JSON or Parquet artifact describing the statistical structure of the data, and that artifact is what gets passed to the generation tier. The raw table never leaves.
Tier Two: Synthetic Generation in the Middle
The generation tier receives the schema-and-statistics artifact and produces synthetic records. At Synthpylon, this is where we apply the generative modeling step: fitting a Gaussian copula or a conditional tabular GAN over the captured marginal distributions and correlation structure, then sampling from that model to produce rows that statistically resemble the original but share no records with it.
This tier can run in a separate compute environment with no network access to the source database. It needs only the artifact. The artifact itself is not sensitive: it describes statistical properties at a population level, not individuals.
A detail that matters in regulated industries: the generation tier can also be configured with a privacy amplification step. Differential privacy noise can be applied at the generation layer, providing a formal epsilon-delta privacy guarantee that even the aggregate statistics cannot be used to infer information about any individual record. For most ML training use cases, the noise budget is small enough that downstream model accuracy is not meaningfully affected. We're not saying differential privacy is always necessary; for many internal development use cases the statistical separation is sufficient. But the option is there when a compliance review requires it.
A practical example: one team using this architecture ingests a claims table with roughly 800,000 records, generates 1.2 million synthetic rows (intentionally oversampling to balance minority diagnosis codes), and passes those downstream. The generation step takes about four minutes. No claims records leave the source environment at any point.
Tier Three: Model Training on Clean Synthetic Output
The ML environment receives only the synthetic dataset. From the perspective of the training code, this looks like any other dataset. The data scientists working in this tier do not need access to the source environment. They do not need to know what real records look like. They can iterate on feature engineering, model architecture, and hyperparameters without ever touching regulated data.
This has a concrete benefit beyond compliance: it decouples access permissions from engineering velocity. The team that has database credentials to the HIPAA environment is often a small, carefully audited group. The team that needs to run training experiments can be broader. With a PII-free pipeline, those two teams can operate independently. The first team runs schema ingestion periodically (or on a schedule); the second team trains continuously on synthetic data that's already available.
The MLflow or Weights and Biases tracking that logs model runs doesn't capture any real records, because there aren't any in the environment. Your model lineage is clean.
What This Architecture Doesn't Solve
We should be direct about the limits here. This pipeline design prevents PII from entering your training environment. It doesn't prevent a poorly trained model from memorizing distributional information that could, in adversarial conditions, leak information about the source distribution. If you're in a high-stakes regulatory setting, model output should itself be reviewed for membership inference risk before deployment.
The architecture also assumes your real data is structured and tabular. Text data, images, and audio don't fit this tier-one extraction approach directly. Statistical properties of unstructured data are less cleanly separable from the content itself. That's a different problem requiring different techniques, and we will cover it separately.
There is also an organizational question: the schema ingestion step needs to run inside the source environment, which means someone in that environment needs to own it. In practice, this usually means a light Python or SQL script that a DBA or data engineer can review and schedule. It is simple code, but it requires buy-in from whoever controls the source.
Implementation Notes
A few things we've found matter in practice:
Snapshot frequency. If the source data evolves over time (new columns added, distributions shifting), the schema ingestion step needs to run again. A weekly scheduled job is usually enough for slowly evolving datasets. If you skip this and the synthetic distribution drifts from the source, you'll see it in the TSTR (train on synthetic, test on real) evaluation metric before it causes production problems.
Constraint documentation. The statistical structure captures what the data looks like but not necessarily what it means. A column called "status" with values 1, 2, 3 could be anything. If there are business rules (status 3 can only appear when another column is non-null, for example), those need to be explicitly passed as constraints to the generation step. Implicit knowledge in the heads of the data owners needs to be made explicit here.
Referential integrity across tables. If you're generating from a multi-table schema with foreign keys, the generation step needs to preserve those relationships. A synthetic "orders" table that references synthetic "customers" with correct key relationships. This is not automatic; it requires explicit modeling of the join structure during generation. Get this wrong and your downstream model will fail on joins before it even starts training.
The three-tier PII-free pipeline isn't a novel idea. The novelty is making the extraction, generation, and delivery steps fast and reliable enough to fit into normal engineering workflows. When schema ingestion runs as a scheduled job and synthetic generation is a five-minute API call, teams stop treating it as a compliance ceremony and start treating it as how they always work.