Class imbalance is one of the most common and most mishandled problems in tabular ML. A fraud detection dataset where 99.4% of transactions are legitimate. A medical diagnosis dataset where the condition of interest appears in 2% of records. A churn prediction dataset where churned customers are 7% of the total.
The standard response is oversampling: generate additional minority class examples so the training distribution is less skewed. SMOTE (Synthetic Minority Oversampling Technique) has been the go-to method for this since 2002. It works by finding nearest neighbors in feature space for each minority class sample and interpolating between them to create new synthetic points.
SMOTE is useful and worth knowing. It is also quite limited in ways that become apparent when you move from toy datasets to real production schemas. Understanding those limits is the first step toward choosing a better approach when the problem calls for it.
What SMOTE Actually Does (and Doesn't Do)
SMOTE operates in numeric feature space. It takes minority class samples, finds their k-nearest neighbors, and creates new samples by linear interpolation between a sample and a randomly selected neighbor. That's it. The result is new points that lie on line segments between existing minority class examples in the continuous feature space.
This approach has a few structural limitations. First, SMOTE works on numeric features. For categorical features, the interpolation idea doesn't apply directly. Various extensions (SMOTE-NC, SMOTENC) handle mixed categorical/numerical data, but they do so with heuristics that don't capture the conditional distribution of categorical values given numeric ones. If "account_type" takes different values for different income ranges in your source data, SMOTE-NC doesn't model that dependency.
Second, SMOTE doesn't enforce schema constraints. If your fraud cases always have transaction amounts above $500 (because small transactions trigger different fraud patterns), SMOTE may generate interpolated samples with transaction amounts below that threshold, creating synthetic fraud examples that don't resemble real fraud patterns. The generated points lie in feature space, not in the manifold of valid data your schema defines.
Third, SMOTE doesn't model the joint distribution of the minority class with the full feature set. It models the local geometry of the minority class in isolation. If the minority class has a specific relationship between two features that isn't captured by nearest-neighbor geometry, SMOTE will miss it.
The Probabilistic Synthesis Alternative
The alternative to SMOTE for oversampling is to fit a generative model specifically on the minority class and use that model to generate additional samples. Instead of interpolating between existing points, you're sampling from an estimated probability distribution over the minority class conditional distribution.
Concretely: fit a Gaussian copula or a CTGAN model on the minority class subset of your dataset. The model learns the marginal distributions of each feature within the minority class, the correlation structure between features, and (if you're using a conditional model) the dependencies between categorical and numeric features. Then sample from that model to produce as many additional minority class rows as you need.
This approach handles categorical features correctly because the generative model estimates the actual probability distribution over categorical values, conditioned on the numeric features. It can represent that fraud cases with transaction amount above $5000 have a different distribution of account_type than fraud cases with lower amounts. SMOTE cannot.
Schema constraints can be explicitly encoded into the generation step. A constraint that says "amount must be positive" or "account_tenure cannot exceed account_age" is applied as a hard filter on generated samples or incorporated into the model structure. The output satisfies your business rules by construction.
A Concrete Example: Medical Claim Escalations
Consider a claims management dataset at a growing health administration firm. The dataset has roughly 400,000 records with 18 features: claim type, diagnosis code category, payer type, time-to-submission, appeal flag, and others. The target variable is whether a claim required manual escalation. Escalations occur in about 3.8% of records.
A model trained on this dataset with SMOTE oversampling hits 0.71 AUC with 0.38 F1 on the minority class. The SMOTE-generated escalation samples look realistic in terms of individual feature ranges but have poor joint fidelity: the correlation between diagnosis code category and time-to-submission in the synthetic minority class doesn't match the real pattern (certain complex diagnoses consistently take longer to submit, and that pattern predicts escalation). SMOTE interpolates across that correlation without knowing it exists.
Replacing SMOTE with probabilistic synthesis on the minority class, preserving the joint distribution, brings F1 on the escalation class up to 0.46 with similar AUC. The improvement comes from the synthetic escalation samples having correct joint feature distributions, giving the model better signal about what a real escalation looks like.
We're not saying this result will generalize identically to every dataset. The improvement depends on how much of the minority class signal lives in joint feature interactions vs. individual feature values. But in tabular datasets with meaningful inter-column dependencies, which is most production tabular datasets, the joint fidelity difference matters.
When to Use Each Approach
SMOTE is a reasonable default when your minority class is small enough that fitting a generative model on it is statistically risky (under a few hundred samples), your features are primarily numeric with limited categorical interaction, and you don't have strong schema constraints that need to be satisfied.
Probabilistic synthesis is preferable when your schema has important categorical features or constraint rules, when you believe joint feature relationships within the minority class are important for model performance, when you have enough minority class samples to fit a generative model reliably (typically at least 500-1000 rows), or when your dataset has complex multi-table relationships with foreign key constraints that SMOTE doesn't model.
There's also a privacy argument for probabilistic synthesis when you're working with regulated data: SMOTE creates new samples by interpolating between real minority class records. Those interpolated samples are still close to the real records in feature space. A probabilistic synthesis approach, especially with differential privacy applied, provides cleaner separation from the source records. If the minority class contains sensitive records (fraud victims, patients with rare diagnoses), generating realistic but statistically independent synthetic samples is a stricter privacy posture than interpolating between real ones.
Validation: Don't Trust F1 Alone
Whichever oversampling approach you use, the validation approach matters. A common mistake is tuning F1 on the minority class while ignoring whether the oversampled training distribution has damaged the model's majority class performance. A model optimized purely for minority class recall can learn to classify everything as the minority class, producing acceptable F1 numbers while being useless in production.
The validation framework should include: precision and recall for both classes, AUC-ROC evaluated on a held-out real dataset (no synthetic samples in the test set), and a calibration check to make sure predicted probabilities are meaningful. If you generated synthetic minority class samples, those samples should never appear in the test set. The test set must be real data only, or you are evaluating against your own synthetic distribution, not the production distribution you care about.
Oversampling is a tool for changing what the training set looks like. It doesn't change what the production distribution looks like. The model will be evaluated on real production data, so that's what your test set must represent.