Testing NVIDIA's Structured Data Models: Zero-Training Predictions Work, But Three Default Settings Fail Silently
MarkTechPost consulting intern Sana Hassan tested NVIDIA's Structured Data Models library on CPU, finding that its pretrained tabular models match scikit-learn trained baselines on benchmarks like breast cancer classification without any training. However, three default settings silently corrupt results: the default preprocessing recipe drops timestamp and text columns, the explainer reports only the last estimator, and the relational model reads future data to produce inflated scores when no time column is declared.
ImportanceLocalEvidenceE2 unreplicatedWrite-upQuick
Sana Hassan published a hands-on tutorial on MarkTechPost systematically testing NVIDIA's newly released Structured Data Models library. The library wraps several pretrained tabular foundation models behind a single PyTorch interface: the user provides labeled context rows and unlabeled query rows, and the model returns probabilities or a full predictive distribution in one forward pass, with no training step and no hyperparameters to tune. On the breast cancer dataset, using 300 labeled rows as context, both TabICLv2 and KumoTabular reached 0.970 accuracy with AUCs of 0.996 and 0.995 respectively, without any training, matching logistic regression and gradient boosting baselines fitted on the same 300 rows. On the diabetes regression task, the model's median prediction achieved an RMSE of 53.6, slightly better than RidgeCV at 54.3 and clearly better than gradient boosting at 64.7, while a single forward pass simultaneously returned 999 quantiles forming a full predictive distribution whose 80% and 90% intervals covered close to 81% and 91.5% of held-out targets, showing good calibration. These results explain why practitioners would want to reach for this library directly.
But Hassan's real message is not about the models themselves—it is about the defaults. He found three places where the library quietly does the wrong thing without raising an error. The first is the preprocessing recipe: TabICLv2's default recipe dispatches on semantic type with branches only for categorical and numerical columns, so timestamp and text columns never become features. On a support-ticket escalation prediction task—where the escalation signal comes from when tickets were opened and keywords in the ticket text—the default recipe scored an AUC of only 0.525, close to chance. The library does emit a RuntimeWarning naming the ignored column types, but it is a warn-once: it fires the first time and is suppressed for the rest of the process, so every later model call in the same notebook or service drops those columns in silence. Prepending AddCalendarFields to convert timestamps into cyclic hour and weekday features lifted the AUC to 0.634; adding TFIDF to extract terms from the comment text lifted it further to 0.789.