Amazon Mitra-v2: Expanding Synthetic Pretraining Diversity to Replace Network Depth, Self-Reported Leading TabArena
The Amazon Science team has released the tabular foundation model Mitra-v2. Its architecture remains identical to its predecessor; performance gains stem from scaling up synthetic pretraining data (10x rows, ~3x columns) and a novel hybrid causal model prior. This prior combines neural networks, decision trees, Gaussian processes, and autoregressive mechanisms within a single causal graph, allowing one synthetic table to carry mixed inter-column relationships found in real tables. Self-reported results show it leads the single-model track on 51 datasets in TabArena and ranks first on the ScoringBench regression benchmark.
The Amazon Science team released the tabular foundation model Mitra-v2. Author Chris Kong provided a clear assessment in his blog post: the substantive improvement of this model over the previous generation, Mitra-v1, does not come from architectural changes—the Transformer still attends to both rows and columns simultaneously—but rather from two changes in the synthetic pretraining data: increased scale and diversity.
By 'increased scale,' we mean the number of rows in the synthetic tables used for pretraining is 10 times that of v1, with approximately three times as many columns, enabling the model to encounter tables comparable to actual business scales during training. By 'increased diversity,' we refer to the introduction of a new hybrid structural causal model (SCM) prior. While v1's synthetic tables were generated separately by classical SCMs and tree models, v2's hybrid SCM prior constructs a single causal graph where each edge relationship can be defined by a small neural network, a decision tree, a smooth function from a Gaussian process, or a time-dependent autoregressive process. This allows a single synthetic table to carry the mixed relationship types common in real tables. A new hybrid optimizer is employed to maintain training stability.
At deployment, two lightweight wrappers extend the model's boundaries: singular value decomposition is used to compress dimensions for extremely wide tables; for classification tasks exceeding 10 classes, predictions from each node in the output layer are fed into a hierarchical tree for further refinement. Notably, Mitra-v2 is not strictly a zero-shot model—it performs brief automatic fine-tuning on each new table by default, producing eight copies whose predictions are averaged (bagging). This incurs additional computational overhead but yields higher accuracy.