Schema matching · Language models · Data integration

Magneto: Combining Small and Large Language Models for Schema Matching

Magneto combines efficient retrieval with selective language-model reranking to improve schema-matching accuracy without paying the cost of applying a large model to every candidate.

The matching bottleneck

Schema matching requires finding semantically equivalent attributes across datasets. Large language models can reason about ambiguous candidates, but applying them exhaustively is expensive and slow.

A staged architecture

Flexible retrieval and reranking

The implementation supports several pretrained retrievers, including MPNet, RoBERTa, E5, Arctic, and MiniLM. An optional fine-tuning path generates synthetic matches and adapts the retriever to a target domain. The reranking stage can use GPT-4o mini or Llama 3.3 70B.

Reproducible benchmarks

The repository includes experiment drivers for the Genomic Data Commons and Valentine schema-matching benchmarks, ablation studies, and top-k retrieval metrics. A domain-adapted GDC retriever is also available through Hugging Face, allowing the published pipeline to be used without rebuilding the model locally.