Research Scientist
Pretraining, Posttraining & Agentic AI
I develop foundation models and agentic systems through large-scale training, posttraining and synthetic data, including contributions to H Company’s Holo and Holotron models.
I want to make foundation models more capable for the compute spent training them. This connects my work on efficient generative models, continual pretraining, synthetic data and distributed systems.
I’m particularly interested in compact models with strong agentic capabilities, and the training methods and data that make them possible.
Complete publication record on Google Scholar ↗An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models
A compressed latent diffusion architecture for efficient text-to-image generation.
Designed experiments, conducted ablation studies and co-wrote the paper. Presented the work as an ICLR oral.
Simple and Scalable Strategies to Continually Pre-train Large Language Models
Matched a retraining-from-scratch baseline at 10B parameters with less compute, using learning-rate rewarming, redecay and data replay.
Engineered the distributed training setup for ablations across model scales. Adapted PyTorch, DeepSpeed and Megatron forks to Summit’s POWER9 CPUs and six-GPU nodes, enabling runs on up to 13,824 GPUs.
Open web-scale pretraining data
A 422B-token pretraining corpus built from permissive-first text sources, including instruction and reasoning data.
Designed and implemented distributed data-processing pipelines for the corpus.
An Open Dataset for Training Multimodal Models on Document and Code Tasks
An open multimodal dataset for document understanding and code tasks.
Designed and implemented distributed data-processing pipelines for the dataset.
For Holo1–3, I built synthetic-data generation and desktop/web-agent evaluation pipelines, and ran supervised fine-tuning (SFT). My broader work spans Holo and Holotron model training and releases.
Ranked #1 at the time of evaluation. 35B total / 3B active parameters.
Official OSWorld leaderboard ↗NVIDIA collaborationCollaborated on official Nemotron releases at H.
Updates on LinkedInHolo1 · Holo2 · Holo3
Holo3.1 · Holo4
Computer-use models
Surfer-H · Surfer 2
From leading industry teams to academic research and foundation-model releases.
Research Scientist · Core Team
Built synthetic-data and agent-evaluation pipelines and ran SFT for Holo1–3. Trained models for Holo releases on 1,024 H100 GPUs and collaborated with NVIDIA on official Nemotron releases.
Senior Research Scientist
Foundation-model safety evaluation and automated red teaming in the Trust and Governance Lab.
Postdoctoral Researcher · DAAD IFI Fellow
Designed experiments and ablation studies for Würstchen. Engineered distributed training for continual-pretraining studies on Summit and Frontier.
Research Associate
Neural architecture design and computer vision, including medical imaging.
Data Scientist → Head of Data Science
Led a team of 10 data scientists and software engineers on applied machine-learning projects.
Teams, infrastructure and research mentorship.
Elected representative for Université de Montréal postdoctoral researchers; served on Mila’s Leadership Committee.
Contributed benchmarks and code for future deep-learning clusters through Mila’s IDT Committee.
Created Advanced Topics in Deep Learning at Osnabrück and mentored Mila master’s interns. Primary supervisor for four theses; second supervisor for one.
I welcome conversations about training efficiency, synthetic data and capable agentic models.
matsleonrichter@gmail.com Based in Berlin, Germany