Machine Learning Engineer — Distillation

About the Role

We’re looking for a Machine Learning Engineer focused on model distillation to help us build smaller, faster, and more efficient models without sacrificing quality. You’ll work at the intersection of research and production—taking cutting-edge techniques and turning them into systems that scale.

This is a hands-on role with real ownership: you’ll design distillation pipelines, run large-scale experiments, and ship models used in production.

What You’ll Do

Design and implement knowledge distillation pipelines (teacher–student, self-distillation, multi-teacher, etc.)

Distill large foundation models into smaller, faster, and cheaper models for inference

Run and analyze large-scale training experiments to evaluate quality, latency, and cost tradeoffs

Collaborate with research to translate new distillation ideas into production-ready code

Optimize training and inference performance (memory, throughput, latency)

Contribute to internal tooling, evaluation frameworks, and experiment tracking

(Optional) Contribute back to open-source models, tooling, or research

What We’re Looking For

Strong background in machine learning or deep learning

Hands-on experience with model distillation (LLMs or other neural networks)

Solid understanding of training dynamics, loss functions, and optimization

Experience with PyTorch (or JAX) and modern ML tooling

Comfort running experiments on multi-GPU or distributed setups

Ability to reason about model quality vs. performance tradeoffs

Pragmatic mindset: you care about shipping, not just papers

Nice to Have

Experience distilling LLMs or large sequence models

Experience with inference optimization (quantization, pruning, kernels, etc.)

Familiarity with evaluation for language models

Open-source contributions or research publications

Experience in early-stage or fast-moving startups

Why Join

Work on core model quality and cost efficiency—not side projects

High ownership and direct impact on product and roadmap

Small, senior team with strong research + engineering culture

Competitive compensation + meaningful equity

Remote-friendly, async-first environment

Originally posted on Himalayas