Member of Technical Staff - Distributed Training Engineer
Open 15 months. Roles open this long are often filled, or kept open all year. Check the date on the employer's site before you apply.
Not mentioned in the posting. Liquid AI filed 5 H-1B, H-1B1 or E-3 applications for AI roles in FY2025–FY2026, 2 for workers changing employer. Its US visa filings
In Liquid AI's visa applications for similar roles: $230K – $325K (the middle half of 5 base wages offered, FY2025–FY2026). Not this posting's range.
ABOUT LIQUID AI
Spun out of MIT CSAIL, we build general-purpose AI systems that run efficiently across deployment targets, from data center accelerators to on-device hardware, ensuring low latency, minimal memory usage, privacy, and reliability. We partner with enterprises across consumer electronics, automotive, life sciences, and financial services. We are scaling rapidly and need exceptional people to help us get there.
THE OPPORTUNITY
Our Training Infrastructure team is building the distributed systems that power our next-generation Liquid Foundation Models. As we scale, we need to design, implement, and optimize the infrastructure that enables large-scale training.
This is a high-ownership training systems role focused on runtime/performance/reliability (not a general platform/SRE role). You’ll work on a small team with fast feedback loops, building critical systems from the ground up rather than inheriting mature infrastructure.
WHAT WE'RE LOOKING FOR
We need someone who:
- Loves distributed systems complexity: Our team builds systems that keeps long training runs stable, debugs training failures across GPU clusters, and improves performance.
- Wants to build: We need builders who find satisfaction in robust, fast, reliable infrastructure.
- Thrives in ambiguity: Our systems support model architectures that are still evolving. We make decisions with incomplete information and iterate quickly.
- Aligns with team priorities and delivers: Our best engineers align with team priorities while pushing back with data when they see problems.
THE WORK
- Design and build core systems that make large training runs fast and reliable
- Build scalable distributed training infrastructure for GPU clusters
- Implement and tune parallelism/sharding strategies for evolving architectures
- Optimize distributed efficiency (topology-aware collectives, comm/compute overlap, straggler mitigation)
- Build data loading systems that eliminate I/O bottlenecks for multimodal datasets
- Develop checkpointing mechanisms balancing memory constraints with recovery needs
- Create monitoring, profiling, and debugging tools for training stability and performance
DESIRED EXPERIENCE
Must-have:
- Hands-on experience building distributed training infrastructure (PyTorch Distributed DDP/FSDP, DeepSpeed ZeRO, Megatron-LM TP/PP)
- Experience diagnosing performance bottlenecks and failure modes (profiling, NCCL/collectives issues, hangs, OOMs, stragglers)
- Understanding of hardware accelerators and networking topologies
- Experience optimizing data pipelines for ML workloads
Nice-to-have:
- MoE (Mixture of Experts) training experience
- Large-scale distributed training (100+ GPUs)
- Open-source contributions to training infrastructure projects
WHAT SUCCESS LOOKS LIKE (YEAR ONE)
- Training throughput has increased
- Overall training efficiency/cost has improved
- Training stability has improved (fewer failures, faster recovery)
- Data loading bottlenecks are eliminated for multimodal workloads
WHAT WE OFFER
- Greenfield challenges: Build systems from scratch for novel architectures. High ownership from day one.
- Compensation: Competitive base salary with equity in a unicorn-stage company
- Health: We pay 100% of medical, dental, and vision premiums for employees and dependents
- Financial: 401(k) matching up to 4% of base pay
- Time Off: Unlimited PTO plus company-wide Refill Days throughout the year
Attention Head lists this role as published on Liquid AI's own job board. Applications go to the employer.
Similar roles at other employers
- Sr. Staff Forward Deployed EngineerDatabricks · Remote - Washington D.C.; Washington, D.C.Nov 21, 2026
- Staff Applied AI EngineerBILL · United States$209K–250K
- Staff AI Agent Engineer – Moveworks | Partner SuccessServiceNow · New York, New York, United States$167K–291K
- Principal AI EngineerGallatin · El Segundo, CA; Washington, D.C.; Austin, TX; San Francisco, CA$240K–310K
- Member of Technical Staff, Core InfrastructureParallel Web Systems · San Francisco; New York, New York; Palo Alto$150K–300K
All staff and principal AI/ML Engineering roles in the United States →
More roles at Liquid AI
- Member of Technical Staff - GPU Inference EngineerSan FranciscoNov 19, 2026
- Member of Technical Staff - Product Engineer (Console/API)San Francisco; BostonNov 18, 2026
- Nov 11, 2026
- Member of Technical Staff - EvaluationsSan FranciscoNov 10, 2026
- Oct 30, 2026
- Oct 27, 2026
- Member of Technical Staff - Developer RelationsSan Francisco; New YorkOct 10, 2026
- Aug 26, 2026