Backend Engineer - Perm - US/ UK/ Western Europe
Description
Backend Engineer (APIs, Load Balancer, Scheduler) Full-Time · Remote or Hybrid · High-Impact Role
About Odyn
Odyn is building transformative AI solutions powered by cutting-edge, high-performance infrastructure. We're seeking a Backend Engineer to architect high-performance APIs, load balancers, and workload schedulers for our GPU infrastructure platform.
What You'll Do
Build Core Backend Systems & Load Balancing ● Architect RESTful and GraphQL APIs for model inference, fine-tuning, and resource management. ● Build API gateways handling authentication, rate limiting, routing, and multi-tenant isolation. ● Develop intelligent load balancers distributing inference requests across GPU clusters based on latency, cost, and availability with health checking and circuit breakers. ● Implement token streaming, batching, and request queuing for LLM inference workloads. Design Workload Schedulers & Resource Orchestration ● Design GPU resource allocation schedulers with bin-packing algorithms optimizing for topology (NVLink, PCIe, NUMA), workload type, and cost. ● Implement preemption, checkpointing, and graceful eviction for multi-tenant environments. ● Integrate with Kubernetes schedulers, custom operators, or standalone orchestration systems. Ensure Reliability, Performance & Collaboration ● Build observability systems (Prometheus, Grafana, OpenTelemetry) tracking API performance, load balancer health, and scheduler efficiency. ● Define and monitor SLOs for latency, throughput, and availability. ● Partner with infrastructure, ML, and product teams to optimize systems and shape developer- facing features. ● Participate in on-call rotation supporting production systems.
What We're Looking For
Must-Have
● 4–7+ years in backend/distributed systems or infrastructure engineering. ● Strong programming in Python, Go, or Rust; deep API development experience (REST/GraphQL). ● Proven distributed systems expertise: load balancing, service discovery, failover, microservices, message queues. ● Production cloud experience (AWS/GCP/Azure); Kubernetes and Docker proficiency. ● Strong networking fundamentals (TCP/IP, DNS, TLS, HTTP); SQL/NoSQL database skills.
Nice-to-Have
● GPU-accelerated workloads or HPC scheduling experience. ● LLM inference frameworks (vLLM, TensorRT-LLM) or Kubernetes scheduling internals. ● API gateways (Kong, NGINX) or service meshes (Istio); infrastructure-as-code (Terraform, Helm). ● Observability tools (Jaeger, Datadog); high-throughput systems (100k+ req/sec); AI infrastructure startup experience.
Why Join Us
● Work at the frontier of AI infrastructure building systems powering next-generation AI applications. ● Own critical components from the ground up. ● Collaborate with world-class teams. ● Competitive compensation + remote flexibility.
If you have experience building high-performance APIs, distributed schedulers, or load balancers for cloud infrastructure, we strongly encourage you to apply.