Senior Backend Engineer (Java, C++, Scala)

On behalf of our client, we are looking for a Senior Backend Engineer (Java, C++, Scala).

About the Project Apache Spark powers data infrastructure worldwide, but at petabyte scale, performance bottlenecks translate directly into massive infrastructure costs. The company is building a novel engine designed to dramatically accelerate Spark jobs at their core without requiring data teams to change how they work. The startup has raised $4.5M in seed funding to build a next-generation data infrastructure optimization engine for the AI era.

Responsibilities:

• Own SQL-acceleration projects end-to-end from planning to implementation and measurable outcomes
• Implement algorithms (targeting a custom DSL) for complex SQL functions within a Scala codebase
• Drive performance optimizations for Big Data processing algorithms primarily in Java but also C++
• Research and develop new greenfield development lying at the intersection of Big Data analytics and compilers
• Implement (in a custom DSL) a major SQL operator (e.g. Window aggregate) or smaller SQL expressions
• Migrate a complex code-generated algorithm in a custom DSL to a C++ equivalent
• Invoke Java code via C++ JNI to provide execution of a SQL function for compatibility reasons
• Implement a variant of repartitioning algorithm optimized for edge-case application configurations
• Refactor existing code generation algorithms to accommodate new improvements of a custom DSL
• Improve GitHub actions workflows to accommodate new build options or new comparative benchmarks

Requirements:

• 5+ years of experience with Java
• 3+ years of experience with C++
• 2+ years of experience with Scala
• Experience working with Big Data processing systems (Apache Spark or similar)
• Strong understanding of SQL processing and data processing algorithms
• Experience working with performance optimization and low-level system improvements
• Upper - Intermediate or higher level of English

Company offers:

• Long-term employment with possibilities for professional growth
• Fully remote work
• Reasonably flexible schedule
• 15 days of paid vacation
• Regular performance reviews