Senior Data Engineer

We are seeking a Senior Data Engineer to design, implement, and optimize data pipelines utilizing Scala, Spark, and Java. The ideal candidate will develop and maintain real-time data processing systems essential for business operations. Collaboration with data scientists and analysts is crucial to understand data requirements and deliver high-quality solutions. Responsibilities include ensuring data quality through robust testing, monitoring workflows, and troubleshooting pipelines. Candidates should possess a degree in Computer Science or Engineering, with proven experience in data engineering, real-time processing, and SQL proficiency. Familiarity with cloud platforms and data governance is preferred. We offer a competitive salary, benefits, and opportunities for professional growth in a collaborative environment.

Key Responsibilities:

• Design, implement, and optimize data pipelines using Scala, Spark, and Java.
• Develop and maintain real-time data processing systems to support business-critical operations.
• Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver high-quality solutions.
• Ensure data quality and reliability through robust testing and validation procedures.
• Monitor and troubleshoot data pipelines and workflows to ensure high availability and performance.
• Stay current with emerging technologies and industry best practices to continuously improve our data infrastructure.

Qualifications:

-Bachelor’s or Master’s degree in Computer Science, Engineering, or a related field.

• Proven experience with Scala, Spark, and Java in a data engineering or similar role.
• Strong understanding of real-time data processing and streaming technologies.
• Experience with big data platforms and tools such as Hadoop, Kafka, and Flink is a plus.
• Proficiency in SQL and experience with relational databases.
• Excellent problem-solving skills and attention to detail.
• Strong communication and collaboration skills to work effectively with cross-functional teams.

Preferred Skills:

• Experience with cloud platforms (AWS, Azure, Google Cloud) and their data services.
• Knowledge of data warehousing solutions and ETL processes.
• Familiarity with data governance and security best practices.

Budget: USD 25/hour

Proposals: 13 freelancers have applied