Associate Manager Data Engineering

Responsibilities

• Develop and maintain large-scale ETL and data pipelines.
• Design, implement, and support data warehouse and big data solutions.
• Build data ingestion, transformation, and loading processes for downstream systems.
• Monitor and resolve issues in production data pipelines.
• Lead data engineering projects from design through implementation and maintenance.
• Implement data governance and data quality best practices.
• Mentor data engineers and provide technical guidance.
• Collaborate with global teams and business stakeholders.

Requirements

• Experience with Hadoop, Spark, and Sqoop.
• Experience with Kafka and streaming frameworks.
• Experience building ETL pipelines using NiFi.
• Strong programming skills in Python, PySpark, Java, or Scala.
• Knowledge of HDFS, YARN, MapReduce, Hive, and Impala.
• Proficiency in Linux, Unix, Shell Scripting, and SQL.
• Experience designing scalable distributed systems.
• Ability to troubleshoot production and Hadoop/Hive issues.