We are looking for a Senior Data Engineer to join our team. As a Senior Data Engineer, you will be responsible for designing, building, and maintaining our data pipeline, as well as optimizing our data infrastructure for performance, scalability, and reliability.
Responsibilities:
- Design, build, and maintain our data pipeline, ensuring high data quality and reliability.
- Develop and maintain data processing and ETL pipelines using tools such as Apache Spark, Kafka, and AWS Glue.
- Collaborate with data scientists and analysts to ensure that data is readily available and accessible for analysis and reporting.
- Optimize our data infrastructure for performance, scalability, and reliability.
- Work with cross-functional teams to identify and solve technical challenges related to data processing, storage, and access.
- Implement data security and compliance measures.
- Mentor and coach junior engineers in the team.
Requirements:
- Bachelor's degree or above in Computer Science or a related field.
- At least 5 years of experience as a Data Engineer.
- Strong experience with big data technologies such as Hadoop, Spark, and Hive.
- Strong experience with data processing and ETL tools such as Apache Beam, Apache NiFi, or Talend.
- Strong experience with cloud platforms such as AWS, GCP or Azure.
- Strong experience with SQL and NoSQL databases.
- Experience with data warehousing and data modeling.
- Experience with data visualization tools such as Tableau or Power BI.
- Excellent problem-solving skills and ability to work independently.
- Excellent communication and collaboration skills.