Data Engineer - IAG INSURTECH INNOVATION HUB PTE. LTD.｜Meet.jobs

About The Role

The Data Engineer role ensures that data pipelines are scalable, repeatable and secure. It helps facilitate getting data from a variety of different sources, getting it in the right formats, adhering to data quality standards, and assuring that downstream users can get data quickly.

Additional responsibilities include developing prototypes and proof of concepts for the selected solutions, and implementing complex data projects with a focus on collecting, parsing, managing, analysing and visualising large sets of data using multiple platforms. Data Engineers should also have the capability to architect highly scalable distributed systems, using open source tools.

The Data Engineer role is a technical job that requires substantial expertise in a broad range of software development and programming skills. These professionals have knowledge of data analysis, end user requirements analysis, and business requirements analysis to develop a clear understanding of the business needs and to incorporate these needs into technical solutions. They have a solid understanding of physical database design principles, and the software development life cycle. 

Preferred Skills and Expertise

· Experience in building and maintaining data pipelines on Hadoop, AWS, Google Cloud, or Azure.

· Experience with Spark, Kafka, Hadoop, Hive to process both batch and streaming data.

· Experience with schedulers or distributed processing tools such as Airflow, Nifi or Oozie.

· Proficiency in Python or Scala.  Being a polyglot is a big plus.

· Experience with Docker containers and container management services such as Kubernetes or Docker Swarm.

· Proficient understanding of code versioning tools such as git.

· Experience with AWS components are a huge plus.

· Experience in Agile methodology and TDD.

· Good understanding of data-structures and algorithms.

· Excellent written and verbal communication skills.

· Experience with NoSQL and RDBMS databases.

 

Business Development Javascript Java Communication Skills Python MySQL HTML5 & CSS3 Business Analysis

At IAG INSURTECH INNOVATION HUB PTE. LTD., we are innovative, passionate, vibrant, open and adaptable.

Benefits and perks of working with us include:

Family benefits: Paid maternity / paternity leave
Lifestyle: Flexible hours, Free food
Progression: Professional development
Welfare: Health insurance, Paid sick days