Data Engineer - Shopee Singapore Pte Ltd｜Meet.jobs

Job Description:

• Design and grow the Shopee data platform to support a variety of big data applications using open-source technologies including Kafka, Hadoop, Presto, HBase, Spark, Hive, Druid, and our own creations. Some examples include a real-time data streaming platform, a unified query platform, a cluster management system, and a machine learning platform
• Design and grow Shopee’s data warehouse, build reliable and smart ways to ingest data to the warehouse, and help engineer efficient data pipelines. Some examples include self-service data ingestion systems, Airflow-enabled workflows with code-as-configuration, and data validation tools
• Build critical data marts and applications to support products and solve specific business needs, design data models for optimal storage and retrieval, and optimize data architectures to meet critical product and business requirements. Some examples include a real-time Campaign Mart (used to serve the business with live intelligence) and Order Mart (to supply low latency seller performance system)

Requirements:

• B. Sci. / Ms / PhD in Computer Science or a related technical field
• 2+ years of working experience in software development in at least one of these languages: Java, Scala, Python, C/C++, under Linux / Unix. Scala is a plus
• Familiar with SQL; strong scripting ability in Bash is a plus
• Familiar with Hadoop, Spark, Kafka, Presto, and other big data experience is a plus
• Familiar designing and operating of a robust distributed system is a plus
• Love to use and develop open-source technologies
• Excited to work intimately with data
• Passionate, self-motivated, and takes ownership