Data Engineer

SP Group

  • Singapore
  • Permanent
  • Full-time
  • 29 days ago
Why Work for UsWe Power the Nation.Make the most of your talents and develop products that can create impact on a national scale. We are an in-house software team, assembled to move with speed and deliver with quality.We Build Reliable Solutions. For Customers, Company and Country.You will be part of the Digital Technology Team and together, you will innovate, create, and deploy digital products that will empower more than 3,800 employees within SP Group and improve the quality of life for more than 1.7 million commercial, industrial and residential customers that SP Group serves. We build solutions that enable sustainable high quality lifestyles and help consumers save energy and cost, as well as supporting national goals for a sustainable livable city. Now, imagine the impact you can create.What You'll Do:
  • Create and manage a single master record for each business entity, ensuring data consistency, accuracy, and reliability.
  • Implement data governance processes, including data quality management, data profiling, data remediation, and automated data lineage.
  • Create and maintain multiple robust and high-performance data processing pipelines within Cloud, Private Data Centre, and Hybrid data ecosystems.
  • Assemble large, complex data sets from a wide variety of data sources.
  • Collaborate with Data Scientists, Machine Learning Engineers, Business Analysts, and Business users to derive actionable insights and reliable foresights into customer acquisition, operational efficiency, and other key business performance metrics.
  • Develop, deploy, and maintain multiple microservices, REST APIs, and reporting services.
  • Design and implement internal processes to automate manual workflows, optimize data delivery, and re-design infrastructure for greater scalability.
  • Establish expertise in designing, analyzing, and troubleshooting large-scale distributed systems.
  • Support and work with cross-functional teams in a dynamic environment.
What You'll Need:
  • Minimum 7 years of experience in a Data Engineer role.
  • Experience building and operating large-scale data lakes and data warehouses.
  • Experience with Hadoop ecosystem and big data tools, including Spark and Kafka.
  • Experience with Master Data Management (MDM) tools and platforms such as Informatica MDM, Talend Data Catalog, Semarchy xDM, IBM PIM & IKC, or Profisee.
  • Familiarity with MDM processes such as golden record creation, survivorship, reconciliation, enrichment, and quality.
  • Experience in data governance, including data quality management, data profiling, data remediation, and automated data lineage.
  • Experience with stream-processing systems including Spark-Streaming.
  • Experience working with Cloud services using one or more Cloud providers such as Azure, GCP, or AWS.
  • Experience with Delta Lake and Databricks.
  • Advanced working experience with relational SQL and NoSQL databases, including Hive, HBase, and Postgres.
  • Deep understanding of SQL and the ability to optimize data queries.
  • Experience with object-oriented/object function scripting languages: Python, Java, Scala, etc.
  • A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
  • Experience applying modern development principles (Scrum, TDD, continuous integration, and code reviews).
  • Proven ability to support and work with cross-functional teams in a dynamic environment.
#LI-DNI#LI-NF1Please click on the blue ' ' button at the top right hand side corner of this screen, if you are keen to apply for this job.Thank you for your interest in SP Group. You will be contacted if you are shortlisted for an interview.

SP Group