Data Engineer Intern

• Design and develop data pipelines: Architect, build, and optimize data pipelines to ingest, transform, and store large volumes of data from various sources into our data warehouse using AWS technologies such as AWS Glue, AWS Lambda, and AWS S3.
• Data modeling and schema design: Collaborate with data scientists and analysts to design efficient and scalable data models and database schemas that meet business requirements and enable accurate and timely reporting and analysis.
• Data transformation and processing: Utilize your expertise in Python programming to transform raw data into clean, structured datasets, applying data validation, cleansing, and enrichment techniques.
• AWS infrastructure management: Leverage your skills in AWS services such as EC2, RDS, Redshift, and EMR to build and maintain scalable and reliable data infrastructure on the cloud. Implement best practices for security, performance optimization, and cost efficiency.
• Terraform deployment and management: Utilize Terraform to automate the provisioning, configuration, and deployment of AWS resources, ensuring reproducibility, consistency, and scalability of our infrastructure.
• ETL optimization and performance tuning: Identify bottlenecks and optimize the performance of ETL processes and data pipelines. Implement data partitioning, indexing, and caching strategies to improve overall system performance.
• Data quality and governance: Implement data quality checks, data monitoring, and data governance processes to ensure data accuracy, integrity, and compliance with regulatory requirements.
• Visualization and reporting: Collaborate with business stakeholders to understand their reporting and analytics requirements. Utilize Tableau or other visualization tools to create interactive dashboards and reports that provide actionable insights to drive decision-making.
• Collaboration and mentoring: Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand their data needs and provide guidance on best practices.
• Strong proficiency in AWS services such as Glue, Lambda, S3, EC2, RDS, Redshift, and EMR.
• Solid understanding of infrastructure-as-code concepts and experience with Terraform for infrastructure provisioning and management.
• Expertise in Python programming for data transformation, manipulation, and processing.
• Proficiency in designing and optimizing data models and database schemas.
• Experience with data visualization tools such as Tableau or similar.
• Strong understanding of ETL processes, data warehousing concepts, and data integration techniques.
• Familiarity with data quality assurance and governance practices.
• Excellent problem-solving skills and ability to troubleshoot and debug complex data issues.
• Strong communication and collaboration skills, with the ability to work effectively in cross- functional teams.
• Proven ability to mentor and provide technical guidance to junior team members.

InternSG