Data Engineer

Location: PAN India
Shift: General Shift
Experience: 8+ Years
Education: Bachelor's Degree – Graduation is compulsory

Job Overview

We are looking for an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data integration solutions across multiple heterogeneous data sources. The role involves working with cloud technologies, data lakes, data warehouses, ETL/ELT processes, and large-scale structured and unstructured data.

Key Responsibilities

• Design, build, and maintain scalable data pipelines for heterogeneous data sources.

• Develop end-to-end ETL/ELT processes for data extraction, transformation, and loading.

• Build and optimize data solutions using AWS, Azure, or GCP.

• Work with data lakes and data warehouses such as Snowflake, Amazon Redshift, and Databricks.

• Develop reusable pipelines, libraries, templates, and data integration standards.

• Process and transform large volumes of structured and unstructured data.

• Develop data quality, integrity, and validation checks.

• Work with APIs, JSON, XML, and various data transfer mechanisms.

• Collaborate with cross-functional teams, vendors, architects, and downstream data users.

• Troubleshoot and optimize data pipelines and resolve data integration issues.

• Provide data visualization and ad-hoc analysis as required.

• Support continuous improvement and automation of manual data processes.

• Provide technical leadership and contribute to project delivery and stakeholder requirements.

Required Skills

• 5+ years of relevant Data Engineering experience.

• Strong experience in Python/R and SQL.

• Experience with NoSQL databases and database concepts.

• Hands-on experience with AWS, Azure, or GCP.

• Experience with Apache Spark / Databricks.

• Experience with Snowflake, Amazon Redshift, or similar data warehouses.

• Strong knowledge of ETL and ELT.

• Experience with GitHub/GitLab and Jenkins.

• Experience deploying data pipelines in the cloud.

• Experience handling large datasets and unstructured data.

• Knowledge of APIs, JSON, and XML.

• Experience with data lakes and cloud-based data integration.

• Strong problem-solving, project management, and communication skills.

Good to Have

• Experience developing R Shiny or Python applications.

• Experience with Hadoop.

• Experience with Agile development methodologies.

• R or Python certification.

Important Requirements

• Graduation is compulsory.

• PAN and DOB are required for employee profile creation in the database.

• Former and/or current TCS/Tata employees are not eligible.