Role Overview
IBM Consulting (IBM India Private Limited) is seeking an experienced Data Engineer – Data Platforms – AWS to join its hybrid cloud and data engineering practice in Pune, Chennai, or Bangalore, India. Operating in a hybrid daytime shift model, you will architect, build, and optimize scalable, secure enterprise data platforms and batch/real-time ETL pipelines on AWS using Apache Spark, Python/Scala, AWS Glue, and SQL.
Core Responsibilities
- AWS Data Platform & Pipeline Engineering: Design, develop, and maintain batch and real-time data pipelines using AWS Glue, Athena, EMR, Kinesis, and Apache Spark.
- Serverless, Containers & IaC: Build and operate scalable workloads on AWS Lambda, EC2, and EKS while provisioning cloud infrastructure via AWS CloudFormation.
- Storage & SQL Analytics Optimization: Manage data layers across Amazon S3, RDS, Redshift, Aurora, and DynamoDB, writing high-performance SQL queries for analytics, reporting, and validation.
- Cloud Security & Workload Governance: Enforce AWS security best practices across IAM policies, VPC networking, encryption, and least-privilege access while optimizing pipeline cost and reliability.
- Cross-Functional Stakeholder Delivery: Collaborate with architects, application teams, and business stakeholders to support enterprise analytics and regulatory reporting initiatives.
Qualifications & Requirements
- Educational Background: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related technical field (Master's degree preferred).
- Professional Experience: 5 to 10 years of hands-on experience designing and operating enterprise data platforms and distributed data pipelines on AWS.
- Core Technical Stack: Strong proficiency in Python and/or Scala, Apache Spark, advanced SQL tuning, and core AWS services (S3, Lambda, EC2, EKS, Glue, Athena, RDS, Redshift, VPC, IAM, CloudFormation).
- Data Engineering & Open-Source Ecosystem: Solid foundation in ETL/ELT concepts, dimensional data modeling, and open-source tools like Apache Airflow and Kafka/Kinesis.
- Preferred Qualifications: Hands-on exposure to dbt, Snowflake, Databricks, Lakehouse architectures, AWS Glue DataBrew, and Redshift Spectrum within regulated enterprise environments.
- Work Mode & Travel: Full-time hybrid daytime role based in Pune, Chennai, or Bangalore (may require up to 20% travel or ~1 day per week).
About IBM Consulting
IBM Consulting is IBM's global professional services and business transformation arm, partnering with leading organizations worldwide to accelerate hybrid cloud and AI adoption. Combining deep industry expertise with open hybrid cloud architectures, IBMers build resilient, data-driven solutions that drive measurable business and societal impact.