Role Overview
Capgemini features an opening for an experienced PySpark Developer within its Data & AI practice, with opportunities across Bangalore, Hyderabad, Chennai, Navi Mumbai, and Pune. In this senior data engineering role, you will design, engineer, and deploy scalable data-driven solutions and high-throughput pipelines that drive critical business and analytics operations. Leveraging advanced programming proficiency in PySpark, Python, and SQL analytical functions, you will architect robust data models, design automated CI/CD build workflows, and guide technical and architectural strategies across modern cloud environments (AWS, Azure, or GCP).
Core Responsibilities
- Distributed Data Processing: Architect, develop, and optimize clean, scalable, and resilient data processing pipelines using PySpark and Apache Spark.
- Data Modeling & Analytics: Build and maintain complex relational and dimensional data models, optimizing SQL queries and advanced analytical functions for high-volume datasets.
- Cloud Infrastructure & Integration: Deploy, manage, and scale data solutions across major hyperscaler cloud platforms (AWS, Azure, or GCP).
- CI/CD & Build Management: Design, implement, and maintain automated CI/CD pipelines, release mechanisms, and build management workflows for data products.
- Technical & Architectural Decision-Making: Actively contribute to technical design reviews, system architecture discussions, and key business roadmap decisions.
- Cross-Functional Collaboration: Partner with data architects, software engineers, and business stakeholders to translate functional requirements into scalable technical deliverables.
Qualifications & Requirements
- Professional Experience: 6+ years of dedicated, hands-on software and data engineering experience focused on PySpark development.
- Programming Proficiency: Strong programming command of Python and PySpark for large-scale distributed computing; experience with Scala is highly valued.
- Database & Querying: Advanced proficiency in SQL, execution plan analysis, analytical/window functions, and database performance tuning.
- Cloud & DevOps: Direct hands-on experience building solutions on cloud platforms (AWS, Azure, or GCP) combined with automated CI/CD pipeline implementation.
- Architecture & Design: Demonstrated capability in data pipeline architecture, data modeling standards, and distributed system troubleshooting.
About Capgemini
Capgemini is a global business and technology transformation partner driven by AI and human ingenuity. Operating across more than 50 countries with a diverse workforce of 420,000 employees—including nearly 235,000 in India—Capgemini partners with the world's leading organizations to unlock the full potential of technology, digital engineering, and cloud platforms.