Role Summary
Bayer is seeking an experienced Senior Data Engineer to join the Data & Analytics organization supporting Enabling Corporate Functions in Bangalore, Karnataka, India. In this full-time role, you will design, build, and maintain scalable, reliable, and cost-efficient data pipelines across Finance, Supply Chain, HR, and Procurement business units. Operating under Bayer's Dynamic Shared Ownership (DSO) model, you will leverage Python, PySpark, Databricks, Delta Lake, and modern DataOps practices to build FAIR (Findable, Accessible, Interoperable, and Reusable) data foundations.
Key Responsibilities
- Pipeline Engineering: Design, develop, and maintain idempotent, scalable, and automated ETL/ELT pipelines using Python, PySpark, Delta Lake, and MERGE INTO patterns.
- DataOps & Quality Assurance: Enforce DataOps practices including automated data-contract enforcement, data-quality monitoring, unit and integration testing, and proactive incident remediation.
- Data Governance: Establish fine-grained access control policies, data lineage tracking, and compliance frameworks across Databricks workspaces using Unity Catalog.
- Infrastructure as Code (IaC): Provision and manage Databricks workspaces, compute clusters, serverless SQL warehouses, and cloud storage integrations using Terraform and Azure DevOps.
- CI/CD & Cost Optimization: Build automated CI/CD pipelines using Databricks Asset Bundles, Databricks CLI, and GitHub Actions; monitor compute consumption and right-size clusters for cost efficiency.
Key Qualifications
- Education: Bachelor's or Master's degree in Computer Science, Data Engineering, Information Technology, or an equivalent technical field.
- Experience: At least 5 years of hands-on experience in data engineering, DataOps, DevOps, or an enterprise cloud data engineering capacity.
- Core Technical Skills: Databricks Lakehouse Platform, Delta Lake, PySpark, Python, Advanced SQL, Unity Catalog, and Microsoft Azure Cloud Services.
- Key Methodologies: Infrastructure as Code (Terraform), CI/CD Automation (Databricks Asset Bundles/GitHub Actions/Azure DevOps), DataOps, and Data Quality Frameworks.
- Preferred Capabilities: Working knowledge of Databricks Serverless SQL, cloud FinOps cost optimization, and multi-cloud exposure (AWS or GCP).
About Bayer
Bayer is a global enterprise with core competencies in the life science fields of healthcare and agriculture. Powered by its Dynamic Shared Ownership (DSO) philosophy, Bayer eliminates bureaucratic layers to drive high productivity, faster innovation, and impactful business results across its global divisions.