Lead Data Engineer | Kroll Careers India
Job Description
Summary
Kroll is seeking an experienced Lead Data Engineer in India to drive the design, development, and integration of advanced data infrastructure and analytics pipelines. This role offers the opportunity to work with cutting-edge data engineering technologies, cloud platforms, and AI/ML-driven analytics across global teams, serving financial institutions, government agencies, and corporate clients. The ideal candidate will have strong expertise in cloud data platforms (Azure, AWS, Databricks), ELT/ETL processes, Python, Spark, and data governance. Join Kroll to contribute to high-impact data solutions while accelerating your career in a collaborative, innovative environment.
Job Details at a Glance
| Title | Lead Data Engineer |
|---|---|
| Company | Kroll |
| Location | India |
| Job Type | Full-time, Remote/Hybrid |
| Experience Required | 5–12 years |
| Education | Bachelor’s/Master’s in Computer Science, Engineering, Data Science, or related field |
| Tools / Technologies | Azure, AWS, Databricks, Python, PySpark, Spark SQL, SQL, FastAPI, Pandas, Polars, Deltalake, Docker, Kubernetes |
| Travel | As needed |
Key Responsibilities
Data Architecture & Engineering
- Design and build scalable organizational data infrastructure and architecture.
- Develop and manage ELT/ETL pipelines for data ingestion, transformation, and distribution.
- Implement Lakehouse and Medallion architecture, ensuring robust data governance and quality.
- Optimize data pipelines for performance, scalability, and reliability.
Process Optimization & Automation
- Identify and implement internal process improvements to automate manual workflows and optimize data delivery.
- Select and configure the best tools, services, and resources for data engineering projects.
- Build fault-tolerant, high-quality data pipelines in collaboration with global teams.
Cloud & Technology Enablement
- Leverage Azure, AWS, and Databricks platforms for cloud-based data solutions.
- Develop REST APIs, Python SDKs, Spark Jobs, and libraries for advanced data processing.
- Apply best practices in data orchestration, CI/CD, and containerization (Docker/Kubernetes).
Collaboration & Reporting
- Work closely with cross-functional teams to deliver integrated, data-driven solutions.
- Conduct data profiling, cataloging, and mapping to support technical data flows.
- Communicate effectively across global teams, ensuring transparency and project alignment.
Required Skills & Competencies
- Advanced ETL/ELT experience and data pipeline architecture.
- Strong expertise in cloud platforms (Azure, AWS, Databricks).
- Proficiency in Python, Spark ecosystem (PySpark + Spark SQL), SQL databases.
- Experience with Lakehouse architecture, data governance, and pipeline orchestration.
- Knowledge of open-source tools and Python libraries: FastAPI, Pydantic, Polars, Pandas, Deltalake.
- Strong analytical, problem-solving, and communication skills.
- Ability to work effectively in international, collaborative teams.
Desired Skills
- Strong understanding of cloud architecture principles: compute, storage, networking, security, and cost optimization.
- Advanced SQL and Spark query optimization skills.
- Experience building Lakehouse solutions using Azure Databricks, Data Lake, SQL, and PySpark.
- Knowledge of programming paradigms (OOP, async programming, batch processing).
- Familiarity with CI/CD, Git, Docker, Kubernetes for data engineering workflows.
Why Join Kroll
- Work with global teams and high-impact clients in finance, government, and corporate sectors.
- Collaborate on AI/ML-driven analytics, advanced data engineering, and cloud solutions.
- Access to career growth, learning, and mentorship in a collaborative environment.
- Be part of Kroll’s mission to protect, restore, and maximize client value while shaping your career trajectory.
Apply Now
Take your career to the next level as a Lead Data Engineer at Kroll, India, driving advanced data engineering, cloud analytics, and AI/ML-enabled solutions.