Senior Data Engineer

We are looking for an experienced data engineer responsible for expanding and optimizing our big data pipelines. You will be working alongside other engineers and developers working on different layers of the infrastructure. Therefore, a commitment to collaborative problem-solving, sophisticated design and the creation of quality products is essential.

Responsibilities:

· Designing and implementing data pipelines to collect, clean, and transform data from various sources

· Building and maintaining data storage and processing systems, such as databases, data warehouses, and data lakes.

· Ensuring data is properly secured and protected

· Developing and implementing data governance policies and procedures

· Collaborating with data analysts, data scientists, and other stakeholders to understand their data needs and ensure they have       access to the data they need

· Sharing knowledge with the wider business, working with other BA’s and technology teams to make sure processes and ways of  working are documented.

· Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize

data ingestion pipelines so that data is shared effectively across various business systems.

· Ensure the design, code and procedural aspects of the solution are production ready, in terms of operational, security and compliance standards.

· Participate in day-to-day project and agile meetings and provide technical support for faster resolution of issues.

· Clearly and concisely communicating to the business, on the status of items and blockers.

· Have an end-to-end knowledge of the data landscape within Lebara.

Requirements

· 6+ years of design & development experience with big data technologies

· Proficient in Python, PySpark

· 2+ years of development experience in cloud technologies like Azure, AWS or GCP

· Proficient in querying and manipulating data from various DB (relational and big data).

· Experience of writing effective and maintainable unit and integration tests for ingestion

pipelines.

· Experience of using static analysis and code quality tools and building CI/CD pipelines.

· Excellent communication, problem-solving, and leadership skills, and be able to work well

in a fast-paced, dynamic environment.

· Experience of working on high-traffic and large-scale Software Products

 

We have an amazing team of 550+ individuals working on highly innovative enterprise projects & products. Our customer base includes fortune 5 retail and CPG companies, leading store chains, fast growth fintech and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is ISO 9001:2015 certified. We have a vibrant culture of learning via collaboration and making workplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves. This is obviously above and beyond the usual stuff (excellent pay, medical benefits, provident fund, stock options, and frequent appraisals).

To know more about Confiz Limited, visit: https://web.facebook.com/lifeatconfiz