Senior Data Engineer (Python)

  • Lahore, Punjab, Pakistan
  • Permanent
  • Software Engineering

We are looking for an experienced developer responsible for the design, development and maintenance of applications. You will be working alongside other engineers and developers working on different layers of the infrastructure. Therefore, a commitment to collaborative problem solving, sophisticated design, and the creation of quality products is essential.

Responsibilities:

  • Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems.
  • Build ETL/ELT and Ingestion pipelines and design optimal data storage and analytics solutions using cloud and on-perm technologies.
  • Ensure the design, code and procedural aspects of solution are production ready, in terms of operational, security and compliance standards.
  • Participate in day to day project and production delivery status meetings, and provide technical support for faster resolution of issues.

Qualifications

  • Minimum 4 years of design & development experience with at least 1 year of experience with big data technologies like Spark/Flink and Kafka
  • Proficient in Python or Java/Scala.
    Hands-on experience with some of the following technologies:
    • YARN, HDFS/ADLS/GCS, Hive/BigQuery, ELK/Splunk/Stack Driver
    • Kubernetes
    • Airflow/Oozie
    • Spring/Guice or any other DI framework,
    • RESTful Web Services.
  • Proficient in querying and manipulating data from various DB (relational and big data).
  • Experience of writing effective and maintainable unit and integration tests for ingestion pipelines.
  • Experience of using static analysis and code quality tools and building CI/CD pipelines.
  • Exposure to microservices architecture (nice to have).
  • Bachelor's degree from an accredited college/university or equivalent work experience.

We have an amazing team of 350+ individuals working on highly innovative enterprise projects & products. Our customer base includes fortune 5 retail and CPG companies, leading store chains, fast growth fintech and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz is CMMi Level 3 and ISO 9001:2015 certified. We have a vibrant culture of learning via collaboration and making workplace fun.

People who work with us work with the cutting-edge technologies while contributing success to the company as well as to themselves. This is obviously above and beyond the usual stuff (excellent pay, medical benefits, frequent appraisals and bonuses).

To know more about Confiz Limited, visit: https://web.facebook.com/lifeatconfiz 

Apply Now