Senior Big Data Engineer (Azure)

Avanceon Middle East & South Asia


Date: 1 week ago
City: Lahore
Contract type: Full time
As a Big Data engineer you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data. You will develop and implement data Lake solutions for company-wide applications and manage large sets of structured, semi structured and unstructured data. You will work in a modern cloud-based data lakehousing environment alongside a team of diverse, intense, and interesting co-workers.

Job Responsibilities

  • Performing analysis of large data stores and derive insights using Big Data querying
  • Design and develop code, scripts, and data pipelines that leverage structured, semi-structured and unstructured data
  • Should be able to manage Data ingestion pipelines and stream processing
  • Perform computations using Azure functions
  • Responsible for the documentation, design, and development of Hadoop applications
  • Perform day-to-day tasks to accuracy with and minimal directions while meeting deadlines

Qualification & Skillset

  • Bachelor/master’s degree in computer science
  • At least 4-6 years of relevant working experience with Cloud Computation
  • Hands-on experience on Azure/AWS
  • Experience working with Apache Spark, Apache Hive, Apache HBase
  • Hands-on experience on:
  • Azure/AWS Architectures
  • SQL, Scala, Node.js, JS, Java
  • Python with Pandas, NumPy, TensorFlow
  • ETL

Preferred Requirement

  • Implementing Lambda Architecture with stream processing and batch processing
  • Implement security standards and practices in the Architecture of Cloud Solution
  • Training/Certification on Data Lake HDFS
  • Training/Certification on Azure Data Lake Gen 2
  • Training/Certification on Data Bricks
  • Experience with Azure Data Explorer and Kusto query language would be a plus.
  • Experience with Azure Data Factory would be a plus.

How to apply

To apply for this job you need to authorize on our website. If you don't have an account yet, please register.

Post a resume

Similar jobs

Future Talent Wanted - SEO Executive

BAT, Lahore
2 hours ago
BAT is evolving at pace into a global multi-category business. With products like VELO, VUSE and GLO we are on a mission to decrease the health impact of our industry To achieve our ambition, we are looking for colleagues who are ready to Be The Change. Come, and join us on this journey!British American Tobacco Pakistan has an exciting opportunity...

External Audit Manager / Senior Manager / Director (Financial Services Sector) Lahore

EY, Lahore
1 day ago
From small businesses to some of the world’s best-known brands, our clients across the MENA region count on reliable financial statements, information and processes to inform their on-going strategies. More than ever, business is about interpreting and reacting to complex data - and they look to us for advice and services they can trust every time. You will use your...

Legal Case Manager

Cedar Financial, Lahore
3 days ago
Job Summary:The Legal Case Manager at Cedars Business Services is responsible for the efficient review and placement of debt collection files with partner attorney offices. This role involves closely monitoring the progress of legal actions, coordinating with legal customer service representatives (CSRs), and collaborating with other departments to ensure the smooth liquidation of accounts. The Legal Case Manager plays a...