Apply Now »

Azure Data Engineer

Date:  Nov 9, 2024
Location: 

Bangalore, KA, IN, 560100

Req ID:  27749

Summary

As an Azure Data Engineer at Gainwell, you will be a part of an innovative healthcare technology company with a mission to serve the healthcare needs of our communities.

Primary skills:

Candidates should possess skills in

  • ADF pipeline set up and management
  • SQL programming (MS SQL Server – SSMS)
  • Issue Resolution & Troubleshooting in Azure environments
  • Testing and validations
  • Automation activities
  • Set up ETL activities in ADF framework
  • Identify process efficiencies opportunities

Secondary skills:

  • Analyze logs and set up monitoring mechanisms
  • Security & Access Management
  • Problem solving and root cause analysis

Candidate should have expertise in:

  • Azure Data Factory, Azure Data Lake, and Azure SQL Database
  • Azure DevOps (for managing CI/CD processes, including pipeline setup, tracking, and performance monitoring)
  • Working knowledge of data integration patterns and ETL processes

Key Responsibilities:

Data Pipeline Management:

  • Build and execute ADF pipelines for continuous integration/continuous deployment (CI/CD) processes.
  • Set up and configure multi-stage pipelines and various environment configurations for development, testing, and production.
  • Track pipeline performance and ensure the health of applications, taking corrective actions when needed.
  • Design processes for monitoring pipeline activities and system health.
  • Integrate data from multiple sources, including SQL databases and third-party APIs.
  • Develop data integrations between source and destination systems to support operational and reporting needs.
  • Build infrastructure for data extraction, transformation, and loading (ETL) using MS SQL and Azure technologies.
  • Identify opportunities for automating tasks within the ADF framework and execute enhancements to improve efficiency.
  • Set up Azure Monitor to track performance metrics, logs, and alerts for Azure resources.

SQL Operations:

  • Run SQL programs within Azure environments, optimizing for performance and data accuracy.
  • Work with data in relational databases and assist with query optimization and troubleshooting.

Issue Resolution & Troubleshooting:

  • Perform root cause analysis and debugging to address issues related to application performance.
  • Coordinate with the Azure IT team to troubleshoot memory leaks, inefficient memory usage, and other performance bottlenecks in applications.
  • Provide rapid response to resolve critical system errors and ensure minimal downtime.

Logs & Monitoring:

  • Create and manage rules and alerts for system monitoring.
  • Analyze logs from various Azure resources to detect and resolve performance issues.
  • Set up comprehensive logging and monitoring solutions for CI/CD processes to ensure system reliability.

Testing & Validation:

  • Conduct thorough testing and validation of pipelines and infrastructure before production deployment.
  • Collaborate with the IT and development teams to perform functional, performance, and regression testing.

Security & Access Management:

  • Coordinate with IT teams to manage access control, granting or revoking access as needed and raising tickets for user management.
  • Maintain documentation of current pipelines, refresh processes, and security configurations.
  • Coordinate and set up various security levels in reports and applications to ensure data privacy and compliance with security protocols.
  • Assist in the review and audit of Azure infrastructure architecture, providing insights for improvements and optimizations.
  • Work on creating a unified strategy for Azure cloud transformation, covering governance, automation, security, and operational aspects.

Other Responsibilities:

  • Assist in the creation and execution of monthly refresh plans to keep data infrastructure updated.
  • Maintain up-to-date documentation for all data pipelines, processes, and operational tasks.

What We’re Looking For:

The ideal candidate should have the following qualifications:

  • Bachelor's degree in Computer Science, Information Technology, or a related field.
  • Azure certifications such as Azure Data Engineer Associate (or equivalent certifications).
  • Working knowledge of Azure DevOps for managing CI/CD processes, including pipeline setup, tracking, and performance monitoring.
  • Proficiency in Power BI for reporting and data visualization.
  • Adequate experience in Azure Data Factory, Azure Data Lake, and Azure SQL Database.
  • Strong working knowledge of data integration patterns and ETL processes for seamless data transfer and transformations.
  • Familiarity with Azure Log Analytics to query and analyze logs from various Azure resources, providing insight into performance and system health.
  • Good understanding of Azure environments, infrastructure, and resource management.

Soft Skills:

  • Good verbal and written communication skills to effectively collaborate with cross-functional teams and business stakeholders.
  • Attention to detail and a proactive approach to identifying and addressing potential issues.
  • Self-driven with the ability to work independently and manage priorities efficiently.

Other Requirements:

  • Ability to work on evening shift to maximize overlap with US teams, ensuring efficient communication and collaboration.
  • Eye for details in monitoring, troubleshooting, and pipeline execution.

 

Apply Now »