About this role
Job Summary
Seeking a Senior Specialist with 7 to 11 years of experience skilled in MS SQL Server SparkSQL and Python to design and optimize big data solutions within the HadoopSpark ecosystem
Job Description
Develop optimize and maintain complex SQL queries and stored procedures using MS SQL Server Design and implement data processing pipelines leveraging SparkSQL for largescale data analytics Write efficient Python scripts for data manipulation automation and integration tasks Collaborate with data engineers and analysts to ensure seamless data flow and highquality data management Utilize knowledge of HadoopSpark frameworks to build scalable and robust data solutions Analyze performance bottlenecks and implement improvements in data processing workflows Ensure adherence to best practices in coding data security and documentation standards Stay updated with emerging technologies and industry trends related to big data and analytics
Roles and Responsibilities
Lead the design and development of data ingestion and transformation processes using MS SQL Server and SparkSQL
Mentor junior team members on technical skills and best practices within the HadoopSpark and Python environment
Collaborate with crossfunctional teams to gather requirements and translate them into technical solutions
Monitor and troubleshoot data pipeline issues to maintain high availability and reliability
Implement data quality checks and validation mechanisms to ensure data accuracy
Participate in code reviews and contribute to continuous improvement initiatives
Drive automation efforts to enhance operational efficiency and reduce manual intervention
Communicate effectively with stakeholders regarding project status risks and mitigation plans Mandatory Skills : MS SQL Server, Python, SparkSQL
Good to Have Skills : Hadoop-Spark SparkSQL - Python
Immigration pathway
ON • Express • onsite • TEER 1