Log In

Don't have an account? Sign up now

Lost Password?

Sign Up

Technology and IT

Data Engineer (PySpark/Informatica) Full Time

Summary:

Highmark Health is seeking a Data Engineer with expertise in data engineering, ETL development, and cloud-based solutions. The role involves designing, developing, and maintaining data infrastructure to ensure efficient data flow and transformation across multiple systems. Candidates should have experience with tools like Python, PySpark, and Informatica, and possess strong skills in building scalable data pipelines and solutions.

Responsibilities:

  • Design, develop, and maintain robust data processes and solutions to ensure the efficient movement and transformation of data across multiple systems.

  • Develop and maintain data models, databases, and data warehouses to support business intelligence and analytics needs.

  • Collaborate with stakeholders across IT, product, analytics, and business teams to gather requirements and provide data solutions that meet organizational needs.

  • Monitor work against the production schedule, provide progress updates, and report any issues or technical difficulties to lead developers regularly.

  • Implement and manage data governance practices, ensuring data quality, integrity, and compliance with relevant regulations.

  • Collaborate on the design and implementation of data security measures, including access controls, encryption, and data masking.

  • Perform data analysis and provide insights to support decision-making across various departments.

  • Stay current with industry trends and emerging technologies in data engineering, recommending new tools and best practices as needed.

  • Other duties as assigned or requested.

Requirements:

  • Bachelor’s degree in Computer Science, Information Systems, Data Science, Computer Engineering, or a related field, or relevant experience and/or education as determined by the company in lieu of a bachelor’s degree.

  • 3 years of experience in design and analysis of algorithms, data structures, and design patterns in building and deploying scalable, highly available systems.

  • 3 years of experience in a data engineering, ETL development, or data management role.

  • 3 years of experience in SQL and experience with database technologies (e.g., MySQL, PostgreSQL, MongoDB).

  • 3 years of experience in data warehousing concepts and experience with data warehouse solutions (e.g., Snowflake, Redshift, BigQuery).

  • 3 years of experience with data pipeline and workflow management tools (e.g., Apache, GCP Tools, Databricks, PySpark).

  • 3 years of experience building ETL and Data Integration pipelines in Python, PySpark, and Informatica.

  • 3 years of experience working with On Prem databases like Oracle, Teradata, and DB2.

  • 3 years of experience with Cloud platforms (GCP and Azure) and their respective data services.

  • 1 year of experience working with a variety of technology systems, designing solutions or developing data solutions in healthcare.

  • 3 years of experience in data governance, data quality, and data security best practices and tools.

  • 3 years of experience translating requirements, design mockups, prototypes, or user stories into technical designs.

  • 3 years of experience with data-related code that is fault-tolerant, efficient, and maintainable.

Benefits:

  • Equal employment opportunity for all individuals, prohibiting discrimination based on veteran status or disability.

  • Adherence to HIPAA and company policies for handling confidential data and business conduct.

  • Compliance with all applicable federal, state, and company regulations and training requirements.

Tagged as: , , ,

Job Overview