Data Engineer 1

Data Science & Engineering

Data Engineering

San Jose/Remote

B2B/UOP

REF1645H

Project - the aim you'll have

Overview
Software Mind is seeking qualified candidates to fill the role of Data Engineer.

In addition to a competitive salary rate and a positive work environment, we are committed to delivering high-quality technology solutions. We also offer:

Flexible schedules
An authentic work-life balance
Payment in US Dollars

About the role:

We are seeking a highly motivated Data Engineer to play a crucial role in our data transformation program. You will be instrumental in the migration of data sources to a centralized cloud environment, with Databricks as our strategic standard on AWS. As a Data Engineer, you will be responsible for designing, developing, and maintaining our data infrastructure and pipelines, utilizing a variety of technologies and tools to ensure efficient data processing, integration, and management. You will collaborate closely with data scientists, analysts, and other stakeholders to ensure the availability, reliability, and scalability of our data systems, while promoting best practices in data governance and architecture.

#LI-DNI

Expectations - the experience you need

Some of the main responsibilities for the role include:

Design and implement processes to ingest data from various sources into the Databricks Lakehouse platform, ensuring alignment with architectural and engineering standards.
Develop, maintain, and optimize data models and ETL pipelines that support the Medallion Architecture (Bronze, Silver, Gold layers) to enhance data processing efficiency and facilitate data transformation.
Utilize Databricks to integrate, consolidate, and cleanse data, ensuring accuracy and readiness for analysis, while leveraging Delta Lake for versioned data management.
Collaborate with business analysts, data scientists, and stakeholders to understand their data requirements and deliver tailored solutions that leverage the capabilities of the Databricks Lakehouse platform.
Engage with clients to support data engineering requirements, address technical queries, and provide guidance on best practices for utilizing the Databricks Lakehouse platform.
Stay updated on industry trends and emerging technologies in data engineering, particularly those related to Databricks and cloud data solutions, continuously enhancing your skills and knowledge.

Job Skills/Requirements
– +90% English written and oral (at least B2 level) with excellent communication skills
– Bachelor’s or Master’s degree in Computer Science, Data Science, Engineering, or a related field.
– Minimum of 5+ years of experience in data engineering or a related data role.
– Proven experience in designing and implementing production-grade Spark-based solutions.
– Proficient in query tuning, performance tuning, troubleshooting, and debugging Spark or other big data solutions.
– Familiarity with big data technologies such as Spark/Delta, Hadoop, NoSQL, MPP, and OLAP.
– Experience with cloud architecture, systems, and principles, particularly in AWS.
– Proficient in programming languages such as Python, R, Scala, or Java.
– Expertise in scaling ETL pipelines for performance and cost-effectiveness, with a focus on tuning queries for big data.
– Familiarity with big data technologies such as Spark/Delta, Hadoop, NoSQL, MPP, and OLAP.
– Experience with cloud architecture, systems, and principles, particularly in AWS.
– Proficient in programming languages such as Python, R, Scala, or Java.
– Expertise in scaling ETL pipelines for performance and cost-effectiveness, with a focus on tuning queries for big data.
– Cloud certification is highly desirable.