Data Engineer Job at Motion Recruitment, Chicago, IL

UjVYdTZRUU5iUjNOcnFaSzNTM0txWlVKa0E9PQ==
  • Motion Recruitment
  • Chicago, IL

Job Description

About the Company

We are looking for a strong core Data Engineer with hands-on experience in PySpark, Databricks, and Azure data platforms to design, build, and support end-to-end data pipelines. The ideal candidate will develop and optimize data transformations, build production-grade Python components, and maintain cloud-native Azure environments while collaborating with application teams and ensuring high-quality, reliable data delivery. This role offers the opportunity to work with large-scale datasets, implement ETL/ELT best practices, optimize Databricks clusters, and leverage modern cloud technologies to support AI/ML initiatives.

About the Role

Location: 3 days Hybrid in Chicago, IL

Duration: 6+ Month Contract

Interview: 2 video interview and final onsite

Responsibilities

  • Design, build, and support end-to-end data pipelines, including ingestion, transformation, validation, and publishing.
  • Develop and optimize SQL and PySpark/Databricks transformations for large datasets.
  • Build production-grade Python modules with logging, error handling, testing, and integration with APIs/files.
  • Create, maintain, and operate Azure Data Factory (ADF) pipelines, including triggers, parameterization, monitoring, and failure handling.
  • Work within Azure environments: ADLS Gen2 (Blob Storage), Azure SQL, Azure App Service, and resource groups.
  • Provision and maintain Azure components using Pulumi (Infrastructure as Code).
  • Optimize Databricks clusters, workflows, and jobs for performance and reliability.
  • Participate in code reviews, documentation, and operational support, including triage and root cause analysis.
  • Collaborate with application teams for integration, troubleshooting, and operational coordination.

Qualifications

  • Education: Bachelor’s degree in Computer Science, Engineering, or a related technical field (or equivalent experience).

Required Skills

  • Experience: 5+ years as a Data Engineer; 3+ years in ETL/ELT concepts, PySpark, and SQL.
  • SQL: Advanced querying, CTEs, views, joins, complex transformations, and performance tuning.
  • Python: 2+ years building production-quality modules, unit/integration testing, logging, and CI/CD integration.
  • Databricks: 1+ year working with notebooks, jobs, workflows, external/managed tables, Delta Lake, and basic cluster configuration.
  • Azure Data Factory (ADF): 1+ year creating and maintaining pipelines, including triggers, parameterization, monitoring, and error handling.
  • Azure Cloud: Hands-on with ADLS Gen2, Azure SQL, Azure App Service, and general Azure portal/resource group operations.
  • Infrastructure as Code: Experience provisioning Azure resources with Pulumi.
  • ETL/ELT Concepts: Strong understanding of pipeline patterns, incremental loads, data validation, and troubleshooting.

Preferred Skills

  • Additional Skills (nice-to-have): R for data validation, TypeScript for Pulumi pipelines, Java/.NET for integration, Angular/Spring Boot for minor troubleshooting.

Job Tags

Contract work

Similar Jobs

Futbol Rebels

U9 Girls Soccer Coach Job at Futbol Rebels

 ...by providing a dynamic environment where they can learn, play, have fun and foster a passion for soccer. Our mission goes beyond teaching the game, as we focus on developing skills, creativity, and confidence in young athletes. We strive to create positive and transformative... 

RRD

Legal Proofreader - 3rd Shift Job at RRD

 ...come from a background in English Literature, Creative Writing, Copywriting, or Professional Editing. In this role, youll dive deep...  ....65 plus $3.00 shift differential Must have extensive experience in Editing, Writing, English Literature, Journalism... 

Barentz

Logistics Specialist Job at Barentz

 ...The Logistics Specialist is responsible and accountable for the effective management of international, cross-border and domestic shipments including but not limited to tendering, tracking, processing of shipments and dispatching Barentz resources. The Logistics Specialist... 

Confidential

Warehouse Loader Unloader - Now Hiring Job at Confidential

As a warehouse loader/unloader, you'll be crucial in handling freightloading it onto trucks and unloading it when it arrives. We're looking for someone detail-oriented, dedicated, and looking for a hands-on role. With competitive pay and plenty of opportunities for advancement... 

The Zimmerman Agency

Sr. Account Manager, Public Relations Job at The Zimmerman Agency

 ...cultural and industry trends, and anticipate opportunities to secure exposure and elevate client relevance. Oversee high-level media relations efforts, build strong media relationships, shape pitch narratives, and serve as a trusted voice in proactive and reactive...