Data Engineer

POSITION SUMMARY
The Data Engineer builds and runs the pipelines that feed Ally's data platform, so the rest of the company can trust and use its data. You'll join a new, growing Data Operations team, reporting to the Director of Data Operations and supporting teams across the business that depend on data from our CRM, TMS and product systems.
The role works primarily in Databricks, owning ingestion, transformations, pipeline monitoring and reliability. You'll be expected to move quickly while keeping strong engineering habits: version control, code review, testing and documentation. Clear written English and comfort working async with a US-based team are essential, with meaningful daily overlap with US business hours.

WORKING STYLE & EXPECTATIONS
●Operates with a high degree of independence, managing your own priorities within the direction set by the Director.
●Takes ownership of problems and fixes them before users notice.
●Treats monitoring, testing and documentation as part of the job, not an afterthought.
●Uses AI-assisted development tools to work faster without sacrificing quality.
●Explains technical issues and tradeoffs in plain language for non-technical stakeholders.

TYPICAL DEVELOPMENT WORKFLOW
This role follows an established workflow designed to keep the data platform reliable as it grows:
●Work is captured and prioritized as issues in Linear.
●All code lives in Git: branches, pull requests and code review before anything reaches production.
●New tables follow the team's medallion layering and naming standards.
●Every production pipeline ships with monitoring, quality checks and a runbook.
●Completed work closes the loop in Linear so stakeholders know what changed.

PRIMARY RESPONSIBILITIES
Ingestion & Integration
●Build and maintain ingestion from internal systems (CRM, TMS and product databases) and external vendor and public data feeds.
●Replace fragile or manual data pulls with scheduled, monitored jobs.
Pipelines & Modeling
●Build transformations from raw data into clean, documented tables.
●Follow the team's medallion layering, naming and code review standards.
●Tune slow queries and inefficient data stores.
Reliability & Monitoring
●Monitor pipelines and resolve failures before they reach users.
●Add data quality checks and flags, so users know which tables they can trust.
●Keep runbooks current, so any team member can operate a pipeline.
●Track compute cost and right-size jobs and clusters.
Support
●Deliver clean, documented data to analysts and business teams for reporting and analysis.
●Pick up backlog items as the Director prioritizes them.

REQUIRED QUALIFICATIONS
●Strong SQL and Python, including PySpark or Spark SQL.
●Production experience building and operating batch pipelines on a lakehouse or cloud warehouse.
●Data modeling for analytics: dimensional or medallion-style layers.
●Git-based workflow: branches, pull requests, code review.
●Habit of monitoring, testing and documenting what you build.
●Clear written English and comfort working async with a US team.
PREFERRED QUALIFICATIONS
●Databricks experience: Unity Catalog, Jobs, Delta Lake, Lakeflow or DLT.
●Ingesting from Salesforce, MongoDB, Postgres or REST APIs.
●AWS basics (S3, IAM, Lambda).
●Experience with AI-assisted or agentic coding tools (Claude Code, Cursor, Copilot or similar).
●Experience in logistics, freight or another operations-heavy industry.

SUCCESS MEASURES
●Pipeline reliability: scheduled jobs succeed on time, and failures are caught and fixed before users report them.
●Data quality: key tables pass automated checks, and other tables carry accurate quality flags.
●Throughput: backlog items delivered at a steady pace with little rework.
●Operability: every pipeline has a runbook and can be run by someone else.
●Cost: compute spend tracked and trending down per job as jobs are tuned.