Senior Data Engineer (Python / Databricks)
ABOUT THE MISSIONRelitix is currently seeking a hands-onSenior Data Engineer to lead a critical transformation of our data platform. We are moving away from legacy structures and re-engineering our ecosystem into a modern, Python-scripted Lakehouse architecture. This is not a maintenance role; it is a builder role for an engineer who values code-first engineering over GUI tools. If you are energized by owning systems end-to-end and want to build a platform that will scale significantly, we want to meet you.You will own the mission of migrating ADF-based pipelines to Databricks, ensuring internal BI data is accurate, and establishing secure governance via Unity Catalog. WHO YOU AREWe don’t just list responsibilities; we define clear outcomes. To achieve them, we look for specific traits in our engineers. These are the principles that will guide your success here:Code-First Mindset | Deep Ownership | Automation First | Governance Minded | Builder EnergyThe purpose of theSenior Data Engineer at Relitix is to modernize core architecture and secure the platform while ensuring reliable downstream BI.You don’t just bolthires problems; you fix them. Job Responsibilities• Modernize Core Architecture: Lead the full migration of priority pipelines into modular Python/Spark jobs running in Databricks, moving away from legacy ADF dependencies. • Optimize Performance: Re-architect pipelines to significantly reduce runtime and compute costs through partitioning, caching, and optimized orchestration. • Stabilize Data Ingestion: Architect and deploy robust Python connectors for API integrations (MLS, CRM, operational systems) with consistent logging and error handling.• Elevate Data Quality: Take ownership of internal BI datasets, implementing automated validation tests to ensure dashboards and operational metrics remain consistent and reliable. • Secure the Platform: Establish a governance foundation by implementing RBAC roles and Unity Catalog controls across bronze, silver, and gold data layers. Job Requirements• Strong Python Engineering: Strong Python engineering skills for ETL/ELT with a preference for writing clean, maintainable code over using drag-and-drop GUI tools.• Spark/Databricks Expertise: Experience with PySpark and SparkSQL inside Databricks, comfortable designing and building out bolthires Lake and medallion architectures. • Governance Knowledge: Understanding of the importance of governance with exposure to Unity Catalog or similar frameworks for managing data permissions. • Automation Mindset: Approach problems with an automation-first mindset, utilizing Git-based workflows and bolthires/CD principles. • Accountability: High sense of ownership and accountability to ensure reliable downstream BI.Preferred Qualifications• Experience with data quality tools (e.g., Great Expectations). • Familiarity with MLflow or AI-ready pipeline structures. • Prior experience in multi-tenant SaaS data environments. • Madison, WI, Milwaukee, WI or Chicago IL area preferredJob Types: Full-time, PermanentPay: $90,000.00 - $115,000.00 per yearBenefits:• 401(k)• Dental insurance• Flexible schedule• Health insurance• Health savings account• Paid time off• Vision insuranceWork Location:Remote Apply tot his job