Note: The job is a remote job and is open to candidates in USA. CloudTech Innovations is seeking a highly experienced Solution Architect with a strong background in building and scaling cloud-native data platforms. This role focuses on Databricks and involves collaborating with cross-functional teams to design and implement well-architected data solutions across enterprise environments.
Responsibilities
• Design and lead implementation of end-to-end Databricks Lakehouse Platforms using Delta Lake, Delta Live Tables, and MLflow
• Architect Medallion Architecture (Bronze/Silver/Gold) for structured, semi-structured, and streaming workloads
• Implement governed Lakehouse patterns using Unity Catalog for access control, lineage, data classification, and secure sharing
• Build scalable ETL/ELT pipelines using Databricks Notebooks, Workflows, SQL Warehouses, and Spark-based transformations
• Develop real-time streaming pipelines with Auto Loader, Structured Streaming, and event-driven platforms (Kafka, Kinesis, Pub/Sub)
• Integrate Databricks with cloud-native services such as AWS Glue, Azure Data Factory, and GCP Dataform
• Define distributed integration patterns using REST APIs , microservices, and event-driven architectures
• Enforce data governance, RBAC/ABAC, encryption, secret management , and compliance controls
• Optimize Delta Lake tables, Spark workloads, and cluster configurations using Photon and autoscaling patterns
• Drive cloud cost optimization across storage, compute, and workflow orchestration
• Participate in architecture reviews, set standards, and support engineering teams throughout execution
• Stay current on Databricks capabilities including Unity Catalog updates, Lakehouse Federation, serverless compute, and AI/ML features
Skills
• Bachelor's or Master's degree in Computer Science, Data Engineering, or related field
• 10+ years of experience in enterprise software, cloud architecture, or data engineering roles
• Strong hands-on experience with Databricks, Apache Spark, Delta Lake, and Lakehouse platform design
• Experience implementing and administering Unity Catalog for governance, lineage, and fine-grained access control
• Experience designing Medallion Architecture for analytics and engineering workloads
• Hands-on experience with cloud platforms such as AWS, Azure, or GCP, including storage, compute, and networking services
• Experience with streaming technologies such as Kafka, Kinesis, or Pub/Sub
• Strong understanding of data modeling, workflow orchestration (Airflow, Databricks Workflows, dbt), and pipeline automation
• Familiarity with Scala-based Spark workloads in addition to PySpark and SQL pipelines
• Skilled in performance tuning, Spark optimization, cluster policies, and cloud cost management
• Excellent communication skills for technical leadership and stakeholder collaboration
• Certifications in Databricks, AWS/GCP/Azure Solution Architecture, or TOGAF are a plus
Company Overview
• Comprehensive Staffing Solutions for Specialized Engineering Roles: Software Development Engineering: We offer staffing solutions with skilled software engineers proficient in various programming languages and technologies. It was founded in 2023, and is headquartered in Dallas, TX, US, with a workforce of 51-200 employees. Its website is https://cloudtech-innovations.com.