[Remote] Data Platform Engineer
Note: The job is a remote job and is open to candidates in USA. Lucid Services Group is seeking an experienced Data Platform Lead to design, build, and scale a modern cloud-based data platform. This role involves overseeing the architecture, development, governance, and operational excellence of the client's data ecosystem while collaborating with various teams to meet data requirements.
Responsibilities
- Define and execute the organization's data platform roadmap and architecture
- Design scalable ELT/ETL frameworks leveraging dlt for data ingestion and pipeline development
- Establish standards, best practices, and governance frameworks for data engineering and platform operations
- Lead platform modernization initiatives and cloud data architecture improvements
- Build and maintain robust data ingestion pipelines using dlt (Python)
- Develop scalable transformation layers using dbt, including data modeling, testing, and documentation
- Design and optimize data warehouse solutions in Snowflake
- Implement workflow orchestration, monitoring, and dependency management using Dagster
- Ensure data quality, lineage, observability, and reliability across all platform components
- Lead and mentor a team of data engineers
- Conduct architecture reviews and code reviews
- Promote engineering best practices including CI/CD, testing, version control, and infrastructure as code
- Collaborate with leadership to prioritize platform investments and technical initiatives
- Monitor platform performance, reliability, and cost optimization
- Implement observability and alerting frameworks for pipelines and platform services
- Establish SLAs and operational processes for production support
- Drive continuous improvement in data platform scalability and resilience
- Partner with Analytics, Data Science, Product, and Business teams to understand data requirements
- Translate business needs into scalable technical solutions
- Support self-service analytics and data product development initiatives
- Lead the design and implementation of enterprise-scale data warehouse and data lakehouse architectures
- Define and enforce enterprise data modeling standards, including dimensional, normalized, and data vault methodologies where appropriate
- Collaborate with business and analytics teams to translate business processes into scalable, maintainable data models
- Design conformed dimensions, fact tables, and semantic layers to support self-service analytics and reporting
- Ensure consistency, governance, lineage, and reusability across enterprise data assets
Skills
- Significant experience designing and implementing Enterprise Data Warehouses (EDW)
- Deep expertise in data modeling, data architecture, and Snowflake platform engineering
- Experience defining strategic data architecture and working hands-on within Snowflake to optimize performance, scalability, governance, and cost
- Experience building and maintaining robust data ingestion pipelines using dlt (Python)
- Experience developing scalable transformation layers using dbt, including data modeling, testing, and documentation
- Experience designing and optimizing data warehouse solutions in Snowflake
- Experience implementing workflow orchestration, monitoring, and dependency management using Dagster
- Experience ensuring data quality, lineage, observability, and reliability across all platform components
- Experience leading and mentoring a team of data engineers
- Experience conducting architecture reviews and code reviews
- Experience promoting engineering best practices including CI/CD, testing, version control, and infrastructure as code
- Experience collaborating with leadership to prioritize platform investments and technical initiatives
- Experience monitoring platform performance, reliability, and cost optimization
- Experience implementing observability and alerting frameworks for pipelines and platform services
- Experience establishing SLAs and operational processes for production support
- Experience driving continuous improvement in data platform scalability and resilience
- Experience partnering with Analytics, Data Science, Product, and Business teams to understand data requirements
- Experience translating business needs into scalable technical solutions
- Experience supporting self-service analytics and data product development initiatives
- Experience leading the design and implementation of enterprise-scale data warehouse and data lakehouse architectures
- Experience defining and enforcing enterprise data modeling standards, including dimensional, normalized, and data vault methodologies where appropriate
- Experience collaborating with business and analytics teams to translate business processes into scalable, maintainable data models
- Experience designing conformed dimensions, fact tables, and semantic layers to support self-service analytics and reporting
- Experience ensuring consistency, governance, lineage, and reusability across enterprise data assets
Company Overview
Company H1B Sponsorship