Note: The job is a remote job and is open to candidates in USA. Trulieve is one of the fastest growing companies in the nation, dedicated to providing patients with trusted products for their medical needs. They are seeking a Senior Data Engineer to design and implement Snowflake-native data architectures and lead the development of complex data pipelines, ensuring data security and governance standards are met.
Responsibilities
- Design and Implement Snowflake-Native Data Architectures
- Lead the creation and optimization of modern data architectures on Snowflake, including multi-layer pipelines (bronze/silver/gold), real-time streaming ingestion, and governed analytics layers
- Ensure that the architecture supports high-volume data workloads, near-real-time freshness requirements, and integrates seamlessly with upstream operational systems and downstream analytics consumers
- Lead the Development of Complex, End-to-End Data Pipelines Using Native Snowflake Services
- Architect and build data pipelines using Dynamic Tables for declarative SQL-based transformations with automated dependency management and incremental refresh
- Implement real-time and near-real-time ingestion using Snowpipe and Snowpipe Streaming
- Design event-driven and procedural pipeline logic using Streams and Tasks for complex orchestration scenarios (MERGE, SCD patterns, external function calls)
- Leverage Snowpark (Python) for advanced transformations that require procedural logic beyond SQL
- Collaborate with Data Scientists, Analysts, and Other Stakeholders
- Work closely with analytics and data science teams to understand data requirements and translate them into scalable, performant Snowflake solutions
- Build and maintain Semantic Views and governed consumption layers that provide self-service access to curated data
- Provide technical guidance on Snowflake best practices for data usage, cost optimization, and warehouse sizing
- Ensure Data Security, Governance, and Compliance Standards Are Met
- Implement and manage Snowflake data governance features including Dynamic Data Masking, Row Access Policies, Object Tagging, and Data Classification
- Establish and maintain data governance frameworks ensuring data quality and compliance with relevant regulations (SOX, GDPR, HIPAA)
- Manage role-based access control (RBAC), data sharing, and cross-account governance using Snowflake's native security model
- Deep Understanding of Data Engineering Concepts
- Extensive knowledge of data modeling, including designing and maintaining relational, dimensional, and semi-structured data models within Snowflake
- Proficiency in data warehousing concepts with hands-on experience designing multi-layer transformation pipelines (staging, intermediate, marts)
- Strong understanding of incremental processing patterns, change data capture (CDC), and slowly changing dimension (SCD) strategies
- Expertise in Snowflake Platform and Native Data Pipeline Services
- Deep proficiency with Snowflake Dynamic Tables (TARGET_LAG, REFRESH_MODE, pipeline dependency graphs, incremental vs. full refresh)
- Hands-on experience with Snowpipe and Snowpipe Streaming for continuous and real-time data ingestion
- Strong knowledge of Snowflake Streams and Tasks for event-driven and procedural pipeline orchestration
- Experience with Snowpark (Python/Scala) for complex data transformations and UDFs/UDTFs
- Familiarity with Snowflake Cortex AI functions for embedding AI/ML capabilities into data pipelines
- Proficiency in SQL, Python, and Data Engineering Frameworks
- Advanced SQL skills including window functions, CTEs, recursive queries, semi-structured data handling (VARIANT, OBJECT, ARRAY), and performance optimization
- Python proficiency for Snowpark development, automation scripting, and integration work
- Experience with modular SQL transformation development, testing, and documentation using native Snowflake patterns and shared engineering standards
- Familiarity with orchestration platforms such as Apache Airflow / Astronomer for pipeline scheduling and monitoring
- Experience with Real-Time Data Processing and Streaming Architectures
- Hands-on experience designing solutions for real-time and near-real-time data pipelines using Snowpipe Streaming and Kafka connectors
- Understanding of event-driven architectures and their integration with Snowflake's continuous data pipeline features
- Experience with change data capture (CDC) patterns and tools (Debezium, Fivetran, custom CDC)
- Strong Knowledge of Cloud Infrastructure and Cost Optimization
- Expertise in cloud platforms (AWS, Azure, or GCP) with a focus on integration with Snowflake (external stages, storage integrations, PrivateLink)
- Experience with Snowflake cost management including warehouse sizing strategies, auto-suspend/resume, resource monitors, and query optimization
- Familiarity with infrastructure-as-code tools (Terraform, Pulumi) for managing Snowflake resources declaratively
- Lead the Technical Design of New Projects
- Responsible for making critical decisions regarding Snowflake architecture patterns, product direction, and delivery tradeoffs, including when to use Dynamic Tables, Streams/Tasks, Materialized Views, and external orchestration based on business requirements and user needs
- Develop and enforce best practices for pipeline development, testing, deployment, and monitoring within the team
- Design and document data contracts and SLAs for pipeline freshness (TARGET_LAG), data quality, and availability
- Mentor and Develop Junior Engineers
- Provide technical leadership and mentorship to junior and mid-level engineers, fostering a culture of continuous learning and improvement
- Lead code reviews, pair programming sessions, and technical workshops focused on Snowflake-native development patterns and engineering best practices
- Create reusable patterns, templates, and documentation for common pipeline scenarios (CDC ingestion, SCD management, real-time aggregations)
- Operate as a Product Minded Data Engineering Leader
- Partner with business stakeholders, analytics teams, and engineering leaders to shape the roadmap for data products and platform capabilities based on business value, user needs, and operational priorities
- Translate ambiguous business problems into clearly defined product requirements, success metrics, delivery plans, and prioritized engineering work
- Own the lifecycle of key data products and shared platform services, including intake, prioritization, stakeholder alignment, adoption, and continuous improvement
- Continuously Evaluate and Improve Existing Systems
- Regularly review current data systems to identify opportunities for migration to modern patterns (for example, migrating legacy Streams/Tasks to Dynamic Tables where appropriate)
- Monitor pipeline health, refresh performance, and cost efficiency using Snowflake's INFORMATION_SCHEMA, ACCOUNT_USAGE, and alerting capabilities
- Implement optimizations including incremental refresh tuning, warehouse right-sizing, and query performance improvements
Skills
- Typically requires 7+ years of experience in data engineering or related fields
- Proven track record of designing and implementing large-scale data solutions on Snowflake or similar cloud data platforms
- 2+ years of hands-on experience with Snowflake-native pipeline services (Dynamic Tables, Streams/Tasks, Snowpipe)
- Deep Understanding of Data Engineering Concepts
- Extensive knowledge of data modeling, including designing and maintaining relational, dimensional, and semi-structured data models within Snowflake
- Proficiency in data warehousing concepts with hands-on experience designing multi-layer transformation pipelines (staging, intermediate, marts)
- Strong understanding of incremental processing patterns, change data capture (CDC), and slowly changing dimension (SCD) strategies
- Expertise in Snowflake Platform and Native Data Pipeline Services
- Deep proficiency with Snowflake Dynamic Tables (TARGET_LAG, REFRESH_MODE, pipeline dependency graphs, incremental vs. full refresh)
- Hands-on experience with Snowpipe and Snowpipe Streaming for continuous and real-time data ingestion
- Strong knowledge of Snowflake Streams and Tasks for event-driven and procedural pipeline orchestration
- Experience with Snowpark (Python/Scala) for complex data transformations and UDFs/UDTFs
- Familiarity with Snowflake Cortex AI functions for embedding AI/ML capabilities into data pipelines
- Proficiency in SQL, Python, and Data Engineering Frameworks
- Advanced SQL skills including window functions, CTEs, recursive queries, semi-structured data handling (VARIANT, OBJECT, ARRAY), and performance optimization
- Python proficiency for Snowpark development, automation scripting, and integration work
- Experience with modular SQL transformation development, testing, and documentation using native Snowflake patterns and shared engineering standards
- Familiarity with orchestration platforms such as Apache Airflow / Astronomer for pipeline scheduling and monitoring
- Experience with Real-Time Data Processing and Streaming Architectures
- Hands-on experience designing solutions for real-time and near-real-time data pipelines using Snowpipe Streaming and Kafka connectors
- Understanding of event-driven architectures and their integration with Snowflake's continuous data pipeline features
- Experience with change data capture (CDC) patterns and tools (Debezium, Fivetran, custom CDC)
- Strong Knowledge of Cloud Infrastructure and Cost Optimization
- Expertise in cloud platforms (AWS, Azure, or GCP) with a focus on integration with Snowflake (external stages, storage integrations, PrivateLink)
- Experience with Snowflake cost management including warehouse sizing strategies, auto-suspend/resume, resource monitors, and query optimization
- Familiarity with infrastructure-as-code tools (Terraform, Pulumi) for managing Snowflake resources declaratively
- Lead the Technical Design of New Projects
- Responsible for making critical decisions regarding Snowflake architecture patterns, product direction, and delivery tradeoffs, including when to use Dynamic Tables, Streams/Tasks, Materialized Views, and external orchestration based on business requirements and user needs
- Develop and enforce best practices for pipeline development, testing, deployment, and monitoring within the team
- Design and document data contracts and SLAs for pipeline freshness (TARGET_LAG), data quality, and availability
- Mentor and Develop Junior Engineers
- Provide technical leadership and mentorship to junior and mid-level engineers, fostering a culture of continuous learning and improvement
- Lead code reviews, pair programming sessions, and technical workshops focused on Snowflake-native development patterns and engineering best practices
- Create reusable patterns, templates, and documentation for common pipeline scenarios (CDC ingestion, SCD management, real-time aggregations)
- Operate as a Product Minded Data Engineering Leader
- Partner with business stakeholders, analytics teams, and engineering leaders to shape the roadmap for data products and platform capabilities based on business value, user needs, and operational priorities
- Translate ambiguous business problems into clearly defined product requirements, success metrics, delivery plans, and prioritized engineering work
- Own the lifecycle of key data products and shared platform services, including intake, prioritization, stakeholder alignment, adoption, and continuous improvement
- Continuously Evaluate and Improve Existing Systems
- Regularly review current data systems to identify opportunities for migration to modern patterns (for example, migrating legacy Streams/Tasks to Dynamic Tables where appropriate)
- Monitor pipeline health, refresh performance, and cost efficiency using Snowflake's INFORMATION_SCHEMA, ACCOUNT_USAGE, and alerting capabilities
- Implement optimizations including incremental refresh tuning, warehouse right-sizing, and query performance improvements
- SnowPro Advanced: Data Engineer, SnowPro Core, or AWS/Azure/GCP data engineering certifications
- Continuous learning through relevant certifications and training to stay current with Snowflake platform releases and modern data engineering practices
- Familiarity with Snowflake's release cycle and ability to evaluate new features (for example, Cortex AI, Document AI, Iceberg Tables) for team adoption
Benefits
- A comprehensive benefits package including paid time off
Company Overview