Note: The job is a remote job and is open to candidates in USA. Insomniac Design is a global digital agency headquartered in Washington D.C., specializing in human-centered design and digital transformation. As a Senior Data Engineer, you will architect and optimize scalable data platforms, enforce data governance best practices, and mentor junior engineers to ensure high-performance data solutions.
Responsibilities
- Architect and implement enterprise-grade data platforms, including data lakes, warehouses, and real-time pipelines
- Define and enforce data engineering standards, including ETL/ELT frameworks, data modeling, and quality controls
- Lead performance optimization of databases, queries, and pipelines (e.g., partitioning, indexing, cost-efficient cloud solutions)
- Stay updated with emerging trends, evaluating and integrating new technologies
- Design and implement scalable data architecture that ensures high availability, security, and performance
- Define best practices for data modeling, storage, retrieval, and governance
- Optimize database structures, schemas, indexing, and partitioning for efficient query performance
- Implement CI/CD for data pipelines and automate infrastructure using Infrastructure-as-Code (IaC) tools
- Architect and develop end-to-end ETL/ELT pipelines for structured and unstructured data ingestion, processing, and transformation
- Automate and optimize data workflows using tools like Apache Spark, Kafka, Airflow, and dbt
- Monitor, troubleshoot, and optimize data pipeline performance
- Implement data governance policies, including metadata management, lineage tracking, and role-based access control (RBAC)
- Ensure compliance with data privacy regulations (e.g., GDPR, CCPA) and enforce security best practices
- Develop frameworks for data quality validation, anomaly detection, and integrity checks
- Collaborate with stakeholders (clients, PMs) to translate business needs into technical specifications and scalable architectures
- Develop interactive dashboards and reporting solutions to provide actionable insights
- Lead technical discussions, workshops, and knowledge-sharing sessions within the team
- Mentor junior engineers, conduct code reviews, and set best practices
- Drive automation of data workflows, including CI/CD for pipelines and Infrastructure-as-Code (IaC) solutions
Skills
- 7+ years of experience in data engineering with a proven track record of building scalable data solutions
- Expertise in SQL and relational database management systems (MySQL, PostgreSQL, Oracle)
- Proficiency in Python (or similar languages) for data processing and automation
- Deep understanding of cloud platforms (AWS, GCP, Azure) and cloud-native data services: AWS (Redshift, Glue, S3), Azure (Synapse, Data Factory), GCP (BigQuery, Dataflow)
- Hands-on experience with big data tools: Apache Spark, Kafka, Airflow, dbt, Snowflake, Databricks
- Expertise in data modeling
- Strong knowledge of data security, governance, and compliance frameworks
- Proven track record of leading data projects, including migrations, warehousing, and real-time analytics
- Ability to lead cross-functional teams and manage client expectations
- Strong problem-solving skills for debugging distributed systems and resolving data anomalies
- Experience with Agile/Scrum methodologies, including tools like Jira, Git
- Excellent communication and stakeholder management skills
- Certifications in cloud/data technologies (e.g., AWS Certified Data Analytics, Google Cloud Data Engineer)
- Experience with ML pipelines (e.g., feature stores, model deployment)
- Contributions to open-source data projects
Benefits
- Health and life insurance
- 401k contribution
Company Overview