Senior Databricks Solution Engineer

BlueFlag is hiring Databricks Solutions Engineers to set the standards and patterns for a premier data and artificial intelligence (AI) platform at the Department of Veterans Affairs. The platform is moving on-premises and existing cloud workloads to Databricks in Azure Commercial, where it will serve a thousand teams building reporting, analytics, and machine learning. You will define how the platform is built and governed, build the reference implementations teams copy, and teach those teams to deliver on their own. We're looking for an engineer first: someone who built production pipelines on Databricks, then grew into setting standards and guiding customers. You'll join a friendly, collaborative team doing work that makes a real difference for Veterans and their families. What You'll Do -Design the Databricks platform foundation in Azure Commercial: workspace topology, VNet injection and Private Link, and secure connectivity to ADLS Gen2, Azure SQL, and Event Hubs, all provisioned with Terraform. -Set Unity Catalog governance: catalog and schema design, data classification, row- and column-level security, attribute-based access, lineage, and Delta Sharing. -Define lakehouse standards for medallion data flow, table layout (liquid clustering, OPTIMIZE, VACUUM), and performance (Photon, serverless). -Build reference pipelines for batch, streaming, and change data capture (CDC) with Lakeflow Declarative Pipelines, Lakeflow Jobs, Auto Loader, and Structured Streaming. -Set MLOps patterns with MLflow, the Unity Catalog model registry and feature tables, and model serving. -Set guardrails for a platform that thousands of teams share: cluster and budget policies, tagging and chargeback, monitoring, alerting, and audit logging to Azure Monitor. -Build reusable templates (Databricks Asset Bundles, CI/CD with Azure DevOps or GitHub Actions), and run workshops, office hours, and playbooks that help teams adopt them. -Work with security, networking, compliance, and VA stakeholders to agree on standards, review exceptions, and report adoption. Why Join BlueFlag At BlueFlag, we're passionate about leveraging cutting-edge technology to make a real difference. You'll be at the forefront of cloud innovation, working on projects that directly impact people's lives. We offer a high-growth, entrepreneurial environment that values fresh ideas and authentic teamwork. If you're ready to take your data engineering career to new heights and contribute to meaningful projects that push the boundaries of technology, we want to hear from you. Join BlueFlag and be part of a team that's shaping the future of cloud solutions! Requirements -12+ years in data engineering or analytics, including 5+ years on cloud data platforms (Azure preferred) and 3+ years hands-on with Databricks and Delta Lake -Proven Unity Catalog implementation with granular governance (row- and column-level security, attribute-based access, lineage) -Strong Spark skills (PySpark,…