Position Summary
The Director of Data Engineering is responsible for building and operating the data engineering function within Ryan Specialty’s corporate data organization. This role is accountable for the silver and gold layer of the medallion data architecture thus transforming raw ingested data into clean, conformed, and business-ready data assets that serve as the foundation for downstream consumption. Beyond technical delivery, this role builds and scales a high-performing engineering team, establishes disciplined software development practices, and manages the contractor and external parties that augment internal capacity.
An ideal candidate should be collaborative and dynamic and be able to work across the organization on a fast-paced and quickly evolving team.
The leader will have five primary responsibilities:
1. Own and deliver the silver and gold layer of the medallion data architecture
2. Establish and enforce a standardized software development lifecycle for all data engineering work
3. Build, mentor, and develop a high-performing data engineering team
4. Partner with IT on data platform infrastructure, tooling, and integration
What will your job entail?
- Own the Databricks medallion architecture from silver forward: transformation patterns, data quality enforcement, business logic application, gold-layer curation, and pipeline orchestration
- Establish and enforce a standardized SDLC that all team members must follow: version control, code review, testing, CI/CD, and documentation
- Design and implement a data quality framework in partnership with the VP of Data Governance who defines it from a business perspective
- Lead Master Data Management (MDM) technical implementation: entity resolution pipeline design, match-and-merge workflow configuration, golden record integration into the gold layer, and performance optimization
- Serve as the primary interface between the data organization and the IT data platform team
- Translate engineering requirements into platform specification across Azure and Databricks for the IT data platform team to deliver
- Conduct structured code reviews that are explicitly developmental for data engineers
- Build a library of engineering standards, patterns, and examples
- Own the technical planning for contractor-staffed project delivery pods
- Design and maintain the employee and contractor onboarding process
- Provide ongoing technical oversight for all data engineers
- Own technical compliance enforcement in the Databricks environment: data lineage tracking via Unity Catalog, access audit logs, encryption verification, and contractor access governance
- Hire, mentor, develop, and assess a team of data engineers and contractors
Position Qualifications:
- Bachelor’s degree in computer science, information systems, engineering, or a related technical field
- 7+ years in data engineering with at least 3 years in leadership or management role
- Proven experience owning large-scale data engineering development in a cloud-native environment
- Deep expertise in Azure Databricks, including Unity Catalog, Delta Lake, Delta Live Tables, PySpark, Spark SQL, Databricks SQL, Jobs/Workflows, and cluster management
- Hands-on experience with data quality frameworks, including implementing quality gates, monitoring pipelines, and designing alerting systems
- Strong SQL skills with experience in large-scale dimensional modeling (star schema, slowly changing dimensions, multi-source fact table design)
- Experience with cloud data lake architectures on Azure (ADLS Gen2, Azure Data Factory, Azure DevOps, Azure Key Vault). AWS experience is acceptable if Databricks experience is strong
- Proficiency in Python for data engineering (PySpark, pandas, orchestration scripts, SDK integrations)
- Hands-on experience with CI/CD pipelines for data (Azure DevOps or equivalent); comfortable reviewing infrastructure-as-code, deployment scripts, and PR pipelines
- Demonstrated ability to push back on senior stakeholders and enforce process discipline
- Track record of building and scaling data engineering teams in a high-growth environment: establishing code review norms, architecture decision record, knowledge-sharing sessions, and technology onboarding programs
- Experience managing contractors and external development partners
- Experience in the insurance industry, particularly E&S lines preferred; or a strong willingness to learn
- Ability to understand and communicate with technical and business experts
Unique Skill Requirements:
- Intellectual curiosity with a strong drive to learn
- Critical thinker with an open mind to innovative solutions
- High emotional intelligence
- Exceptional communication skills
- Strong collaboration and relationship building skills
- Ability to navigate ambiguity
- Highly structured in execution
- Skilled at simplifying complex environments
- *****Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment visa at this time*****
Ryan Specialty is an Equal Opportunity Employer. We are committed to building and sustaining a diverse workforce throughout the organization. Our vision is an inclusive and equitable workplace where all employees are valued for and evaluated on their performance and contributions. Differences in race, creed, color, religious beliefs, physical or mental capabilities, gender identity or expression, sexual orientation, and many other characteristics bring together varied perspectives and add value to the service we provide our clients, trading partners, and communities. This policy extends to all aspects of our employment practices, including but not limited to, recruiting, hiring, discipline, firing, promoting, transferring, compensation, benefits, training, leaves of absence, and other terms, conditions, and benefits of employment.
How We Support Our Teammates
Ryan Specialty seeks to offer our employees a comprehensive and best-in-class benefits package that helps them — and their family members — achieve their physical, financial, and emotional well-being goals. In addition to paid time off for company holidays, vacation, sick and personal days, Ryan offers paid parental leave, mental health services and more.
The target salary range for this position is $132,000.00 - $165,000.00 annually.
The wage range for this role considers many factors, such as training, transferable skills, work experience, licensure and certification, business needs, and market demands. The pay range is subject to change and may be modified in the future. Full-time roles are eligible for bonuses and benefits. For additional information on Ryan Specialty Total Rewards, visit our website https://benefits.ryansg.com/.
We provide individuals with disabilities reasonable accommodations to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment in accordance with applicable law. Please contact us to request an accommodation at HR@Ryansg.com
The above is intended to describe this job's general requirements. It is not to be construed as an exhaustive statement of duties, responsibilities, or physical requirements. Nothing in this job description restricts management's right to assign or reassign duties and responsibilities to this job at any time. Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions.
