Ditching Legacy: Scaling Modern Medallion Architectures in Databricks vs. Fabric
About this event
⚠️ Important Note:
PyData Amsterdam is transitioning to Luma. ✨
- Please subscribe to PyData Amsterdam on Luma: https://luma.com/pydataamsterdam
- And register at the Luma link for this event: https://luma.com/f8fvgdh3
What’s up, PyData Amsterdam community!
For our next meetup, we are bringing you two massive, real-world migration stories from major engineering teams who tore up the old playbook to build next-gen analytics platforms.
Mark your calendars! We are gathering at the Xomnia office in Amsterdam on July 30 to dive deep into these architectures over cold drinks and great food.
We are putting the spotlight on Holland Casino and a major public-sector organization to pull back the curtain on their architectural pivots. From open-source dbt workflows to massive PySpark-driven migrations, this event is packed with raw engineering insights you can actually use.
💡 Why you can't miss this:
- Two Ecosystems, One Goal: Compare how Databricks + dbt stacks up against Microsoft Fabric + PySpark for Medallion (Bronze/Silver/Gold) architectures.
- Zero Marketing Fluff: Both talks focus heavily on the real engineering decisions, mistakes made, and lessons learned.
- Community & Networking: Stick around at Xomnia after the talks to grab a drink, talk shop, and swap migration stories with fellow engineers.
Agenda
- 18:00 - 18:55: Welcome with food and drinks!
- 18:55 - 19:00: Xomnia intro
- 19:00 - 19:45: Talk 1: Building a Modern Lakehouse at Holland Casino by Sebastiaan de Leeuw & Don de Lange
- 19:45 - 20:00: Short break
- 20:00 - 20:45: Talk 2: Escaping the Legacy Black Box: Rebuilding a Public-Sector Data Platform with PySpark and Microsoft Fabric by Daniel Martin & Samridh Pandey
- 20:45 - 21:30: Networking + drinks
Talk 1: Building a Modern Lakehouse at Holland Casino
By Sebastiaan de Leeuw & Don de Lange
At Holland Casino, we implemented a new Data Warehouse solution in Databricks with the data models in DBT. We want to use this talk to give you insight into the steps we took to implement this solution and the impact this change has had at Holland Casino.
To increase efficiency and reduce the complexity of reporting and analytics processes we developed a modern analytics platform on Databricks with DBT at Holland Casino. The platform reduced the complexity of the modeled data and increased efficiency, security, and flexibility considerably faster than the existing approach allowed.
This session examines the architectural and engineering decisions behind the migration. We show how the Databricks components, DBT modeling, Lakeflow ingestion, and Databricks Workflows work together, where DBT and the medallion structure add the most value, and where future opportunities lie beyond what has been achieved so far.
About the speakers
Bio - Sebastiaan de Leeuw
Sebastiaan de Leeuw is a Data Architect at Holland Casino, focusing on Agentic AI, Data Architecture, Graph Technology and Data Platforms. He has worked in data and AI for close to a decade, serving as a data scientist, data engineer, solutions architect and tech lead for organizations including IKEA, Nike, Unilever, the Dutch National Police and Capgemini, before joining Holland Casino to lead the redesign of their analytics platform on Databricks and dbt.
Bio - Don de Lange
Don de Lange is a Senior Data Engineer at Xomnia, focusing on Agentic AI, Data Architecture, Data Modeling and Data Platforms. He has worked as a data engineer and cloud engineer for about 6 years now at various organizations, focusing on the public sector. Outside of work he likes to travel, try a lot of different foods, exercise, and learn new things.
Talk 2: Escaping the Legacy Black Box: Rebuilding a Public-Sector Data Platform with PySpark and Microsoft Fabric
By Daniel Martin & Samridh Pandey
A large public-sector organisation was implementing a new CRM solution based on Dynamics 365. The plan was to integrate its data into an ageing SQL data warehouse. Complex dependencies, an ageing codebase, and long lead times made that approach increasingly impractical.
The organization changed course, and built a new analytics platform on a new, modern stack: Microsoft Fabric. Using PySpark as the basis of an accelerator for ingestion, transformation and orchestration, the team was able to deliver considerably faster than the original approach allowed – meeting deadlines that were seen as impossible before.
The resulting architecture combines OneLake, Lakehouses, notebooks, pipelines and semantic models in a layered Bronze, Silver and Gold design. PySpark is used as the primary engineering vehicle throughout the production data flow, turning data from operational systems into governed datasets for reporting and analytics.
This session examines the architectural and engineering decisions behind the pivot. We show how the Fabric components work together, where PySpark adds value, and where future opportunities lie beyond what has been achieved so far.
We will also discuss the less polished parts of the journey: the constraints we encountered, the choices that did not work as expected, and what we would approach differently today.
About the speakers
Bio - Daniel Martin
Working at UWV Customer & Service as a Data/Business Analyst since late 2022 – responsible for supporting the Operations department with reporting, analysis, dashboards, modeling, and more. Prior to this, I worked at Conduent for 9 years, the majority of which was as a Data Analyst.
As a versatile analyst, I focus on various facets, including both ETL development and front-end design. My primary motivation is facilitating and enabling the workforce by helping to gain better insights.
Extensive experience in customer contact centers/call centers, familiar with COPC definitions and standards, and experienced in modeling data from various systems within a call center environment.
Bio - Samridh Pandey
As a managing solutions architect Data & AI at Capgemini, I leverage my expertise in data analytics and visualization to help clients make informed technological and strategic decisions. With over 11 years of experience in this role, I have successfully assisted various organizations in optimizing their existing data landscape and enhancing their data analytics capabilities.
Prior to joining Capgemini, I worked as an Application Developer/Consultant/ICT Specialist at Rabobank and a Senior System Engineer at Infosys, where I gained valuable experience in data analytics and business intelligence. I hold a Bachelor of Engineering degree in Computer Science from Maharaja Agrasen Institute Of Technology and multiple certifications from Aspiring Minds. My skills include data architecture, Microsoft Fabric, and data analysis. I am passionate about using data to drive business growth and innovation.
Directions:
📍 Raamstraat 7, 1016 XL Amsterdam
Just around the corner of Leidseplein!
Source: meetup