Medallion architecture is the go-to for handling analytical data, particularly with the prominence of lakehouses in Databricks and Fabric. Whilst the layers are well defined, their definitions don’t tell you where particular tasks belong. So here I wanted to present my view from a more practical perspective of what goes into each layer. Layering Let’s start by […]
We recently dived into Spark and saw how DataFrames use lazy evaluation to defer execution until needed. This is useful for defining segments of logic to be reused, but if this causes large, complex calculations to execute multiple times, there may be a better way. Lazy evaluation Let’s take an example with some sales data: The display triggers execution […]
I’ve spent years shaping data with SQL Server, however after pulling at the threads of Fabric I’m opening notebooks and finding PySpark. At first glance the difference is stark, but it’s not quite the dramatic shift it appears. If you’re not familiar, let’s look at what’s very similar, and where the true differences are. Same data, different […]
I’ve had Fabric gnawing at me for too long. Whilst I’ve dabbled in a couple of areas, I haven’t taken the time to explore the breadth of what Microsoft’s flagship data platform offers. That is, until I recently took the plunge. For those who may be in the same position, I wanted to share the […]
When developing Power BI models, we don’t always want to refresh the entire dataset before we start working with the data. We can speed up the refresh – and therefore our development – by reducing the volume we’re working with. Here we’ll look at a couple of ways to use parameters to adapt production-ready models to reduce […]
Last week we looked at utilising Teams to make notifications more collaborative. This week I want to show how we can add more visual and interactive elements by adopting Adaptive Cards. What are Adaptive Cards? Adaptive Cards are structured messages which can be used across various Microsoft services as alternatives to plain text. They use JSON […]
Whether running Data Factory, Synapse, or Fabric pipelines, things go wrong – and the de facto response is to send an email. We’ve looked at sending emails from pipelines before, but at scale they can become noise and are easy to ignore. A more effective option is to surface alerts where collaboration already exists, such as Teams. […]
If you’ve tried to get header rows to repeat or scroll as you move through your paginated reports, you may have felt the frustration of getting them working correctly. Whether it’s Reporting Services, Power BI, or Fabric, the flavour doesn’t matter. The properties on the tablix should provide the functionality, but they’re not reliable. In this post […]
When using Power BI or Fabric workspaces to browse reports, we’re greeted with a list of items and their attributes. While attention is given to report visuals such as bars, pies, candles, and RAG highlights, the surrounding experience is neglected. When it comes to consumption, the standard interface falls short. Apps fill this gap. They’ve […]
