Your Fabric Bill Explained in Euros: 6 Hidden Costs Dutch SMEs Keep Paying For

Scenario: The Logistics SME with a Quietly Expensive Lakehouse

Take a mid‑sized logistics company running Fabric in an F2 or F4 capacity:

• One lakehouse with ~200 GB of parquet data.
• A Dataflows Gen2 pipeline that copies order data from Azure SQL every hour.
• A second pipeline landing CSV files from an SFTP drop.
• A semantic model in Power BI Direct Lake on top of the lakehouse.

The team thinks they’re careful:

• Pipelines are simple copy activities.
• Capacity is sized modestly.
• No machine learning, no crazy transformations.

Yet every month, the Fabric bill shows a persistent extra few hundred euros in capacity usage. The culprits are mostly:

• Always‑on hourly copy jobs, including at night when nobody uses the reports.
• Full refreshes of large tables instead of incremental loads.
• A capacity that never pauses, even on quiet weekends.

We’ll walk through six concrete cost drivers in this scenario and the fixes that actually move the needle.

1. Hourly Copy Jobs: The €300–400 Per Month Leak

Fabric billing for Data Factory (pipelines and Dataflows Gen2) is tied to:

• Activity runtime (compute) on your Fabric capacity.
• Data volume processed (rows/bytes moved and transformed).

In our logistics scenario, the team has:

• An hourly pipeline that copies the full Orders table from Azure SQL.
• Another hourly pipeline that ingests all CSV files dropped in a day.

Those jobs run:

• 24 hours a day, 7 days a week.
• Even when the warehouse is closed and nobody opens the reports.

If each run scans tens of millions of rows and writes to Delta tables, that’s a non‑trivial amount of capacity time. Multiply by 24 runs per day and 30+ days per month, and you get exactly the “mysterious” €300–400 leak.

How to See the Damage

You can quantify this in Fabric:

• Go to Admin portal → Capacity settings → Metrics.
• Filter by workload (Data Factory, Data Engineering).
• Look at activity runtime and peak utilization by hour.

You’ll usually see:

• Clear daytime peaks (08:00–18:00).
• A surprisingly high baseline at night driven by hourly copy jobs.

If your capacity is often above recommended utilization thresholds at night, you’re paying for compute nobody is using.

2. Night‑Time Pausing: Turning Off the Meter When Nobody’s Driving

Fabric capacities can be paused and resumed on demand. For pay‑as‑you‑go SKUs, pausing stops compute charges while keeping your artifacts (lakehouses, warehouses, semantic models) intact. Storage continues to accrue, but that’s usually the cheaper part.

For our SME, the realistic usage pattern is:

• Business hours: roughly 07:00–19:00 on weekdays.
• Light usage: Saturday mornings.
• Near zero usage: nights and most of Sunday.

Practical Pausing Pattern

A simple pattern that teams actually live with:

• Weekdays:
  • Resume capacity at 06:30.
  • Pause capacity at 20:00.
• Weekends:
  • Resume at 08:00 on Saturday, pause at 14:00.
  • Keep capacity paused all day Sunday.

This only works if you:

• Move non‑critical pipelines out of the paused window.
• Accept that any scheduled refresh during pause will fail and must be rescheduled.

Automating Pauses with PowerShell

You can automate pausing/resuming with Azure PowerShell and a scheduled task or Azure Automation. The key is using the Fabric capacity management APIs.

Example PowerShell script to pause a capacity by ID:

# Requires Az.Accounts and Az.Resources modules

param(
    [Parameter(Mandatory = $true)]
    [string]$SubscriptionId,

    [Parameter(Mandatory = $true)]
    [string]$ResourceGroupName,

    [Parameter(Mandatory = $true)]
    [string]$CapacityName
)

Connect-AzAccount
Select-AzSubscription -SubscriptionId $SubscriptionId

$capacity = Get-AzResource -ResourceGroupName $ResourceGroupName -Name $CapacityName -ResourceType "Microsoft.Fabric/capacities"

if (-not $capacity) {
    throw "Capacity not found."
}

$capacity.Properties.state = "Paused"

Set-AzResource -ResourceId $capacity.ResourceId -Properties $capacity.Properties -Force
Write-Host "Capacity $CapacityName paused."

You’d pair this with a matching script that sets state = "Active" and schedule both via:

• Windows Task Scheduler calling PowerShell.
• Azure Automation runbooks.

Effect on our scenario:

• Night‑time hourly copy jobs no longer run (they’d fail while paused).
• You reschedule those jobs to a slightly lower frequency during business hours.
• The capacity’s utilization baseline drops sharply, and that regular €300–400 line item shrinks.

3. Incremental Loads vs Full Refresh: Stop Re‑Copying the Lake

The second big leak is full refresh behavior:

• Pipelines copy the entire Orders table every hour.
• Dataflows Gen2 reload all rows from source tables.

Fabric supports incremental patterns both in pipelines and Dataflows Gen2. You don’t get automatic change tracking for every source, but you can design incremental behavior using:

• Watermark columns (e.g. LastModifiedDate).
• Change tracking or CDC where your source supports it.

Incremental Load in Dataflows Gen2

In Dataflows Gen2, you can:

1. In the dataflow settings, enable Incremental refresh for a table.
2. Define:
   • A date/time column used for partitioning (e.g. LastModifiedDate).
   • Refresh ranges (e.g. last N days) and store ranges.

This reduces:

• The amount of data scanned on each run.
• The compute time in your capacity.

Incremental Load in Pipelines with SQL Source

For the hourly Azure SQL copy, a typical pattern is:

• Keep a watermark (max LastModifiedDate) in a small control table or a Fabric lakehouse/warehouse.
• On each run, query only rows where LastModifiedDate is greater than the stored watermark.

Example SQL query used in a Copy activity source (for Azure SQL Database):

DECLARE @LastWatermark datetime2 = @WatermarkValue; -- passed via pipeline parameter

SELECT OrderId,
       CustomerId,
       OrderDate,
       LastModifiedDate,
       Status
FROM dbo.Orders
WHERE LastModifiedDate > @LastWatermark;

You then:

• Pass @WatermarkValue into the query via a pipeline parameter.
• After a successful copy, write back the new max LastModifiedDate to your watermark store.

Effect on our scenario:

• Overnight, only new or changed orders are copied.
• The hourly job reads thousands of rows instead of millions.
• Capacity runtime for Data Factory drops significantly.

4. Over‑Refreshing Power BI Semantic Models

Even with Direct Lake, semantic models have refresh behavior that costs compute:

• Incremental refresh policies partition data and reload only recent partitions.
• Full refreshes scan all partitions and can be expensive.

In the logistics scenario, the team configured:

• A semantic model with incremental refresh on OrderDate.
• Scheduled refresh every hour, 24/7.

This is often unnecessary:

• Direct Lake already reads the latest data from the Delta tables for most standard tables and measures.
• Frequent refresh is only needed if you rely on calculated tables or certain transformations that must be materialized.

Practical Refresh Schedule

For most SME workloads, a realistic pattern is:

• Business‑critical report:
  • Refresh every 15–30 minutes during business hours.
  • Pause refresh at night.
• Operational report:
  • Refresh every 1–2 hours on weekdays.
  • No weekend refresh unless required.

Configure this in the Power BI workspace (Fabric) under Datasets → Scheduled refresh:

• Set multiple time windows rather than a blanket hourly refresh.
• Disable refresh on days where the capacity is paused.

Effect on our scenario:

• The semantic model stops hammering the capacity at night.
• Daytime refreshes still keep data fresh enough for operations.

5. Data Engineering Jobs Left Running Too Long

Notebook and Spark job usage is another quiet cost driver:

• Interactive notebooks that stay attached to a running session.
• Spark jobs that run with more executors than needed.

Fabric’s Data Engineering workload charges based on:

• The compute used by Spark sessions and jobs.
• The duration those sessions are active.

In our SME scenario, the data engineer might:

• Leave a notebook session open after testing a transformation.
• Use a larger Spark configuration than necessary.

Practical Controls

1. Short‑lived jobs:

   • Convert exploratory notebooks into parameterized jobs.
   • Schedule them to run once, then stop.

2. Session discipline:

   • Explicitly stop sessions after use.
   • Avoid leaving notebooks connected overnight.

3. Right‑sizing Spark:

   • Use default configurations for moderate data volumes.
   • Only increase executor counts when you have measured a bottleneck.

These changes don’t alter functionality but prevent background compute that quietly inflates the bill.

6. Capacity Size and Workload Mix: Paying for Peaks You Don’t Need

The last hidden cost is capacity sizing itself.

Fabric capacities have:

• A fixed hourly price per SKU.
• A maximum throughput they can handle comfortably.

Our logistics SME picked an F4 capacity “for safety” and then:

• Runs pipelines, Dataflows Gen2, Spark, and Power BI on the same capacity.
• Sees daytime peaks, but night‑time usage is low if pausing is configured.

If you’ve implemented:

• Night‑time pausing.
• Incremental loads.
• Sensible refresh schedules.

You might find that:

• Peak utilization is well below what F4 can handle.
• An F2 capacity would be sufficient.

How to Decide

Use the capacity metrics:

• Look at CPU and memory utilization during your busiest hour.
• If you’re consistently far below recommended thresholds, consider downsizing.

When you change capacity size:

• Confirm all workloads still meet SLAs (pipeline durations, report responsiveness).
• Adjust job concurrency if needed.

Effect on our scenario:

• After optimization, the team moves from F4 to F2.
• Combined with pausing and incremental loads, the monthly bill drops significantly without sacrificing reliability.

Putting It Together: A Before/After for the Logistics Team

Before:

• Hourly full‑table copy jobs, 24/7.
• Semantic model refreshing every hour.
• Spark notebooks left running.
• F4 capacity active all month.

After:

• Incremental loads using LastModifiedDate watermarks.
• Capacity paused at night and most of Sunday.
• Semantic model refresh aligned to business hours.
• Spark jobs converted to short‑lived scheduled runs.
• Capacity downsized to match real peaks.

The result is exactly what you want from "Your Fabric Bill Explained in Euros": the vague €300–400 leak becomes a set of specific, controllable knobs.

One Concrete Takeaway: Start with the Hourly Copy Jobs

If you only do one thing this week, do this:

1. List every pipeline and Dataflow Gen2 job that runs more than once per day.
2. For each job, answer:
   • Does it really need to run at night?
   • Can it use an incremental pattern instead of a full refresh?
3. Move non‑critical jobs into business hours and implement a simple watermark‑based incremental load.

Once those hourly copy jobs stop re‑scanning your lakehouse around the clock, your Fabric bill will start making sense – and you’ll have room to invest in the workloads that actually help the business.