Excelgoodies logo +31 97 010285556

LEARN THIS HANDS ON

Power BI Reporting

. Live Online FILLING FAST
View all upcoming batches
Power BI Incremental Refresh vs Fabric Real-Time Analytics: Fast Dashboards Without Burning Capacity

Power BI Incremental Refresh vs Fabric Real-Time Analytics: Fast Dashboards Without Burning Capacity

You’ve got a sales dashboard that needs to show yesterday’s numbers for thousands of stores plus near-real-time orders from the last few minutes. It works in dev, but once you turn on incremental refresh and point it at Fabric, capacity starts screaming and users see spinning circles instead of charts. This article walks through how to architect that mix of historical and live data using Power BI Incremental Refresh and Fabric Real-Time Analytics without blowing your budget.


The Scenario: One Dashboard, Two Very Different Data Behaviours

You’re working with a central BI team supporting an online retail operation:

  • Fact table: FactOrders, ~2–3 billion rows, partitioned by OrderDate.
  • Historical data: stored in a Lakehouse, refreshed daily.
  • Recent data: events landing in a Fabric Real-Time Analytics KQL database (formerly Eventhouse), arriving with a latency of seconds.
  • Single Power BI semantic model feeding executive and operational dashboards.

The pain points:

  1. Full refreshes on the historical model take hours and spike capacity.
  2. Adding DirectQuery to the KQL database makes visuals feel sluggish under load.
  3. Mixing incremental refresh with near-real-time data leads to confusing behaviour and unexpected capacity consumption.

We’ll keep this scenario in mind while comparing Power BI Incremental Refresh (import mode) with Fabric Real-Time Analytics (KQL database, DirectQuery) and then design a pattern that keeps both performance and cost under control.


What Incremental Refresh Actually Does (and Doesn’t)

Incremental refresh in Power BI is fundamentally about import-mode tables and partition management in the semantic model.

For our FactOrders table:

  • You define RangeStart and RangeEnd parameters in Power Query.
  • You configure incremental refresh in Power BI Desktop (Model view → table → Incremental refresh).
  • On publish to a Premium or Fabric capacity, the service uses those settings to create partitions per time slice and only refreshes recent partitions.

Key behaviours and constraints that matter in 2026:

  • Supported storage: works for import-mode tables in semantic models hosted on Power BI Premium or Fabric capacities. It’s also supported for Direct Lake tables, but the mechanics differ; here we focus on import.
  • Partitioning: the service creates partitions based on the Store rows in the last and Refresh rows in the last settings. This is not visible in Desktop; you inspect it via XMLA or tools like Tabular Editor.
  • First refresh is full: the initial refresh after enabling incremental refresh is effectively a full load of the defined historical window.
  • Policy is per table: you configure incremental refresh per table, not per model.
  • Query folding is required for true incremental: if the Power Query step that uses RangeStart/RangeEnd doesn’t fold to the source, the service may fall back to full refresh of the table range.

What incremental refresh does not do:

  • It does not provide streaming or sub-minute latency. You’re still on scheduled refresh (or APIs) with a minimum of 30 minutes for scheduled refresh and lower latencies only when using XMLA/REST-triggered refresh or the enhanced refresh API.
  • It does not reduce query-time CPU cost; it optimizes refresh-time cost and duration.
  • It does not help DirectQuery tables; it’s an import/Direct Lake concept.

In our scenario, incremental refresh is ideal for the historical FactOrders data in the Lakehouse: we can keep, say, 5 years of data but only refresh the last 3 days.


What Fabric Real-Time Analytics Provides

Fabric Real-Time Analytics (RTA) is built on Kusto Query Language (KQL) and is optimized for high-volume, append-only event data.

For our recent orders:

  • Events land in a KQL database table, e.g. OrdersLive.
  • We connect Power BI to the KQL database using the Fabric Real-Time Analytics (KQL) connector (DirectQuery).
  • Visuals query the KQL database at runtime.

Key behaviours to be clear about:

  • Execution mode: the standard pattern is DirectQuery. Each visual sends KQL translated from DAX and gets results from the KQL engine.
  • Latency: data is typically queryable within seconds of ingestion, depending on your pipeline and ingestion method.
  • Scale: RTA is designed for high event throughput, but each Power BI query still consumes CPU and memory in the Fabric capacity.
  • Model features: you can build a semantic model over RTA as a DirectQuery source. Many core features work (measures, relationships), but some features behave differently or are constrained compared to import.

Things RTA does not give you automatically:

  • It doesn’t magically make every dashboard sub-second. If your DAX is complex or joins multiple large tables, queries can be slow.
  • It doesn’t remove capacity cost; queries still consume compute from your Fabric capacity.
  • It doesn’t manage historical partitions; that’s the job of incremental refresh on an import model.

In our scenario, RTA is perfect for the last few minutes of order events. But using it for all history would be expensive and slower than an import-based model with incremental refresh.


Why Mixing Incremental Refresh and Real-Time Often Hurts

Teams often try this first:

  1. Build an import-mode model over the Lakehouse with incremental refresh.
  2. Add a DirectQuery table pointing to RTA for recent events.
  3. Create measures that union or combine both tables.

Symptoms:

  • Some visuals query both sources; performance becomes bound by the slower one.
  • Capacity load spikes when many users hit DirectQuery visuals concurrently.
  • Complexity in DAX to avoid double-counting when a record exists in both historical and real-time sources.

The root problems:

  • Mixed storage modes: when a visual touches both import and DirectQuery tables, the engine has to orchestrate across sources, which is inherently slower and more expensive than import-only.
  • Overlapping time ranges: if the historical import data includes today and RTA also holds today’s events, you risk double-counting unless you carefully exclude overlaps.
  • No clear SLA boundaries: all visuals look similar to users, but some are backed by cheap import queries and others by expensive DirectQuery calls.

We need a cleaner architecture.


Architectural Pattern: Historical via Incremental Refresh, Recent via RTA, With a Clear Cutover

The pattern that usually works best for this scenario:

  1. Historical semantic model (import)

    • Source: Fabric Lakehouse (or other warehouse) with curated, batch-loaded data.
    • Storage mode: Import.
    • Incremental refresh: enabled on large fact tables.
    • Time window: e.g. keep 5 years, refresh last 3 days.
  2. Real-time semantic model (DirectQuery)

    • Source: Fabric Real-Time Analytics KQL database.
    • Storage mode: DirectQuery.
    • Time window: last N minutes/hours, depending on business needs.
  3. Report design

    • Use separate pages or clearly separated visuals for historical vs real-time views.
    • Avoid visuals that query both models for the same metric unless absolutely necessary.
    • If you must combine, consider aggregating real-time data into a small summary table to reduce query cost.
  4. Operational process

    • Periodically move recent data from RTA into the Lakehouse (e.g. every hour) and let the next incremental refresh pick it up.
    • Adjust RTA retention so it only holds the short real-time window you truly need.

This pattern helps you:

  • Keep most user queries on the cheap, fast import model.
  • Reserve DirectQuery to RTA for a small number of operational visuals where seconds matter.
  • Avoid overlapping time windows that complicate DAX and double-counting.

Implementing Incremental Refresh Correctly for the Historical Model

Let’s focus on FactOrders in the historical Lakehouse.

1. Define Range Parameters in Power Query

In Power BI Desktop, create two parameters using the UI (Home → Manage Parameters → New Parameter):

// Example definition for RangeStart
let
    RangeStart = #datetime(2020, 1, 1, 0, 0, 0)
in
    RangeStart
// Example definition for RangeEnd
let
    RangeEnd = #datetime(2020, 1, 31, 0, 0, 0)
in
    RangeEnd

The key points:

  • Type: DateTime.
  • Values: design-time placeholders; the service overwrites them during refresh based on the incremental refresh policy.

2. Filter the Fact Table Using Range Parameters

Assume your Lakehouse table has a OrderDateTime column.

let
    Source = Lakehouse.Contents("MyWorkspace", "SalesLakehouse"),
    OrdersTable = Source{[Name="FactOrders", Kind="Table"]}[Data],
    FilteredRows = Table.SelectRows(OrdersTable, each [OrderDateTime] >= RangeStart and [OrderDateTime] < RangeEnd)
in
    FilteredRows

Important:

  • Ensure this filter step folds to the Lakehouse source. You can check query folding in Power Query (View → Query plan / View Native Query where available).
  • Avoid transformations before the filter that break folding.

3. Configure Incremental Refresh Policy

In Desktop (Model view):

  • Right-click FactOrders → Incremental refresh.
  • Example policy:
    • Store rows in the last: 5 years.
    • Refresh rows in the last: 3 days.
    • Optionally, enable Get the latest data in real time with DirectQuery only if you want hybrid mode (see next section).

Once published to a Fabric or Premium capacity, the first refresh will create partitions for those 5 years. Subsequent refreshes will only process partitions covering the last 3 days.

This cuts refresh time and capacity cost dramatically for large tables.


Hybrid Mode: When to Use "Get the Latest Data in Real Time" and When to Avoid It

Power BI incremental refresh has an option: Get the latest data in real time with DirectQuery. This creates a hybrid table:

  • Historical partitions: import.
  • Latest partition: DirectQuery to the same source.

This can be useful when:

  • Your source supports good DirectQuery performance.
  • You want near-real-time data for the most recent period without a separate RTA pipeline.

However, in our scenario we already have Fabric Real-Time Analytics as a specialized real-time store. Using hybrid mode against the Lakehouse and RTA at the same time can:

  • Increase complexity: two different real-time mechanisms.
  • Make capacity usage harder to reason about.

Practical guidance:

  • If you have RTA for true events, keep incremental refresh in pure import mode and use RTA only for real-time views.
  • Consider hybrid mode instead of RTA for sources that don’t need high-frequency ingestion but still require fresher-than-scheduled-refresh data.

Building the Real-Time Model Over Fabric RTA

For the real-time semantic model over OrdersLive in RTA:

  1. In Power BI Desktop, use Get data → Fabric Real-Time Analytics (KQL).
  2. Connect to the KQL database and select the OrdersLive table.
  3. Keep storage mode as DirectQuery.
  4. Model only the minimal fields you need for operational views:
    • OrderId, StoreId, OrderDateTime, Amount, etc.

Example DAX measure for real-time sales in the last 15 minutes:

RealTime Sales (Last 15 min) = 
VAR NowUTC = NOW()
VAR WindowStart = NowUTC - (15.0 / 1440.0)  // 15 minutes as fraction of a day
RETURN
CALCULATE(
    SUM(OrdersLive[Amount]),
    FILTER(
        OrdersLive,
        OrdersLive[OrderDateTime] >= WindowStart &&
        OrdersLive[OrderDateTime] <= NowUTC
    )
)

Considerations:

  • Ensure OrderDateTime is in a timezone that matches your reporting logic (often UTC). If you need local time, convert consistently.
  • Keep filter windows small; wider windows mean more data scanned per query.

To reduce capacity load:

  • Limit real-time pages to a smaller audience (e.g. operations team) using workspace and app permissions.
  • Avoid high-cardinality slicers on DirectQuery tables.
  • Use simple aggregations; complex DAX over DirectQuery can produce heavy KQL queries.

Stitching Historical and Real-Time Views Without Double-Counting

We want executives to see daily totals (historical, import) and operations to see recent orders (real-time, RTA) without confusing overlaps.

Practical design:

  • Historical pages: all visuals bound to the import model with incremental refresh. Time granularity: day or higher.
  • Real-time pages: all visuals bound to the RTA DirectQuery model. Time granularity: minutes/seconds.
  • Navigation: buttons or tabs clearly labelled “Daily performance” vs “Live orders”.

If you must show a combined metric (rare, but sometimes requested), use a semantic boundary:

  • Historical model: up to end of last completed day.
  • Real-time model: from start of current day to now.

Then a combined value in a separate report can be built via a composite model that:

  • Imports a daily summary from the historical model.
  • DirectQueries the real-time model for today.

But be aware: this adds complexity and potential performance cost. Often it’s simpler to show two numbers side by side.


Capacity and Cost: Keeping the Dashboard Fast Without Oversizing

To avoid blowing the budget:

  1. Push most queries to import

    • Incremental refresh on large tables.
    • Historical pages used by the broad audience.
  2. Constrain real-time usage

    • RTA DirectQuery only where latency truly matters.
    • Short time windows and simple aggregations.
  3. Tune refresh schedules

    • Historical model: refresh after ETL completes, not more often than needed.
    • Use XMLA or REST APIs (including the enhanced refresh API where appropriate) to trigger refresh on demand when upstream data lands.
  4. Monitor capacity

    • Use Fabric/Power BI capacity metrics to watch for spikes on DirectQuery operations.
    • If RTA queries dominate, consider:
      • Aggregating events in RTA into summary tables.
      • Reducing page complexity (fewer visuals querying RTA at once).

In our scenario, the team ended up with:

  • Historical model: 5 years of orders, incremental refresh, daily scheduled refresh + on-demand triggers after ETL.
  • Real-time model: last 2 hours of events in RTA, DirectQuery pages only for operations.
  • Executives mostly use the historical pages; operations use real-time during business hours.

Capacity stabilized, refresh times dropped, and users stopped complaining about slow visuals.


Practical Takeaway

Before you reach for Fabric Real-Time Analytics, make sure every metric that can tolerate batch latency is handled by an import model with incremental refresh. Use RTA only for the narrow slice of truly real-time needs, keep the two worlds separated in your report design, and you’ll get fast dashboards without needing to scale Fabric capacity just to feed a few live tiles.

Editor's Note

This article reflects the shift toward combining incremental refresh for bulk historical data with carefully scoped Fabric Real-Time Analytics for live slices, especially in teams running large semantic models on constrained Premium or Fabric capacities.

Professionals who want to apply these patterns to their own data can explore Excelgoodies' Microsoft Fabric & Power BI programme - taught live by instructors, with certification awarded once a real project is running at work.

Insights compiled through ongoing industry research and discussions within the Excelgoodies Analytics Community.

Power BI

New

Next Batches Now Live

Power BIPower BI
SQLSQL
Power AppsPower Apps
Power AutomatePower Automate
Microsoft FabricMicrosoft Fabrics
AzureAzure Data Engineering
Explore Dates & Reserve Your Spot → Reserve Your Spot →