clement13.net
Original photo: Passerelle de nuit, dont les cadres lumineux décalés forment un tunnel au-dessus d’une vallée enneigée.
Description
Passerelle de nuit, dont les cadres lumineux décalés forment un tunnel au-dessus d’une vallée enneigée.
Source
Tony Webster tonywebster · original page
Licence
CC0 Public domain
Original photo, unretouched. The background is a version of it stylised by the site.

Comparison — OneLake / Fabric vs Managed SQL Server for Power BI Reporting

You will be able to position, across nine architecture dimensions, a Fabric / OneLake platform (fed by SaaS mobile applications) against a managed SQL Server database (Azure SQL Database…

Guide≈ 23 min readCheckedCheck doneFacts, figures and references were checked against the sources cited in the page.Written with AI assistance; sources are cited at the bottom of the page. An error? Report it via clement13.net.Reading
In brief
  • You will be able to position, across nine architecture dimensions, a Fabric / OneLake platform (fed by SaaS mobile applications) against a managed SQL Server database (Azure SQL Database, Managed Instance, SQL database in Fabric) as a Power BI source.
  • You will have a /5 rated heatmap, a decision tree, and typical scenarios to make decisions in meetings, without dogma.
  • You will know how to estimate in orders of magnitude (clearly marked as such) what each option costs, and what makes you switch from one to the other.
  • You will understand coexistence paths: most often, the right answer is neither one nor the other, but managed SQL + mirroring to Fabric.
  • You will leave with links to the detailed dossiers (OneLake 01 to 04, SQL Server 05 to 08).
On this page
  1. 1. Framework, interpretation, and the PaaS vs SaaS nuance
  2. 2. The compared options
    1. 🟢 Beginner — Four reference architectures
    2. 🟡 Practical — What really distinguishes A from B/C
  3. 3. Rating grid and how to read ratings
  4. 4. Side-by-side comparison by aspect
    1. 4.1 Freshness
    2. 4.2 Performance
    3. 4.3 Cost
    4. 4.4 Complexity
    5. 4.5 Governance and security
    6. 4.6 Scalability
    7. 4.7 Required skills
    8. 4.8 Technology maturity
    9. 4.9 Lock-in risk
  5. 5. Summary heatmap
  6. 6. Costs Compared in Orders of Magnitude
    1. 🟡 Practical — Cost Items
    2. 🟡 Practical — Numerical Reasoning (illustrative)
    3. 🔴 Expert — Hidden Costs
  7. 7. Decision Tree
    1. 🟢 Novice — Five Questions in Order
    2. 🟡 Practical — Hard Constraints That Bypass the Tree
    3. 🔴 Expert — Make the Decision Reversible
  8. 8. Typical Scenarios
    1. 8.1 Scenario 1 — Field sales force, 150 users, existing SQL datamart (the common thread)
    2. 8.2 Scenario 2 — Multi-SaaS (CRM + SFA + support), changing schemas, data engineering team
    3. 8.3 Scenario 3 — Broad distribution (2,000 readers), few developers
    4. 8.4 Scenario 4 — Operational real-time (route tracking, alerts)
    5. 8.5 Scenario 5 — Small organization, 15 users, tight budget
    6. 8.6 Scenario 6 — Strict compliance (regulated sector)
  9. 9. Migration and coexistence paths
    1. 🟡 Practice — “Mirroring first” path (the safest)
    2. 🟡 Practice — “SaaS extraction to OneLake” path
    3. 🔴 Expert — Long-term coexistence and dual-source governance
  10. 10. What drives the switch
    1. 🟡 Practice — Switch thresholds (orders of magnitude, to calibrate)
    2. 🔴 Expert — Reasons not to switch
  11. 11. Decision pitfalls
  12. 12. AI contribution to the decision
  13. 13. See also

Level: 🟢→🔴

1. Framework, interpretation, and the PaaS vs SaaS nuance#

This document uses the interpretations adopted for the dossier (details in the dossier's README).

  • “OneLake + SaaS mobile applications” refers to Power BI reporting whose analytical source is Microsoft Fabric / OneLake (lakehouse, warehouse, Direct Lake, mirroring, shortcuts, pipelines, dataflows Gen2, eventstream), fed by SaaS mobile applications (field SFA/CRM, Dynamics 365, Salesforce, Power Apps, data collection apps). This implies API quotas, drifting schemas, data synchronized after offline usage, latency, and identity questions.
  • “SQL Server in SaaS” refers to a cloud managed relational database — Azure SQL Database, Azure SQL Managed Instance, SQL database in Fabric; SQL Server on virtual machine is used as a contrast — used as a Power BI source (Import, DirectQuery, composite, or Direct Lake via mirroring), with the Power BI service itself in SaaS.
⚠️ Pitfall

The term “SaaS” is misleading. Azure SQL Database is strictly PaaS (you do not manage the OS or engine, but you do manage schema, indexes, scaling, logical backup, and permissions). Only Power BI (service) and, to a large extent, Fabric (“unified” SaaS) are SaaS. SQL database in Fabric is a SQL database inside Fabric SaaS. So saying “SQL in SaaS” incorrectly suggests that no one has to operate it.

The two options are not mutually exclusive: Fabric mirroring of managed SQL (see § 9) is exactly the bridge. The comparison helps choose a center of gravity, not ban a building block.

Illustrative thread (anonymous): a SQL Server datamart fed by a field sales application, with multilingual Power BI reports, versioned in TFVC and delivered as installation kits. It is typically the starting point for a “should we move to OneLake?” discussion.


2. The compared options#

🟢 Beginner — Four reference architectures#

Code Architecture In one sentence
A Native Fabric / OneLake SaaS data lands in a lakehouse/warehouse (pipelines, dataflows Gen2, mirroring from the SaaS source when available), and the semantic model reads in Direct Lake.
B Managed SQL + Import A datamart in Azure SQL Database or Managed Instance; Power BI imports data on a schedule.
C Managed SQL + DirectQuery / composite Same database, but Power BI queries the source on the fly (all or part of the tables).
D Hybrid: managed SQL + mirroring to Fabric The datamart stays in SQL; its Delta replica lives in OneLake and feeds Direct Lake, notebooks, AI.

For contrast, E — SQL Server on VM: same principles as B/C, but you operate the OS, patches, high availability, and backups. Keep this only for compatibility reasons (features not available in PaaS, existing licenses): see 06 — SQL Server architecture.

SaaS mobile applicationsOneLakeManaged SQLPower BI semantic modelReports, mobile, CopilotAPI / export / mirroringETLmirroringDirect LakeImport / DirectQuery
Hover over or tap an element to read its details.

🟡 Practical — What really distinguishes A from B/C#

  1. Where transformation happens. In A, in Fabric Spark/SQL, in notebooks, pipelines, and dataflows; in B/C, in T-SQL procedures, with the relational engine enforcing integrity constraints (keys, constraints, transactions).
  2. Which engine serves report queries. In A (Direct Lake), the VertiPaq engine reads Delta/Parquet files without copying; in B, it reads its own imported copy; in C, the SQL engine responds, query by query.
  3. What the billing unit is. A relies on Fabric capacity (capacity units, shared by all workloads); B/C on a sized database (DTU or vCore) plus Power BI licenses; D combines both.
  4. What the relationship is with the SaaS source. Neither A nor B removes the SaaS extraction problem: API quotas, pagination, evolving schemas. The choice changes where it is handled. See 03 — difficulties and pitfalls (OneLake) and 07 — difficulties and pitfalls (SQL Server).

3. Rating grid and how to read ratings#

The nine dimensions are those from the “reporting architectures” brief. A 5/5 rating is always favorable (including for “cost” and “complexity”: 5 = low cost, low complexity). Ratings are contextual expert assessments, not measurements (see Rating: Reading Ratings, Scores, and the Legend).

Rating context — BI team of 3 to 8 people, 20 to 300 report users, data volumes from 10 GB to 2 TB, main source is an SFA SaaS application plus internal reference data, refresh requirement between 15 minutes and one day. Beyond that (thousands of users, terabytes, near real time), ratings shift: see § 10.

Dimension What a 5/5 rating means
Freshness Data available within seconds to minutes, without heavy orchestration
Performance Responsive reports (< 2 s on common visuals) at high data volume
Cost Low and predictable total cost (infrastructure, licenses, human time)
Complexity Few moving parts, straightforward diagnostics
Governance / security Fine-grained control, traceability, RLS/OLS, lineage, compliance, strong tooling
Scalability Grows in data and users without redesign
Required skills Common skills, easy to hire or train
Technology maturity Stable features (GA), established practices, few surprises
Lock-in risk Easy exit, open formats, limited proprietary dependency (5 = low risk)

4. Side-by-side comparison by aspect#

4.1 Freshness#

Option A. Freshness depends on the slowest link in the chain: the SaaS API (quotas, pagination), orchestration, then Direct Lake “framing,” which refreshes model metadata. With mirroring from a supported source, replication is nearly continuous; with pipelines on SaaS APIs, expect from 15 minutes to an hour. An eventstream can get down to a few seconds for event flows, at the cost of added complexity.

Option B. Import refresh is scheduled (8 scheduled refreshes per day in shared capacity, up to 48 on Premium, PPU, or Fabric capacity, according to Microsoft documentation; status 2026-10) and can be triggered by API at the end of loading. Realistic freshness is measured in hours, at best a few tens of minutes on small models or with incremental refresh.

Option C. Freshness matches the database: excellent (real time) but achieved by shifting the load to the database at every user interaction.

Option D. SQL → OneLake mirroring is designed for low latency (on the order of a few seconds to minutes under normal conditions; order of magnitude, to be verified based on load), but the freshness of your datamart is still determined by its own loads.

💡 Expert tip

A report's freshness is the sum of latencies: mobile app → sync (sometimes offline, so hours) → API → ETL → model → visual cache. Measure the worst link before promising “real time.” Often, mobile device sync is the bottleneck, not the platform.

4.2 Performance#

Option A. Direct Lake combines Import speed (VertiPaq in-memory columns) with no copy. Performance depends on Delta file hygiene: too many small files, no V-Order optimization or compaction, poorly sized row groups. Beyond capacity limits (model size per SKU, Direct Lake guardrails), behavior can switch to DirectQuery fallback: performance then drops without any change to the report. See 02 — OneLake architecture.

Option B. Import provides the best query performance at medium scale, with a well-designed model (star schema, low-cardinality columns, efficient measures). The ceiling is importable model size based on license/capacity.

Option C. DirectQuery makes performance dependent on the SQL engine and indexing: columnstore indexes, statistics, query plans. Well designed (aggregations, aggregated Import-mode tables in a composite model), it fits large fact tables; poorly designed, it produces 10-second visuals and a saturated database.

Option D. Direct Lake on the replica delivers A's performance without migrating processing: the team keeps T-SQL and its procedures.

4.3 Cost#

See details in § 6. In short: B is the most economical for a small scope (a database costing a few hundred dollars per month and Pro licenses); A requires a capacity entry ticket but pools Power BI and analytics; C combines high database cost (sizing must absorb interactive queries); D adds both, partly offset by no replication ETL.

4.4 Complexity#

  • B is the simplest: few moving parts, standard skills, familiar diagnostics.
  • C adds diagnostics for DAX → SQL generated queries, a source of opacity.
  • A multiplies components (lakehouse, warehouse, notebooks, pipelines, dataflows, capacity, workspaces, domains) and a capacity model to monitor (smoothing, throttling).
  • D combines B + a thin A layer: more components than B, but without rewriting processing.

4.5 Governance and security#

  • A: OneLake provides a single storage point, lineage, sensitivity labels (Microsoft Purview), and “OneLake security” (data roles with RLS/CLS, documented with no preview mention; only integration of authorized third-party engines remains in preview, status 2026-10). The risk is security sprawl: workspace permissions, item permissions, model RLS, SQL endpoint-side security, Direct Lake-side security. An RLS rule misaligned across layers leaks or blocks access.
  • B/C: the model is familiar — Entra ID authentication, database roles, native SQL RLS (DirectQuery), and Power BI RLS (Import). In DirectQuery with SSO identity, SQL RLS applies to the end user; without SSO, a service account sees everything and Power BI RLS alone carries security.
  • D inherits both: make sure the OneLake replica does not bypass source protections (data masked in SQL must not be readable in clear text in OneLake).
⚠️ Pitfall

Symptom: a user prohibited from seeing a customer can see it through a notebook or the lakehouse SQL endpoint. Cause: RLS was applied only in the semantic model. Fix: handle security at every access point (or expose only the model), document the “layer × mechanism” matrix, and test it.

4.6 Scalability#

  • A: designed for very large volumes and scaling up capacity by SKU, with the constraint of shared capacity between interactive reports and batch processing (noisy neighbor).
  • B/C: vertical scale-up (vCore/DTU), hyperscale for very large databases; B's limit is importable model size, C's limit is the relational engine under interactive load.
  • D: analytics scales on the Fabric side, transactional writes on the SQL side.

4.7 Required skills#

Option Required profiles Note
A Data engineer (Spark/PySpark, Delta, pipelines), analytics engineer, capacity administrator, semantic modeler Rarer and more expensive skills; real learning curve
B T-SQL/ETL developer, Power BI modeler Very widespread skills
C Same as B + SQL performance specialist Index/plan tuning becomes critical
D B + OneLake/Direct Lake basics Gradual upskilling

4.8 Technology maturity#

Managed SQL and Power BI Import/DirectQuery are very mature. Fabric is GA but evolves quickly: some features move from preview to GA in a few months, and the ecosystem of practices (CI/CD, testing, observability) is solidifying. Azure SQL Database mirroring has been GA since November 2024, and Azure SQL Managed Instance mirroring is GA as well; extensions (gateway for a non-public instance, tables without a primary key) have arrived in stages since 2025 — check the exact status for your case in the up-to-date documentation (status 2026-10).

4.9 Lock-in risk#

OneLake stores Delta/Parquet, an open format, which reduces data lock-in; but lock-in shifts to orchestration, semantic models, permissions, and the capacity model. Managed SQL is locked into T-SQL (but portable to on-prem SQL Server or VM); Power BI remains the lock-in point for presentation in both cases.

🧪 Example

The common thread (SQL datamart, multilingual reports, installation kits) has T-SQL procedures for consolidation that form the primary asset. Migrating these procedures to Spark notebooks is not done by “drag and drop”: it's a rewrite. Mirroring preserves this asset and opens OneLake.


5. Summary heatmap#

Legend: 🟥 1–2 · 🟧 3 · 🟨 3.5–4 · 🟩 4.5–5. Scores /5, the higher the better.

Aspect A — Native Fabric / OneLake B — Managed SQL + Import C — Managed SQL + DirectQuery D — Managed SQL + Fabric mirroring
Freshness 🟩 4/5 🟧 3/5 🟩 5/5 🟩 4/5
Performance 🟩 4/5 🟩 5/5 🟥 2/5 🟩 4/5
Cost 🟧 3/5 🟩 4/5 🟧 3/5 🟥 2/5
Complexity 🟥 2/5 🟩 4/5 🟧 3/5 🟧 3/5
Governance / security 🟨 4/5 🟨 4/5 🟧 3/5 🟧 3/5
Scalability 🟩 5/5 🟧 3/5 🟧 3/5 🟩 4/5
Required skills 🟥 2/5 🟩 4/5 🟧 3/5 🟧 3/5
Technology maturity 🟧 3/5 🟩 5/5 🟩 5/5 🟧 3/5
Lock-in risk (5 = low) 🟧 3/5 🟨 4/5 🟨 4/5 🟨 4/5
Unweighted average 3.3/5 4.0/5 3.4/5 3.3/5

Honest reading:

  • B wins for modest scope. It has the highest average: that's no coincidence, it's the least sophisticated combination.
  • The average doesn't decide. Weight based on your context: if scalability and AI matter three times more than complexity, A and D rise sharply. Recalculate with your weights.
  • C is rarely a destination choice. It's a point solution (very large fact table, real-time requirement, no-copy constraint), never a default.
  • D is mostly valuable for its optionality: it buys the possibility of Direct Lake, notebooks, and AI without rewrites, at the cost of additional expense.
✅ Good practice

Have the grid rated by three people (a developer, an operator, a key user) independently, then compare gaps. A 2-point gap on “complexity” reveals an unspoken operations issue, more useful than the average.


6. Costs Compared in Orders of Magnitude#

⚠️ Pitfall

All figures below are indicative orders of magnitude, not quotes. Public pricing, regions, contract discounts, commitments (reservations), currencies, and licenses change over time; status as of 2026-10, verify on official pricing pages before setting any budget. A verification search indicated Power BI Pro around $14 and Premium per user around $24 per user per month with annual billing, and Fabric capacities billed per capacity unit (F4 around $500/month pay-as-you-go, so F2 around $260; to be reconfirmed).

🟡 Practical — Cost Items#

Item A — Native Fabric B — SQL + Import C — SQL + DirectQuery D — SQL + mirroring
Platform storage / compute Fabric capacity (F2 to F64+; from a few hundred to ~10,000 $/month depending on size) Right-sized SQL database (from ~100 to several thousand $/month depending on vCore/DTU) Larger database (it absorbs interactive load) SQL database + Fabric capacity (mirrored storage is often included up to a capacity-linked quota: verify)
Power BI licenses Pro/PPU for creators; free readers possible with sufficient capacity (F64 capacity or higher, with Reader role on the workspace, per Microsoft documentation) Pro per user (including consumers) or capacity Same as B Same as A
Orchestration / ETL Pipelines, dataflows, notebooks: consume capacity T-SQL / Azure Data Factory / agent: marginal cost Same as B SQL ETL retained; little replication ETL
Human time Highest at startup (new skills) Lowest Ongoing performance tuning Intermediate
Data gateway Often avoided in pure SaaS; required for on-prem sources Depends on database accessibility (private endpoint, VNet) Same as B Depends on source

🟡 Practical — Numerical Reasoning (illustrative)#

🧪 Example

Small team, 40 users, 20 GB database.

  • B: a modestly sized SQL database (order of magnitude: $150 to $400/month) + 40 Pro licenses (≈ 40 × 14 = $560/month) ⇒ ~$700 to ~$1,000/month.
  • A: an F2 to F4 capacity (≈ $260 to $530/month) does not eliminate Pro licenses for reading (free readers require larger capacity) ⇒ ~$800 to ~$1,100/month, with added learning cost. Conclusion: at this level, cost does not decide; complexity and skills do, so B.
🧪 Example

Large deployment, 2,000 readers, 1 TB.

  • B/C: 2,000 × 14 ≈ $28,000/month for Pro licenses alone.
  • A/D: an F64 capacity (around $8,000 to $10,000/month pay-as-you-go, less with reservation) enabling reading without individual licenses for consumers (condition to verify) ⇒ half the cost or less. Conclusion: beyond a certain number of readers, capacity is cheaper than per-user licenses. The break-even point is around a few hundred readers (order of magnitude, recalculate with your pricing).

🔴 Expert — Hidden Costs#

  1. Smoothing and throttling (Fabric): undersized capacity slows down everyone; the workaround (move up SKU, split capacities by criticality) has a cost. Plan capacity usage monitoring (Capacity Metrics app).
  2. Re-runs: a SaaS load that fails on API quota consumes compute for nothing; idempotency and replay are both a cost and design item.
  3. Mirroring: includes OneLake storage up to a capacity-proportional quota, beyond which it is billed as storage; replication compute is, per documentation, free under capacity (verify for your setup). The source carries additional load from transaction log reads.
  4. DirectQuery: every user interaction generates queries; on a database billed by compute size, interactive usage requires sizing for peak.
  5. Skills debt: without in-house Spark/Delta skills, A turns into recurring external services.
  6. Cost of indecision: running two platforms without a clear target often costs more than choosing the reasoned “wrong” option.

7. Decision Tree#

🟢 Novice — Five Questions in Order#

Q1. Do you already have a SQL datamart that works and that the team has mastered?
    ├─ Yes → Q2
    └─ No (new project, dominant SaaS sources) → Q4

Q2. Is anything preventing you from staying with managed SQL + Import?
    (volume beyond imported model limits, need for AI/notebooks, need for
     many readers without individual licenses, freshness requirement < 15 min)
    ├─ No → OPTION B  (managed SQL + Import) — do not change what does not need changing
    └─ Yes → Q3

Q3. Can you keep T-SQL processing as is?
    ├─ Yes → OPTION D  (managed SQL + mirroring + Direct Lake)
    └─ No, you also need to move beyond T-SQL → OPTION A, phased migration (§ 9)

Q4. Is data mostly arriving through SaaS APIs/exports, with changing schemas?
    ├─ Yes, and you have/hire Spark/Delta skills → OPTION A
    └─ Yes, but the team is T-SQL → OPTION D or B (landing in SQL database,
       extraction via lightweight pipeline), A later

Q5. Need strict real-time on a large fact table, without copying?
    └─ Add a single table in DirectQuery as needed (composite model),
       never the whole model — OPTION C as a complement, not by default.

🟡 Practical — Hard Constraints That Bypass the Tree#

Constraint Consequence
Data prohibited outside a region or tenant Verify Fabric/OneLake and capacity data residency first (capacity region)
Database behind strict private network Verify support for private endpoints, gateways, or VNet for mirroring or reading (status varies, verify)
SQL features unavailable in PaaS SQL Server on VM or Managed Instance rather than Database
No training budget B or D, not A
Contractual commitment on Power BI Premium per capacity Assess migration to F-SKUs (Microsoft no longer sells P-SKUs and each subscription ends at the current agreement term; status as of 2026-10, verify for your contract)

🔴 Expert — Make the Decision Reversible#

Always choose the variant where the exit costs the least: land raw SaaS data in open storage (OneLake or object storage), keep processing in a portable language, version models (PBIP/TMDL) and schemas, and hide the source behind a semantic layer referenced by reports. section 7 of dossier 06 and dossier 02 detail the semantic layer on each side.


8. Typical Scenarios#

8.1 Scenario 1 — Field sales force, 150 users, existing SQL datamart (the common thread)#

  • Context: SFA application feeding a SQL datamart, multilingual reports, delivery by kits.
  • Recommendation: B now (nothing breaks), D if a need for advanced analytics or AI appears. Monitor imported model refresh time.
  • What changes the game: demand for near real-time visit data, or growth that exceeds importable model size.

8.2 Scenario 2 — Multi-SaaS (CRM + SFA + support), changing schemas, data engineering team#

  • Context: three SaaS platforms, quota-limited APIs, required historization, notebook-based analysts.
  • Recommendation: A: medallion lakehouse (raw bronze, cleaned silver, semantic gold), Direct Lake. Preserved raw data absorbs schema drift.
  • Guardrails: pipeline idempotency, quota management, data contract. See 03 — OneLake + SaaS Mobile Apps: Difficulties and Pitfalls.

8.3 Scenario 3 — Broad distribution (2,000 readers), few developers#

  • Recommendation: imported model or Direct Lake on capacity (readers without individual licenses if capacity threshold is reached), D if the database remains in SQL. License cost dominates (see § 6).

8.4 Scenario 4 — Operational real-time (route tracking, alerts)#

  • Recommendation: A with eventstream (or Real-Time Intelligence) for the stream; Power BI reports in DirectQuery/streaming mode on this scope only; all other reporting in B/D. A single end-to-end “real-time” model is an anti-pattern.

8.5 Scenario 5 — Small organization, 15 users, tight budget#

  • Recommendation: B, no hesitation. The simplest platform that meets the need is the best one. Reassess annually.

8.6 Scenario 6 — Strict compliance (regulated sector)#

  • Recommendation: whichever your security team can audit. B/C offer long-established auditing practices; A requires mastering multilayer security (workspace, item, OneLake security, RLS). A compliance proof that does not exist is not a technical argument.
🤖 AI

Assistants (Copilot in Fabric and Power BI, code assistants) mostly accelerate options with a heavy declarative code share (notebooks, DAX, T-SQL). They do not change platform choice, but reduce the cost of missing skills: a factor to weigh, never overestimate. See 04 — AI (OneLake) and 08 — AI (SQL Server).


9. Migration and coexistence paths#

🟡 Practice — “Mirroring first” path (the safest)#

  1. Step 0 — Scope: inventory tables, volume, consumers, freshness requirements, and security requirements. Choose a pilot scope (3 to 5 fact tables, 1 report).
  2. Step 1 — Mirroring the SQL database to a “mirrored database” item; check prerequisites, data type limits, non-replicated tables, and specifics (primary key, supported DDL), which evolve: check up-to-date documentation (status 2026-10).
  3. Step 2 — Direct Lake semantic model on the replica (or on a lakehouse via shortcut), pilot report in parallel with the existing one.
  4. Step 3 — Number comparison: acceptance testing “old report vs. new report” on a fixed KPI set (see 11 — checklist).
  5. Step 4 — Report cutover one by one, keeping the old Import model as backup for a defined period.
  6. Step 5 — Decommissioning of what is redundant only once the coexistence period is validated.
SQL datamart + Import modelsMirroring to OneLakePilot Direct Lake in parallelIdentical numbers?Cut over report by reportDecommission the old modelOption: shift processing to Fabricnoyes
Hover over or tap an element to read its details.

🟡 Practice — “SaaS extraction to OneLake” path#

When the new part is the SaaS source (not the SQL database):

  1. Land SaaS extractions as raw, timestamped, immutable in OneLake (bronze zone).
  2. Standardize into silver (types, keys, deduplication, logical deletion handling).
  3. Publish gold in a warehouse or lakehouse (star schema); the SQL datamart can remain the source for reference data (shortcuts or mirroring).
  4. Let the SQL datamart consume gold, or retire it progressively.

🔴 Expert — Long-term coexistence and dual-source governance#

Topic Rule
Source of truth by domain Only one per entity (e.g., customer master data in SQL, visit events in lakehouse). Never two concurrent writes.
Keys Same surrogate keys and natural keys on both sides; the replica must not “reinvent” keys.
Schema versions Versioned data contract; a column renamed in SQL breaks the replica and models: notify upstream.
Security Single permissions matrix, applied to every layer.
Observability Single dashboard: freshness by table, SQL/OneLake count deltas, pipeline failures.
Costs Tag resources (database, capacity) to track coexistence spending; set a review date.
✅ Good practice

Set a coexistence exit criterion from the start (“at the end of quarter T+2, either we shut down X, or we decide to keep it for reason Y”). Without this, coexistence becomes the architecture.

⚠️ Pitfall

Symptom: numbers in the Direct Lake report differ from those in the Import report by a few rows. Common causes: replication window (mirroring lags behind the database), date/time columns with different time zones, truncated decimal types, deletions not propagated, different RLS filters. Remedy: an automated, published reconciliation script (counts and sums by day and key).

Example SQL-side reconciliation (to compare with the equivalent result on the replica's SQL endpoint):

-- Daily fingerprint of a fact table, comparable between source and replica
SELECT CAST([DateVisite] AS date)         AS [Jour],
       COUNT_BIG(*)                       AS [NbLignes],
       SUM(CAST([Montant] AS decimal(19,4))) AS [SommeMontant],
       CHECKSUM_AGG(CHECKSUM([ID_Visite])) AS [Empreinte]
FROM   [dbo].[FACT_Visite] WITH (NOLOCK)
WHERE  [DateVisite] >= DATEADD(day, -30, CAST(GETDATE() AS date))
GROUP BY CAST([DateVisite] AS date)
ORDER BY [Jour];

10. What drives the switch#

🟡 Practice — Switch thresholds (orders of magnitude, to calibrate)#

Signal Measurement Switch direction
Import model refresh time > 60% of the overnight window or > 1 hr Direct Lake (D/A) or incremental refresh
Model size close to the license/capacity limit Direct Lake, aggregations, partitioning
Number of readers several hundred Capacity (read without individual license, if threshold reached)
Freshness requirement < 15 min end-to-end Mirroring/eventstream
Analysts in notebooks / data science recurring demand Read-accessible OneLake
Multiple SaaS sources with changing schemas ≥ 3 sources Raw landing in lakehouse
Database load due to queries CPU > 70% during reporting hours Move reads off the database (Import or replica)
Skills arrival of Spark engineers A becomes realistic
Budget licenses > cost of capacity Fabric capacity

🔴 Expert — Reasons not to switch#

  • The need is met and the team has no spare time: an architecture change without felt pain is an expense, not an investment.
  • The data is small and reports are few.
  • Fabric governance is not ready (domains, permissions, workspace lifecycle): migrating disorder to a lake creates a data swamp.
  • Client or third-party reports are tied to existing models (measure IDs, links): the impact of a model change goes beyond the team. See 10 — measures and lineageTag.

11. Decision pitfalls#

# Pitfall Symptom Remedy
1 Choosing based on hype The platform is chosen before the need First write 5 measurable requirements
2 Comparing a mature product to an announced product Ratings based on demo videos Run a measured 2-to-4-week pilot
3 Forgetting licenses Infrastructure budget with no Power BI line item Recalculate (§ 6) with the real number of readers
4 Underestimating governance Workspace sprawl, inconsistent permissions Define domains, conventions, owners before opening
5 Overestimating required freshness “Real-time” requested, D-1 is enough Ask “what decision changes in 5 minutes?”
6 Neglecting the SaaS source API quotas hit, manual retries Plan retry, backoff, cursor-based delta, immutable raw
7 Not testing DirectQuery fallback Slow production report with no change Monitor the query mode actually used
8 Migrating without number acceptance testing Silent drift Fixed KPI set and automated reconciliation
9 Double write Divergence between two sources of truth One source of truth per entity
10 Evaluating without operations “Easy” for the developer, unmanageable in production Have operations provide the rating
11 Ignoring exit Lock-in discovered when leaving Document the exit plan at selection time
12 Choosing DirectQuery “to avoid copying” Saturated database and slow visuals Reserve DirectQuery for tables that require it (composite)

12. AI contribution to the decision#

Use case Gain Risk Maturity Prerequisites Guardrails
Drafting a requirements grid from interviews 4/5 2/5 4/5 Interview notes Review by business stakeholders
Producing an initial license and capacity estimate 3/5 4/5 3/5 Verified rates provided by you Never accept a quoted rate from memory
Converting T-SQL procedures into notebooks 3/5 4/5 3/5 Non-regression tests Number comparison on a fixed dataset
Generating reconciliation scripts 4/5 2/5 4/5 Table schema Run in read-only mode first
Summarizing official documentation 3/5 3/5 3/5 Source links Check date and GA/preview status

Prompt example: “Here are our five measurable requirements and the attached datamart schema. Propose three architectures, with for each one the risks, exit cost, and points I need to verify in the official documentation — without inventing pricing.”

🤖 AI

AI does not replace a measured pilot, trade-off arbitration, or decision accountability. Its main risk here is confidently producing pricing, limits, and preview statuses that are outdated or fabricated: this is exactly the kind of volatile fact to verify at the source.


13. See also#

✻ Written by Claude Sonnet 5.5