Microsoft Data Platform
A HeadToNet publicationMicrosoft architecture & engineering

Azure data engineering

Build dependable data flows across the Microsoft services and enterprise systems your organization already uses.

Which pipelines should move, and which should stay?

Azure Data Factory and Fabric Data Factory have different artifact models and feature coverage. Start with pipeline behavior, connectors, network boundaries, and operational dependencies. Retention, integration, and migration are all valid architecture choices.

Inventory behavior, not just artifacts.

A pipeline inventory should explain what runs, why it runs, what it consumes, and how it recovers. Counting activities alone will miss the integration and support work.

  • Record connectors, triggers, authentication, dependencies, and service owners.
  • Identify on-premises access, gateways, self-hosted integration runtimes, and SSIS requirements.
  • Compare each required activity and deployment mechanism against the current Fabric feature matrix.

Connect the right execution engines.

Azure SQL can remain a source or operational system while Fabric serves analytical use cases. Azure Databricks can continue data engineering or machine learning responsibilities within a wider Microsoft architecture.

  • Define source-to-target contracts, data formats, and schema evolution rules.
  • Evaluate supported OneLake access or catalog integration rather than copying data by default.
  • Test source load, identity behavior, freshness, and operational recovery end to end.

Make operations part of the design.

Pipeline modernization includes monitoring, deployment, reconciliation, and support ownership. A working notebook is one part of a production data flow.

  • Define retry and replay behavior, including idempotency and late-arriving data.
  • Plan environment promotion, connection configuration, and secret ownership.
  • Measure representative workloads before making cost or throughput commitments.

Microsoft documentation

Product guidance checked 17 November 2026. Architecture recommendations are HeadToNet’s editorial analysis; validate feature availability for your tenant, region, and workload.

Put your architecture in context.

Bring your current platform, constraints, and priorities to a Microsoft data-platform assessment discussion.

Explore the assessment