Microsoft Fabric

Complete Analytics Platform

In its new Software as a service offering, Microsoft basically clubbed every tool in their Analytics portfolio and gave it a new name - Fabric :). Claims are Fabric can serve every Data stakeholder ranging from a developer working with Data Lake to a Sales associate working on a self-serve Powerbi report. Microsoft has implemented tenant centric architecture in Fabric like office 365, In optimal design an organization will have 1 fabric similar to 1 office 365 tenant for entire organization.

Lake centric and Open

All the data and apps built on Fabric provided solutions will get stored at a single lake, It auto calculates the lineage for objects stored on a single data lake. It uses delta file format and parquet data storage for all the objects.

Advantage:

Table storage is shared across the fabric workspace, suppose you have a data issue in a Synapse datawarehouse query, just run a fix on the data set using Synapse data engineering python notebook, and refresh your query you will see the fix in the result. No need to refresh the datawarehouse like before, as every application across Fabric is using the same Delta table.

Concept of Shortcuts will allow one lake to communicate with other external data storages such as azure data lake , Amazon S3.

One Security- Only 1 security format across one lake for all the engines. Segregations can be made on Domain levels by Administrators- For instance Sales, HR, Marketing etc.

Fabric integration with office for Business Users

Excel integration on top of One Lake data.
Fabric Power Bi Integration in Teams.
Interactive reporting integration into PowerPoint

Data Activator

Microsoft's brand new AI powered scheduling/orchestrating tool for triggers and runs.

Comments

ACLs in Azure Data Lake

Access Control Lists(ACLs) in azure are an extremely powerful toolset to provision granular levels of access in Azure Data Lake. Role-Based Access Control (RBAC) is best option to setup broader access levels however with ACLs you can reach the lowest possible grains as low as a file inside a blob container. Think of a scenario where you want to add more than 1 user to a folder inside a blob container and each one of them sees only their data - Possible with ACLs Prerequisites Azure Subscription Storage blob with hierarchical namespace enabled Reader Access on the storage object via RBAC How to setup ACLs in Azure Data Lake Like any other offering, Microsoft has a broad spectrum of tools/ways to setup ACLs, ranging from Azure Portal to writing python code . All the steps involved are available in Microsoft documentation, and in a very descriptive manner, therefore needless to rephrase again in this article. Instead, lets walk through some of the challenges one can c...

n8n Tutorial: Automate Your Workflows in 4 Simple Steps

Getting Started with n8n: A Practical Guide to Workflow Automation n8n (pronounced "n-eight-n") is a powerful, open-source workflow automation tool that enables you to connect different apps and services without writing code. Think of it as a visual programming environment where you design automated sequences, known as workflows, by linking together pre-built nodes. Unlike some other automation platforms, n8n is fair-code, meaning its source code is available for you to inspect, modify, and self-host, giving you unparalleled control over your data and processes. This guide will walk you through the core concepts and initial steps to harness n8n's potential, turning complex, manual tasks into efficient, automated workflows. What is n8n? Understanding the Core Concepts At its heart, n8n is an integration platform designed to move and transform data between various applications. The fundamental building block of any n8n project is the node . Each node represents a spe...

Integration Runtime in Azure Data Factory

Integration runtime joins Activity and Linked services. It provides a compute environment to the Activity to process enlisted actions. Azure IR Self-Hosted Azure-SSIS Running Dataflows, data movement inside Azure Data movement from externally hosted systems To execute SSIS packages Azure data factory can be hosted in any azure region of Customer’s choice and IR location can be independent of ADF’s azure regions. Generally, IRs are hosted in the Azure region where data movement, activity dispatching etc. is required. IR behavior with AutoResolve Region in public network Time To Live TTL in Integration Runtime Auto resolve, Adhoc integration runtime clusters add a cluster acquisition time (approximately 4-5 mins) every time it spins up a new cluster for being used in a data flow. Thus it adds an additional compute setup time in total job time for dataflow execution, this behavior...

StoredProcs

Search This Blog