Microsoft Fabric: Loading existing Delta Tables to Silver!

Fabric Espresso: Loading existing Delta Tables to Silver! Welcome to our Fabric Espresso series! In this video, Stijn shows how to tap into existing delta tables residing on ADLS Gen2 with Microsoft Fabric! We will show you how to link those tables using shortcuts and how to start querying it with the SQL endpoint!

"Exploring Microsoft Fabric and Delta Tables in Depth" likely refers to learning about Microsoft Fabric's Lakehouse and its use of Delta Lake tables. Here's what I found:

• Microsoft Fabric Lakehouse is a data architecture platform for storing, managing, and analyzing both structured and unstructured data in a single location. For seamless data access across all compute engines in Microsoft Fabric, Delta Lake is chosen as the unified table format.

• Microsoft Fabric Runtime for Apache Spark is built on the same foundation as Azure Synapse Analytics Runtime for Apache Spark, but it has key differences that provide more streamlined behavior across all engines in the Microsoft Fabric service. Key performance features are turned on by default, but advanced Apache Spark users can revert configurations to previous values to better align with specific scenarios.

• Microsoft Fabric Lakehouse and the Apache Spark engine support all table types, including both managed and unmanaged tables. This includes views and regular non-Delta Hive table formats. Tables defined using PARQUET, CSV, AVRO, JSON, and any Apache Hive compatible file format work as expected.

• The Lakehouse Explorer user interface experience varies depending on the table type, and currently, it only renders table objects.

• The Lakehouse Explorer provides a tree-like view of the objects in the Microsoft Fabric Lakehouse item. It has a key capability of discovering and displaying tables that are described in the metadata repository and in OneLake storage. Auto-discovery also applies to tables defined over OneLake shortcuts.

• Microsoft Fabric Lakehouse supports tables defined over OneLake shortcuts to provide utmost compatibility and no data movement. If multiple tables are present in the destination, one shortcut should be created per table. Folders with files should use Apache Spark to use the destination directly using relative paths and load the data into Lakehouse native Delta tables for maximum performance.

• There is a feature called "Load to Tables" that streamlines loading data into Delta tables. This feature provides a visual experience for loading common file formats to Delta to boost analytical productivity for all personas.

• Microsoft Fabric Lakehouse proactively enables important parameters to minimize common problems associated with big data tables, such as compaction and small file sizes, and to maximize query performance. There are scenarios where these parameters need changes, and Delta Lake table optimization guides provide in-depth instructions on how to efficiently maintain Delta tables for maximum performance.

It's worth noting that as of the time the information was gathered, Microsoft Fabric was still in the preview stage.

Exploring Microsoft Fabric and Delta Tables in Depth