Data Leaps – Liping's Data Blog

02/11/2023

3+ Ways to Connect Excel to Databricks

Since all roads lead to Excel, this blog is inevitable. At the time of writing, there are mainly three ways to let business Excel users consume the data in Databricks. 1. Connecting through ODBC Driver If you google “connect Excel to Databricks”, this is the documentation you will find. It requires your end users to install ODBC driver on their local computer and...

Databricks/Misc

14/10/2023

Why is thinking the most important skill in the era of AI

When participating in Advancing Analytics‘ Advancing Innovation Day, I said during a panel discussion: “Thinking is the most important skill to have in this era of AI explosion. Picking the right problem to solve and the right question to ask is the most important skill.” A few people came to me/DMed me to discuss this point. So I thought I...

Career/Data Strategy/Misc

04/05/2023

Getting into technical Data & AI roles from a Non-STEM background

Recently I had some time to reflect on my academic and professional career. Having done multiple degrees that are not STEM related and started my career working for P&G in supply chain a decade ago, it feels incredible that I am working in Data and AI as a solution architect today. Over the years, I have also coached several graduates...

Career/Misc

05/04/2023

Why the pig?

Someone had noticed the logo change in my blog and asked why there is a pig in the logo now? Well, because originally I was gonna name this blog SQL Oinks. Why SQL Oinks? Here is the story: I had the pleasure to be in the room when my brilliant colleague Mike (amazing human and solution architect, who also writes SQL...

Databricks/Lakehouse/Misc

31/03/2023

Copy from/into Unity Catalog Tables using Copy Data Tool in Azure Data Factory

Those of you who have adopted Unity Catalog in Databricks and are using Azure Data Factory(ADF) in conjunction might have noticed that ADF’s delta lake connector has not been fully updated for Unity Catalog tables yet. You can only select database name and table name when creating a dataset using the connector, which means the three-level namespace Unity Catalog tables...

Azure Data Factory/Databricks/Uncategorized

13/08/2022

5 ways to integrate Azure Synapse Analytics with Azure Machine Learning

Before the integration between Synapse and AML (Azure Machine Learning) was possible, when you wanted to bring Azure Machine Learning capability into your data platform, you would store your data in the data lake, set the lake up as a datastore in AML, do all the training and scoring of machine learning models in AML, push the scoring back to...

Azure Machine Learning/Synapse