In this article we’ll review how we can automatically create directory structure in Data Lake Gen2 container utilizing Azure DevOps Repo and Release pipeline with custom PowerShell script. We’ll create new repo for Data Lake, add the PowerShell script in different repo where we store the DevOps scripts (or use inline), and create release pipeline […]
Tag: DevOps
Categories
Synapse Analytics Workspace CI/CD
In this article we’ll review how to implement continuous integration and continuous delivery on Azure Synapse Analytics Workspace using Azure DevOps. We’ll be covering how to setup source control, and how to build & deploy ARM templates. Setup source control Setting the source control for Synapse Analytics is absolutely the same as Data Factory. Skip […]
Categories
Data Factory CI/CD
In this article we’ll review how to implement continuous integration and continuous delivery on Azure Data Factory using Azure DevOps. We’ll be covering how to setup source control, and how to build & deploy ARM templates, including the new auto-validating and auto-publishing capabilities that Microsoft just introduced. We’ll start with source control setup, then we’ll […]
Categories
Databricks CI/CD
In this article we’ll review how to implement continuous integration and continuous delivery on Azure Databricks using Azure DevOps. We’ll be covering how to build and deploy Databricks notebooks, interactive clusters and libraries for the interactive clusters. We’ll start by organizing the authorization via Service Principals and Databricks Personal Access Tokens, including setup of service […]
I’m assuming you already have:1. DevOps organization available with project’s service connection setup.(i.e. permissions to view your Data Factory)2. PowerBI Desktop installed or license to use PowerBI online. Currently there is no way to clearly see all scheduled triggers in Data Factory. You can check them one by one, but if you have very big […]