Categories
Azure

Backup Data Lake Gen2 containers with Data Factory

In this article we’ll review how to create a Data Factory pipeline, linked services, dataset and trigger in order to copy files between Data Lakes. The storage accounts can be setup to have disaster recovery and replication (and some storage accounts have soft-deletes), but this does not cover accidental deletes cause by human error. We’ll […]

Categories
Azure

Azure Active Directory authorization strategy utilizing nested groups

In this article we’ll review how we can create Azure Active Directory authorization strategy utilizing nested groups with assigned role-based access control (RBAC) and access control lists (ACL). What’s the difference between the two access types. How we can access resources and fine-grain the control with examples for Data Lake and Databricks. Role-Based Access Control […]

Categories
Azure

Automatic creation of directory structure in Data Lake Gen2 container

In this article we’ll review how we can automatically create directory structure in Data Lake Gen2 container utilizing Azure DevOps Repo and Release pipeline with custom PowerShell script. We’ll create new repo for Data Lake, add the PowerShell script in different repo where we store the DevOps scripts (or use inline), and create release pipeline […]