Managed Virtual Network and managed private endpoints - Microsoft Purview (2023)

Table of Contents
In this article Conceptual overview Supported regions Supported data sources Managed Virtual Network Managed private endpoints Interactive authoring Deployment Steps Prerequisites Deploy Managed VNet Runtimes Deploy managed private endpoints for data sources Register and scan a data source using Managed VNet Runtime Next steps
  • Article
  • 10 minutes to read

Important

Currently, Managed Virtual Network and managed private endpoints are available for Microsoft Purview accounts that are deployed in the following regions:

  • Australia East
  • Canada Central
  • East US
  • East US 2
  • North Europe
  • West Europe

Conceptual overview

This article describes how to configure Managed Virtual Network and managed private endpoints for Microsoft Purview.

Supported regions

Currently, Managed Virtual Network and managed private endpoints are available for Microsoft Purview accounts that are deployed in the following regions:

  • Australia East
  • Canada Central
  • East US
  • East US 2
  • North Europe
  • West Europe

Supported data sources

Currently, the following data sources are supported to have a managed private endpoint and can be scanned using Managed VNet Runtime in Microsoft Purview:

  • Azure Blob Storage
  • Azure Cosmos DB
  • Azure Data Lake Storage Gen 2
  • Azure Database for MySQL
  • Azure Database for PostgreSQL
  • Azure Dedicated SQL pool (formerly SQL DW)
  • Azure Files
  • Azure SQL Database
  • Azure SQL Managed Instance
  • Azure Synapse Analytics

Additionally, you can deploy managed private endpoints for your Azure Key Vault resources if you need to run scans using any authentication options rather than Managed Identities, such as SQL Authentication or Account Key.

Important

If you are planning to scan Azure Synapse workspaces using Managed Virtual Network, you are also required to configure Azure Synapse workspace firewall access to enable Allow Azure services and resources to access this workspace. Currently, we do not support setting up scans for an Azure Synapse workspace from the Microsoft Purview governance portal, if you cannot enable Allow Azure services and resources to access this workspace on your Azure Synapse workspaces. If you cannot enable the firewall:

  • You can use Microsoft Purview REST API - Scans - Create Or Update to create a new scan for your Synapse workspaces including dedicated and serverless pools.
  • You must use SQL Authentication as authentication mechanism.

Managed Virtual Network

A Managed Virtual Network in Microsoft Purview is a virtual network that is deployed and managed by Azure inside the same region as Microsoft Purview account to allow scanning Azure data sources inside a managed network, without having to deploy and manage any self-hosted integration runtime virtual machines by the customer in Azure.

Managed Virtual Network and managed private endpoints - Microsoft Purview (1)

You can deploy an Azure Managed Integration Runtime within a Microsoft Purview Managed Virtual Network. From there, the Managed VNet Runtime will use private endpoints to securely connect to and scan supported data sources.

Creating a Managed VNet Runtime within Managed Virtual Network ensures that data integration process is isolated and secure.

Benefits of using Managed Virtual Network:

  • With a Managed Virtual Network, you can offload the burden of managing the Virtual Network to Microsoft Purview. You don't need to create and manage VNets or subnets for Azure Integration Runtime to use for scanning Azure data sources.
  • It doesn't require deep Azure networking knowledge to do data integrations securely. Using a Managed Virtual Network is much simplified for data engineers.
  • Managed Virtual Network along with Managed private endpoints protects against data exfiltration.

Important

Currently, the Managed Virtual Network is only supported in the same region as Microsoft Purview account region.

Note

You cannot switch a global Azure integration runtime or self-hosted integration runtime to a Managed VNet Runtime and vice versa.

A Managed VNet is created for your Microsoft Purview account when you create a Managed VNet Runtime for the first time in your Microsoft Purview account. You can't view or manage the Managed VNets.

Managed private endpoints

Managed private endpoints are private endpoints created in the Microsoft Purview Managed Virtual Network establishing a private link to Microsoft Purview and Azure resources. Microsoft Purview manages these private endpoints on your behalf.

Managed Virtual Network and managed private endpoints - Microsoft Purview (2)

Microsoft Purview supports private links. Private link enables you to access Azure (PaaS) services (such as Azure Storage, Azure Cosmos DB, Azure Synapse Analytics).

When you use a private link, traffic between your data sources and Managed Virtual Network traverses entirely over the Microsoft backbone network. Private Link protects against data exfiltration risks. You establish a private link to a resource by creating a private endpoint.

Private endpoint uses a private IP address in the Managed Virtual Network to effectively bring the service into it. Private endpoints are mapped to a specific resource in Azure and not the entire service. Customers can limit connectivity to a specific resource approved by their organization. Learn more about private links and private endpoints.

Note

To reduce administrative overhead, it's recommended that you create managed private endpoints to scan all supported Azure data sources.

Warning

If an Azure PaaS data store (Blob, Azure Data Lake Storage Gen2, Azure Synapse Analytics) has a private endpoint already created against it, and even if it allows access from all networks, Microsoft Purview would only be able to access it using a managed private endpoint. If a private endpoint does not already exist, you must create one in such scenarios.

See Also
De Microsoft Beheerd startscherm-app configureren - Microsoft IntuneBeheerde identiteit - Azure Data FactoryUse Intune to manage Microsoft Defender for Endpoint Security on devices not enrolled with Microsoft IntuneDoor de klant beheerde sleutels voor accountversleuteling - Azure Storage

A private endpoint connection is created in a "Pending" state when you create a managed private endpoint in Microsoft Purview. An approval workflow is initiated. The private link resource owner is responsible to approve or reject the connection.

Managed Virtual Network and managed private endpoints - Microsoft Purview (3)

If the owner approves the connection, the private link is established. Otherwise, the private link won't be established. In either case, the Managed private endpoint will be updated with the status of the connection.

Managed Virtual Network and managed private endpoints - Microsoft Purview (4)

Only a Managed private endpoint in an approved state can send traffic to a given private link resource.

Interactive authoring

Interactive authoring capabilities are used for functionalities like test connection, browse folder list and table list, get schema, and preview data. You can enable interactive authoring when creating or editing an Azure Integration Runtime that is in Purview-Managed Virtual Network. The backend service will pre-allocate compute for interactive authoring functionalities. Otherwise, the compute will be allocated every time any interactive operation is performed which will take more time. The Time To Live (TTL) for interactive authoring is 60 minutes, which means it will automatically become disabled after 60 minutes of the last interactive authoring operation.

Managed Virtual Network and managed private endpoints - Microsoft Purview (5)

Deployment Steps

Prerequisites

Before deploying a Managed VNet and Managed VNet Runtime for a Microsoft Purview account, ensure you meet the following prerequisites:

  1. A Microsoft Purview account deployed in one of the supported regions.
  2. From Microsoft Purview roles, you must be a data curator at root collection level in your Microsoft Purview account.
  3. From Azure RBAC roles, you must be contributor on the Microsoft Purview account and data source to approve private links.

Deploy Managed VNet Runtimes

Note

The following guide shows how to register and scan an Azure Data Lake Storage Gen 2 using Managed VNet Runtime.

  1. Go to the Azure portal, and navigate to the Microsoft Purview accounts page and select your Purview account.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (6)

  2. Open Microsoft Purview governance portal and navigate to the Data Map --> Integration runtimes.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (7)

  3. From Integration runtimes page, select + New icon, to create a new runtime. Select Azure and then select Continue.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (8)

  4. Provide a name for your Managed VNet Runtime, select the region and configure interactive authoring. Select Create.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (9)

  5. Deploying the Managed VNet Runtime for the first time triggers multiple workflows in the Microsoft Purview governance portal for creating managed private endpoints for Microsoft Purview and its Managed Storage Account. Select on each workflow to approve the private endpoint for the corresponding Azure resource.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (10)

  6. In Azure portal, from your Microsoft Purview account resource window, approve the managed private endpoint. From Managed storage account page approve the managed private endpoints for blob and queue services:

    Managed Virtual Network and managed private endpoints - Microsoft Purview (11)

    Managed Virtual Network and managed private endpoints - Microsoft Purview (12)

    Managed Virtual Network and managed private endpoints - Microsoft Purview (13)

    Managed Virtual Network and managed private endpoints - Microsoft Purview (14)

  7. From Management, select Managed private endpoint to validate if all managed private endpoints are successfully deployed and approved. All private endpoints be approved.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (15)

    Managed Virtual Network and managed private endpoints - Microsoft Purview (16)

Deploy managed private endpoints for data sources

To scan any data sources using Managed VNet Runtime, you need to deploy and approve a managed private endpoint for the data source prior to create a new scan. To deploy and approve a managed private endpoint for a data source, follow these steps selecting data source of your choice from the list:

  1. Navigate to Management, and select Managed private endpoints.

    See Also
    Azure Disk Storage overview - Azure Virtual MachinesConfigure the Microsoft Managed Home Screen app - Microsoft IntuneIntroduction to solutions (Developer Guide for Dynamics 365 Customer Engagement (on-premises))Managed virtual network and managed private endpoints - Azure Data Factory

  2. Select + New.

  3. From the list of supported data sources, select the type that corresponds to the data source you're planning to scan using Managed VNet Runtime.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (17)

  4. Provide a name for the managed private endpoint, select the Azure subscription and the data source from the drop-down lists. Select create.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (18)

  5. From the list of managed private endpoints, select on the newly created managed private endpoint for your data source and then select on Manage approvals in the Azure portal, to approve the private endpoint in Azure portal.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (19)

  6. By clicking on the link, you're redirected to Azure portal. Under private endpoints connection, select the newly created private endpoint and select approve.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (20)

    Managed Virtual Network and managed private endpoints - Microsoft Purview (21)

  7. Inside the Microsoft Purview governance portal, the managed private endpoint must be shown as approved as well.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (22)

Register and scan a data source using Managed VNet Runtime

Register data source

It's important to register the data source in Microsoft Purview prior to setting up a scan for the data source. Follow these steps to register data source if you haven't yet registered it.

  1. Go to your Microsoft Purview account.

  2. Select Data Map on the left menu.

  3. Select Register.

  4. On Register sources, select your data source

  5. Select Continue.

  6. On the Register sources screen, do the following:

    1. In the Name box, enter a name that the data source will be listed with in the catalog.
    2. In the Subscription dropdown list box, select a subscription.
    3. In the Select a collection box, select a collection.
    4. Select Register to register the data sources.

For more information, see Manage data sources in Microsoft Purview.

Scan data source

You can use any of the following options to scan data sources using Microsoft Purview Managed VNet Runtime:

  • Using Managed Identity (Recommended) - As soon as the Microsoft Purview Account is created, a system-assigned managed identity (SAMI) is created automatically in Azure AD tenant. Depending on the type of resource, specific RBAC role assignments are required for the Microsoft Purview system-assigned managed identity (SAMI) to perform the scans.

  • Using other authentication options:

    • Account Key or SQL Authentication- Secrets can be created inside an Azure Key Vault to store credentials in order to enable access for Microsoft Purview to scan data sources securely using the secrets. A secret can be a storage account key, SQL login password, or a password.

    • Service Principal - In this method, you can create a new or use an existing service principal in your Azure Active Directory tenant.

Scan using Managed Identity

To scan a data source using a Managed VNet Runtime and Microsoft Purview managed identity perform these steps:

  1. Select the Data Map tab on the left pane in the Microsoft Purview governance portal.

  2. Select the data source that you registered.

  3. Select View details > + New scan, or use the Scan quick-action icon on the source tile.

  4. Provide a Name for the scan.

  5. Under Connect via integration runtime, select the newly created Managed VNet Runtime.

  6. For Credential Select the managed identity, choose the appropriate collection for the scan, and select Test connection. On a successful connection, select Continue.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (23)

    See Also
    Add and assign Managed Google Play apps to Android Enterprise devices - Microsoft IntuneIntroduction to managed metadata - SharePoint in Microsoft 365Use managed identities on a virtual machine to acquire access token - Azure AD - Microsoft Entra

  7. Follow the steps to select the appropriate scan rule and scope for your scan.

  8. Choose your scan trigger. You can set up a schedule or run the scan once.

  9. Review your scan and select Save and run.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (24)

Scan using other authentication options

You can also use other supported options to scan data sources using Microsoft Purview Managed Runtime. This requires setting up a private connection to Azure Key Vault where the secret is stored.

To set up a scan using Account Key or SQL Authentication follow these steps:

  1. Grant Microsoft Purview access to your Azure Key Vault.

  2. Create a new credential in Microsoft Purview.

  3. Navigate to Management, and select Managed private endpoints.

  4. Select + New.

  5. From the list of supported data sources, select Key Vault.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (25)

  6. Provide a name for the managed private endpoint, select the Azure subscription and the Azure Key Vault from the drop-down lists. Select create.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (26)

  7. From the list of managed private endpoints, select on the newly created managed private endpoint for your Azure Key Vault and then select on Manage approvals in the Azure portal, to approve the private endpoint in Azure portal.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (27)

  8. By clicking on the link, you're redirected to Azure portal. Under private endpoints connection, select the newly created private endpoint for your Azure Key Vault and select approve.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (28)

    Managed Virtual Network and managed private endpoints - Microsoft Purview (29)

  9. Inside the Microsoft Purview governance portal, the managed private endpoint must be shown as approved as well.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (30)

  10. Select the Data Map tab on the left pane in the Microsoft Purview governance portal.

  11. Select the data source that you registered.

  12. Select View details > + New scan, or use the Scan quick-action icon on the source tile.

  13. Provide a Name for the scan.

  14. Under Connect via integration runtime, select the newly created Managed VNet Runtime.

  15. For Credential Select the credential you've registered earlier, choose the appropriate collection for the scan, and select Test connection. On a successful connection, select Continue.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (31)

  16. Follow the steps to select the appropriate scan rule and scope for your scan.

  17. Choose your scan trigger. You can set up a schedule or run the scan once.

  18. Review your scan and select Save and run.

    Managed Virtual Network and managed private endpoints - Microsoft Purview (32)

Next steps

  • Manage data sources in Microsoft Purview
Top Articles
Latest Posts
Article information

Author: Eusebia Nader

Last Updated: 03/01/2023

Views: 6102

Rating: 5 / 5 (80 voted)

Reviews: 87% of readers found this page helpful

Author information

Name: Eusebia Nader

Birthday: 1994-11-11

Address: Apt. 721 977 Ebert Meadows, Jereville, GA 73618-6603

Phone: +2316203969400

Job: International Farming Consultant

Hobby: Reading, Photography, Shooting, Singing, Magic, Kayaking, Mushroom hunting

Introduction: My name is Eusebia Nader, I am a encouraging, brainy, lively, nice, famous, healthy, clever person who loves writing and wants to share my knowledge and understanding with you.