Category: Python

Python + Dataverse Series – #07: Running a Linear Normalization Algorithm on Dataverse Data Using Python

This is continuation in this series of Dataverse SDK for Python, if you haven’t checked out earlier articles, I would encourage to start from the beginning of this series.

Python + Dataverse Series – How to run Python Code in Vs Code

Machine learning often begins with one essential step: data preprocessing. Before models can learn patterns, the raw data must be cleaned, scaled, and transformed into a form suitable for analysis. In this example, let me demonstrate how to retrieve numerical data from Microsoft Dataverse and apply a linear normalization algorithm using Python.

Normalization is a fundamental algorithm in machine learning pipelines. It rescales numeric values into a consistent range—typically between 0 and 1—making them easier for algorithms to interpret and compare.

1. Retrieving Data from Dataverse

Using the DataverseClient and Interactive Browser authentication, we connect to Dataverse and fetch the revenue field from the Account table. This gives us a small dataset to run our algorithm on.

from azure.identity import InteractiveBrowserCredential 
from PowerPlatform.Dataverse.client import DataverseClient  
credential = InteractiveBrowserCredential()
client = DataverseClient("https://ecellorsdev.crm8.dynamics.com", credential)
account_batches = client.get(
    "account",
    select=["accountid", "revenue"],
    top=10,
)

We then extract the revenue values into a NumPy array.

2. Implementing the Linear Normalization Algorithm

The algorithm used here is min–max normalization, defined mathematically as:normalized=x−min⁡(x)max⁡(x)−min⁡(x)This algorithm ensures:

  • the smallest value becomes 0
  • the largest becomes 1
  • all other values fall proportionally in between

Here’s the implementation:

import numpy as np
revenues = np.array(revenues)
min_rev = np.min(revenues)
max_rev = np.max(revenues)
normalized_revenues = (revenues - min_rev) / (max_rev - min_rev)

This is a classic preprocessing algorithm used in machine learning pipelines before feeding data into models such as regression, clustering, or neural networks.

3. Visualizing the Normalized Output

To better understand the effect of the algorithm, we plot the normalized values:

import matplotlib.pyplot as plt 
plt.plot(normalized_revenues, marker='o')
plt.title('Normalized Revenues from Dataverse Accounts')
plt.xlabel('Account Index')
plt.ylabel('Normalized Revenue')
plt.grid()
plt.show()

The visualization highlights how the algorithm compresses the original revenue values into a uniform scale.

4. Why Normalization Matters

Normalization is not just a mathematical trick—it’s a crucial algorithmic step that:

  • prevents large values from dominating smaller ones
  • improves convergence in optimization-based models
  • enhances the stability of distance‑based algorithms
  • makes datasets comparable across different ranges
#Running Machine Learning Algorithm on data retrieved from Dataverse to run a linear normalization
from azure.identity import InteractiveBrowserCredential
from PowerPlatform.Dataverse.client import DataverseClient
import numpy as np
# Connect to Dataverse
credential = InteractiveBrowserCredential()
client = DataverseClient("https://ecellorsdev.crm8.dynamics.com", credential)
# Fetch account data as paged batches
account_batches = client.get(
"account",
select=["accountid", "revenue"],
top=10,
)
revenues = []
for batch in account_batches:
for account in batch:
if "revenue" in account and account["revenue"] is not None:
revenues.append(account["revenue"])
revenues = np.array(revenues)
# Apply a simple linear algorithm: Normalize the revenues
if len(revenues) > 0:
min_rev = np.min(revenues)
max_rev = np.max(revenues)
normalized_revenues = (revenues – min_rev) / (max_rev – min_rev)
print("Normalized Revenues:", normalized_revenues)
#visualize the result
import matplotlib.pyplot as plt
plt.plot(normalized_revenues, marker='o')
plt.title('Normalized Revenues from Dataverse Accounts')
plt.xlabel('Account Index')
plt.ylabel('Normalized Revenue')
plt.grid()
plt.show()
view raw Python-RetrieveData-ApplyLinearAlgorithm hosted with ❤ by GitHub

The use of this code is to transform raw Dataverse revenue data into normalized, machine‑learning‑ready values that can be analyzed, compared, and visualized effectively.

You can download the Python Notebook below if you want to work with VS Code

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-RetrieveData-ApplyLinearAlgorithm.ipynb

Once you have opened the Python notebook, you can start to run the code as below

You should see something like below

For authentication in another browser tab, once authenticated, you should be able to see the

Hope you found this useful…it’s going to be interesting, stay tuned for upcoming articles.

Cheers,

PMDY

Python + Dataverse Series – #06: Data preprocessing steps before running Machine Learning Algorithms

Hi Folks,

If you were already a Power Platform Consultant and new to working with Python, then I would encourage to start from the beginning of this series.

Python + Dataverse Series – How to run Python Code in Vs Code

Now in this series, we entered an interesting part where Machine learning algorithms were run to analyze Dataverse Data and in this post we will understand why feature scaling is a critical preprocessing step for many machine learning algorithms because it ensures that all features contribute equally to the model’s outcome, prevents numerical instability, and helps optimization algorithms converge faster to the optimal solution

Primarily before running any Machine Learning Algorithm, we need to do some data preprocessing like scaling the data, in this case we will use a formula which is used to scale using min–max normalization (feature scaling to the [0, 1] range).

#preprocessing step before running machine learning algorithms
from azure.identity import InteractiveBrowserCredential #using Interactive Login
from PowerPlatform.Dataverse.client import DataverseClient #installing Python SDK for Dataverse
import numpy as np #import Numpy Library to perform calculations
# Connect to Dataverse
credential = InteractiveBrowserCredential()
client = DataverseClient("https://ecellorsdev.crm8.dynamics.com", credential) #Creates Dataverse Client
# Fetch account data as paged batches
account_batches = client.get(
"account",
select=["accountid", "revenue"],
top=10,
) #Fetches top 10 accounts with accountid, revenue columns
revenues = []
for batch in account_batches:
for account in batch:
if "revenue" in account and account["revenue"] is not None:
revenues.append(account["revenue"])
revenues = np.array(revenues)
#Normalize the revenue
if len(revenues) > 0:
min_rev = np.min(revenues)
max_rev = np.max(revenues)
normalized_revenues = (revenues – min_rev) / (max_rev – min_rev)
print("Normalized Revenues:", normalized_revenues)
#visualize the result
import matplotlib.pyplot as plt
plt.plot(normalized_revenues, marker='o')
plt.title('Normalized Revenues from Dataverse Accounts')
plt.xlabel('Account Index')
plt.ylabel('Normalized Revenue')
plt.grid()
plt.show()
view raw preprocessing step before running machine learning algorithms hosted with ❤ by GitHub

You can download the Python Notebook below if you want to work with VS Code

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-PreProcessingStepBeforeMachineLearning.ipynb

Hope you found this useful…

Cheers,

PMDY

Python + Dataverse Series – How to run Python Code in Vs Code

Hi Folks,

As you folks know that Python currently is the number #1 programming language with a massive, versatile ecosystem of libraries for data science, AI, and backend web development. This post kicks off a hands‑on series about working with Microsoft Dataverse using Python. We’ll explore how to use the Dataverse SDK for Python to connect with Dataverse, automate data operations, and integrate Python solutions across the broader Power Platform ecosystem. Whether you’re building data-driven apps, automating workflows, or extending Power Platform capabilities with custom logic, this series will help you get started with practical, real‑world examples.

https://www.microsoft.com/en-us/power-platform/blog/2025/12/03/dataverse-sdk-python/

With the release of the Dataverse SDK for Python, building Python-based logic for the Power Platform has become dramatically simpler. In this post, we’ll walk through how to download Python and set it up in Visual Studio Code so you can start building applications that interact with Dataverse using Python. Sounds exciting already. Let’s dive in and get everything set up..

1. Download and install Python from official website below and then install it in your computer.

https://www.python.org/ftp/python/3.14.3/python-3.14.3-amd64.exe

2. Install VS Code

Important: During installation, make sure to check “Add Python to PATH”. This ensures VS Code can detect Python automatically.

3. After installation, open VS Code and install the Python extension (Microsoft’s official one). This extension enables IntelliSense, debugging, and running Python script

4. That’s it, you are now able to run Python logic inside Vs Code

5. Create or Open a Python file in the system, opened a sample file below

5. If you want to run Python Programmes in your VS Code, follow below options

a. Select Start Debugging

b. You will be prompted a window like below

You can select the first option highlighted above, it automatically runs your Python Code

This is very easy to setup…

If you want to continue reading this series, check out the next article.

Python + Dataverse Series – #01 – Console Application using Python

Hope this helps…

Cheers,

PMDY

Python + Dataverse Series – #05: Remove PII

Hi Folks,

This is in continuation in the Python + Dataverse series, it is worth checking out from the start of this series here.

At times, there will be a need to remove PII(Personally Identifiable Information) present in the Dataverse Environments, for this one time task, you can easily run Python script below, let’s take example of removing PII from Contact fields in the below example.

from azure.identity import InteractiveBrowserCredential
from PowerPlatform.Dataverse.client import DataverseClient
# Connect to Dataverse
credential = InteractiveBrowserCredential()
client = DataverseClient("https://ecellorsdev.crm8.dynamics.com", credential)
#use AI to remove PII data from the dataverse records, let's say contact records
def remove_pii_from_contact(contact):
pii_fields = ['emailaddress1', 'telephone1', 'mobilephone', 'address1_line1', 'address1_city', 'address1_postalcode']
for field in pii_fields:
if field in contact:
contact[field] = '[REDACTED]'
return contact
# Fetch contacts with PII (Dataverse client returns paged batches)
contact_batches = client.get(
"contact",
select=[
"contactid",
"fullname",
"emailaddress1",
"telephone1",
"mobilephone",
"address1_line1",
"address1_city",
"address1_postalcode",
],
top=10,
)
# Remove PII and update contacts
for batch in contact_batches:
for contact in batch:
contact_id = contact.get("contactid")
sanitized_contact = remove_pii_from_contact(contact)
# Prepare update data (exclude contactid)
update_data = {key: value for key, value in sanitized_contact.items() if key != "contactid"}
# Update the contact in Dataverse
client.update("contact", contact_id, update_data)
print(f"Contact {contact_id} updated with sanitized data: {sanitized_contact}")
view raw Easy way to remove PII in Dataverse using Python hosted with ❤ by GitHub

If you want to work on this, download the Python Notebook to use in VS Code…

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-DataverseSDK-RemovePII.ipynb

Cheers,

PMDY

Python + Dataverse Series – #04: Create records in batch using Execute Multiple

Hi Folks,

This is continuation in this Python with Dataverse Series, in this blog post, we will see how can we create multiple records in a single batch using ExecuteMultiple in Python.

Please use the below code for the same…to make any calls using ExecuteMultiple…

import pyodbc
import msal
import requests
import json
import re
import time
# Azure AD details
client_id = '0e1c58b1-3d9a-4618-8889-6c6505288d3c'
client_secret = 'qlU8Q~dmhKFfdL1ph2YsLK9URbhIPn~qWmfr1ceL'
tenant_id = '97ae7e35-2f87-418b-9432-6733950f3d5c'
authority = f'https://login.microsoftonline.com/{tenant_id}'
resource = 'https://ecellorsdev.crm8.dynamics.com'
# SQL endpoint
sql_server = 'ecellorsdev.crm8.dynamics.com'
database = 'ecellorsdev'
# Get token with error handling
try:
print(f"Attempting to authenticate with tenant: {tenant_id}")
print(f"Authority URL: {authority}")
app = msal.ConfidentialClientApplication(client_id, authority=authority, client_credential=client_secret)
print("Acquiring token…")
token_response = app.acquire_token_for_client(scopes=[f'{resource}/.default'])
if 'error' in token_response:
print(f"Token acquisition failed: {token_response['error']}")
print(f"Error description: {token_response.get('error_description', 'No description available')}")
else:
access_token = token_response['access_token']
print("Token acquired successfully and your token is!"+access_token)
print(f"Token length: {len(access_token)} characters")
except ValueError as e:
print(f"Configuration Error: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
#Get 5 contacts from Dataverse using Web API
import requests
import json
try:
#Full CRUD Operations – Create, Read, Update, Delete a contact in Dataverse
print("Making Web API request to perform CRUD operations on contacts…")
# Dataverse Web API endpoint for contacts
web_api_url = f"{resource}/api/data/v9.2/contacts"
# Base headers with authorization token
headers = {
'Authorization': f'Bearer {access_token}',
'OData-MaxVersion': '4.0',
'OData-Version': '4.0',
'Accept': 'application/json',
'Content-Type': 'application/json'
}
# Simple approach: create multiple contacts sequentially
# generate 100 contacts with different last names
contacts_to_create = [
{"firstname": "Ecellors", "lastname": f"Test{str(i).zfill(3)}"}
for i in range(1, 101)
]
create_headers = headers.copy()
create_headers['Prefer'] = 'return=representation'
created_ids = []
print("Creating contacts sequentially…")
for i, body in enumerate(contacts_to_create, start=1):
try:
resp = requests.post(web_api_url, headers=create_headers, json=body, timeout=15)
except requests.exceptions.RequestException as e:
print(f"Request error creating contact #{i}: {e}")
continue
if resp.status_code in (200, 201):
try:
j = resp.json()
cid = j.get('contactid')
except ValueError:
cid = None
if cid:
created_ids.append(cid)
print(f"Created contact #{i} with id: {cid}")
else:
print(f"Created contact #{i} but response body missing id. Response headers: {resp.headers}")
elif resp.status_code == 204:
# try to extract id from headers
entity_url = resp.headers.get('OData-EntityId') or resp.headers.get('Location')
if entity_url:
m = re.search(r"([0-9a-fA-F\-]{36})", entity_url)
if m:
cid = m.group(1)
created_ids.append(cid)
print(f"Created contact #{i} (204) with id: {cid}")
else:
print(f"Created contact #{i} (204) but couldn't parse id from headers: {resp.headers}")
else:
print(f"Created contact #{i} (204) but no entity header present: {resp.headers}")
else:
print(f"Failed to create contact #{i}. Status code: {resp.status_code}, Response: {resp.text}")
# small pause to reduce chance of throttling/rate limits
time.sleep(0.2)
if created_ids:
print("Created contact ids:")
for cid in created_ids:
print(cid)
except Exception as e:
print(f"Unexpected error during Execute Multiple: {e}")
print("Failed to extract Contact ID from headers.")
view raw Python – Dataverse – ExecuteMultipleCreate hosted with ❤ by GitHub

Please download this Jupyter notebook to work on it easily using VS Code.

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-Dataverse-ExecuteMultipleUsingPython.ipynb

If you want to continue reading this series, follow along

Python + Dataverse Series – #05: Remove PII

Hope this helps..

Cheers,

PMDY

Python + Dataverse Series – Post #03: Create, Update, Delete records via Web API

Hi Folks,

This is continuation in this Python with Dataverse Series, in this blog post, we will perform a full CRUD(Create, Retrieve, Update, Delete) in Dataverse using Web API.

Please use the below code for the same…to make any calls using WEB API to Dataverse.

import pyodbc
import msal
import requests
import json
import re
# Azure AD details
client_id = 'XXXX'
client_secret = 'XXXX'
tenant_id = 'XXXX'
authority = f'https://login.microsoftonline.com/{tenant_id}'
resource = 'https://XXXX.crm8.dynamics.com'
# SQL endpoint
sql_server = 'XXXX.crm8.dynamics.com'
database = 'XXXX'
# Get token with error handling
try:
print(f"Attempting to authenticate with tenant: {tenant_id}")
print(f"Authority URL: {authority}")
app = msal.ConfidentialClientApplication(client_id, authority=authority, client_credential=client_secret)
print("Acquiring token…")
token_response = app.acquire_token_for_client(scopes=[f'{resource}/.default'])
if 'error' in token_response:
print(f"Token acquisition failed: {token_response['error']}")
print(f"Error description: {token_response.get('error_description', 'No description available')}")
else:
access_token = token_response['access_token']
print("Token acquired successfully and your token is!"+access_token)
print(f"Token length: {len(access_token)} characters")
except ValueError as e:
print(f"Configuration Error: {e}")
print("\nPossible solutions:")
print("1. Verify your tenant ID is correct")
print("2. Check if the tenant exists and is active")
print("3. Ensure you're using the right Azure cloud (commercial, government, etc.)")
except Exception as e:
print(f"Unexpected error: {e}")
#Get 5 contacts from Dataverse using Web API
import requests
import json
try:
#Full CRUD Operations – Create, Read, Update, Delete a contact in Dataverse
print("Making Web API request to perform CRUD operations on contacts…")
# Dataverse Web API endpoint for contacts
web_api_url = f"{resource}/api/data/v9.2/contacts"
# Base headers with authorization token
headers = {
'Authorization': f'Bearer {access_token}',
'OData-MaxVersion': '4.0',
'OData-Version': '4.0',
'Accept': 'application/json',
'Content-Type': 'application/json'
}
# Create a new contact
new_contact = { "firstname": "John", "lastname": "Doe" }
print("Creating a new contact…")
# Request the server to return the created representation. If not supported or omitted,
# Dataverse often returns 204 No Content and provides the entity id in a response header.
create_headers = headers.copy()
create_headers['Prefer'] = 'return=representation'
response = requests.post(web_api_url, headers=create_headers, json=new_contact)
created_contact = {}
contact_id = None
# If the API returned the representation, parse the JSON
if response.status_code in (200, 201):
try:
created_contact = response.json()
except ValueError:
created_contact = {}
contact_id = created_contact.get('contactid') or created_contact.get('contactid@odata.bind')
print("New contact created successfully (body returned).")
print(f"Created Contact ID: {contact_id}")
# If the API returned 204 No Content, Dataverse includes the entity URL in 'OData-EntityId' or 'Location'
elif response.status_code == 204:
entity_url = response.headers.get('OData-EntityId') or response.headers.get('Location')
if entity_url:
# Extract GUID using regex (GUID format)
m = re.search(r"([0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12})", entity_url)
if m:
contact_id = m.group(1)
created_contact = {'contactid': contact_id}
print("New contact created successfully (no body). Extracted Contact ID from headers:")
print(f"Created Contact ID: {contact_id}")
else:
print("Created but couldn't parse entity id from response headers:")
print(f"Headers: {response.headers}")
else:
print("Created but no entity location header found. Headers:")
print(response.headers)
else:
print(f"Failed to create contact. Status code: {response.status_code}")
print(f"Error details: {response.text}")
# Read the created contact
if not contact_id:
# Defensive: stop further CRUD if we don't have an id
print("No contact id available; aborting read/update/delete steps.")
else:
print("Reading the created contact…")
response = requests.get(f"{web_api_url}({contact_id})", headers=headers)
if response.status_code == 200:
print("Contact retrieved successfully!")
contact_data = response.json()
print(json.dumps(contact_data, indent=4))
else:
print(f"Failed to retrieve contact. Status code: {response.status_code}")
print(f"Error details: {response.text}")
# Update the contact's email
updated_data = { "emailaddress1": "john.doe@example.com" }
response = requests.patch(f"{web_api_url}({contact_id})", headers=headers, json=updated_data)
if response.status_code == 204:
print("Contact updated successfully!")
else:
print(f"Failed to update contact. Status code: {response.status_code}")
print(f"Error details: {response.text}")
# Delete the contact
response = requests.delete(f"{web_api_url}({contact_id})", headers=headers)
if response.status_code == 204:
print("Contact deleted successfully!")
else:
print(f"Failed to delete contact. Status code: {response.status_code}")
print(f"Error details: {response.text}")
except requests.exceptions.RequestException as e:
print(f"Request error: {e}")
except KeyError as e:
print(f"Token not available: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
view raw FullCRUDWebAPI hosted with ❤ by GitHub

You can use the VS Code as IDE, copy the above code in a python file, next click on Run Python File at the top of the VS Code

Hope this helps someone making Web API Calls using Python.

If you want to try this out, download the Python Notebook and open in VS Code.

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-Dataverse-FullCRUD-03.ipynb

Looking to continue following this series, don’t forget the next article in this series

Python + Dataverse Series – #04: Create records in batch using Execute Multiple

Cheers,

PMDY

Python + Dataverse Series – #02 – use Datavese Web API using Python

Hi Folks,

This is in continuation to the previous blog post…if you haven’t gone through the earlier post on connecting to Dataverse using Python, please have a look here

Python + Dataverse Series – #01 – Console Application using Python

Now, we will see how you can retrieve the records in Dataverse using Web API using Python…

  1. Follow the previous blog post for connecting to Dataverse using Python
  2. Once you get the access token via the TDS End point, we can invoke the Dataverse Web API using below code…
import pyodbc
import msal
import requests
import json
# Azure AD details
client_id = 'XXXX'
client_secret = 'XXXX'
tenant_id = 'XXXX'
authority = f'https://login.microsoftonline.com/{tenant_id}'
resource = 'https://XXXX.crm8.dynamics.com'
# SQL endpoint
sql_server = 'XXXX.crm8.dynamics.com'
database = 'XXXX'
# Get token with error handling
try:
print(f"Attempting to authenticate with tenant: {tenant_id}")
print(f"Authority URL: {authority}")
app = msal.ConfidentialClientApplication(client_id, authority=authority, client_credential=client_secret)
print("Acquiring token…")
token_response = app.acquire_token_for_client(scopes=[f'{resource}/.default'])
if 'error' in token_response:
print(f"Token acquisition failed: {token_response['error']}")
print(f"Error description: {token_response.get('error_description', 'No description available')}")
else:
access_token = token_response['access_token']
print("Token acquired successfully and your token is!"+access_token)
print(f"Token length: {len(access_token)} characters")
except ValueError as e:
print(f"Configuration Error: {e}")
print("\nPossible solutions:")
print("1. Verify your tenant ID is correct")
print("2. Check if the tenant exists and is active")
print("3. Ensure you're using the right Azure cloud (commercial, government, etc.)")
except Exception as e:
print(f"Unexpected error: {e}")
#Get 5 contacts from Dataverse using Web API
import requests
import json
try:
print("Making Web API request to get contacts…")
# Dataverse Web API endpoint for contacts
web_api_url = f"{resource}/api/data/v9.2/contacts"
# Set up headers with authorization token
headers = {
'Authorization': f'Bearer {access_token}',
'OData-MaxVersion': '4.0',
'OData-Version': '4.0',
'Accept': 'application/json',
'Content-Type': 'application/json'
}
# Add query parameters to get only 5 contacts with specific fields
params = {
'$top': 5,
'$select': 'contactid,fullname,emailaddress1,telephone1,createdon'
}
# Make the GET request
response = requests.get(web_api_url, headers=headers, params=params)
if response.status_code == 200:
print("Web API request successful!")
contacts_data = response.json()
print(f"\nFound {len(contacts_data['value'])} contacts:")
print("-" * 80)
for i, contact in enumerate(contacts_data['value'], 1):
print(f"Contact {i}:")
print(f" ID: {contact.get('contactid', 'N/A')}")
print(f" Name: {contact.get('fullname', 'N/A')}")
print(f" Email: {contact.get('emailaddress1', 'N/A')}")
print(f" Phone: {contact.get('telephone1', 'N/A')}")
print(f" Created: {contact.get('createdon', 'N/A')}")
print("-" * 40)
else:
print(f"Web API request failed with status code: {response.status_code}")
print(f"Error details: {response.text}")
except requests.exceptions.RequestException as e:
print(f"Request error: {e}")
except KeyError as e:
print(f"Token not available: {e}")
except Exception as e:
print(f"Unexpected error: {e}")
view raw Python – Dataverse – Web API hosted with ❤ by GitHub

You can use the VS Code as IDE, copy the above code in a python file, next click on Run Python File at the top of the VS Code

So, once you get the Access token, you can invoke the Web API using Python similar to how we did it using Javascript…

Please download the Python Jupyter Notebook if you want to work on this in VS Code

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-Dataverse-WebAPI-GetData-02.ipynb

If you want to follow along in this series, please see below post

Python + Dataverse Series – Post #03: Create, Update, Delete records via Web API

Hope this helps…

Cheers,

PMDY

Python + Dataverse Series – #01 – Console Application using Python

Hi Folks,

This series is for Pro Code Developers especially those working on Dataverse and want to know how to work with Dataverse and Python. I am starting this series as I see little to no content in this area.

So, in this post, first we will try to understand how to write a console application using Python Code utilizing(Tabular Data Stream) the TDS end point. Well, there were many posts in the internet for connecting to Dataverse using Python but uses more libraries and requires bit more code

Below posts will have hardcoded configurations as they are meant for initial trial purposes, going further, we will align with the best practices.

import pyodbc
import msal
# Azure AD details
client_id = '0e1c58b1-3d9a-4618-8889-6c6505288d3c'
client_secret = 'qlU8Q~dmhKFfdL1ph2YsLK9URbhIPn~qWmfr1ceL'
tenant_id = '97ae7e35-2f87-418b-9432-6733950f3d5c'
authority = f'https://login.microsoftonline.com/{tenant_id}'
resource = 'https://ecellorsdev.crm8.dynamics.com'
# SQL endpoint
sql_server = 'ecellorsdev.crm8.dynamics.com'
database = 'ecellorsdev'
# Get token with error handling
try:
print(f"Attempting to authenticate with tenant: {tenant_id}")
print(f"Authority URL: {authority}")
app = msal.ConfidentialClientApplication(client_id, authority=authority, client_credential=client_secret)
print("Acquiring token…")
token_response = app.acquire_token_for_client(scopes=[f'{resource}/.default'])
if 'error' in token_response:
print(f"Token acquisition failed: {token_response['error']}")
print(f"Error description: {token_response.get('error_description', 'No description available')}")
else:
access_token = token_response['access_token']
print("Token acquired successfully and your token is!"+access_token)
print(f"Token length: {len(access_token)} characters")
except ValueError as e:
print(f"Configuration Error: {e}")
print("\nPossible solutions:")
print("1. Verify your tenant ID is correct")
print("2. Check if the tenant exists and is active")
print("3. Ensure you're using the right Azure cloud (commercial, government, etc.)")
except Exception as e:
print(f"Unexpected error: {e}")
view raw ConsoleApplicationUsingPython.py hosted with ❤ by GitHub

The logic just uses two libraries in Python

  1. pyodbc
  2. msal

The code efficiently handles all the errors for efficient tracking….

You can easily work with Python using VS Code as below,

Hover over Run option –> Click Start Debugging

You will able to get the Access Token after invoking the Dataverse.

Download the Python Jupyter Notebook if you want to work on this in VS Code.

https://github.com/pavanmanideep/DataverseSDK_PythonSamples/blob/main/Python-Dataverse-ConsoleApp-01.ipynb

Hope this posts helps…

If you want to continue reading this series, please follow along

Python + Dataverse Series – #02 – use Datavese Web API using Python

Cheers,

PMDY

Embed Python Visuals in Power BI Desktop – Quick Review

Hi Folks,

This post is all about embedding Python visuals in Power BI, you will need to install the respective dependent libraries like Seaborn, Matplotlib when you were creating visuals as we are using the respective libraries.

Thank you @Dr.S.Gomathi for sharing insights at GPPB Tamil Nadu, 2024, while I don’t know that Power BI has this capability. I am writing this down.

The first thing you need to do is to install Python, you can install the latest version from internet. Click here to Download Python for Windows.

Once downloaded and installed in your local machine, you can find a folder created under your Windows Start menu like below.

You need to right click on Python 3.11(64-bit) icon which is the current latest version and then click on open file location.

Then you will be able to see the contents in the folder

You need again right click on Python 3.11 (64 bit) and open its actual contents where the library files reside.

Copy this path, we need this in a while.

Now open Power BI Desktop and navigate to File –> Options and Settings –> Options

Now in the options and settings, you need to select on the Python scripting and specify the path which you just copied above as below.

Now you were ready to use Python visuals in Power BI.

Next step is to click on Python visual as highlighted below

You will be then asked to enable Python scripts as below

You will need to click on Enable as shown above. Once it is done, you are ready to start using Python visuals in Power BI.

Then you need to load data from your DataSource. Here is the link to the excel I have used. Once data is loaded into your Power BI report, you need to select respective data fields which you want to visualize. Here I am using two fields for X and Y axis, then in Power BI Desktop, you should be able to see something like below.

and in order to effectively visualize the sales trends, I will be visualizing the data using Violin Chart, which is using Seaborn library, while the Seaborn is actually based on Matplotlib library. So, I need to make sure I have those two libraries installed in my machine. You can install by using Command Prompt in your PC, you need to enter below commands and press enter to install.

pip install matplot lib

pip install seaborn

Once installed, we can plot using the below command in the python script tab in Power BI Desktop

# The following code to create a dataframe and remove duplicated rows is always executed and acts as a preamble for your script:
# dataset = pandas.DataFrame(Sales, Country)
# dataset = dataset.drop_duplicates()
# Paste or type your script code here:
import pandas as pd
import seaborn as sns
import matplotlib.pyplot as plt
# Set the aesthetic style of the plots
sns.set_style("whitegrid")
# Create a violin plot for Sales Satisfaction across different Product Categories
plt.figure(figsize=(12, 8))
sns.violinplot(x='Year', y='COGS', data=dataset, palette='muted', split=True)
plt.title('Sales by Product Category')
plt.xlabel('Product Category')
plt.ylabel('Customer Satisfaction Rating')
plt.show()
view raw Violin Plot Python Code.py hosted with ❤ by GitHub

Then you may need to click on run script as highlighted below

This gives your Violin chart showing the sales distribution for different product categories in your Power BI Desktop. If you were facing any problems viewing the report, check the error in the pop-up message displayed by Power BI, you can also follow the Microsoft article on this mentioned in the references.

Hope this helps someone trying to use Python visuals inside Power BI. Same way, you can use different visualizations available with Python which were not available in Power BI by default.

References:

https://learn.microsoft.com/en-us/power-bi/connect-data/desktop-python-scripts

Cheers,

PMDY