Monday, 15 November 2021

Speaking at the European Collaboration Summit 2021 - Infuse AI into your Power Platform solutions

The European Collaboration Summit (ECS) is a fantastic Microsoft-focused conference in Europe with around 2100 attendees expected, and I'm hugely excited to be speaking there again this year. While we're clearly not done with the pandemic yet, it feels great to be out speaking and networking in-person again and the recent South Coast Summit event here in the UK showed that many others share the same feeling. This year's European Collaboration Summit will be a special event - it essentially combines two conferences in one since the European Cloud Summit which focuses on Azure and Google Cloud has been rolled into this event. I'll be speaking on automation, AI and the Power Platform - some one of my favourite topics of recent times as we embark on the next frontier of process automation.

ECS is held in Düsseldorf this year and as usual has a strong line-up of speakers and significant Microsoft representation. Keynote speakers include Karuana Gatimu, Dan Holme, Vesa Juvonen and others from Microsoft and Bernd Wagner from Google. With over 150 sessions, you can expect deep dives on Microsoft 365, Azure, the Power Platform, Google Cloud as well as business and strategy-focused talks on digital transformation and modern workplace in general. 

The event is almost sold out but it's not too late to get tickets - see below for more details. 

My session - "Infuse AI into your Power Platform solutions"

The session abstract is:

Bringing AI into your applications has never been easier, and huge benefits can be unlocked when some level of intelligent processing of your data happens. From auto-tagging of incident report images uploaded from a Power App, to Flows that detect anomalies in documents, to apps which detect if a retail shelf is low on stock - so many improvements to your SharePoint-based processes are now within easy reach.

In this session we'll cover several scenario demos showing how you can instantly get better insights from your data and better outcomes with little effort - and no developers required! This session is aimed at anyone building solutions on the Power Platform.

The session has a couple of pillars:

  • Demonstrating how to build AI into solutions using the example of an incident reporting Power App. The app uses image recognition to understand photos captured on a mobile phone, auto-tagging the image in Microsoft 365 and integrating with Microsoft Teams to alert a responder group for further triage if the incident appears to be serious
  • An exploration of 3 different techniques to integrate AI. Specifically we look at:
    • Power Apps AI Builder
    • Azure Cognitive Services through code
    • Azure Cognitive Services through Power Automate

Each of these options brings different capability and cost implications, and being able to make the right decisions in this space is vital. One area of focus is cost modelling based on Power Platform premium licensing where I show how what could be considered an architectural choice can have a big impact on the TCO of your solution. 

ECS conference details

Some more high level details of the conference:
  • When - 29 November - 01 December 2021
  • Where - Düsseldorf, Germany
  • Language - English
  • Conference size - expected to be 2100+ attendees
  • Pricing - see the Tickets page on the conference website
It would be great to see you there! The conference website has all the details you need of course:

Wednesday, 8 September 2021

Bringing external data into Microsoft 365 - custom search result types and verticals

 In the previous article we looked at how to integrate other data sources into Microsoft 365, using ServiceNow as an example. This approach of using a Graph Connector to bring external data in can be used with Azure SQL, on-premises SQL Server, Windows file shares, Azure Data Lake, Azure DevOps, Salesforce, Oracle, ServiceNow and several other major platforms - regardless of the target, the process is largely the same. One benefit is that Microsoft Search can now provide a consolidated search engine across all of these platforms - improving discoverability and breaking down silos. There's more to it than that though - beyond search, the other platforms can now be used as "intelligence sources" for Microsoft 365, with some capabilities available now and more to come. Examples of this include:

  • Viva Topics using the other platform to mine knowledge and expertise (later in 2021)
  • "People and workplace intelligence" integrations, where the other platforms feeds information to the Microsoft Graph to power experiences in Microsoft 365. Examples of this might be: 
    • Recommended content from a custom SQL CRM system being displayed to the user on the home page
    • Entities in another system (e.g. a client in Salesforce) appearing in 'used', 'shared' and 'trending' lists shown to the user within Microsoft 365 (date unknown)   
In this article I want to stay focused on search, and in particular discuss how to control the appearance of search results which come from an external platform. Continuing my scenario from the previous article, I'm bringing in ServiceNow Knowledge Base content - in cases like this you will typically want to do two things:
  • Create a custom search vertical - this is a tab in the search results which only shows results from the external platform. This allows users to filter their search to just this content source
  • Create a custom result type - this allows you to control exactly which pieces of data are displayed in the search result and where
Microsoft provide a layout designer to help with this process. This helps you construct an adaptive card layout which gives you the formatting control of the search result, allowing you to specify all of the individual bits of data and where in the result they appear:

So, in this post we'll create a custom search vertical and a custom result type for my ServiceNow scenario. Configuration for both of these can be done at the tenant level in the Microsoft 365 admin portal or at the site collection level in Site Settings. I'm using the former option here, requiring tenant admin permissions. Before we dive in, note that:

Creating a custom search vertical for external search results

If you have extended SharePoint search in the past, you'll probably be familiar with the concept of a search vertical - effectively a custom tab of search results appearing in the search engine. This model continues to be used in modern day Microsoft Search, and the process only takes a few minutes. Essentially we're providing a place for the results to show in a way that's clear where they are from - results do not get interleaved in the main search results. 

To start, find the Search & intelligence section within the Settings menu in the admin portal. From the last post we already have a data source defined (ServiceNow in my case), so we go into the Customizations area and select Verticals and begin the process of adding one:

In the next step, select the definition of the remote content source you are using (i.e. ServiceNow):

You now have a couple of options for pre-filtering the results returned - using KQL or property matching. For example, you could ensure only KB articles created in the last 12 months are used. I'm choosing not to do any pre-filtering:

You can now review the settings used before finishing up:

Now it's just a question of adding and enabling the vertical you just created:

So that's step one. Now that we have a search vertical defined to show results from the data being integrated, it's time to move on to how the results are presented.

Creating a custom search result type

Microsoft Search needs to understand how to present a result from the system you are integrating. Within the admin portal we need to be in the same areas as above (Search & intelligence > Settings > Customizations. This time select Result types and begin the process of adding one:

In the next step we name the result type. Then area on the left in the image below shows the steps we'll walk through:

Now we assign this result type to our previously-created content source:

In the next step we can define criteria for the result type to apply. This gives the control to use multiple result types for a given data source, for example in my case I could have a different layout depending on the category of the ServiceNow Knowledge Base article being returned: 

At this point we can jump off into the layout designer mentioned earlier:

The designer is an external tool currently hosted at - once opened, the first step in the tool is to select one of the provided base templates to use as a starting point:

Once you've chosen a good base, click the "Get Started" button at the bottom of the page. That will take you into this interface where you can map values in the search result to tokens in the layout template:

At this point things can be confusing initially because it may not be clear what to enter into each of the "Property" boxes to do the mapping for things like the title, URL, description etc. - they're just text boxes after all, no pickers. However, that might be because, like me, you didn't pay enough attention to what was on the previous page in the admin center:

If you scroll down on the page we jumped off from, there's a table of "Available properties" which lists the name of every piece of data which can be used in your search result:

Armed with this, the process involves copy/pasting keys into the slots provided in the layout designer.

These are "property labels" you assigned when setting up the data source itself - we did this in the last article in this step:

For my ServiceNow result, I use mappings such as:
  • title = ShortDescription
  • titleUrl = AccessUrl
  • modifiedBy = SysUpdatedBy
..where the item on the left is the "slot" in the SharePoint search result, and the item on the right is the ServiceNow attribute. Once you've filled all the slots, the 'Submit' button the bottom of the page will become enabled and clicking it copies the JSON template which is generated:

You then paste this into the textbox provided in the admin center page:

From there you can review your previous selections and then click the 'Add result type' button. On doing so you should see a successful result:

Your custom search vertical and result type are both now created, complete with all the formatting rules you chose. 

The result should be something like this:

(In Microsoft 365/SharePoint):

(In Microsoft Search in Bing):


Integrating other platforms with Microsoft 365 provides some compelling ways to simplify the user experience. The approach described here with Graph Connectors can provide a unified search experience and single place to go to, but there are other benefits too. As described at the beginning of this article, Microsoft are increasingly looking for ways to drive value from external data - including expanding the content Viva Topics can use for knowledge mining as well as proactively guiding the user to new or modified data and recommended content from the other platform. This is likely to be a technique we'll use heavily in the future.

Monday, 2 August 2021

Bring external data into Microsoft 365 using Graph Connectors - a ServiceNow example

Microsoft 365 has become the centre of the digital universe for many organisations, and it makes sense to explore the integration of other platforms and data sources so that the employee experience is simpler and less fragmented. A great example is search - perhaps most of the organisation's documents are now in Microsoft 365, but when technical knowledge base articles are stored in ServiceNow, client data in Salesforce and there's still a file share of archived (but still useful) content, the net result can be a lot of context switching and time lost to searching. No wonder those McKinsey and IDC research papers suggest an average knowledge worker spends 20-30% of their time simply looking for information. 

The benefits of providing a consolidated search experience often increase as each additional location is integrated - great things can happen when data in a certain system is made more accessible or put in the line of sight of new audiences. The sales and marketing teams might spend all day in CRM, but when client data is put in front of other groups in the company that can help others make better decisions too. In Microsoft 365, the Graph Connectors framework makes this possible - so that's what we're looking at here. 

This article comes in two parts:

For us here at Content+Cloud, a good example of a valuable content source is ServiceNow - it's the platform that powers the Managed Services Provider side of our business, providing core ITSM capabilities around problem/incident/change management as well as various forms of automation, AI and analytics which help us maintain a great service. Something that ServiceNow does quite well is knowledge base tooling - which isn't the easiest thing for me to say as a Microsoft person immersed in Viva Topics and SharePoint Syntex. But it's true - ServiceNow has all sorts of concepts like the Article Quality Index scoring mechanism, the ability to create knowledge from tasks easily and identify possible duplicates - capabilities that suit it's role in providing curated knowledge to support engineers where misinformation can be disastrous. We follow Microsoft and ServiceNow integration closely here at C+C, and there have been LOTS of recent developments - one example is a Microsoft Graph Connector for ServiceNow, and when this became available I knew it would be something useful for us. 

Integrating external data with Microsoft Graph Connectors

At the time of writing Microsoft provide nine native connectors:

In addition, the Microsoft Graph connectors gallery provides a showcase of 3rd party connectors created by other vendors - usually these are paid products, but allow you to go beyond what's possible with Microsoft's connectors. Finally, it's possible to create your own Graph connector as illustrated by the Graph connector GitHub sample

A single framework and approach for all sources - not just ServiceNow
In this article I use ServiceNow as the external data to bring into Microsoft 365, but with Graph connectors the process is 90% the same regardless of data source. So if you want to connect to Azure SQL, Azure Data Lake, a file share, Salesforce or any of the other above sources, the process is much the same. 

The work you do on the non-Microsoft 365 side is generally around creating an app registration or account for the connection to use. You might need to take care of additional things around authorisation and permissions, but there's some flexibility in this in the Graph connector side too. 

Improving search and discoverability through integration 

Being able to index content in other systems allows you to provide a consolidated experience, with the additional location appearing as a search vertical (tab) in the results. You can control how results are displayed, including changing the pieces of data shown and the formatting. Clicking the tab runs your search across this content source (i.e. ServiceNow in my case): 

Viva Topics and Graph connectors
Being able to search and discover across other sources is powerful, but in the future Viva Topics will also be able to use external sources for knowledge gathering. This has the potential to make a huge impact in organisations where authoritative content is split across multiple repositories. Viva Topics will not only do the hard work of identifying, organising and surfacing knowledge across platforms - but will also "bring the knowledge to where the user is" through the topic cards which appear in Teams, Outlook and SharePoint. That could be very compelling indeed as a way to unify platforms and ensure data is brought into the core digital workplace experience.

Configuring the link between Microsoft 365 and ServiceNow

Configuring an OAuth app registration in ServiceNow

In the case of ServiceNow, the first step is to create an OAuth endpoint in ServiceNow for your Graph connector to use. This is documented at ServiceNow Graph connector for Microsoft Search. In my case I'm using a ServiceNow PDI (Personal Developer Instance) and the authentication type I used is ServiceNow OAuth. Here's what my ServiceNow endpoint looks like:

In ServiceNow authZ/authN, a ServiceNow account is also required for the connection - this needs to have access to the knowledge role in the platform. The app registration and the identity are used together in authentication.

Once you have ServiceNow side in place, it's time to create the Graph connector in Microsoft 365.

Creating a Graph connector in Microsoft 365

To start, head into the Admin Center and into Settings > Search & Intelligence.

Look for the 'Data sources' tab and click 'Add':

Select ServiceNow as the data source:

On the next screen you supply a name, ID and description for the connection - these can be any values you like but the ID can only contain alphanumeric characters:

The next screen is where the actual ServiceNow instance details are specified:

When the 'Sign-in' button is clicked a pop-up window will appear for you to login and provide in ServiceNow - the credentials to use at this point are those for the identity which has the knowledge role in ServiceNow:

You are then taken through a consent flow for that user:

Once the consent is granted the connection is tested by Microsoft 365 and the result shown:

The next few screens allow you to specify exactly how the ServiceNow data should be indexed. This process will be different depending on your data source, but for ServiceNow I can first pick the properties/fields to bring in and any additional filter I want to supply: 

The connector does a great job of specifying good defaults on your behalf. For example, in the "Filter data" section above you can see a default query string being used to filter out ServiceNow KB articles which aren't active or at a state of 'published' - this will be what you want in the vast majority of cases. 

The Graph connector framework has a nice preview capability to help you see the results ahead of full configuration. Hitting the 'Preview results' button gives me a few records from the external source:

Security trimming is the next section - depending on the connector used and how you configured authentication, you can specify that permissions in the underlying data source are respected so that the viewing user doesn't see anything different to within the source system:

I allow 'Everyone' to be used in my case, since I'm effectively using a form of app authentication (ServiceNow OAuth + ServiceNow named user) rather than delegated auth with the user's identity which would support this.

In the next step we map properties in the external source to properties in Microsoft 365 search - effectively the same as SharePoint managed properties for those familiar with those. This allows a common set of properties to be used across different content types within search - so that there's always a title, URL, last modified timestamp etc:

Related to this, you can set which properties are Queryable, Searchable, Refinable and Retrievable within Microsoft search. This allows you to control which you might want to use as refiners/filters in search and which can be queried on - for example, you might want to query on a person name and have it return ServiceNow articles where he/she is the author.

Again Microsoft's ServiceNow connector does a great job of providing defaults so you can probably accept them and continue:

The final config setting is how often the indexing process from Microsoft 365 to the source should run - with settings for an incremental and full crawl:

The config is now complete, so let's review settings before confirming:

It's now for Microsoft 365 to publish the connector and start indexing the external source

The result

Once the indexing has happened Microsoft Search is able to show results from the external data source, as shown earlier:

Courtesy of what is now known as 'Microsoft Search in Bing', another surfacing point that will be interesting to some is Bing itself - if it suits me, I can go direct to Bing in my browser and if the integration with my workplace is enabled I get the same results there too in the 'Work' tab:

By extension it should be possible to enable Windows search to cover this too - meaning you could hit the Windows key, type a search term and it search all configured sources including your external data source brought into Microsoft 365 through your Graph connector:


So that's the process and it's not too difficult - most of the effort is on the side of the external system since that's where you'll need to configure authentication. In the next post we'll look at controlling the appearance of search results with:

  • Custom search verticals
  • Custom search result types

Thursday, 8 July 2021

Using RPA to control my ISP settings for kids gaming - Power Automate Desktop for web automation - part 2

In the last post I talked about how I automated my home wi-fi/ISP settings with Power Automate Desktop so that my kids can only use gaming websites like Roblox at permitted times. Since the scheduler in the portal doesn't support what I need, I had been manually visiting the site many times each week to block and unblock sites, which was a huge pain to both me and the kids. Enter Robotic Process Automation - RPA technologies are designed to "automate what cannot be automated", in other words applications without APIs, applications which run on the desktop and other legacy systems. By driving the keyboard and mouse directly, RPA opens the door to wider automation since it simply replicates how a human would interact with the application. Power Automate Desktop is now free with Windows 10 and Windows 11 - Microsoft's move to bring RPA technology to the masses and gain market share as RPA moves from exotic and niche to democratised and commonplace.

In this follow-up post I want to cover a few things:

  • Turning a Power Automate Desktop flow into a cloud flow which can run on a schedule
  • Sending a mobile notification once the flow has run
  • Licensing considerations for Power Automate Desktop
  • Authentication challenges and solutions - working around Captcha requests

In the last post I walked through how I got Power Automate Desktop to open a browser, navigate to my ISP's portal, find the page with the URL blocking, enter the two gaming URLs I want to block into a textbox and then click 'Apply' - with the final step being an announcement to Alexa. This now runs like clockwork every day once the kids have had their prescribed hour of fun. To give you a sense of what that looks like, the portal looks like this:

Where we got to in the last post is that my process was automated, but only in the sense that I have to click a button manually to trigger all the steps to execute. At this point I have a couple of "desktop flows" (one to block the URLs and one to unblock), and they show up in the relevant area in the Power Automate portal:

Of course, what I really want is for this to run in a 100% automated way and on a schedule - with no manual intervention involved. For that I need a cloud flow, so from our points above let's start there. 

Calling a Power Automate Desktop flow from a cloud flow for full automation

Once you have an RPA Flow it can be "wrapped" in a standard cloud flow - a cloud flow being the type of flow that you've been using all along if you've been working with Power Automate previously. Of course, a cloud flow can be triggered in a huge number of ways including when some data is changed in Microsoft 365, SQL or Dataverse or perhaps when something has happened in Salesforce, Workday or another cloud service, or even when an e-mail is received. In my case I just need a simple schedule. 

To get started I go into Flow and create a new scheduled cloud flow:

I then give my flow a name and define the schedule:

Once in the flow I can use the 'Desktop flows' connector to call the flow on my machine - this is the critical thing that links a cloud flow to a desktop flow:

This action allows me to select from a few Microsoft RPA options - note that all are premium actions (more on licensing later):

This action allows me to select from desktop flows I've already created - they are detected because of the connection previously established, which links my Microsoft 365 identity with my physical laptop:

Once I select one of my desktop flows, the action is intelligent enough to recognise any input parameters to that flow. In my case I parameterised the following to avoid them being hard-coded into my desktop flow:
  • RobloxUrl
  • ScratchUrl
  • AlexaAccessCode
The UI adapts to provide a way for me to enter these:

Take note of the "Run Mode" parameter above which is used to specify whether the desktop flow should run in attended or unattended mode - this is important for usage and licensing. We'll come to this later. 

Once things are saved, that's all I need to schedule and fully automate my desktop flow.

However, there's one more thing - the kids hear about the fact that gaming is either open or closed through the automated Alexa announcement I talked about last time. But I'd like to know that everything happened as expected too, so I add a Flow mobile notification:

...and specify the message:

Now wherever I am the notification comes through on my phone each time one of my flows run:

So now everything is fully automated and I have confidence that my automated process is behaving properly. 

Choosing between attended and unattended mode 

When we were creating a cloud flow to "wrap" our desktop flow earlier, we chose "Attended" for the run mode. When working with Power Automate Desktop and RPA in general, attended vs unattended mode is is an important decision to plan for - on the surface, unattended might seem sensible because you can't guarantee that a user will be at the machine when the flow runs, especially for a process that runs a few times through the day. However, an unattended flow can only run under the following circumstances:
  • All users are completely signed out
  • The screen is locked
  • The gateway connection between the cloud flow and the desktop flow has user sign-in information
In fact, a common use case for unattended flows is for a virtual machine to be used - thus giving you more control around guaranteeing these stipulations can be met. Unattended flows can also run concurrently on the same device if different user accounts are used, and this is useful with long-running processes or doing RPA at scale. 

Notably, licensing is significantly different for unattended vs. attended flows - so this is another reason why planning is required here. Let's turn to licensing now.

Licensing for Power Automate Desktop

I mentioned earlier that Power Automate Desktop is free with Windows 10/11, and that's true. What that gets you is the ability to click a button in Power Automate Desktop client on your machine and have your process run - the further automation that comes with a cloud flow requires pay-for Power Automate licensing however. As usual with Power Automate there are two ways of licensing this - per user and per flow. For per user licensing, you need a specific plan named the "Per-user plan with attended RPA", as shown below on the Microsoft pricing page:

Notice the additional "per bot" consideration if you want to run unattended (the second row). If you're doing per-user licensing you add the "unattended RPA add-on" to the per-user price:

So in summary, the options are:

Mode License approach Base cost Unattended add-on cost Total cost per month
Attended Per user $40 $15 per user per month (until 31 September 2021) - $15 (until 31 September 2021)
Attended Per flow $500 per month - $500
Unattended Per user $40 $15 per user per month (until 31 September 2021) $150 per month $165 (until 31 September 2021)
Unattended Per flow $500 per month $150 per month $650

As ever, you should refer to the official Power Automate licensing page for the current pricing and considerations between per user and per flow licensing.

From Microsoft's pricing you might observe:
  • There's a fairly significant premium for unattended processing. That's because you can automate at scale with this option with many processes running concurrently - if you're using RPA to process 10,000 invoices at month end, this incremental cost is likely to be justified of course.
  • The decision between per user and per flow licensing for Power Automate overall is relevant in this area too - most orgs decide this based on automation strategy and requirements they have in view  
  • Whilst there are some incentives at the time of writing (June/July 2021), it seems that the new pricing applied to Power Apps at this time doesn't carry over to Power Automate in case you're wondering
Overall, you can see this is designed for organizational use rather than anything personal or home-based. In that context of course, if a high value process is being automated the value of this to the organization could be many multiples of the cost - so comparing unattended RPA in Power Automate Desktop with non-enterprise solutions doesn't really make sense. And of course, my scenario of using this business technology to automate my wi-fi settings at home doesn't really make sense either - I created my solution on trial licensing, but if I needed to license this for the long-term the costs (even at $15 per month, which is all I'd need) would be a factor. None of this takes away from the power of the technology however - in a business context Power Automate Desktop is extremely powerful.

And when the trial licensing expires on my solution, I'll just click a single button each day and it will still be less work than going to my ISP portal and entering/removing URLs!

A final challenge for RPA - Captcha prompts

One final topic to address is how to use RPA to automate around visual Captcha challenges like these: 

Of course, the whole point of this kind of check is to ensure there's a human making the request rather than an automated bot - but this is to foil a malicious bot of some kind, rather than our wholesome, legitimate and approved automation. The image above shows the Captcha challenge presented when logging into my ISP's portal.

I suspect it may be possible to use a combination of DOM scraping and image recognition (e.g. Azure Cognitive Services Vision API) to get past something like this - however, reliability could be an issue since these images are small and frankly difficult enough for humans. So, my solution is far less exotic unfortunately:

  • Create a dedicated browser profile and pre-authenticate to the ISP site
  • Ensure the RPA process always uses this browser profile:
  • Enjoy the fact that persistent cookies last a long time with my ISP

So perhaps working around the Captcha problem more than solving it, but since I only need to take action every few weeks when the cookies expire that's good enough in this case. Clearly this could be a challenge for some automation scenarios which target public SaaS services in the enterprise, but wouldn't be common with other targets such as legacy non-cloud applications.


Over these posts we've seen how effective Power Automate Desktop can be in "automating what cannot be automated". We've discussed the basic "Power Automate Desktop only" approach vs. bringing a desktop flow into a cloud flows to facilitate scheduling or integration with other cloud processes - the latter method requires pay-for licensing in the form of a Power Automate per-user or per-flow license, and unattended processing comes with an extra cost but does enable automation at scale. 

As we noted that the technology is pitched at business automation rather than personal automation. The potential is clear, and it's great to see Microsoft have a strong easy-to-use offering in this space.