Wednesday 8 September 2021

Bringing external data into Microsoft 365 - custom search result types and verticals

 In the previous article we looked at how to integrate other data sources into Microsoft 365, using ServiceNow as an example. This approach of using a Graph Connector to bring external data in can be used with Azure SQL, on-premises SQL Server, Windows file shares, Azure Data Lake, Azure DevOps, Salesforce, Oracle, ServiceNow and several other major platforms - regardless of the target, the process is largely the same. One benefit is that Microsoft Search can now provide a consolidated search engine across all of these platforms - improving discoverability and breaking down silos. There's more to it than that though - beyond search, the other platforms can now be used as "intelligence sources" for Microsoft 365, with some capabilities available now and more to come. Examples of this include:

  • Viva Topics using the other platform to mine knowledge and expertise (later in 2021)
  • "People and workplace intelligence" integrations, where the other platforms feeds information to the Microsoft Graph to power experiences in Microsoft 365. Examples of this might be: 
    • Recommended content from a custom SQL CRM system being displayed to the user on the Office.com home page
    • Entities in another system (e.g. a client in Salesforce) appearing in 'used', 'shared' and 'trending' lists shown to the user within Microsoft 365 (date unknown)   
In this article I want to stay focused on search, and in particular discuss how to control the appearance of search results which come from an external platform. Continuing my scenario from the previous article, I'm bringing in ServiceNow Knowledge Base content - in cases like this you will typically want to do two things:
  • Create a custom search vertical - this is a tab in the search results which only shows results from the external platform. This allows users to filter their search to just this content source
  • Create a custom result type - this allows you to control exactly which pieces of data are displayed in the search result and where
Microsoft provide a layout designer to help with this process. This helps you construct an adaptive card layout which gives you the formatting control of the search result, allowing you to specify all of the individual bits of data and where in the result they appear:

So, in this post we'll create a custom search vertical and a custom result type for my ServiceNow scenario. Configuration for both of these can be done at the tenant level in the Microsoft 365 admin portal or at the site collection level in Site Settings. I'm using the former option here, requiring tenant admin permissions. Before we dive in, note that:


Creating a custom search vertical for external search results

If you have extended SharePoint search in the past, you'll probably be familiar with the concept of a search vertical - effectively a custom tab of search results appearing in the search engine. This model continues to be used in modern day Microsoft Search, and the process only takes a few minutes. Essentially we're providing a place for the results to show in a way that's clear where they are from - results do not get interleaved in the main search results. 

To start, find the Search & intelligence section within the Settings menu in the admin portal. From the last post we already have a data source defined (ServiceNow in my case), so we go into the Customizations area and select Verticals and begin the process of adding one:


In the next step, select the definition of the remote content source you are using (i.e. ServiceNow):


You now have a couple of options for pre-filtering the results returned - using KQL or property matching. For example, you could ensure only KB articles created in the last 12 months are used. I'm choosing not to do any pre-filtering:



You can now review the settings used before finishing up:


Now it's just a question of adding and enabling the vertical you just created:




So that's step one. Now that we have a search vertical defined to show results from the data being integrated, it's time to move on to how the results are presented.

Creating a custom search result type

Microsoft Search needs to understand how to present a result from the system you are integrating. Within the admin portal we need to be in the same areas as above (Search & intelligence > Settings > Customizations. This time select Result types and begin the process of adding one:


In the next step we name the result type. Then area on the left in the image below shows the steps we'll walk through:

Now we assign this result type to our previously-created content source:

In the next step we can define criteria for the result type to apply. This gives the control to use multiple result types for a given data source, for example in my case I could have a different layout depending on the category of the ServiceNow Knowledge Base article being returned: 

At this point we can jump off into the layout designer mentioned earlier:

The designer is an external tool currently hosted at https://searchlayoutdesigner.azurewebsites.net - once opened, the first step in the tool is to select one of the provided base templates to use as a starting point:

Once you've chosen a good base, click the "Get Started" button at the bottom of the page. That will take you into this interface where you can map values in the search result to tokens in the layout template:

At this point things can be confusing initially because it may not be clear what to enter into each of the "Property" boxes to do the mapping for things like the title, URL, description etc. - they're just text boxes after all, no pickers. However, that might be because, like me, you didn't pay enough attention to what was on the previous page in the admin center:

If you scroll down on the page we jumped off from, there's a table of "Available properties" which lists the name of every piece of data which can be used in your search result:

Armed with this, the process involves copy/pasting keys into the slots provided in the layout designer.


These are "property labels" you assigned when setting up the data source itself - we did this in the last article in this step:

For my ServiceNow result, I use mappings such as:
  • title = ShortDescription
  • titleUrl = AccessUrl
  • modifiedBy = SysUpdatedBy
..where the item on the left is the "slot" in the SharePoint search result, and the item on the right is the ServiceNow attribute. Once you've filled all the slots, the 'Submit' button the bottom of the page will become enabled and clicking it copies the JSON template which is generated:

You then paste this into the textbox provided in the admin center page:


From there you can review your previous selections and then click the 'Add result type' button. On doing so you should see a successful result:


Your custom search vertical and result type are both now created, complete with all the formatting rules you chose. 

The result should be something like this:

(In Microsoft 365/SharePoint):



(In Microsoft Search in Bing):


Summary 

Integrating other platforms with Microsoft 365 provides some compelling ways to simplify the user experience. The approach described here with Graph Connectors can provide a unified search experience and single place to go to, but there are other benefits too. As described at the beginning of this article, Microsoft are increasingly looking for ways to drive value from external data - including expanding the content Viva Topics can use for knowledge mining as well as proactively guiding the user to new or modified data and recommended content from the other platform. This is likely to be a technique we'll use heavily in the future.

1 comment:

MidniteCandle said...

Great articles on this topic! Followed your steps and managed to get what I wanted. One question though. The search results for Service Now content does not appear under the All vertical. This has led to some confusion among users. Is there anyway to create verticals that contain both SharePoint AND external data search results?