Tuesday, 30 June 2020

Using Azure DevOps to manage teamwork

It's been interesting observing the rise of Azure DevOps in the last couple of years at Content and Code, and in particular, seeing the usage expand beyond the development team. We now have several teams which are not dev-focused using the platform to manage their work - including architecture and platform/infrastructure teams whose work is oriented towards envisioning, design, configuration, security or testing. In many cases, team leaders have been looking for a task management tool somewhere within the Microsoft stack with deeper capabilities than Planner or To Do, and without the heavy project management emphasis of Project Online. In some cases their work involves files (e.g. PowerShell scripts or JSON snippets) but often does not. The common thing is that a team's work needs to be managed - and once introduced to Azure DevOps Boards, everyone seems to love the feature set offered by Boards and the clarity it brings to the work. Being able to understand priorities, who is working on what and how various tasks relate to each other are perhaps fundamental to teamwork but are done very well in DevOps Boards. Being able to stay up-to-date on specific items without chasing the owner starts to change the team dynamic in significant ways - it's amazing how many interruptions can be avoided and how efficient a team can become. 

Over the next couple of articles, I want to talk about how Azure DevOps can help in many scenarios - not just for advanced development teams. 

I use the slide below to summarise some of the benefits - this was in a presentation aimed at developers, but it's interesting how only a couple of the points are specific to dev work:

By the way, the slide shows a screenshot of the "Work Details" pane in Azure DevOps. This alone is hugely powerful, providing a view of the volume of work assigned to each team members and how this relates to pre-defined limits. The limits come into play by defining the capacity that each person has available in a particular period, meaning that vacations, work on other projects and anything else that takes away from time available to this work can be accounted for - thus keeping forecasts and time management accurate:
In addition to the examples in my company of architecture and platform/infrastructure teams, I think there's even a case for small support teams outside of a full ITSM process to use Azure DevOps over other tools - more on that later. Overall, I see a scale of different levels of Azure DevOps usage where only development teams running agile sprints are likely to move to level 2 or 3, but level 1 can work for so many different teams:

To unpack that a little, here's an overview of what I mean by those different usage levels:


Some of that might not mean too much at the moment, but we'll explore the different areas further in these articles.

A closer look at Azure DevOps Boards

If you're not familiar with the Boards capability, part of the premise is that the team can see a visual board of a task backlog where an item can simply be dragged to another column to update the status, similar to Post It notes on a physical board (Kanban). This is similar to Trello, Planner or some other task management tools of course. Personally, I love the following features:
  • Being able to define your own columns - often these will map to an individual item status such as e.g. New, Approved, In Progress etc. but you can also map multiple states to a column such as "Build and Test"
  • Being able to define swimlanes to go across the board horizontally - to help categorise items regardless of their state
  • Styling rules - some examples I like to implement include:
    • Changing the card color to red for blocked or overdue items
    • Highlighting the tag displayed on the card for certain tags - the image above shows any item tagged with "Data" in a yellow highlight so the team can identify those tasks quickly for example
  • Live updates - meaning that another team members changes are immediately reflected without a page refresh
Here's an example of a board showing some of those features:


Setting up an Azure DevOps project

In the video below, I demonstrate starting from nothing - creating a new DevOps project, and then adding some issues and tasks to assign to team members. I think it gives a good feel for the user experience of working with items and using some of the collaborative features:


Quite a lot of the basic features are shown in this video - defining tasks, tagging, using drag and drop to update an item's status on the board, using collaborative features such as @mentions and "following" to receive update when an item is changed, and more. The elements shown can be summarized into this list, broken down into three areas:


Summary

Getting started with Azure DevOps is quick and easy. Licensing is generally considered a good deal for what you get, with the first 5 users free and then $6 per user per month after that. In this post we explored some of the capabilities to support modern teamwork - and my experience is that they really can work amazingly well for a variety of teams. Whilst Azure DevOps has many benefits for developers, we've certainly seen it work well in other contexts too at Content and Code. 

Coming back to the different levels of usage, this article and video focuses on the first level ("Simple"):   

In the next article, I'll cover some of the agile features which help a team run sprints, including use of tools such as the burndown chart Cumulative Flow Diagram - tools which help a team understand their progress through a series of tasks and overall velocity over time. 

Monday, 18 May 2020

Teams and SharePoint provisioning solutions - new Group.Create permission

It's common for organizations using Office 365 to provide a custom experience for requesting and creating Microsoft Teams and/or SharePoint sites - often giving the ability to select from pre-defined templates, provide metadata and important settings, check for a possible duplicate, implement an approval process and so on. For many, this is key part of governance and provides a more controlled approach than "allow everyone to create and perform some clean-up later". We've been building solution like this for years, and as one example, the workspace request and provisioning capability in our Fresh digital workplace solution has increased in richness and control as Teams has come to prominence and APIs have gotten better.

You want WHAT permission in our tenant??
Until recently, there was a big problem with these solutions. Since both Microsoft Teams and SharePoint team sites are underpinned by Office 365 Groups (though not SharePoint communication sites of course), creating either involves creation of an Office 365 Group underneath. Whether a logged-in user or application or tool is performing this step, a very high permission is needed - Group.ReadWrite.All. This is essentially allowing the app to read and write to any Microsoft Team, SharePoint site or Office 365 Group within your entire tenant - think about that for a moment!

As you might expect, many security-conscious organizations were reluctant to grant this permission to something running in their tenant. Practically every business on the planet using Office 365 to it's full extent has Teams and SharePoint sites which contain sensitive information, and the whole idea of a single application being able to change or delete data across the entire estate is very far away from the principle of least privilege. If an attacker or some malicious code code was to impersonate this identity, the impact to the business could be huge. 

With one global organization, I spent around 8 weeks convincing the Digital Security (Info Sec) group and other stakeholders that the other mitigations our team and other internal teams were taking were enough to offset this risk. No less than 7 distinct sign-offs were required.

So, something had to change. I knew others would be feeling this pain and I'd been speaking to folks at Microsoft like Jeremy Thake about it for some time. I finally got round to raising a request on UserVoice, but also mentioned it on Twitter. I was pleased to see this response from Bill Bliss, Microsoft's Platform Architect for Teams:


So, that gave me hope. Fast forward a few months, and the good news now of course is that a solution landed some time ago - making the lives of anyone building on the Office 365 that bit easier.

The new Group.Create permission in the Microsoft Graph

Solutions which create Microsoft Teams or Group-connected SharePoint sites can now use a new permission scope in the Graph authorization model named Group.Create. This arrived without much noise or fanfare, and I suspect many people who this is relevant to may still not be aware. 

As you might expect, this allows the user or application to create new Teams, Groups and sites, without having access to all existing instances. As the documentation says:

Allows the calling app to create groups without a signed-in user. Does not allow read, update, or deletion of any groups.

 This is hugely important for any organization using Office 365 who cares about their security posture and wants to make use of provisioning solutions. Here's what it looks like in AAD:


Hoorah! So now we have something that fits both requirements:
  • Allows new Teams and sites to be created
  • Minimises the surface area so that other actions cannot be taken

Call to action - is this relevant to your environment?

If you're using any kind of tailored approach to creating Teams and SharePoint sites, whether it's solution developed in-house or by a partner, a 3rd party product or one of starter solutions found on Github, you should ensure the permission set used is Group.Create. If you find there's a change to be made, this should be carefully managed with testing in your non-production tenants before being going ahead with the switch in live of course - but the process shouldn't be too complex.  

I previously mentioned this new option on Twitter a while ago, and was quite a few people responded in some way:


However, I wanted to also post about this here too. The pace of change in Office 365 and underlying platform elements such as the Microsoft Graph is relentless, and it's easy to miss things. And whilst there will always be imperfections, it's good to know that things like this are being smoothed out.

Hopefully this is useful to some folks!


Wednesday, 22 April 2020

Infuse AI into your Office 365 apps – three approaches and pricing: Part 3

This is the 3rd article in my series on AI in Office 365 apps. Obviously the world has turned upside down since the last post a few weeks ago - firstly I hope you and the people you care about are well and you are coping. We're learning more about new world each day, and it's clear some things might not be quite the same again. I think it's fair to say we'll see greater use of the technologies discussed in this series - the Power Platform, Azure and AI. Many organizations now need to transform very rapidly - and that means new forms of I.T., new processes, and in many cases more automation. You've probably seen Power Apps such as the Crisis Communications app and Emergency Response app emerge from Microsoft, and the fact that these apps were produced so quickly highlights how effective these technologies can be. Being able to plug AI into such apps provides a strong foundation for a wide range of scenarios, as you've hopefully seen in the "Incident Reporting" sample app I built for this series.

As a reminder, the articles in this group are:

  1. AI in Office 365 apps - a scenario, some AI examples and a sample Power App 
  2. AI in Office 365 apps - choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate 
  3. AI in Office 365 apps - pricing and conclusions (this article)
As you might expect, pricing is another area where things get interesting as we compare the three approaches. We looked at capabilities last time (and some factors which can guide you to which form of AI is right for a given situation) - but pricing surely has to be part of the overall decision making process, so let's dive into that now.

In writing this article I compiled costs for the three approaches based on the following parameters for the Incident Reporting app used in the previous articles:
  • 200 users
  • Varying transactions (incidents being reported with an image to tag/describe) - 1000, 100,000, 200,000 and 1 million
  • Calculate app running cost per month

I ended up with a spreadsheet that looks like this - but the costs on their own don't mean much without commentary so I've blurred out the numbers for now, but we'll come back to this later. I'm just providing this here so you can understand the process I went through:

 
So let's dive into the different approaches, starting with Power Apps AI Builder.

Power Apps AI Builder - pricing

Use of AI Builder in Power Apps is enabled by purchasing "AI Builder service credits". Prices can be obtained on the Power Apps pricing page, and the idea is that these are purchased in blocks of 1 million service credits. Prices at the time of writing are $500/£377/€421 per block per month. So the immediate question of course is "how many service credits will I consume?". In the case of the scenario discussed in this series, the specific question would be "how many service credits will my use of Object Detection consume?", since that's the particular type of AI Builder functionality which is appropriate.

Microsoft provide the Power Apps AI Builder pricing calculator to help with this kind of cost forecasting. There, you can enter the number of transactions you'd like to price for - so I entered my range of numbers (1000, 100,000, 200,000 and 1 million) and assumed my model would be trained once per month:


This gives me an estimate of how many AI Builder add-on units are required for that many transactions, and the cost in USD:


So, we get a monthly cost of $500 for 1000 transactions - and we can use the calculator to see costs for varying usage levels The caveat is that they are only estimates rather than a guaranteed price, but they should be reasonably accurate.

However, the big thing to also consider for this approach is that we also need Power Apps per user/per app licensing to cover our 200 users. As the Power Platform licensing FAQ states, "AI Builder is licensed as an add-on to standalone Power Apps and Power Automate licensing as well as Dynamics 365 licenses." What that means of course, is that if you're using Power Apps through Office 365 licensing, you cannot use AI Builder - Power Apps licensing itself is required too. As we'll see, this makes a HUGE difference to the pricing - something that won't be a surprise to anyone who has previously looked at Power Platform licensing.

So, pricing for this approach ends up like this - these numbers are PER MONTH for our varying amounts of usage:



So, things feel expensive when using Power Apps AI Builder - but the costs are mainly coming from the core Power Apps licensing requirements for our 200 users, rather than the AI Builder costs.  

Azure Cognitive Services - pricing

So, does sidestepping Power Apps AI Builder and plugging directly into AI services help in any way? Well, as you might expect, the Computer Vision API in Azure (which again, is the particular AI service relevant to the needs of our scenario) is priced by number of API transactions i.e. it is a consumption-based Azure service. There are tiers of 0-1 million, 1-5M, 5M-10M, 10M-100M and so on, with pricing getting cheaper the more you use. Pricing will vary per region, and there's a different pricing rate for each capability.

For our scenario, we would use the following capabilities of the Computer Vision API:
  • Tag
  • Detect
  • Describe
Pricing looks like this:

ItemPricing (current, UK, lowest tiers)1000100k200k1m
Tag0-1M transactions — £0.746 per 1,000 transactions
1M-5M transactions — £0.597 per 1,000 transactions
£0.75£74.60£149.20£597.00
Detect0-1M transactions — £1.118 per 1,000 transactions
1M-5M transactions — £0.746 per 1,000 transactions
£1.12£111.80£223.60£746.00
Describe£1.864 per 1,000 transactions£1.86£186.40£372.80£1864.00
TOTAL£3.73£372.80£745.60£3207.00

So if our app had around 1000 incidents reported per month, we'd be looking at a total of £3.73 per month. 2000 incidents would be £7.46, and so on.
As you can see, this is pretty cheap unless you're running at scale. However, the gotcha is that we would *still* need Power Platform per app/per user licenses to tap into this though - this is because we need to use the HTTP connector to call to Azure, which means that we are in "pay for" Power Platform territory. Assuming we would use the per user licensing, current UK pricing is £7.50 per user per month. 

The impact of Power Apps licensing for HTTP calls

The issue here for our incident reporting app and any similar scenario, is that the Power App can be used by many users. Since our app needs to call into Azure via the HTTP connector, *each* user needs a per app/per user license. Even just 100 users would mean £750 per month, and that's before the Azure costs or costs for any other aspect. So, just when you think you have an extremely cost-effective way of building AI into your Office 365 application, if you're using a Power App as the front-end, you'll find that the Power Platform licensing can get in the way if you envision large numbers of users.

So, pricing for this approach ends up like this - again, PER MONTH and for our different usage levels:


Again, it's the need for Power Apps licensing that makes this expensive, rather than the costs for Azure AI services.

Power Automate (Flow) for AI - pricing

The final option we looked at was using Power Automate to plug into AI. As we detailed last time, this is essentially a wrapper around Azure's Vision API anyway - making the services easily available to a Flow you might create. Is pricing any better here? Well, yes - depending on how your Flow is called. If your Flow is triggered by an item being added to SharePoint, rather than being directly triggered by the user - which is the case in my Incident Reporting app - then only a single user license is required for Flow. Clearly this is very different to requiring each user of the Power App to have a license - the difference is substantial if you have a large number of users, and this is completely valid according to Microsoft's licensing guidelines.

So, pricing for this final approach ends up like this - again, PER MONTH and for our different usage levels:



As you can see, the costs are dramatically different! Removing the need for Power Apps per user/per app licensing for our fictional 200 users is by far the biggest factor, but it also helps that our AI costs are massively cheaper with Azure Cognitive Services compared to Power Apps AI Builder. It seems the conclusion here is that if your solution can be built in certain ways, then you can save substantially on license costs. Some of these aspects aren't specific to the use of AI in our scenario, but are common considerations:
  • Ensuring AI is used via the Azure Computer Vision actions in Power Automate as an alternative approach to Power Apps AI Builder 
  • Ensure your Power Automate Flow is triggered "indirectly" 
    • In other words, the trigger used is either "When an item is created" or "When an item is created or modified" provided by the SharePoint connector. This way the metadata is added to the file in SharePoint after it lands in the document library, and the process isn't triggered by the user directly (e.g. from a button in the Power App)
Of course, these choices only stack up if you or your team are happy to build in this way. If, for example, you don't have skills in Power Automate, this specific approach may not be an option for you.

Summary

There are numerous approaches to AI in Office 365, and they come with a big variance in operational costs. The decisions your implementation team or partner make during the design of the solution will have a significant impact on the TCO of your application! Understanding the techniques to use which can reduce costs but not contravene any Microsoft licensing guidelines is the key to success here - so having the right expertise is vital.

Two principles which can help overall are:
  • Store data in Office 365 where possible - avoiding data sources which require a premium connector (e.g. Azure, SQL, CDS, other SaaS applications) or custom connector means you stay within the bounds of "Power Platform use rights for Office 365" described in the licensing guidelines
  • Use Office 365 triggers for Power Automate - avoiding implementations where the Flow is triggered directly by the user, but instead uses an Office 365 trigger (e.g. SharePoint's "When an item is created", or OneDrive's "When a file is created") will again keep you within the bounds of Office 365 licensing
None of this will be a surprise to people who know Power Platform licensing well, but it's interesting to see a quantified comparison of costs for a detailed scenario. Of course, in many cases storing data in Office 365 will not be the right solution design - perhaps you need true database features such as relationships (one-to-many or many-to-many), cascading integrity, or business rules/triggers related to your data. Maybe you need better support for role-based access. In cases like these, choosing CDS or SQL is likely to provide a better platform and stakeholders need to accept that costs come along with this - but that should be reflected in the value provided by the application.

In any case, these decisions should be made mindfully. Hopefully the information in this article series has been useful!

Tuesday, 3 March 2020

AI in Office 365 apps – choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate: Part 2

In the last article we talked about some examples of using AI in Office 365, and looked in detail at the the idea of building an incident reporting app which combines common Office 365 building blocks with AI. Whilst Power Apps and SharePoint underpin our solution, we use AI to triage the incident by understanding what is happening in the image. Is this a serious emergency? Are there casualties or emergency services involved? Our image processing AI can interpret the picture, and add tags and a description to where the file is stored in Office 365 - this can drive some automated actions to be taken, such as alerting particular teams or having a human review the incident. We also looked at the results of feeding in a series of images to the AI to analyze the results of different scenarios.

Overall, this article is part one of a series:

  1. AI in Office 365 apps - a scenario, some AI examples and a sample Power App 
  2. AI in Office 365 apps - choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate (this article)
  3. AI in Office 365 apps - pricing and conclusions
In this article, we'll focus on the how - in particular, comparing three approaches we could use in Office 365 to build our incident reporting app and make use of AI. Which is easier? What skills are required? How long might we expect it to take for each approach?

Choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate

As mentioned in the last article, there are a number of ways we could build this app:
  • Use of Power Apps AI Builder
  • A Power App which talks directly to Azure Cognitive Services (via a Custom Connector)
  • A Power App which uses a Power Automate Flow to consume AI services
For each approach, we’re looking at how the app would be built, pricing, and any constraints and considerations which come with this option.

Option 1 - Power Apps AI Builder

AI Builder is still in preview at the time of writing (February 2020 - release will be April 2020) with four models offered:

As you might expect, the idea is to make AI relatively easy to use and to focus on common scenarios. No coding is required, and anyone with Power Apps experience can now take advantage of what would previously have been highly-complex application capabilities.
How the app would be built
In our scenario, it’s the “Object Detection” model that is relevant. This can detect specific objects in images that are supplied to it, as well as count the number of times the recognized object is found in the image. The first step is to define a model and supply some sample images:



You'll need an entity pre-defined in CDS to use AI Builder - I'm skipping through a few screens here, but ultimately you select a CDS entity representing the object you are detecting:


In my case, I was building an app related to transport and my entity is "Bus". The next step is to start to train the AI model by providing some images containing the entity:



We then tag the object in each image with the entity:



Once all this is done, you can use the Object Detector control in your Power App and configure it to use this model:



Since the whole topic of how to build apps with AI Builder is interesting, I'll most likely go through this process in more detail in a future article - but hopefully you get a feel for what the process looks like.

In the case of our scenario, we said that we wanted the images to be tagged in SharePoint - and here's where we run into a consideration with AI Builder:

Power Apps AI Builder - great for SOME forms of image detection

The Object Detector capability allows us to detect whether a certain object is in an image or not, and how many times. However, our scenario demanded the capability to recognize *what* was happening in an image, not simply whether a predefined object is present or not! And that's what AI Builder provides - a percentage certainty of whether your single object is present or not. This is much less flexible other forms of AI image processing, and we'd need to somehow supplement this to achieve the goals of our application. After all, we can't provide an AI model with every known object in the universe..

Option 2 - Azure Cognitive Services

Another way of bringing AI into an app is to plug directly into Azure Cognitive Services. As you might expect, this as a developer-centric approach which is more low-level - we're not in the Power Platform or other low-code framework here. The big advantage is that there's a wider array of capabilities to use. Compared to the other approaches discussed here, we're not restricted to whatever Microsoft have integrated into the Power Platform. The high-level areas of Cognitive Services currently extend to:
  • Decision - detect anomalies, do content moderation etc.
  • Language - services such as LUIS, text analytics (e.g. sentiment analysis, extract key phrases and entities), translation between 60+ languages
  • Speech - convert between text and speech (both directions), real-time speech translation from audio, speaker recognition etc.
  • Vision - Computer Vision (e.g. tag and describe images, recognize objects, celebrities, landmarks, brands, perform OCR, generate thumbnails etc.), form data extraction, ink/handwriting processing, video indexing, face recognition and more
    • NOTE - this is the service that's relevant to the scenario in this article, in particular the Computer Vision API's ability to tag and describe images) 
  • Web search - Bing autosuggest, Bin entity/image/news/visual/video search and more 
In terms of what this looks like for our scenario, let's take the following image:


How the app would be built
If I can write some code to consume the Computer Vision API and send the above image to it, I get a response that looks like this (notice the tags such as "person", "indoor", "ceiling", "event", "crowd" and so on:

The code I used to do this is:

Code sample:

The relationship between Azure Cognitive Services and other options

Whilst we're talking about Cognitive Services, it's worth recognising of course that all of the options listed in this e-mail use these services underneath. Power Apps AI Builder, the Power Automate activities discussed here, and many other facilities in Microsoft cloud technologies all use Azure Cognitive Services underneath. When you're thinking about technology options, it's worth considering that the more direct your approach to Azure is, the cheaper it is likely to be.

Option 3 - Using Power Automate (Flow) to consume AI

The final option presented here is to create a Flow which will do the work of tagging and describing the incident report images. This is by far the easiest way I think, and is perhaps an overlooked approach for building AI into your apps - I recommend it highly, Power Automate is your friend. Note, however, that these are premium Flow actions - we'll cover licensing and pricing more in the next post, but for now understand that bringing AI capabilities this way does incur additional cost (as it does with the other two approaches).

In the scenario of our Power App for incident reporting, the simplest implementation is probably this:
  1. Power App uploads image to SharePoint document library
  2. Flow runs using the "SharePoint - when a file is created in a folder" trigger
  3. The Flow calls Azure Cognitive Services (using the native Flow actions for this)
  4. Once the tags and image descriptions have been obtained, they are written back to the file in SharePoint as metadata
The beauty is really in step 3. Since Microsoft provide hooks into many AI services as Flow actions, infusing AI into your app this way is extremely simple - no code is required, and it's as simple as lining up some actions in the Flow. Some skills and experience in Power Automate are certainly required, but the bar is certainly much lower than the other options. 

Here's what my end-to-end Flow looks like:

In more specific terms, the trigger is on a new file being created in the SharePoint site and list where my Power App pushes the image to:

For each file that has been added, I get the file and then call:
  • Describe Image Content 
  • Tag Image
The Flow section below shows how we get the details of the file added to SharePoint to be able to pass the contents to the Azure actions for processing:
On the Power App side of course, I need something to use the camera on the device to take the photo and upload the file to SharePoint - but that's not too complex, and it's just a question of adding the Power Apps camera control to my app to facilitate part of that. Of course, a major capability of Power Apps is being able to plug into device facilities such as camera, GPS and display functions, so it should be no surprise that's simple. If you remember, I showed my sample app briefly in the last post:



However, I do need to do some work to get my image into SharePoint once it has been captured - in my case I use the integration between Power Apps and Power Automate to do this. I create a Flow which uses the Power Apps trigger and ultimately uses the SharePoint "Create File" action. The important part though, is the Compose action in the middle which uses the "DataUriToBinary" function to translate the image data from how Power Apps captures it to how SharePoint needs it:

I then link the Flow to the Power App:

I can then use this in my formulas, as:

UpdateContext( { PictureFilename: "Incident_" & Text( Now(), "[$-en-US]yyyy-mm-dd-hh-mm-ss" ) & ".jpg" } );

IncidentPhotoProcessing.Run(PictureFilename, First(Photos).Url);

..and there we go, a fairly quick and easy way to get the photo for my incident into SharePoint so that the AI can do it's processing.

Summary


We've looked at three possible approaches in this post to building an Office 365 application which uses AI - Power Apps AI Builder, use of Azure Cognitive Services from code and use of actions in Power Automate which relate to AI. The findings can be summarized as:
  • Different skills are needed for each approach:-
    • Power Automate is the simplest to use because it provides actions which plug into AI easily - just build a Flow which can receive the image, and then use the Computer Vision actions shown above
    • Direct use of Azure Cognitive Services APIs requires coding skills (either use provided SDKs for .NET and JavaScript etc. or make your own REST requests to the Azure endpoints), but is a powerful approach since the full set of Microsoft AI capabilities are exposed
  • Capabilities are different across the options:-
    • Power Apps AI Builder has some constraints regarding our image processing scenario. The "object detection" model is great for identifying if a known object is present in the image, but can't help with identifying any arbitrary objects or concepts in the image
    •  Azure Cognitive Services underpins all of the AI capabilities in Office 365, and offers many services not exposed in Power Apps or Power Automate. As a result, it offers the most flexibility and power, at the cost of more effort and different skills to implement
  • Requirements and context are important:-
    • In our scenario we're talking about a Power App which captures incident data and stores it in SharePoint - in other words, we've already specified that the front-end should be Power Apps. In this context, integrating Azure Cognitive Services directly would be a bit more challenging than the other two approaches (but is relatively simple from a coded application). In Power Apps, we'd need a custom connector to bring the code in (probably in the form of an Azure Function), and that's certainly more complex than staying purely in the Power Platform 
    • In the design process, other requirements could lead to one technical approach being much more appropriate than another. As another example, if the app needed a rich user interface that was difficult to build in Power Apps, the front-end may well be custom code. At this point, there's an argument for saying that using code for the content services back-end of the application also makes sense too
As usual, having a team or partner who understands this landscape well and has capability across all options can lead to the best result. In this case, all options can be on the table rather than being limited to one or two because of skills. The architecture decision can be merit-based, considering the use cases and scenarios for the app, the AI capabilities needed, the user experience, the range of devices used, speed to implement and cost.

And cost, of course, is the element that we haven't covered yet! Let's discuss that in the next post:



Friday, 28 February 2020

Infuse AI into your Office 365 apps – three approaches and pricing: Part 1

We’ve all heard how Microsoft are on a mission to democratize AI and bring it to the masses. The last Ignite conference (fall 2019) continued to bang this drum heavily, with several keynote demos featuring AI in interesting scenarios. In fact, Microsoft’s AI democratization journey started back in early 2015 with Project Oxford, a set of APIs which developers could use to recognize faces, perform speech-to-text processing, categorize images and more. Looking back, I remember presenting at Microsoft's Tech Ed 2014 conference showing an extension I created for Word which would use non-Microsoft AI to find similar documents based on the content - so AI has been around for some time. Also making it's debut around this time was LUIS (Language Understanding Intelligent Service), the Microsoft service that a bot developer would typically use to infer intent from some words. All this was great for developers, but true democratization would mean lowering the barrier to entry much further than that.

Fast forward to early 2020, and I think we’re in a different situation altogether. Microsoft’s AI capabilities have moved into well-defined Azure offerings as part of Azure Cognitive Services, and there will have been significant investment in performance, scaling, reliability and accuracy as individual APIs transition into service across the Azure datacenter infrastructure. Along with AWS, Google, IBM and other major cloud providers, Microsoft are out to land-grab as much of the global workload as possible, in order to recoup their infrastructure investments and make their margins work. Competition is fierce, and even if your organization isn't using AI significantly at the moment, it's you they are targeting. 

Alongside this, the Power Platform has emerged as Microsoft’s model for business applications which can be built without professional developer skills. So how easy is it now for an organization using Microsoft cloud tech to use AI, and what profile of person will be able to build such an app? This series of articles looks at several approaches, and also analyzes the pricing to consume AI in each case. Does easier to use AI come with an added cost? Which option is best if the organization *does* have developers or a partner?

This article is part one of a series:

  1. AI in Office 365 apps - a scenario, some AI examples and a sample Power App (this article)
  2. AI in Office 365 apps - choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate
  3. AI in Office 365 apps - pricing and conclusions
As we think about different approaches, a scenario is useful to base our analysis on - so let's think about that first.

The scenario

Some form of incident or situation reporting is a requirement across many sectors, ranging from monitoring hazards on a construction site, health and safety monitoring in a hospital, or even a store manager submitting evidence of merchandising to head office.So let’s say we’re building an incident reporting app which mobile workers will use in the field on their phones - I use this example frequently with clients, as it combines several ingredients for intelligent data capture apps. We’ve decided that the app itself will be a Power App to avoid costly native app development and for easy distribution to mobile devices, and that the photo and details of the image will be stored in SharePoint Online. AI will be used to detect what’s happening in the image – specifically, we’ll add some metadata to the image in the form of a description and keywords. The keywords will describe objects detected in the image and the overall setting. This is useful, because it could be used to automatically categorize the incident and/or alert different teams – using the health and safety example, if the keywords contained “casualty”, “injury” or “blood”, an alert could be raised immediately to a certain team. Other processing of the incident could also be built-in, depending on what other rules or workflows might be appropriate.

There are a number of ways we could build this app:
  • Use of Power Apps AI Builder
  • A Power App which talks directly to Azure Cognitive Services (via a Custom Connector)
  • A Power App which uses a Power Automate Flow to consume AI services
For each approach, we’ll look at how the app would be built, pricing, and any constraints and considerations which come with this option.

AI image processing - looking at examples

Before we go on to look at implementation approaches and costs, exactly how can AI help in an application like this? Whilst the range of AI tools include text-to-speech (and vice-versa), language translation, pattern matching, data extraction, text processing and various data science related tools, in our scenario it's a form of image processing that is most relevant. Potential benefits we can unlock here include:
  • Images (and the associated incidents in our case) become searchable when we have some textual data for them. Unless some interpretation has been performed, any search capability (including Office 365) is unable to determine what the various pixels and colours represent
  • Images/incidents can be categorised once the application knows what they relate to
  • Some automated 'triage' is possible once we've turned the image into information. Using the example described earlier, if the AI does identify concepts such as "casualty" or "injury" our system would take specific action - even if the process was simply to route these incidents for urgent human processing and/or we accepted that some would be false positives, there could be huge benefits across the a busy system
  • AI can easily process large amounts of historic data. So if I already have a repository of existing files where I want to perform some automated image processing (or in the case of other files, pattern identification/translation/text to speech generation etc.), I can do that easily even if I didn't have the capabilities back when the app was first introduced
Overall, there are an almost unlimited array of possibilities here.

Anyway, back to the image processing. Here are some examples - using a couple of images captured with a test Power App I created to perform the above functions and one from the internet. For each one, consider the image and what that the AI detected:

Image

Result

Image

Result



Image

Result


I think there are some pretty amazing results there. Consider some of the objects and concepts being recognized - public transport, subway, conference room, emergency services, police - that's quite a lot of intelligence to be able to successfully detect those contexts, and to have this power available to you within easy reach opens the door to lots of innovative solutions. How could you use it in your organization?

A Power App for incident reporting

So that's what the back-end can do. But on the front-end, our scenario would need a means of capturing the image and reporting the incident. Here's a quick Power App I created to do this - it uses the camera control with Power Apps and allows me to plug in any of the three architectures we're looking at in this series:

In the maker view, it looks like this (featuring me hard at work, because the front-facing camera on my Surface Pro is activated):

Summary

So that lays the groundwork for this series. AI is one of those topics that gets a lot of coverage, but I see lots of organizations struggling to make practical use of it. In these articles my aim is to show some approachable methods which can add real value to a common business scenario, and between the options there are a few variables to consider - capabilities, cost profiles, skillset required to implement to name a few.

Having a good understanding of the options provided to you in the Microsoft stack can help you bring real innovation to your organization or clients. In the next article we'll go through our 3 implementation options in close detail, so you can see what's needed to tap into AI in these ways.

Next article - AI in Office 365 apps - choosing between Power Apps AI Builder, Azure Cognitive Services and Power Automate

Friday, 31 January 2020

SharePoint Conference 2020, Las Vegas - my thoughts (and session details)

I’m excited to be speaking again at this year’s official SharePoint Conference, held by Microsoft in Las Vegas in May. SharePoint is still a huge focal point of Office 365, but like other SharePoint events it’s certainly the case that the whilst “SharePoint” is in the name, the truth is that the content extends to most of Office 365 and many aspects of Azure too. I think a few reasons combine to make this an *extremely* interesting time to be talking about SharePoint in context of other elements of Office 365. For most of us of course, this is against a backdrop of how to provide the best tools to an organisation and what a digital workplace should look in these times, and specific areas for me include:

  • The relationship between Teams and SharePoint, and how the two can be combined to provide amazing experiences
  • Project Cortex, the forthcoming toolset that we can expect to be a step-change to how knowledge is generated, discovered, managed and evolved within an organisation
  • How to exploit the Power Platform whilst staying in control of how data is used, and being able to provide effective support for end-user developed apps
  • How to provide a world-class modern workplace, based on an approach that is more than just technology. What specifically are the practices that work in achieving business change and great adoption of the tools, and allow the business to hit the objectives of the program?
  • The forthcoming “new Yammer”, with the move towards community-oriented groups with a different feature set with interesting mobile capabilities
  • Changes in the development landscape, including greater capabilities in the SPFx platform and the broadening out into other areas of Microsoft 365 development. The ability to think beyond Teams and SharePoint, and to understand what kind of experiences can be provided across Office apps is a huge opportunity for most organisations
  • The need for a considered, appropriate security posture
  • Moving forward with AI, so that it becomes something that is weaved into your applications rather than just something discussed aspirationally. High-end data science tools in Azure are one thing, but what about the easier to achieve possibilities across SharePoint, Power Apps and Power Automate? What are they, and how can they be used without developer skills?
I keep saying that we’re in a “what a time to be alive!” period with Microsoft technologies. At an event like this, being able to hear about key developments from Microsoft execs and program managers, as well as some of the best practitioners in the field (and me!), is a great way to accelerate learning and position yourself to drive your organisation or clients forward.

Speakers and sessions

The conference will host over 200 sessions and 20 workshops, with over 100+ exhibitors on the show floor. Speakers from Microsoft include Jeff Teper, Jared Spataro, Dan Holme, Bill Baer, Mark Kashman, Vesa Juvonen, Naomi Moneypenny, Murali Sitaram, Navjot Virk, Karuana Gatimu and many more. There’s a long list of very talented speakers from the industry too, including Andrew Connell, Susan Hanley, Benjamin Niaulin, Eric Shupps, Erwin van Hunen, Paolo Pialorsi, Sebastien Levert, Vlad Catrinescu, John White and more.

The conversations will be great, and I know the people above are always willing to talk in corridors and around the conference.

My session

I’ll be delivering my popular “Office 365 dev hitlist” session – here’s the blurb:

Top Office 365 Development techniques to master - the hitlist:
Things move fast in Office 365 development, and as Microsoft evolve the platform and APIs, new techniques and approaches become available all the time. As the head of a talented dev team, I regularly update my list of techniques that I believe are essential to have good capability building Office 365 solutions. Between SPFx, the Graph, Teams development, coding in Azure Functions and building solutions in PowerApps and Flow, let's walk through high-value scenarios in mid-2020 that should be in every experienced coder's toolbox, with a demo and code sample for each. This session will help you close any skills gaps, and should be a great conversation with some bright minds in the room.

Use code “OBRIEN” for a $50 discount

As usual with this event, if you sign-up and use my surname as the discount code, you’ll get $50 off the ticket price - and of course, the organizers get to know which speakers attendees are interested in. Since this is Las Vegas we’re talking about here, so I’ll be amazed if you can’t find a good use for that $50 😉

Even simpler than typing my name into the box on the form is to use this link which will do it for you http://obrien.spc.ms. Clicking on the image below will take you there too:

More details on the conference

The conference website is at https://sharepointna.com, and has all you need to know about the event, location, pricing, hotels and more. You can also tap into "SharePointTV", which has some great content streamed most Wednesdays going forward.

Hopefully see you at the event!

Wednesday, 8 January 2020

Improving Power Apps governance and analytics

Some of the work I’ve been doing (alongside some talented colleagues) recently is around improving governance of Power Platform use within an organization – in particular Power Apps, since a lot of the risk tends to be centred there. It’s becoming increasingly common that the Power Platform can be be widely-adopted within a company (or at least, adoption is growing), but a whole new set of problems become apparent in terms of exactly who is doing what with which apps. In some ways, ungoverned use of Power Apps and Power Automate can become a free-for-all in an organization – and this can cause serious operational problems if a critical app created by someone in the business has problems, or that person changes role or leaves the company. In many cases, people 'out there in the business' can create apps which become critical and users expect I.T. to provide support – but I.T. have a blind spot to what is happening in this area and don’t know the first thing about the application.

Common questions are:

  • How do I.T. get on top of this? How do we become aware of which apps exist already, and which are being created?
  • How do we discover whether apps are connecting to Azure, SQL, SharePoint, or perhaps even SAP, Workday, ServiceNow or ungoverned cloud services such as Dropbox?
  • Which accounts are used for connections? Are they appropriate?
  • Are we protected if that person who made the app leaves the organisation or moves to another role?
We’ve done some work around this with some organizations, partly based on the Power Apps COE starter kit. However..

The Power Apps COE starter kit is not a turnkey solution

Whilst the COE starter kit is a great baseline, as the name suggests this isn’t a turnkey solution. For one thing, unfortunately it’s based on CDS which means immediately that every maker of a significant app in your organization may need additional licensing just to use the governance solution! This seemed crazy to us, since many makers within our clients are using Power Apps extensively, but focusing on apps which only talk to Office 365 sources – and so these users would not therefore would not have a ‘per app’ or ‘per user’ plan license.

So, we decided to create a fork of the COE kit which is re-engineered to store data in Office 365. This side-steps this and provides some additional benefits over the baseline, and is the solution we’re using with our clients.

A peek at what the solution provides

The solution provides quite a few things around analytics, doing a lot of data collection in the background (including app launches from the Office 365 audit logs – another area we had to tweak) to provide a Power BI dashboard which provides some very useful insights. Here's just ONE screen from it, but the tabs across the bottom give you an idea of what else is in there:

The governance framework also introduces the idea of ‘compliant’ and ‘non-compliant’ Power Apps to your organization. This consists of a few things, including requests to app makers to provide a mitigation plan for their app. Exactly what should happen if the app becomes unusable or an update breaks something? Since I.T. aren’t necessarily in a good place to provide SLA support, having these things in place can de-risk the situation significantly across the enterprise. Administrators get to see a traffic light rating of each app, as well as having built a good understand of each app including it’s connections and data sources, the environment used, the usage patterns, the maker(s) and users and so on.

We also do a lot more to facilitate a well-functioning Power Apps maker community – including creation of a Power Apps Knowledge Center site with some policy content we wrote, and using the data we collected about makers, creation of a Yammer group or Microsoft team where all the top makers are invited and introduced to each other. From there, they have a place to share experiences, ask for help and so on.

A podcast interview about this

If you want to hear more, I was interviewed recently by Jeremy Thake for the Microsoft 365 Developer Podcast on this. In fact, we only decided that this would be the topic at the last minute, but I think it came out OK! If the embedded version below doesn’t work, the direct link is:

https://www.podbean.com/eu/pb-s8iq2-cab99d