Friday, 6 October 2017

My Ignite wish list – what got delivered and what didn’t?

In the week or two before the recent Ignite conference, I published a wish list of things I was hoping Microsoft would announce. As usual for me, this focuses *mainly* on building on and extending Office 365, and SharePoint in particular. I like to publish these lists occasionally - it helps me shape my thoughts, and keep up with “top of mind” developments in Office 365 which would help the organizations I work with. Now that the event has happened, there were a TON of announcements at Ignite – clearly there is HUGE amount of investment and development happening across Office 365 right now, and it’s an awesome time to be working with Microsoft tech. If ever you wondered whether you backed the right horse for your career (or made the right choice for your organization), I think it’s hard to have those doubts these days. But that said, I notice that several items on my wish list did NOT get dealt with, so I thought it would be good to reflect on those somewhat.

Before we jump into the items, I have two high-level thoughts on Ignite:

  • Hey, I see what you did there!
    • Although there *were* lots of announcements, pay attention to the timeline on each of them. I’ve tried to gather together the most relevant ones to me in my table below – but I note that many are actually quite far away into 2018. Nothing wrong with it, but I feel perhaps there was a Microsoft tactic to provide a “big wave” of announcements to increase the overall effect, even if many are quite far away from launch (or even just “things that might happen”). I guess this fits with today’s “disclosure strategy” of using these 2 or 3 large events through the year to make announcements – and frankly, I much prefer this compared to having less overall visibility of the roadmap. You just need to pay attention to dates and caveats.
  • Office 365 and cloud revenue is powering investment!
    • Remember a couple of years ago people were asking if SharePoint was dead? Feels like a different world now. I think one big factor is that Microsoft are further up the adoption curve of Office 365, and the corresponding revenue and outlook means that big ambitious goals can be set. I can imagine that various product groups are a little larger and have more resources. Another one is that lots of infrastructure/scale/foundational challenges have clearly been dealt with now – leaving an increased focus on providing great tools and a modern development platform to users.

My original wish list

Anyway, back to my list – here’s what I said Microsoft should provide. I didn’t blog this, but put it on LinkedIn strangely - maybe it was a prophesy related to the increased integration between LinkedIn and Office 365 ;)

    image

    What got delivered/announced, and what didn’t?

    I initially just had some checks/crosses against each item, but then I added some notes, and then some timelines, and finally I ended up with the table below. This might only be useful to me personally, but hey, if it’s useful to you too, then great. It could have more comprehensive notes/links, but that’s probably a bigger task than I can deal with right now! So:


    Item

    Announced?

    Timeline

    Notes

    Tagging/metadata for modern pages

    Not really

    H1 2018

    “Categorization” mentioned for 2018

    Associate existing SP site with
    Group/Team

    Yes

    Early 2018

    PowerShell will also be possible. See “BRK2434 – No team site left behind”

    Provisioning/templating for
    Communication Sites (ideally PnP)

    Yes

    Q4 2017

    Site Designs (previously known as “Recipes”).*See special note on Site Designs below

    Page layouts for Communication
    Sites

    Yes

    Q1 2018



    Profile page extensibility

    No

    ?

    ?

    Web hooks on Group creation
    (site)

    No

    ?

    Nothing imminent, but a *custom*
    site design can call out to a Flow

    Provisioning/templating for Teams
    (tabs, connectors, bots)

    No

    ?

    ?

    More modern web parts (e.g.
    search)

    Yes

    Q4 2017
    • Planner/Forms web parts
    • Enhancements to Highlighted Content web part (provide query)
    • Enhanced web part picker

    SPFx enhancements

    Yes

    Multiple
    • Site scripts/site designs (Q4 2017)
    • Simplification of calling Graph/custom web APIs secured with AAD
    • Site Collection App Catalog etc. (Q4 2017)

    Some ‘under the radar’ things!

    Yes

    Multiple
    • Multi-geo (OneDrive/EXO now, SPO in 2018)
    • Hub Sites (early 2018)
    • OneDrive Files on Demand (Q4 2017)
    • Conditional Access per site (early 2018)
    • Simpler sharing e.g. verification code links for external users (Q4 2017)
    • List enhancements e.g. formatting, attention view, indexing and Flow improvements etc. (Q4 2017)
    • Better site/content analytics (early 2018)
    • LinkedIn integration with Office 365 people card (Q4 2017)

    Previously-announced,
    but not yet delivered

    Yes

    Multiple
    • New Admin Center (First Release Tenants in early CY2018)
    • App Launcher updates
    • PowerApps for SharePoint lists (October 2017 for FR tenants)
    • PowerApps enhancements e.g. upload attachments, simpler conditional views etc. (before end 2017)

    NOTE:– I’d love to hear from anyone who spots an error or something I missed with these details (e.g. a timeline). Please leave a comment if you don’t mind and I’ll update!

    My thoughts – especially on the missing items

    So Microsoft didn’t deliver everything I was hoping for. BUT, they did deliver a whole bunch of things that weren’t in my list! I list most of these in the “under the radar” and and “previously announced, but not yet delivered” categories. Us MVPs are fortunate enough to have something of an inside track on most of them (thanks Microsoft, your work here is MUCH appreciated by the way) so I had familiarity lots of them before the event. It’s great that those categories have so many items, and no doubt many more could be added too. BUT:

    •  Tagging/metadata for modern pages
      • I was disappointed not to hear more on this. It feels like there are still significant gaps when it comes to building solutions around modern pages, especially when it comes to roll-ups and displaying content of different types (e.g. pages tagged with X). Yes, there’s a new PnP re-usable control, but you’d have to do some work to integrate this, and that feels like Microsoft’s job to be honest.
    • Profile page extensibility
      • Still nothing. What’s up with that? This has been talked about for a LONG time, but still we can’t add custom widgets or tailor the Office 365/Delve profile page. I didn’t even hear any mention of it to be honest, yet lots of organizations I work with want to do something there.
    • Web hooks on Group creation (site)
      • Again, nothing here either. Vesa has mentioned some engineering considerations around providing this, but I was hoping for an announcement so that applying a custom PnP templates to self-service created Office 365 Group sites would become easier. Sure, a Flow can be used with the forthcoming Site Designs capability, but that seems to still leave a gap for default sites which don’t use any kind of Site Design, like an out-of-the-box Group site. The info was only released on this yesterday, so I could be missing something though – hope so..
    • Provisioning/templating for Teams (tabs, connectors, bots)
      • Lots of progress on Microsoft Teams, but still no proper templating story. I was *really* surprised not to hear something on this - I definitely have clients who want to use Teams at scale, but with the same set of custom tabs etc. Configuring by hand just isn’t an option, so let’s hope for something soon (again, unless I missed it).

    Of course, let me know if I am missing anything, or you disagree with my interpretation of things.

    * A note on Site Designs

    So, we do have a way of applying templating to Communication Sites (and it goes beyond that – Site Designs can be also applied to Team Sites created from the out-of-the-box UI, so this is a big deal). But, right now I do have some reservations on the model here. Obviously there’s ANOTHER way to define a site template now (the JSON format used by Site Designs, which doesn’t look too easy to work with actually – seems like there are quite a few attributes/actions to understand), and that feels sub-optimal given that the PnP provisioning building blocks have evolved so much. We *can* integrate PnP provisioning (woohoo), but the architecture is Site Design > Flow > Azure queue item > QueueTrigger/Azure Function > my PnP provisioning code. Which is fine and ticks many boxes, but:

    • Frankly I would have preferred some form of Microsoft-hosting my PnP provisioning template and taking care of the execution. The fact that we still need Azure can be a blocker for some organizations, and means there is still a level of complexity, and a chunk of work for us to do. It was pointed out to me that Microsoft may have chosen this approach because they don’t want to host/support PnP provisioning themselves (as a community effort rather than pure Microsoft), and I guess I can understand that actually. But still..
    • Templating Office 365 Group sites still seems to be a gap. Yes, I can call out to my Flow for a site which *does* have a custom Site Design – but what about Group sites which are springing up as users create Groups/Teams/Planner plans/Power BI workspaces etc.? There’s a way in which a default Site Design can be specified at the tenant level, but will this apply to Group Sites? I’m not clear at this stage. But I certainly want the capability of applying a custom template to a Group-connected site in a timely way.
      • UPDATE – just minutes after publishing, I picked up on the detail in the video linked below, that it IS possible to target a Site Design at an out-of-the-box Group Site. This is done by associating your Site Design to WebTemplate=”64” for a Group site, or WebTemplate=”68” for a Communication site etc. Happy days.

    I still think it’s something of a shame to have another templating language/approach though. Which bits of the template will be done in the Site Design and which in a PnP template? I can imagine lots of different approaches being used for this. But of course, the main advantage is that this form of templating integrates with the out-of-the-box UI for creating SharePoint sites, which opens up a lot of possibilities. I just wonder if in the end, we’ll be implementing “shell” Site Designs which call out to a PnP template which does the real work of creating content types/lists/pages/web parts and so on in the site. Let’s see..

    For more on Site Designs, see https://techcommunity.microsoft.com/t5/SharePoint-Developer/SharePoint-Patterns-amp-Practices-PnP-Core-and-PnP-PowerShell/m-p/114082#M3496

      Conclusion

      Despite not hitting all items on my list (which are just the views of one guy of course – everyone else has their priorities too), I think Microsoft are actually exceeding what I hoped for. Yes, there will always be gaps and as you can imagine, there are LOTS of things I’m not covering here. Whether it’s general developments such as new Office 365 Plans or Bing for Business, the raft of enhancements to Microsoft Teams (e.g. taking the place of Skype for Business for online), or developer-focused things such as Graph enhancements (e.g. Extensions in Azure Functions, SP list data in the Graph etc.), there’s lots of other things to keep up with. Some good starting points for further reading are:

      Wednesday, 20 September 2017

      Use Azure App Insights to track events in your app/web part/provisioning code

      In my recent post Add Azure App Insights or Google Analytics to your SharePoint pages with an SPFx Application Customizer we focused on the page tracking/analytics capability of App Insights. But what I really think is cool is the ability to track what is happening your code – whether it’s simply that your web part has executed or your app has been launched (so you can get counts), or that you showed an error message to a user (with the details), or maybe to understand where users are navigating in your app or which options they select. As an example, we have a tabbed interface in one particular app, and I’d really love to know how many users are actually navigating to the 2nd and 3rd tabs (my bet is very few). Well, App Insights event tracking is great for these scenarios and more. I finished the last post with this list:

      • Server side applications (Office 365/SharePoint Add-in)
        • How many app launches are happening? By who/where?
        • What is happening within the app (e.g. which buttons are being clicked/what functionality is being used)?
      • SPFx/Graph
        • How many executions is our web part getting per day?
        • How long do my web parts take to execute on the client side?
        • How long are my async calls taking (e.g. to the Graph)? How different are they for users around the world?
      • Provisioning code
        • How many sites are we creating per day/week/month?
        • How many times do we hit an issue during provisioning?
        • How long does provisioning take?
      • General
        • How many times are we showing an error message to a user?

      You get the idea.

      A quick getting started recap (for JavaScript/SPFx)

      Remember that to use App Insights you need an Azure subscription and an App Insights Resource to be created – this will give you the instrumentation key which you obtain from the Azure portal (see my last post for more details). If you’re working with JavaScript/TypeScript/SPFx, there’s an npm package you can install with:

      npm i applicationinsights-js --save

      From there, you'll need the couple of lines which get you started. The import statement at the top of your module:

      import {AppInsights} from "applicationinsights-js";

      ..and then the initial bootstrap code which references your App Insights key:

      let appInsightsKey: string = "[YOUR KEY FROM THE AZURE PORTAL HERE]";
      
      AppInsights.downloadAndSetup({ instrumentationKey: appInsightsKey });

      Now you're ready to log various things which happen in your code to App Insights. In essence, we drop a line in when we want to log that something has happened - let's look at some examples.

      Example 1 – logging a call to the Microsoft Graph (in SPFx code)

      The code below fetches calendar events for the current user in SPFx code, and is taken from one of the SPFx React samples. This is a fairly typical use of the Microsoft Graph, and the logging approach used here could be used across lots of similar scenarios. The fact that it is in a React component is not important, but the bits to focus on are the calls to App Insights - note that what I’m also doing is recording the *time* taken to make the call, so as well as finding many times my web part is being used, I also start to understand how long calls to the Graph are taking for users around the world. To do this, I simply create a timestamp before the call, and then another in the promise which executes once the data has been fetched and subtract the difference:

      In App Insights, I can then see an event for each execution of my web part along with the time taken to call the Graph:

      SNAGHTML1f5d5c2c

      SNAGHTML203e67e

      As I showed in my last App Insights post, as you accumulate data you can use the ‘Analytics’ section in App Insights to filter and sort as you need (e.g. on the 'timeTaken’ value):

      SNAGHTML1f6b05fa

      Example 2 – logging a call in site provisioning/templating code

      So that’s an example of logging in SPFx. As a different flavor, let’s say you are creating some SharePoint sites based on a template – perhaps as part of a self-service site creation tool. You might be interested in things like:

      • How many sites are being created? With what details?  
      • How long is Office 365 taking to create the base site collection?
      • How long does it take for your template to be applied?

      In this case, you’ll probably be using the PnP Core library in C# code – so you’ll need the App Insights nuget package from https://www.nuget.org/packages/Microsoft.ApplicationInsights/

      To get started in C# code, you’ll need some bootstrap code:

      // at top of class..
      using Microsoft.ApplicationInsights;
      
      // to initialize..
      TelemetryClient telemetry = new TelemetryClient(); 
      telemetry.InstrumentationKey = APP_INSIGHTS_KEY;

      As the first scenario, let’s say you want to log the fact that a site collection was created and how long that step took. If you’re using PnP provisioning code, it might look something like the below – the key things are:

      • Simple use of the .NET stopwatch clock for timings
      • Creation of ‘metrics’ and ‘properties’ dictionaries to pass details to App Insights
      • Use of the TrackEvent() method to actually log the data

      If using PnP provisioning code, you might end up with something like: 

      This would then show up in App Insights with details of the SharePoint URL, and the time taken for site collection creation (shown here in the Analytics tool):

      SNAGHTML16524535

      We can extend this to PnP templating too. In a similar way, I would add logging statements around PnP 'ApplyProvisioningTemplate' call, and perhaps log any errors too.

      Code sample:

      As expected, you can then get the details of your template being applied with whatever details you logged (site URL, template ID and processing time in my case):

      SNAGHTML1f82d233

      Summary

      App Insights is awesome for integrating into all sorts of Office 365 and SharePoint dev things. By simply dropping a couple of statements in your code, you get to report on it later and get way better visibility of what is happening than you might otherwise have. Whether it’s executions of your web part, errors which are happening, or how often users are going down particular paths in your app, it’s a great way to build this logging in with low effort. In terms of practicalities, you do need an Azure subscription of course and App Insights is chargeable if you pass more than 1GB of data per month (at the time of writing), and your data is only retained for 90 days. BUT, you don’t need to build any kind of front-end or query tool, you can get graphs/charts, weekly summary e-mails, and importantly, alerts if any conditions you define are not met. Additionally, there are lots of ways to integrate with the data (including BLOB download to keep the data for longer). All considered, the features are pretty awesome.

      I feel App Insights is definitely under-utilized by Office 365 and SharePoint developers, but there are lots of possibilities. I’m looking forward to building it in to more of our solutions!

      Monday, 11 September 2017

      Manage tenant-scoped SPFx extensions across your SharePoint sites

      As I mentioned in Use an SPFx Application Customizer to add JavaScript (e.g. header) to every page in a site, it’s now possible to *globally* deploy SPFx extensions (e.g. page headers, footers or other random pieces of JavaScript) or do a controlled roll-out across *many* SharePoint sites - without the app needing to be installed to each individual site. This is great news, and it was a gap for modern pages until now. SPFx web parts deployed at tenant scope will appear everywhere in the picker, but for SPFx extensions there is still something you need to do locally, and that’s “associate” your extension with the site/web/list/field. For Application Customizers, it’s this step which allows you to control exactly which sites use your extension. To do this you add a CustomAction to your site or web, specifying the GUID of your extension in the ClientSideComponentId property (new for SPFx). Although I’m focusing more on site-level customizations (Application Customizers) in this post, it’s a similar story for SPFx Field Customizers too (ClientSideComponentId is specified on the field) and SPFx Command Set Customizers (CustomAction with ClientSideComponentId is specified on the list). All this can be done a couple of ways:

      • Using CSOM or REST – perhaps in PowerShell or C# code
      • As part of PnP XML, if you are applying a custom template to the site – the XML schema and PnP Core provisioning library now supports this (Sept 2017 release onwards)

      In this post I’ll provide some PowerShell and C# code to help you apply Application Customizers across your sites – you could modify for other types of customizer without too much trouble. However, there are some prerequisites in all cases - in some ways, the association step is one of the last things you will do. So let's cover that quickly:

      Tenant-scoped SPFx extensions - recap/prerequisites

      SPFx extensions and webparts are possible from v1.2 of SPFx onwards. Broadly, the prerequisites needed before the script/code in this article can be used are:-

      • You specified "skipFeatureDeployment": true in the package-solution.json file
      • The app was packaged and then installed to the App Catalog, and the administrator checked the box for 'Make this solution available to all sites in the organization' (shown in image below)
      • The JavaScript bundle for the SPFx app has been deployed to a CDN or other web-hosting location

      Here's what the administrator will see when installing to the App Catalog, and the checkbox (which they need to check) that appears when "skipFeatureDeployment": true:

      SNAGHTMLf7281c2

      The PnP XML option

      I'll focus on the C#/PowerShell options in this post, but there’s a PnP XML option which is very useful too. This allows you to include the association as part of a custom site template, and therefore is great for any sites being newly-created from such a template. In fact, you could also use it to apply a ‘partial’ template to existing sites, but I think most people might choose PowerShell or C# code at that point. The main info I want to convey here is that this is available from version 2.18.1709 onwards of PnP Core (Sept 2017 release). The 2017-05 schema has places to specify the ClientSideComponentId property on a CustomAction and on a field, since that’s what the association consists of. The XML extract to provision an Application Customizer at the web level would be:

      In next sections, I’ll cover PowerShell first and then C#.

      Using PowerShell code and PnP PowerShell

      Here are some PowerShell functions to add, remove and list the SPFx global extensions across a selection of sites, done by adding a Custom Action at the root web level – tweak if you need something else. I'm using a simple array of site URLs here, but you could fill that array however you like. Other notes:-

      • I'm using PnP PowerShell cmdlets here - you'll need to install those if you don't have them already, and then get connected to your tenant with Connect-PnPOnline etc.
      • At the time of writing, I had an issue with the PnP cmdlet that they provide (Add-PnPCustomAction), so I'm using direct CSOM in the 'add' method. I raised a GitHub issue about this (also noted in comments in script below), and I'm sure the guys will fix it soon (or tell me I'm doing something wrong ;), but the direct CSOM approach works fine too)
      Output:

      Registering a globally-deployed extension with addSpfxExtensionCustomAction(ctx) will give:

      SNAGHTMLf6529dd

      Listing the extensions on a site with:

      Get-PnPCustomAction -Web $ctx.Web | Where-Object {$_.Location -eq "ClientSideExtension.ApplicationCustomizer" }

      will give:

      SNAGHTMLf67a240

      Removing an extension with

      Remove-CustomActionForSPFxExt $spfxExtName $site $ctx

      will give:

      SNAGHTMLf69e124

      Using C# code and PnP core

      But instead of PowerShell, perhaps you want to use C# code instead. Some notes on this:-

      • I'm using the PnP Core library here - you'll need to install the NuGet package to your solution/Azure Function/whatever if you don't have it already. Get this from https://www.nuget.org/packages/SharePointPnPCoreOnline
      • In contrast to the PowerShell above I'm only processing a single site here, but it would be trivial to extend the code to run across whatever sites you need

      Sample code:

      Output:

      As you’d expect, registering a globally-deployed extension with addSpfxExtensionCustomAction(ctx) will give:

      SNAGHTMLf59848b

      Listing the extensions on a site with getCustomActions(ctx) will give:

      SNAGHTMLf55b850

      Removing an extension with removeSpfxExtensionCustomAction() will give:

      SNAGHTMLf5790d9

      Another option – CLI scripts

      As another option, note that my esteemed colleague Vardhaman Deshpande also has a super-cool CLI tool to help you manage SPFx extensions. He’s so hipster ;) His scripts offers the ability to manage SPFx Command Set Customizers too. See https://github.com/vman/spfx-extensions-cli for more details.

      Summary

      For tenant-scoped SPFx Application Customizers, you need to ensure the sites or webs which should use it have a CustomAction with the ClientSideComponentId of your extension (in addition to dealing with the other prerequisite steps i.e. getting the app package and corresponding JavaScript bundle deployed). Although not addressed with this code, it’s a similar approach for SPFx Field Customizers and Command Set Customizers too. Hopefully the options presented in this article (together with the underlying PnP awesomeness) are of some use.

      Thursday, 17 August 2017

      Add Azure App Insights or Google Analytics to your SharePoint pages with an SPFx Application Customizer

      Azure App Insights is fantastic to help understand how your site or app is being used, and I’ll spend more time on this in future articles. For now, let’s start with a simple case – you want to know how many users are hitting your SharePoint site, which pages are the most popular, which countries the users are based in and so on. We can add page tracking to all pages in a modern SharePoint site (where the master page cannot be customized to add the script) by using an SPFx Application Customizer – previously, script on modern pages was a problem but it is is easily possible now. Of course, the same approach can be used to add Google Analytics or other similar tracking scripts. However, whatever script you’re adding you should consider that:

      • If your site has classic SharePoint pages too, an SPFx App Customizer will not cover these – you’ll need a separate approach for that, such as using a Custom Action to add the script there. (N.B. this is mainly a concern only for publishing sites or older team sites – Office 365 Group sites, modern team sites and communication sites only have modern pages.)
      • Any script added with a SPFx App Customizer will appear on system pages too, such as list/library pages or the Site Contents page. That can be useful so you can see which libraries are getting visited the most, but you might decide only true pages are useful, and add a couple of lines of code to filter out these pages – up to you.

      Regardless of which page tracking system you use, to get this added to modern SharePoint pages the starting point is always to create an SPFx Application Customizer. If you need a getting started guide, I wrote about these in Use an SPFx Application Customizer to add JavaScript (e.g. header) to every page in a site – in this post I’ll focus on the specific code snippets to add Google Analytics or Azure App Insights page tracking, and then talk a bit more about App Insights.

      Integrating Google Analytics

      OK, let’s get this one out of the way first. For GA, you just need to add the script reference to the page – and since Google give you a block of JavaScript to add, the classic way of dynamically adding some script works just fine (as opposed to using the SPFx module loader which can be used with external scripts).

      It's OK to do this in the onInit() method of your customizer because Google's code specifies that the script should be loaded asynchronously - so the browser won't delay loading the page just for the analytics. Not that choosing the Render() method of an SPFX App Customizer instead would make a huge difference anyway, but in any case, onInit() works just fine for this.

      Integrating App Insights

      App Insights comes with an SDK with various objects and methods (both client-side and server-side SDKs are available) - arguably this makes it even easier to add than Google Analytics. For a client-side web application such as SPFx, there’s an npm package (applicationinsights-js) so once your SPFx Application Customizer is created you can get started with:

      npm i applicationinsights-js --save

      At the top of your code, you’ll then need to import the App Insights module to use it:

      import {AppInsights} from "applicationinsights-js";

      Just like with Google Analytics, one of the other first steps is to ensure you have the service set-up. In this case, you need an Azure subscription and an App Insights resource - this will provide a tracking ID to use in your code:

      SNAGHTML1e867b14

      An App Insights resource will be automatically created for any Azure web apps you have in your subscription (e.g. you might have some Office 365 apps or provider-hosted SharePoint add-ins there), but if none of those are appropriate you should create a new instance. Once you have the instrumentation key, the code you need is this:

      The code shown above won’t include the App Insights JS into your bundle – the AppInsights.downloadAndSetup()method just references Microsoft’s JS on their CDN. The core method to track a page view is simply:

      AppInsights.trackPageView();

      Your choices now with an Application Customizer are to test it in debug mode by adding parameters to the URL of a modern page, or package it for real deployment as an app package and add to your side (my previous article details both of these approaches). Once your code is executing, you’ll see results over in the Azure portal for any users hitting pages which run that line of code, and you can now track page load performance and other things. Just like other analytics software, various details about the user (location, browser, device etc.) will be recorded:

      SNAGHTML19ce5502

      SNAGHTML19d0667d

      However, things get more interesting when we help out by changing our code to pass better data as the request is tracked. The other overload which can be used is:

      So, you might choose to log some useful info about the page request:

      • User login
      • User display name
      • If user is external

      In particular we can use the ‘dimensions’ parameter to pass this kind of data. In SPFx, that code would look like:

      Warning!

      Be careful storing user Personally Identifiable Information (PII) of course! I use it purely as an example of what can be done - but in real-life, you might need to store some kind of user token which doesn't actually allow identification of an individual.

      But what about other forms of user information? By also fetching some data from the user's Office 365/SharePoint user profile, you could answer things like:

      • Which division/department are most of my users from?
      • Which region are most of my users from?
      • How many directors/managers are accessing the site/app?

      And lots of other possibilities too no doubt. Unfortunately you lose the page duration timings with the more detailed overload, unless you do some custom work to put it back – specifically, writing some code to record timing of browser events, and then using the final parameter in the overload above. If you do this, be sure to use the browser events (e.g. the load event) rather than a simple stopwatch, as your code executing in the SPFx App Customizer is only *part* of the full page load of course.

      But now I have my page views tracked, and I can do some nice analysis in the Azure portal. I can create some charts in Metrics Explorer (and some charts and tables *are* generated automatically), but the Analytics tool is even more powerful. This definitely isn’t an an end-user/site owner tool, however, as a technical person I can do some querying using an easy syntax with Intellisense-style auto-complete:

      SNAGHTML19e93f15

      (N.B. Those results include page load durations because they used the simple overload).

      When you include custom information as part of your call to trackPageView(), by default this comes out in a “customDimensions” column as JSON:

      SNAGHTML1aa6f52f

      ..but that’s not much good if you want to sort/filter on it. You can use the “extend” keyword in an App Insights query to get around this – so to unfold the 3 custom bits of data we logged earlier, we can use:

      pageViews |
      where url !contains "workbench" | 
      extend userLogin = tostring(customDimensions.userLogin), userDisplayName = tostring(customDimensions.userDisplayName), isExternal = tostring(customDimensions.isExternalGuest) 

      ..and now we get that data in individual columns:

      SNAGHTML1ab5c82b

      You can now play with that data as much as you like.

      So overall, adding page tracking is relatively simple and you can work out the best way to analyze your data as you go. The important thing is to get it captured in the first place, either using the default call to trackPageView() or the more advanced one with custom data.

      App Insights data retention

        At this point it’s probably worth remembering that App Insights only retains data for 90 days, so that’s the longest period you can analyze in a single query. However, if you need longer you can set up Continuous Export to copy your data out (to Azure BLOB storage) and store it forever. Working with it becomes slightly different, but there are details on that page.

        Going beyond page tracking

        I mention page tracking here because it’s a relatively simple case and combines well with an SPFx Application Customizer. However, I think other use cases for App Insights are perhaps more interesting. What I *really* think is valuable is the idea of dropping AppInsights.TrackEvent() statements through your code, whether it’s an SPFx web part, a custom web API, an Office 365 app, or a provider-hosted SharePoint Add-in (e.g. an MVC site). I’ll talk about this in a future post, but I like the idea of scenarios such as:

        • Server side applications (Office 365/SharePoint Add-in)
          • How many app launches are happening? By who/where?
          • What is happening within the app (e.g. which buttons are being clicked/what functionality is being used)?
        • SPFx/Graph
          • How many executions is our web part getting per day?
          • How long do my web parts take to execute on the client side?
          • How long are my async calls taking (e.g. to the Graph)? How different are they for users around the world?
        • Provisioning code
          • How many sites are we creating per day/week/month?
          • How many times do we hit an issue during provisioning?
          • How long does provisioning take?
        • General
          • How many times are we showing an error message to a user?

        Of course, there are a million things that could be nice to track once you start to think about it. I’ll talk more about tracking events in the next article.

        Thursday, 29 June 2017

        Office 365 developer wish list (SPFx and modern sites), summer 2017

        Clearly we’re in a transitional period in Office 365 at the time of writing (June 2017), where modern SharePoint experiences are available - modern sites and pages for example - but not everything is fully joined-up yet. That said, it’s a fast-moving landscape and part of the consultant’s role is to keep up with the best way to deliver solutions. For once, I hope this is a blog post which dates very quickly – certainly I have had to add bits of info as I’ve been writing it, and I’ll come back again and update the table below as things get announced/released.

        Going back to that ‘transitional period’ - this is especially the case for organizations with collaboration workloads, where there’s a need to create some kind of site template for team sites. This is still very common for our clients, even if it’s just a need to provide a different home page experience or add some lists/libraries/content types/global admins to the site. After all, I think that *whatever* Microsoft provide as the default experience, many orgs benefit from some lightweight changes to this – and so site templating continues to be important in SharePoint. I hope Microsoft don’t lose sight of this. Certainly when I consider my wish list, many items relate to “doing team sites at scale” in SharePoint – so perhaps let’s think about that first.

        Current site templating challenges

        Currently site templating is challenging because:

        • Want to use Group sites? Well, it’s challenging to template these currently, because:
          • Can’t currently specify a custom template
          • Can’t currently be notified that a new site has been created, because there are no web hooks
            • [By the way, I agree there are  ways around this (e.g. web job/function which polls for new sites), but none are pretty because users may start using the site in one state, only for it to change as they are working in it..]
        • Want to use non-Group SharePoint sites? Currently challenging to template these *and get modern experiences*, because:
          • Even the Patterns and Practices (PnP) site templating doesn’t currently allow provisioning of modern pages – at least, not without some dev effort to extend it

        For our clients who don’t want to use Groups, that 2nd approach is becoming common for us. But it would be nice if it was more baked-in/required less work. It’s coming I know (see info in my table below), but there’s no harm in nudging people along the way ;)

        My current wish list

        Here’s an extract from a PowerPoint slide I recently to discuss my current list of “asks” to Microsoft:

        SPFx and modern pages wishlist - June 2017

        Let me expand on those in a bit more detail:

        Thing

        Status (June 2017)

        Notes

        Global deployment of SPFx web parts

        • Needed so that SPFx app does not need to be installed to each site
        • Ideally at different scopes e.g:
          • All sites of a certain type e.g. all group sites, all regular team sites etc.
          • Sites with X in property bag
          • All site collections except [list]
          • etc.

        Partial solution “imminent” (weeks not months)

        Sept 5th 2017 – now delivered, see https://dev.office.com/sharepoint/docs/spfx/tenant-scoped-deployment

        See slide at the end of this article. Initial solution will allow web parts to be available across sites, but without much control (e.g. my “scopes” examples).

        Expand PnP schema to include provisioning modern pages and web parts

        • Needed to allow provisioning of team sites *with modern page as home page* – without code/PnP extensibility provider)

        Expected in next release of PnP Core (August 2017 release). It’s in the XML schema already...

        More web hooks

        No news

        I’d like to see web hooks for:

        • Site creation e.g. group sites
        • Subsite creation
        • List creation
        • Permission changes
        • Other changes

        Remove restrictions on “no script” sites (esp. property bag)

        • Needed to bring more extensibility to Group sites (or other sites with “no script” enabled
        • Writing to property bag is common in many customization scenarios

        Done!

        It’s now possible to disable “no script” on modern team sites, including Group sites. The Customizing modern team sites page has been updated to reflect this (June 26 2017)

        Missing controls and tagging support

        • Taxonomy control
        • Person/Group control
        • Calendar control (for non-Group sites)
        • Full Content Search web part (or similar)

        No specific news

        Modern pages in team sites are great for editing and display, but currently not so good when it comes to some types of content and tagging. Missing controls include the ones I’ve listed (difficult to have a “Page contact” or “Site owner” for example), but the tagging/metadata support is lacking too due to the (lack of) fields on the Site Page content type. Rolling up these pages with a filter is tricky in the current arrangement.

        Pages

        • Multi-column support
        • Fix banner/header height

        Coming “soon”

        Sept 5th 2017 – now delivered, see the “Section layouts” section of this post around Communication Sites (but note same page model applies to modern pages in team sites too) - https://techcommunity.microsoft.com/t5/SharePoint-Blog/Reach-your-audience-via-SharePoint-communication-sites-in-Office/ba-p/70079

        More known issues with flexibility of these pages – the single column aspect in particular. But, this is relatively low-hanging fruit for Microsoft and we can expect these resolved soon I think.

        Expand SPFx extensions model (bonus item)

        No specific news

        Additional areas to target, not just “PageHeader” and “PageFooter”. The product group have said more areas are coming for modern pages in team sites (and presumably communication sites too).

          In terms of the first item, global deployment of SPFx web parts, Vesa recently used this slide to discuss what’s coming:

            SNAGHTML46ce695

            That sounds like a reasonable plan, since at least the long-term solution would give us full control. The short-term solution would clearly mean that all SPFx web parts are available everywhere (e.g. in my team sites, my intranet and my communication sites) which might not make sense, so hopefully the long-term arrangement isn’t too far away.

            Communication sites

            Communication sites are starting to launch and Microsoft’s “Ask Me Anything” session was yesterday (28 June 2017). I was disappointed not to hear any detail about templating of communication sites, so that would certainly be another wish list item. I know there are “new” templating options for those coming, but I still want the ability to use PnP site provisioning and XML too – after all, we still need the control and ability to specify all the aspects of a site template that PnP provides, so why use another approach? Also, having a mismatch between what I’m doing everywhere else and for comms sites would be sub-optimal too. Hopefully PnP provisioning will be possible there too – I really hope so!

            What else?

            I could certainly think of a few more items. But what did I miss that’s on your list at the moment?

            UPDATE 26 JULY 2017 – Wait, how did I miss the lack of API for Microsoft Teams? Again, I know it’s coming, but I’m really looking forward to the ability to create a template for a team with appropriate tabs and connectors, and the ability to create Teams programmatically (perhaps alongside an Office 365 Group/SharePoint site). Hopefully not too long to wait!

              Sunday, 11 June 2017

              Use an SPFx Application Customizer to add JavaScript (e.g. header) to every page in a site

              [Updated September 2017 for SPFx 1.2 RC0]

              New tools for customizing modern SharePoint sites and pages in Office 365 have arrived (in preview at the time of writing, June 2017).  These are known as “SharePoint Framework (SPFx) extensions”, and replace some tools that SharePoint developers have long used to deliver key scenarios such as:

              • Adding JavaScript to every page in a site/web
              • Injecting some content (e.g. a mega-menu/global navigation or message bar) into every page
              • Popping up dialog boxes in an integrated way
              • Adding items into certain toolbars/menus in SharePoint
              • Changing the rendering/behavior of a specific field in a list

              In other words, SPFx extensions provide the equivalent of CustomActions and JSLink – previous dev approaches which didn’t necessarily translate to modern pages.

              In this article I want to focus on the first two scenarios listed above (in bold) – referencing some JS on every page, and also running some code to put something in the header area of the page. The documentation provided by Microsoft does a good job on the 2nd scenario, but sometimes it’s good to have something a bit more visual so I’ll provide more screenshots. I’ll also talk about the scenario where you don’t necessarily want to add some *content* to the page, but you do want to add *some other form of script* to run on every page (e.g. analytics/whatever).

              In terms of injecting content into the page, we now have the following zones in modern pages (N.B. these are the names from SPFx 1.2 onwards):

              • Top
              • Bottom
              • DialogContainer

              N.B. We can expect more zones in the future! Here’s what the Top (header) and Bottom (footer) zones look like:

              SNAGHTML4eec1a

              Key information

              Microsoft are currently saying that SPFx extensions will hit General Availability (i.e. fully-released in all tenants and suitable for production use) in fall/autumn of 2017. Until this time they are in preview.

              Also be aware that what makes the new extensions possible is Microsoft's updates to tenants (only in developer tenants at the time of writing, not even in First Release), and updates to the Yeoman Generator that developers use to get started - this has a new set of component types which get you started with the right default code.

              SPFX 1.2 changes

              • Changes to placeholder names – “Top” and “Bottom” insteaad of “PageHeader” and “PageFooter”
              • The onRender() method is deprecated/should no longer be used in SPFx extensions

              Previous limitations with modern pages

              Modern pages have been frustrating because:

              • No possibility to run custom script
                • Global JS added with previous methods (CustomAction + JSLink) did not run here – only on “classic” pages
              • Corresponding lack of page extensibility
                • No way to inject content into the page

              What’s changing here is that Microsoft are providing a hook to run your code, and are also providing named placeholders on modern pages – zones of the page which you can add content to. So long as you stick to these zones and don’t arbitrarily “hack” the page by changing other DOM elements (e.g. with jQuery or similar), then Microsoft effectively guarantee that updates to Office 365 will not impact your customizations.

              The script you provide has to be installed to an app catalog and deployed that way, meaning that there is effectively an approval step. This means that simply editing the page to add a Script Editor web part no longer exists as the easy option – the script must be OK’d by an administrator. Lots of debate on this one of course, but ultimately it’s what Microsoft need to do to facilitate more governance and safeguard Office 365 as a stable platform.

              Targeting placeholders such as the Top and Bottom zones

              In earlier versions of SPFx, some pages only had the Top zone but missed the Bottom zone. That’s now been fixed and it seems that if the Top zone exists on a page type (e.g. modern page, Site Contents page, document library or list page etc.), the Bottom one will too:

              SNAGHTML53485bb

              I showed a relatively narrow bar above, but there’s nothing to stop you making that top zone larger if you want to with CSS (this image is zoomed out):

              SNAGHTML2418631

              But of course, all this only applies to modern pages – classic pages do NOT have these zones or support SPFx extensions in general:

              SNAGHTML48946ed

              I’ll talk about the end-to-end process later, but to get straight to the code - with some minor tweaks/simplification to the suggested code in the documentation, mine looks like this:

              And the CSS is implemented by adding an SCSS file in your extension’s directory – mine is named AppCustomizer.module.scss and has the following content:

              Remember this is imported to the class for your customizer e.g:

              import styles from './AppCustomizer.module.scss';

              So, the key elements here are:

              • A class that derives from the ApplicationBaseCustomizer class
              • Use of the this.context.placeholderProvider.tryCreateContent() method to get a reference to the appropriate placeholder and it’s content - and the fact that it gives you the DOM element to manipulate (e.g. set innerHTML)

              Deployment options – global or site-by-site

                In terms of what associates your customizer to the site, there are two ways of doing this in production:

                • Site-by-site – in this approach, you add some declarative XML to your app packaging, and then ensure the app is installed from the App Catalog to each site where your extension should operate. Specifically, your customizer has a manifest file which contains it’s ID ([MyCustomizer].manifest.json), and on top of this you actually add an elements.xml file with a CustomAction element (just like the old days!). This has a new "’ClientSideComponentId” attribute, and this must point to the ID of your customizer.
                • Global/scripted – in this approach, you set the skipFeatureDeployment attribute to “true” in youre package-solution.json file, and then use CSOM or REST to add a CustomAction programmatically to each web as you need (i.e. by iterating, or including into some provisioning code). See https://dev.office.com/sharepoint/docs/spfx/tenant-scoped-deployment for more details. When using this approach, the admin has the option of making the SPFx web part/extension globally available when installing to the App Catalog:

                  SNAGHTMLf7281c2

                  SPFx web parts will show up in every site, but as I say, for SPFx extensions you also need to take care of the programmatic association/registration to each site/web you require, using CustomAction/ClientSideComponentId. See my post Manage tenant-scoped SPFx extensions across your SharePoint sites for some PowerShell/C# code to do this.

                But before packaging for production, there’s a mode when you can dev/test your customizer before worrying about packaging. This works by running a “gulp serve” locally and adding some querystring parameters to a modern page so that the manifest is loaded from localhost – it’s a bit like the “local SPFx workbench” equivalent but for SPFx extensions/customizers.

                But I don’t need placeholders – I just want to reference some JavaScript on every page!

                In this case, the code is somewhat simpler. If you have an external JS file you want to reference in a quick and dirty way, you could do this by dynamically adding a script tag to the <head> element of the page. My testing shows it seems safe to do this in the onInit method, but the onRender method would be fine also – in any case, it’s just the old-fashioned method like this:

                But consider!

                • If the JS is hosted on another domain, you may need to enable CORS there (depending on what your JS is doing)
                • If you're referencing a module script, you could do this in a cleaner way by referencing it as an external module in the "externals" section of your config.json file (see Add an external library to your SharePoint client-side web part for more). I've tested and this approach does work with an Application Customizer
                • You could also choose to bundle your script if that made sense, and ensure it was referenced in the onRender method for your customizer. That should work too..

                Process

                The process is effectively the same whether you're targeting page placeholders or just referencing script on every page:

                Update the SPFx Yeoman Generator if needed

                The first step you might need to do is to update your SPFx Yeoman Generator – assuming you already have all the bits installed, you can do this by typing “yo” at the command-line and then going through the update process:

                SNAGHTML38f4e932

                Choose the “Update your generators” option and select “@microsoft/sharepoint”:

                SNAGHTML38f64e11

                SNAGHTML38f7128a

                Creating an Application Customizer extension

                [N.B. I’m essentially duplicating/walking through the main “Build your first extension” documentation here – you should reference that too.]

                Once you’re ready to actually create your app customizer, do this by running that generator:

                SNAGHTML3902e5e2

                Give your solution a name, and ensure you select the “Extension” option:

                SNAGHTML1dbcd6b

                In this case, we’re using Application Customizer (rather than ListView Command Set Customizer [CustomAction/toolbar replacement] or Field Customizer [JSLink/field replacement]):

                SNAGHTML1d772eb

                Provide a name for your customizer and then a description:

                SNAGHTML212d82d

                The generator will then get busy creating your application with the appropriate files, and then you’ll see:

                SNAGHTML2114dd4

                Your application has now been created and you’ll get the boilerplate code (which may look a little different to this in later versions of SPFx):

                SNAGHTML2152c30

                It’s a good idea to test running this in debug mode before making any code changes, so do this by running a gulp serve with the “nobrowser” switch:

                SNAGHTML216506c

                The next step is to browse to a modern page, but adding some querystring parameters in the URL so that our *local* manifest for the customizer is loaded. First, open a browser to a modern page – a document library is a good choice:

                SNAGHTML21b7c46

                And then in Notepad or similar, build the querystring parameters you need. This basic format of this is:

                ?loadSPFX=true&
                debugManifestsFile=https://localhost:4321/temp/manifests.js&
                customActions={"badba93c-7f98-4a68-b5ed-c87ea51a3145":{"location":"ClientSideExtension.ApplicationCustomizer","properties":{"testMessage":"Hello as property!"}}}
                  

                However, you’ll need to replace the ID with the one from your customizer’s manifest file:

                SNAGHTML224e8a1

                SNAGHTML226aa94

                If you paste that onto the end of the URL to the document library in your browser window and hit enter, you should see a warning message related to debug mode:

                SNAGHTML228610e

                Click the “Load debug scripts” button, and then your code should execute and you should see the results – in the case of the boilerplate code, it’s an alert box:

                SNAGHTML22a488b

                Success! You’ve now run an Application Customizer in debug mode.

                Packaging for production (site-by-site/declarative approach)

                For this, I recommend following the steps in the documentation (start at Deploy your extension to SharePoint) – but below is an extract of the main steps. Ultimately it revolves around:

                1. Building your app, and deploying the bundled JS files to somewhere like a CDN (just like an SPFx web part)
                2. Adding some packaging files to your app, so that your customizer is called when the app is added to a site (a bit like feature activation – in fact, it IS feature activation ;))
                3. Deploying the app package to an App Catalog, and then adding the app to a site

                In terms of the process, key steps are:

                • Create SharePoint/Assets folder and add an elements.xml file:

                  SNAGHTML3954c9a1
                • Add the contents to elements.xml – set the “ClientSideComponentId” to identifier of your customizer i.e. the one found in the [MyCustomizer].manifest.json file (remember, you can skip this if you plan to use skipFeatureDeployment=true and globally deploy via script):

                  <?xml version="1.0" encoding="utf-8"?>
                  <Elements xmlns="http://schemas.microsoft.com/sharepoint/">
                      <CustomAction
                          Title="COB Global JS"
                          Location="ClientSideExtension.ApplicationCustomizer"
                          ClientSideComponentId="5dba1a34-6bbe-42ef-be72-94e01b527ce2">
                      </CustomAction>
                  </Elements>
                        

                • Edit the config\package-solution.json file – add a “Features” node to reference your elements.xml file. It needs contents similar to the following:

                    "features": [{
                        "title": "COB AppCustomizer - global JS",
                        "description": "Adds some JavaScript to every page in the site",
                        "id": "456da147-ced2-3036-b564-8dad5c1c2e34",
                        "version": "1.0.0.0",
                        "assets": {        
                          "elementManifests": [
                            "elements.xml"
                          ]
                        }
                      }]
                •       
                • Take care of some other steps related to CDN-hosting of your JS bundle (e.g. updating the ‘cdnBasePath’ property in the ‘write-manifests.json’ file), and then bundle and package your app using 'gulp bundle --ship' and 'gulp package-solution --ship' respectively.
                • As I say, head to the documentation for the full steps when you actually come to do this.

                  The app is then upload to the app catalog:

                  SNAGHTML2dc2e6a

                  Notice that at this point, the admin needs to trust the application and will see where the remote files are hosted - in my case, I used the Office 365 public CDN:

                  SNAGHTML9a5ac5

                  You should then see your customizer take effect, and if you go looking you’ll see a web-scoped feauture (by default) which is binding your customizer to the site:

                  SNAGHTML2cbe9ae

                Other matters 

                • Property bag – as shown in the “Build your first extension” page, there’s a property bag of sorts that can be used with customizers. In production mode, properties are specified in the CustomAction element in your elements.xml file. In my example, I chose to use values specified directly in the code, but this property bag provides some level of separation (but it is still burnt into your package)

                Happy customizing!