Thursday, 29 October 2015

My wish list for Office 365 and SharePoint (late 2015)

Sometimes, it’s good to take a step back and think about the technologies you bet on for your clients, employer and career. I had cause to do this recently, and I see no reason not to publish these thoughts publically – so in this post I’d like to take you through my list of things that I’d like to see Microsoft address. In terms of timing, as we move closer to the release of new things in Office 365 (Next Gen Portals, “Planner” and so on), and of course the on-premises SharePoint 2016 release, it’s probably not a bad moment to do some thinking like this – hopefully some items on my list will be ticked off sooner rather than later :)

I’ve broken the list down into smaller/easier things to accomplish which would be good to see in the next 12 months, and then a list of perhaps bigger/longer-term things. Here we go:

Within 12 months:

  • Progress the hybrid story from "basic" to "joined-up". For orgs with on-premises SharePoint and SharePoint Online, the lack of integration/sync between taxonomy, user profiles, user experience (branding, navigation, customizations etc.) is a big problem. It's great that identity, search and OneDrive are becoming reasonably well integrated - but there's a long list of other areas which are not (e.g. see
  • Take steps to integrate recent and upcoming features with the things that SharePoint does well. Right now Office 365 Groups and Next-Gen Portals seem to be missing the mark for some of our clients, because whilst they're great at "quick and easy" they're not so good for "planned and governed". The lack of integration with taxonomy, search and Yammer creates more siloes and mean that some organizations are unhappy with the direction in these areas.
  • Provide a better top-level experience. SharePoint could be so much better if Microsoft “did more” about the top-level, in terms of a default page. Sure, lots of orgs have strong requirements around their intranet/digital workplace homepage – but many do not, yet are forced to expend effort building the top-level experience. Even those with requirements would probably benefit from better out-of-the-box pieces here.
  • Remove some of the need for site templating by fixing content types. This applies particularly to collaboration scenarios. It's great that the Content Type Hub allows an organization to easily define and manage their content types without development (especially in Office 365), but frankly it's useless because those content types do not get applied to document libraries (without manual steps in each site).
  • Fill the void on forms within SharePoint.
  • Fill the void on analytics within SharePoint.
  • Clarify the social situation/roadmap. As Wictor pointed out recently in Has Yammer played out it’s role?, there hasn’t been a great deal of innovation in Yammer, and things have gone suspiciously quiet. Whilst I agree with many of Wictor’s points, I also note that Yammer has been a pretty huge success at Content and Code (around 100 employees), despite the many sucky aspects. Which, of course, include the useless search, lack of profile integration, inability to edit comments, strange inbox, poor handling of multiple networks, lame mobile apps and so on. But genuinely, the platform still manages to work well in many contexts I see, and really good conversations and team-working can happen. Maybe 1000 employees or less is a sweet spot for Yammer? Anyway – the lack of clarity on Yammer and social needs to be addressed so that organizations can plan.

Within 3 years

  • Retain strengths around documents and collaboration, but leap forward in terms of CMS sites/pages, mobile, user experience and certain specific workloads. In terms of the last point, SharePoint could do such a better job at things common to most organizations - running projects, idea generation, on-boarding new employees and so on. These things don't need to be best-of-breed, but having capabilities in these areas which integrate with other pieces would be hugely valuable.
  • Address key end-user scenarios which still cause pain. Moving/copy and pasting items, sharing content *easily* between sites (cross-site publishing doesn't currently cut it), seeing all the sites I can contribute to, having a proper site directory (or other discovery tools) and so on. It's a big shame that we still have these constraints in 2015.
  • Provide a simple option for self-service sites created from a simple custom template (e.g. team/project sites with some tweaks). The OfficeDev Patterns and Practices initiative helps developers implement this, but the "clone from this master site" feature should be baked into the product and have a FAR lower barrier to entry than it currently does IMHO.
  • Resolve the constraints and pain points which come with the current architecture. The site collection boundary is a good example of this - branding, many site settings, customizations and many more are challenging to implement globally.
  • Eliminate some of the complexity around the remote code model (e.g. more streamlined integration with Azure). Yes, it is HUGE that we now have an enterprise-grade way of customizing SharePoint/SharePoint Online, whilst not impacting the stability of core SharePoint. Kudos Microsoft! However, whilst it’s great for folks like me (and my employer) who can wrap their heads around Azure, Azure AD, development models such as Office 365 apps and SharePoint Add-ins, authentication options and so on - I’d say the world would probably be better off if Microsoft could simplify this landscape somewhat. More progress on integration between pieces in Office 365 and Azure is probably the right angle here.


So that’s my wish list as we go into November 2015. No doubt with a bit more time I would add some more items, and we could go on forever in some ways. But what’s on your list? Do you agree with some of my points, or do you have a completely different set of burning issues?

And don’t forget, the various UserVoice channels are a great way to get your message across to Microsoft. Personally, now I’ve got this off my chest I’ll go back over this list and consider what should be added there, and I suggest you do the same for yours too :)

Wednesday, 16 September 2015

Using Azure Deployment Slots to implement dev/test/production ALM for Office 365 apps and SharePoint Add-ins

This is the 4th article in my Challenges in Office 365 development – and how to address them series. In the past, I’ve recommended the use of *multiple* Office 365 tenancies where custom development around Office 365/SharePoint Online is happening - after all, the need for separate dev/test/production environments typically doesn’t go away in the cloud, just because these aren’t physical environments you yourself are hosting. BUT, this multiple tenancy arrangement sometimes brings additional challenges – in previous articles I’ve discussed how dev/test Office 365 environments often don’t have production-like aspects such as identity integration and SSO, but any development of custom apps also brings challenges. Fundamentally this is because both Office 365 apps and (typically) SharePoint Add-ins need to be registered in each environment where the app will be used – and this gets problematic because each registration gives a different client secret and password, but your app typically only caters for one. But before we get too deep into the problem (and potential solutions), let’s consider this in the context of my article series:

The problem – using the Office 365 APIs when using multiple environments

So I’m focusing primarily on Office 365 apps here - in other words, some kind of application (a web or client app) which uses the Office 365 APIs. Of course, there are other flavors of apps, such as provider-hosted SharePoint Add-ins – I’ll discuss these specifically towards the end of the article, since they have some different considerations.

But Office 365 apps are becoming “the new normal” for app development in Office 365, not least because you can step outside of the Office 365 APIs and use good old-fashioned SharePoint CSOM/REST APIs if you need to (but without having to use SharePoint app authentication). So, in my view it’s starting to make less sense to create apps which will run in Office 365 as a SharePoint Add-in, rather than just creating an Office 365 app. But back to the story - to be able to read/write data to Office 365, the critical factor is that the app is registered with the Azure AD behind the Office 365 tenancy you’re using.

If you’re not ultra-familiar with this area and that sentence doesn’t make sense to you, consider that each Office 365 tenancy has an Azure subscription behind it. By default, this is a background thing only, and only contains the Azure AD directory, which holds your Office 365 users. But it can be turned into a “full” Azure subscription by providing a credit card – this unlocks the ability to use the full set of other Azure capabilities in this subscription, such as Azure web apps.

Now, if your app is a web application and you want to host it in Azure, the app itself does NOT need to be published to (and run from) from this Azure subscription. But it IS mandatory that the app is registered in this Azure AD. I’ve seen quite a few developers trip up over this – perhaps because they published the application to an Azure subscription they have access to, but this in fact has no relationship to the Azure instance (and specifically the Azure AD directory) behind Office 365.

So, we could show this relationship like this:

So, what’s the problem with our app needing to be registered in different places? Well, in addition to the general overhead which comes with this, the big issue is that each registration gives a different client ID and password. However, the design of both Office 365 apps and SharePoint Add-ins (provider-hosted) is that the client ID and password are stored in web.config – Microsoft’s authentication-handling code which gets included in every app looks for them here.

Assuming we don’t want some weird arrangement of having different web.config files for each environment, what can we do? Well, I guess some other options could be web.config transformations or rewriting Microsoft’s auth/credential-handling code – but neither of those are appealing.

If you’re hosting your app in the cloud, Azure has some nice functionality in the form of “deployment slots” which can help here.

Azure Deployment Slots – a brief introduction

Deployment Slots are a capability of Azure Web Apps (previously known as Azure Websites). They are only available when you’re using a Standard or Premium pricing plan. If you go into the “Settings” area of your web app, you can click into “Deployment Slots” and from here create a slot:

You might name this slot something like “Staging”. You can choose whether to copy any config from your existing “main” slot (which is there by default), or not:

For Office 365 apps, this isn’t important as there aren’t lots of config settings to worry about. Using this process you can create up to 5 deployment slots on a Standard plan, and up to 20 on a Premium plan – each slot effectively gives you an additional “instance” of your web application, accessible on a different URL. So for a website at, if you create two slots named “test” and “staging”, you’ll end up with a set of URLs like this:

This is great Azure feature, which provides test/staging capabilities for your site in a simple way. There are two key aspects which provide useful support:

  • The ability to “swap” the content of deployment slots – this is effectively your deployment mechanism to promote a version of your site’s code from test/staging to production
  • Each deployment slot can have its own configuration, such as connection strings and AppSettings values

For the configuration settings, values can be entered in the Azure portal (or via the Azure APIs), and when you do this these values override any specified in web.config - it’s this feature that we can leverage for Office 365 and SharePoint applications. There are a few other things to line-up so that your app can authenticate to Office 365/SharePoint, but I’ll go through those later.

In terms of working with deployment slots, you can deploy your site/app to a particular slot using the typical “publish to Azure” approaches – publish from Visual Studio, Web Deploy, PowerShell, FTP, publish from source control and so on. Once you’re at the point where your app is deployed to a staging slot, to push this version of the code to production you can swap the staging and production slots – either with this button in the Azure portal:

..or with the Switch-AzureWebsiteSlot PowerShell command. This truly is a swap, and in fact happens via a simple DNS update – so if all is good with your code update, you now have an out-of-date codebase back in your staging slot. You can fix that by simply overwriting the content there with a subsequent direct publish to that slot, but what’s useful here is that you have a quick, robust roll-back mechanism to lean on if things didn’t go to plan.

You can read Set up staging environments for web apps in Azure App Service to understand more about working with Deployment Slots in general, but I want to now discuss them specifically for Office 365 apps and SharePoint Add-ins – with the former first.

Implementing dev/test/production for Office 365/SharePoint apps using Deployment Slots

So now that we understand something of how deployment slots work, let’s consider ways in which they can be useful for Office 365 apps and SharePoint Add-ins. For one thing, the “slot-specific configuration” capability is great because we could use it to specify a different client ID and password for different environments, thus side-stepping the single web.config file problem. But interestingly, deployment slots also give us a way to have dev/test/production instances of our app, without having to register them with different Office 365/SharePoint environments. Let’s think about that for a second:

Implementing dev/test/production PURELY on the Azure side

What I’m suggesting here is that deployment slots are used to implement dev/test/production PURELY on the Azure side. Using an Office 365 app as an example, all instances of the app are actually registered with the production Office 365 environment only. This is great because it allows us to avoid many of the problems we listed earlier in this series – things like dev/test environments suffering from different configuration (e.g. authentication, Office 365 plan type etc.), a reduced set of users, lower quality content/data, and so on. We get to now build and test our app in the context of production, and in particular, against the Azure AD directory that holds our production Office 365 users. However, we are still maintaining the dev/test/production separation of our app that we need to do a good job of releasing and maintaining the thing. Our app still has robust ALM practices, but they’re implemented on the Azure side – and that can work well, since that’s where the app’s code and implementation is.

There are few details to line-up, since you need a registration of your app for each “instance”, even if they all happen in one Office 365 environment. But it’s simple enough, and I’ll go through these shortly.

So, hopefully I’m demonstrating that this can be a good option. But before we go much further, it’s also worth considering things to be careful of and when it might NOT be appropriate:

But consider if your app can do "dangerous" things in testing!

So, we have our dev/test/production instances of our app, and these may or may not be running the same version of the codebase (e.g. it might be different during an upgrade cycle). But all instances are talking to the same Office 365 tenancy – this is usually good because the same Azure AD is used, but it can also be bad (or at least need some consideration) depending on what your app does. If your app can read/write to Office 365 data, and presumably it can, then consider that it’s the same set of Office 365 mail, calendar items, tasks and OneDrive for Business files that all instances of your app are running against – and this is the user’s production data! Clearly if your app can WRITE data, send e-mails or make other changes, you need to bear this in mind, especially when testing. Similarly, if you’re doing anything with SharePoint Online data (e.g. writing data to lists or libraries), then the same consideration exists. You could implement some specific logic in your app, but ultimately the “separation only in Azure” approach may not be appropriate for all cases – you’ll need to use some judgement as to whether you’d be better handling ALM another way. For example, this could be registering your app in each environment you have and publishing each to a separate Azure web app.

What it looks like – using Deployment Slots for dev/test/prod in an Office 365 app

So, creating a new Deployment Slot will give you a new URL such as, and it will now be listed in the Azure Portal:

But at this point, the slot is empty and has no website content:

So, we need to publish our app/website content to this slot – either using one of the approaches described above (such as publishing from Visual Studio), or perhaps something better. If we’re using Visual Studio Online for source control, it’s very simple to have automated builds deploy the latest build to a particular deployment slot – so your CI process always deploys to a test or staging slot, but you manually choose when to swap that with production for example. I’ll show some of this CI approach in a video in a future post - but in the end we just need to ensure our files get published to the staging slot somehow. HOWEVER, your app will NOT be fully working at this point! At first glance, however, all looks good and you should see your app is now visible at the slot’s URL (N.B. I’m using the Office 365 Starter Project for ASP.NET MVC here):

BUT, your app will not be able to authenticate to Office 365 at this time – it is not known to Azure AD. If you click the “Sign in” link and enter credentials, you’ll get an error effectively telling you that you can’t sign in – most likely saying “the reply address ‘foo’ does not match the expected reply addresses for configured for the application [app ID]”, like this:

What’s happening here is that the values in web.config for the Client ID and password are being used. The ClientID found there matches the Azure AD app registration for the production slot – but we’re on a different URL in the staging slot, hence the “reply address does not match” error. Additionally, there’s no real way of getting to the staging version of the app (except typing the URL in the address bar), because it doesn’t appear in the Office 365 My Apps page or App Launcher. So, the two things we need to do now are:

1. Manually register the staging instance of the app in Azure AD – we’ll specify the staging slot URL for the app’s Redirect URL here, and we also need to grant appropriate permissions.

2. Configure “slot-specific” override values for the Client ID and password in Azure – to do this, we obtain the Client ID and password from the staging registration in Azure AD, and then use the Azure portal (or API) to configure these against the staging slot, to override web.config.

We’ll go into these steps in more detail next, but at this point I like to create a simple icon to distinguish the staging version of the app in the My Apps page – something like this will do:

All we’re trying to do is ensure we can tell which app is which in the My Apps page later on. But let’s now go into those two steps in more detail:

Manually registering the staging instance of the app in Azure AD

If you’ve developed an app which talks to the Office 365 APIs, you might be familiar with the “Add Connected Service..” dialog in Visual Studio which enables you to register the app in Office 365 and grant permissions. Here, we need to perform the same steps that Visual Studio does but directly in Azure AD.

To register the staging app, we go into the Directory area of Azure AD (N.B. we have to use the “old” portal at the time of writing, since Azure Active Directory is not yet available in the preview portal), and in the Applications area we click the ‘Add’ button:

The next few images show the next steps in the app registration wizard – here we’re supplying some details of our app, and notably, specifying the URL of our staging slot as the sign-on URL:

Once we’re through the wizard, our staging app is registered in Azure AD:

We now need to configure it:

On the “Configure” tab, we upload our amazing logo and more importantly, obtain the client ID and password that Azure has generated for this app registration. For the key, you need to select a key duration from the dropdown and the string representing the key will be available when you click ‘Save’. At this point, copy these values and store them in a safe place – we’ll need them in the next step:

Depending on what the application actually does, we also need to grant permissions to the respective data and services in Office 365. For example, maybe it needs to read or write to Exchange Online mailboxes and calendars, or work SharePoint sites and files:

Once saved, the registration is complete and your app will appear in the My Apps page (if it is assigned to the current user):

However, if you try to sign-in at this point you’ll STILL get the sign-in error:

So, we now need to override the web.config values to match our staging app details.

Add slot-specific config for the Client ID and password (key):

Now we need to add our slot-specific config, using the details we noted down earlier. This has to be done in the *new* Azure portal of course, because deployment slots are only available there J

In the “Application settings” area, we use the same key names that web.config uses, but for the values we’re using the ones we noted down earlier:

Now save the settings, and ensure you see a success notification:

And that’s it – our staging app is now registered in Azure, and everything should now be usable. I can now sign-in to the app successfully..

..and the app’s functionality works fine. For example, it can read the files in my OneDrive For Business folder:

So there we have it. We now have different versions of our running in Azure, but all hooked up to the production Office 365 tenancy. Although I’ve presented deployment slots for use in a dev/test/production way (perhaps the most common usage), clearly you could use them in whatever way you like and use whatever ALM approaches you prefer. Along these lines, a couple of other Azure capabilities could be useful too, such as Traffic Routing.

Azure coolness - Traffic Routing for deployment slots

Traffic Routing is a feature which allows me to direct a certain percentage of traffic to my app to a particular slot. This could be useful in a few scenarios – a soft rollout of some new functionality, A/B testing and so on. Essentially I can define what proportion of traffic each slot should receive:

Traffic Routing is cookie-based, and so there are a couple of ways to override the core behavior (which doesn’t allow segregation of traffic by any particular logic) by implementing code around this, but there’s probably not a huge amount more to say about Traffic Routing. It’s certainly a feature which could be useful in some usages though, so bear it in mind. Let’s switch our attention now to other aspects of Office 365/SharePoint development, and how things work here.

SharePoint Add-ins (provider-hosted) – what’s the deal with these?

Many of the same considerations apply for SharePoint Add-ins – here, the app gets registered with AppRegNew.aspx, and the client ID/secret is then added to the SharePoint environment’s list of “known remote callers” (i.e. AppPrincipals). Clearly, if you’re using dev/test/production SharePoint environments, then the Add-in needs to be registered with each. However, there are a couple of other things to consider for this type:

  • An alternative to registering with AppRegNew.aspx is to use the Seller Dashboard instead. This *isn’t* purely for app vendors selling apps through the Store – it makes a lot of sense to use it in an in-house corporate environment too, because you get to sidestep having to register with each different environment. (Back in the opening section of this article, it’s this option which made me say that SharePoint add-ins *typically* need to be registered in different environments – i.e. not always :))
  • If you do use AppRegNew.aspx to register your SharePoint add-in, a notable difference compared to Office 365 apps is that you can provide the same client ID/secret across your different environments (since the fields on the AppRegNew.aspx form are textboxes which you can you type into). However, consider that even with this approach you’d still need different app packages to be deployed, since each will have a different redirect URL.
  • If you’re hosting your remote components in Azure, all instances of your app could live in the same Azure subscription, with no worries about the fact that they all share the same Azure Active Directory (in contrast to Office 365 apps). This is because Azure AD isn’t used for SharePoint add-in authentication – it doesn’t come into the picture.


Azure deployment slots offer some interesting capabilities for development of Office 365 apps and SharePoint add-ins. The idea that we could implement dev/test/production versions of our app which are all talking to the same Office 365 tenancy can be very useful, as it helps get past many of the challenges with non-production Office 365 environments outlined at the beginning of this series – lack of production data, lack of a full directory of users, lack of SSO due to different authentication methods, lack of Yammer Enterprise and so on. As discussed, you need to consider if the approach outlined here is appropriate for your case. You may need to take care if your app can write data to Office 365/SharePoint for example.

But, deployment slots offer some useful possibilities around app development, and they are certainly be a useful tool in the implementer’s toolbox.

Friday, 7 August 2015

Enabling Yammer Enterprise with SSO in dev/test Office 365 environments

This is the 3rd article in my Challenges in Office 365 development – and how to address them series. Today we’ll focus on Yammer – for my team, it’s increasingly common to do small bits of development around Yammer, and so having the right set-up in dev/test environments starts to become important. This could be non-coded approaches such as use of the “Yammer Embed” approach such as showing the feed for the “Development” group in Yammer on their team site home page, or using Yammer for comments on an intranet page. But we also find ourselves implementing simple things with the Yammer API, such as automatically adding users to a Yammer group when they are granted access to a team site. With these cases, you really want the behaviour in test environments to be exactly the same as in production. But before we go any further, let’s put this in context of the overall series content:

Why Yammer Enterprise is important in dev/test environments

Let’s be clear, it *is* possible to develop for Yammer without having Yammer Enterprise – and the two scenarios I mentioned above can indeed be accomplished with the free version of Yammer. But it’s also fair to say you’ll hit some trade-offs around user identities and single sign-on (SSO) between Office 365/Yammer. If you have multiple Office 365 environments (e.g. dev/test/production), you’ll most likely be in a situation where only production has the true user experience that end users will see. This might be acceptable to you, but equally it can make things challenging for testing (“OK, but just imagine you don’t have to sign-in here!”)
Specifically, the things you’ll miss without Yammer Enterprise are:
  • Users have to manually register with the Yammer network (or be invited) using their e-mail address – this step will create their Yammer profile
  • Single sign-on will not work – any Yammer functionality on your pages (e.g. a feed, or a “Like” button) will show a link to sign-in to Yammer, and navigating to the main Yammer site will also require sign-in, rather than just automatically passing you through
  • All of the Yammer features which come with Yammer Enterprise, such as the reporting and administration tools
On a related note, developers using Yammer Embed will notice the “use_sso” flag on the JavaScript object in the Embed code (see From memory, even with Yammer Enterprise enabled this has to be set to “true” to avoid the behaviour listed in the 2nd bullet point above.
Also, it’s probably also worth mentioning that there are different flavors of SSO with Yammer. Here I’m referring to “Office 365 SSO with Yammer”, but note it was always possible to implement a form of Yammer SSO before Office 365 came along – this was typically using a SAML-based IdP (such as ADFS) for both Yammer and your other corporate systems. Depending on your organization’s situation, even if you use Office 365 you may actually find that you can actually only use “plain” Yammer SSO, rather than Office 365 SSO (e.g. you have users without e-mail addresses or multiple Yammer networks). See for more info.

A note on licensing and other pre-requisites

So, we’d like to use Yammer Enterprise even in dev/test environments if possible. However, this does require licenses for any user who will access Yammer since we’re going beyond the free version. Consider the following:
· If your dev/test Office 365 tenant is on an Enterprise plan (e.g. E1, E3) your users will automatically have a Yammer license – these do not need to be assigned in the Office 365 tenant admin screens
· If you’re on some other kind of plan (e.g. a SharePoint-only plan such as SharePoint P2), then you can “additively” pay for individual Yammer license as a bolt-on – these do need to be assigned to individual users
In addition to licensing, another key pre-req is that Yammer Enterprise can only be activated on an Office 365 tenant which is integrated with a custom domain - in other words, your user accounts will be “” rather than “”. This would either be done in the normal way during the initial set-up process, or for dev and test environments you could use the approach I discuss in Implementing AD integration with Office 365 using a sub-domain (for dev/test)
But assuming licensing and domain integration are in place, you’re good to go with Yammer Enterprise.

HOW-TO: configure Yammer Enterprise for an Office 365 environment

Go to Office 365 tenant admin dashboard, then click “Included Services”, then on the right of the page, click “Yes, activate Yammer Enterprise for my network”:
You’ll then be asked which domain you want to use for Yammer – assuming you only have one verified/integrated domain, you’re simply confirming you want to proceed:
At this point, the Yammer network provisioning process starts:
Once complete and you see the above message, you should be able to click the “log in to Yammer” link and sign-in. On the next screen, some home realm discovery stuff kicks in and detects that you can be signed-in on your current identity without providing a password:
From there you can complete the sign-up wizard – adding any Yammer colleagues, joining/starting groups, and adding a profile photo:
And voila – you now have Yammer Enterprise!
If you haven’t done already, you should switch the Office 365 tenancy to use Yammer for social:
Once all these steps are complete, you’ll know the Yammer network is a Yammer Enterprise because you should see the additional administration tools:


So that’s how to enable Yammer Enterprise in an Office 365 tenancy being used as a dev or test environment (or production for that matter – the process is the same). If you go through this process, your dev/test Office 365 environments should behave just like your production environment. This can often be an important point for testing, especially around mobile devices and other non-domain joined devices, and especially where Yammer Embed or any custom Yammer functionality has been implemented.
This series will continue to discuss options and techniques for improving Office 365 development. Other posts:

Wednesday, 15 July 2015

Debugging errors in SharePoint add-in (app) development

If I was to summarize this post in one sentence, I’d say “if your SharePoint app/add-in is giving a generic error such as ‘An error occurred while processing your request’, then you’ve probably got the add-in registration/authentication stuff wrong!”. This applies to provider-hosted SharePoint add-ins rather than SharePoint-hosted add-ins, and also doesn’t apply to apps which use the Office 365 APIs as an alternative to the SharePoint add-in model. But for anyone developing/deploying SharePoint add-ins, this is a really common experience for the modern SharePoint/Office 365 developer – everybody screws this up at some point :) There are numerous forum posts and probably some existing articles about this, but I always seem to fall into this trap (usually with a typo or something) and then spend 1-2 hours debugging and searching the internet until I remember what specifically the problem is. So, I just wanted to take a short diversion from my Challenges in Office 365 development – and ways to address them series to write about this; partly just to have a quick reference for the next time I run into the problem, because I know I definitely will :)

Symptoms of the problem

When you click on your add-in, you are taken to the remote website (e.g. on localhost, or in Azure, IIS, or wherever you published it to) but instead of seeing the default page you simply see a white page with a simple message:
This happens because of this boilerplate code which is found in most SharePoint add-ins – see the 'CanNotRedirect' case in particular:

Uri redirectUrl;

switch (SharePointContextProvider.CheckRedirectionStatus(Context, out redirectUrl)) 
    case RedirectionStatus.Ok: 

    case RedirectionStatus.ShouldRedirect: 
        Response.Redirect(redirectUrl.AbsoluteUri, endResponse: true); 

    case RedirectionStatus.CanNotRedirect: 
        Response.Write("An error occurred while processing your request."); 

If you perform any debugging and step through the code, you’ll probably end up in the TokenHelper class, looking at this method:

public static string GetContextTokenFromRequest(HttpRequestBase request) 
    string[] paramNames = { "AppContext", "AppContextToken", "AccessToken", "SPAppToken" }; 
    foreach (string paramName in paramNames) 
        if (!string.IsNullOrEmpty(request.Form[paramName])) 
            return request.Form[paramName]; 
        if (!string.IsNullOrEmpty(request.QueryString[paramName])) 
            return request.QueryString[paramName]; 

    return null; 

In many cases, what you’ll find is that null is being returned from this method – effectively no value can be found for any of the specified form parameters, either in the URL or the post body. We’ve successfully passed lots of earlier code and a valid context token is passed from SharePoint/Office 365 into the add-in, but we’re failing at this point.

Digging deeper

When you click on an add-in (e.g. in the SharePoint Site Contents page), you are taken to AppRedirect.aspx – this page eventually does a redirect to the redirect URL you should have specified in the app registration. This is a form POST, and in the body of the request is token needed by your remote code. Or at least, it should be. In the case where things aren’t working, you’ll notice that a token is NOT passed (e.g. in Fiddler):
Add-in redirect - Fiddler 1
Add-in redirect - Fiddler 2
We can see two things about the body of the form post from the image above:
  • The SPAppToken parameter is empty
  • The SPRedirectMessage parameter gives us a clue about my specific problem, with a value of “EndpointAuthorityDoesNotMatch
To be clear, the EndpointAuthorityDoesNotMatch message is just one flavor of this problem – you might see something else here depending on precisely how you’ve messed up your app registration/authentication setup :) For example, you might have the add-in registration all correct, but you’re trying to run the application on HTTP rather than HTTPS – in that case, you’ll see:
  • An empty SPAppToken
  • The SPErrorInfo parameter containing something like “The requested operation requires an HTTPS (SSL) channel. Ensure that the target endpoint address supports SSL and try again.”
The message will be encoded so will look more like this (for anyone pasting this into an internet search):
If you’re developing a high-trust add-in (for on-premises SharePoint), another flavor of the problem is that you haven’t lined up the authentication requirements – in particular the token-signing via a certificate aspects. See Package and publish high-trust apps for SharePoint 2013 for more info if this is your scenario.
In my case, EndpointAuthorityDoesNotMatch is telling me that the URL of the remote application does not match what is expected from the app registration for this client ID. I can use the /_layouts/15/AppInv.aspx page to lookup the details I specified when I registered my app with /_layouts/15/AppRegNew.aspx. IMPORTANT: if that last sentence doesn’t mean much to you and/or you’re starting out in add-in development, then your mistake is probably that you just haven’t registered the add-in at all! The next section might help:

Background - key things to know/remember when developing add-ins

Add-in registration

Add-ins need to be registered with SharePoint/Office 365 with a set of required permissions requested (which the person installing the add-in needs to consent to) – without this, you effectively just have some random code outside of SharePoint which is trying to read/write data, and so the security mechanisms are kicking-in and your code is blocked. This can be done via the Seller Dashboard for add-ins that will be widely distributed (even if just amongst your own environments), or using the AppRegNew.aspx page for other types. I suggest reading the following if you need some background on this - Guidelines for registering apps for SharePoint 2013
“Development mode” vs. “packaging for deployment mode”
Visual Studio really tries to help you out when developing add-ins. When I hit F5 to launch and debug my add-in, quite a few things are taken care of so I can get running quickly and see my code - I think of this as “development mode”. Let’s talk about what happens here, and also what you need to think of later on when packaging for deployment to another environment.
Development mode:
Upon pressing F5, the add-in is packaged up and deployed to the SharePoint site specified - this can be local or in SharePoint Online, and any previous installations there are removed. Additionally, the add-in is automatically registered with this SharePoint environment. Assuming the website for the add-in (i.e. the remote ASP.NET site) can indeed be browsed on the URL specified, then the F5 process opens up a browser window where the app is being installed/trusted, and you can then click on the icon and enter the app. Some other points to note:
  • The URL for the remote ASP.NET website is specified in the properties for the add-in web project – by default, a “localhost” URL such as https://localhost:44311/ is used against your local IIS Express instance. It’s possible to change this in the project properties, for example to a named virtual directory URL for use with a full IIS instance.
  • If using IIS Express, you should consider if localhost can be used with HTTPS on your machine. I had it in my head that it was possible somewhere to specify that it was SSL should NOT be used for the remote website – but frankly I can’t now remember if this true or find how it’s possible now, so that could be nonsense :) I always run on the full local IIS instance and have SSL running on localhost, and that works for me.
  • Authentication also needs to be considered. Often you’ll need to ensure “Windows Authentication” is enabled on the web project, so that the current user can be identified and authentication back to SharePoint can take place. If, however, you’re developing an Office 365 add-in which will authenticate to Azure AD, this isn’t the case.
  • In development mode, your app manifest is expected to have a ClientID value of “*”. Visual Studio will replace this at packaging/debug time with the correct value.
Packaging for deployment mode:
When you’re ready to think about deploying the add-in where other people can see it (e.g. Azure, some IIS servers, or somewhere else that isn’t your local dev box), you need to change a couple of things in your files before deploying. For an end-to-end walk through of the deployment process (using Azure Web Apps as the hosting platform), see my Deploy SP2013 provider-hosted apps/Remote Event Receivers to Azure Websites (for Office 365 apps) post – it was written a while back but I’ve updated it since to account for changes in Azure and Visual Studio. But here are the key changes to make from “development mode”:
  • Deal with the add-in registration – go to AppRegNew.aspx in the SharePoint environment the add-in will be used in, and register the add-in (reminder – either see my last link or Guidelines for registering apps for SharePoint 2013 if you’re not clear on this step!) For the app/add-in ID, either generate a new GUID on the page or use the existing ClientId value in your project files – the key thing is that they match, that’s all.
    • Make a note of the ClientId and ClientSecret used/generated on the AppRegNew.aspx page – you’ll need these!
  • Ensure the AppSettings section of your web.config contains the ClientId and ClientSecret from the add-in registration (previous step) – overwrite the existing values if needed. The authentication code baked into every add-in by Visual Studio tools (TokenHelper.cs) will read the values from here.
  • Optionally, find your appmanifest.xml file and replace the “*” in the ClientId value with the add-in ID from the registration – effectively hard-coding the value in there. This step is technically optional, because the process of packaging the app in Visual Studio (next step) will actually replace the “*” with a GUID value you specify anyway, but it can make sense to not rely on this and explicitly specify the GUID in appmanifest.xml instead - for example, if you’re distributing the VS project files, or perhaps aren’t expecting to do much more local development/testing. Otherwise, we’ll deal with things in the next step.
  • Optionally, replace the “~remoteAppUrl” value in appmanifest.xml also – it’s the exact same deal /consideration as the previous point.
  • Publish both elements of the add-in (the add-in remote website and the add-in package) using the Visual Studio “Publish..” mechanism. Right-click on the add-in project and click “Publish..”, and you should see the dialog helping you publish the outputs of both VS projects:
    VS SP add-in publish dialog 1
    You’ll need to press both of these buttons :) The first one will help you publish your web project – if you want to publish to Azure easily, ensure you have the Azure SDK installed. This will allow you to “discover” and publish to Azure Web Apps (websites) within your subscription, without having to separately download the publishing profile used for the connection.

    The second button will package the app – you’ll be asked for some key details which will be baked into the package:

    VS add-in publish dialog 2

    During the packaging process, a couple of things will be baked into the appmanifest.xml in the .app file which is generated (and this is why some of the previous steps are optional):
    • The ~remoteAppUrl token will be replaced with the remote website URL you specify
    • The “*” in the ClientId value will be replaced with the value from the Client ID textbox above

      NOTE - you won’t see any changes in your source appmanifest.xml file. It’s only if you crack open the add-in, by renaming from .app to .zip, and examining the files inside will you see where this has happened.
  • Take the add-in package and deploy it to the SharePoint add-in catalog for your environment (be in Office 365 or on-premises SharePoint). Drag the .app file into the catalog, and add any additional details (e.g. add-in description, screenshots etc.) to the list item for the file.
The add-in is now available to be installed to SharePoint sites.

Back to my problem

Anyway, AppInv.aspx allows me to enter a client ID/app ID (which identifies an add-in/some remote code) and see the details of the registration:
When I click the “Lookup” button, the other textboxes get populated with the details of the registration, as shown above. In my case, I can now see the mistake I made – it’s the “www.” in the app domain/URL. When I consider the URL my remote site is actually running on, it’s actually just https://cob-[foo] without the “www.” – as you can see if you look back at the first image in this article.

So, the solution in my case is to re-register the add-in with the correct URL.
It’s not possible to modify an existing add-in registration, so you need to do a new registration with a new App ID (and update the Client ID value in your app manifest etc.) You can tidy up the old app registration by going to AppPrincipals.aspx and deleting the original app principal there.


There are a few common pitfalls around developing SharePoint add-ins, and even with a reasonable level of experience it’s easy to fall into one of these! Some key things to look for include mistakes in the add-in registration, ensuring you know the difference between “development mode” and “packaging for production mode”, and also the need for HTTPS on the remote web application. If everything is straight, your add-in should work fine however.
Steve Peschka has some more good tips in this area at

In the next post, I’ll resume my series on Challenges in Office 365 development – and how to address them.

Wednesday, 17 June 2015

Implementing AD integration with Office 365 using a sub-domain (for dev/test)

As discussed in Challenges in Office 365 development – and how to address them, it’s fairly common to create multiple Office 365 tenancies to get around the fact that there’s currently no such thing as a “test environment” in Office 365. However, as the previous post discusses in detail, it’s certainly true to say that some trade-offs come with this method, generally related to environmental differences, code issues, testing, Application Lifecycle Management and so on. This article series attempts to provide some options and techniques which might be useful:

Summarizing the problem

Non-production tenants used for dev and test typically lack some of the aspects of the production environment – things like:

  • Directory integration (i.e. users sign-on with their “real” Windows account – e.g. “” rather than “” cloud accounts)
  • Synchronization of user profile data between AD and Office 365
  • Yammer SSO (or Yammer in general!)

And this leads to a long list of compromises – from not being able to test the real user experience properly, to sometimes having to write code to use a different “mode” in dev/test compared to production (perhaps because user account names work in different ways, or profile data isn’t populated).

Background – using one AD and multiple subdomains for Office 365 integration

So, we’ll look at one option for mitigating this issue – a technique which allows configuring non-production Office 365 tenancies with full directory integration and synchronization of user account data, but WITHOUT the need for a separate custom domain and Active Directory for each (and all that entails). This post is effectively a big HOW-TO article which describes the configuration process. In a team working on perhaps 15+ Office 365 implementations at any one time, you can probably see the attraction of this technique for us! Frankly, it can be challenging in the enterprise to get a test AD and domain set up which can be used with test Office 365 environments – with servers, networks, AD, security and operations all involved, quite a few people across I.T. might be needed there. So, the idea of having ONE Active Directory which could be used for directory integration with MULTIPLE Office 365 environments can be useful for lots of different teams in many different contexts.

What we're doing here
Ultimately the way we achieve this is to perform the regular “custom domain integration with Office 365” config steps, but with a couple of tweaks. Office 365 administrators and platform people will be very familiar with the standard process, and it’s a key element of hybrid SharePoint/Office 365 configuration. However, most developers will be less familiar with this stuff – but I personally think it’s hugely beneficial for any technical person working with Office 365 to have been through this process at least once.

At this point, I need to make clear that a lot of the smart thinking behind this approach actually comes from one of my colleagues at Content and Code – I’m really just being the mouth-piece here ;) Tristan Watkins is Head of Research and Innovation at Content and Code, and Tristan did the initial work of proving the approach – awesome work as usual.

What we actually do is use a sub-domain for each different Office 365 tenancy we want to integrate with the Active Directory/domain. So this could be:


Of course, you might substitute “Client” with “Project”, or anything else that you’re using different tenancies for.

Or perhaps:


..and so on.

Ultimately where we’re going with this is that we’ll have one AD which can provide accounts to multiple tenancies. Whilst it may not be appropriate for production, this can really help us with our mission to make dev/test environments more like production. User accounts in Office 365 don’t *have* to come from AD when cloud identities are used (more on this later), but it can be useful to stay closely with the model that you’ll use in production and implement sync between AD and O365.

Things you’ll need

We need to be ready with each item in the list below – if not, you’ll need to purchase/obtain the following:

  • An Office 365 tenant ready and provisioned
  • The top-level domain (e.g. – this needs to be something you/your organization already owns and has registered
  • Ability to manage DNS for the domain (usually at the ISP hosting the domain)
  • Ability to create user accounts in the AD you’ll use

At a high level, we go through the usual steps to add a custom domain to an Office 365 tenancy – I’m using GoDaddy to host my domain/DNS, and Office 365 offers some useful integration/simplified admin with this and some other hosters. However, with the sub-domain approach, even if we ARE using GoDaddy to host our domains, we’ll need to manually configure our DNS records rather than can click any of the “magic buttons” to buy or configure the domain (like this one):

GoDaddy shortcut

So, just be aware that although Office 365 has numerous “simplified admin” options if you’re using GoDaddy, for the most part we’ll be ignoring these and doing some extra configuration ourselves.

HOW-TO: configure Office 365 domain integration with a sub-domain

For reference, in this walkthrough I’m using the following:

Office 365 domain

Custom domain (top-level)

Subdomain used for this tenancy

cobsp this gives:

- URL =

- Usernames =

So here’s the process. Firstly we go to the Office 365 admin center, and then into the “Domains” area. Once there, we click the “Add domain” button on the “Manage domains” screen:

Add domain

We then start to step through the configuration of adding a custom domain to Office 365:


Hit the “Let’s get started” link on this screen. On the next screen (below), we should enter the subdomain/domain we are intending to use for this Office 365 tenant – as mentioned previously, we’re going to need to add DNS records there so we should be ready for that.


Once the domain is entered, hit “Next” and see the screen below. Here, you need to NOT sign-in to GoDaddy and have Office 365 attempt the config for you. Instead, the DNS records need to be created manually, so click the “use a TXT record” link as shown below:


This next screen gives details of the TXT record you’ll need to add at your hoster (GoDaddy in my case):


Doing this helps authenticate the fact you do indeed own and control the domain, and aren’t trying to pull some sneaky internet trick. So, we now need to go to the control panel of the ISP hosting our domain, and specifically the DNS area. At GoDaddy, it looks like this:


I use the “Add Record” link to add a TXT record with the details Office 365 gave me. Once done, I wait a while and then go back to Office 365 and confirm I’ve now done this step. If I added the TXT record as specified and DNS has propagated, I should see something like the following:


I click the “Next” button here. This will update any user’s e-mail addresses from to in my case:


You can then optionally add some additional users (manually). If you plan to sync users from an on-premises AD using AAD Sync you probably won’t use this option, but for dev/test environments you might want to add a couple more users at this stage (assuming you have licenses for them):


Now we get to the important part – adding the main set of DNS records to support the integration with your custom domain:


Office 365 asks you which services you plan to use/integrate, so that you only need to worry about the relevant DNS records:


On the screen which follows, there’s another one of those super-handy shortcut links that unfortunately we can’t use (because we’re doing the non-standard thing of using a sub-domain) J Skip the “Add records” easy button, and instead click the “add these records yourself” link:


The screen which follows lists all the DNS records you need to add at your hoster. You now need to cross-reference this screen with the DNS panel there – for each item listed, configure the DNS record in the admin panel. The key difference compared to the standard process is that the sub-domain is specified in each DNS record, rather than just the top-level domain:


Here’s an example of adding a record in the GoDaddy DNS admin panel:



Once you’ve added all the records, go back to Office 365 and click that “Okay, I’ve added the records” link. Office 365 will now check the records you added, and if successful you should see a confirmation message as shown below:


Your custom domain is now integrated, and the status should be reflected on the “Manage domains” admin page:


Review - so what do we have now?
At this point, we now have our on-premises AD integrated with our Office 365 tenancy, using a sub-domain. However, we don’t have user accounts actually existing in Office 365 yet – accounts can be created there in different ways, and we need to use one of them to actually get identities up into Office 365 for testing. This applies regardless of whether we’ll actually log-in with these accounts, or simply use them as “passive” users. Ultimate these are cloud identities, so if these users will log-in our model is to use cloud authentication rather than the other option (federated authentication e.g. with ADFS or similar).

By the way, if some of this is confusing and you need a good background on the different options for identities and authentication in Office 365, I’d suggest starting with User Account Management on TechNet.

Actually creating users in Office 365 – AAD Sync, manual, or bulk update

With or without a custom domain (or subdomain), users can be created in the following ways:

  • Manually in the Office 365 administration UI
  • Using PowerShell
  • Using bulk upload from a CSV file
  • Using AAD Sync
  • Using an Exchange mailbox migration

In other words, it’s worth remembering that even when you’ve integrated a custom domain with an Office 365 tenancy, users do not *have* to come from AD. However, this is definitely possible if you want to stay close to production and perhaps have synchronization occurring from AD to Office 365.

Using AAD Sync to sync users from AD to Office 365

Implementing sync from AD with this sub-domain approach is made possible by adding an alternative UPN suffix to AD domain (as detailed in the “Assign a UPN domain suffix” section of Configure Office 365 for SharePoint hybrid on TechNet) – this effectively allows us to have different users in the directory be matched to different Office 365 tenancies:


Once the UPN suffix has been added in AD, it’s then possible to assign this suffix to individual user accounts – thus “matching” them to a particular Office 365 tenancy and URL domain:


I also tend to put each set of accounts in a different OU so I can easily see accounts for each tenancy.

At this point, you could now implement AAD Sync to synchronize accounts – you would download and install/configure the AAD Sync tool, which is a step that Office 365 administrators will be familiar with:



This will provision FIM and the scheduled task for ongoing sync from AD. In FIM, you could then restrict the sync to a particular tenancy to users you earlier put in a specific OU:


If you want to do this against multiple Office 365 tenancies, one thing to be aware of here is that you’ll need multiple instances of FIM and the sync tool/task (as far as I’m aware). Since none of this sync stuff to dev/test is mission-critical for you, this can be managed pretty easily by just deploying AAD Sync in a couple of different small VMs (i.e. one per tenancy).

At this point, you’d now have users being sync’d from your AD instance to your dev/test Office 365 tenancy. Cool!

And finally, don’t forget that if you want these test users from AD to be able to login and consume Office 365 services, you’ll need to assign them a license in the tenancy.


Implementing directory integration is required in some Office 365 contexts (e.g. hybrid, and where Yammer Enterprise is used), but is useful in many others too. If you’re using multiple Office 365 tenancies to represent non-production environments, it’s a pain when dev/test don’t match the production set-up, especially if you find yourself implementing any custom functionality around user profiles. The approach detailed here provides one option for facilitating directory integration for dev/test Office 365 environments, without some of the infrastructure requirements and headaches that would usually come with this.

This series will continue to discuss options and techniques for improving Office 365 development. Other posts: