Thursday, 16 April 2015

Improving the Office 365 SharePoint hybrid experience

Back in February 2014, I wrote Office 365 SharePoint hybrid – what you DO and DO NOT get. In that article, I discuss the fact that whilst there’s lots of discussion about hybrid and indeed lots of organizations doing *something* in this space – in actual fact, what Microsoft provide is far from a turn-key solution and there’s quite a long list of things you need to work on. That is, if your organization wants to provide any kind of “joined-up” experience that works well for end-users, across sites in the cloud and sites in an on-premises SharePoint environment.

It’s fair to say that at the current time, hybrid from Microsoft really just consists of:

  • Search
  • BCS
  • A model for personal sites (OneDrive for Business sites) to live in the cloud, with some navigation for users, when other things are on-premises

The list of things you don’t get includes things like (N.B. the earlier article has a more complete list):

  • Global navigation
  • A global site directory
  • A joined-up social experience (unless you use Yammer)
  • Any kind of synchronisation of:
    • Branding
    • Customizations
    • Taxonomy/Managed Metadata
    • Content types
    • Apps
    • User profile data

..and so on.

I fully expect SharePoint 2016 to improve on this situation – I think it will be a key focus area for the product. But here in April 2015, having SP2016 in production is still some considerable time away for most.

Common requests for improving hybrid – user profiles, taxonomy and user photos

Since then I’ve worked on several hybrid deployments, and have been involved in work to close the hybrid gap and improve the experience. In this post I want to talk about some of the ground my team and I have covered – what we’ve implemented, and what our experiences have been. From where I stand now, I still believe that hybrid is the most complex option in some ways (for now at least). If you have a choice of simply running all of your SharePoint “things” in Office 365, at a stroke you will reduce complexity, cost and effort! However, this isn’t always possible and indeed, I talk to more and more organizations who are on a roadmap to eliminating on-premises SharePoint in favour of SharePoint Online – but it will take time to get there. So, hybrid definitely makes sense in many cases.

For one client in particular, we focused on solving three core problems:

User profiles

Since hybrid effectively gives each user two completely independent/disconnected user profiles, we wanted to provide one place for a user to edit their profile, with changes replicated to the other. We picked the on-premises SharePoint profile as the “editable master”, since some investment had already been made in integrating data from other on-premises data sources in this case (via BCS) and also custom AD fields. Our job was to replicate the changes to the SharePoint Online user profile (particularly custom/organization-specific fields not sync’d by Azure Active Directory sync – more on this later), so that user data was consistent wherever it was accessed

Taxonomy data

There were two drivers behind the need to sync taxonomy data from the on-premises SP2013 term store to the SharePoint Online term store – tagging of documents (as you’d perhaps expect), but also the fact that several term sets were used in custom user profile properties. Examples would be user profile properties for, say, “Business unit” or “Division” – values for these come from custom term sets to ensure consistency and valid input. In order for us to sync user profile data, this taxonomy data would need to also exist in Office 365, otherwise the profile cannot be saved because it’s pointing to a missing value. In other words, there was a dependency on taxonomy data so that we could sync user profile data.

User photos

On the surface this was simpler than the others – the client didn’t want to allow users to edit their own photos, and had some processes to deal with user photos in existing systems (including resizing, applying a border for consistency etc.). The desire was for the SharePoint Online user profile to also use this photo. If you’re going to bulk sync photos up to SharePoint Online, in reality the best way to do this is to deal with this at the Office 365 level (so that other service elements also show the photo), and the Set-UserPhoto cmdlet is provided in this space. However, we found that there was more complexity here than we expected – more later.

What we implemented – leveraging the OfficeDev Patterns and Practices code

Overall, custom code was needed to bridge these gaps. From the start, we knew that the OfficeDev Patterns and Practices libraries had code which would help, and we designed our solution around some pieces in there. With much of the heavy lifting taken care of, our focus was implementing some of their “engine code” in an enterprise kinda way – something that could run on a schedule, have the right level of resilience, and be monitorable so that any failures could be identified. Helpdesk calls of the “hey, when I go here it doesn’t have the right value for me, but when I go there it does!” and “I can’t tag this document properly” would no doubt occur otherwise – to a certain extent we definitely expect this occasionally, but we want to be able to identify/manage any issues proactively.

High-level architecture

Whilst the various bits of code supplied by the OfficeDev P and P team do their particular job (e.g. help sync taxonomy or user profile data), exactly how they are implemented and where they run from is up to you – options could include:

  • A simple console app, installed to an on-premises server
  • As above, but wrapped in a PowerShell script
  • A cloud option, e.g. WebJobs in Azure (but you might need to do extra work, e.g. with the Data Management Gateway or similar if access to on-premises data is needed)
  • Some combination of the above e.g. middle-tier code in Azure, “client” code or script on-premises

In our case, because all three aspects start with on-premises data we needed the code to talk to on-premises systems – be it the SharePoint 2013 environment, or on-premises fileshares (e.g. for photos). So, we used the “simple console app” option - this gave us the connectivity we needed, but also could be easily scheduled/monitored, and offered a familiar architecture that everyone was comfortable with. In code terms, however, we did separate out the core engine code, so that it could be hosted somewhere else (e.g. Azure) if ever required.

User profiles

The solution here focused on sync’ing custom user profile data from on-premises to SharePoint Online. Of course, Azure Active Directory sync (was DirSync) is used to synchronize some user data from on-premises systems (Active Directory) to Office 365, including the initial creation of profiles, in fact AAD Sync only deals with a core set of fields/attributes! Organizations commonly define *many* custom user profile fields (e.g. division, department, employee ID etc.) – and so a custom solution is currently needed if you want this data imported to Office 365/SharePoint Online. Our approach uses the relatively recent (November 2014) CSOM remote code methods to write the data to user profiles in SharePoint Online (not to Azure AD in this case). I implemented this as a two-step process, where a CSV file is generated of users whose profile has been updated (whether through the user, BCS fields or a sync from AD). The import then picks up this data as a 2nd step.

In fact, because CSOM is used for both reading from the source and writing the target, simply by changing values in a config file (e.g. URLs, credentials) we can actually synchronize profile data between ANY SharePoint environments we have connectivity to – either side could be an on-premises or Office 365 environment. This mean we have a pretty good solution which could be adapted if ever the needs change.

The end result is that custom profile data is synchronised to Office 365, and effectively users “look the same” in both environments - for example in people searches, or where a “created by” or “last modified by” link is clicked, and so on. In other words, the user experience just makes sense. (N.B. the images shown below are of the new profile experience in Office 365 [April 2015]):



Specific fields can be set to "do not import" (e.g. to not conflict with fields updated by AAD Sync), or for any other reason: clip_image004
User accounts can also be "mapped" between the source/target environments - this is useful where * accounts are used/AAD Sync and identity integration isn't implemented (e.g. test environments):


Any users who fail to import for some reason will automatically be re-tried (even if their profile hasn't changed since the last attempt). Information on which users are failing, how many times they've failed, and the last failure reason is maintained as the import jobs run:


Since they run on an ongoing basis, the tools are designed to be monitorable. They write a summary to the Windows Event Log, ready for SCOM or another monitoring tool to pick up and send e-mail notifications of any failures:


Further to this, Log4Net is implemented to provide rich logging to help troubleshoot any problems:


Taxonomy data (e.g. term sets for tagging)

Taxonomy sync can synchronize most types of changes (new term sets, new terms, renames etc.) from source to target. The heavy lifting is actually done by Microsoft code (the OfficeDev Patterns & Practices Core.MMSSync solution).

The end result is that any term set created in the source environment (on-premises SP2013 in this case) is replicated to the target environment (Office 365):

This allows documents to be tagged in a consistent way across online/on-premises sites, and also user profile fields which are bound to term sets to work properly in both environments.


Synchronization of user photos

Our mechanism here was mainly a PowerShell script which used the Set-UserPhoto PowerShell cmdlet (which is actually an Exchange Online cmdlet – and requires the user to have an Exchange Online mailbox in Office 365), but integrates with some existing photo-handling processes. However, it was decided to change this approach entirely in the end, due to some issues we hit – more on this in the next section.

Challenges we encountered

For the most part, the development of these tools went fine – however, we definitely did encounter a couple of road-bumps along the way. These were things like:

  • Typical client infrastructure issues – tools would run fine in our environments, but would be less reliable communicating with the cloud when running on more locked down servers and running through additional network infrastructure (e.g. reverse proxies).
    • In the end, this proved a significant issue for this client (but perhaps somewhat specific to their case). The user photo PowerShell script in particular would fall over with authentication problems when the network connection was less than perfect – and it was hard to “engineer in” absolute reliability here. In the end, it was decided a simpler approach would be to change policy to allow users to edit their own photo (in SharePoint Online) :)
  • Some taxonomy sync challenges
    • Although the P and P code is pretty resilient here, we did hit the occasional error (most likely related to connectivity/the previous point). For the most part, these would get resolved on the next run however.
  • Throttling and batch issues with remote code
    • During the development process, clearer guidance emerged on how to handle throttling in Office 365 code. This was useful, but we also found that some bits of the OfficeDev Patterns and Practices internal code caused problems in the way we were using it (even though the later versions implemented the throttling/retry pattern). There was nothing invalid about the PnP code *or* the calls we were making – it was more that some tweaking was required to be able to successfully process everything we needed to.


Ultimately we were able to implement our sync tools so that the end-user’s experience of hybrid SharePoint is improved. I think that any form of cloud development will always encounter occasional errors in communication, and at enterprise scale it’s usually a good idea to build in some kind of tolerance to this (e.g. our re-try mechanism for failed user profile updates). The OfficeDev Patterns and Practices code helped us out a ton – there’s no doubt that closing these gaps would have required WAY more effort without it.

As mentioned earlier, I’d expect SharePoint 2016 to improve on the hybrid situation natively – but this work wasn’t a huge effort to make big steps towards this right now.

Monday, 6 April 2015

Speaking at Ignite and SharePoint Evolutions 2015 Conferences

As you might know if you’re in the SharePoint/Office 365 space, there’s a couple of big conferences around the corner – I’ll be delivering sessions at Microsoft’s Ignite conference in Chicago in May, and also the most excellent SharePoint Evolutions conference before then in April, here in London. It always takes quite a lot of preparation to pull new talks together, but after the conference I’ll be looking forward to writing in more detail about some of the topics I’ve been exploring. Before then, I think both conferences are going to be pretty special, and if you’re going to one of them (or Build, in San Francisco in April) I think you’ll have a great time and learn a lot. Here are the details of my sessions (N.B. there were some updates to these summaries which hadn’t made it to the websites at the time of writing):


MS_Ignite_400pxDealing with Application Lifecycle Management in Office 365 app development 

Thursday, May 7th, 10:45AM - 12:00AM

For teams who are undertaking cloud-friendly SharePoint or Office 365 development, apps are likely to be a key area of focus - be they Apps for SharePoint or newer Office 365/Azure AD apps. One of the benefits of stepping outside the traditional SharePoint development model is that aspects such as ALM and Continuous Integration become easier - finally, an end to "development is harder because it's SharePoint!". In this session we'll talk about how to do app development the "right" way - with a focus on ASP.NET as the framework and Azure as the hosting platform. We'll cover good practices such as implementing automated builds, take a close look at “continuous deployment” for apps, and also discuss the upgrade/push-to-live process for existing applications. Over several demos, we'll show how Visual Studio Online and Azure are a winning combination, and see how concepts such as "Deployment Slots" in Azure can help.

Transforming Your SharePoint Full Trust Code to the Office App Model (panel discussion)

Tuesday, May 5th - 5:00PM - 6:15PM
Panel – Vesa Juvonen, Frank Marasco, Bob German, Erwin van Hunen, Chris O’Brien

This session is a panel discussion covering examples and patterns for moving SharePoint customizations to the cloud app model - for use either in Office 365 or "cloud-friendly" implementations of on-premises SharePoint. The panel comprises members of the Microsoft OfficeDev Patterns and Practices team and independent MVPs. Both bring their experiences from the field, but different perspectives too. Since the landscape changes when developers must leave behind server-side SharePoint API code and feature framework elements, the discussion will centre around 5 related hot topics - branding, options around remote code (including .NET, JavaScript and PowerShell), provisioning of customized sites, the story with sandbox solutions and finally how the Office 365 APIs fit in with the app model. We promise a lively discussion, examples of code from the field, and time for Q&A!


SharePoint Evolutions


Dealing with Application Lifecycle Management in Office 365 app development

Same as Ignite session above

New developer choices - comparing Apps for SharePoint with external Office 365 apps

No sooner have we all learnt how to develop Apps for SharePoint, when Microsoft go and release a new development model. Don’t you just love it when they do that?! In an "external" or "standalone" app, there's no need for the app to be installed to a SharePoint site - and this can provide huge benefits. However, new APIs must be used and the developer must understand how Azure AD fits in. Whilst this model is currently Office 365 only, Microsoft have hinted that this approach may come to on-premises SharePoint soon - so this will become important to all developers. This session is designed to get you up to speed.

Closing notes

I also plan to publish videos of the demos from these sessions. There have been quite a few changes in Azure recently, and my ALM session covers some of the cool things such as Deployment Slots, with a hat tip to other related capabilities such as the “Traffic Routing” feature which can be used for A/B testing and the like.

I’m really looking forward to the panel discussion at Ignite too. Like the other panel members I’ve definitely got some opinions on the topics being discussed :)

Anyway, hope to see you at one of the conferences. Please come and say hello if you’re there!

Friday, 27 March 2015

Key skills and topics for today’s SharePoint/Office 365 developer

Chris OBrien SharePoint Office 365 skills topics 2 As I prepare to talk at the upcoming Ignite and SharePoint Evolutions conferences (will post details soon), I was reflecting on the technology landscape I’m currently working in.  As I’m not the first to mention, for many SharePoint developers the world is a different place now to how it’s been in previous years. If you work with the cloud in some form (Office 365, Azure, Yammer or other platforms) then you’ll probably be nodding your head at that statement. If not, then maybe the wave hasn’t hit you yet but could do in the next 12 months. Either way, the level of change seems to have ramped-up from mere evolution to full-on revolution.

I often maintain a list of skills and topics which I think are relevant to me and the kind of projects I’m involved in. I do this partly for my development and that of my team around me at Content and Code, but for other reasons too. As someone involved heavily in recruitment of technical people, some of this powers my list of current interview questions and what I look for in candidates. As such, I always thought publishing it would probably be a bad idea, because:

  • The list is ever-changing – at the very least, it evolves every few months
  • Recruiters (and candidates) would take advantage of this info

Call me cynical, but I know that in the UK market at least, recruiters do seek to boost the chances of their guy getting an interview by ensuring CVs are targeted to what is being looked for. Nothing wrong with that if everything stays honest - but I’ve seen quite a lot of cases where the truth is stretched too far. Fortunately not all recruiters use such tactics of course, but it can make the job of identifying good developers somewhat harder. Still, I found myself writing my skills/topics list on a whiteboard in a meeting the other day, and the idea of jotting it down somewhere public came to mind again. I now figure “Ah, what the hell – if a candidate actually *does* know all this stuff, then however they gained the knowledge I probably want to work with him/her anyway!” After all, seeing the list isn’t the same as having genuinely deep knowledge in each area. I’m not sure what other technical hirers do but, like the Microsoft exams, I personally take the ‘adaptive’ questioning approach – if the candidate appears to know a certain topic, I’ll do my best to keep asking questions to see just how deep the knowledge goes. I certainly don’t claim to be the world’s best developer, interviewer or anything else for that matter (and not even on my “strongest” topics!), but I often find I *can* establish where the limits of the candidate are for a given area.

Key skills/topics

So, anyway here’s my list – it’s just a dump from my OneNote notes rather than anything particularly structured. And there’s no special sequence or anything:

  • Remote SharePoint/SharePoint Online development techniques
    • .NET CSOM and JSOM
    • Using CSOM in PowerShell
    • The REST API
    • The App Model
      • Apps for SharePoint/Apps for Office
        • Provider-hosted vs. SharePoint-hosted
        • App authentication – context tokens/access tokens, app-only policy etc.
        • Infrastructure requirements
      • Office 365 apps
        • App registration – and the App Launcher/My Apps page
        • App authentication – ADAL and adal.js, the Common Consent framework, options for app-only tokens via a certificate
        • Office 365 APIs and client libraries
      • Deciding between Office 365 apps and Apps for SharePoint
      • Deploying apps to Azure (both provider-hosted Apps for SharePoint/Office and Office 365 apps
        • Protecting the app with Azure AD authentication
    • OfficeDev
      • The Patterns and Practices core libraries
      • The Patterns and Practices samples
    • Office 365
      • Identity management options
      • AAD Sync
      • Hybrid
    • Office 365 and Azure
      • Azure AD
      • How the "special" restricted Azure subscription behind an Office 365 tenancy works (i.e. just AAD)
  • General SharePoint/SharePoint Online development
    • Key building blocks for out-of-the-box solutions
    • Search-based solutions - Content Search web part, search APIs etc.
    • Managed Properties/Crawled Properties - provisioning (including differences in SharePoint Online), auto-created properties, using in search refiners etc.
    • Display templates - provisioning templates, the end-to-end process of having values from a custom column displayed in a display template (via a custom Managed Property)
    • JSLink
    • Considerations around use of custom master pages, web templates etc.
    • Branding options – Office 365 theming, alternate CSS, custom master pages etc.
  • Azure
    • Azure Web Apps (was Azure Websites) – development considerations, scaling models, deployment slots etc.
    • Azure Web Jobs
    • Azure SQL Database
  • General web development
    • Responsive design - level
    • JavaScript
      • jQuery (DOM manipulation, AJAX methods etc.)
      • Patterns for OO JavaScript
      • Promises
      • Cross-domain issues and options
      • Knockout.js
      • Angular.js
    • ASP.NET
      • MVC
      • WebAPI - especially:
        • Calling from JavaScript
        • Dealing with incoming/outgoing JSON
        • Implementing a REST service
        • Implementing CORS etc.
    • Token-based authentication models
    • Developing for performance - bundling and minification, page weight issues, useful tools etc.
  • Yammer
    • Yammer concepts - networks, groups, topics etc.
    • Using Yammer Embed
    • User sync (i.e. Yammer DSync)

No doubt this isn’t comprehensive and there are lots of items that could be added to this. Feel free to leave a comment if you think I’m missing something big :)

Summary – reflecting on this list

What’s interesting about this to me, is that if I’d written a similar list a couple of years ago it would be full of things like “web parts, timer jobs, event receivers, Feature XML” and lots of other SharePoint-only constructs. Some of these things can still be very relevant depending on the type of work you’re doing of course - but in general there are so many new concepts for most SharePoint developers in the last couple of years. Key themes here include general web development (i.e. non-SharePoint specific), new platform aspects such as Office 365 and Yammer, and Azure and Azure Active Directory (AAD) deserve a special mention since they become extremely important in the new world. For example, for Office 365 I highly recommend that even developers get their hands dirty with some infrastructure/platform aspects such as implementing domain integration and perhaps AAD Sync. After all, the production environment you’ll be targeting will probably use this configuration! Doing this in my dev environment certainly helped solidify my understanding of a few things – for example, what controls the attributes which are sync’d between the on-premises AD and Azure AD:

AAD Sync chris obrien com

One thing is for sure – things have changed irreversably for SharePoint developers, and moving forward I think these types of changes will apply more and more to “on-premises only” SharePoint developers too. More details on SharePoint 2016 will be announced at Ignite very soon, and whilst we already know the full-trust/farm solutions model will live on, I think various aspects of cloud app development are likely to come into that world too. Exciting times my friends!

Thursday, 26 February 2015

Podcast interview: customizations and Office 365 – Microsoft Cloud Show

I was interviewed last week for the Microsoft Cloud Show by Andrew Connell and Chris Johnson. If you need something to listen to whilst you're working/travelling/exercising, I think it was a fun chat about lots of Office 365/SharePoint things! I’m a regular listener of the show (perfect for my morning runs!), so it was cool to be a guest and catch up with those guys.

The overall show lasts for just over an hour - like all their shows there's a segment at the beginning where AC and Chris talk about the week's news (something they do really well IMHO), so the interview itself starts after around 20 minutes. We covered things like:

  • The recent changes in Microsoft guidance for Office 365 development (e.g. avoid custom master pages, avoid WebTemplates, avoid use of Feature XML etc.) - this was the main topic of the interview
  • Microsoft now releasing SharePoint updates via Windows Update
  • Overall changes in Microsoft strategy (i.e. the new "openness" discussed in another Yammer thread recently)

The podcast episode can be listened/downloaded from:

Chris OBrien MS Cloud Show interview

Sunday, 4 January 2015

Developing for Office 365 – thoughts on use of custom master pages and web templates etc.

As more and more organizations are choosing to deploy SharePoint workloads on Office 365, it’s interesting to see Microsoft change the guidance given to developers on how customizations should be implemented. If you’ve been following closely, you might be aware that the following messages have started to emerge in recent months:

  • Avoid custom master pages
  • Avoid using web templates
  • Avoid using declarative XML (i.e. Feature XML) in a WSP (even a sandbox WSP – and remember, we already should not be using server-side code here either!)

These messages are mainly relevant to the Office 365/SharePoint Online space – but arguably have some relevance for on-premises SharePoint too. At the time of writing (Christmas 2014), these messages aren’t really on MSDN yet but will very soon be published in Microsoft Virtual Academy training. I have a lot of respect for the guys in Microsoft behind this thinking. I’ve had some great conversations with Vesa Juvonen, Steve Walker, Bert Jansen and so on about these topics, and these guys ultimately have to battle slightly opposing forces - “making Office 365 a good development platform WITHOUT making the service too difficult/expensive to run” would be a summary of that. Hopefully this doesn’t get me crossed off their Christmas card list (!), but in this post I’ll explain why I’m not sure that I agree with the guidance in all cases.

After all, these are BIG changes!! No custom master pages? No web templates? Really?? Web templates were the answer to world hunger not that long ago – after all, if you’ve got several hundreds/thousands of team or project sites to create, and they all should have some tools/widgets above the out-of-the-box team site, web templates are the tool in SharePoint way that Microsoft provided to do this! And custom master pages have long been the way to implement a reasonable level of branding in SharePoint – and that’s important to many organizations. So what is behind this guidance? Do you need to follow it? For our projects, I’m taking the view that we SHOULD be following it for collaboration/team site-based stuff compared, but NOT necessarily for highly-branded publishing intranets which are more communications-oriented. So to summarize this decision process:

Custom master pages and web templates - collab vs publishing - small

I’ll explain my thinking later, but first let’s understand why Microsoft are giving this guidance.

Using custom master pages/web templates means you might not *automatically* get new Office 365 functionality

As we all know, Office 365 is an “evergreen” platform – it is regularly updated with new functionality for end-users (the “ripples, not waves” approach). To roll out these updates, Microsoft sometimes need to update out-of-the-box site templates and master pages. Some examples of this could be:

  • New functionality added to SharePoint Online team sites (e.g. new lists or settings used) = Microsoft update to team site template (STS#0)
  • New functionality added to SharePoint Online via new page controls = Microsoft update to OOTB master pages

Of course, if you’re using a custom web template and master page, you won’t receive these kind of changes.  That’s a pain because ultimately you WANT this stuff in many cases – it could be some high-value functionality that your end-users would really benefit from (and perhaps that you were thinking of building a customization for!). And indeed, you’re paying for this “service” in your Office 365 licenses; it seems a shame not to get the full benefits. As time goes by, it’s also true to say that your custom master page will become more and more out-of-sync with OOTB master pages, as depicted by this diagram from Microsoft training:

Custom master page vs OOTB

Of course, what you could do is manually copy over any master page updates to your custom master page(s) for example – but this comes with effort each time. Some level of analysis, code changes, testing, deployment through multiple environments and so on – all these tasks are needed for each Microsoft update. Microsoft refer to this as the “customization tax” that you must pay if you do use custom web templates/master pages in Office 365 – and I think that’s a good description.

The flip side – having control of updates

So, some extra effort is required if we do decide to use custom web templates or master pages. But on the flip side, it could give you more control over when and how Office 365 updates are applied to your environment, and this can be appealing. Certainly if you have 50,000 users on your Office 365 global intranet and a Microsoft update breaks the home page one day (and such “breaking” changes have occurred), that’s a lot of folks calling the helpdesk and a very unhappy business! That’s probably the worst case scenario here, and it’s true to say that Microsoft have improved communications around updates – but even less dramatic scenarios such as changes which don’t break anything but do change an intranet experience need to be managed properly. Compared to team/project sites, publishing intranets tend to have more “designed” user experiences, with custom branding/look and feel and potentially more customizations. And this means a greater chance of a Microsoft update causing an issue, perhaps some kind of conflict with a minor customization – usually you’d want the opportunity to test these things together before applying in production.

If nothing else, perhaps using a custom web template/master page is one layer of protection against uncontrolled updates for publishing intranets. It won’t cover all scenarios (depending on the type of update), but certainly will cover many. However, I do strongly feel that collaboration scenarios such as team sites and project sites generally should NOT use custom web templates/master pages. There’s way less benefit to heavy branding/look and feel customization in these areas (so less need for a custom master page), and I think you really want any updates which come from Microsoft here – it could be valuable new functionality that your business users will appreciate. Additionally, since you often have many collaboration site collections (but just one publishing intranet site collection), the idea of manually copying any updates from Microsoft to these areas has more complexity too. So, my starting position is a custom web template/master page for publishing scenarios but not for collaboration scenarios – obviously each implementation may then have further variables to consider, but that’s my default thinking.

Other layers of protection

As a slight aside, other things you could do around Microsoft updates to Office 365 include:

  • Close monitoring of Microsoft communications around updates – particularly:
  • Running other Office 365 tenancies for test purposes with “First Release” mode enabled
  • Signing up to the NDA Preview program if possible

And just how much of a burden is the “customization tax” anyway??

For many organizations rolling out Office 365 (especially large enterprises), I can’t help but think this “customization tax” is often a drop in the ocean in many ways. Most of the clients I work with already have a team (or teams) handling other roll-out or customization tasks, and they know that it takes much more than switching the thing on and sending a couple of e-mails to get the most of the platform. Having to occasionally have a developer make some updates is perfectly fine in the overall running of the intranet.

All these factors feed into my thinking about using a different approach for collab vs. publishing intranet scenarios.

Alternative approaches to custom web templates and master pages

But let’s say you don’t want to use a custom web template or master page or both) - perhaps because you’re doing team sites, or for other reasons.  How can you get the same effect (and what do Microsoft suggest instead)?

If you need…

“Classic” approach

Alternative approach

To apply custom branding/look and feel
  • Custom master page
  • Use a Composed Look/theme/alternate CSS file for look and feel
  • JavaScript injection to add custom page components/controls
To have a template to create many sites from
  • Custom web template
  • Site is based on an OOTB site type (e.g. team site) and..
  • ..use Remote provisioning code to configure content types, lists/libraries and so on after site creation. The two key approaches here are:

Obviously the alternative approaches need some thought during your implementation, and extra effort is usually involved – but you should find that they at least work well for non-publishing scenarios (e.g. team/collab sites), thus aligning somewhat with my thoughts of treating these implementations differently. You might find though, that NOT using a custom master page for a highly-bespoke intranet would be difficult anyway, particularly if aspects such as responsive design are involved. On the templating side, the steer from Microsoft is to use remote code approaches to apply customizations to out-of-the-box sites. This goes back to the other point in my opening bullet point list, which we haven’t discussed yet:

Avoid using declarative XML for provisioning (use imperative/code-based approaches instead)

So yes, whilst many of us have spent the last few years using Feature XML for provisioning, Microsoft would now prefer us NOT to use this approach in Office 365 even if it is still 100% supported. Certainly the “XML vs. code for provisioning” debate has been around for many years now, but Microsoft are now coming down on the code side heavily – and in SharePoint Online this means remote code (CSOM or JSOM). It’s interesting to pause and consider the types of things we’d commonly use XML for here:

  • Provisioning fields and content types
  • Provisioning lists and libraries
  • Provisioning files (either “site infrastructure” files, such as master pages/page layouts/CSS/JavaScript/images etc., or default content such as pages or documents)
  • Provisioning web parts
  • Declaring templates – list templates and web templates (and site definitions before that)
  • Adding a CustomAction (e.g. to link a JavaScript file, or to add a ribbon customization)
  • Custom field controls
  • Delegate controls
  • Declaring event receivers
  • Feature receivers

Only a couple of those (e.g. Feature receivers, custom field controls, delegate controls) can’t be used at all in the sandbox (and therefore SharePoint Online), but you can achieve the same thing in other ways.  The rest can, but Microsoft are starting to say we should use remote code where possible instead of XML. The reason for this is that inside SharePoint, the functionality becomes dependent on an XML file somewhere instead of purely being within “provisioned” items in the content database – and this could make some kinds of system upgrade more complex. Whilst this is an interesting SharePoint engineering “thing”, it’s somewhat hard to see this as our problem rather than the Office 365 engineering team’s – after all, these upgrades are mainly things that Microsoft will undertake as part of running SharePoint Online. The “content database upgrade” step as part of, say, upgrading Office 365 to use SharePoint 2015 (or whatever) would be an example. 

As someone who leads a group of 20 developers working in many SharePoint/Office 365 engagements at any one time, I currently have some reservations about the team switching entirely to imperative/c0de-based provisioning for all scenarios. I’ve mentioned this to Steve/Vesa etc. in the past – in the spirit of open discussion, here’s a slightly expanded list of my thoughts on the “don’t use declarative XML” message from that conversation:

Some quick thoughts from me are:

  • Some aspects of switching to imperative provisioning are pretty inconvenient - my team (and SharePoint devs generally) are very efficient at declarative provisioning (~7 years experience ;)), and lots of the resources on the internet are heavily-oriented towards this.
  • The declarative/XML approach has been pushed by Microsoft for 7 years and even now is still 100% supported in Office 365.
  • I completely get that if all customers used imperative provisioning rather than declarative, certain things/upgrades would be simpler for Office 365 engineering. But frankly, I'm not sure I'm on board yet that this is my problem too! I know that in some places it *can* be very relevant to me as the initial implementor (e.g. initial deployment can be done declaratively, but changes/updates require code), but even with this in mind, often we *still* prefer declarative for initial provisioning!
  • Imperative provisioning obviously doesn't cover some templating scenarios easily - list templates, site templates etc. Sure, I can implement some remote provisioning solution which does a similar thing, but even with the OfficeDev samples I’d say it's more effort than the declarative approach - AND, sometimes there's no easy way to have things appear in the expected place in the end-user UI (especially for list templates). Big gap here really.
  • Providing a production-grade, resilient “remote code hosting” solution can be difficult for some clients (e.g. those not comfortable with Azure for this). 

It will be interesting to see how this area plays out in time. These are just my opinions obviously, and it’s certainly true that using code rather than XML can bring advantages (such as better debugging and logging). Still, it seems a shame to no longer have both options. Ultimately this sub-decision might now only be a consideration for publishing intranets and the like, if you’re happy to go with the thinking that collaboration-oriented sites should indeed use OOTB site types with remote provisioning code (in other words, you’ve already decided to use code here). But I’m certainly reluctant to abandon the declarative approach across the board just yet.


There’s quite a lot to think about with the new guidance, and what’s right in one situation may not be right in another. It’s up to you to consider the factors (alongside what you need to achieve), but as usual understanding the landscape as much as possible helps you make the best decision. My thoughts are to consider publishing intranets and collaboration sites differently – but hopefully this post at least helps you with some detail around what to think about!

Tuesday, 16 December 2014

Office 365 apps – deciding between the SharePoint app model and the Office 365 APIs

Just like me, many other SharePoint developers have spent the last year or two getting to grips with the SharePoint app model – this was introduced in SharePoint 2013 as a way of building customizations which do NOT run on the SharePoint server. Enhancements built this way can run on a remote machine (e.g. Azure, IIS, whatever) and therefore any stability/performance issues do not affect core SharePoint. If you’re using Office 365 of course, the app model is a big cornerstone of extending the platform, since Microsoft do not allow custom SharePoint code to be deployed to the Office 365 servers (server-side code at least). These were big changes in the SharePoint development world, with lots to learn around authentication, remote APIs and patterns for solving common needs.

However, there’s a new kid in town. Microsoft have introduced a series of new APIs for Office 365 which span SharePoint, Exchange and Lync – and rather than needing to know each platform as a developer, more general concepts are exposed such as:

  • Mail
  • Calendar events
  • Contacts
  • Files

Of course, those who know the territory will recognize that “mail”, “calendars” and “contacts” are related to Exchange and that “files” are related to SharePoint (N.B. right now this is primarily about files stored in the user’s OneDrive for Business area). But the point is that not everyone who wants to work with Office 365 data these days has that background. Initially this new direction can be confusing to the experienced developer – after all, can’t I access SharePoint files using the existing SharePoint CSOM or REST APIs? Also, why is authentication different? I just spent months learning about provider-hosted apps and the “new” authentication models with context tokens, access tokens and so on! Why did we need something new?  If this is a new way of building apps for Office 365, is the old model going away? These are all logical questions in my opinion, so I wanted to talk about them here, as well as discuss some things to look out for in your apps which suggest you SHOULD be using the new approach. But for sure, if you’re about to start building an app you might be faced with this decision (with a gratuitous graphic to illustrate):

COB - provider-hosted SP app vs Office 365 app

After all, there are many similarities between the two – in both cases, the core implementation could be ASP.NET MVC, on Azure or some IIS servers for example. Let’s start by understanding what an Office 365 external app is about.

What an Office 365 app looks like

Here are some key tenets of these apps:

  • The application is an “external” or “standalone” app of some kind, perhaps implemented in ASP.NET (most likely MVC flavor these days). And when I say website, remember that it could really be something else altogether, like an App for Office which surfaces in Word or Outlook (since these use web pages underneath). Equally it could also be a Windows Store/Phone app or one which targets other devices, such as an iOS/Android app
  • We want to somehow integrate the app with Office 365 (perhaps in terms of user experience/navigation, data storage or authentication)
  • The app is registered in Azure Active Directory – specifically the Azure AD behind your Office 365 tenant
  • Apps can implement the same authentication mechanism as Office 365. This means your users get single sign-on, and can pass between Office 365 and your app without re-entering their username/password
  • Users can navigate to the app via the Office 365 App Launcher or “My apps” page
  • In coding terms, the apps are likely to use the new Office 365 APIs to access data there, but can also revert to using lower-level APIs such as the SharePoint CSOM/REST APIs
    • N.B. To do this, you use the authentication token obtained from Office 365 as the “bearer” token in the HTTP headers for the request to the other API:

Developing with the Office 365 APIs

If I was to summarize the development model, I’d say there are two high-level approaches:

  • Using the Office 365 REST API (more operations, but more complex since you need to craft the HTTP requests yourself and deal with authentication etc.)
  • Using a flavor of the Office 365 client libraries (simpler to use, but don’t have 100% coverage of what’s possible). These flavors include:
    • .NET (for use in ASP.NET websites, Windows Store/Windows Phone apps, Xamarin apps etc.)
    • iOS
    • Android
    • Cordova apps (multi-device hybrid apps)

Key differences between Office 365 apps and Apps for SharePoint/Apps for Office

But what if your app is SharePoint/Office focused, and could therefore be implemented as EITHER an Office 365 app or an App for SharePoint/Office? Let’s consider that decision for a while - here are some of the differences as I see them (N.B. I’m also summarizing these differences in a table below):

Difference 1 - how the app is accessed (by end-users)

Office 365 app:

An Office 365 app can be accessed in a few ways:

  • From any link (e.g. on your intranet home page), since it’s a standalone website
  • From the Office 365 App Launcher (if the app is “pinned” there for quick access)
  • From the Office 365 “My apps” page

App Launcher experience:

COB app pinned to app launcher

“My apps” page experience:

COB My apps page

App for SharePoint:

An App for SharePoint can be accessed in the following ways:

  • From the Site Contents page of a SharePoint site where the app is installed
  • From a developer-provided SharePoint link, using AppRedirect.aspx?instance_id=[app instance GUID here]

In other words, these apps can only be accessed from a SharePoint/SharePoint Online site – assuming that is, that the app accesses some SharePoint data (as most would). This is because the authentication model relies on the context token being passed from SharePoint into the app – otherwise the app cannot talk to SharePoint in any way.

Site Contents page experience:

App deployed to target site

Difference 2 – app registration (for implementer)

Office 365 app:

Firstly, these apps must be registered in the Azure AD which relates to the Office 365 tenancy:

COB Azure AD COB Azure AD - applications

Note that Visual Studio (more specifically, the Office 365 dev tools) perform this step for you when you create an app for a single Office 365 tenancy. Azure AD administrators can also register apps, and this is what needs to happen for multi-tenant apps.

App for SharePoint:

These apps are registered in SharePoint/Office 365 using AppRegNew.aspx (or using the Office Seller Dashboard for Store apps):


Difference 3 - how authentication and authorization works e.g. for SharePoint data (for implementer)

Office 365 app:

Once the app is registered in Azure AD, you can write code using the Office 365 client libraries to authenticate to the platform (simpler), or alternatively go lower level and obtain tokens yourself through OWIN (for MVC) or ADAL for .NET (both more complex, but what is needed when you’re coding directly against the Office 365 REST APIs). In terms of the client libraries approach, here’s a snippet showing how auth works in the .NET Office 365 library (taken from the Office 365 Starter Project for ASP.NET MVC sample):

You can cache these access tokens as you see fit to avoid repeated auth prompts. But if a new token needs to be obtained, this code does the work of redirecting the user to sign-in to Office 365 (whether they are in a web browser or on a device):

COB Office 365 sign-in - web


App for SharePoint:

Authentication for an App for SharePoint works in the following ways:

  • SharePoint-hosted app: authentication happens implicitly, since web pages are hosted in SharePoint context (and only the SharePoint JSOM API can be used)
  • Provider-hosted app: app must be registered with AppRegNew.aspx (or the Office Store), and .NET code can authenticate using the SharePointContext and TokenHelper classes which are added to your Visual Studio web project on creation:

Difference 4 – app deployment (for implementer)

Office 365 app:

Whatever is needed to to copy the app to the hosting location e.g. publish to Azure (from Visual Studio – shown below) or FTP or similar:

Publish web app - validate settings  

App for SharePoint:

Similar to above in that app files must be published to the hosting location, but with two extra steps:

  1. The app package (.app) must also be deployed to an App Catalog:

    Publish app - upload to app catalog
  2. Once the app is in the App Catalog, it must be installed to any SharePoint site which should use the app (or, a tenant-scoped installation should be performed):

     App install - in site

Summary table of differences/commonalities

Since the above sections are a bit difficult to cross-reference, let me try to summarize in a table:

Office 365 app

App for SharePoint

How the app is accessed (for end-user)
The App Launcher/My Apps page

Some other link (e.g. from some other intranet site you have) or bookmark – it’s a standalone website
Typically from the Site Contents page of a SharePoint site
Authentication to Office 365/SharePoint (for implementer) App is registered in Azure AD, then authentication helper in Office 365 client libraries typically used (e.g. AuthenticationContext object in .NET Office 365 SDK) App authentication – context token is passed from site into the app, and access token obtained from there. Helper code supplied in form of SharePointContext and TokenHelper classes (added automatically to app projects)
App hosting (for implementer) Any web platform – Azure or other cloud-hosting, on-premises IIS, or other web servers Any web platform – Azure or other cloud-hosting, on-premises IIS, or other web servers
App registration (for implementer) Registered in Azure AD Registered in SharePoint/Office 365 using AppRegNew.aspx (or using the Office Seller Dashboard for Store apps)
App deployment (for implementer) Deployed once to the hosting platform (e.g. Azure/IIS) Deployed to a SharePoint/Office 365 App Catalog, and then installed by site owners to various SharePoint sites or tenant-scoped install.

Alternatively, submitted to the Office Store for sale there.

Other differences

There are some wider differences not covered above too. Don’t forget things like:

  • For Office 365 apps, I no longer need knowledge of Exchange and SharePoint APIs to pull together an app which (for example) stores some files in an area I can access (i.e. my OneDrive) and sends an e-mail on my behalf. In other words, I don’t need to be a “career developer” in Exchange or SharePoint to make use of this stuff
  • Office 365 apps are probably easier to develop for devices – having SDKs for iOS and Android means that some common dev tasks are pretty simple on those platforms. I’d say the fact that these apps don’t need to be accessed from a SharePoint site can help here too

Summary - reasons to consider the Office 365 approach

So, there are clearly some things to consider if you’re building an app which could work as either a provider-hosted SharePoint app OR an Office 365 app. Here’s my summary list of indicators that the Office 365 app could be a good fit:

  • Your app doesn’t need to work with on-premises SharePoint/Exchange
  • Your app doesn’t really “fit” in SharePoint:
    • It’s an external app, not associated with a “site”
    • It sits at the “Office 365 suite” level
  • Your requirements match files/messages/events etc.
  • You want to integrate with the Office 365 App Launcher
  • You don’t want your app to need to be installed into SharePoint sites by site owners (and a tenant-scoped SharePoint app doesn’t suit your requirements)
  • You’re developing for devices and want to take advantage of device-specific SDKs
  • It’s the future!

In short, there will always be cases where either an Office 365 app or an App for SharePoint is a better fit – both models are valid going forward. The key is understanding the differences, both to the end-user and the implementer.

Feel free to leave a comment if you think I’m missing anything, or have any thoughts related to this!

Sunday, 16 November 2014

More speaking – SharePoint Connect 2014 (Amsterdam) and SharePoint Saturday UK

I’ll be doing a couple more talks in the next few weeks – these will cover different topics, ranging from the new Office 365 APIs to techniques for cloud-friendly SharePoint development, and also Apps for Office. Hopefully see you at one of these events!

SharePoint Connect 2014 (Amsterdam)

  • Modern SharePoint development - techniques for moving code off servers (Tuesday November 18, 14:50)
    • Those implementing Office 365 know that custom code cannot run on the SharePoint servers, and remote code alternatives must be used. However, it can be valuable to adopt the same "cloud-friendly" development techniques even for on-premises implementations. In addition to increasing platform stability, this enables you to build in flexibility to your deployment - leaving the door open to a potential move to Office 365 in the future without major re-engineering. With several code-focused demos, this session examines approaches for switching to "off-box" code - including Remote Event Receivers, PowerShell and CSOM scripts and client-side JavaScript.
  • An intro to the new Office 365 APIs (Wednesday November 19, 11:15)
    • Microsoft have stated that they will move away from SharePoint, Exchange and Lync APIs towards generic Office 365 APIs. The preview version of the first APIs were announced in March 2014, and cover several core building blocks for user collaboration - including mail, OneDrive files, calendar entries and contacts. Developers who need awareness of Office 365/hybrid scenarios can expect more such APIs in the future.

      Get on the bus early - come learn about the APIs and the new permissions model ("the Common Consent framework") in this code-focused session.

SharePoint Saturday UK (Hinckley, UK)

  • Apps for Office - the hidden gem that's easier than you think (Saturday November 28, 12:15)
    • Most developers know that it's now possible to develop Apps for Office which are based on core web technologies - HTML, CSS and JavaScript. But the take-up appears to be fairly limited - why is that when development and deployment (even in the enterprise) are fairly easy? This session takes a renewed look at Apps for Office, now that the Word, Excel and Mail APIs have been enhanced and the app model is no longer the weird new kid on the block. Some cool demos are involved, including a Word task pane app which analyses the contents of a document to find you similar items across your SharePoint environment/tenancy whilst you're working with the document.

Sunday, 9 November 2014

Slides/video from my Apps for Office talk at TechEd 2014 now available

I presented at TechEd 2014 last week (Barcelona) on “Weave your enterprise solutions directly into Office products with the app model”, alongside Andrew Salamatov from the Exchange product group - in this post I want to provide the PowerPoint deck and link to the TechEd video, but I also to provide some thoughts on Apps for Office in general. First off, here is the slide deck:


Link to TechEd video

The TechEd video can be watched in the embedded video at the bottom of this post, or the link to the video over on Channel 9 is - 

General information

If you’ve read other posts of mine, you’ll know that I’ve spent a lot of time developing Apps for SharePoint and Office 365 recently and doing “cloud-friendly” SharePoint development in general – but (like many people I suspect) I hadn’t spent too much time around Apps for Office. Having spent the last couple of months preparing to talk on the subject at TechEd last week, I feel like my eyes have been opened a bit here – Apps for Office have HUGE potential, and I feel you could easily become a hero in your client or organization with the right idea and a bit of dev effort!

The big game-changer is the development model for how Office is extended these days:



Deployment to users

Some weird VSTO/VSTA stuff that I never did :) Write code in C# against slightly strange APIs specific to Word, Excel etc. Have to get a DLL deployed to each user’s machine – good luck with that
Apps for Office

Web page hosted in client app (e.g. Word) – developer writes HTML, CSS and JavaScript.

JavaScript API for interacting with document. Many methods work across all client apps (e.g. Word, Excel, PowerPoint etc.)

No deployment to user’s machine – it’s a web page

Additionally, an App for Office could optionally make use of SharePoint/Office 365 building blocks if useful. Some apps will just relate to Office (e.g. the Wikipedia app in the Office Store), but in the enterprise there are quite a few scenarios where it would be useful to bring in SharePoint/Office 365 – perhaps to store files (e.g. in a team site, or the user’s OneDrive For Business site), or access the user’s calendar, mail, contacts etc. When you need an Office app which uses these things, there are two high-level approaches:

  • Implement your app as an App for SharePoint (usually provider-hosted), which encompasses an App for SharePoint
    • This works for both on-premises SharePoint and Office 365, but has some trade-offs (e.g. App for SharePoint needs to be deployed to sites, or tenant-scoped install)
  • Implement your app as a standalone/external app, which consumes Office 365 services via the new Office 365 APIs and is registered in Azure AD
    • This only works for Office 365, but lots of advantages (e.g. app can be made available in the new App Launcher)

This is a decision that will come up quite frequently for Office 365 development from this point (autumn/fall 2014) onwards, and I’m sure I’ll be touching on it in quite a few future articles.

My “find similar documents” sample app

For the talk, I developed an app which surfaces in Word - specifically in the “task pane” which sits alongside the document. My app is nowhere near production-ready, but it tries to illustrate a few concepts, including use of SharePoint/Office 365 for surrounding functionality. Since I wanted it to work with both on-premises SharePoint and Office 365, I chose the provider-hosted App for SharePoint model – but my conclusion now is that the Azure AD/Office 365 APIs approach is preferable if the app only needs to work with Office 365 and you can go that way. Anyway, my app is based around the idea of “finding documents similar to the one you are working with” – perhaps to identify similar or duplicate content within your organization, find other people with expertise on the same topic, and so on. Here’s a screenshot to give you the idea (click on an image to enlarge):

Find similar docs app

Here’s what my app does:

  1. Analyzes the document content (or the user can choose to analyze just a specific part), and extracts the most relevant keywords/phrases:

    Find similar docs app - keywords

    It uses the top 3 keywords/phrases to search your SharePoint/Office 365 environment for similar content. I wanted something which worked purely on the content of the document, so that the user doesn’t need to tag the document in any way (to find related documents). By using the Office JavaScript API, I’m able to fetch the content of the document (or even just the content selected by the cursor) and then perform some analysis on it. Other, less important keywords/phrases are listed for information.

    1. As a bonus, it also analyzes the document for “sentiment” – and reports whether the document is mainly positive/negative:

      Find similar docs app - sentiment

      At this point, you might be wondering what kind of analysis I do to extract the keywords and determine the sentiment score. I’d love to tell you it’s some amazing algorithm that I spent weeks on and have registered for patent :) In reality of course, there’s actually a bunch of services on the internet that do this kind of thing – so (unfortunately) my smartness only extends to combining the Apps for Office model with one of these services! The one I chose is called AlchemyAPI, and it seems pretty cool. It’s a pay-for service to use in production, but you can obtain a free key for demos/evaluation etc.

  2. As shown in the image above, those top 3 keywords/phrases are used to power a SharePoint search query. The results are shown in the task pane – each result has a “Save” button (floppy disk icon!) next to it:

    Find similar docs app 
  3. The user can save any interesting references for follow-up – these get saved into a list in the user’s OneDrive For Business site (i.e. what we used to call their “My Site” in SharePoint). Here, the user can edit the list item and add notes if they choose – the main thing is they have the link and author details stored for future reference:

    Find similar docs app - saved references in OneDrive

    From here I thought it might be interesting if these “saved references” could be easily searched by other users – almost in a social bookmarking kinda way. So, I thought some enhancement of the search experience would be useful here:

  4. If any “saved references” appear in search results, they are highlighted with some special presentation (mainly just a grey background, some tweaks to the display so that the author’s notes and details of the “linked to” item are shown, along with a custom hover panel):

    Find similar docs app - saved references in search 

I also implemented a custom Result Source for saved references, so that a search only within these items can be done if desired.

Over the next few weeks, I’ll tidy up the implementation somewhat and publish the code somewhere.


  • Apps for Office have huge potential in putting useful business tools in front of users, so that they don’t need multiple windows/applications open. We can put the experience right there in the applications they would already be in – and this makes even more sense when the functionality somehow relates to documents or e-mail
  • The development model is WAY simpler than previous versions of extending Office – if you’re comfortable with a little JavaScript, you can interact with Office documents and implement apps which in Word, Excel, PowerPoint, Outlook and so on. The beauty is that your app will appear in the rich client, as well as web *and* mobile versions where appropriate – there are some caveats here, such as Word task pane apps only appearing in the rich client (for now), but for other permutations (e.g. mail apps and Excel apps) it’s a pretty good story already
  • Deployment and making your app available to end–users is also much simpler than before. There *are* some things to take care of on the IT Pro side (e.g.. deploying Office Trust Center settings through Group Policy), but these aren’t too tricky – I cover them towards the end of the slides

I’ll definitely be looking for opportunities to bring in Apps for Office to my client solutions moving forward – cool stuff indeed.

TechEd Video


Wednesday, 15 October 2014

Speaking at TechEd 2014 on Apps for Office

TechEd Europe is just round the corner (October 28-31, in Barcelona), and recently I’ve been heads-down preparing TEEU_2014_Im speaking_3to speak there on the Office 365 track. Most of the other speakers on this track are Microsoft folks from the Office 365/SharePoint/Exchange product groups (including Rob Howard, Brian Jones, Vesa Juvonen, Steve Walker, Jeremy Thake, Mauricio Ordonez and Sonya Koptyev) so there’ll be some *great* information being dispensed – my aim is to sprinkle in some field experience there too :) As an added bonus, I hear that TechEd will have an announcement or two that developers in this space will like.

My session is on Apps for Office, and I’m fortunate to be co-presenting with Andrew Salamatov from the Exchange team. Andrew is a Program Manager whose team builds the capabilities around mail apps, and he has some very cool stuff to show in this area. Apps for Office in general are becoming very interesting to me, as it kinda feels like there hasn’t been huge amounts of take-up of Apps for Office since the launch – maybe that’s confined to my corner of the Office 365/SharePoint-focused development world, but I think it’s surprising given the power and possibilities there. After all, many users LIVE in Office applications, and from the dev perspective it’s really just web development with some Office-specific JavaScript APIs these days – WAY less scary than previous models :)

Overall, Andrew and I have a couple of cool demos to show, including a Word task pane app which analyses the contents of a document to find you similar items across your SharePoint environment/tenancy whilst you're working with the document.

Here are the session details:

DEV-B307: Weave your enterprise solutions directly into Office products with the App Model (Friday, October 31 12:00 PM - 1:15 PM Room: Hall 8.0 Room D4)

Apps for SharePoint and Apps for Office allow you as an enterprise developer to weave your existing enterprise solutions throughout the Office products that your users live in day to day. In this session you will see compelling real world examples that light up existing Office products in context of key enterprise scenarios. The session will highlight tips and tricks on architecting this solution in Visual Studio along with Patterns & Practices guidance.

  • Understand what scenarios are available to surface your customizations in Office products
  • See real world enterprise scenarios that span multiple products stitching together an enterprise solution
  • Learn tips on how to architect enterprise solutions with the App Model

Hopefully see you there!

Monday, 8 September 2014

JavaScript-based provisioning in SharePoint/Office 365

In this post I want to specifically focus on the idea of JavaScript-based provisioning code – a potentially useful approach which has a place in every modern SharePoint developer’s toolkit. This is the idea of using SharePoint’s JSOM API (i.e. the JavaScript version of CSOM) to perform various one-time configuration steps for SharePoint sites - I was initially somewhat sceptical of this technique (JavaScript? For provisioning? Would that be reliable??), but having seen some team-mates use it with success on projects (kudos guys!), I think it’s worthy of consideration.


For SharePoint developers who are working with SharePoint Online/Office 365, or are working on-premises but choosing not to use full-trust WSPs simply to ensure their solution is “cloud-friendly”, the previous options for custom SharePoint code are no longer available. And some common scenarios still require code – especially when some form of custom site template is being developed, or we are working towards automated configuration of sites.

Some examples of provisioning steps that a developer might need to use code for could be:

  • Binding Managed Metadata/taxonomy fields
  • Setting a theme/Composed Look on a site
  • Setting the master page of a site
  • Modifying navigation settings
  • Branding OneDrive (personal) sites
  • [Optional] Provisioning fields and content types through code, rather than XML

This presents a challenge when server-side SharePoint code cannot be used. As we all know, Microsoft have deprecated the use of sandboxed SharePoint code (which we might previously have used for say, a Feature receiver), and this facility may be turned off in Office 365 in in the future. So, we have to use one of the remote APIs instead – and the code will either run from some other server (e.g. the .NET CSOM) or the user’s browser (JSOM).

Options for remote code

Broadly, the options for remote code include:

  1. Some use of the .NET CSOM API:
    1. Code deployed to a cloud service such as Azure, or running off your own server somewhere. This could be an “App for SharePoint”, or it could just be remote code which does not use app authentication.
    2. A “PowerShell + CSOM” script you wrote
  2. Some use of the JSOM API (i.e. the JavaScript version of CSOM)
  3. Some use of the REST API (either from JavaScript or remote server-side code)

As Microsoft put it, your custom code either needs to move UP to the cloud or DOWN to the user’s browser.

The JavaScript-based (JSOM) option

The idea of using JSOM code for site provisioning steps is a strange one initially. What happens if the user closes their browser as the code is running? Will the site get configured properly? What about if the user is very far away from the server on a slow connection? It was questions like these that made me sceptical initially, but with some safeguards and caveats in place I think it’s an interesting option for remote code. I’ll talk about how it works in a second, but first here’s why I think it’s interesting:

Side-stepping the “server-side remote code” issue

As a modern SharePoint developer, I’m all onboard with apps and remote code. In several cases I’ve implemented remote SharePoint code in Azure (e.g. see my Deploying SP2013 provider-hosted apps/Remote Event Receivers to Azure Websites (for Office 365 apps) post). So yep, I’m cool with that stuff! And I’m also down with kids in terms of Microsoft’s OfficeDev Patterns and Practices samples, which cover a ton of scenarios such as those I list above – they’ve effectively written a great wrapper around the .NET CSOM API calls, which make it really easy to get these tasks done.

BUT THIS TYPE OF CODE STILL HAS TO BE DEPLOYED SOMEWHERE! On-premises servers (e.g. some IIS boxes) can bring complexity in terms of high-availability, access outside the firewall, SSL certificates, backup and restore, and so on. Of course, cloud options exist too. If you (or your client) is already using Azure, then GREAT! It’s really not too difficult to get the code deployed and running there, and Microsoft take care of the operational/I.T. Pro factors listed above. But in many of the organizations I’ve worked with, unfortunately Azure (or similar cloud service) is not yet embedded in the I.T. department’s toolkit. There is no Azure subscription, with someone in I.T. already paying the monthly invoice. And there are all sorts of operational I.T. topics to discuss and get agreement on before we could get to that position – who owns responsibility, what kind of data will we be storing there, who has access, how does it fit with any existing SLAs/OLAs, which department/cost centre pays the bill and so on.

Interestingly, working in an “implementation partner” as my team does, we almost have an option of deploying some remote code for a client to Azure and not needing to involve them in this decision. The Azure Websites offering has a free option, and therefore all the billing questions/set-up could be avoided. However, you do not get high availability in free mode – so if the machines your code is running on are being patched, you’ll have downtime. So in the end, unless your code is trivial this probably isn’t an option.

Consequently, the idea of implementing remote code in JavaScript can be quite appealing in some cases.

A robust implementation for JavaScript-based provisioning

So, we’d need some way of making a JavaScript-based approach somewhat reliable. The pattern my guys have used is this:

  • The SharePoint site is created (perhaps from a custom WebTemplate which takes care of some settings in XML) – at this point, the site is NOT fully configured
  • A custom JavaScript file is referenced on every page – this checks to see if set-up steps have been completed yet. If not, the user’s permissions are checked to see if they are a Site Owner and whether we can proceed (with a “A site owner needs to visit this site to configure it” message shown if not).
  • When a user hits the site, our code runs. If we can proceed with set-up, a dialog is shown - “Please wait – we are getting your site ready”
  • The code starts to execute provisioning steps – perhaps setting a Composed Look and/or custom master page, binding taxonomy fields and so on
    • When each step completes, an entry is written to the web property bag e.g. “BrandingApplied = true”
  • When all steps are complete, a success message is shown and the “getting your site ready” dialog is automatically closed. The site is now ready for use.

As you can guess, it’s only when all confirmations are written to the property bag that we no longer attempt to run the steps. This works surprisingly well in practice – users/site owners are comfortable seeing these kind of messages, and having a site owner hit the site initially tends to work out fine, since they are the site creator. If any failures occur, well that’s fine too because the uncompleted steps will be re-tried on the next page load.

To illustrate the experience, here’s what the site owner might see during configuration (shown here on a test page, rather than a “real” site home page):

Dialog large

Of course, you could do something a bit fancier in terms of look and feel.

Some sample code for JavaScript-based provisioning

In my other article Three cloud-friendly ways for developers to provision Managed Metadata fields in SharePoint/Office 365, I show some JavaScript code which does the basics described above – specifically for binding taxonomy fields to MM Term Sets. This is the same sample as in the other article - I’ve included it again below for convenience, but I recommend reading the other article too as it has some extra notes about how the code should be used (like the fact you might want to enhance with JavaScript promises). Note that it doesn’t deal with setting confirmations in the web property bag, but it would be fairly simple to add this. The code will work with Office 365 or on-premises SharePoint:

I guess the main thing to take from this code is the approach - showing the dialog whilst several steps of JSOM code are being executed. In this simple example, when the code completes our taxonomy field will be bound and ready for use – but in our real-life projects obviously many more steps are performed.


Whilst the idea of using JavaScript/JSOM code for provisioning can seem strange, I believe it has it’s place and can be the right choice sometimes. Certainly, if you have no other need for remote code and/or do not have a hosting location (such as Azure or on-premises IIS servers) ready to go in a production sense, then the JSOM approach does offer you a way of running remote SharePoint code without the hassles that come with the alternatives. As always, consider the trade-offs and think about what’s right for you – but hopefully the information here helps you along the way.