Office 365 performance – our Azure CDN image renditions solution

In the last post image renditions causing slow page loads in SharePoint Online, I talked about how Office 365/SharePoint Online has some sub-optimal performance around images and image renditions, at least at the present time. Numerous people got in touch to say they also see the same issue. However, we are implementers - and we bring solutions, not problems! So in this post I’ll go into a bit more detail on our way of working around this challenge, and how it can improve page load times.

To recap, the problem is related to the image renditions functionality of SharePoint. This is a useful feature which automatically creates additional versions (sizes) of images in a publishing site such as an intranet. However, when a user hits a page which has these images - often a home page or section page – and they need to be downloaded, we see a big delay of up to 3 seconds. Clearly if a page is taking say, 5 or 7 seconds to download in total, this is a big chunk of this time. Surprisingly, the delay is NOT the actual image being sent over the wire to the user. Instead, analysis shows the 3 seconds or so pause happens on the server in SharePoint Online – most likely because of “cache misses” due to the fact that the renditions framework wasn’t originally designed for architectures used in Office 365. So, performance of this bit of the platform isn’t optimal - our solution was to roll our own renditions framework, and this post describes what we did.

Using Azure to implement renditions

Before delving into the implementation, here’s how I described the process last week:

1. An intranet author adds or changes an image in SharePoint
2. A remote event receiver executes, and adds an item to a queue in Azure (NOTE – RERs are not failsafe, so we supplement this approach with a fall-back mechanism which ensures broken images never happen. More on this below).
3. An Azure WebJob processes the queue item, taking the following steps:
   1. Fetches the image from SharePoint (using CSOM)
   2. Creates different rendition sizes (using the sizes defined in SharePoint)
   3. Stores all the resulting files in Azure BLOB storage
4. The Azure CDN infrastructure then propagates the image file to different Azure data centers around the world.
5. When images are displayed in SharePoint, the link to the Azure CDN is used (courtesy of a small tweak to our display template code). Since CDNs work by supplying one URL, the routing automatically happens so that the nearest copy of the image to the user is fetched.

For those interested, let’s go into the major elements:

The Remote Event Receiver and associated SharePoint app/add-in

There are two elements here:

• A SharePoint add-in used for our remote code (hosted in Azure) to authenticate back to SharePoint
  • We register this add-in using AppRegNew.aspx, specifying a Client ID and client secret which will be used for SharePoint authentication
• A remote event receiver used to detect when new images are added

The two are related because we implement the RER code as a provider-hosted add-in using the Visual Studio template (which gives 2 projects, one for the app package and one for the app web). In actual fact, this particular RER doesn’t need to communicate back to SharePoint – when it fires, it simply adds an item to a queue in Azure which we created ahead of time. The object added to the queue contains the URL of the image which was just added or modified, and we use the Azure SDK to make the call.

We apply this RER to all the image libraries in the site which needs the solution. We do this simply as a one-off setup task with a PowerShell/CSOM script that iterates through all the subsites, and for each image library it finds it binds the RER. My post Using CSOM in PowerShell scripts with Office 365 shows some similar snippets of code which we extended to do this. The script can be run on a scheduled basis if needed, so that any new image libraries automatically “inherit” the event receiver.

The Azure WebJob

The main work is done here. The job is implemented as a “continuous” job in Azure, and we use an Azure QueueTrigger to poll the queue for new items. This is a piece of infrastructure in Azure that means that a function in our WebJob code is executed as soon as a new item is added to the queue – it’s effectively a monitor. We initially looked at using a BlobTrigger instead (and having the RER itself upload the image to Azure BLOB storage to facilitate this), but we didn’t like the fact that BlobTrigger can have a bigger delay in processing – we want things to be as immediate as possible. Additionally, remote event receivers work best when they do minimal processing work – and since a quick async REST call is much more lightweight than copying file bytes around, we preferred this pattern. When a new item is detected, the core steps are:

1. Fetch details of the default rendition sizes defined in SharePoint for this site. This tends to not change too much, so we do some caching here.
2. Fetch details of the *specific* rendition sizes for this image, using a REST call to SharePoint. We need to do this to support the cool renditions functionality which allows an author to specifically zoom-in/crop on a portion of the image for a specific rendition – y’know, this thing:
   
   
   
   If an author uses this feature to override the default cropping for a rendition, these co-ordinates get stored in the *file-level* property bag for the item, so that’s where we fetch them from.
   
3. Fetch the actual image from the SharePoint image library. We use CSOM and SharePoint add-in authentication to make this call – our WebJob knows the Client ID and client secret to use. We obtain the file bytes i.e. effectively downloading the file from Office 365 to where our code is running (Azure), because of course we’re going to need the file to be able to create different versions of it.
4. For each rendition size needed:
   1. Resize the image to these dimensions, respecting any X and Y start co-ordinates we found (if the author did override the cropping for this image). There are many ways to deal with image resizing, but after evaluating a couple we chose to use the popular ImageProcessor library to do this.
   2. Upload each file to Azure BLOB storage. We upload using methods in the Azure SDK, and ensure the file has a filename according to a URL convention we use – this is important, because our display templates need to align with this.

Once the files have been uploaded to Azure BLOB storage, that’s actually all we need to worry about. The use of Azure CDN comes automatically from files stored there, if you’ve configured Azure CDN in a certain way. I’ll cover this briefly later on.

Authentication for the Azure WebJob

I thought long and hard about authentication. In the end, we went with SharePoint app-only authentication, but we also considered using Office 365/Azure AD authentication for our remote code. Frankly that’s my “default” these days for any kind of remote code which talks to SharePoint (assuming we’re talking about Office 365) – as discussed in Comparing Office 365 apps with SharePoint add-ins, there are numerous advantages in most cases, including the fact that there is no “installation” of an add-in, and the authentication flow can be started outside of SharePoint.

However, one advantage of using SharePoint authentication is that we aren’t tied to using the same Azure subscription/directory as the one behind the Office 365 tenant. Our clients may not always be able to support that, and that was important for us in this case – using this approach means we don’t have that dependency.

Display templates

As mentioned previously, a big part of the solution is ensuring SharePoint display templates align with file URLs in the CDN. So if we’re using roll-up controls such as Content Search web parts around the site and these reference rendition images, these also need to “know the arrangement”. Effectively it’s a question of ensuring the thing that puts the file there and the thing that requests the file are both in on the deal (in terms of knowing the naming convention for URLs). It’s here that we also implement the fall-back mechanism (more on this later) to deal with any cases where a requested image isn’t found on the CDN. In terms of the swapping out the default behaviour of fetching images from SharePoint to fetching them from the CDN instead, it just comes down to how the value used within the <img src> attribute is obtained:

<img src="_#= imgSrc =#_" />

Simply implement a function to get that value according to your URL convention, and you’re good. Although not shown in the snippet above, it’s here that our fall-back mechanism is called, courtesy of the ‘onerror’ handler on the <img> tag.

WebAPI

Since we’re talking about architecture pieces, there’s some WebAPI thrown in there too – this is part of the fall-back mechanism, described later.

Azure CDN configuration

As mentioned earlier, the CDN part is easy with Azure. When a file gets uploaded to Azure BLOB storage, it gets a URL in the form:

https://[MyAzureContainer].blob.core.windows.net/MyImage.jpg

..but if you configure the CDN, it can also be accessed on something like:

https://[MyCDNEndPoint].azureedge.net/MyImage.jpg 

When the latter URL is used, in fact the file will be requested from the nearest Azure CDN data center to the user. If the file hasn’t propagated to that location yet, then the first user to be routed through that location will force the file to be cached there for other users in that geographical region. Our testing found this additional delay is minimal. There are a few more CDN things to consider than I’ll go into detail on here, but initial configuration is easy – simply create a CDN configuration in Azure, and then specify it is backed by Azure storage and select the container where you’re putting your files. The images below show this process:

The fall-back mechanism

So I mentioned a few times what we call “the fall-back mechanism”. I was always worried about our solution causing a broken image at some point – I could just imagine this would be on some critical news article about the CEO, on a big day for one of our clients. Fortunately, we were able to implement a layer of protection which seems to work well. In short, we “intercept” a broken image using the HTML 5 ‘onError’ callback for the <img> tag. This fires if an image isn’t found on the CDN for any reason, and this kicks off our mechanism which does two things:

1. Substitutes the original rendition image from SharePoint - this means we’re “back to the original situation”, and we haven’t made anything worse.
2. Makes a background async call to our WebAPI service – this adds an item to our queue in Azure, meaning the image gets processed for next time. This is the same as if the RER fired against this particular file.

The image below shows what happens (click to enlarge):

One nice thing about this mechanism is it works for existing images in a site. So if the mechanism is implemented in an existing site with lots of images, there’s no need to go round “touching” each image to trigger the remote event receiver. Instead, all that needs to happen is for someone to browse around the site, and the images will be migrated to the CDN as they are requested.

Challenges we encountered

Along the way we faced a couple of challenges, or at least things to think about. A quick list here would include:

• Thinking about cache headers from Azure CDN, cache expiration and so on – this relates to scenarios where an author may update an image in SharePoint but not change the filename. Clearly end-user browsers may cache this image (and an end-user can’t be expected to press CTRL F5 to do a hard refresh just because you’ve updated a file!). My colleague Paul Ryan wrote a great post on this at Azure CDN integration with SharePoint, cache control headers max-age, s-maxage
• Parallel uploads to Azure (e.g. if we’re creating 8 different sizes for image found, may as well upload them in parallel!)
• Ensuring we understand how to handle different environments (dev/test/production tenants with different Azure subscriptions)
• Implementing a nice logging solution
• Testing
• We used the ever-awesome Azure Storage Explorer tool to monitor our files appearing in Azure BLOB storage (click to enlarge):
  
  

Summary

As I summarized last time, it would be great if the original performance issue in Office 365 didn’t occur. But CDNs have always been useful in optimizing website performance, and in many ways all we’re doing is broadening Microsoft’s existing use of CDNs behind Office 365. The building blocks of Azure WebJobs, Azure file storage, CDN, SharePoint add-ins, remote event receivers, WebAPI and so on mean that the Office 365/SharePoint Online platform can be extended in all sorts of ways where appropriate. This was a solution developed for clients of Content and Code so it’s not something I can provide the source code for, but hopefully these couple of articles help awareness of the issue and architectural details of one way of working around it.

Office 365 performance – image renditions causing slow page loads in SharePoint Online

Just like any website, there are many reasons why page load times might not be amazing in SharePoint Online. Perhaps it’s a page with too many ‘heavy’ controls (e.g search web parts), a particularly slow custom control, the amount of data going over the wire (e.g. due to large images JavaScript/CSS files), use of a known performance killer such as structural navigation, or maybe things are slow from the office due to network infrastructure such as reverse proxies slowing things. If users are far away from where the Office 365 tenant is located, that can certainly exacerbate things. As always, if the site has any kind of customization, some optimization steps need to be taken - good performance won’t always happen by default. Recently however, we’ve been noticing slow page loads even in:

• Sites we have optimized
• Out-of-the-box publishing sites

Analysis showed that the issue was related to image renditions in SharePoint. If you’re not familiar with this feature, it does something useful which, ironically, is intended to improve site performance. For each image uploaded, multiple resized versions are automatically created in the background – the idea is that end-users don’t download a large ‘original’ image when only a tiny thumbnail is needed. A classic example is large images added to content pages, which are then shown as a list of rolled-up links on a home page e.g. “most recent news articles”.

If it wasn’t for the performance issue, the principle works well – a 4MB image is typically shrunk to around 200k for a typical size, and that’s a lot less data being downloaded to users. I’m not sure if anything has changed recently in Office 365 (since most sites I’ve been involved in use image renditions), but a couple of clients noticed the issue around the same time we did. Specifically, pages with renditions are slow on “first-time” page loads i.e. whenever the images need to be downloaded because they are not served from the local browser cache. But unfortunately it’s not just that - rendition image files are served with expiry headers of 24 hours, meaning even regular users will have at least one very slow page load every 24 hours. And of course, we might not just be talking about their home page – rather, every page they hit could be slow once every 24 hours. That’s certainly enough to damage user perceptions about Office 365.

Sidenote – first-time page loads vs. returning user page loads

So renditions are slow even outside of first-time page loads. But whilst we’re on the subject, how much do we need to care about first-time page loads anyway? I typically advise my clients not to worry too much – frankly, Office 365 will always be slow here since there are some pretty heavy JavaScript files that need to be downloaded (even if they do come from a CDN). It’s a rich, highly-functional platform after all. But this is very much part of the Office 365 design – I believe Microsoft take the view that users are forgiving, so long as their *subsequent* browsing experience is quick. Most users don’t have the same expectations of their corporate intranet/collaboration platform as they do of public consumer sites such as Facebook and Google – and since most intranet usage is *not* first-time page loads, things work out in the end. I agree with this viewpoint frankly.

But why are image renditions slow?

When further analysis is performed, we see the delay happens on the server – the big surprise is that the delay is not for the actual image to be downloaded, which is what you’d normally expect. The following image shows a real home page being loaded, and we can see a delay of multiple seconds for many images (each one being a “rendition” image), as indicated by the long green bars:

When we dig deeper, we see the delay is not in the content download, but is in the “waiting” stage – this indicates the delay is with Office 365 itself, and not in the actual file being downloaded:

We believe this happens because there are typically “cache misses” on rendition images being served from the BLOB cache. When the image is not served from the blob cache, the SharePoint Online infrastructure is very slow to process and serve the image. It seems that cache hits are very rare for end-users – possibly due in part to BLOB cache settings in SPO (e.g. disk size allowed), but more likely due to sheer number of front-end servers in a typical SPO farm. I’m told that some of the larger farms in the service have between 100 and 200 front-end servers now – clearly this is a very different situation to an on-premises environment which SharePoint was originally designed for. So whilst the renditions architecture would be very effective in a typical on-prem farm of say, 3-6 front-end web servers, in the Office 365 world this is not the case. Of course, if you work closely with the product you sometimes see some examples of things like this that didn’t quite translate perfectly to the cloud world. That said, having worked on many deployments I’m always amazed at just how well SharePoint does work as a service (no doubt due to some hard work from talented Microsoft engineers, including some folks I know) - but there will always be “opportunities for improvement”, and the service often evolves to include these.

Our solution (based on Azure CDN)

So what can we do about it? Well, we could just avoid using SharePoint image renditions completely, but then performance would still be poor due to large files being downloaded to users. So we definitely do want to use different image sizes – and since the core work of resizing images isn’t that hard, why not do it ourselves? We could then take advantage of other things, like some automatic use of a CDN such as Azure CDN (N.B. that link explains what a CDN does if you’re not familiar). This is the direction we took to work around the performance issue in SharePoint Online. Things work pretty well, and what we implemented has the following benefits:

• The Office 365 renditions delay does not occur
• There is no impact to intranet end-users (except the improved performance)
• Intranet authors do not need to do anything different or have additional training
• The image files are hosted in Azure CDN (Content Delivery Network), which places the files in various Azure datacentres around the world, to ensure they are close to the user. This can significantly boost performance further for some users, especially those far away from the Office 365 datacentres (e.g. non-European users in the case of most of our clients)
• All of the capabilities of the image renditions framework are supported (e.g. the ability for an author/administrator to crop an image rendition so that a certain portion of the image is used)

Technical architecture of our solution

I’ll go into more detail in the next post, but briefly our solution works as follows follows:

1. An intranet author adds or changes an image in SharePoint
2. A Remote Event Receiver executes, and adds an item to a queue in Azure (NOTE – RERs are not failsafe, so we supplement this approach with a fall-back mechanism which ensures broken images never happen. More on this next time).
3. An Azure WebJob processes the queue item, taking the following steps:
   1. Fetches the image from SharePoint (using CSOM)
   2. Creates different rendition sizes (using the sizes defined in SharePoint)
   3. Stores all the resulting files in Azure BLOB storage
4. The Azure CDN infrastructure then propagates the image file to different Azure datacentres around the world.
5. When images are displayed in SharePoint, the link to the Azure CDN is used (courtesy of a small tweak to our display template code). Since CDNs work by supplying one URL, the routing automatically happens so that the nearest copy of the image to the user is fetched.

This is depicted below:

Fall-back mechanism

Clearly it’s critical that an intranet home page never displays a broken or missing image – that would be A Very Bad Thing for most organizations. So how can we guard against that? Also, we said that Remote Event Receivers cannot be 100% reliable (and also should not do “heavy” processing work), so…..what about that? And what about existing images in a site that was running before a solution like this is implemented? My colleague Paul Ryan and I wrestled with these challenges and more as we architected the solution and wrote the code - I’ll talk more about the fall-back mechanism (which takes care of these aspects) and go into more detail on the technical implementation in the next post. 

Summary

It would be great if this issue didn’t exist in Office 365 in the first place of course. But I write this post to show that with the right building blocks, we can certainly supplement functionality around Office 365/SharePoint with some effort. This was a solution developed for clients of Content and Code so it’s not something I can provide the source code for, but hopefully this write-up helps awareness of the issue and potential ways of working around it. More next time..

Get started with Office 365/SharePoint Online dev – part 2: Developing SharePoint Add-ins (apps)

This is the second post around getting started with Office 365/SharePoint Online development. In these couple of articles, I discuss a process where developers new to this space can get started by using trial environments and an Azure virtual machine. The idea is that you don’t need an MSDN subscription, existing Office 365 environment or even a development machine to get running – in the last post, we created all those things. Now it’s time to use them. Specifically, we’ll get some add-in code running in the Azure VM, which talks back to SharePoint Online - we’ll do this by obtaining a sample app and configuring our dev environment to run it. In my opinion, this is a good way to learn about modern SharePoint development which uses “provider-hosted” remote code – which, of course, can be used on-premises or in the cloud. Before we get started, a reminder on the contents of these two articles:

1. Get started – part 1: Create trial environments and a VM
2. Get started – part 2: Developing SharePoint Add-ins/apps (this post)

Deploying a provider-hosted SharePoint Add-in in your VM

Here’s an overview of what we’ll cover here:

• Create a Developer Site in Office 365
• Complete some information in your user profile in SharePoint Online (since our lab demo will use it)
• Prepare the development environment to host apps locally:
  • Create a local IIS website to run apps
  • Create a self-signed SSL certificate and bind it to the site
• Register the SharePoint Add-in using AppRegNew.aspx
• Download the “app script part” SharePoint Add-in sample from from Github (from the Microsoft OfficeDev Patterns and Practices library), and run it locally in the VM
• Review how the solution is working

Here are the step to go through..

Create a developer site collection in Office 365

Navigate to the SharePoint admin area of Office 365, and go to the “Site Collections” page. Click “New” to create a new site collection:

Use settings like the below:

Once the site has created, check it can be accessed in the browser:

Enter some details into your user profile in Office 365/SharePoint Online

In any SharePoint page in your Office 365 tenant, click the user photo in the top right corner, and then click the “About me” link to go to the profile page:

Click the “Edit profile” link:

Enter some details into fields such as “About me” and “Ask Me About”. Also use the “Change your photo” link to change your photo:

Once done, click “Save all and close”.

Create a new site in IIS for app hosting

Open IIS Manager, and create a new website. Use the following settings:

Config item	Value
Name	spsites
Path	C:\inetpub\wwwroot\spsites (N.B. the “spsites” folder will need to be created)
Host name	spsites

The Add Website dialog should look like this:

Click “OK” to finish creating the IIS site.

Add the site to your hosts file:

Create a self-signed SSL certificate and apply it to the site

Open a PowerShell window as an administrator. Create a self-signed cert using the following command:

New-SelfSignedCertificate -DnsName spsites -CertStoreLocation cert:\LocalMachine\My

Now we need to install the certificate as a Trusted Root certificate in the store, and apply it to the “spsites” IIS website. The following steps are used:

1. Certificate is exported to a file.
2. Certificate is installed from the file on the filesystem to the “Trusted Root Certification Authorities” cert store.
3. Certificate is applied to the IIS website.

Go to “Server Certificates” in IIS, and find the certificate you just created. Follow the sequence in the steps below to export the certificate to a file:

The certificate should now be exported as spsite.cer.

Now find the file on the filesystem, and right-click > “Install Certificate”:

Follow the process in the images below to install the certificate to the Trusted Root cert store:

The final step around IIS and SSL certificates is to apply it to the site. Find the website in IIS, right-click on it and select “Edit bindings..”:

Add a new site binding on port 443 – select the “spsites” certificate:

Click “OK”, and the certificate should now be applied to the site. You should be able to browse it on https://spsites (although note you will get a default IIS page, since there is no site running there at this point).

Obtain the “app script part” sample from the Microsoft OfficeDev Patterns and Practices library

Navigate to https://github.com/OfficeDev/PnP - click the “Download zip” link:

Save the zip file to C:\Code on your virtual machine (create the folder since it won’t exist already), and then unzip:

Now run Visual Studio 2015 as an administrator. Note that it can take some time to open in an Azure virtual machine.

In Visual Studio, use “File > Open > Project/Solution…” to open the “App Script” sample from the zip in Visual Studio – this can be found at:

C:\Code\PnP-master\PnP-master\Samples\Core.AppScriptPart\Core.AppScriptPart.sln

Once the project has opened, enter the URL for your Office 365 developer site in the Site URL property of the main project in Visual Studio:

A dialog box should appear – enter the credentials for your Office 365 identity:

Visual Studio should now be signed-in to your developer site in Office 365.

Configure the web project to match your hosting environment

Go to the properties page for the “Core.AppScriptPartWeb” project, and go into the “Web” area. Change the settings to match the IIS site we created, and click the “Create Virtual Directory” button to allow the files to be hosted in their dev location:

Save the Visual Studio project.

Also edit the userprofileinformation.webpart file in the project – find the line which sets the URL of the JavaScript file being linked by the web part in the sample, and set it to https://spsites/Core.AppScriptPartWeb/scripts/userprofileinformation.js:

Create the registration for your Add-in in SharePoint Online

In a browser window to your developer site, navigate to the following URL:

[site]/_layouts/15/AppRegNew.aspx

..so in my case, this is:

https://chrisob1.sharepoint.com/sites/Dev/_layouts/15/AppRegNew.aspx

Click the “Generate” button to generate new values for both the Client ID and Client Secret fields, then complete the other information as below:

On the next screen, be sure to copy the Client ID and Secret somewhere safe – you’ll need these later:

Now return to Visual Studio and perform the following steps:

Open the web.config file in the web project, and update the ClientId and ClientSecret app settings values:

Open the AppManifest.xml file in the app project, and update the ClientId attribute:

Your add-in should now be configured to run in development – in the next step we will run the app to test it.

Run the add-in

Press F5 in Visual Studio to run the project. Enter the credentials for your Office 365 identity if prompted:

You should be prompted to trust the app:

Click “Trust It”, and you should then be taken to the default page for the add-in. Note that this page is not a key piece of the add-in – it simply uploads the web part in an automated way to make it available.

To do this, click the “Run scenario” button:

Now click the “Back to Site” link:

Edit the page, and add the web part deployed by the add-in to the page. Go to add a web part, and you’ll find it in the “Add-in Script Part” category:

Once the web part has been added to the page, it should show details from your user profile (including a photo if you have one – I don’t in this case!):

If you got this far, well done! Now let’s consider what we just got working.

Key takeaways

• We implemented a web part which is truly cloud-friendly – it has NO server-side code, because it is not a legacy web part.
• In fact, it’s just an out-of-the-box Script Editor web part – which points to a particular JavaScript file, which has the actual functionality implemented
• The JavaScript file has some CSOM code to fetch the user profile details
• The JavaScript file is actually hosted remotely from SharePoint. In the case of our development scenario, it was running in IIS – but we could publish it to Azure or similar for production use (remember we’d need a new app registration with the appropriate URL etc.)

The key bits of code are essentially some simple JSOM code to fetch user profile details and output to HTML:

So that’s an introduction to the world of remote code and cloud-friendly development in SharePoint. Hope someone found it useful!

Get started Office 365/SharePoint Online dev with an Azure VM – part 1: Create trial environments and a VM

I’m always surprised by the number of roadblocks which developers in large organisations have to face. Whether it’s locked-down desktops, lack of virtual machines (or very low spec virtual machines), inadequate licensing for developer tools, or any of the other myriad of issues, it can be frustrating trying to get started with new technologies. I often talk to developers in such companies, and often see some reluctance to do things which I find myself doing fairly frequently to test things out – creating a trial Office 365 environment, spinning up a virtual machine in Azure, downloading/testing an Office 365 or SharePoint app and so on. Recently I put together a guide to steps like these (for attendees of a Microsoft dev event here in the UK), and I thought it might be nice to repurpose the guide for other newbie developers in case it’s useful. This is split into two articles:

1. Get started – part 1: Create trial environments and a VM (this post)
2. Get started – part 2: Get started with developing SharePoint Add-ins (apps)

I guess the type of SharePoint developer this is aimed at is:

• Doesn’t have access to an Office 365 environment to play in – may not even have touched SharePoint Online before
• Isn’t able to quickly obtain a new virtual machine
• Doesn’t have an MSDN license
• May not have touched Azure before
• Needs a developer environment with the latest version of Visual Studio to look at/run code locally (i.e. hosting apps on https://localhost). Perhaps their existing desktop or virtual machines aren’t appropriate for this

If one or more apply to you, you might find this useful. We’ll go from creating the Office 365 and Azure dev environments, to downloading and running a sample app (a provider-hosted SharePoint Add-in) from Microsoft’s OfficeDev Patterns and Practices samples. This can help someone experience add/add-in development for the first time, but it can be just a generally useful set of steps that can be handy in several scenarios.

Disclaimer!

Although I've taken care with these steps, I make no guarantee they're 100% accurate/will work in all regions/at all times, and can't really dedicate much time to support around this topic I'm afraid! Hopefully they're enough to get you up and running though.

Here’s what is covered:

This post:

• Create trial Office 365 E3 tenant
• Create a trial Azure subscription
  • NOTE – you WILL need a credit card for this step unfortunately. BUT, it’s possible to avoid spending money, since you receive credit to the value of £125 and by default, the Azure subscription is “locked” to prevent further charges. This will enable you to get started at least
• Prepare the SharePoint Online tenancy for apps – create an App Catalog etc.
• Create a development VM in Azure with Visual Studio 2015
• Prepare Azure VM for development (e.g. enable IIS and ASP.NET, install the OfficeDev tools for Visual Studio etc.)

Next post:

• Walkthrough of deploying a provider-hosted SharePoint add-in in your VM (next post)
• Create a self-signed SSL certificate
• Create a local website in IIS to run apps
• Download the SharePoint Add-in from Github and run locally

Create a trial Office 365 environment (E3)

In this step, we’ll create a trial Office 365 environment to enable us to do some development. Note that another option here is to sign-up to the Office 365 Developer program – this will provide you with a tenant which will last 365 days, but only contains one user. If I’m just spinning up something which I don’t care about being durable, I often prefer to create an E3 trial – this only lasts for 30 days, but comes with 25 users. Both options are useful (depending on what you’re doing), but in this case we’ll go through the process for creating an E3 trial.

To do this, navigate to the Office 365 Enterprise E3 product page (the UK version of this page is at the following link, but simply change the market using the selector at the bottom of the page if you are in another country - https://products.office.com/en-GB/business/office-365-enterprise-e3-business-software?omkt=en-GB). Click the “Free trial” link on this page:

Enter some details into the form and click “Next”:

On the next screen, provide a username and also select a prefix for the “.onmicrosoft.com” domain – NOTE, this must be unique in Office 365, so you may need to try a few names until you find one which is unique:

Click “Next” when done. On the next screen, provide your mobile phone number to receive the text message verification. Once entered, click “Text me”:

Enter the code in the box provided, then click “Create my account”:

On the next screen, make a note of the user ID you provided and then click the “You’re ready to go…” link:

Your Office 365 environment is now creating – this will take around 20 minutes:

Prepare Office 365 tenant for development

Once your tenant has finished provisioning, go to the Admin Center and navigate to the SharePoint area. If your tenant is using the current Admin Center experience (rather than the new preview experience), the link is here:

Once in the SharePoint area, navigate to the “settings” page:

On this page, scroll to the “Custom Script” section. Change both radio buttons to the “allow” option to enable custom scripts to execute. The radio buttons should look like this:

Click “OK” to save the settings.

Now go into the “apps” area, and select “App Catalog”:

Ensure the option to create a new App Catalog site is selected, then click “OK”:

Complete the form using settings similar to the below – ensure you enter YOUR username for the administrator field:

Click “OK” when done. The App Catalog site will now create.

Provide payment details to enable the “full” Azure subscription

Within the Office 365 admin area, click the “Azure AD” link:

On the next page, click the “Azure subscription” link to start the process of signing-up to Azure:

Complete the details on the sign-up page which appears next. You’ll need to verify your identity by providing a phone number to receive a text message or call, and it’s here that you’ll need to provide credit card details. As a reminder, you WON’T be charged for anything unless you explicitly change to a paid subscription – as the page tells you:

Once you’re through the process, click “Sign-up”:

Your Azure subscription is now creating. You’ll see a screen like the following (though some of the messages may be different) - wait for this process to complete:

When the process completes, click the “Start managing my service” button as shown below:

Your Azure subscription should now be ready, and you should then be taken into the Azure Portal:

Create a development VM in Azure

In this step, we will create a new virtual machine in Azure using the Visual Studio Community 2015 template. Go into the “Virtual machines (classic)” area of the portal, and click the “Add” button:

In the Virtual Machines area in the Azure portal, filter for ‘Visual Studio Professional 2015’ and then select the resulting VM from the results.

In the filter box, type “visual studio community 2015” and hit enter. In the list of VM templates which appears, locate the item we want:

Select this item from the list. On the next screen, select “Resource Manager” in the dropdown box and then click “Create”:

On the next page, enter some details for the virtual machine. Be sure to make a note of the username and password you use: Also be careful with the “Resource Group” setting – you’ll need to create a new Resource Group here (any name is OK). Make sure it then becomes selected in the dropdown:

On the next page, select size “A2 - Standard” for the virtual machine size:

In the next area, leave the default settings and click “OK”:

Click “OK” on the confirmation page:

The virtual machine will now start to create – this can take around 20 minutes.

Once the VM has been created, select it in the portal and click on ‘Connect’:

This will instantiate the RDP connection. Take care in the RDP box – you’ll need to select “use another account” and be sure to enter the username in the following form:

[machine name]\[username]

This is shown below:

Click “OK” once the details are entered. After the security prompt, your RDP session should open and you should now have control of the virtual machine. We now need to perform some configuration steps within the VM to be able to develop SharePoint Add-ins and Office 365 apps.

Setting up IIS

One of the first things we need to do is enable apps to be developed/hosted on http://localhost within the VM. The first step is to enable IIS.

Within the VM, right click on the Start button and select ‘Programs and Features’

Select “Turn Windows features on or off”:

In the “Add Roles and Features Wizard” which appears, click “Next” until you reach the “Server Roles” page of the dialog. Here, check the box for “Web Server (IIS”) and click “OK” to any prompts which appear:

Then, click “Next” until you reach the “Role Services” page for IIS. In this dialog, expand “Application Development” and check the boxes for “ASP.NET 3.5” and “ASP.NET 4.5”:

Click “OK” to any prompts which appear. On the confirmation screen, check the box for automatic restarts then click “Install”:

Wait until process has completed. Once complete you now have IIS running on your VM with ASP.Net enabled.

At this point, restart the machine.

Install Office Development tools

Start Visual Studio 2015 – ensure you right-click and run as administrator:

In the dialog which appears, click the “Sign in” button:

Enter the details you used for your Office 365 subscription and Azure identity:

Accept any prompts which appear until Visual Studio is started.

Once VS has started, close it again. Open a browser window, and navigate to http://aka.ms/OfficeDevToolsForVS2015 - save the file at the prompt, and then run from the download location to run the installer:

Accept the prompts which appear, and then wait for the install to complete.

Your virtual machine is now broadly ready for SharePoint Add-in and Office 365 development. Some other steps you may perform are:

• Create a self-signed certificate to run apps on HTTPS
• Configure a new IIS website to host apps locally (I prefer this to running IIS Express)

We’ll look at these steps in the next post, “Get started with developing SharePoint Add-ins (apps)”. Until then, happy developing!