Tuesday, 9 August 2016

Introduction to the SharePoint Framework – presentation slides available

I presented recently at the SharePoint Saturday London 2016 event, on the topic of developing with the new SharePoint Framework. As more and more SharePoint developers are coming to realize, it’s a time of big changes and although the SharePoint Framework has not yet hit general availability (at the time of this article), it’s very very close and so it’s good to start soaking up the information. I held back publishing this slide deck, as I knew Microsoft were making some changes to the Framework since I first saw it and were going to release some updated bits to us folks with early access. I wanted this ‘early preview’ information to remain as accurate as possible for when *you* get the Framework, and I’ve now verified everything in the deck is still accurate.

The deck is available for download on SlideShare at http://www.slideshare.net/chrisobrien/chris-obrien-introduction-to-the-sharepoint-framework-for-developers, but you can also flick through the slides below. Unfortunately the demos weren’t recorded at the event – so I’ve added a couple of slides to describe them in case that helps.

 

Looking forward

Hopefully the SharePoint Framework will start rolling out to Office 365 tenants in First Release mode soon. It’s a radically different development experience, and the payback is that you can develop performant web parts that work well in responsive design and can be used in “modern pages”. Rememeber that it's not possible to use what we're now calling "classic" web parts in modern pages, and given that both team sites and publishing sites in Office 365 will soon be able to use modern pages, that alone makes the new model worthy of attention. Additionally, it will come to on-premises SharePoint 2016 in a Feature Pack in 2017, so even if you're on-prem only it's perhaps advisable to start looking at this.

Onwards and upwards!

Monday, 1 August 2016

Sandbox code no longer available in Office 365 – with immediate effect

Most technical people working with Office 365 have known that sandbox code has been deprecated for some time. This doesn’t mean any sandbox solution is deprecated however – instead, the deprecation specifically refers to *code* in sandbox solutions, and by that we mean code which uses the server-side API. No Code Sandbox Solutions (NCSS) continue to be supported. This advance warning about sandbox code was given back in January 2014 in https://blogs.msdn.microsoft.com/sharepointdev/2014/01/14/deprecation-of-custom-code-in-sandboxed-solutions/. Since then however, no timeline for the actual removal of the feature was announced – and in fact there was no news whatsoever. I even heard from some Microsoft folks that the urgency of removing sandbox code from SharePoint Online had gone away, since the original operational concerns weren’t materializing and in fact the engineering team were able to run and scale the feature just fine in Office 365.

So a big surprise last week was the sudden removal of the feature in many tenants. The “Activate” button against a solution is unavailable, and the following message is shown:

"Activation of solutions with sandboxed code has been disabled in this site collection. Contact your administrator to enable activation using the guidance published here."

SPO - sandbox code disabled

One very interesting aspect of this is that you will see this even if you’re not using sandbox code, but *are* deploying a code assembly – and this is the default in Visual Studio! So, lots of solutions which aren’t really using sandbox code at all will still need some attention – developers will need to repackage the WSP again, this time without including the auto-generated assembly. I discuss the specific change later in the “How do I remove auto-generated sandbox assemblies?” section.

Official Microsoft announcement

What effect will this have?

Well, it depends on which camp you fall in:

  1. You’re not actually using sandbox code, but have some auto-generated assemblies deployed.
  2. You genuinely are using sandbox code.

For number 1, your developers will need to make some relatively simple updates. See the “How do I remove auto-generated sandbox assemblies?” section.

For number 2, the short answer is you have a problem. Just how big depends on how sandbox code is being used, and how much of it you have. Ultimately you need to take action to re-engineer your solution now. You can’t wait until some code changes are scheduled, since bits of your solution may stop working very soon and you may have a service issue. Already, you will not be able to add new sandbox code or reactivate an existing sandbox WSP with code. Additionally, the Microsoft message states that you have 30 days before existing code will stop working. In technical terms, that means the following things will stop working:

  • Event receivers
  • Feature receivers
  • Workflows (coded)
  • InfoPath forms (if using code behind the forms)
  • Web parts (if coded using sandbox code)

Exactly what that means for your intranet or solution is down to how those building blocks have been used by the original developers.

Considering the impact

I’m really surprised at this – in the original deprecation announcement, Microsoft said:

We realize that our customers have made investments in coded sandboxed solutions and we will phase them out responsibly. Existing coded sandboxed solutions will continue to work in on-premises SharePoint farms for the foreseeable future.

Happily, none of my clients/projects are affected because we knew better than to build solutions using this approach. But I feel for organizations in a different situation. I was certainly expecting a significant “final warning” period and I’d heard it said that there would be at least a further 12 months from this point. But apparently not – and I know Microsoft have probably not breached any contract or element of the Office 365 Service Description here, but I struggle to see that this is “responsibly” taking a feature away from the service.

On a related note, I’ve being doing some work alongside Microsoft with a global enterprise (over 100,000 users) who have sandbox code in their Office 365 intranet – in light of this, it’s perhaps fortunate that they are not yet live. We didn’t create the initial solution, but flagged this in our review of their code, along with a few other things. I suggested the Microsoft folks reach out to their colleagues for clarification on the timeline for switching off sandbox code. Interestingly, the message that came back is that “everything should be fine for at least 12 months”, and so if it’s a surprise to these guys I think it’s fair to say it’s a surprise to many others too.

Communication troubles

Notably, there was some confusion as this was playing out initially. At first, this was being reported as a temporary service issue, and administrators may have seen a message to this effect in the Office 365 Message Center:

SPO - sandbox code service issue_

However, sandbox code really is being disabled – it just seems that for a while, other teams within Office 365 didn’t actually know either. You will now see one of two things in the Office 365 Admin Center:

Status

Message

You have a problem – sandbox code was detected

MC73347 - We’ve detected that you are using a code-based sandbox solution with your tenant account.

Please be advised that we’ve moved forward on our plans to remove code-based sandbox solutions as previously announced in 2014, which can be seen here: http://aka.ms/sandboxcode

Please note that declarative (no-code) sandbox solutions are still fully supported.

How does this affect me?

As part of the removal process, activation of new code-based sandboxed solutions, as well as updates of existing solutions are no longer available. In approximately 30 days, currently running, code-based sandbox solutions in the SharePoint Online environment will be disabled. If you have an extenuating circumstance, please contact us as our Product and Customer Support teams are ready to support you during this transition.

You’re OK – no sandbox code was detected

SP73009 - Custom Solutions and Workflows - False positive

Incident customer impact scope has been updated. Ongoing analysis of customer impact has determined that your service is not impacted by this incident

So that’s the final story it seems.

How do I remove auto-generated sandbox assemblies?

As mentioned earlier, developers may need to repackage WSP solutions without the auto-generated assembly. You can do this by setting the “Include Assembly in Package” project property to false:

clip_image005

This step may be necessary the next time some development or maintenance is carried out – effectively anything that would require activation of another version of a WSP.

Summary

We rely on Microsoft to make the right decisions in terms of running Office 365 (and on-premises SharePoint). I’ve no idea of the scale of sandbox code out there in real-life solutions, but I really hope the way this is being handled doesn’t affect too many organizations too badly. Are you affected by this? Or do you otherwise have views? I’d be interested in hearing how this is being received – feel free to leave a comment and let me know.

Friday, 1 July 2016

Understanding the web part manifest, bundle.json and other key files and folders in the SharePoint Framework

The new “SharePoint Framework” model for client web parts and client-side applications uses a whole series of files that developers must learn to successfully extend SharePoint using this approach. There is a new web part manifest file (now in JSON rather than XML) and files such as bundle.json, package-solution.json and upload-cdn.json which interact with the new Gulp-based developer tools to control how the solution is packaged and deployed – all wrapped up in a new folder structure. So, there’s a learning curve and understanding these files and folders is crucial. In the previous post Develop a client web part in the SharePoint Framework - a walk-through we looked at the overall “getting started” process which creates the files for you using Yeoman Generator – but in this article I’ll explain the files and folders.I

Folders used in the SharePoint Framework

Let’s start with a rundown of the various folders and what they contain. My last post had a cut-down version of this table, but I expand on the descriptions here, and add a list of key files and an image so you can see what that folder looks like:

Folder

Purpose

Key files

Image

src The place where you add/edit code files – you will build up your structure of files here.
  • [MyWebPart1.ts] (TypeScript)
  • [MyWebPart1.manifest.json]
SNAGHTML72de5c5_thumb
lib Contains “processed” code files which are ready to move into the bundle which is distributed with the app.

For example, TypeScript files have been compiled to JavaScript files in this directory.
  • [MyWebPart1].js
  • [MyWebPart2].js
SNAGHTML72e6ac4_thumb
dist Contains the final code files which are distributed with your application.

The most important file is the final JavaScript bundle file  [MyWebPart].bundle.js – this is all of the JS for your web part, including some framework stuff.
  • [MyWebPart].bundle.js
  • [MyWebPart].bundle.js.stats.json
  • webpart.manifest.js
SNAGHTML4ed93fd_thumb
config Contains a set of JSON files used by the tooling for the build process. In particular to control how your app is packaged - in terms of the .spapp file, JavaScript/CSS bundling and so on.
  • build.json
  • bundle.json
  • package-solution.json
  • serve.json
  • upload-cdn.json
SNAGHTML4e02bfb_thumb
node_modules Contains JavaScript modules used by the solution. TypeScript follows the standard set by node.js for resolving modules (i.e. when you use the import keyword to use an object defined in another module/library), and the node_modules folder is a key part of this.  Whatever JavaScript libraries your code OR the underlying SharePoint Framework has a dependency on. SNAGHTML73042df_thumb
typings Contains TypeScript typings files – these are used to give you auto-complete (IntelliSense) against JavaScript libraries you are using, such as jQuery. Typings for whatever JavaScript libraries you’re using in code. These typically get added with TypeScript Definition Manager with e.g.
tsd install jquery jqueryui --save
SNAGHTML14200135
sharepoint Contains the .spapp file which is generated by ‘gulp package-solution’. This is what you’ll add to the App Catalog of your real environments such as production.

Also has a ‘debug’ folder so you can see what went into the package.
  • [MyApp].spapp
SNAGHTML20029fca

The web part manifest

The [MyWebPart].manifest.json file is where core details such as the display name and description are specified. Critically, other associations are specified here such as the associated JavaScript file (usually a bundle) and any dependent JavaScript modules which will be loaded – yes, there’s an in-built JS loading framework for you. Here’s a rundown of some key properties specified in the manifest.json file:

Element

Sample value

Purpose

ManifestVersion 1 Represents the version of the manifest schema that the build system is expected. Currently this must be “1” – any other value will be rejected.
id 41d9c141-e10f-4373-8a24-84383fa95592 ID of the client web part – the same as we’re used to, i.e. it provides a concrete way of identifying a web part (e.g. through APIs)
bootstrapModule http://localhost:4321/dist/cob-latestnews-wp.bundle.js

https://cob.azureedge.net/intranet/cob-latestnews-wp.bundle.js
This is where the JS file for the web part lives. In dev, you’ll have a localhost path (which the tooling hosts for you using node.js). For production, you’ll replace this with the path where you deployed the JS file to for real – a CDN, website path, or similar.
moduleName CobLatestnewsWebPart JavaScript module name of the web part object.
localizedScripts ? I believe this is an array of locale/JavaScript file mappings – to support localization.
preloadModules

"preLoadModules": ["jquery","jqueryui"]

Specifies JavaScript modules that the web part should load before execution. The module names specified here must match names/paths specified in the “config” element described next – in other words, you name the module and specify where it’s loaded from.
config

"config": {
        "paths": {
            "jquery": "//code.jquery.com/jquery-1.10.2.js",
            "jqueryui": "//code.jquery.com/ui/1.11.4/jquery-ui.js"
        },
        "meta": {
            "jqueryui": {
                "scriptLoad": "true",
                "deps": [
                    "jquery"
                ]
            }
        }
    }

Used to specify paths to JavaScript libraries pre-loaded by the web part (as specified in the “preloadModules” tag). Dependencies between scripts are also specified here - so that jQuery is loaded before jQuery UI for example (shown left).
properties

"properties": {
    "description": "Displays recent docs for the current user.",
    "headerText": "Your recent documents:",
    "searchQuery": "*",
    "itemLimit": "5"
  }

Specifies default values for web part properties. Note that other elements of web part properties are defined in the web part code – specifically the interface which represents the web part’s properties, and the propertyPaneSettings method. You can see these in the code sample above.

Note that there are other properties in the manifest file which I’m not listing here – but these are some important ones for now!

Other key files in the “src” folder

In addition to the web part manifest, files your code files also live here. These other files here can be summarized more easily, so I’ll switch to bullet points. You will create sub-folders in here, and build our your codebase as you need. But let’s take the example of single web part, and look at the set of files used:

  • [MyWebPart].test.ts – a file to add Mocha JavaScript tests to
  • [MyWebPart].less – a file to add LESS CSS directives to. If you’re not familiar with LESS, it’s a way of structuring CSS in a neater way – it allows variables (e.g. @cob-title-color: #FFB900; – which you can then use in different styles). In addition to variables, CSS markup can be nested in a way which matches HTML structure. Both of these things make your CSS code easier to maintain.  See http://lesscss.org for more details.
  • [MyWebPart].strings.ts – a file to add resource strings to (to keep them in a separate place from your code, and therefore avoid magic strings)

Files in the “lib” folder

The lib folder serves as an intermediate folder in the build system. A few things happen between the src and lib folders:

  • TypeScript files get compiled to JavaScript files
  • LESS files get compiled to CSS files 
  • CSS files get minified, and prepared for the CSS loader
  • HTML templates files get compiled to JavaScript (N.B. I’m still working that one out!)
  • TypeScript typings files are generated for each of your .ts files (i.e. “foo.d.ts”, as per the TypeScript convention)

Files in the “dist” folder

These are files which are distributed with the app – several of these are used at run-time for example:

Key files:

  • [MyWebPart].bundle.js – this is all the JavaScript required to run your solution, and is CRUCIAL to your web part or client-side application working properly. When you package for production, you can host this file anywhere you like (CDN, Azure web app, on-premises web server etc.) but whatever URL it lives at must be reachable by end-users AND it must be specified in the “bootstrapModule” key in the web part manifest.
  • [MyWebPart].bundle.js.map – map file to support browser debugging
  • [MyWebPart].manifest.js – this is the shipped version of your web part manifest file.
  • [MyWebPart].bundle.js.stats.json – a file which contains metadata about the generated JS bundle. One really useful thing in here is the “modules” node, which lists all the original JavaScript files which went into the bundle. This is really useful if you’re troubleshooting why a JavaScript library you’re using isn’t making it’s way into the bundle (or vice-versa) – much easier than trying to read the uglified JavaScript in the bundle itself (and trying to work out where jQuery stops and jQuery UI starts for example!).

Files in the “config” folder

The config directory holds a collection of JSON files which mainly control how the app gets packaged. The most critical files are:

  • bundle.json – this controls which JavaScript files make their way into your final JS bundle. Note that by default, any 3rd party JS libraries you load as a module into your code (e.g. with something like import $ = require('jquery'))  will get included into the bundle. However, you can override this (for example if you’re referencing jQuery on a CDN) by adding an “exclude” entry with the same name. More on this in another article.
  • package-solution.json – controls the details of the app package which is produced by gulp package-solution. This is the .spapp file that you’d upload to the App Catalog to make the web part available when packaging for a non-dev environment
  • upload-cdn.json – I’m unsure if this will be in this form in the final toolchain at release time, but this file provides a convenient way to push files (e.g. your JavaScript bundle) to Azure file storage (and optionally Azure CDN, if you’ve configured that). The file takes parameters which identify your area in Azure BLOB storage so the files get uploaded to the right place, such as:
    • account
    • container
    • accessKey

Bundle.json is an important file, especially as you add multiple JavaScript files and reference more and more external libraries. I’ll dig deeper into this one in a future article, but for now let’s just consider a couple of key points. Here’s what bundle.json looks like in the current tooling:

The “entries” node specifies elements of your solution – for example each JavaScript file which should go into the bundle. Notably you can have 3rd party JavaScript libraries bundled in with your JS code if you like – in fact, you have to explicitly exclude them if you they are used as a module in your solution but you’re referencing a CDN at run-time for example. The key children of the “entries” element are:

Element

Sample value

Purpose

entry “./lib/webparts/cobLatestnewsWp/CobLatestnewsWpWebPart.js” Path to a source file to go into the JavaScript bundle. The “lib” folder is expected to contain JavaScript files, not TypeScript files – any TS files in the “src” folder are compiled to JavaScript files which are output here (think of gulp.dest() if you’re familiar with it).

However, it is not the final bundle of all JavaScript files - that is the one generated into the “dist” folder.
outputPath "./dist/cob-latestnews-wp.bundle.js" Path to the final JavaScript file for the entire bundle. This is what will be distributed with your app and what end-users will hit. 
exclude

"@ms/sp-client-platform",
  "jquery"

Modules used which should NOT go into the bundle (e.g. because you’re referencing jQuery [for example] on a CDN).

Files in the root folder

And finally, we get to files in the root folder. Several important files live here, such as gulpfile.js which is the entry point to the build system – you might choose to extend this to add custom Gulp tasks. The following list details the key files:

  • gulpfile.js – the “top” of the build system, and this defines the Gulp tasks you call from the command-line such as:
    • gulp build
    • gulp bundle
    • gulp serve
    • gulp package-solution
    • gulp upload-cdn
    • [N.B. the Gulp tasks themselves are defined within the node_modules\@ms\ms-core-build\tasks folder]
  • package.json – similar to “packages.config” in NuGet, in that it defines the JavaScript library dependencies (and their versions) used by your solution.
  • tsconfig.json – defines TypeScript compilation settings
  • tslint.json – defines TypeScript style checking settings

Summary

Phew! As you can see there are lots of new files and folders to get to grips with in the SharePoint Framework. Remember that this information is based on an early version of the Framework, and there may be some changes by the time things actually become generally available – I will update this post with any changes I find to keep the information accurate. I’ll also be covering more aspects of the SharePoint Framework in the next few articles. 

Monday, 27 June 2016

Speaking at SharePoint Saturday London - An intro to the new SharePoint Framework development model

imageI’m looking forward to giving my first public talk on the new SharePoint Framework soon – this will be at SharePoint Saturday London, on July 09 2016.

The event will be held at Imperial College in west London, and the sessions and speaker line-up looks great for a community event. I think now is a great time to soak up some information on current developments in SharePoint and Office 365, and along with the other speakers I’ll be doing my best to provide some useful info. Developers in particular are going to have a lot to learn over the next year or so if they want to build with modern pages and client web parts.

Here are the details of my session:

An introduction to the new SharePoint development framework

Microsoft are making big changes to SharePoint sites and pages, and this includes how developers can write code to extend them. There's a new web part framework, and a completely revamped page model which is designed to be fast and lightweight. We'll focus on the developer angle in this session, and take a close look at developing a "client web part" - using Yeoman, TypeScript, Gulp, npm, and node.js, and with some discussion of where JavaScript frameworks such as Angular and React can fit in. I'll also show some preview tools from Microsoft's private "dev kitchen" event to help you visualise the development process. The aim of the session is to kick-start your understanding of where this approach fits, and what you can do now to be productive in this brave new world!

Unfortunately the session won’t be recorded, as the venue doesn’t have those kind of facilities. I’ll be talking more about the SharePoint Framework in other talks though going forward.

Event information and registration

There are still spaces left at the end, and it would be great to see you there if you’re in London or around! Registration is done in EventBrite, and you can get event info and the registration link from the SharePoint Saturday site at this link:

http://www.spsevents.org/city/London/London2016/home

Sunday, 12 June 2016

Develop a client web part in the SharePoint Framework - a walk-through

I covered an intro to the SharePoint Framework in my previous article The new SharePoint development model – JavaScript frameworks, npm, Gulp, TypeScript, client web parts etc., so now let’s now take a closer look at developing a client web part (something most developers will do commonly). We’ll walk through the web part creation process and start to look at the various config files, and also cover things like bringing in a JavaScript library such as jQuery, incorporating CSOM code, and dealing with async operations such as fetching data and so on. I’ll link to separate articles for many of those, since they deserve some decent coverage. We’ll also consider the “what now?” moment, when you have the default code running and need to start adding your own.

Remember – this is simply *sample guidance* at this point to help you prepare. Some details WILL change before release!

I can’t emphasize enough that you shouldn’t take anything in this post as gospel. At this time (May/June 2016), the framework is still not in official preview and things WILL change before most folks get their hands on things. The Framework pieces shown here come from working with the product group at a private “dev kitchen” event. Still, I think there is benefit of having a heads-up of what the development process looks like – most developers I know like to see the moving parts to understand a technology stack, so hopefully this helps folks prepare. Even if only mentally!

Getting started - creating a client web part “project”

Seasoned SharePoint developers are used to opening Visual Studio and doing File > New > Project > and selecting one of the SharePoint templates, even if just an empty SharePoint project, or adding an item to an existing project using Add > New item > SharePoint > Empty element or similar. However, in the SharePoint framework we don’t need to use Visual Studio and the Yeoman Generator is used to get us started with the right files. This works from the command-line, and isn’t tied to a particular code editor such as Visual Studio. So, we need the following installed:

  • The Yeoman Generator itself
  • A template for a SharePoint client web part – this will become available when the tools are released

We open a command line in the directory we want to use, and we will run the Yeoman Generator to drop the right files into this directory. We tell it the name of the generator to use – I have one called “@ms/sharepoint” (from the preview bits I have access to – final names might change). This is effectively a “templated set of files”, and these templates will be supplied by Microsoft (but the community will add additional ones no doubt):

Yo1

The generator will ask me some questions in order to provide the right scaffolding (and note the dev kitchen references illustrating that these are preview tools), and then starts unpacking some files into the directory – especially the “node_modules” folder used for JS dependencies:

Yo3_1

I am then asked if I want to create a client-side web part or full page application  – I’ll select the first option here:

SNAGHTML1afe09d9

After some further information gathering on my web part name and description (highlighted with red box below), some core files are created in the directory (see yellow box):

Yo5_1

The generator completes, and now I have some boilerplate files in my folder, including some “Hello World” code in the main web part file. I can now open these in a code editor, examine them, and start implementing some real code. To open this folder in Visual Studio code (and remember it’s just a folder, not a VS project or anything), I can type “code .” – the dot tells VS Code to open in the current folder:

SNAGHTML378acf80

Now we see our files. The main sub-folders to be interested in are the “config” and “src” folders, highlighted below:

Client web part - initial code - 1_1x

Usefully, this is a working sample and as shown in the last post, I can run this in the local “workbench” environment which has no dependency on SharePoint. To do this, I run “gulp serve” from the command-line:

SNAGHTML378c3a8a

This will execute the Gulp task named “serve”. This is defined in the tooling, and it’s job is to spin up node.js on your local machine to allow the files to be served (as an alternative to IIS) – the default URL is based off http://localhost:4321. A browser window is also opened onto the workbench page, and all this is logged in the console:

SNAGHTML37902c8c

In the browser window which opened, you should see an instance of your web part added to one of the the new style pages. The content displayed will match the boilerplate HTML code which was shown earlier:

SNAGHTML379207b5

If we go to edit the web part properties, we see the single property which the boilerplate code defines to help us get started:

SNAGHTML37947e91

Here's what the boilerplate code looks like for the web part:

** N.B. There is a code sample here but it will not show in RSS Readers - click here for full article **

Early challenges when adding your code

So we now have some boilerplate code and we’ve tested that in the browser. But now the fun starts, and we really need two things:

  • To understand how to start adding code, to implement the specific functionality we plan to build in this web part
  • To understand the various folders and files provided by the SharePoint Framework – in other words, understanding the SharePoint Framework  development model
Adding code

Looking at the boilerplate code, you can see the main entry point to a client web part is the public render() method provided by the SharePoint Framework. A DOM element representing the web part’s container on the page is passed as a parameter, and it’s up to you to output your content in there – using whatever approach you choose. You can see then, that this is a massive formalization of the “JavaScript embed” approach that many of us have been using via a Content Editor web part or Script Editor web part – where basically the web part emits a DIV onto the page, and some companion JavaScript then populates that DIV. The big difference, of course, is that things are baked-in and we now have proper framework support. We get all the things long-time SharePoint developers would expect from a web part framework – support for web part properties, a web part manager object to help manage web parts on a page, context data such as whether the page is in edit or display mode and so on. But we also get other things - support for loading resources such as CSS, a logging framework, and various helpers for common needs (such as making a GET/POST request).

So, it comes down to whatever we build around that render() method. For non-trivial web parts, usually you’ll create additional TypeScript modules so that your core implementation of fetching data (or whatever) isn’t in presentation code, and if you’re new to TypeScript even that can be “fun” the first time! As I was creating my first client web parts, I hit challenges such as these as I was building out my code:

  • How to add additional TypeScript modules to better structure your code (and consuming this code in your web part class)
  • How to add jQuery or other JavaScript libraries to your web part (including TypeScript typings for code completion)
  • How to add CSOM to your web part (again, with TypeScript typings so that CSOM is easier to write)

I plan to cover all those in future articles.

Understanding the SharePoint Framework files and folders – an overview

At this point it’s worth thinking about the files and folders we are working with:

SNAGHTML1997d549

Here’s a cut-down version of a table from my next post – I’ll go into more detail there, but here’s an overview of the folder structure used:

Folder

Purpose

src The place where you add/edit code files
lib Contains “processed” code files which are ready to move into the bundle which is distributed with the app.
dist Contains the final code files which are distributed with your application.

The most important file is the final JavaScript bundle file  [MyWebPart].bundle.js
config Contains a set of JSON files used by the tooling for the build process. In particular to control how your app is packaged - in terms of the .spapp file, JavaScript/CSS bundling and so on.
node_modules Contains JavaScript modules used by your code or the SharePoint Framework. Some may be loaded at run time, but others may be used at design time only (e.g. by TypeScript code).
typings Contains TypeScript typings files – these are used to give you auto-complete (IntelliSense) against JavaScript libraries you are using (e.g. jQuery).

 

Summary

There are a few things to get used to when developing in the new SharePoint Framework. The process to create custom code is different (and doesn’t even need to involve Visual Studio!), and the files and folders SharePoint developers will work with are different. I’ll continue digging into things in my next post Understanding the web part manifest, bundle.json and other key files and folders in the SharePoint Framework (published soon!)

Wednesday, 4 May 2016

The new SharePoint development model – client web parts, JavaScript frameworks, npm, Gulp, TypeScript etc.

In my end-user focused companion post Overview of the new SharePoint – modern team sites, pages, web parts and applications, I talked about the overall set of changes coming to SharePoint, but here I’m focusing on the new development model. I spent time with Microsoft engineers building the framework at a fantastic “dev kitchen” event earlier in the year, and have been playing with bits and forming thoughts since. I’ll cover an overview of the framework, and talk specifically about new building blocks – we’ll focus on client web parts, but will also touch on full page client applications that are also coming. To get started, even developers are visual people so take a look at what you’ll be working with in code terms in the editor (and note this is Visual Studio Code, the lightweight/free code tool that Microsoft now have):

SNAGHTML23253fb7

Whoah, that looks pretty different to SharePoint development as you’ve seen it before! A couple of quick observations before delving deeper:

  • That code is TypeScript code – these files have a .ts extension, and get compiled down to JavaScript by a tool.
  • The project structure is different – the “src” directory is where you edit your files, the “config” directory holds a bunch of config files (that we’ll walk through later) and so on.
  • We have lots of new files. There’s a new web part manifest file (the new equivalent of a .webpart file), new config files to control how bundling works, what happens in the build process, how the solution gets packaged and more.

This is SharePoint development, but not as you’ve known it before!

Key aspects for developers

I’m condensing a lot of information into this list, but I think these are the key takeaways:

  • Client web parts and client-side applications are the new building blocks. As discussed in the last post, these go along with the new page model.
  • It’s a JavaScript world baby! Config files are in JSON, and code is implemented in JavaScript on the client-side.
  • The packaging of artifacts is different! There are new manifest files to learn about (e.g. a web part manifest), and other files such as bundle.json, package-solution.json, upload-cdn.json and more. Gulp tasks are used for packaging.
  • Files for your web part or app can live anywhere (e.g. a CDN, or a website you host) - they don’t have to live in SharePoint. Anywhere that can be accessed on a URL by the end-user basically.
  • The “local development” model is very different – Gulp and node.js are used to host files locally, so you don’t need to use IIS on your local machine. A special “workbench.aspx” page is used to support this.
  • No particular JavaScript framework is mandated – you can use Angular, React, Knockout, Handlebars or whatever you like. The one thing to consider is that React will already be on the page as it’s used by some SharePoint components, so that’s one less file to download the first time users hit your site or customization – useful, but probably not a massive factor for intranets though.
  • You should consider learning TypeScript – at least the key parts such as modules, the type system and so on.
  • You no longer NEED to work in Visual Studio - if you prefer another tool (e.g. because you’re not a “career” SharePoint developer), that’s fine. Other lightweight code editors such as Visual Studio Code or Sublime are now 100% viable options because the packaging of artifacts is different. Even I (as a 10 year SharePoint developer) have started to prefer using VS Code, even if some bits are slightly painful initially.
  • SharePoint Webhooks – these are the new “event receivers”. This is a move towards a more standards-based approach to responding to changes in SharePoint/Office 365.
  • The App Catalog is used as a packaging and registration method – both client web parts and client-side applications are packaged in this way (moving away from the web part gallery we’re used to)
  • Page security needs some consideration – because anything on the page can theoretically access anything else (e.g. by scraping the DOM), that could raise some interesting questions when the user’s mail is shown (for example) and web parts from different sources/vendors are present. To counter this, some operations are blocked in the new framework but other governance may be required. More on this in the future.
Whoah, why all these changes? Do I have to use this new model?

The driver for the changes on the dev side is the new page and web part model. As discussed in Overview of the new SharePoint– modern team sites, pages, web parts and applications, the user experience of working with SharePoint pages and web parts has never been the slickest, and there was a lot of baggage from earlier SharePoint versions. So, since Microsoft were making big changes to the page model it also makes sense to move to modern web development approaches too. Yep, it’s interesting that things aren’t wholly based on MVC, but I’d argue most developers are choosing client-side approaches instead these days.

Do I have to use this new model?
 

No. This evolution of the development model doesn’t replace existing options (e.g. provider-hosted apps, use of JavaScript embed approaches etc.). However, if you choose to implement your solution using the new building blocks (especially the new pages and web parts), then you *do* need to adapt your skills and approaches.

What about TypeScript? Do I have to use that?
 

No, but at this stage I recommend it. The version of the dev tools used by those of us in the preview *is* oriented around TypeScript, and frankly things are just easier if you write TS code and allow it to be compiled down to JavaScript. There may be other options at the time of general availability, but there’s a lot of love about TypeScript once you get used to it. There is a learning curve, but I recommend getting into it personally.

Client web parts – a simple example

Let’s take an overview of a new web part solution. In my example here, I’ve created a client web part which displays the last 5 documents you have created or modified. It runs a search query using the Office Graph extensions (actors and edges) to do this. To keep things (relatively) simple, I’m choosing NOT to use React, Angular or another framework to build on for this first example. – I do use TypeScript though, and I also use TypeScript’s ability to do simple “JavaScript templating” with string literals and parameters though. Here’s the structure of my core files – the main bit to focus on is the “src” directory, since these are the files that get edited:

SNAGHTML17b59b5

I’ll provide a detailed walkthrough of files outside of the “src” directory and their purpose in a post coming very soon. The main code files in my example are:

File Purpose
CobRecentDocsWebPart.ts The core web part implementation (in TypeScript). Has a “render” method which is called by the page framework.
Search.ts Encapsulates the code needed to call the search REST API with the right parameters.
SearchResult.ts A simple class to represent the core properties of a search result that I’m interested in here – Title, Description, URL and file icon.

These files all get compiled down to JavaScript by a Gulp task provided by the developer tooling – this is the equivalent of MSBuild tasks used by Visual Studio to compile DLLs or build WSPs. Here’s what the core code looks like for my web part – from this you can get a heads-up of things like:

  • The TypeScript side of things
  • Referencing other TypeScript modules
  • The web part framework – interfaces and methods, how rendering can work etc.
  • Understanding context – whether the code is executing in the test workbench page or a real environment
  • How to define web part properties

But remember! This is preview code only, and some of these files and the surrounding “dev toolchain” may change between now and when the framework becomes generally available – all discussion at this point is partly just for illustrative purposes!

Once I’m ready to test/debug my code (and usually you’d do this WAY before you’ve added so much of your own code), you can see what things look like on your own machine by running “gulp serve” at the command line in the folder where your files are:

 gulp serve

This does a couple of things, all taken care of by the developer “toolchain” that Microsoft give you (so long as you have the prerequisites such as node.js installed):

  • Runs build tasks to package up your files for runtime – this includes pre-processing any TypeScript and/or React if you’re using those, processing web part manifests, running any JavaScript tests you have, and finally combining your JS and CSS files into one bundle.
  • Starts an instance of node.js to run the JavaScript files in a browser – no SharePoint or IIS needed for this, and there’s a “live reload” facility so that you can edit code and immediately see changes reflected in the browser (the page is automatically refreshed when you save a code file).

This launches a “workbench” page which hosts all your client-side files. So long as Gulp and Node are serving your JavaScript files, you’ll be able to add your custom web part to this page:

SP workbench - add client web part

Your web part will then appear on the page and you can use browser tools for debugging etc:

SP workbench - add client web part 2

You can edit web part properties in the new property pane (I’ve added some custom ones for my web part here):

COB client web part - recent docs - wp props

But I need “real” SharePoint!

Of course, the local workbench page helps you get up and running and build presentation code, but sooner or later you’re probably going to need to see your web part in the context of a real SharePoint environment (e.g. so you can access data, call search/user profiles/taxonomy, or whatever). As you might have noticed, it’s possible to write code to detect the host environment and use dummy data if you’re running on localhost. The Workbench.aspx file provided by the toolchain can simply be uploaded to an Office 365 tenancy though, and again, so long as your files are being served you can add your web part to that page. This time, you can interface with real SharePoint things such as search and test your web part with real data – now my web part finds files in my Office 365 sites:

COB client web part - recent docs

Deploying to production (e.g. deploy to CDN)

Once we’re ready to have our web part used in test or production, we need to move away from the locally-hosted model. Now the JavaScript, CSS and any other files needed at runtime need to live in a location which can be accessed by all users – this can be a CDN, a simple website such as an Azure web app, or any other location of your choosing. You *can* continue to deploy to SharePoint libraries (e.g. Site Assets) if you choose, but now we have an option which moves us away from needing key files deployed (and duplicated) in each site collection – woohoo! You are responsible for providing this location (unless you’re choosing SharePoint)

The preview developer tooling I’ve been playing provides some support for deploying to CDN – this is in the form of a Gulp task, which deploys to Azure BLOB storage/CDN based on a config file. Your manifest files need to be updated to point to your CDN URLs, but things basically are straightforward in terms of deploying assets to a production-ready location.

I’ll delve deeper in other blog posts, but remember that the framework isn’t available yet. You’ll be able to get your hands on things later this summer. 

Client applications 

As well as client web parts, let’s touch briefly on client applications. These are coming later in the year, and in the same way that client web parts offer a JavaScript-only version of web parts, client applications do a similar thing for full page apps. I summarized the flavors in my previous post like this:

  • Page-based apps – an alternative to provider-hosted apps (which remember, are implemented with server-side code).
  • List-based apps – think of these as an alternative to JSLink for transforming the display/edit/new experience around list items

Client applications are implemented purely in JavaScript, but have benefits such as having full “context” and using the Office 365 suite bar etc. There is a framework or scaffolding page in your SharePoint site/Office 365 tenancy, but the main page body is implemented in your JavaScript/HTML/CSS. It’s pretty easy to understand the page-based apps, but the list-based flavor is interesting too. Recently I’ve worked on several mini-applications which provide a custom interface, but store their data as items in a SharePoint list. We built a page which takes a URL parameter for the item ID, and then issued a REST call to fetch the data and build the page around that. One example was a “office locations directory”, where the locations were stored in the list but we provided a nice presentation with an embedded map, particular layout of the data elements and so on. You can consider list-based client applications as a formalisation of that – it will be quicker and easier to build such solutions, almost in a “JSLink on steroids but without the proprietary display template framework”-kinda way. Nice.

Summary and other resources

So, we now have a page and web part model that’s fast, lightweight, simple for end users, and nice to develop on. There are also new ways of solving common requirements around building mini-applications, but in a way which moves away from the very SharePoint-y building blocks such as JSLink and display templates that we had in the past, to a way where the implementer can choose how to build the UI using the approaches of their choice. Additionally, we now have an open development model more in line with the rest of the world, and great support for quick development without the need for Visual Studio and where much can be done without even having access to a SharePoint environment. Kudos Microsoft!

In future posts I’ll provide a more detailed walk-through of the framework and it’s key files.

Other resources:

Overview of the new SharePoint – modern team sites, pages, web parts and applications

We’re heading into a new era with SharePoint at the moment, with BIG changes coming that will bring a new user experience and also a radically different development model. Team sites and publishing sites get the biggest update I’ve seen in years, and pages and web parts work differently - there’s a new web part framework based on JavaScript. These updates will come first to SharePoint Online but eventually to on-premises SharePoint too. You might have seen the “Future of SharePoint” announcements on May 4 2016 - in this post I want to go over the new things and add some thoughts, having been fortunate enough to be looking at this stuff for a while now. I’ll do this over the following posts in this series, with MANY more to come:

Modern team sites – new home page, list and library UI and “SharePoint home”

If you run SharePoint team sites (i.e. most organizations using the platform), you’ll soon have the option of using a pretty attractive new home page provided by Microsoft. Here are some screenshots of what things might look like (click to enlarge):

New SharePoint team site-800

As you’d expect, the mobile view looks good too:

New SharePoint team site - mobile

A more branded site might look something like this:

New SharePoint team site 2 - small

New SharePoint team site - mobile - small

It’s great to see innovation happening in team sites (arguably the core of SharePoint), and this is fairly sexy compared to what we’re used to! The home page has the following features:

  • An area for curated/highlighted content
  • Activity on the site (powered by the Office Graph). This consists of:
    • Conversations
    • Other activity e.g. activity around files in the site
  • Responsive design so the experience on a mobile device works well

The new home page design won’t be forced on you – after all, you may have invested in a tailored experience when users land in a site (either with customizations or just content). Instead, administrators will have the ability to opt-in to using this as the default landing page for the site or not.

The Site Contents page gets a makeover too, now providing quick access to the most active content and showing some high level stats:

Site contents page - activity and stats-800

New document library and list UI

In addition to the new home page, lists and libraries get an update too. We’ve already seen the new document library interface (if you’re on First Release), and a similar experience will be rolled out to lists. There are some nice features, like the ability to drag and drop between groups in a “grouped” view, and this will automatically update metadata. For example, you could drag an item from “In progress” to “Complete”, and the corresponding metadata will be updated. This makes it possible to use a SharePoint list as something more like Trello, or perhaps a sprint planning board in TFS or similar.

More information on the changes to the user interface of lists will come soon.

SharePoint home

This one is nice too. I’ve previously complained about how SharePoint doesn’t really have a “top-level” – you might have lots of team sites and a publishing intranet, but you have to deal with the top-level thing yourself. This is even more fun if you actually don’t have a publishing intranet, but do have lots of team sites. So, Microsoft have acknowledged this by giving a big update to the “Sites” page. This is now renamed “SharePoint” and shows activity across sites you might be interested in (via Office Graph), recommended sites and so on. You can continue to create new sites from here (though as before, these are site collections based on the stock OOTB team site template, and created in a certain place - which might not be what you need), but one big difference is that these now also get an Office 365 Group! The next section describes this in more detail.

HOWEVER – it’s good to put “SharePoint home” in perspective. I can’t imagine many organizations being happy to actually use this as the browser default page across all their users. Most will already have some kind of intranet, and whether the home page shows company news, a social feed, key links or whatever – I can’t imagine SharePoint home replacing that. “SharePoint home replace not the intranet!”, as Yoda might say. Still, I see it as a massive improvement over what was there before and might work great in a smaller firm (click to enlarge):

SharePoint-home-small

Team sites are now Office 365 Groups, and vice-versa!

We’ve known for a while that there would be more harmonization between Office 365 Groups and team sites, and this is what it looks like. Later in 2016, when a site is created from the “SharePoint home” page you’ll actually be creating an Office 365 Group *and* a team site together. This is a big step forward for Office 365 Groups, since previously all you got before was the cut-down OneDrive library which didn’t have full capabilities such as metadata.  So that definitely helps on that particular “what to use when” question. Additionally, any existing Office 365 Groups you have will gain a team site. Clearly some planning work will usually still be required to establish policies and governance on how Groups are used, but at least now things are a lot more enterprise-ready.

Modern pages and web parts – a new page model

At the heart of these changes to team sites (and publishing sites – updates coming to those too!) is the new page model. To support some of the other changes such as the new web part framework, a new type of pages were needed too. Although us SharePoint folks often saw past it, if you sit with an end-user who is completely new you start to realise how clunky the current page edit experience is. Challenges included using the ribbon, editing web part properties and some aspects of adding content into the rich text editor. ALL that has been replaced in modern pages, with the goal being to provide a simplified experience closer to WIX or Medium.

Here’s what you need to know:

  • New “client” web parts – with a completely new development model
  • A new page “canvas” – this is the page framework which provides a simplified editing experience for end-users. There is no ribbon, and the whole process of adding/editing web parts is streamlined.
  • New “client applications”
    • Page-based apps – an alternative to provider-hosted apps (which remember, are implemented with server-side code). Client applications are implemented purely in JavaScript, but have benefits such as having full “context” and using the Office 365 suite bar etc. 
    • List-based apps – think of these as an alternative to JSLink for transforming the display/edit/new experience around list items

These images don’t show the final version of the new page/canvas, but we will have a radically simplified text editor (similar to the Delve blogs UI you might have noticed):

New page canvas 5

The interface to add a new web part to the page is also much simpler:

New page canvas 2

A new publishing infrastructure

In addition to team sites, publishing sites also become much simpler for end-users. As you’d expect, the new page and web part model is used there too, meaning pages should be more lightweight. New web parts will be available, and there will continue to be a method of implementing a custom look and feel on such a site: 

New publishing framework

What all this means for implementers

The introduction of new pages and web parts means that some choices are needed – should you implement your solution with the new building blocks, or stick to what’s out there already? What about solutions that are in use already? Should you migrate to the new page/web part model?

Some of the factors to consider include:

  • New pages exist in a new pages library – they cannot simply be added to the existing ‘Site Pages’ library in a team site for example. This means navigation, roll-ups and so on need to be thought about if you’re considering some kind of migration or “mixed” solution.
  • New web parts can be used in “classic” pages, but not vice-versa. So you can’t expect to use existing web parts (even existing out-of-the-box web parts) in new pages. To ensure new pages can do the things we need, Microsoft will provide a new set of web parts, equivalent to commonly used ones such as the content search web part, content editor web part and so on. These will most likely be simplified versions to align with the overall aim of making things easier for page authors. The aim is to have around 5-10 common OOTB web parts available in the new model at launch time, and then others will come later (including the ability to purchase web parts in the store).

    Note that when a new client web part is used in a classic page, there’s some “joining-up” of the edit experience for web part properties. There’s a single “Edit properties” button in the place where settings usually appear, and the user must click this to show the new style property pane and change settings there. It’s a bit clunky, but it works.
  • The edit experience will be different between new pages/web parts and other page types. That’s the whole idea, but worth remembering if you’re considering a migration or mixed model.
  • The page and development model will be different between new pages/web parts and other page types. As above.

So, it might be sub-optimal to mix the models too much within one site/solution. A simpler approach might be to consider the new building blocks for new sites and development projects, but to leave existing investments as they are. Your mileage may vary though.

What it means for developers (high-level)

I cover this in MUCH more detail in the next post The new SharePoint development model – client web parts, JavaScript frameworks, npm, Gulp, TypeScript etc., but let’s include a high-level view here:

  • If you want to build new style client web parts and/or client applications, you’ve probably got some new skills to learn! The general framework and tooling are new, but web development and JavaScript are at the core and this brings things much more in line with the rest of the dev world.
  • Core technologies include npm, Gulp, a little node.js, the Yeoman generator, and TypeScript. But the good news is you don’t have to be an expert in all these, and when you get your hands on the tooling you’ll see that it takes care of lots of things for you. As ever though, the more you know, the more you’ll be able to resolve any little quirks you hit if/when you need to do something a bit different.
  • There are new config and manifest files to learn about, for example the web part manifest which describes a web part, it’s dependencies on other JavaScript libraries and so on..
  • The “local development” model is very different – you don’t need to use IIS to host files on your local machine, since Gulp and nodel.js are used to serve files instead
  • This “evolution” of the development model doesn’t *necessarily* replace existing options (e.g. provider-hosted apps, use of JavaScript embed approaches etc.). However, if you choose to implement your solution using the new building blocks, then you do need to adapt your skills and approaches

Other bits – SharePoint mobile app and PowerApps/Flow:

But that’s not all. I’ve tried to summarize the key changes for how SharePoint sites will be used (and built) above, but other things that will have an impact are the new mobile app and a new options for integrating your SharePoint sites/data with other tools. Let’s look at both of those briefly:

SharePoint mobile app

The new app looks great – one key pillar I really like is that you’ll have quick access to recently used sites and documents, in a way that will actually work. We’ve already had the OneDrive app, but the SharePoint app will cover team sites and publishing sites too. Additionally there are areas to help with finding people and and key links for your environment as defined by administrators. The iOS app will come first (early summer), followed by Windows Phone and Android:

SharePoint-mobile-app-2

PowerApps/Flow

You might already be familiar with PowerApps, Microsoft’s no-code platform for creating simple business apps which can also work on mobile devices. If not, my article PowerApps – no-code Azure apps which talk to Office 365, SharePoint, SQL and more may help. In a similar vein, Microsoft also recently announced a new service called “Flow” – this helps you take simple actions across common services under certain circumstances e.g. when something changes in SharePoint. It’s commonly described as being a bit like If This Then That (IFTTT) for the enterprise, and can talk to popular service such Salesforce, CRM, SharePoint/OneDrive, Dropbox, Twitter and so on:

Flow templates - 1

I guess I’m particularly interested in some of the ones that can act as simple “event receivers” in SharePoint:

Flow templates

To add a flow related to something in SharePoint, a new “Add flow” button will take you into the simple designer where you can define the steps:

Microsoft Flow inside SharePoint

Clearly this isn’t a heavyweight workflow or forms tool, but it’s quite a nice option for taking simple actions related to things stored in SharePoint (and elsewhere).

Summary

So, big changes all round then! Like many others, I’m hugely excited about the future of SharePoint and it’s great to see the innovation that’s happening. I think the proposition for organizations using SharePoint and Office 365 is getting even stronger, and many of the the gripes and gaps are being addressed. My next post on the SharePoint framework, The new SharePoint development model – client web parts, JavaScript frameworks, npm, Gulp, TypeScript etc., looks at things from a developer perspective.

Other reading:

For developers: