I’m a bit late in posting this, but if you’re still considering whether to attend this conference (London, April 15-17th 2013) then I highly recommend it. As usual, the speaker line-up looks awesome and after missing out last year (due to moving house) I’m really looking forward to it.
I’m giving two talks centred around SharePoint 2013:
DEV 203 - Deep dive into SharePoint-hosted apps (Monday, 1:30pm)
Even though apps can be outside of SharePoint, often it makes sense to leverage what's inside. This session takes a close look at how to use lists, content types and pages within a SharePoint-hosted app - whether there are additional remote components or not. We'll also look at advanced approaches for using a custom site definition, and explain why some web parts can be used and some cannot. In addition to working with the app web, we'll discuss pros and cons of "high-privilege" apps which provision into the host web - potentially bringing collaboration-based solutions closer to the app world.
P&M 306 - Using JavaScript templates to customise the SharePoint user interface (Tuesday, 9am)
Developers often have a need to amend the HTML rendering of SharePoint, especially around lists and list forms, but in previous versions of SharePoint this was not always easy. Many developers resorted to using JavaScript to manipulate the page after it had loaded, even though this was potentially unreliable and unsupported. SharePoint 2013 solves this problem by providing the ability to specify a JavaScript 'template override' for several user interface elements. Even better, these can be applied globally or locally as needed. Come and learn how to transform the SharePoint interface the supported way!
So, let’s say you’re working with the Content Search web part in SP2013 and you’ve got it displaying the correct items. (If not, see my big post on Using the Content Search web part (and understanding search). In the case of this article, I’m searching for items I’ve created and tagged with “SharePoint”. Whatever the query, you’ll most likely be confronted by something like this:
Your client will be thrilled with something so aesthetically pleasing! By the way, the above search is for “documents in my SharePoint farm tagged with SharePoint” - having a thumbnail image shown might not really be needed for documents, but just for fun we’ll talk about both options. In a later post, I’ll cover using the CSWP to roll up some news pages (publishing pages) – these do tend to benefit from thumbnails, but the good news is there’s less work to get them there.
Anyway, if we break down the issues we’d like to fix, we would say:
An image is rendered for each item, but one is not found (so a default image is used)
The fields may not be correct – certainly for the 2nd and 3rd items, a fairly useless value is being fetched from the Office document properties. This is being displayed in the absence of a ‘Description’ field on the list item (N.B. this is actually a great ‘fallback’ feature of SP2013 search – Office doc properties get pulled into the property bag of the list item, and indexed by search if no other values take precedence [via Managed Property mappings]), but in this case, it’s some irrelevant data about who created the original PowerPoint template for the conference I presented at:
The overall formatting (HTML and CSS, especially colors) do not match our desired rendering
There are several settings in the CSWP web part properties at work here:
Thing
Screenshot
Notes
Control Display Template
This determines the overall style of the presentation – it’s effectively the “outer HTML” that is output.
I’d say List will be the most commonly-used value, but it’s great to see List with Paging come for free. Slideshow is an image rotator.
Item Display Template
This dictates the style applied to each item displayed i.e. “the HTML for each item”. Later when we create a new Item Display Template, we’ll see this become available for selection in this dropdown.
Property Mappings
These are very much like ‘slots’ in the Content Query web part – they are a feature of the Item Display Template which is selected (specified in JavaScript). Item templates use generic properties (e.g. ‘Picture URL’, ‘Line 1’ etc.), and you can ‘map’ some specific fields which exist on your list item. The image shows the default mappings. This means that if you only need to get a different field displayed in an existing slot, you can do thiswithout modifying XSL (CQWP) or JavaScript (CSWP).
If you needed 10 slots rather than 5, editing the Item Display Template’s JavaScript controls this.
Note that can specify multiple values specified by a semi-colon (as shown).
Choosing the Item Display Template
As you change the selected Item Display Template in the dropdown, the Property Mappings section in the CSWP tool pane will change accordingly, allowing the mapping for each slot. Once configured appropriately (see next section), “Picture on left, 3 lines on right” can have 3 lines of information displayed:
Whereas “Two lines” of course, will give you no picture and just two lines:
We’d probably say that “Two lines” works just fine for documents – unless you do happen to want thumbnails (e.g. for some key policies/procedures/CAD files/PDFs/whatever). Whilst thumbnails kinda happen by default for publishing pages, the process for other items is useful for illustrating the ‘Property Mappings’ aspect – first let’s cover the possibilities for working with these mappings.
Deep dive - getting the right data coming through
If you haven’t guessed already, those property mappings such as ‘PublishingImage’, ‘Path’, ‘Title’ and ‘Description’ above are Managed Properties in the search service app. If you are rolling up content from multiple locations (Content by *Search* remember!), consider that these mappings allow you to show results from across your farm/tenant even if you don’t have standardized content types with the same fields everywhere. For example, perhaps ‘Description’ could really come from a custom field named ‘Project Description’ in one place but ‘Task Description’ for other items.
The use of Managed Properties means you might need to go down a different route to get your desired values coming through:
Option 1 – no work needed, the CSWP automatically displayed my data where I want it!
Awesome! You’ll see this for the ‘Title’ field and some others (e.g. roll-up images in publishing pages). This is happening because for this ‘slot’ in the selected item display template, the default Property Mapping contains a Managed Property which points to the field on the list item/content type which holds the data. Perfect. If this isn’t happening and you think it should be, check that you have populated some content and it has been successfully crawled – you won’t get anywhere until it has.
Option 2 – use an existing Managed Property and site column, but edit the CSWP properties for a particular slot (to point to this Managed Property). You would do this when the CSWP isn’t looking at this Managed Property by default for this slot:
It’s a big old dropdown, but all you have to do is select the existing Managed Property you want:
Additionally you can add to the semi-colon separated list of Managed Properties, by typing into the box.
Option 3 - use a new site column but add it to an existing Managed Property:
Rare maybe, but you might want to do this if you want the same mapping to be use in different places e.g. multiple CSWP instances, or CSWP and the search results page etc. The process would be:
Add a new column to your list/library/content type. Populate some content and perform a full/incremental crawl.
Find an existing, somewhat generic, Managed Property which represents the thing you are working with – this could be something like ‘Title’ or ‘Description’. Edit the Managed Property so that the corresponding Crawled Property for your field (e.g. ‘ows_COBDescription’ for a field named ‘COBDescription’) is added to the list of Crawled Properties.
Ensure your Crawled Property is higher in the mappings list than others (if using the ‘Include content from the first crawled property that is not empty..’ option. As an illustration:
Option 4 – use an entirely custom Managed Property and site column:
This might be the best approach if there are no appropriate fields in existence, or you really don’t want to pollute any other CSWP instances and/or the search results page with these settings. I used this approach to add thumbnails to my documents when using “Picture on left, 3 lines on right”. The process would be:
Add a new column to your list/library/content type. Populate some content and perform a full/incremental crawl.
Create a new Managed Property, and map it to the corresponding Crawled Property for your field (e.g. ‘ows_COBThumbnailUrl’ for a field named ‘COBThumbnailUrl’).
Edit the Content Search web part properties, so that the Property Mappings for the desired slot contain the name of the Managed Property – here I’ve added my Managed Property (‘COBThumbnailUrlManaged’) to the ‘Picture URL’ slot:
In general, the deciding factors will be things like:
Do I want this change to take effect everywhere? (editing an existing Managed Property [i.e. adding a new Crawled Property to it] could mean that other Content Search web parts/search results show data from an instance of this column)
Is there an existing field and/or Managed Property which could represent this data?
“CAD files/documents with thumbnails (in team sites)” example
Recently I had to implement thumbnails for CAD files for a client demo – to appear in both CSWP results and the search hover panel (I’ll talk about customizing the search hover panel another day). The thumbnails aren’t automatically generated, but could be in the future. At it’s core, there’s an additional column which contains a hyperlink to the thumbnail image (e.g. in a Site Assets library).
I chose option 3 above in this case – since I don’t need to ensure I’m scoping only to one CSWP instance or anything.
Here are the specific changes I made to get my thumbnails and descriptions to come through to the CSWP:
Added two columns to my content type, to contain the thumbnail URL and description
Populated these fields for each item (for the thumbnails, as mentioned the images themselves live in the local Site Assets library)
Item thumbnail:
I created a site column (‘COBThumbnailUrl’), and mapped this to the ‘PictureThumbnailURL’ Managed Property
Item description (for display in the ‘Line 2’ value):
In this case I re-used the OOTB ‘Description’ Managed Property, but added a new column ‘COBDescription’ and included this in the Crawled Properties mapped to that Managed Property
(N.B. There are a bunch of OOTB fields named ‘Description’ in a team site, but most are in the ‘Hidden’ group and didn’t actually seem suitable for re-use)
Once the thumbnail is against that file, it’s fairly easy to surface it in search hover panels (as well as CSWP rollups):
“Publishing pages with thumbnails” example
If the items you’re rolling up are publishing pages (e.g. news pages), then things are slightly simpler – this is because the default mappings of, say, “Picture on left, 3 lines on right” work better. Assuming you’ve configured the query correctly and the pages have a roll-up image set (e.g. a rendition of an existing image), then you’ll probably get thumbnails but no descriptions by default:
In this case, you might notice that my thumbnails aren’t high quality. This is because they are simply generated using the SP2013 publishing framework’s image renditions capability (i.e. a 100 x 100 version of the much larger image displayed on the page itself) – if I produced special images for thumbnails, the result would be higher quality (but more work).
We have no item descriptions so far, only titles. However, if we edit the Property Mappings for “Line 2” and “Line 3”, we can start to bring in other data – here I’m selecting the article byline and also the property which lists which terms this item has been tagged with:
..and now we get the those details displayed by the Display Template:
Next considerations – modifying the HTML/CSS/JS, and packaging/deployment
So far, we haven’t worried about changing the actual HTML which is output. My next articles look at this, for both publishing and non-publishing scenarios (the process is slightly different).
There are also a couple of considerations for working the “developer way” and provisioning these artifacts through a WSP. I’ll also discuss these next time.
Over the past couple of years I’ve been a big advocate of continuous integration (automated builds) for SharePoint projects, and I’ve previously published some resources to help people get started with this. Specifically I have:
A custom TFS build workflow – this is for use with TFS Build. It integrates with some PowerShell scripts I also supply, so that WSPs can be deployed automatically as part of the build. This is my “SharePoint CI starter pack”.
Assembly versioning workflow activity – this is an optional custom workflow activity which will increment the version numbers of your assemblies as they are built in the right way for SharePoint. To simplify things I supply one workflow with this activity dropped in and configured, and one without.
Because Microsoft made some changes between TFS 2010 and TFS 2012, these things do not “just work” for TFS 2012 unfortunately. In fact, migrating a build workflow from TFS 2010 to TFS 2012 is a bit of a nightmare. But, I’ve now updated these with new releases for TFS 2012 so things should be a bit easier for if you work with this stuff.
A while back someone pinged me on Twitter asking for an update, and then some Microsoft folks got in touch to say “can we take ownership of your stuff, or could you update it?” and then last week I was in meetings about a TFS 2012 upgrade at my place of work (Content and Code), and since our larger SharePoint projects use this stuff I knew I had to bump this up the priority list :)
Both new releases are labelled “for TFS 2012”, and instructions can be found on each site. Briefly, you’ll need to download both if you want to perform versioning within your automated builds – if not, you can just use the CI starter pack, selecting the “no versioning” workflow in there.
Release notes
I’ve added release notes on Codeplex, but just to summarize here:-
TFS simple assembly versioning
This is a brand new workflow activity (with a new name - COB.CI.Workflow.AssemblyVersioning), due to the incompatibility between TFS 2010 and TFS 2012. As I say above, you can either drop this into your own (custom) TFS build workflow to have your assemblies versioning automatically, or use my workflows from my CI starter pack.
This has been fairly well tested and is robust.
SharePoint CI starter pack
As before, this contains 2 workflows (with versioning/without versioning) and a set of PowerShell scripts which deploy WSPs etc. If you’re an existing user, you should add the 2 workflow XAML files to TFS, and then edit your existing build definition(s) to use one of these workflows.
The TFS/PowerShell integration to deploy WSPs as part of the build has been well tested. However, at the time of writing I haven’t yet tested including automated tests (unit tests/UI tests) into the build – at this point, I just want to get something out there quickly to support any TFS upgrades of existing users.
Incompatibility issues around TFS 2010/TFS 2012
For completeness, I just wanted to show the errors you’ll see if you upgrade to TFS 2012 and are using my resources (i.e. attempting to use TFS 2010 workflow stuff in TFS 2012). In fact, I think you can expect to see errors if you open any build workflow after the upgrade – whether you use my stuff or not. You’ll see something like:
System.IO.FileNotFoundException: Could not load file or assembly ‘Microsoft.Team.Foundation.Build.Client, PublicKeyToken=b03f5f7f11d50a3a’ or one of its dependencies.
System.IO.FileNotFoundException: Could not load file or assembly ‘Microsoft.Team.Foundation.Build.Workflow, PublicKeyToken=b03f5f7f11d50a3a’ or one of its dependencies.
Various other error messages all related to versioning.
Yes, it seems many TFS 2010 build workflows appear to have version numbers stamped into the XAML. I understand this is due to an unfortunate bug in VS 2010, specifically Team Explorer - this KB article has more details and a hotfix. However, chances are that (like mine) your workflows already have ended up with the version numbers in them by now. So, it’s certainly more painful than it should be.
You have two options to make your build workflow run in TFS 2012:
Rebuild it from scratch in VS 2012/TFS 2012 – that’s what I did in the end with my CI starter pack workflows
Attempt to clean the XAML:
I tried this by hand several times and failed. The workflow failed to display in the designer, and there is precious little diagnostic information.
The Jason Prickett post linked above also has other variations of error messages you might see.
Related error messages you might see if you use my resources:
Again for completeness, here are some other errors you might encounter if you’re using my resources but are “doing it wrong” according to TFS 2012. This is what you might see if you open a workflow with my (TFS 2010) assembly versioning activity after upgrading to TFS 2012/VS 2012:
You’ll need to fix the workflow to use the latest version of the activity if so.
And here’s what you’ll see if you try to drop my old (TFS 2010 version) assembly versioning activity into a TFS 2012 workflow:
Conclusion
Upgrading to TFS 2012 brings some pain if you do automated builds, whether in relation to SharePoint or not. For any SharePoint folks who use my resources, I’ve upgraded these to hopefully save you some pain. Please leave a comment on the Codeplex site if you hit any issues with them.
Changing the user interface of certain bits of SharePoint has always been somewhat challenging – at least, if we want to avoid the SharePoint Designer route and produce something which is packaged, deployable as a WSP and repeatable across many sites. Not to mention source-controlled. A common area of customization is the user interface around SharePoint lists - when it comes to this, there are a few “flavours” of common requests:
Custom list forms – e.g. a more “designed” form for adding and/or editing items
Custom behaviour for a certain field - e.g. a “UK address” field which allows postcode lookup
Custom rendering of a list in display mode (e.g. the ‘All items’ view) – a good example of this is conditional formatting e.g. “Add a red background to the row if the value for ‘Days overdue’ is over 5”
As you might guess from the article title, it’s the third scenario that this article focuses on, for SharePoint 2013 specifically.
Earlier versions of SharePoint gave us a couple of approaches for fulfilling some of these requirements. We could create a custom field control (item number 2 above) or modify the XSLT of an XsltListView perhaps (number 3 above) – a method that was new in SP2010 of course, since we previously had CAML rendering for list views (yuck). SharePoint 2013 brings a new JavaScript-based approach for modifying the UI of fields and lists, and many more things too. You’ll often see this referred to as “JSLink”, because many things in SP2013 have a new JSLink property which you use to point to your JavaScript file. Here’s a list of SharePoint objects which you can modify in this way:
Lots of interesting possibilities there – all the field types are obviously represented, but binding UI changes to a:
content type
form
view
list view web part instance (rather than to the list/view itself)
..could all be extremely useful.
[As an aside, if you came to this article looking for a solution to the 1st scenario above (modifying forms), you’ll probably be interested that SPForm has a JSLink property. However I imagine there’s still a place for providing entirely custom forms as opposed to making relatively small changes with JavaScript. The old approaches still apply here - modifying list forms is easy in SPD, but requires a bit more thought in the Visual Studio world. In that case, we’d probably want to NOT edit the original .aspx, but instead provide a different .aspx file and update the SPList.NewFormUrl and/or SPList.EditFormUrl properties to point to it (e.g. in a Feature receiver).]
Implementing changes to list rendering with JSLink
For a recent SharePoint talk, I wanted a fairly dramatic example of changing the UI of a list (when looking at list items). So, my example showed changing the list from this:
..to this:
There are some slightly nasty scrollbars in these images, but that’s just because I’ve reduced the size of the window for my screenshots - normally the accordion looks great. As you might imagine, I get a nice sliding “expand and contract” experience when I select different items:
If you’ve worked with it before, you’ll immediately recognise this as the accordion from jQuery UI. In many ways, it’s a nice example, since although YOU are unlikely to need/want to use jQuery accordion for your “customizing a list” needs, it uses custom CSS, images and JavaScript to provide rendering which you probably ARE going to need. So, I’ll walk through most aspects of this process, but as you’ll see only around 20% of the work relates to the JSLink stuff – the rest is just scaffolding. We’ll cover the JSLink specifics first.
The full Visual Studio project can be downloaded at the end of this article.
The important bit (part 1)– how to hook up your custom rendering with JSLink (e.g. call into jQuery UI accordion)
The first thing to say is that when we want to change the rendering of a SharePoint list, it’s actually individual views (SPView) we’ll be working with. The SPList class does NOT have a JSLink property, which makes sense given this is all about presentation. Getting SharePoint to “see” your custom rendering will probably depend on what you’re doing:
Scenario
Approach
Creating a new list
Specify the path to your .js file in the schema.xml file for the list (specifically within the declaration for the view)
Modifying an existing list
In code, update the SPLink property (i.e. SPView.JSLink) to provide the path to your .js file (using PowerShell/server-side/client-side API as appropriate)
This bit only ensures your .js file is referenced by the page. You also need to make sure it has the right contents to register your display templates – we’ll come to that in a second.
In my case, I’m creating a new list – it’s worth noting that I’m assigning a list type ID of “11000” – we’ll use this later with respect to JSLink:
When developing for SharePoint 2013 onwards, when a list is created in Visual Studio the JSLink element for any views will contain “clienttemplates.js”:
..but we should change that to our custom JavaScript file which has our display template implementation (you’ll see me deploy this later):
Our file will then be loaded when this view is requested (i.e. SharePoint will add the .js file to the page). But that’s not enough – we now have to think about what JavaScript is needed to actually register the templates. This is done by specifying two properties of the list views to match:
OPTIONAL - the BaseViewID property (e.g. BaseViewID=1 for a standard view like “All items”, but you could use another ID for a custom view)
The ListTemplateType property (for a new list, you’ll be specifying a unique integer value e.g. 10000)
So, it’s usually a combination of THREE controls overall which dictate how rendering is applied to a list view – the JSLink path, BaseViewID and ListTemplateType. Although BaseViewID appears to be optional, it seems sensible to set it to avoid unforeseen problems with Explorer View/Datasheet View etc. So for a given list, if you wanted one view to have custom rendering and one to have default rendering (even though they share a BaseViewID), simply ensure the JSLink property for the default one is NOT set to your custom .js file.
I also note that it appears possible to specify multiple values in a JSLink path – I haven’t tried this, but I see things like <JSLink>mquery.js|contentfollowing.js</JSLink> (note the pipe character) within out-of-the-box files under the 15 folder.
The important bit (part 2)– what your JavaScript should look like:
Here’s my full AccordionListView.js file specified in the JSLink property for my view – I supply a header and footer and then a JavaScript method (function pointer) to execute for each list item. Notice some context gets passed to this function, including details of the list item:
As you might be able to infer, the jQuery UI accordion expects a H3 and div element for each item – so that’s what my template does, in addition to actually calling the accordion() method.
An issue with JSLink and the Minimal Download Strategy (MDS)?
Continuing my recent tradition of discovering slightly strange behaviour around what I’m writing about, this week is no exception. In my testing, I noticed that if a list has multiple views and the user switches between them, the client templates specified by JSLink do not get applied if MDS is enabled (even though the debugger shows they are called). I’m hoping that I’m doing something wrong, but I can’t rule out a bug in SharePoint 2013’s MDS framework at this stage.
If anyone sees this/has any info, please leave a comment :)
Integrating CSS/JS (such as jQuery UI) into the solution
OK, so if all you wanted to know about was the JSLink aspect, we’re now done covering that. Hopefully that was useful. The remaining bits of this article will cover the “scaffolding” aspects of my particular jQuery UI accordion example – integrating jQuery/jQuery UI, and so on.
My first step was to go to the jQuery UI Download Builder and download the files (plus jQuery too if you don’t have it already). If you haven’t done this before, you can basically configure some options (e.g. styles, behaviors, jQuery UI widgets to use) to build a package for download – this means you get a smaller package than if you were opted for all styles and all jQuery UI components. You’ll therefore have lower page weight and better performance, than if you were using the full set. However, I had an issue where my solution would only work with the full jQuery UI file rather than the one I “built” – if this was production code I’d stop and resolve this, but for demoware I was OK with it.
When you get your download and crack it open, you’ll have some CSS and JavaScript files:
I chose a grey-ish theme called “overcast”, and within the CSS folder come a stack of images – we’ll need to integrate all of these files into our VS project:
In my project, I created a “Site Assets” folder to house my supporting images, CSS and JavaScript – since I’m working in a sandboxed solution, I need these to go into the content database and in my case the contents get deployed to the SharePoint library of the same name. After copy/pasting these files into my Visual Studio project, I get this:
..and, 0f course, Visual Studio is kind enough to detect the new files and generates my elements.xml file accordingly:
The next step is to ensure the site we’re deploying to references these files – we’ll deal with JavaScript first and then CSS.
Add jQuery/jQuery UI to the page (in this case, every page in the web)
Here we need to ensure jQuery and jQuery UI are added to pages in the site. Since we’re not deploying a custom master page (and don’t want to use the Content Editor web part to add JS to the page), we’ll need to use either CustomAction + ScriptLink or a delegate control going into ‘AdditionalPageHead’. In my case I’m choosing the former so that my solution works in the cloud (as a sandboxed solution):
Notice that I’m referencing a non-minified version of jQuery UI – that was the issue I said earlier that I would resolve for production use. The next step is CSS.
Ensure the jQuery UI branding/CSS is applied
Since we’re not using a custom master page, we’re just using the AlternateCssUrl property of the web to ensure our custom CSS file is referenced by our pages:
Our site should now receive all branding and JavaScript dependencies – so if we were to deploy a static page with the correct HTML for a jQuery UI accordion, then it should work fine. However, we want ours to serve as the UI for a SharePoint list, so we’ll now create that list. In the final step we’ll do the actual JSLink work – this will change the rendering of the list at run-time.
Create the list we will use with JSLink
Nothing special here and I won’t go into every step – but it is worth mentioning that creating a list via Visual Studio 2012 is much easier than before:
..then add whatever columns your list needs (here I’m adding them direct to the list rather than to a content type):
As we mentioned earlier, we are allocating a new list type ID (the value used was 11000) for the underlying template for this list – this is important since, in this “new list” scenario, we’re using a combination of this and the BaseViewID for the "All items” view to hang our JSLink customizations off:
Finally I add some dummy data (some old blog articles) to my list so each time I deploy/test, I don’t have to manually add list items:
SharePoint 2013 brings some great possibilities for customizing the user interface, and JSLink/client-side rendering is central to much of this. The great news is that it’s all cloud-friendly, so these techniques can be used in Office 365. The solution above was implemented as a sandboxed solution, and the only code was really to set the AlternateCssUrl of the site. As a result, it would be fairly easy to move all this to a SharePoint app if needed.
Hopefully this has been useful in understanding how JSLink/client rendering might be a valuable tool.
Sidenote - although this article focuses on search, I’ve tried to generalize slightly so that it may also be useful for:
Understanding the pattern for working with *any* SharePoint 2013 REST API (e.g. social, querying lists/libraries etc.) – the URL used for the REST call is the main piece that would change
Seeing (broadly) how to process REST API results formatted in the two available formats - JSON or XML
Understand how to call a REST API from both a SharePoint-hosted app (JavaScript) and a cloud app (provider-hosted or auto-hosted – C#)
Briefly discuss using CSOM to call search instead of REST
Often it’s valuable to leverage SharePoint services such as search, user profiles, Managed Metadata and so on from an app. With search specifically, an example I’ve been giving recently is my ‘learner’ time-tracking app, which displays a ‘Related content’ bar with links to documents and other things which relate to completing timesheets:
My search query is somewhat simplistic (I’m currently just searching for any documents and pages which contain the term ‘timesheets’), but it’s easy to imagine that with some refinement we could return only highly-relevant items. A nice approach in real-life could be to have a configurable value to allow each client (who installs the app) to edit/store the search query that works best for them. Regardless, I think the main point is that search can be a valuable ‘bridge’ between an app and the rest of the SharePoint environment. In my example, consider that:
As the app vendor, I have no idea what your organization’s timesheet policy/guidelines are
If I can bring in some of your content around this, users will have more success with my app, and that’s good for everyone
In terms of implementation, I’ll now talk about the basics of accessing search via REST, and also how to do this from an app.
Broadly, the process is:
Identify (and test) the REST URL to use
Deal with app authentication if necessary (i.e. you’re calling from outside SharePoint, e.g. a cloud app)
Add a Permission Request to your app, so that a request to use search is successful
Develop the success handler code to process the results
Accessing search via REST (_api)
The beauty of REST is that it’s just a URL, and you can test/refine it in the browser before writing a single line of code. From SharePoint 2013, such a URL takes the format:
[site]/_api/[name of SP API]/[name of operation]?[parameters]
So in the case of search, a basic query can be executed with the following:
[site]/_api/search/query?querytext='timesheets'
If such a URL is entered into the browser address bar, then you should see an XML response like this – you’ll find some interesting properties, like those I’ve highlighted in the images:
Common operations
So far we’ve just looked at executing a basic query where the query text is specified. Here are a couple of other common operations you might want to do:
Operation
Sample REST URL
Use a search Result Source (i.e. a scope)
/_api/search/query?querytext='search term'&sourceid='B09A7990-05EA-4AF9-81EF-EDFAB16C4E31' (this example is to search the ‘People’ result source)
For a more comprehensive reference, the best source I can find is the SharePoint 2013 Search REST API post in the ‘Search Space’ blog by ‘nadeemis’ on MSDN – kudos to the author.
Working with the data
As with other SharePoint REST operations, you can choose to have the data in XML or JSON – and JSON might be preferable if you are in JavaScript. As I’ve mentioned before, this is accomplished with the “Accept” header – specify one of the following:
Your code which processes the results will need to deal with JSON/XML as appropriate. In either case, you’ll usually have some drilling down to do to get the valuable data – for example, the main result rows are in a property called jsonObject.d.query.PrimaryQueryResult.RelevantResults.Table.Rows.results (where jsonObject is the object which was parsed from the JSON string). And when you’re iterating through the results, if you want the URL for the item, then you can get that from the cell at index 6 with result.Cells.results[6].Value. The title is at index 3, so result.Cells.results[3].Value. These are basically just the things you work out with a JavaScript debugger, or from a code sample like mine below.
Calling the search REST service from the JavaScript Object Model (JSOM)
In this sample, I have a web page which has a DIV waiting to output the search results:
<div id="related-content-results"></div>
If I’m in a page running on SharePoint (including a SharePoint-hosted app), I can use SP.RequestExecutor (as opposed to a jQuery.get() or jQuery.getJson() for instance) and this simplifies things by meaning I don’t have to worry about adding an OAuth header to my request. So, this code would look like:
The URL we’re using is the app web, rather than needing to reference the host web somehow – this is fine for search (but isn’t for accessing data in the host web, for example)
We need something to iterate over the result items within the JSON – jQuery’s each() operator is a good choice here
There are some ‘magic numbers’ in processing the JSON results to get to the values e.g. result.Cells.results[6] to get to the URL of the item. That’s just a reflection of the data structure SharePoint returns, and you’ll see the same if you’re working with XML instead
Calling the search REST service from a cloud app (provider-hosted/auto-hosted) in .NET CSOM
Accessing the search REST API (just like any SP2013 REST API) in other contexts can be a bit more involved. The main consideration is that if your page isn’t on the SharePoint server, you’ll need to deal with OAuth and adding the ‘Authorization’ token to the REST request. If you’re in C# (i.e. the .NET CSOM), then the TokenHelper class automatically added to your Visual Studio project simplifies this – you’ll need to use a HttpWebRequest object (or similar) to call the REST URL, and then process the response. Since I’m on the server this time and have access to .NET, XML could be a better choice here:
And just for clarity, here’s my ASPX page which renders this – as you can see, I’m using an ASP.NET ListView control to put the results on the page:
Sidenote: Other contexts – calling SP2013 REST APIs using jQuery in a cloud app
As I worked through these samples, I realized that another useful approach would be calling the REST API client-side from a cloud app (in an AJAX way). In this case, we’d probably be coding in jQuery, and would need to deal with obtaining an OAuth token and adding it to the web request – but in this case, we don’t have TokenHelper to support us. Sure, you could perhaps wrap TokenHelper in a WCF service and call that from jQuery, but that would mean an inefficient process of multiple web requests (firstly to the service, then the exchange to get the OAuth token, and finally the call to the SharePoint REST API).
Far better would be to eliminate the extra call and just directly obtain the OAuth token and call the REST URL directly from jQuery – but some work is needed to facilitate this. I’m sure someone will look at this soon as I expect it to be a common pattern for SharePoint 2013 app development.
In order to be able to call the search REST API from an app (of any hosting type), the app needs to be granted permission to use the search service – this doesn’t happen by default. Like any permission which relates to “core SharePoint” rather than the app itself, the developer must specify the requirement in the form of a Permission Request, and the administrator installing the app must agree to granting this permission at install time.
So, the developer needs to add the following to the appmanifest.xml file:
Note that the value specified for the Right attribute gives a very descriptive string reminding you what’s being asked for – although most actions within an app run “as the app’s identity” (i.e. as the ‘app principal’), the search will instead be performed as the user (with the associated security-trimming).
In the Visual Studio designer for the appmanifest.xml file, this looks like this:
And, of course, when someone is installing (or upgrading) the app, they’ll see search listed as one of the permissions that the app requires – and can make an informed decision on whether they are OK with this:
An issue for on-premises deployments?
So everything above works great for Office 365 environments. However, I see something weird when I deploy an app which uses search to my local developer environment. I haven’t yet tested the app against another on-premises environment, but I’d be interested to hear if anyone else sees this – the app in fact cannot be added, and I see a message saying “Sorry, only tenant administrators can add or give access to this app”:
As a concept, I understand “tenant administrator” to apply to Office 365 environments but not on-premises
So, does anyone else see this? Am I missing something? Please let me know in the comments.
Finally, it’s worth pausing to consider – that fetching content via search across an entire SharePoint environment is a fairly elevated level of permission. The app is basically being given access to content created within your business, and anything in the search index is available. Clearly an app could potentially do some ‘interesting’ things with this, so it might be prudent to do some evaluation of such apps before blindly installing.
Conclusion
Calling into search from an app can be valuable, and here we’ve looked at how to do it from both a SharePoint-hosted app (in JavaScript) and a cloud app (in C#). The template used here for making such REST calls can be applied to many other operations – ultimately the pattern is to identify the URL, deal with authentication if outside SharePoint, ensure a permission request exists, then drill down in the JSON/XML response to get to the valuable data.
I seem to have been doing a bit more speaking than writing recently, and I just wanted to publish the decks for attendees (or anyone else interested) to find. The two talks were:
Getting to grips with SharePoint 2013 apps (SharePoint Saturday UK)
What’s new in SharePoint 2013 for devs (SharePoint User Group UK)
There’s around 50% overlap in the decks (mainly because apps *are* such a big part of what’s new in SP2013 for devs), but the “What’s new” talk covers other topics also. I’ll generally be blogging in detail about most of this stuff (or have done already in some cases), but here are some more detail if you’re wondering if anything in the slide decks could be useful to you:
3 key decisions to make on apps (for any company implementing SharePoint 2013)
App development fundamentals/app hosting options
Apps and SSL
How users obtain apps
Developing SharePoint-hosted apps
Developing cloud apps
Possible reasons to need a cloud app
5 lessons learnt developing apps
Side note - I’ve given a similar talk a few times now, but each time the session has 30-40% of new stuff in it as I learn more and form deeper opinions. Frankly the topic is so deep that I could imagine speaking/writing about it regularly for the next 2 years with deeper and better info each time.
But, I don’t particularly want to focus on apps exclusively – there are too many other interesting things to write/talk about. Which is why I included a Display Templates/JSLink demo in my “What’s new” talk.
SP2013 developer investments, by order of potential impact (my view)
Tooling enhancements (e.g. list/content type designer, IntelliTrace etc.)
Apps
Definitions
Official Microsoft guidance on apps
3 key decisions to make on apps (for any company implementing SharePoint 2013)
Display Templates (aka JSLink, aka Client Side Rendering)
Demo showing a standard SharePoint list being rendered in a custom way (using a jQuery UI accordion)
Managed navigation
Cross-site publishing
I learnt a few things putting the demos together for these talks, so have some detailed blog posts in the queue. Thanks to all who came to these talks, and asked questions/gave feedback etc. – much appreciated :)
There is an interesting behavior in SharePoint-hosted apps (or apps with a SharePoint-hosted component), which many app developers will run into at some point. I mentioned it in an earlier article - at the time, I said that it was a bug in the Preview version of SP2013 which would presumably be fixed in the RTM release. But it hasn’t! So, developers need to be aware of this issue and depending what your app does, potentially write special code to work around it.
Sidenote: a future Visual Studio update may deal with this problem for you, by injecting code similar to my workaround below (if this happens, I will update this article here). For now, however, developers must deal with the problem themselves – the relevant Microsoft folks agreed more awareness of the issue would be a good thing in the interim, so this article is my attempt to help!
These are the circumstances where you’ll need to care:
Your app is SharePoint-hosted (or at least has some part of it which is SharePoint-hosted, and therefore uses an app web – maybe it’s predominantly a cloud app [auto-hosted/provider-hosted] but uses some lists/pages in the app web)
Your app accesses data in the host web, and uses the SPHostURL querystring parameter for this
The issue
To ‘enter’ an app, an end-user clicks on the app’s icon in the Site Contents page – I think of this as the front door to the app. When they do this they hit a system page called appredirect.aspx, which (unsurprisingly) redirects them to the app’s home page – as it does so, it passes some information as querystring parameters in the URL to the app, including:
SPHostUrl – the full URL of the host site (i.e. where we have just been redirected from)
SPAppWebUrl – the full URL of the app web (i.e. where we are being redirected to)
SPLanguage – the supported locale of the app
SPClientTag/SPProductNumber – other tokens used to identify the app
Often, the app’s code will make use of these. For example, in my time-tracking application I use SPHostUrl in some JavaScript CSOM code, because there’s a list in the host web which I read some information from. All these parameters are correctly populated when the user is redirected from the host web into the app. However, once in the app, if the user navigates around some pages there and uses the in-built breadcrumb link to return to the app’s home page – the parameters are put back in place by the app framework, but theSPHostUrlparameter now contains the wrong value. Specifically, it now has the value for the app web URL (also in the SPAppWebUrl parameter), which can cause all sorts of fun for your code!
Before we discuss workarounds, let’s just consider how this might manifest itself in your code.
Possible symptoms of the bug
Symptom 1 – asking for the lists in the web gives the ‘wrong’ set:
Let’s say you had some code to fetch the lists in your host web (i.e. end-user data). Using the Developer Site template, I see this if I print them out to the screen (this code is taken from example 3 in my previous article):
..but when I move around pages/lists in my app, when I come back to the page with this code I now get a completely different set of lists!
Symptom 2 – referencing a list by name results in an error:
Similarly, let’s consider some ‘real’ code in my learner app – this asks for a list called ‘Utilisation targets’ with the following line of CSOM code:
var targetHoursList = hostWebContext.get_web().get_lists().getByTitle('Utilisation targets');
Once I’ve got the list, I query for an item associated to the current user and ultimately display the value of the ‘Target’ field on the page:
However, when I navigate around my app and come back to the app’s home page, I find the same code gives an error and I’m suddenly in the ‘failure handler’ method for this code – this uses a simple JavaScript alert to display the error to the user:
..and for clarity, here’s that code:
function onGetTargetHoursByRESTFail(data, errorCode, errorMessage) {
alert('Failed to get host site. Error:' + errorMessage);
}
As you can imagine, this behavior can be quite confusing to the developer! And, of course, there’s a whole range of potential symptoms in addition to the two I’ve listed here – it really depends what your code is doing. What’s that saying? If it looks like a bug and feels like a bug…!
The workaround
Since the SPHostUrl value is correct when the user initially enters the app, the workaround is to save the value at this point and refer to the saved string from then on. You could either save it client-side in a cookie, or perhaps use some server-side persistence with an appropriate key. My learner app takes the JavaScript/cookie approach. First we need some helper methods to get/set the cookie - there are many samples on the internet, so either find a nice jQuery plugin or use whatever else does the job. I’m using the code below to set a session cookie, which includes a jQuery plugin to simplify actually getting the querystring parameters from the URL (apologies for lack of credit, but thanks/kudos to whoever the authors are):
EDIT JAN 2013 – updated this code so that cookie has a PATH and did some tidying:
Then in my sharePointReady() method, I detect if this is either the first entrance (and grab the host URL from the querystring/store in cookie) or a subsequent page load (and use the value stored in the cookie) accordingly:
Hopefully this illustrates possible workarounds. In production, you may want to think about the server-side option or least encrypting the value if using a client-side cookie – that way URLs to the SharePoint sites which use the app aren’t hanging around in plain text on lots of client machines. Of course, vendors and individuals who supply apps to the ‘public’ via the Store should particularly bear this in mind.
Personally I find it weird that this one got through to the released version of SP2013, but there you go. I guess it’s not too catastrophic to work around it once the issue has been identified. In summary, an app which uses the SPHostUrl parameter is likely to run into problems, but some custom code to persist the initial value can be used to work around this.
As I say, if Microsoft do provide a fix I’ll keep you posted and will also update this article.
I'm a technology leader with over 25 years' experience of working with technology to transform how organisations work, with the last decade in leadership roles. I'm the CTO at Advania UK and while more strategic these days, I have a background as a deep hands-on technologist - managing development teams, leading Microsoft 365 and SharePoint delivery, and writing code. I've held Microsoft MVP status for the last 18 years and regularly speak at Microsoft and industry events. In technology terms, I mainly focus on AI these days, with some Azure, Microsoft 365, SharePoint, and Power Platform.
