SharePoint 2013 – my view on what’s new (particularly for developers)

So the SharePoint 2013 (previously known as ‘SharePoint 15’, the internal name) public beta is finally here. And that means that MVPs, TAP participants and other folks with early access are no longer bound by their non-disclosure agreements and can now talk about the product publicly. No doubt there will be a flurry of blog posts, but I wanted to write up my thoughts on what has struck a chord with me in the next version – partly because I have friends and colleagues who might look to me for this information, but mainly because it helps me crystallize my thinking on some of the new aspects. This started as a “developer perspective” article, but hopefully also gives a sense of what the new version brings for everyone.

If you’re a technical person, my view is that developers have a much bigger learning curve than IT Pros in this release. I might get flamed for that, and certainly IT Pros who need to deal with very large scale or need to know low-level detail on infrastructure topics might disagree, but that’s my view and I’m sticking to it :)

So this post represents my list – not in any order of importance.

Social

So it’s pretty interesting in the light of the Yammer acquisition, but yes, mature social capabilities are actually native in SharePoint 2013. Of course, they’re not quite the full hit that Yammer and Newsgator are, but my guess would be that the core SharePoint functionality will now meet many folks’ requirements – for the current generation of social intranets at least. I saw Steve Ballmer’s comments about the viral Yammer model too, but frankly I have to think that a big driver for the acquisition was to remove the “compete” element and bring Yammer into the fold.

The newsfeed (as it is now referred to - no longer “activity feed”) is much richer, and supports many of the enhancements I helped build for a client running SharePoint 2010 (see my post Extending SharePoint 2010 social capabilities). The newsfeed is now fully “two-way”, meaning items can be commented on, liked, and so on – so it’s a now a full Facebook-style feed:

As you might be able to tell from the image, some other capabilities include:

• @mentions
• #hashtags
  • These are searchable, and can be followed. Effectively the new hashtag bit has been merged with the existing SP2010 functionality of being able to follow a Managed Metadata tag.
• Ability to ‘follow’ a site or specific documents
  • This is very similar to the ‘subscriptions’ feature we built for SP2010. Effectively, you’ll see updates about this content (site or document) in your newsfeed
• Groups:
  • In addition to posting to everyone, I can opt to post only to the members of a specific team site – which could be a specific community with the company
• Ability to post links and pictures (with thumbnails automatically generated/used in the feed)
• Document/media preview from within the feed
  • No need to open a new window or be taken out of context to get a feel for the document/video/whatever
• Autocomplete:
  • The UI feels highly-usable – when I type @ to get a person’s name, or # for a hashtag, I get an autocomplete box which gives me categorized suggestions. In the case of an @mention, it initially gives me only people I’m following (left image), but as I type more and no results are found, it expands the search to all people (right image):
    
    

There is also a ‘team’ newsfeed for keeping up with what’s happening in a particular team site. So effectively SharePoint 2013 has just about all of the social features (and a couple more) we built for one of my old projects (see Extending SharePoint 2010 social features). I’m happy to see that the feature set looks good.

As I’ve predicted before, the API has been substantially re-engineered and a lot of the cruft I talked about in my “how we did it” post has gone away. You’ll see an option in Central Admin (in My Site settings within the UPA) to enable ‘SharePoint 2010 activity migration’.

Everything is an app

Certainly an interesting move on Microsoft’s part - but now if you create a couple of lists in SharePoint, in the SharePoint UI that’s “an app”. I can understand where Microsoft are coming from – not every new user understands what a SharePoint list, or a view, or a content type gives them, but frankly even my parents know what an “app” is. Still, there will be a level of confusion for users familiar with earlier versions – and folks may need some hand-holding around this. But when you think about the future and the next 2 versions of SharePoint (e.g. to “SharePoint 2019”), maybe it’s a good thing and these are concepts which should be switched earlier rather than later.

Of course, developers can create custom apps – that’s really the main point here, and it’s important since some of the challenges around getting code into an enterprise environment have arguably been addressed (more on this later). Here’s what the experience can look like – the site owner can install an app from the ‘Your Apps’ area:

From then on, Joe User can access the app from the All Site Content page:

With apps that give a full page experience, when he/she clicks on the app, they get taken to a completely different location which has a link back to the site which hosts the app. This is effectively the start page for the app:

App marketplace – plus remote apps

The big news is SharePoint finally gets an “app store”, meaning it’s far easier to add small solutions to SharePoint which fulfil a particular need (e.g. a timesheeting app, some social extensions, whatever). Given the success of this model in the consumer phone space, and it’s integration into Windows 8, it would have been bizarre for SharePoint 2013 not to have this also. It’s pretty revolutionary, since several things come together to make it much easier to get a 3rd party customization onto a SharePoint environment – whether the person trying to acquire the app is a team site owner somewhere, or the core SharePoint admin team:

• No need to have server administrators be involved with deployment/activation
• Integrated payment/procurement:
  • Got budget? Well if the facility is enabled by server administrators, all you might need is a credit card! Of course, more streamlined payment options (e.g. set up company account details, pay with that) are also possible, though management teams everywhere will be pleased to hear there are a number of governance controls in this area
• Regulated (by Microsoft) marketplace:
  • In the same way phone apps have to go through a certification process (involving checks on performance, legality, use of user interface controls etc.), the same will apply to SharePoint apps. It only gets into the public marketplace if all those boxes are checked, and many of the checks are performed by humans
• Corporate marketplace option:
  • In addition to the public marketplace, SharePoint 2013 provides a framework to have a corporate marketplace – in other words, an internal app catalog where only apps approved for use within the organization are added. Whoever owns the SharePoint platform controls whether the public and/or corporate marketplace are enabled. In the case of SharePoint Online, this is decided on a tenant-by-tenant basis
• App requests:
  • When governance controls are enabled, SharePoint provides a mechanism for end-users to “log a request” for a certain application from the public marketplace. Administrators can then track which apps are commonly requested, perform some internal validation, and then make them available in a controlled way.
• Safety:
  • There’s no point in having apps if they can cause harm to the SharePoint environment. Any individual ‘rogue’ app should not compromise the SharePoint environment as a whole. Of course, Microsoft introduced support for this with sandbox solutions, and it was just strange that the app marketplace piece didn’t come with it – but still, the foundations were laid. The big issue of course, was that sandbox solutions were/are intentionally limited in their capability (to support the safety aspect), and developers constantly hit the limiter in terms of what they could implement. To a large extent, Microsoft have removed these constraints, in quite an innovative way – I discuss this lower down in the ‘App hosting options’ section
• Upgrade framework:
  • Just in the same way we’re now used to updates to apps on our phone being pushed out by the vendor (and having the choice of whether we apply the update or not), a similar framework exists for SharePoint apps. This is great news for both users and vendors – it can mean quicker development cycles, a more efficient way of rolling out bug fixes and enhancements (e.g. it’s turns into at least 50% push, as opposed to 100% pull)

Needless to say, no doubt many vendors and developers with product ideas have been (or are about to start) scrambling to work out how they can leverage this. Of course, the marketplace won’t be empty during the beta phase – big name product vendors will have worked with Microsoft on the ISV TAP program. After all, an empty marketplace isn’t in anyone’s interest.

App hosting options – including remote apps with OAuth

So, I mentioned that many of the constraints around sandbox solutions have been removed in SharePoint 2013. Here’s how – apps can now run separately from the SharePoint environment itself. So if you have a need to run code which does some heavy processing (which could be shut down by SharePoint because it’s using too much CPU/memory/disk IO), then run it on a non-SharePoint server. In other words, take the problem outside of the SharePoint equation and deal with it there – the SharePoint admins (who might be Microsoft if you’re a SharePoint Online customer) are happy because the SharePoint performance/uptime is assured, and the developers (or the vendor selling the product) are now happy because they have a solution to the constraints of the sandbox. This means you can effectively build anything and target the sandbox or O365 – you just have to provide the resources (e.g. web hosting, processing power and if needed, data storage) elsewhere. This external “engine” can then talk to SharePoint (e.g. to read and modify data) using the client APIs such as CSOM and REST – these are *much* more capable now. OAuth is used for authentication, whereby the external app is granted a token to access a particular SharePoint site for a specified duration. Here are the top-level options:

• Remote - Azure
  • A lot of work has gone into enabling this. This is known as an Azure auto-provisioned app, and is integrated into the app deployment process – the developer/vendor can ensure that any “external to SharePoint” infrastructure is created on Azure as the app is deployed in SharePoint. This could include a SQL Azure instance to store data (so in the sandbox we’re no longer constrained by having to store data in SharePoint lists within the site collection), and Azure processing capability to do any heavy-lifting. Of course, this needs to be scaled appropriately for the app to work well, but it does work well as far as SharePoint is concerned.
• Remote - Developer-hosted
  • In this context, “developer-hosted” means “whoever is building the app supplies the hosting/infrastructure”. You don’t have to use Azure for any external portions of an app. In fact. you can use anything – another SharePoint farm, some other servers you have running IIS and SQL, Amazon Web Services, and more. By that, I mean this separation brings some interesting possibilities – since SharePoint doesn’t care about the processing that goes on here so long as you call into it using the client APIs, it’s effectively technology-agnostic. You could implement this part on a LAMP stack (Linux, Apache, MySQL, PHP) for all SharePoint cares – I nearly fell off my chair the first time I heard this! I’m not saying that particular aspect is hugely interesting to me personally – though it could certainly be relevant to product vendors, hosters or an organization with dev teams with different skills – but I do find it interesting that things aren’t restricted to IIS. And of course it does illustrate the separation of SharePoint and the remote piece of a remote app.
• On-premise - SharePoint-hosted
  • If you don’t need any external processing/data storage, then an app can be purely SharePoint-hosted. Notably, this is not a sandboxed solution – that model still exists, but this is something different. Server-side code is *not* allowed in a SharePoint-hosted app, so all SharePoint code is CSOM or REST (in addition to the HTML, CSS and JavaScript elements). What’s really important to understand about this model is that any SharePoint artifacts required by the app (e.g. pages, lists, content types, and so on) do not live in the actual site collection where the app is installed – rather, they get created in a special “app web” on a separate web application which is isolated from the site where the app is installed. Visual Studio 2012 with SharePoint dev tools understands this architecture when you deploy and test your app.

Note that SharePoint gives support for extending the branding of the hosting site into the app site (remember, even if it is hosted in the same SharePoint farm, it is still a separate web application/site collection).

Of course, we still have the traditional development models too. So if you’re asked to build a customization in SharePoint and you can deploy code to the farm, you could potentially choose from:

• Farm solution
• Sandbox solution
• SharePoint-hosted app

You’d have to decide whether your solution can be built using an app (i.e. whether the client APIs do what you need), and whether the app model is giving you anything (i.e. should users acquire the app from the marketplace).

Enhanced client APIs

No doubt partly to support apps, the client APIs have been given some love and are much extended. The Client Object Model (CSOM) has many new capabilities, and a new OData/REST API is introduced. My understanding is that both have the same capabilities, but the different programming styles are meant to provide choice – so if you’re coding in JavaScript or Silverlight you’d probably use CSOM, but if you’re talking to SharePoint from another platform (e.g. mobile app) then the REST API would be convenient. The early documentation listed the following as capabilities:

• Existing CSOM capabilities, plus:
• User Profiles
• Search
• Taxonomy
• Feeds
• Publishing
• Sharing
• Workflow
• IRM
• Analytics
• Business data
• ...and more

Note that the new REST API is known as _api since it can be accessed using the format http://somesite/_api/Web/title (with that example returning the title of the root web).

Use of .NET 4.5, but no MVC (within SharePoint at least)

Yes, of course the latest SharePoint is using the latest version of .NET. So we get a new GAC, new language features (e.g. web API, await/async, new HttpClient class etc.) and Visual Studio workflows are simpler, but the bigger impact is that if your code has to target both SharePoint 2010 and 2013, then you have some things to deal with (like avoiding the new language features in your codebase for one). If you follow developments outside of SharePoint in the wider ASP.Net world, it’s interesting that there is no option for using MVC and that a custom SharePoint page will continue to use the WebForms model – meaning Viewstate, postbacks for click events of .NET controls and so on. SharePoint developers the world over probably breathe a sigh of relief of not having to learn a new paradigm on top of a new product, but then pause to think if that’s ultimately a good thing. I don’t think it is personally. If you care that much, there *are* in fact options for surfacing MVC pages within SharePoint 15, but only in SharePoint ‘remote apps’ – and that stuff is far bigger for SharePoint than MVC vs. WebForms.

Use of Metro styling

Some new UI paradigms to learn, but on the plus side even I can style some colored blocks :)

Search-driven content

A capability some other CMS have, which I’ve wanted to see in SharePoint publishing sites for a long time, is the idea that a piece of content can effectively be surfaced in different locations (possibly with different branding/surrounding content), despite the fact that it is really just one piece of content. This content could be edited in one location. I might be showing my age here, but Content Management Server 2001/2002 kind of had this with connected postings, so it was annoying that nothing like it existed in the product for so long.

In SP2013, search is used to deliver this. That makes sense because search has long been the only way to “see past” the site collection boundary in SharePoint, and so this facilitates content sharing across site collections and web applications. You’ll find a new category of web parts called ‘Search-driven content’ – the web parts in here are all pre-configured variants of the Content by Search Web Part (e.g. “Items Matching a Tag”). The base Content Search web part can be found in the Content Rollup category. This is like a Content by Query Web Part on steroids, and using search:

Along similar lines,  any list/library in SharePoint can be nominated as a ‘catalog’ and then shared across site collections. This is part of the ‘Cross-Site Collection Publishing’ Feature and is also search-powered.

Skydrive Pro - Dropbox/Skydrive-like sync to PC, and simple sharing

One nifty feature that will go down well is that users can sync any document library to a folder on their PC, making offline access smoother – no client such as SharePoint Workspace is required. Additionally, SharePoint 2013 introduces the concept of ‘sharing’ – this is really the same permissions model we’re used to already, but with different semantics to hopefully makes things simpler for users. So I can ‘share’ a document library, rather than ‘grant someone contribute permissions’.

Apps for Office

An “app for Office” (known as an ‘Agave’ during pre-beta) is effectively a new form of “Office Add-in” – some good examples I’ve seen include a Bing map appearing next to some Excel rows containing addresses, or a Word Add-in which shows some SharePoint list items and allows them to be dropped into the document. Hopefully you get the idea. What’s interesting about it is that despite the client being Office, the development is done in web technologies such as JavaScript, CSS and HTML. It’s almost like an IFrame hosted in Office. This is kind of cool because there’s now a lot of potential for re-use across the browser and Office clients – i.e. I no longer need to learn Office APIs to target those clients.

AppFabric

AppFabric is a caching technology which exists as a standalone install (and has done for some time now – it doesn’t require Windows Server 2012). SharePoint installs it as a pre-requisite if it is not already present. AppFabric essentially joins together the memory of multiple web servers, and allows you to use this as a cache – in this way you don’t need to worry about different machines having different values in the cache, or having to use a CacheDependency or anything like that. Effectively you store name/value pairs and access them from anywhere. The cache can be divided into different areas (partitions), and there are options for making it highly available.

Summary

So, lots of new capabilities and lots of things  for developers to get to grips wit – and there’s lots I haven’t mentioned too. Like JavaScript templates (think jQuery templates for well-known SharePoint controls e.g. a list view) and use of Azure for workflow hosting. I’m sure good content on these can be found elsewhere (if not now, then soon).

Of course, this has been a technically-focused post and other folks will take a closer look at end-user enhancements. I’ll be covering a few technical areas in detail, as will many others, so it might be time to dust off the RSS reader and get on Twitter if you’re not there already :)

SharePoint MVP for another year

I’m privileged again to receive the MVP award for SharePoint. This is the 4th year for me and I definitely enjoy being part of the program. I’ve met a lot of great people, and I’ve had a lot of interesting conversations with product group folks - e.g. recently helping define how some upcoming IntelliTrace features for SharePoint development should work. Every year I like to jot down on my blog the community “things” I did in the previous year - this list is mainly for me rather than you, but here are some the things I got up to:

• Talks on Continuous Integration and customizing the SharePoint ribbon at the SharePoint conference in Anaheim
• Other talks in the UK (e.g. “jQuery and AJAX in SharePoint”) – at the UK user group and SharePoint Saturday UK
• Continuous Integration article series on the official MSDN SharePoint Developer blog
• 15 new articles on this site, including things like Extending the SharePoint 2010 social features and WebPartAdderExtension – a better way of deploying web parts
• Open source contributions such as:
• http://sharepointci.codeplex.com
• http://tfssimpleversioning.codeplex.com
• Interview with the SharePoint Podshow guys on performance and optimization

I also gave a couple of talks at Tech Ed U.S. recently, but I think technically they fall into the next “MVP year”.

As always though, I know I could do a far better job of serving the community, and blog comments are something I continue to struggle with. Seems the more articles are added to your “back catalogue”, the more comments will be generated. More recently I’ve started publishing them even if I haven’t got time to contribute an answer myself, but specific questions on what I’ve written will always get priority.

Anyway, it’s been a fun year – and with the next version of SharePoint round the corner, I think the next one will be even better :)

I’m speaking at TechEd North America, June 11-14

This year’s edition of Microsoft’s biggest and glitziest conference, TechEd, is nearly upon us, and I’ll be flying the SharePoint flag with two sessions aimed at SharePoint developers. TechEd is a great event in my eyes – with over 10,000 attendees, I love it for the scale, the speakers, and the focus on upcoming technologies. This year’s event is held in Orlando, Florida, and has now sold out. Several of the speakers are Microsoft Technical Fellows and Distinguished Engineers, and include the likes of Scott Guthrie, Mark Russinovich, Juval Lowy, David Chappell and Jeffrey Snover. It’s definitely nice to be on the same bill as such luminaries.

For us SharePoint folks, remember of course that the big unveiling of SharePoint ‘15’ is later in the year, in November – at the official SharePoint Conference in Las Vegas.

Here are the details of my sessions:

OSP433: Deep Dive on SharePoint Ribbon Development and Extensibility – Weds, June 13 - 5pm

Take advantage of the Ribbon in your SharePoint applications for a tightly integrated and great user experience! Developers can customize and extend the Ribbon for custom solutions. In this session we examine the different components of the Ribbon as well as how to create page components, asynchronous callbacks and prompt the user with intuitive dialogs. Best of all you can do all this from the sandbox and avoid getting admins involved in deploying farm solutions!

OSP432: Application Lifecycle Management: Automated Builds and Testing for SharePoint Projects – Thursday, June 14 – 8:30am

Whether experienced or novice, most SharePoint development teams can improve their processes, and this session looks at how Microsoft Visual Studio Team Foundation Server and Visual Studio can help—specifically with automated WSP builds and Visual Studio 2010 UI testing (which can have a much lower barrier to entry than unit testing). When a few of these capabilities are strung together, the results are incredible for dev teams. Over several demos, we cover how to get started with automating the build, deploying the resulting WSPs to a remote SharePoint environment, then automatically running UI tests against the site. We show examples of testing custom functionality, and share lessons learned from implementing these techniques in the real world.

Here’s looking forward to a great conference!

Being a SharePoint Premier Field Engineer

Since the beginning of the year I’ve been contracting to Microsoft as a Premier Field Engineer, and I’ve been meaning to write about my experiences. Since my team (here in the UK) are looking for new people, now seems like a good time. Also, although I was already fairly close to Microsoft in many respects, it’s interesting being on the inside and seeing how certain things work - especially how MS IT deal with some of the challenges within a large organisation. I’ll talk about some of the cool things I notice at the end.

It’s been a fun few months for me – I work in the SharePoint group within Premier Field Engineering, and the focus is definitely oriented to infrastructure/IT Pro topics, such as:

• Performance
• Scale
• High Availability
• Security
• Patching/upgrade
• Backup and recovery
• DR
• Monitoring
• etc.

From a personal perspective, my background is all about development, customization, end-user experience, consulting and so on – within Microsoft, I’d say my “natural home” would probably be Microsoft Consulting Services (MCS), and consequently I thought working instead for PFE would be a good challenge for me. I always felt slightly guilty that some folks labelled me a SharePoint “expert”, but if a client asked me some “Achilles heel” questions (“What should my backup strategy look like in detail?”) I wouldn’t be able to give a good answer. How can that be right if you’re a product expert? If you’re a developer, I highly recommend trying “the other side” – you’ll learn about what’s really involved in running SharePoint (hint – it’s not all about your cool functionality). If you’re a SharePoint IT Pro, I can’t think of many better places to do this work than PFE. 

The UK team

The SharePoint group within Premier Field Engineering UK currently consists of 16 engineers. Two of the guys are SharePoint 2010 MCMs (Sam Hassani and Chris Whitehead), and another colleague is on the current rotation. Another of the UK MCMs, Neil Hodgkinson, now works within the SharePoint Online group but is an ex-member of my team. I’m not sure how many SharePoint MCMs there are in the UK overall, but I’d guess that’s the highest concentration for any group here. Needless to say, the rest of the team are talented too - there’s a lot of technical capability here, and that makes it an appealing place to work. I’ve certainly had some highly interesting conversations about the nuts and bolts of SharePoint.

My first interview felt like it went pretty badly. It was in the middle of a heavy period with my project at the time, and I went into the phone interview (with Chris W, one of the MCMs) immediately following a draining 3 hour conference call with my client. At the time, my day-to-day work was more around heavy customization, development sprints and story points, so it was tough to be thrown into quick-fire questions on infrastructure. I guess I’m used to performing quite well in dev/consulting interviews, so it was humbling to not have answers to some of the questions. After I put the phone down, I was kicking myself for not having prepared better – I realised that yes, I *did* in fact want to do this kind of work and that it would be great to plug some of those gaps. In the end, I heard I actually scored in the mid 80s during the technical test so I guess it just felt worse than it was! From there it was onto the second stage. This was a mock presentation to my manager and a SharePoint PFE, and since this is a key part of the transactional PFE role (more on this later) it was a good insight to the kind of conversations I’d have with clients.

PFE work

A PFE can be “dedicated” or “transactional”. The former typically serves 1 or 2 clients for a period of several months (e.g. during a rollout/upgrade) and spends most of his/her time on the client site. So far I’ve been transactional, which means I spend just a few days with each client before moving on to the next one. This is great for seeing many SharePoint farms and how clients are using the product in the field. I’m mainly involved with the “Risk and Health Assessment Program” for SharePoint (often abbreviated to “RAP”) so I’ll talk about this some more, but other things I’ve done include “chalk and talk” sessions (sometimes these last several days) and client visits as part of a reactive incident (e.g. “Crit Sit”). Some of the engagements have several aspects to them, and I often think back to a description I read in the hiring process – “Frequently, these problems will not only be technically complex, but will be politically-charged situations requiring the highest level of customer skill.”

Other PFE engagements include performance labs and several types of workshops, some of which the engineer needs to be accredited to deliver. Only organisations with a Premier Support contract can make use of these services.

Needless to say, SharePoint PFEs aren’t the only breed – different groups exist for many Microsoft technologies, and each will have engineers who can run the respective RAP offering:

• AD RAP (Active Directory)
• EX RAP (Exchange)
• SQL RAP (SQL)
• SP RAP (SharePoint)
• CM RAP (System Center Configuration Manager)
• OPS RAP (Operational Excellence/Service Management)
• WD RAP (Windows Desktop)

Risk and Health Assessment Program

A SharePoint RAP is a deep SharePoint health check, which assesses where the client may have deviated from what Microsoft consider to be good practice, particularly in terms of product configuration and management processes. From the service description - “The Risk and Health Assessment Program for Microsoft SharePoint Server includes an extensive environment review completed by a highly skilled, accredited field engineer. The onsite engineer will also complete hands-on knowledge transfer with your engineering staff, passing on some of the best real-world knowledge.” A big part of the RAP process is passing on key information which will help the client get to a better place with their SharePoint deployment, often down to low-level detail such as whether each of the changes being recommended will require a service outage. Being able to closely understand Microsoft’s view of health/risk issues for a SharePoint farm is certainly interesting, and I’ve definitely learnt a lot about SharePoint in the process.

If you’re interested in the RAP process, I came up with this diagram which represents the different phases (my interpretation only):

A good proportion of the “magic” of the RAP process is the use of an analysis tool written by a central group within Microsoft. This is effectively a shell, where different rule sets can be plugged in depending on the RAP being performed (e.g. a SharePoint rule set for SP RAP, Exchange rules for Ex RAP etc.). Like many developers, I might look at some tools and say “Pah, with enough time I could build that!”, but frankly, I’m deeply impressed with the RAP tooling and how it works. In addition to the core “environment interrogation” tasks, several other tasks are simplified – one aspect of a SharePoint RAP is performance analysis, and tool helps out with the process of monitoring all the servers in the farm for a period (usually 24 hours) using appropriate PerfMon counters. Of course, interpreting the results of much of this data and applying knowledge of how the organisation is using SharePoint is completely down to the engineer.

Still, it’s clear that a lot of development has gone into the tool and the tests it performs, and to protect this IP the tool is digitally-signed to each Premier client who receives a RAP. This means that if the tool ever finds it’s way onto the internet, it’s of zero use to anyone other than the original client for their original environment.

No tool can find everything though, and so other aspects of the RAP include an “operations interview” where several topics are covered. These include:

• Change management
• Operational Excellence/Service Management (e.g. SLAs)
• DR/backup and recovery
• Database maintenance procedures
• Security (e.g. anti-virus arrangements, service account configuration etc.)
• Monitoring

So the RAP process is quite wide-ranging, and personally I’ve yet to have a delivery where the client hasn’t concluded it was extremely valuable.

Working at Microsoft

As I mentioned earlier, it’s interesting seeing how various things happen inside Microsoft. Perhaps more than anything, it’s the MS IT stuff that really gets my attention – after all, I’m just an end-user in terms of the helpdesk and operations teams. Here are my rough notes on “cool things” I’ve noticed along the way – what’s striking is that many are just good uses of Microsoft technologies:   

• Efficiency of helpdesk
  • Immediately take control of desktop via Lync
• Search
  • When connected to Corpnet and browse to Bing, see beta/dogfood build
  • SearchVote site – uses an IFrame onto both Bing and Google. Sends your search term to both and allows you to compare results, highlight areas of page and submit feedback (e.g. “this result is missing from Bing”)
• Remote access
  • DirectAccess - DirectAccess is simply awesome. Effectively provides secure VPN access automatically when connected to internet, without needing to connect to VPN or use VPN client
  • VPN via smart card – provided as a fallback option to DirectAccess
• Use of Forefront Identity Manager (FIM) to add yourself to AD groups and distribution lists (self-serve)
• Many internal websites/systems, usually just accessed by an internal DNS name (e.g. http://somesite)
  • Check products for known bugs
  • Access product source code
  • See slide decks from Microsoft conferences
  • Large array of intranet sites, all running on SharePoint
  • Case management systems (MS CRM)
• Secure FTP site creation (for sharing large files with customers)
• Bitlocker
• IRM'd e-mails - can't open on personal PC/iPhone
  • Can't forward/copy from/print etc.
• Automatic software updates via System Center 2012 Configuration Manager

SharePoint PFE’s needed in the UK (May 2012)

The team is on the lookout for talent, and I’ve been asked to help out. If you are based in the UK, are looking for a permanent role and think you have the technical expertise, then the team would like to hear from you. PFE look for people that have a solid technical base, can clearly convey their message, and who are willing to learn. Candidates who are successful in the pre-screening phase will go to face-to-face interviews at the Microsoft UK campus in Reading.

• Read the full job description
• In the first instance, please contact me through LinkedIn - I’ll perform a certain level of CV pre-screening

SharePoint 2010 and unit tests (from Visual Studio 2010/MSTest.exe)

It’s been possible to run unit tests/integration tests for SharePoint 2010 within Visual Studio 2010 since VS2010 SP1, but I’m starting to wonder if the steps to enable this are widely known. Whilst writing another article, I did an internet search and didn’t find any useful guides on page 1 of my results. Indeed, many of the top search results contain information which is now out-of-date, such as the “Unit testing with SharePoint 2010 development” article on the usually good www.nothingbutsharepoint.com. However, I won’t be the first to write about this and I suspect the info is out there if you really go looking – but I’m going to do my bit and write the steps down here too.

The key issue here is being able to call into the SharePoint API from a unit test (usually referred to as an integration test). ‘Pure’ unit tests which do not call SharePoint objects have always been fine. The prerequisite for being able to run SharePoint integration tests are:

• Install Visual Studio 2010 Service Pack 1

From there, just follow the process below.

The process - running SP2010 unit/integration tests from Visual Studio 2010

1. Configure Visual Studio 2010  to allow Test projects (i.e. VS projects which contain tests) to be targeted for .NET 3.5 (the framework version SharePoint 2010 uses):
   
   The framework version for a Visual Studio project can be configured in the project properties, but I’ve always needed to perform a one-off task (per dev VM) – edit the devenv.exe.config file to allow VS to retarget test projects. These steps are listed on MSDN at Possible Additional Steps to Enable Re-targeting of Test Projects to .NET Framework 3.5, but for completeness I’ll summarise them here:
   
   - Close Visual Studio if open
   - Open Windows Explorer to C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\
   - Take a backup copy of devenv.exe.config
   - Open the original devenv.exe.config file in Notepad
   - Paste in the entries listed in the ‘Possible Additional Steps..’ link above. Take note that there is an appSettings key as well as the main set of bindingRedirect entries
   - Save the config file
   - Reopen Visual Studio
   
   
2.  Create a Test project and ensure it gets targeted to .NET 3.5:
   
   There are a couple of routes to doing this, made slightly confusing that the ‘Test Project’ template which targets 3.5 is listed underneath a confusingly-named ‘Test Documents’ heading in the dialog. Either use that one (and check it does target 3.5), or use the main ‘Test Project’ template and go on to retarget it from .NET 4 to .NET 3.5. I’ll show that process here:
   
   
   
   Once the project has been created, go into the project properties and on the Application tab, ensure the Target Framework setting is ‘.NET Framework 3.5’:
   
   
   
   You’ll see a dialog warning that the project must be closed and reopened – click ‘Yes’ to do this.
   
   
3. Ensure the ‘Platform target’ of the project is set to ‘Any CPU’:
   
   
   
4. Ensure the .testsettings file specifies tests should run in a 64-bit process:
   
   You should now have a couple of .testsettings files in your Visual Studio solution. The deafult one is called Local.testsettings, and we need to edit this one – but note if you use any other file, you’ll need to edit that too. On the Hosts tab, ensure the ‘Run tests in 32 bit of 64 bit process’ option is set to 64-bit:
   
   

The result

You should now be able to write tests which call into the SharePoint API. For example, here’s a slightly theoretical test showing I can call a utility class which uses SPSite/SPWeb objects:

   1: [TestMethod()]

   2: public void GetRootTeamSiteTitleTest()

   3: {

   4:     string expected = "BVT test"; 

   5:     string actual;

   6:     actual = CobCiHelper.GetRootTeamSiteTitle();

   7:     Assert.AreEqual(expected, actual);

   8: }

..where the code in the CobCiHelper class being called looks like this:

   1: public static string GetRootTeamSiteTitle()

   2: {

   3:     string title = null;

   4:     using (SPSite site = new SPSite(Urls.Homepage))

   5:     {

   6:         using (SPWeb web = site.RootWeb)

   7:         {

   8:             title = web.Title;

   9:         }

  10:     }

  11:     return title;

  12: }

If your test should pass, you’ll see something like this (but either way your text should execute successfully):

Running SharePoint integration tests within TFS automated builds

The process above helps you run tests manually from within Visual Studio, but note that it does not help if you want to run the same tests from within an automated build. Unfortunately, integration tests which call the SharePoint API are not currently supported in Team Foundation Server 2010/Visual Studio 2010. ‘Pure’ unit tests, or those which use a mocking framework such as Typemock can be used though. Let’s hope this challenge goes away in the next wave of technologies.

Extending SharePoint 2010 social features

In terms of social, it seems that many SharePoint 2010 customers wanted and/or continue to want a little bit more than “out-of-the-box”. This is especially the case for clients who use the activity feed – one of the features which changed SharePoint to be more of a social platform. “How do I reply/comment on one of these things in my activity feed?” clients usually say. “Where’s the ‘like’ button?!”. And so, a whole host of 3rd party products sprang up – some dedicated to SharePoint (Newsgator, Beezy, SnapworkSocial etc.), some more generic and sometimes cloud-based but with SharePoint integration (Yammer etc.). I’m not an expert in those products, though I have looked at some. What I do know is that most are still evolving, and if you started this journey a while ago, frankly you might not have liked any of them. Maybe you have reasons for still not liking any of them, such as privacy concerns. Or perhaps the fact that many want to be a parallel universe alongside SharePoint, where content is “sync’d” or “published” to. Something I notice about several products is that they completely replace the SharePoint activity feed, and so many of the good things about what Microsoft built are unfortunately lost - notifications around tags I’m interested in, job title changes, someone specifying they have the same interest as me, and so on. These are all valuable features which I’ve seen work well in the enterprise – they help connect people, content and company projects/workstreams which are relevant to each other.

Here I want to show you what we did for one client, perhaps taking a more SharePoint-centric route. I’m not here to sell it (it’s not a shrink-wrapped product, and I don’t own it), but I thought it would be fun to talk about some of the pinch points in SharePoint, and what you might do about them. I won’t be publishing source code. This was part of a project I worked on for Content and Code, a partner in the UK. I worked as part of a small but talented team, consisting of myself, Wes Hackett and Rich Browne (Rich is an extremely capable ‘under the radar’ kinda guy). Some of the client’s offshore team (a great group) also worked on other functionality around this set.

The activity feed – from “one way” to “two way”

So let’s start with the activity feed. Let’s go from this:

..to this (click to see larger image):

Notice that the content is pretty much the same in both images. The look and feel is clearly stolen from influenced by Facebook, but I’d like to think there’s more to it than that. Obviously we have the big ‘tick-list’ items such as liking and commenting (more on these later), but we also enhanced some other aspects of the SharePoint 2010 activity feed too:

• Activity feed items related to SharePoint content – did it ever strike you as weird that SharePoint is an ECM system, but only status updates, job title changes and other ‘social’ things appear in the activity feed? Our client thought so, and we changed that. Our feed has the following content:
  • Announcements, discussions/discussion replies, wiki pages/enterprise wiki pages, blog posts and blog comments. Notice that documents are missing – obviously most SharePoint environments have huge volumes of documents, so this content type requires some thought. Later I’ll talk about our ‘subscriptions’ framework which gives a user the option of seeing documents for certain locations.
  • [Side note: the ECM content types appear in the first image above too, but only because our ‘activity publish’ code is working with the OOTB ‘presentation’ code – they wouldn’t show in a standard SP2010 installation.]
  • [Side note: my buddy Wes has the best reference on the internet on how to do this, including some code samples – see Extending the activity feed with enterprise content]
• Richer activities:
  • The blog article in the images above is a great example – in the OOTB activity feed, users can only see the article title. In our version, you see the title and the first bit of the content - providing more context to help you decide whether the link is worth following.
  • Also check out the ‘Rich Browne added a new colleague item’ item. If he’s added a new colleague, it might be nice to show a picture of him.
• Iconography – this makes it easier to quickly identify what an entry is about.
• ‘Likes’ in the feed - under certain circumstances, ‘likes’ are published to the feed as an activity. See the first item in the second screenshot). More on this later.
• ‘My activity’ view – this helps me find things I’ve published. For example, if I uploaded a PowerPoint presentation recently and know someone made an interesting comment, this view will help me find it again.
• Status update bar – no longer any need to navigate off to my SharePoint profile page just to update my status.
• Look and feel – subjective, but arguably prettier :)

Commenting/note board

Dialogue and conversation is the backbone of most social solutions, and no doubt you’re familiar with how it works on Facebook et al. But SharePoint 2010 already has the note board which allows comments/notes to be added on every document, page and list item – so isn’t that enough? Well, possibly not. For one thing, consider that we want to be able to comment on ‘social’ activities such as status updates, job title changes and so on. On a different tangent, the SharePoint 2010 note board architecture is effectively incompatible with a Facebook-style display of activities and comments hanging off them (as shown in our enhanced activity feed). This is because to fetch the set of comments for a feed containing 20 items, I have to make 20 individual calls to SocialCommentManager.GetComments() – which is clearly not going to perform or scale. So, we need a different approach.

In our case, we took the big decision of scrapping the standard note board. Some big considerations here include the migration path to vNext, but we concluded that to write some migration scripts (and migrate the volume of data we would have) would be completely in-line with the expectations of an upgrade project. So without the shackles of the OOTB design, it’s possible to go back to first principles and use whatever data schema fits best. The key is simply being able to retrieve all your likes/comments (and anything else) in one operation, for whatever page/web part designs you come up with. Nothing more to it. When implemented this way, it means that a user can add a comment to a SharePoint item either from the activity feed OR from the item’s note board, and such comments would show wherever I looked at the item. Notably, the way most ( maybe all?) 3rd party products deal with this is to side-step the issue completely – comments in the activity feed only show in the activity feed, and notes on the note board purely live on the note board – and never the twain shall meet. Our client wasn’t comfortable with this due to the huge overlap between the two concepts - the feeling was this disjointed behaviour could be confusing to users. We debated that our path could mean that some “noise” comments would be stored against items (“Dude, nice kitten photo LOL!”), but our client preferred this to the alternative.

So whilst we’re having to do work around the note board, we may as well give it some UI love. Rich, my partner in crime, decided that the native “show 5 items and do paging” approach was pretty poor in this enlightened age of Facebook infinite scrolling. Even weirder, did you notice that any user can delete another user’s note? Governance, who needs it?! Here’s a reminder of the out-of-the-box note board:

Here’s what we ended up with:

If I’d been logged in as one of the users above, then of course ‘Edit’ and ‘Delete’ links would show next to my notes as appropriate (i.e. we fixed the lack of security around ‘Delete’). Notice also that our dates are a bit more user-friendly than SharePoint’s ‘raw’ display of a date/time.

Note board web part

Consider also that in addition to the ‘tags and notes’ window, the SharePoint note board is also a web part which can be dropped on a page. This can be a great idea for a team site home page for example, so we ensured our note board had the same capability:

De

Profile note board

In SharePoint 2010, the note board also appears on a user’s profile page and can be used to leave a note for that person. In that context, it looks like this:

This is definitely a useful piece in the social feature set - people are certainly getting used to sending messages using something other than e-mail. However, the lack of ‘reply’/threaded discussions frequently becomes an issue – you end up with the ridiculously disjointed situation of writing on each other’s profile in order to have a conversation. We thought it should be something more like this:

If you ever want to do something similar, don’t forget the e-mail notifications which alert a user that a note has been left on their note board. Of course if you are doing something custom, you’ll have the opportunity to enhance the e-mail content and behaviour – for instance, not only sending a notification when someone posts on your note board, but also when someone replies to a conversation you’re involved in.

Entry points

So it’s great that we have this super-duper note board, but it’s also worth mentioning the experience to get to it. Since we already had much of the back-end code to support it, we thought we may as well display the number of likes/comments for the current page (or highlighted list item, or whatever). This is great for instantly seeing “interaction” around the current item, and probably encourages people to perform these social actions:

These are basically direct replacements for the social controls added to the ribbon for ‘I like it’ and opening the tags and notes window.

Speaking of the ‘I like it’ functionality in SharePoint, let’s talk about how we deal with ‘likes’.

Likes – more important than you might think

SharePoint 2010’s ‘I like it’ is implemented as a social tag – this enables Microsoft to re-use the plumbing for that feature, and in some ways is a nice way of doing things. However, we had a different vision. For us/our client, the main point of liking was to “generate footfall” – in other words, to put content under the eyes of people who would not otherwise see it. Facebook does this. So if a colleague of mine “likes” some content which I haven’t seen in my activity feed because it was created by a non-colleague or in a site/list/library which I don’t have a ‘subscription’ to (N.B. ‘subscriptions’ are another custom piece, which I’ll talk about later), then I’ll see it in my feed:

So in the image above, Tristan Watkins is my colleague but David Bowman is not. We tossed around several ideas for ensuring a good signal-to-noise ratio here. You could definitely get quite sophisticated in terms of “keeping score” of which users have received likes for which content, and such conversations were a good reminder of the complex algorithms behind Facebook. However, simply ensuring I don’t see likes for content I’m already probably aware of was a good starting point. Remember that if the original activity (e.g. status update/document/whatever) appeared in my feed, then I’ll be able to see likes/comments “hanging off” it right there – but it’s the likes “broadcast” activities we need to be careful with. In the end, our implementation of comments also used the same “broadcast” logic – so I could see comments my colleagues were making on items I might not have seen.

The end result is pretty powerful. Our client had some examples across the business where liking brought people together (e.g. they went on to work together on some initiative which was a success for the company) – if you’re cynical about social, seeing this happen might change your mind.

In user-experience terms, of course we also need to be able to see who liked any particular item. We put that as a new tab in the existing ‘tags and notes’ window. Although not shown in this image, I would see ‘Add as colleague’ buttons (and presence icons) next to any people who I wasn’t already colleagues with, since we want to encourage connections across the business:

Subscriptions

If you’re familiar with the social aspects of SharePoint, you’ll know it’s all about colleague relationships. My activity feed gives me a view of what my colleagues are doing and helps me stay in touch. However, in the environment we were working in, this was considered a little “one-dimensional” - colleague relationships aren’t always ideal. What happens, for example, if I discover a great team site where the content is extremely interesting to my work. Let’s say it’s another part of the business working on SharePoint, and let’s say these guys are producing some great architecture documents. I probably don’t want to add all of the contributors to that site as colleagues – for one thing, I may not particularly want their status updates, birthdays, manager changes and so on. I probably do care if they tag documents with ‘SharePoint’ or related terms, but remember that in the out-of-the-box activity feed I can be notified of that simply by ‘following’‘ those tags. So what I really want is just to be notified if new content is created in the site. I guess I could use alerts, but frankly my inbox is busy enough and anyway, surely this is exactly the kind of stuff I occasionally dip into my activity feed for?

So, we created a subscriptions framework. If a user sees some content in SharePoint, they can use it to be notified in their activity feed about new items, without requiring colleague relationships. Users can subscribe at the site (web) level:

..or they can subscribe to a particular list or library:

It’s also possible to subscribe to a site but exclude a “noisy” list or library. Once I have a subscription, I’ll see new content appear in my activity feed, and this includes documents. As I said earlier, we didn’t implement notifications for documents from colleagues because we felt that would result in noise, simply because documents are so prevalent. With subscriptions, it felt like the user would have more control. The image below shows a document in my activity feed – consider also that we now have multiple ways of things coming into my feed, so we added a context menu so that a user can understand why they were seeing a particular item:

Group activity feed

In addition to the main enhancements to the activity feed, we also developed a team/group activity feed. We found this was a great web part to have on a team site home page – visitors to the site could then instantly see new content and other recent activity related to the site. This is a slightly tricky concept to implement with what SharePoint provides, but we were happy with our final design. In our model, this view shows recent activity by any of the owners/members of the site which is relevant to the site, and also any new content, including documents (i.e. as though you had a subscription to the site).

Summary

With SharePoint 2010, the direction of the product moved firmly towards the social arena – of course, the flexibility is there for organisations to opt-in to using this capability or not. However, amongst adopters it’s fairly common to want more than is provided out-of-the-box, and I’ve hopefully given some food for thought on what an enhanced version might look like.

SP2010 Continuous Integration–pt 6:Running UI tests as part of a build

The 6th article in our CI with SharePoint 2010 and TFS series is now live on the official SharePoint Developer blog (as mentioned on Twitter last week) – see Running Tests as Part of a Build. It took a while to write, and it took a while to publish (you’d think Mike Morton and his colleagues had actual products to ship or something) but it’s there now.

The article shows how to integrate tests into a build, particularly user interface tests via Visual Studio 2010’s Coded UI Tests. After all, if you’re going to continually check that what you’re putting together works, usually you’ll want to go beyond simply checking that it builds and deploys. You might choose to run unit tests, integration tests or UI tests (or most likely a combination), but UI tests seemed like a good place to start since they can slot into a build very easily.

As a side-note, running integration tests (unit tests which call the SharePoint API) within a build is not so straightforward (but pure unit tests can be done). I might discuss this in a future post.

Links:

• Running Tests as Part of a Build
• SharePoint/TFS Continuous Integration Starter Pack

 

WebPartAdderExtension – a better way of deploying web parts

Recently, that nice young man Wictor Wilén blogged about WebPartAdderExtension and web part gallery sources – this is a little-known (and currently undocumented) capability in SharePoint which allows you to deploy custom web parts in a different way, with huge benefits. In summary, SharePoint 2010 provides a means for you to specify web parts to appear in the web part ‘picker’, without requiring a .webpart file in the local web part gallery (which is the usual route). Code is required, but SharePoint 2010 provides a ‘hook’ for you to add this in the form of the WebPartGallerySourceBase class. To help visualize this, the web part highlighted in the image below is deployed using WebPartAdderExtension – it appears just like a custom web part which uses a .webpart file and the user is blissfully unaware of any differences:

I read Wictor’s two excellent articles, and on a recent project I decided to extend his sample into a model which I think works well for many scenarios. Now, I don’t want to be too dramatic, but I think WebPartAdderExtension in general is A Big Deal – I just can’t imagine going back to the old way of doing web parts at all. In this article we’ll first have a recap of what the problem is and why this new way is better, and I’ll then talk about my implementation and where you can download the files.

The problem

In a nutshell, web parts and the web part ‘gallery’ model in SharePoint are great, until you have to make a change to the definition of a web part that’s already out there. This usually takes the form of:

Client: “Hey can we change the title of ‘foo’ web part? Actually that name we came up with isn’t great and users aren’t finding it.”
Client: “Hey can we move ‘foo’ web part into a different category? That’s a quick change right?”

As a developer you say “Sure, I’ll get right on it!”. But then you realise that it’s deployed to everyone’s My Sites. And you have thousands of them. Or maybe it’s in team sites, but come to think of it you have quite a few of those too. This matters, because the web part is in each site collection’s web part gallery (in the .webpart file), and you’ll have to perform some operation to iterate every site and do some sort of processing on each one. Depending on how many sites you have to deal with, this could take could take significant time to run, and will place a certain amount of load on the infrastructure whilst it does. On my last project, it usually took around 7-8 hours to iterate the 25,000 My Sites we had, and we could only run it during a long maintenance window due to the load on both the server doing the processing and the database servers. Of course, 25,000 sites isn’t that big. In larger enterprises, I’ve even heard of folks writing custom multi-threaded applications to deal with such “iterate the sites” scenarios. I’m not saying you won’t need to occasionally perform this iteration for other reasons, but if we could take one common use case (i.e. changes to web part definitions) out of the equation, that’s got to be good in my book.

The solution

Consider that with WebPartAdderExtension, rolling out such a change to (say) 25,000 sites has the following characteristics:

• No iteration required, therefore no load on servers
• Takes immediate effect across all sites
• Depending on your implementation, may not even require an app pool recycle!

How it works: WebPartAdderExtension is a Feature element (XML), and this can be defined at farm/web app/site collection scope (which is interesting in itself!). This ‘points’ to a class which derives from WebPartGallerySourceBase and in here you can do what you like – all you have to do is return a collection. Each item in the collection will then appear as a web part in the web part gallery. Your custom code is fired when a user browses for a web part (in page edit mode). In Wictor’s sample, he fetched items from a SharePoint list, meaning the pseudo web parts were actually list items rather than each having a traditional .webpart file.

 My implementation

Wictor’s sample conveyed the principle to me perfectly, but I ended up with a different model. Here’s what I came up with to meet our requirements:

• A series of XML files, which define web parts by web application (e.g. an XML file for ‘teams’ web parts, an XML file for ‘My Site’ web parts, and so on).
  • N.B. It could be the nature of our web parts, but by web app was definitely the most appropriate factoring for us.
• Code which reads the XML file for the ‘current’ web application
• A web application-scoped Feature, which uses WebPartAdderExtension
  • In fact, several variations of this Feature are created, one per web app. The ‘teams’ web parts Feature uses WebPartAdderExtension to point to the class which knows about ‘teams’ web parts – this in turn reads from the ‘teams’ web parts XML file.

So rather than Wictor’s list-based sample, I’m happy to use XML files on the filesystem – when we specify the web parts which can be picked, we must specify the assembly/type name etc., and since this is in developer territory I’m comfortable with the fact that a WSP deployment is required to update this XML (the files live under ‘_layouts’). Somewhat surprisingly, SharePoint doesn’t cache this data so even in my file-based implementation, an app pool recycle is not required – just update the WSP (and ensure ResetWebServer = false) and the changes take effect immediately. Even if it DID require a recycle, remember this is still several million times better than a long-running process to iterate site collections.

Here’s my XML (custom schema) for defining the set of ‘teams’ web parts (using dummy web parts here, in my COB namespace): 

   1: <?xml version="1.0" encoding="utf-8" ?>

   2: <WebParts>

   3:   <WebPart

   4:     Title="My web part for team sites - deployed via WebPartAdderExtension"

   5:     Category="COB"

   6:     Description="A dummy web part for team sites"

   7:     ID="TeamsWebPart1"

   8:     Assembly="COB.SharePoint.WebPartAdderExtension, Version=1.0.0.0, Culture=neutral, PublicKeyToken=23afbf06fd91fa64"

   9:     Type="COB.SharePoint.WebPartAdderExtension.TeamsWebPart1.TeamsWebPart1"

  10:     IconUrl="_layouts/images/webpart.gif"

  11:     RibbonCommand=""

  12:     OnClientAdd=""

  13:     />

  14:   <WebPart

  15:   Title="My second web part for team sites - deployed via WebPartAdderExtension"

  16:   Category="COB"

  17:   Description="Another dummy web part for team sites"

  18:   ID="TeamsWebPart2"

  19:   Assembly="COB.SharePoint.WebPartAdderExtension, Version=1.0.0.0, Culture=neutral, PublicKeyToken=23afbf06fd91fa64"

  20:   Type="COB.SharePoint.WebPartAdderExtension.TeamsWebPart2.TeamsWebPart2"

  21:   IconUrl="_layouts/images/webpart.gif"

  22:   RibbonCommand=""

  23:   OnClientAdd=""

  24:     />

  25: </WebParts>

We then need some code to consume this. As discussed earlier, this must be a class derived from WebPartGallerySourceBase – since my implementation is ‘XML file per web application’, I created a base class which understands my XML schema and then a derived class per web app. It’s this latter class which knows where to find the XML file:

   0: [Guid("98FF0ED8-6697-4E13-8BCC-161FEE3E4B0C")]
   1: public class GlobalWebPartGallerySource_Teams : GlobalWebPartGalleryXmlSource

   2: {

   3:     public GlobalWebPartGallerySource_Teams(Page page)

   4:         : base(page)

   5:     {

   6:     }

   7:  

   8:     public override string XmlPath

   9:     {

  10:         get

  11:         {

  12:             string path = SPUtility.GetGenericSetupPath(@"template\layouts\COB.SharePoint.WebPartAdderExtension\GlobalWebParts\GlobalWebParts_Teams.xml");

  13:             return path;

  14:         }

  15:     }

  16: }

..and the base class simply reads from the chosen XML file and returns the collection (of GlobalWebPartGalleryItem instances in my case – the important thing is that it derives from WebPartGalleryItemBase):

   1: public class GlobalWebPartGalleryXmlSource : WebPartGallerySourceBase

   2: {

   3: public GlobalWebPartGalleryXmlSource(Page page)

   4:     : base(page)

   5: {

   6: }

   7:  

   8: public virtual string XmlPath

   9: {

  10:     get

  11:     {

  12:         return string.Empty;

  13:     }

  14: }

  15:  

  16: protected override WebPartGalleryItemBase[] GetItemsCore()

  17: {

  18:     List<WebPartGalleryItemBase> items = new List<WebPartGalleryItemBase>();

  19:     string path = XmlPath;

  20:  

  21:     if (File.Exists(path))

  22:     {

  23:         XElement root = XElement.Load(path);

  24:         var webParts = from wpDef in root.Elements("WebPart")

  25:                         select wpDef;

  26:  

  27:         foreach(XElement wp in webParts)

  28:         {

  29:             string title = wp.Attribute("Title").Value;

  30:             string category = wp.Attribute("Category").Value;

  31:             string description = wp.Attribute("Description").Value;

  32:             string iconUrl = wp.Attribute("IconUrl").Value;

  33:             string id = wp.Attribute("ID").Value;

  34:             string ribbonCommand = wp.Attribute("RibbonCommand").Value;

  35:             string onClientAdd = wp.Attribute("OnClientAdd").Value;

  36:             string assembly = wp.Attribute("Assembly").Value;

  37:             string typeName = wp.Attribute("Type").Value;

  38:  

  39:             items.Add(new GlobalWebPartGalleryItem(this, base.Page, title, category, description, iconUrl, id, assembly, typeName, onClientAdd, ribbonCommand));

  40:         }

  41:     }

  42:  

  43:     return items.ToArray();

  44: }

The key bit that links my GlobalWebPartGallerySource_Teams class above with my ‘teams’ web app, is a web app-scoped Feature, having the following elements:

   1: <Elements xmlns="http://schemas.microsoft.com/sharepoint/"> 

   2:   <WebPartAdderExtension 

   3:       Assembly="$SharePoint.Project.AssemblyFullName$" 

   4:       Class="$SharePoint.Type.98ff0ed8-6697-4e13-8bcc-161fee3e4b0c.FullName$"/> 

   5: </Elements>

Note that I’m using the tokenization support to refer to my class using the GUID attribute it’s decorated with. NOTE – when using such tokens the GUID *must* be lowercase (I kid you not). This is just one of a couple of places in Feature/VS schema that I’m aware of which have this requirement.

Also, I’ll let you into a teensy secret, and say that my implementation has a couple of lines of reflection code in GlobalWebPartGalleryItem – this was necessary to carry the title through to the web part instance which was added to the page. Entirely optional, but since this is such an infrequent action in the grand scheme of web server traffic, it’s completely worth it IMHO for a consistent user experience.

In the end, the act of adding the web part to the page is just how the user expects. If you download my sample (link at the end of this article), here’s what it looks like here:

XML which is NOT needed (and must be deleted manually from the VS project)

It’s worth pointing here that you have to explicitly DELETE items from your project to use this approach. Remember that Visual Studio/CKS:Dev just know about the ‘regular’ way of developing web parts, and when you select ‘Add New Item> Visual Web Part’, items are added to your project which you no longer need – specifically this is the .webpart file and the elements.xml file which provisions this into the web part gallery. I’ve not tested, but I’d assume you get an error or duplicate if you forget to remove these. For clarity, it’s these:

Elements.xml (automatically added by Visual Studio) when web part added to project:

   1: <?xml version="1.0" encoding="utf-8"?>

   2: <!-- DO NOT USE - THIS XML IS NO LONGER NEEDED WHEN USING WEBPARTADDEREXTENSION -->

   3: <Elements xmlns="http://schemas.microsoft.com/sharepoint/" >

   4:   <Module Name="MySitesWebPart1" List="113" Url="_catalogs/wp">

   5:     <File Path="MySitesWebPart1\MySitesWebPart1.webpart" Url="MySitesWebPart1.webpart" Type="GhostableInLibrary" >

   6:       <Property Name="Group" Value="Custom" />

   7:     </File>

   8:   </Module>

   9: </Elements>

MySitesWebPart1.webpart:

   1: <?xml version="1.0" encoding="utf-8"?>

   2: <!-- DO NOT USE - THIS XML IS NO LONGER NEEDED WHEN USING WEBPARTADDEREXTENSION -->

   3: <webParts>

   4:   <webPart xmlns="http://schemas.microsoft.com/WebPart/v3">

   5:     <metaData>

   6:       <type name="COB.SharePoint.WebPartAdderExtension.WebParts.MySitesWebPart1.MySitesWebPart1, $SharePoint.Project.AssemblyFullName$" />

   7:       <importErrorMessage>$Resources:core,ImportErrorMessage;</importErrorMessage>

   8:     </metaData>

   9:     <data>

  10:       <properties>

  11:         <property name="Title" type="string">My web part title</property>

  12:         <property name="Description" type="string">My web part description</property>

  13:       </properties>

  14:     </data>

  15:   </webPart>

  16: </webParts>

And that’s it.

Summary

Although the old way of making custom web parts available has worked for years, that doesn’t mean it’s worked well! It turns out there’s an alternative which, in my opinion, has better characteristics – no more having to iterate each site collection if you ever need to make a change to the definition of a web part which is already out there. When I mentioned to our client that I’d used this new approach for our latest web part, the response was “Wow. I think we should go back and switch ALL our custom web parts over to this model.” I think Wictor gets the kudos for bringing it to folks attention though – I’m just providing some icing on the cake :)

Download link

Download my WebPartAdderExtension sample project