Part 2 : Blending publishing/collaboration functionality in SharePoint

A couple of months ago I wrote an article on some of the issues around building sites which are not pure publishing or collaboration solutions, but which mix features of the two. I mentioned at the time that I would follow up on these issues and any others we came across at the end of the project, so here it is.

As a quick recap, we said that characteristics of a site which blends publishing/collab features could be:

site has completely bespoke look and feel/navigation

users will work with document libraries, discussions, calendars etc.

site templates or definitions are used to create several sites with the same content/functionality

custom workflow is used to support a business process (other than standard content publishing), perhaps with InfoPath forms

So let's cover some of the issues you may face in projects like this:

Look and feel of system pages

This refers to the fact that whilst pages you create will happily use your custom master page, any 'system' pages in the '_layouts' directory will not share this look and feel by default since they reference a separate master page, 'application.master', which is also stored on the filesystem in '_layouts'. Such pages are often seen due to the use of collaboration functionality in our site, examples being when a user uploads or edits metadata for a file, starts a workflow etc.

In fact this turned out to be less of an issue than I anticipated because:

• a quick test showed that the use of a HTTP module to switch the master page of these pages works fine. Several people have documented this approach, and I note that Ted Pattison has provided a Codeplex feature for those who would prefer not to write their own code for this. (N.B. though not strictly supported, the idea behind the HTTP module approach is that it is better than modifying files installed with SharePoint such as 'application.master', and can be disabled with a switch if we need to fall back in line with official support.)
• our client decided this was a low priority item, that we may or may not deal with further down the line

On this subject, I see that Microsoft have themselves released KB article How to customize application pages in the Layouts folder in SharePoint Server 2007 and in Windows SharePoint Services 3.0. Bizarrely, the proposed solutions are modifying the standard files but taking backups of the originals, or duplicating the 'layouts' virtual directory in IIS - the HTTP module idea is not mentioned. Without dwelling on this too much, this seems crazy to me and I'm not the only one. I still much prefer the HTTP module solution and will use this on my projects where it is required, though will be sure to regression test thoroughly.

Use of InfoPath with forms authentication

This is an interesting illustration of blending the two types of functionality - collaboration solutions may well use InfoPath for forms, but publishing sites typically use forms auth for a customized/branded login experience. Straight away we ran into problems using InfoPath with forms auth - essentially the link to create a new instance of an InfoPath form in a forms library would disappear (example link shown below [Windows auth enabled here]):

It took a while for me to track down that this is related to the 'Client Integration' switch in Central Admin. For a forms auth site, this gets set to 'off' by default, but defaults to 'on' for windows auth sites. Unfortunately, resolving the problem isn't as simple as flicking the switch back, since what the switch does is remove certain links and disable ActiveX controls which integrate with the browser for link-handling. With the switch enabled again, the InfoPath link reappears on the content type, but when a link to say, a Word document is followed, the browser now shows the basic prompt to download or save the file. This means the more sophisticated 'do you want to check out or open as read-only' prompt is lost and users must work with documents offline and upload to SharePoint at the end of editing, rather than being able to save back directly to the SharePoint repository. In other words, unacceptable to most for a SharePoint 2007 collaboration solution.

In the end I found others in the same position, but didn't find a solution in a short timeframe. Microsoft support tells me "they were unaware of any instances of of these two features working together in this way", so the disappointing conclusion is that these two bits of functionality don't play well together. Our client thus decided to accept windows authentication (our authentication store was always going to be AD rather than SQL anyhow). However, I'm actually convinced it would be possible to get this working given more time - I think the way forward would be to amend the site design to have a link to the InfoPath form in the standard content of a page rather than relying on SharePoint to supply the link. The information supplied in How to: Use Query Parameters to Invoke Browser-Enabled InfoPath Forms would be used here. I'm sure this must be possible and others have done it, I'd be interested to hear from anyone who has.

Overidden control not appearing on standard collaboration pages

Our client had requested some customizations to the search box, and going through our bug list towards the end of development, I realised that our customized version was present on pages we had created, but not existing list pages such as [MyList]/Forms/AllItems.aspx or similar. This seemed strange given that all pages were using our custom master and shared the same header! Digging deeper, this is caused by the fact that SharePoint uses a delegate control to load the search box .ascx file, and to properly override the control across the site, we needed to create a Feature to specify that our .ascx file should be used in preference to the standard control. If we had directly replaced the existing control on the filesystem this would not be required, but when adding custom files in order to avoid modifications to existing files, this extra piece is required. One to look out for when building sites of this nature.

Summary

I read some blogs/discussion threads earlier this week which suggest customizing SharePoint in this way is nigh on impossible, but our project (and surely many others) prove this isn't the case. As is often the case with such a functional platform, there's perhaps a greater chance of success if you understand the nuts and bolts of what SharePoint is doing, but extensive customizations to collaboration-focused sites are inherently possible!

Deployment using STSADM export or content migration API

Having focused on deployment using Features for several articles, back in May I wrote an article titled SharePoint deployment options : Features or Content Deployment?, which explored some of the decisions around deployment strategies for SharePoint projects. There are a variety of methods which can be used to move SharePoint artifacts and content from one place to another, and I think it's fair to say there's still a certain amount of confusion around deployment for many SharePoint developers. I certainly wouldn't claim to have all the answers, but after delivering another project last week, it seems like a good time to go over some of the experiences and reflect on the different approaches.

Needless to say, as far as deployment strategies go in general, the best idea is to have one! I see many newsgroup posts from people approaching the end of the development phase asking how they should move their work to the live servers. The problem I find with leaving deployment until the end of the project, is that none of the approaches are completely straightforward (particularly depending on what your solution consists of), and so if your project is to be delivered on time, it's important to know what steps you might need to go through.

As a sidenote, let's clarify some potentially confusing terminology here:

• Content Deployment - the "paths and jobs" functionality which can be used to move content, surfaced by screens in Central Admin
• Content Migration API - the underlying API (sometimes referred to as PRIME) which actually is used for both STSADM export and Content Deployment (in slightly different ways), in addition to the 'Manage Content and Structure' tool and in migrations from CMS2002

This time round I had decided to use the content migration API to deploy our solution, and it worked well for our circumstances. This is a contrast to developing with Features which I've done in the past, and the main reasons for choosing this approach were: 

• no need for iterative deployment - although our overall project is phased, for this component we were able to develop the solution and then deploy everything from our development environment. (This approach will not work for subsequent deployments since content our client has generated on the live site would be overwritten on each deployment - more on this in an upcoming post.)
• ability to retain object GUIDs - this simplified deployment significantly for our project, since if our lists were allocated new GUIDs on deployment (as happens with STSADM export/import), our components which referenced these lists (ListViewWebPart, InfoPath forms etc.) would not hook up properly on the deployment target. This would add a lot of "fix-up" steps to the deployment process if we were to use STSADM export.
• no direct HTTP access from source Central Admin to target Central Admin - this is a prerequisite to use the Content Deployment functionality (paths and jobs) in Central Admin, but what we needed was a file we could copy to the live server. The content migration API provides this ability and also gives a compression option for large amounts of data.
• automatic inclusion of database dependencies - as with STSADM export, (but not with Features), SharePoint will analyze and collect all dependencies such as fields, content types, master pages etc. for us.

The API is fairly simple to use and you may have seen Stefan's series of excellent articles on the subject - these serve as a good companion to the MSDN documentation.

It's important to remember that any non-database assets (e.g. user controls, assemblies etc.) need to be deployed manually to the target environment - these will not be included by use of something like STSADM export or the content migration API. In our case, since the live environment was a single server (and versioning would be handled by our main source control system), these were deployed by XCOPY since deployment using Solution packages did not offer any compelling advantages here. 

Whilst we're talking about filesystem files, it's useful to be aware that if you see 404 errors on the target after performing (e.g.) an STSADM import, chances are you've forgotten to deploy something like a user control. The 404 is actually coming from the referenced file rather than the actual page, so don't assume something has gone wrong with the import - a check on 'View all site content' and the import log will probably confirm all the site pages are present!

Hopefully this has given some food for thought on an approach you may not have considered. I guess my main message here is that whilst STSADM export is extremely simple, it may not provide the complete answer to all your deployment challenges due to changing GUIDs. In upcoming posts I'll provide a more direct comparison of deployment strategies (extending my 'Features or content deployment' post), and also share my mini-app which provides a front-end onto the content migration API.

[P.S. Sincere apologies to people who left comments whilst I was on holiday which are still not published - I'll publish these and respond over the next few days.

C.]

Blending publishing/collaboration functionality in SharePoint

Most often when creating SharePoint solutions, the requirements often map fairly well to one of the out-of-the-box site definitions which can be used to create new sites. If we're creating heavily-branded internet/intranet sites (WCM sites), we'll probably start with the 'publishing site' template. If we are deploying SharePoint in a document management/collaboration scenario, we'll probably start with the 'team site' template, and so on. Where it gets interesting it when the project requirements effectively have a mix of this functionality. Characteristics of such a site might include:

• site has completely bespoke look and feel/navigation
• users will work with files stored in document libraries
• site templates or definitions are used to create several sites with the same content/functionality
• custom workflow is used to support a business process (other than standard content publishing), perhaps with InfoPath forms

Such requirements present a few challenges, and a current project of mine fits into this category. At a high level, one consideration is that site users will also use 'system' pages provided by SharePoint in many scenarios (e.g. working with document libraries/lists, workflow etc.) and this doesn't happen in most WCM sites. This can lead to situations where there is a disparity between the look and feel of the 'published view' of the website and the 'system' areas. I don't intend to provide answers to all the issues here, but I do want to discuss a few as some food for thought. I'll probably revisit this post at the end of the project and provide a better insight into the issues and solutions, but for now let's cover some high-level decisions:

 

Approach for master page development

Options for starting development here include:

• Using a 'minimal master page' from MSDN or Heather Solomon
• Modifying a copy of default.master (good starting point for customized team sites)
• Modifying a copy of blueband.master (good starting point for WCM sites)

Partly this decision depends on where you are heading. Since the aim in my project is for formatting to be controlled by CSS rather than layout tables, starting with a minimal master page makes more sense (the shipped page layouts use tables). This is an interesting area since there's a lot of rework to be done to eliminate tables in a mixed publishing/collab site (and in fact it often won't be possible to eliminate them completely), and for me the benefit is debatable. Certainly all the 'system' pages which site users will be exposed to use layout tables, so I'm not sure how much is gained by only having some pages using CSS for layout.

Other things to consider here are the usual questions of how to factor responsibility of content items between the master page and page layouts, how to define content types etc., but these are standard decisions in WCM site development so I won't cover them here.

 

Use of Content Editor web parts vs publishing RichHtmlField controls

Most folks in WCM development know there is an overlap in functionality provided by the Content Editor web part and the RichHtmlField control in the Microsoft.SharePoint.Publishing.WebControls namespace, i.e. they can both be used to enter page content such as text/images. However it's important to consider the differences - the RichHtmlField control stores it's content in a column of the list item for the page, whereas the CEWP is a web part and thus stores content in the web part storage architecture. This is important, since if deployment to a different environment is in your project plan or ongoing architecture, things will likely be simpler if you use the field control, since this content will then travel with the page properly.

Additionally, there are some URL fix-up issues with using the CEWP across different environments, as documented in the HawaiianAir.com write up.

In summary, I'd recommend considering the CEWP as a means of entering content in non-publishing SharePoint sites only. 

 

Use of collaboration web parts - in layouts or in WebPartZones?

In a similar vein, since we are mixing the collaboration features into our site we are likely to need to use certain web parts which we wouldn't in a straight WCM site. In our situation the ListViewWebPart is fairly key to some areas, and is used as a means of allowing users to work with different lists from one page. The first decision here is whether the page layouts should include web parts directly (by adding them in SharePoint Designer), or just web part zones to which the individual parts would be added later through the browser. In most WCM scenarios I prefer to add web parts directly to page layouts since they will not be customized/personalized by end users (the main usage scenario for web part zones), and when web part zones are used, again the web part config is not stored in the page which can make deployment more complex. Using the other approach of adding directly from SPD, config is stored in the actual HTML markup of the page and so travels with the page layout itself.

However! The ListViewWebPart has some quirks which means it isn't always possible to use directly from the page layout. Specifically, it is only possible to configure the part to consume a list from the current web, and in the case of a publishing page layout, this means the root web since this is where the master page gallery is stored. Since our lists are stored in a child web, this is problematic - the other solution of using a DataView also had issues. Additionally, the ListViewWebPart configuration stores values specific to it's location, meaning the config XML is not very portable (i.e. export web part definition, modify, use). I'd like to think it would be possible with time to work out exactly which IDs do need to be changed, but alas we don't have time on this project.

As a result, using the ListViewWebPart in a web part zone is actually the best solution in these circumstances as far as I can see. We'll have an extra few steps at deployment, but this will take less time than the alternatives it appears.

 

Look and feel of system pages

As mentioned earlier, for a mixed WCM/collaboration site there can be a disparity of the look and feel of the main pages of the site and the 'system' pages users will see, i.e. pages from the '_layouts' directory. Note this happens even if both the site and system master page is set to point at your custom master page, since these pages are set to use 'application.master' (also on the filesystem in '_layouts') which neither of these properties affect. Sure it would be possible to simply replace 'application.master' with your own version, but that's not an elegant solution and would probably be unsupported. Unfortunately it seems that the architecture doesn't provide an easy way to change the master page used by '_layouts' pages - you have to go a level deeper to explore ways of doing this. Many .Net 2.0 developers will know it's possible to switch a master page dynamically in .Net, and to be fair this is what SharePoint does with the maser pages stored in the master page gallery anyway. I'm not aware of a truly elegant solution to this problem, but this discussion on Serge van den Oever's blog presents a viable approach using this technique. 

 

So those are some of the issues to consider. There are certainly others, including navigation, CSS customization of standard styles (to ensure collaboration web parts integrate well with your look and feel), and possibly the choice of authentication mechanism. I'll cover these and any others which arise in an upcoming post.

Modifying 'system' pages in SharePoint safely - with sample code

In a presentation I gave recently I talked about modifying admin screens within SharePoint. By this, I mean areas such as the site admin area and the central admin area:

Site admin:


Central admin:



Now, if your boss (or your users) asks you to add or change functionality in these areas, how would you handle that?

Well, you'd probably start off by looking to see how SharePoint implements 'system' pages like this. You'd then find they are just standard .aspx files in the 12\TEMPLATE\ADMIN folder on the filesystem. You'd also find that if you make a change to such a file, the changes are immediately reflected on the site - sure, some of the code is in a compiled assembly which is obfuscated (so Reflector doesn't help), but otherwise there's no mysterious abstractions, no caching, it's that simple. So changing the functionality or adding a link is a matter of changing the .aspx page right?

The problem with this approach is that modifying core files in this way is unsupported. This is because when a service pack or hotfix gets applied, your system will be in an inconsistent state and at best your changes will be overwritten. So a better way to implement such changes is to:

• take a copy of the original file
  
• modify the copy to include your customization
  
• create a SharePoint Feature (wrapped in a Solution package [.wsp]) to deploy your customized page to the 12 hive, and also add/modify the link in SharePoint to point to your customized page

The end result of this is that you have deployed your change without modifying any of the original files, and are therefore entirely supported. Additionally, because Features can be activated and deactivated with the click of a button, if you want to revert to the original functionality that's all it takes.

Let's look at an example - I'll link to all the code used here at the end of the article. Imagine your users are asking you to change the Recycle Bin screen so that it only displays items the currently logged on user has deleted - perhaps there's a manual document clean up exercise happening. And let's also say that after the exercise is complete, the users would like to revert back to the original functionality. (Now remember, what I'm demonstrating here is the mechanism for making customizations like this, so whilst this particular change might be unlikely in your organization, you might have other customizations which would help your users.)

So, first we need to modify the functionality of the Recycle Bin page to add the filtering. As mentioned earlier, we don't want to amend the original page, so we take a copy and modify that. In my example, I'm adding a script block to the code in front (since I don't have access to Microsoft's source code for the code-behind) to effectively perform the filtering at the UI level, which is fine for this example. Once we've created the page we want to use, we're ready to create the Feature and Solution to add our page to SharePoint.

Let's start with the manifest.xml file for the solution - this is important because this is how we actually deploy our custom .aspx page to the appropriate place within the SharePoint files. (Remember all this is available for download, the link is at the end of the article.)

<Solution xmlns="http://schemas.microsoft.com/sharepoint/" SolutionId="AA7EF934-C993-4f91-8AF4-0FB1D45927FA">

  <FeatureManifests>

    <FeatureManifest Location="COB.Demos.MyRecycleBinItems\feature.xml" />

  </FeatureManifests>

  <TemplateFiles>

    <TemplateFile Location="LAYOUTS\Custom\MyRecycleBinItems.aspx" />

  </TemplateFiles>

</Solution>

Notice what we are doing here is specifying that our custom .aspx page should be deployed to a 'Custom' subfolder within the 'LAYOUTS' directory. This is a good idea because it keeps our files separate from the core SharePoint files.

Then we have a fairly standard feature.xml file specifying the Feature identification info:

<Feature xmlns="http://schemas.microsoft.com/sharepoint/" Id="29D8B365-6024-4b1a-84D2-FF789FE56355"

        Title="COB.Demos.MyRecycleBinItems"

        Description="Adds a custom page to the Site Settings area to show items in the Recycle Bin which were deleted by the current user."

        Scope="Site" Hidden="FALSE"

        Version="1.0.0.0">

  <ElementManifests>

    <ElementManifest Location="elements.xml"/>

  </ElementManifests>

</Feature>

In this case we're specifying that our Feature is scoped at site collection-level, so we will modify all sites in the site collection.

Finally we have the our Elements file, which specifies what this Feature consists of. In this case, the key is the use of the CustomAction element - this can be used to add a link (i.e. an action) somewhere in SharePoint.

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <HideCustomAction Id="HideDeletedItems"

                    HideActionId="DeletedItems"

                    GroupId="SiteCollectionAdmin"

                    Location="Microsoft.SharePoint.SiteSettings" />

  <CustomAction Id="MyDeletedItems"

                GroupId="SiteCollectionAdmin"

                Location="Microsoft.SharePoint.SiteSettings"

                Sequence="10"

                Title="My recycle bin items"

                Rights="ManageWeb,BrowseUserInfo">

    <UrlAction Url="_layouts/custom/MyRecycleBinItems.aspx" />

  </CustomAction>

</Elements>

Let's walk through this file in detail. The first thing you notice is that we're using a 'HideCustomAction' element. You guessed it, this is used to hide a link in the SharePoint UI. Since we are effectively replacing a link, we actually need to hide the original one and supply our replacement, and this is done by using these 2 elements together. We assign the ID ourselves (if we want to refer to this 'HideCustomAction' elsewhere, this is the value we'll use), but all the other attributes of this tag must correspond to those specified for this action in the Feature Microsoft created to add the link to SharePoint in the first place. So in case you've not come across this yet, much of the original functionality in SharePoint is actually implemented using Features too. The way to find these values is to search in the 12\Template\Features folder. For this particular action, the most reliable thing to search for is the URL of the page, since any 'CustomAction' tag specifies the URL. Since we can find that the URL for the original Recycle Bin page is '/_layouts/AdminRecycleBin.aspx', searching for this finds the file containing the details of the original link. These are then used for our 'HideCustomAction' tag.

Next we specify our 'CustomAction' tag, again using many of the same values since we are replacing an existing action rather than adding a new one. The values we change are for the link title and URL, to point to our replacement page. Note also you can declaratively specify what permissions levels are required to see this link.

Once these files are made into a Solution package, deploying the Solution and activating the Feature has the following effects:

The link on the site settings page has changed from the original (on the left) to our replacement (on the right):

 A 'Custom' folder has been created in the '12\Layouts' directory and our custom .aspx page (MyRecycleBinItems.aspx) has been deployed to this folder. And of course, when a user clicks the link, the modified page has the functionality to only display items they have deleted:



The key thing of course, is that we've 'customized SharePoint functionality' without modifying any original SharePoint files - hence we are are 100% supported and can be confident our customizations won't cause any issues for service packs etc.

On the same lines, the sample stuff has a 2nd example - here we modify the 'Create Site Collection' page in Central Admin to display a guidance message to the SharePoint administrators. The idea is to encourage them to stop and think and perhaps consult the document containing the organization's policy on creating site collections:


Hopefully you might find this useful if you need to customize SharePoint in this way. The full set of files, plus the slides for the presentation I gave on this, are available at:

http://sharepointchris.googlepages.com/customizingsharepointsupportedway

In the next post, I'll be supplying some reference information which will help you make this type of modification in other areas of SharePoint. An important aspect of customizing SharePoint in this way is knowing exactly where and how the 'CustomAction' tag can be used, so that's what we'll look at.

'Customizing SharePoint the supported way' - slides and sample code now available

The slides and code from my presentation at last week's UK SharePoint user group meeting are now available - the topic was 'Customizing SharePoint the supported way - from end user to admin interfaces'. The slides etc. will appear on the user group website at some point but I've also published them at:

http://sharepointchris.googlepages.com/customizingsharepointsupportedway

The areas covered are:

• creating site definitions (and why it can be a good idea even for a WCM scenario with limited numbers of sites/webs being created)
• how to modify the site admin and central admin areas of SharePoint, without modifying any of the shipped files. The examples I used (the sample code zip file contains all the code/Features for these) were:
  
  - modifying the recycle bin to show only files deleted by the current user
  
  - modifying the 'Create Site Collection' page in central admin to display a policy message to the farm administrators
  
  
• using the DelegateControl to override the user controls used such as the 'My sites' link and the publishing console. (This assumes either default.master is being used or you have added the controls to your master page/page layout using <SharePoint:DelegateControl runat="server" ControlId="MyControlID" />). More details in my DelegateControl post.

Since the slide deck alone doesn't really convey the entire process, the next post will cover modifying the admin areas of SharePoint in detail. Included will be the list of areas where you can add/change links, and how to use a <CustomAction /> instruction within a Feature to do this.

If you need to make customizations to SharePoint functionality, the idea behind this technique is to make the changes without modifying the core files and thus putting your SharePoint installation in an unsupported state. This is very important if you need to make these kind of changes!

Considerations when using Features to deploy SharePoint files - ghosting/unghosting

In some of my earlier articles I talk about how to deploy various SharePoint artifacts as a Feature. In particular, 'Deploying master pages and page layouts as a Feature' discusses the idea of deploying these types of file (used in Publishing sites), but the concept applies to any file which will appear somewhere in a SharePoint list of some kind. In addition to the Master Page Gallery, other examples could include CSS/XSLT files being deployed to the Style Library, images to a picture library, and many other similar scenarios.

Occasionally people using this approach find that file updates can be difficult to apply. The process involves updating the Feature with the new version of the file, and then reinstalling and reactivating the Feature (typically the Feature version number will be incremented). In some cases the file updates successfully, in others it doesn't but there are no errors.

So what's going on?

The answer is that the file will not be updated (and therefore changes will not be seen in the browser for example) if that particular file has been 'unghosted' into the database.

<StartUnghostingExplanation>
[Without digressing too much, for anybody new to this concept it can be a confusing term which Microsoft are using less these days - the replacement term is 'customized' which is generally easier to understand. Depending on how they arrived in SharePoint, many files will initially only exist on the filesystem of the SharePoint front-end web servers. However, if such a file is customized e.g. it is checked out and edited (either through the UI, using SharePoint Designer etc.) SharePoint takes a copy of the file (now with modifications) and stores this in the content database instead of saving the changes back to the filesystem. This is the unghosting process. Whenever this file is requested in the future, the modified version from the database will be returned. This architecture allows SharePoint to scale to enterprise level, by only storing separate copies of files when absolutely necessary.]
<EndUnghostingExplanation>

When a file is provisioned in SharePoint using a Feature, SharePoint will copy the file to wherever your Feature specified, and will reference the file from this location. In many cases, this may actually be the 12\Template\Features directory. So in the case of master pages/page layouts etc. for example, when a web page is requested these resources will be retrieved from this location. Or at least, that's what happens if the file isn't customized.

If the file has been customized at some point, what happens when the Feature is updated is that the file on the filesystem is updated and reflects the changes, but SharePoint is now returning the copy in the database and so the update isn't reflected on the site!

So, if you want to use Features as your ongoing deployment strategy for your SharePoint document library (aka 'GhostableInLibrary') files, you should ensure the files can only be modified in this way. So files should not be modified either through the SharePoint UI or via SharePoint Designer. Typically, this won't be the case in the development environment but could be enforced for other environments. This means that unghosting is avoided, and SharePoint will continue to reference the copy on the filesystem.

Some other points of note:

• Another advantage of this approach is that when a Solution is retracted, the files will also be removed.
• Even a check-out/check-in operation without actual changes will cause SharePoint to see the file as customized, and will then be retrieved from the database rather than filesystem.
• A file can be 'uncustomized' in SharePoint Designer by right-clicking on the file and selecting 'Revert to site definition'. However, this will obviously cause you to lose any changes you have made to the file from the original version!

So next time you are using a Feature and wondering why the file isn't being updated, consider if the file has been customized!

Extending the Content Query web part

Something I've been meaning to put on Codeplex for a few weeks now is an extended version of the Content Query web part. This particular web part is one which many SharePoint developers will use quite often for displaying lists of links etc. However, anybody who's ever worked with it will probably tell you they spent a good hour (or more) on the first time, wondering how the hell to get the control to display either:

• custom columns on their list
• custom columns on the content type that the page layout uses (in WCM scenarios)

Since this doesn't happen automatically, it becomes apparent you have to tell the control the names of your custom fields. You might expect this to be exposed (like many other things are) by the control properties, but unfortunately it's not that simple. There is a property (CommonViewFields), but the tool part doesn't provide a UI to enter a value. This means that the value can only be supplied by digging a little deeper and changing the definition of the CQWP using it's .webpart file.

This involves exporting the web part from a page which uses it, amending the CommonViewFields element in the XML, then re-importing the .webpart file to the Web Part gallery. This new definition can then be added to your page, and you'll then get the right results.

Hmm. Why is it not a property on the control again?!

So this was the extension I was going to implement. As it goes, Ishai Sagi has beat me to it and is supplying additional functionality to boot. The shot below shows that now all you have to do is enter your field names into the tool part:

Ishai writes about this at http://www.sharepoint-tips.com/2007/06/adding-custom-fields-to-enhanced.html. His 'Extended Content Query web part' can be downloaded from http://www.codeplex.com/ECQWP.

Definitely worth a look if you use CQWP regularly.

UK SharePoint user group - Customizing SharePoint the supported way

The UK SharePoint user group is meeting this Thursday (28th June) at the Microsoft campus in Reading. There are two sessions, one of which I'm presenting:

Customizing MOSS the supported way – from end-user to admin interfaces

One of the key concepts SharePoint developers should be aware of is that modifications to core shipped files are typically unsupported. This session looks at ways in which you can implement the kind of customizations your users may ask for, but without modifying the original files. Aspects covered include custom site definitions, modifying the site administration/central administration areas, and changing the user controls SharePoint uses with the DelegateControl architecture. The session is packed with demos, and also contains some quick tips on the best way to make other common customizations not shown in the detailed examples.

Speaker: Chris O'Brien

..and also:

Groove: A powerful tool for collaboration, that is now part of Office 07.

In this session, we review the basic functionality and then show, using the latest Microsoft templates (including issue tracking and document review), how Groove is not just a tool but a platform for delivering functionality. We'll also look at how Groove and SharePoint can both effectively work together.

Speaker: TBA

If you're in the area and would like to register, simply leave your name on the following thread over at the UK user group site:

http://suguk.org/forums/thread/3534.aspx

I'll be posting my slides and sample code after the event. The content on customizing the site admin/central admin areas will probably make it's way into separate posts for clarity.

Using the Delegate Control

When starting to build MOSS sites, many developers will start the page template development process by analyzing the shipped master pages and/or using the 'How to create a minimal master page' MSDN article. Whilst very useful, for simplicity the example in the article unfortunately does not include some of the functionality the SharePoint team implemented in default.master, specifically the use of the Delegate Control for the page to load controls. I think this is a shame, since many developers who don't take the time to specifically look at it may therefore be unaware of how powerful the Delegate Control can be.

So what is it?

Essentially the Delegate Control provides an alternative to adding user controls and server controls to a .aspx page in the normal way. By this, I mean adding the control to the page by dragging from the toolbox in the development environment or by adding the appropriate markup manually. Instead, with a Delegate Control all we do is add the markup to instantiate a Delegate Control instance, and use a Feature to specify separately which control should actually be loaded. What this effectively gives us is the ability to control what controls are used on a page without having to directly modify and redeploy master pages/page layouts. This extra level of abstraction can be quite powerful, since any feature scope can be used with the Delegate Control. I'll come back to this, but enough theory for now. Here's how it's used:

First off, we need a Delegate Control declaration on our page. This will look something like:

<SharePoint:DelegateControl runat="server" ControlId="PageHeader">

</SharePoint:DelegateControl>

Only thing to note here at this stage is the ControlId attribute - the Feature we create will use this to substitute the real user/server control.

Then we have the feature.xml file, where we specify the feature details (including scope):

<Feature xmlns="http://schemas.microsoft.com/sharepoint/" Id="373042ED-718D-46e2-9596-50379DA4D522"

Title="COB.Demos.DelegateControls"

Description="Specifies which user control should be used for the 'PageHeader' DelegateControl used on the site master page. The replacement user control is stored in the CONTROLTEMPLATES directory." Scope="Farm"

Hidden="FALSE"

Version="1.0.0.0">

<ElementManifests>

<ElementManifest Location="elements.xml"/>

</ElementManifests>

</Feature>

As always, the 'instructions' for the feature are in the element manifest:

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

<!-- using a sequence number LOWER than default of 100 so our custom control gets loaded -->

<Control Id="PageHeader" Sequence="90" ControlSrc="~/_ControlTemplates/COBPageHeader.ascx" />

</Elements>

This is where we specify which control should actually be used. In addition to specifying the path, the key thing is that the 'Sequence' attribute contains a value lower than any other <Control> instructions for this control (i.e. ID of 'PageHeader'). This is especially important when we are overriding an existing Delegate Control created by Microsoft - here the default value is 100 so your sequence must be lower than this for your control to be loaded instead.

In addition to creating and activating this feature, the actual .ascx file must exist in the location specified. It can be copied to the CONTROLTEMPLATES directory manually, but a better idea is to wrap the feature up as a solution, since solution packages can also deploy files. To deploy the .ascx file along with the solution, your solution manifest file should look something like:

<Solution xmlns="http://schemas.microsoft.com/sharepoint/" SolutionId="E8694626-60F8-4d07-9140-8F9F634020DE">

<FeatureManifests>

<!-- note this is the location in the cab file! -->

<FeatureManifest Location="COB.Demos.DelegateControls\feature.xml" />

</FeatureManifests>

<TemplateFiles>

<TemplateFile Location="CONTROLTEMPLATES\COBPageHeader.ascx" />

</TemplateFiles>

</Solution>

These files would now all be packaged as a solution (.wsp) in the usual way with makecab.exe. When the solution is deployed, the file will be copied to the right place on all the web front-ends in your farm, and when the feature is activated SharePoint will know that it should henceforth load 'COBPageHeader.ascx' for any Delegate Control with an ID of 'PageHeader'.

So what's so great about that?

Well, a couple of things:-

• I can now override which control a page should load without having to go back, edit the template and redeploy
• By using a feature scoped at 'Web', I can effectively use a different page header for different areas of my site without requiring different templates or code. (Note I've not tested this extensively, but since the <Control> element can be used at scope Web, Site, WebApplication or Farm, this should be perfectly feasible.)
• I can use any standard .Net user control or server control, so for anybody familiar with Jan Tielen's SmartPart, this provides similar capability.

In terms of functionality, there's a couple of other things I want to highlight:

• Parameters can be passed to the control via the declaration on the page. To read these, the control's implementation should walk up the control tree to get the values. An example would be:
  

<SharePoint:DelegateControl runat="server"

ControlId="PageHeader" MyParam="MyValue">

</SharePoint:DelegateControl>

• By adding AllowMutiple="true" to the declaration, you can make the Delegate Control load more than one user/server control.
• As mentioned earlier, many of the controls used on default.master are loaded using the Delegate Control. These include global links such as My Site/My Links and the publishing console. So using this approach, customizing the publishing console is a simple matter of providing a replacement .ascx and creating a feature as described here!

Hopefully this has given some insight as to why you should consider using it on your templates. As a final thought, how about combining with a web part so that you can simply re-use existing ASP.Net user/server controls as web parts?

It's 6pm - where are your virtual machines?

I mentioned last time that I'd share a couple more of my SharePoint-related highlights from Tech Ed. On a general note, despite the fact this wasn't my first Tech Ed I'm still amazed at the scale - an estimated 14,000 people in total apparently. Incredible when you think the team can feed 14,000 people in one hour in one room! As my colleague remarked, even Jesus only managed 5,000 ;-)

Anyway, things I didn't mention last time:


Virtual Machine Manager

- Microsoft beefs up it's story on managing virtual machines with Virtual Machine Manager 2007 (part of System Center family). I know other products out there have much of this functionality, but let's face it, for many of us management of virtual machines currently extends to the Virtual Server 2005 admin website. Some of the things I liked were:

• 'library' of virtual machines including metadata - including the facility to have template machines which can be used as building blocks
• fast 'physical to virtual' conversions
• Intelligent Placement tool - analyzes all your physical hardware you have allocated to virtual machines and provides a recommendation as to where to put a new VM. This is based on information it has gathered (analysis of hardware etc.) but the parameters can be weighted, such as processor, memory, disk space etc.
• ability to delegate starting up (etc.) of the VM from the physical box, without granting them remote access to the physical machine or other VM management capability

SharePoint development thoughts

- On other matters, an interesting (and accurate I feel) angle on SharePoint development - whilst the current dev experience is often painful, this is an issue with the tools rather than the platform. Ted Pattison talked about the 'denial, anger, acceptance' cycle often seen in .Net developers starting SharePoint development. SharePoint 2003 developers on the other hand think everything is slick in comparison ;-) Remember that in the .Net world, the tool (Visual Studio) is developed and released in parallel with the platform (.Net). In the SharePoint 2007 world, for whatever reason (likely to be pressure to release at same time as Office client apps) this hasn't been the case, and so the tools are perhaps 18 months behind the client. So there may well be a time when you look back and chuckle at the idea of generating feature files/solution packages without an MS-developed tool.

- Some interesting examples of using the DataView web part from Dustin Miller. This thing alone can get you a long way to building composite applications in SharePoint. Most SharePoint developers will already be switched on to this idea (using WSS as a development platform rather than building from scratch in ASP.Net), but it could be the future for a lot of reasonably simple apps. Some of the functionality includes:-

• Display data from SQL, XML or web service
• Join disparate datasources without code (need at least a string value [or other 'joinable' type] to join on)
• Implement in-line editing (a la DataGrid)
• Perform conditional formatting (if x == 1 show a, otherwise show b)
• Use in a connected way with other web parts

..all without code. Very cool indeed. Note that the control renders using XSLT which can also be customised to get the required output. Also impressive is that SPD actually provides help by allowing in-line editing of the rendering in the design view, and mini-wizards to help with conditional formatting. The underlying XSLT is then written by the tool.

So it's not all bad with SharePoint development ;-)

SharePoint content at TechEd

Some good SharePoint content this week here in Orlando, which is good as it's pretty much the reason I'm here ;-) Thought I'd call out a couple of the things I've heard which have grabbed my attention:-

• an update to VSeWSS which addresses some of the issues I and other people have had with it, specifically the fact that it's not flexible enough for some scenarios. The next version will allow editing of the files generated (manifest.xml, feature and element definitions) before being packaged into the .wsp file. This will allow you to add feature receiver definitions or otherwise modify the generated XML. A CTP should be available in July.
• hearing MS's experiences on their internal MOSS deployment, believed to be the biggest in the world. How d'ya fancy managing 134,000 site collections running on 130 web front-ends?
• an extremely interesting 'participation' session on SharePoint deployment (see my features vs. Content Deployment post) lead by Andrew Connell. This is an area of SharePoint close to my heart (though I'm by no means saying I have all the answers) so it was good to have a beer afterwards with Andrew and continue the discussion. I'll be writing about this more soon.

On a general note it's great to see many SharePoint experts speaking here. A full summary of information useful to SharePoint implementors to come soon :-)

Feature stapling

One aspect of features I wanted to cover quickly is the relationship SharePoint Features have with site definitions. Site definitions are created by a developer and subsequently used by site owners to create new sites. Much of this facility is also available by the 'Save site as template' functionality, though there are some minor differences. Additionally the 'Save site as template' functionality is removed on SharePoint publishing sites (or at least the link in Site Settings is via a HideCustomAction feature element), due I think, to some site information/functionality which SharePoint cannot properly save into a .stp file. So developers often create site definitions as part of their customization, and it's a process well-documented in the WSS SDK.

If you have created a site definition, there are several ways of using features with it. These are:-

1. Create a site from the definition and subsequently deploy your features to the site.
2. Include the features in the site definition (onet.xml) before using it to create sites.
3. Use feature stapling to associate your features to the site definition.

Since the first option doesn't really associate the Features to the site definition, we'll focus on options 2 and 3.

2. Include features in site definition

This option should be used if the site definition has not yet been deployed, and no sites have yet been created from it. The section of the onet.xml file where features are associated with the site definition contains may look something like:

<Configuration ID="0" Name="BLANKINTERNET">

    <SiteFeatures>

        <Feature ID="C85E5759-F323-4EFB-B548-443D2216EFB5" />

        <Feature ID="A392DA98-270B-4e85-9769-04C0FDE267AA" />

        <Feature ID="7C637B23-06C4-472d-9A9A-7C175762C5C4" />

    </SiteFeatures>

    <WebFeatures>

        <Feature ID="00BFEA71-DE22-43B2-A848-C05709900100" />

        <Feature ID="00BFEA71-E717-4E80-AA17-D0C71B360101" />

    </WebFeatures>

    <Modules>

        <Module Name="Home" />

    </Modules>

</Configuration>

The ID value is, of course, the feature GUID. The SiteFeatures element contains features which should be activated when the site definition is used to create a site collection, and the WebFeatures element contains features for when the definition is used to create a standard web within a site collection.

3. Use feature-stapling to associate your features to the site definition

This option should be used when the site definition is already in use (and sites have been created). This is because there are risks in modifying a site definition once it has been deployed, since the site definition files on the filesystem are used for pages which are ghosted (not modified).

To use feature-stapling, you first need the feature(s) you wish to link to the site definition - these should be created in the normal way. The, you should create a 2nd feature which does the actual stapling. Sample values for the various files are shown below:

The manifest.xml file if you are wrapping your feature in a solution:

<?xml version="1.0" encoding="utf-8"?>

<Solution xmlns="http://schemas.microsoft.com/sharepoint/" SolutionId="26E1A1D9-7BB2-4e76-888E-B20184B16E3B">

  <FeatureManifests>

    <FeatureManifest Location="COB.Demos.FeatureStapling\feature.xml" />

  </FeatureManifests>

</Solution>

The feature.xml file which defines the feature 'header' information:

<?xml version="1.0" encoding="utf-8" ?>

<Feature Id="4AF9999A-0517-4224-9ED3-D2F9F87D92E2"

        Title="COB.Demos.FeatureStapling"

        Description="Staples a set of features to custom site definitions. The features/site definitions are defined in stapling.xml."

        Version="1.0.0.0"

        Scope="Farm"

        Hidden="FALSE"

        xmlns="http://schemas.microsoft.com/sharepoint/">

  <ElementManifests>

    <ElementManifest Location="stapling.xml" />

</ElementManifests>

</Feature>

And finally, the core feature definition in the stapling.xml file:

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <FeatureSiteTemplateAssociation Id="F6924D36-2FA8-4f0b-B16D-06B7250180FA" TemplateName="MySiteDefName#0" />

  <FeatureSiteTemplateAssociation Id="94C94CA6-B32F-4da9-A9E3-1F3D343D7ECB" TemplateName="MySiteDefName#0" />

  <FeatureSiteTemplateAssociation Id="02464C6A-9D07-4F30-BA04-E9035CF54392" TemplateName="MySiteDefName#0" />

</Elements>

The FeatureSiteTemplateAssociation element maps feature GUIDs to site defintions - note that the format of the TemplateName attribute value is <SiteDefName>#<ConfigurationID>. This obviously allows a degree of flexibility and allows you to do fairly complex things with different configurations of site definitions.

The final thing to note is that feature-stapling won't affect any sites which have already been created from a site definition, only subsequent sites. The only way to do this is to activate your features individually on the site, though obviously an STSADM script can help you activate multiple features against multiple sites.

SharePoint deployment options : Features or Content Deployment?

[Update 23/05/07 - clarified emphasis here is on WCM sites and added more detail on conclusion]

Today I want to document some of my thoughts on deployment in MOSS - this article is intended to provide a starting point for analysis of the different ways to deploy SharePoint artifacts between environments, particularly with regards to WCM sites rather than team/collaboration scenarios. At a high level, the options are:

• create artifacts as features/solutions
• use SharePoint's Content Deployment framework
• recreate artifacts manually in each environment

Interestingly, the message from Microsoft and other respected sources was initially that features/solutions was the way to go. This seems to have changed recently with Microsoft's white paper on team-based development in SharePoint. I recommend a thorough reading of this paper, but the summary is:

• an authoring environment is provided where developers will create artifacts such as lists, site columns, content types, master pages etc. directly (N.B. this is probably the same environment where your client does content authoring using approved master pages/page layouts)
• artifacts are created directly using the SharePoint UI and SPD rather than using features
• assemblies should be deployed to this environment (and the final production environment) using SharePoint solutions, to leverage the solution framework's ability to deploy to multiple WFE's in one step
• Content Deployment is used to get everything (except assemblies and config settings - see above) to the production environment
• alternatively, the authoring environment can be the production environment if required - use versioning/content approval to control what the end-user site consists of (I recommend having separate environments - I think this is preferable to developing master pages/page layouts in production)

Having gone through the process of creating an entire site with features, I can understand why the guidance would be to avoid using this technique in favour of something simpler. Essentially, I don't think the complexity/overhead of feature development can scale for all MOSS projects, it just takes too much effort. Additionally as I have pointed out earlier, there are issues with updating files which were initially provisioned using one of the original features. However, I also see some limitations with the approach suggested in the whitepaper:

• the integration environment doesn't make too much sense when you consider that some of the code being developed may require the SharePoint artifacts (site columns, lists etc.) to be present. If they are not created until the authoring environment, this type of code will fail/cannot be tested here.
• the deployment process effectively means that full integration happens fairly late in the process. The authoring environment is the first environment where all code and artifacts will be present, and this is just one step away from production. In comparison to an architecture which consists of dev/QA/staging (UAT)/production environments, and thus has more than one opportunity to test integration and deployment, this may pose extra risk.
• one less environment for testing. As in the previous point, some organisations use an internal QA environment for their testing, and then a separate one for client UAT.

An additional environment could resolve some of these issues, but that does mean more content deployment jobs to deal with, and that's not always straightforward.

Despite this, my thinking is that the optimum deployment strategy for a WCM site makes use of both content deployment and solutions/features. The former is used to deploy most SharePoint artifacts (including master pages, layouts etc.), but the latter is used for assets which are mainly based on the filesystem:-

• assemblies
• web part definitions (though these go in the database)
• config settings
• site definitions (including feature stapling)
• DelegateControls

(I cover those last 2 items in a forthcoming article btw).

I think this works the best because the feature framework works particularly well for provisioning, but not so well for ongoing site management. This is why there is something of a distinction between WCM sites and team sites. In a team site, the site is provisioned initially with a set of functionality and default content, and then ongoing management is really restricted to development of new bits of functionality (hence features suit this requirement well) and content/documents are added by users. A WCM site often requires very careful lifecycle management of functionality (and content), and I don't think the feature framework caters well for this.

Comments especially welcome on this topic!