Sunday, 25 November 2007

Part 2 : Blending publishing/collaboration functionality in SharePoint

A couple of months ago I wrote an article on some of the issues around building sites which are not pure publishing or collaboration solutions, but which mix features of the two. I mentioned at the time that I would follow up on these issues and any others we came across at the end of the project, so here it is.

As a quick recap, we said that characteristics of a site which blends publishing/collab features could be:

  • site has completely bespoke look and feel/navigation
  • users will work with document libraries, discussions, calendars etc.
  • site templates or definitions are used to create several sites with the same content/functionality
  • custom workflow is used to support a business process (other than standard content publishing), perhaps with InfoPath forms

    So let's cover some of the issues you may face in projects like this:

    Look and feel of system pages

    This refers to the fact that whilst pages you create will happily use your custom master page, any 'system' pages in the '_layouts' directory will not share this look and feel by default since they reference a separate master page, 'application.master', which is also stored on the filesystem in '_layouts'. Such pages are often seen due to the use of collaboration functionality in our site, examples being when a user uploads or edits metadata for a file, starts a workflow etc.

    In fact this turned out to be less of an issue than I anticipated because:

    • a quick test showed that the use of a HTTP module to switch the master page of these pages works fine. Several people have documented this approach, and I note that Ted Pattison has provided a Codeplex feature for those who would prefer not to write their own code for this. (N.B. though not strictly supported, the idea behind the HTTP module approach is that it is better than modifying files installed with SharePoint such as 'application.master', and can be disabled with a switch if we need to fall back in line with official support.)
    • our client decided this was a low priority item, that we may or may not deal with further down the line

    On this subject, I see that Microsoft have themselves released KB article How to customize application pages in the Layouts folder in SharePoint Server 2007 and in Windows SharePoint Services 3.0. Bizarrely, the proposed solutions are modifying the standard files but taking backups of the originals, or duplicating the 'layouts' virtual directory in IIS - the HTTP module idea is not mentioned. Without dwelling on this too much, this seems crazy to me and I'm not the only one. I still much prefer the HTTP module solution and will use this on my projects where it is required, though will be sure to regression test thoroughly.

    Use of InfoPath with forms authentication

    This is an interesting illustration of blending the two types of functionality - collaboration solutions may well use InfoPath for forms, but publishing sites typically use forms auth for a customized/branded login experience. Straight away we ran into problems using InfoPath with forms auth - essentially the link to create a new instance of an InfoPath form in a forms library would disappear (example link shown below [Windows auth enabled here]):

  • It took a while for me to track down that this is related to the 'Client Integration' switch in Central Admin. For a forms auth site, this gets set to 'off' by default, but defaults to 'on' for windows auth sites. Unfortunately, resolving the problem isn't as simple as flicking the switch back, since what the switch does is remove certain links and disable ActiveX controls which integrate with the browser for link-handling. With the switch enabled again, the InfoPath link reappears on the content type, but when a link to say, a Word document is followed, the browser now shows the basic prompt to download or save the file. This means the more sophisticated 'do you want to check out or open as read-only' prompt is lost and users must work with documents offline and upload to SharePoint at the end of editing, rather than being able to save back directly to the SharePoint repository. In other words, unacceptable to most for a SharePoint 2007 collaboration solution.

    In the end I found others in the same position, but didn't find a solution in a short timeframe. Microsoft support tells me "they were unaware of any instances of of these two features working together in this way", so the disappointing conclusion is that these two bits of functionality don't play well together. Our client thus decided to accept windows authentication (our authentication store was always going to be AD rather than SQL anyhow). However, I'm actually convinced it would be possible to get this working given more time - I think the way forward would be to amend the site design to have a link to the InfoPath form in the standard content of a page rather than relying on SharePoint to supply the link. The information supplied in How to: Use Query Parameters to Invoke Browser-Enabled InfoPath Forms would be used here. I'm sure this must be possible and others have done it, I'd be interested to hear from anyone who has.

    Overidden control not appearing on standard collaboration pages

    Our client had requested some customizations to the search box, and going through our bug list towards the end of development, I realised that our customized version was present on pages we had created, but not existing list pages such as [MyList]/Forms/AllItems.aspx or similar. This seemed strange given that all pages were using our custom master and shared the same header! Digging deeper, this is caused by the fact that SharePoint uses a delegate control to load the search box .ascx file, and to properly override the control across the site, we needed to create a Feature to specify that our .ascx file should be used in preference to the standard control. If we had directly replaced the existing control on the filesystem this would not be required, but when adding custom files in order to avoid modifications to existing files, this extra piece is required. One to look out for when building sites of this nature.


    I read some blogs/discussion threads earlier this week which suggest customizing SharePoint in this way is nigh on impossible, but our project (and surely many others) prove this isn't the case. As is often the case with such a functional platform, there's perhaps a greater chance of success if you understand the nuts and bolts of what SharePoint is doing, but extensive customizations to collaboration-focused sites are inherently possible!

    Sunday, 11 November 2007

    Change a SharePoint site's URL

    Something you may find yourself tasked with at some point, is changing the URL of an existing SharePoint 2007 site. This is a fairly interesting scenario, and it's fair to say the relationship between SharePoint and IIS makes this more complex than for a standard .Net site. However, there are several possible solutions. The first things many of us would think of as potential approaches would probably be:

    • extending the web application onto another URL
    • using Alternate Access Mappings somehow

    Depending on your site both could be valid methods, but as with anything SharePoint-related, there are different things to watch out for with the different approaches. As an example, extending the web application wasn't the right approach for our scenario for the following reasons:

    • the site shouldn't actually exist at the old URL, but a redirect was required
    • InfoPath forms don't seem to deal well with the 'extended web application' configuration. (Problem detail - on one URL everything will be fine, but if the two web applications use separate site collections, on the other you're likely to see security errors when opening forms. This is because the form templates are referenced in the other site collection - a document library can only have one URL to the document template, and publishing a form to a content type stores an absolute link in the content database.)

    Additionally some quick tests with Alternate Access Mappings didn't seem to give the expected results for me, so I decided on another approach since I knew it would work and didn't have much time for experimentation. So this was my process:

    Changing a site's URL by recreating the site (downtime required)

    1. Stop old IIS site.
    2. Create new web application in SharePoint, bind to new IP address in IIS.
    3. Apply SSL certificate if appropriate.
    4. Create new site collection for this web application using the blank site template.
    5. Export content using the SharePoint's content migration API (I have a tool which does this, which will shortly be on Codeplex) ensuring all security data is exported. Alternatives to this step include STSADM -O BACKUP and STSADM-O EXPORT. *
    6. Import content into the new site collection, ensuring to include security.
    7. Amend any absolute URLs in .udcx data connection files used by InfoPath.
    8. Republish any InfoPath forms to the new site.
    9. Configure search:
      1. Ensure new URL is a content source.
      2. Update any crawl rules which use absolute URLs.
      3. Update 'authoritative pages' as appropriate.
      4. Start full crawl.
      5. Update scopes.
      6. Go to Site Settings > Site collection administration > Search scopes, add any custom scopes to search dropdown (if using standard search web parts).
      7. Ensure search web parts use relative URLs/do not reference old site URLs.
    10. If a redirect from old URL is required, create new IIS site to implement this:
      1. Create new site in IIS and bind to old IP address.
      2. On 'Home directory' tab, specify content should come from 'A redirection to a URL' and enter the URL.
    11. Ensure DNS/firewalls are configured appropriately for new URL, remembering to allow appropriate time for DNS propagation.
    12. Perform testing.

    * N.B. Between the content migration API and STSADM export, I prefer the former since this allows control over whether object GUIDs are retained (more information in STSADM export, Content Deployment, Content Migration API, Features/Solutions - deployment options compared). STSADM backup/restore is discussed in next section.

    Considerations to this approach

    • When using the content migration API or STSADM backup/restore, the following items are not included - alerts, workflows, recycle bin state or site collection properties. These must be migrated/recreated separately.
    • Regression testing is absolutely required since the site is effectively recreated

    As a way to improve on the first consideration, another option would be STSADM backup/restore (though I've not tried this approach). Notably this method does collect data for the items which the other approaches exclude, however due to the nature of our site, none of these were significant problems.

    So this method was successful, and hopefully this information allows folks to see some of the pros and cons without having to spend the time going through it themselves. However, I also note an approach based on Alternate Access Mappings suggested by Faraz Khan. Since this was only published in the few days before this article it was too late for my scenario, though I'd encourage you to take a look. Note that Faraz also points out considerations such as certain links not being updated to new URL without fix-ups, though this doesn't seem to be a major issue. It does echo my point about there being different things to watch out for with the different approaches, but both methods provide valid techniques for changing a SharePoint site's URL.

    Thursday, 1 November 2007

    Master pages/page layouts deployed as Feature not updating

    Since Deploying master pages and page layouts as a Feature has been the most heavily commented article on this blog, and several of the posters seem to have run into the same problem, I wanted to write a quick post with some more information from my experiences on this.

    So this is something of a non-standard post, feel free to tune out if it doesn't affect you ;-)

    Anyway, I decided to do some more testing to see if either I'd got something wrong or if perhaps I was doing something differently to the people having problems. My test was basically to knock up a publishing site with a master page and page layout (associated with a custom content type as it often would be), then go through the update process. This is what I found:

    • making updates to the files (outside of the 12 folder) and then XCOPYing these to overwrite the files in the 12\TEMPLATE\features\MyFeature\ directory successfully updated the site. No need to deactivate/activate the Feature.
    • using a Solution package to deploy the files (when using Features this is generally what I do since I'm in a farm environment) - again this updated the site correctly when I upgrade the Solution (stsadm -o upgradesolution). This is to be expected since underneath the exact same thing is happening as in the previous test. (However, I also noticed occasionally the directory would complete disappear even after the solution upgrade had completed, or was there but still locked by another process, meaning the files could not be accessed even in Windows Explorer - this is slightly irritating but running the Solution upgrade again always succeeded.)
    • after causing the file to be customized (e.g. modifying or even just checking out with SPD), any subsequent updates to the files via the Feature/Solution did not appear on the site (though further updates in SPD are fine).

    In short, this is all what I expected. Assuming the file has not been customized, anything which updates the copy on the filesystem will cause an update to the site. If it has been customized, updates on the filesystem will not (since the file has now been added to the content database, and the filesystem version is no longer used). If you've not come across this before, Considerations when using Features to deploy SharePoint files - ghosting/unghosting may help.

    So, I'm guessing that if your master pages etc. are not updating when you overwrite the Feature files, it's because the files have become customized somehow. Unfortunately it's not so easy to tell for publishing files - for other files, SharePoint Designer provides a handy blue dot next to the file in it's Explorer view if the file is customized, but alas this doesn't happen for master pages/page layouts. The blue dot can be seen next to the AllItems.aspx file below (click to enlarge):

    Unfortunately this also means reverting to the file on the filesystem is not straightforward either (we can't right-click the file in SPD and select 'Reset to site definition' as we can with other SharePoint files). So this can be a pain if you do want to keep your page layouts referenced from the filesystem (e.g. because performance is critical), but you've ended up in this state. It is possible to revert the files using the API though. I've not needed to do this myself, but the property to check is SPFile.CustomizedPageStatus and if this returns SPCustomizedPageStatus.Customized, then the SPFile.RevertContentStream() method can be used - this should cause SharePoint to henceforth use the version on the filesystem (though note you may lose some updates which had been made after the file was unghosted (customized) - you will need to re-apply these to the filesystem file after the reversion. And remember, don't use SPD for this or you'll be back where you started!)

    So far, so (reasonably) straightforward.

    However, one poster (deelpunt) had an interesting question about updating page layouts with web parts. As far as I can see, updating all page instances to have web parts in web part zones by updating the layouts will not be possible. This is because if web part zones are used, the web part is associated with the page instance rather than the page layout. Indeed, this can be the power of the architecture. It is possible to either:

    • have default web parts added to a zone when a page instance is created from a page layout. This can be done by deploying the page layout using a Feature, and using the AllUsersWebPart tag. This would not affect pages already created however.
    • add web parts to all the pages by adding them directly to the page markup in SPD, rather than in a zone. Of course, this then means the settings for the web part can only be modified by the page designer in SPD, rather than site users.
    • use the API to iterate through all pages in the site to add/modify webparts using SPFile.GetLimitedWebPartManager(). Needless to say, this is the kind of operation which requires a lot of care and planning in production!

    As I've mentioned before, because of these issues the web part zone architecture is often not the best choice for scenarios such as WCM site development, since here we want our changes to apply across all pages which use the layout.

    Hopefully this has been of some use. As always, leave a comment if you've had different experiences to those detailed here, I'd definitely be interested to hear.

    [I also wanted to say sincere apologies to the commenters on the original post (and any others I've been slow in replying to) that it took a couple of weeks for me to come back. Something to do with my project going live and moving house at the same time, hopefully normal service now resumed ;-) ]