Tuesday, 31 July 2007

Article series - custom permissions with a site definition

This is an introductory article which gives the background to an interesting problem I've seen recently. Over the next 3 articles, I'll be looking at a solution to the problem and will discuss the following:

What I aim to show is how to deploy a site definition which has custom permissions 'attached' to it. By this I mean that any sites created with this template will acquire a set of custom permissions, without a user/admin having to configure them separately. What we're effectively doing is automatically putting a set of users into SharePoint site groups (at the time of site creation) - assuming the site was called 'HR', these would be:

  • HR owners
  • HR members
  • HR visitors

This can be extremely useful in certain scenarios when users are creating their own sites and workspaces in SharePoint, i.e. collaboration scenarios. One example might be a situation where say, the senior management of your organization (who are all fairly familiar with SharePoint of course ;-)) want to create sites for an important initiative they have. They want to do this on an ad-hoc basis without involving IT, but the sites must be restricted access such that only certain people can use them. Additionally, the requirements might be that some people have full control, but others can only read.

Since these security requirements are probably different to those of the parent site collection, the site must implement 'unique permissions'. This is key - if the site can share (and that's share, not inherit - think about the difference) the permissions of the parent site, then custom permissioning is not required. Otherwise if you are happy to set up the custom permissions manually each time a site is created, that can be done simply too without a solution like mine. SharePoint provides dialog screens for this during the site creation process, if you select 'Use unique permissions' in the Permissions section.

However, when it should happen automatically, you need to perform some custom processing during the site provisioning process, which is what I'm illustrating here. So with the background out of the way, the articles will consist of the following:

Article 1 - Site definition basics

  • Process for creating custom site definitions
  • How to deploy using a SharePoint Solution package (.wsp)

Article 2 - Custom code in the site provisioning process

  • When this is necessary
  • How to get your code to run at this time

Article 3 - how to implement unique permissions for sites created with the definition

  • Storing permissions data in a secured SharePoint list
  • Feature Receiver code to apply the permissions

Article 1 in the series should be published in a couple of days. See you then!

Sunday, 22 July 2007

Building and deploying SharePoint Solution packages

Following a comment left elsewhere on this blog, I thought I'd cover exactly what SharePoint Solution packages are, including how they are built and deployed to other SharePoint environments.

First of all, let's be clear on exactly what a Solution package is. Note I'm using 'Solution' with a capital S  in this article to distinguish between what we are discussing here and the general idea of a technical solution built using SharePoint. A Solution is a package of SharePoint items for deployment - in physical terms it is a cabinet file (.cab) which is given the extension of .wsp to differentiate it from standard .cab files. Other articles on this blog cover the idea of using SharePoint Features to deploy functionality, so let's also be clear on the relationship between Features and Solutions. In general terms, some of the tasks which cannot be done with a Feature alone but can be done with a Solution include:

  • Deployment of certain files to the filesystem, e.g. an assembly for workflow or web parts, custom files which will reside in the ‘12’ folder 
  • Deployment of web part definition files (.webpart)
  • Web.config modifications e.g. the ‘SafeControls’ entry required for a custom web part
  • Code Access Security config modifications e.g. those required for custom web parts not running from the GAC

In addition to being able to do these things, Solutions can also contain Features. The way I think of it is that the Solution wraps the Feature. In actual fact, I always recommend that even if they don't need to be (e.g. we're not doing anything in the list above), Features are always deployed as part of a Solution.  The reason for this is that the Solution framework takes care of deploying all required files to all Web Front End (WFE) servers in a SharePoint farm. This alone is incredibly useful in working towards repeatable deployments, and ensures your WFEs stay synchronized.

How Solutions are specified

The key file used to specify what a Solution package consists of is the manifest.xml file. Going back to my earlier post on deploying web parts, the manifest.xml file for that scenario looks like this:

<Solution xmlns="http://schemas.microsoft.com/sharepoint/" SolutionId="122C0F04-78B7-4d42-9378-6F8B4F93ADD1">

  <FeatureManifests>

    <!-- note this is the location in the cab file! -->

    <FeatureManifest Location="COB.Demo.WebPartDeployment.WriteToFileWebPart\feature.xml" />

  </FeatureManifests>

  <Assemblies>

    <Assembly     Location="COB.Demo.WebPartDeployment.WriteToFileWebPart\COB.Demo.WebPartDeployment.WriteToFileWebPart.dll"

        DeploymentTarget="GlobalAssemblyCache">

      <SafeControls>

        <SafeControl Assembly="COB.Demo.WebPartDeployment.WriteToFileWebPart, COB.Demo.WebPartDeployment, Version=1.0.0.0, Culture=neutral, PublicKeyToken=9f4da00116c38ec5"

                    Namespace="COB.Demo.WebPartDeployment"

                    Safe="True"

                    TypeName="*" />

        </SafeControls>

    </Assembly>

  </Assemblies>

  <DwpFiles>

    <DwpFile

     Location="COB.Demo.WebPartDeployment.WriteToFileWebPart\COB.Demo.WebPartDeployment.WriteToFileWebPart.webpart"

      FileName="COB.Demo.WebPartDeployment.WriteToFileWebPart.webpart" />

  </DwpFiles>

</Solution>


This is assuming the web part is being deployed to the GAC - for this illustration this is a simpler scenario than deploying to the web application's bin directory, where Code Access Security (CAS) policy would also be required. Effectively, the manifest specifies that the Solution package consists of the following:

  • a Feature with a header file named 'feature.xml'
  • an assembly for deployment to the GAC named 'COB.Demo.WebPartDeployment.WriteToFileWebPart.dll'
  • an entry in the SafeControls section of the application's web.config file, specifying all types in the specified assembly should be treated as safe
  • a web part definition file named 'COB.Demo.WebPartDeployment.WriteToFileWebPart.webpart' - this will be deployed to the web part gallery on the site

Importantly, all these details in the manifest.xml file are only used by SharePoint when the generated Solution package is deployed. They are effectively the instructions which say "go and get this item from the .wsp file and put it here". The actual process for generating the .wsp file is another task.


Building Solution packages

There are 2 options for building the actual package - build it manually using makecab.exe, or use an automated solution - there are several community-developed tools/techniques available. Since it's always good practise to understand what's actually happening in such processes, we'll cover how to do it manually here.

The first step is to write a .ddf (Diamond Directive File). This is a set of instructions for makecab.exe on how to build the folder hierarchy inside the .cab file. For my web part example, my file looks like:

.OPTION Explicit

.Set CabinetNameTemplate="COB.Demo.WebPartDeployment.WriteToFileWebPart.wsp" 

.Set DiskDirectory1="Package"

;***

 

manifest.xml

 

;** this directory name is used for the folder name under 12\TEMPLATE\Features, so should

;** match up with what you want to call the feature!

.Set DestinationDir=COB.Demo.WebPartDeployment.WriteToFileWebPart

elements.xml

feature.xml

 

.Set DestinationDir=COB.Demo.WebPartDeployment.WriteToFileWebPart

WebPart\COB.Demo.WebPartDeployment.WriteToFileWebPart.webpart

WebPart\COB.Demo.WebPartDeployment.WriteToFileWebPart.dll

 

;***


This file tells makecab.exe to do the following:

  • create a .cab file named 'COB.Demo.WebPartDeployment.WriteToFileWebPart.wsp'
  • put the 'manfest.xml' file at the root of the hierarchy
  • put the 'elements.xml' and 'feature.xml' files in a subfolder called 'COB.Demo.WebPartDeployment.WriteToFileWebPart'
  • also put the 2 other files (with extensions .webpart and .dll) in this same subfolder, but that makecab.exe should look for these files in a subfolder on the main filesystem called 'WebPart'

This instructions file is then passed to makecab.exe with the following command-line command:

D:\SolutionDeployment\Development\COB.Demo.WebPartDeployment>"C:\Program Files\Microsoft Cabinet SDK\BIN\MAKECAB.EXE" /f COB.Demo.WebPartDeployment.ddf


Couple of things to note here. At the command prompt we have ensured the current directory is the directory where all our Solution files are kept. This ensures that our relative references in the .ddf file above can be resolved. We pass the /f parameter to indicate we are passing a directives file, and also pass the location of this file.

Assuming all the files/references are OK, this results in a .cab file with a .wsp extension which will be created in a subfolder under the current directory called 'Package' (the folder will be created for you if it doesn't exist). We now have a Solution package which can be deployed to another SharePoint server. But first let's take a peek inside - this can be done by temporarily renaming the extension to .cab. You should then see something like:



Deploying the Solution to other SharePoint servers

The final part is to actually deploy the Solution. First the .wsp file must be copied to the target server, and then we will use STSADM (SharePoint's command-line admin tool) to actually deploy the Solution. The following are the commands which used to perform the deployment:

stsadm -o addsolution -filename COB.Demo.WebPartDeployment.WriteToFileWebPart.wsp

stsadm -o deploysolution -name COB.Demo.WebPartDeployment.WriteToFileWebPart.wsp
   -url http://myWebApplication -immediate -allowGacDeployment -allowCasPolicies -force


The first command simply adds the Solution package to SharePoint's Solution store in the config database, and the second one actually performs the deployment of the package to the specified web application. These areas are fairly well covered in the SDK at http://msdn2.microsoft.com/en-us/library/aa544500.aspx.

Rather than write these commands at the command-line each time you wish to deploy the package, chances are over time you'll package these commands up into STSADM scripts which do the work. As an example, the script I use for the webparts example looks like:

:begin

@echo off

 

set solutionName=COB.Demo.WebPartDeployment.WriteToFileWebPart

set url=http://myWebApplication

set featureName=COB.Demo.WebPartDeployment.WriteToFileWebPart

@set PATH=C:\Program Files\Common Files\Microsoft Shared\web server extensions\12\BIN;%PATH%

 

echo --- Attempting to deactivate/retract existing solution...

 

stsadm -o deactivatefeature -name %featureName% -url %url% -force

stsadm -o retractsolution -name %solutionName%.wsp -url %url% -immediate

stsadm -o execadmsvcjobs

stsadm -o deletesolution -name %solutionName%.wsp -override

rem stsadm -o execadmsvcjobs

 

echo --- Adding solution %solutionName% to solution store...

 

stsadm -o addsolution -filename %solutionName%.wsp

 

if errorlevel == 0 goto :deploySolution

 

echo ### Error adding solution %solutionName%

echo .

goto end

 

:deploySolution

echo --- Deploying solution %solutionName%...

 

stsadm -o deploysolution -name %solutionName%.wsp -url %url% -immediate -allowGacDeployment -allowCasPolicies -force

stsadm -o execadmsvcjobs

 

if errorlevel == 0 goto :activateFeature

 

echo ### Error deploying solution %solutionName%

echo .

goto end

 

:activateFeature

 

echo --- Activating features in solution %solutionName%...

 

stsadm -o activatefeature -name %featureName% -url %url% -force

 

if errorlevel == 0 goto :success

 

echo ### Error activating feature %featureName%

echo .

goto end

 

:success

echo Successfully deployed solution and activated feature(s)..

echo .

goto end

 

:end

pause


Following the script through, this basically does the work of safely upgrading the Solution (in case it has been deployed previously), and also activating the Feature within the Solution.


Automating Solution builds

Earlier, we mentioned the idea of automating the process of building and deploying Solutions. There are several community tools/ideas in this area, a couple of the ones I'm aware of are:

Both of these effectively use Visual Studio post-build events to automate the build of the .wsp package.

Additionally, Microsoft's Visual Studio Extensions for Windows SharePoint Services effectively performs the same tasks. Although there is some inflexibility in the current version, this add-on for VS means that the whole process is automated by hitting the F5 button. I'd recommend taking a look if you're not aware of this. My post on creating lists with VSeWSS gives an overview of using it for that specific scenario.

Wednesday, 18 July 2007

Using CustomAction to modify system pages

In the last post, I covered 2 examples of how to safely modify functionality in 'system' areas of SharePoint, such as Central Admin and the Site Settings area. The first example was a tweaked Recycle Bin (which listed only items deleted by the current user), and the second was a custom 'Create Site Collection' admin page. In both cases, the technique was to copy the original page which shipped with SharePoint, make the customization to the copy, and then redirect the link in SharePoint to use our new page. The customization is then fairly transparent to users - depending on your implementation, they could simply be clicking the same link in the same place in SharePoint, yet going to somewhere different which is offering your modified functionality.

This last part (of redirecting the link in SharePoint) is accomplished via a Feature which makes use of the CustomAction element. However, this idea of changing an existing link somewhere in SharePoint is not necessarily the type of customization you might regularly make. So it's useful to know the CustomAction element can also be used to add a link somewhere in SharePoint. This opens up many opportunities for customization, and this post aims to detail some of the options.

The CustomAction element is used like this:

<CustomAction Id="MyDeletedItems"

              GroupId="SiteCollectionAdmin"

              Location="Microsoft.SharePoint.SiteSettings"

              Sequence="10"

              Title="My recycle bin items"

              Rights="ManageWeb,BrowseUserInfo">

  <UrlAction Url="_layouts/custom/MyRecycleBinItems.aspx" />

</CustomAction>



I covered most of the values in details in the previous post, but what I want to focus on here is the Location attribute - this essentially specifies where in SharePoint your link will be added to. Some of the common potential values are:

  • Microsoft.SharePoint.SiteSettings (as in the example XML above)
  • Microsoft.SharePoint.Administration.Operations
  • Microsoft.SharePoint.Administration.ApplicationManagement

If you've worked with SharePoint for a while you can probably work out where these values refer to - the first is the Site Settings area, whilst the last two are the 'Operations' and 'Application Management' pages in the Central Admin area respectively. When using a CustomAction to add links in these areas, you can also specify the group your link should appear in. This value refers to an existing 'category' of links within the Location, e.g. the locations listed above. Note that new groups can be created by using a CustomActionGroup element, but the GroupID value of a CustomAction must always specify a group which exists already. The following groups exist by default in the SiteSettings area:

  • UsersAndPermissions
  • Customization
  • QuickLaunch
  • Galleries
  • SiteAdministration
  • SiteCollectionAdmin

These correspond to the columns in the Site Settings area as shown below:




In the Central Admin pages, examples for the 'Operations' page are:

  • Topology
  • GlobalConfiguration
  • Security
  • BackupRestore
  • LoggingAndReporting
  • DataConfiguration
  • Upgrade

And for the 'Application Management' page:

  • WebApplicationConfiguration
  • SiteManagement
  • ApplicationSecurity
  • ExternalService
  • WorkflowManagement

By examining these pages, it's fairly simple to work out where the values refer to.

I mentioned earlier that new groups of links can be created using the CustomActionGroup element. An example of this would be:

<CustomActionGroup

    Id="CustomSiteSettingsGroup"

    Location="Microsoft.SharePoint.SiteSettings"

    Title="New custom group"

    Sequence="100"

    Description="My description" />



This will then create a new group of links on the page with your custom title - SharePoint will take care of the rendering such that your new group will wrap onto another row of groups. Of course if this isn't what you want you'll probably have an interesting time customizing this.

Using CustomAction elsewhere

So having covered how to add links into the admin areas, I'll wrap up by listing some other valid values for the 'Location' attribute. This hopefully illustrates some of the options you have for customizing SharePoint in this way:
  • Microsoft.SharePoint.ContentTypeTemplateSettings
  • Microsoft.SharePoint.ContentTypeSettings
  • Microsoft.SharePoint.Administration.ApplicationCreated
  • Office.Server.ServiceProvider.Administration (Shared Services/SSP links)
  • Microsoft.SharePoint.ListEdit.DocumentLibrary
  • Microsoft.SharePoint.Workflows
  • NewFormToolbar
  • DisplayFormToolbar
  • EditFormToolbar
  • Microsoft.SharePoint.StandardMenu (SiteActions menu)
  • Mcrosoft.SharePoint.Create (_layouts/create.aspx - the screen used to specify what you want to create on your site)
  • Microsoft.SharePoint.ListEdit (the screen used to edit the properties of a list item)
  • EditControlBlock (image below)
I've put the last few in bold and added a description as I think they're particularly interesting. Of course, if you have other customizations in mind one or two of the others might have caught your eye! In particular being able to add custom actions to the EditControlBlock is interesting:




Here, a special attribute called 'RegistrationType' can be added to the CustomAction element, allowing you to control exactly the circumstances where your action should appear. Valid options include 'List', 'ContentType', 'FileType' and 'ProgId'. Vince (aka TheKid) has some good examples of this over on his blog.

Hopefully this has given you some ideas on how you can add or amend the SharePoint UI in this way. Happy customizing!

Thursday, 12 July 2007

Modifying 'system' pages in SharePoint safely - with sample code

In a presentation I gave recently I talked about modifying admin screens within SharePoint. By this, I mean areas such as the site admin area and the central admin area:

Site admin:


Central admin:



Now, if your boss (or your users) asks you to add or change functionality in these areas, how would you handle that?

Well, you'd probably start off by looking to see how SharePoint implements 'system' pages like this. You'd then find they are just standard .aspx files in the 12\TEMPLATE\ADMIN folder on the filesystem. You'd also find that if you make a change to such a file, the changes are immediately reflected on the site - sure, some of the code is in a compiled assembly which is obfuscated (so Reflector doesn't help), but otherwise there's no mysterious abstractions, no caching, it's that simple. So changing the functionality or adding a link is a matter of changing the .aspx page right?

The problem with this approach is that modifying core files in this way is unsupported. This is because when a service pack or hotfix gets applied, your system will be in an inconsistent state and at best your changes will be overwritten. So a better way to implement such changes is to:

  • take a copy of the original file
  • modify the copy to include your customization
  • create a SharePoint Feature (wrapped in a Solution package [.wsp]) to deploy your customized page to the 12 hive, and also add/modify the link in SharePoint to point to your customized page

The end result of this is that you have deployed your change without modifying any of the original files, and are therefore entirely supported. Additionally, because Features can be activated and deactivated with the click of a button, if you want to revert to the original functionality that's all it takes.

Let's look at an example - I'll link to all the code used here at the end of the article. Imagine your users are asking you to change the Recycle Bin screen so that it only displays items the currently logged on user has deleted - perhaps there's a manual document clean up exercise happening. And let's also say that after the exercise is complete, the users would like to revert back to the original functionality. (Now remember, what I'm demonstrating here is the mechanism for making customizations like this, so whilst this particular change might be unlikely in your organization, you might have other customizations which would help your users.)

So, first we need to modify the functionality of the Recycle Bin page to add the filtering. As mentioned earlier, we don't want to amend the original page, so we take a copy and modify that. In my example, I'm adding a script block to the code in front (since I don't have access to Microsoft's source code for the code-behind) to effectively perform the filtering at the UI level, which is fine for this example. Once we've created the page we want to use, we're ready to create the Feature and Solution to add our page to SharePoint.

Let's start with the manifest.xml file for the solution - this is important because this is how we actually deploy our custom .aspx page to the appropriate place within the SharePoint files. (Remember all this is available for download, the link is at the end of the article.)

<Solution xmlns="http://schemas.microsoft.com/sharepoint/" SolutionId="AA7EF934-C993-4f91-8AF4-0FB1D45927FA">

  <FeatureManifests>

    <FeatureManifest Location="COB.Demos.MyRecycleBinItems\feature.xml" />

  </FeatureManifests>

  <TemplateFiles>

    <TemplateFile Location="LAYOUTS\Custom\MyRecycleBinItems.aspx" />

  </TemplateFiles>

</Solution>


Notice what we are doing here is specifying that our custom .aspx page should be deployed to a 'Custom' subfolder within the 'LAYOUTS' directory. This is a good idea because it keeps our files separate from the core SharePoint files.

Then we have a fairly standard feature.xml file specifying the Feature identification info:

<Feature xmlns="http://schemas.microsoft.com/sharepoint/" Id="29D8B365-6024-4b1a-84D2-FF789FE56355"

        Title="COB.Demos.MyRecycleBinItems"

        Description="Adds a custom page to the Site Settings area to show items in the Recycle Bin which were deleted by the current user."

        Scope="Site" Hidden="FALSE"

        Version="1.0.0.0">

  <ElementManifests>

    <ElementManifest Location="elements.xml"/>

  </ElementManifests>

</Feature>


In this case we're specifying that our Feature is scoped at site collection-level, so we will modify all sites in the site collection.

Finally we have the our Elements file, which specifies what this Feature consists of. In this case, the key is the use of the CustomAction element - this can be used to add a link (i.e. an action) somewhere in SharePoint.

<Elements xmlns="http://schemas.microsoft.com/sharepoint/">

  <HideCustomAction Id="HideDeletedItems"

                    HideActionId="DeletedItems"

                    GroupId="SiteCollectionAdmin"

                    Location="Microsoft.SharePoint.SiteSettings" />

  <CustomAction Id="MyDeletedItems"

                GroupId="SiteCollectionAdmin"

                Location="Microsoft.SharePoint.SiteSettings"

                Sequence="10"

                Title="My recycle bin items"

                Rights="ManageWeb,BrowseUserInfo">

    <UrlAction Url="_layouts/custom/MyRecycleBinItems.aspx" />

  </CustomAction>

</Elements>


Let's walk through this file in detail. The first thing you notice is that we're using a 'HideCustomAction' element. You guessed it, this is used to hide a link in the SharePoint UI. Since we are effectively replacing a link, we actually need to hide the original one and supply our replacement, and this is done by using these 2 elements together. We assign the ID ourselves (if we want to refer to this 'HideCustomAction' elsewhere, this is the value we'll use), but all the other attributes of this tag must correspond to those specified for this action in the Feature Microsoft created to add the link to SharePoint in the first place. So in case you've not come across this yet, much of the original functionality in SharePoint is actually implemented using Features too. The way to find these values is to search in the 12\Template\Features folder. For this particular action, the most reliable thing to search for is the URL of the page, since any 'CustomAction' tag specifies the URL. Since we can find that the URL for the original Recycle Bin page is '/_layouts/AdminRecycleBin.aspx', searching for this finds the file containing the details of the original link. These are then used for our 'HideCustomAction' tag.

Next we specify our 'CustomAction' tag, again using many of the same values since we are replacing an existing action rather than adding a new one. The values we change are for the link title and URL, to point to our replacement page. Note also you can declaratively specify what permissions levels are required to see this link.

Once these files are made into a Solution package, deploying the Solution and activating the Feature has the following effects:

The link on the site settings page has changed from the original (on the left) to our replacement (on the right):


 A 'Custom' folder has been created in the '12\Layouts' directory and our custom .aspx page (MyRecycleBinItems.aspx) has been deployed to this folder. And of course, when a user clicks the link, the modified page has the functionality to only display items they have deleted:



The key thing of course, is that we've 'customized SharePoint functionality' without modifying any original SharePoint files - hence we are are 100% supported and can be confident our customizations won't cause any issues for service packs etc.

On the same lines, the sample stuff has a 2nd example - here we modify the 'Create Site Collection' page in Central Admin to display a guidance message to the SharePoint administrators. The idea is to encourage them to stop and think and perhaps consult the document containing the organization's policy on creating site collections:


Hopefully you might find this useful if you need to customize SharePoint in this way. The full set of files, plus the slides for the presentation I gave on this, are available at:

http://sharepointchris.googlepages.com/customizingsharepointsupportedway

In the next post, I'll be supplying some reference information which will help you make this type of modification in other areas of SharePoint. An important aspect of customizing SharePoint in this way is knowing exactly where and how the 'CustomAction' tag can be used, so that's what we'll look at.

Sunday, 8 July 2007

'Customizing SharePoint the supported way' - slides and sample code now available

The slides and code from my presentation at last week's UK SharePoint user group meeting are now available - the topic was 'Customizing SharePoint the supported way - from end user to admin interfaces'. The slides etc. will appear on the user group website at some point but I've also published them at:

http://sharepointchris.googlepages.com/customizingsharepointsupportedway

The areas covered are:

  • creating site definitions (and why it can be a good idea even for a WCM scenario with limited numbers of sites/webs being created)
  • how to modify the site admin and central admin areas of SharePoint, without modifying any of the shipped files. The examples I used (the sample code zip file contains all the code/Features for these) were:

    - modifying the recycle bin to show only files deleted by the current user

    - modifying the 'Create Site Collection' page in central admin to display a policy message to the farm administrators

  • using the DelegateControl to override the user controls used such as the 'My sites' link and the publishing console. (This assumes either default.master is being used or you have added the controls to your master page/page layout using <SharePoint:DelegateControl runat="server" ControlId="MyControlID" />). More details in my DelegateControl post.

Since the slide deck alone doesn't really convey the entire process, the next post will cover modifying the admin areas of SharePoint in detail. Included will be the list of areas where you can add/change links, and how to use a <CustomAction /> instruction within a Feature to do this.

If you need to make customizations to SharePoint functionality, the idea behind this technique is to make the changes without modifying the core files and thus putting your SharePoint installation in an unsupported state. This is very important if you need to make these kind of changes!

Wednesday, 4 July 2007

Considerations when using Features to deploy SharePoint files - ghosting/unghosting

In some of my earlier articles I talk about how to deploy various SharePoint artifacts as a Feature. In particular, 'Deploying master pages and page layouts as a Feature' discusses the idea of deploying these types of file (used in Publishing sites), but the concept applies to any file which will appear somewhere in a SharePoint list of some kind. In addition to the Master Page Gallery, other examples could include CSS/XSLT files being deployed to the Style Library, images to a picture library, and many other similar scenarios.

Occasionally people using this approach find that file updates can be difficult to apply. The process involves updating the Feature with the new version of the file, and then reinstalling and reactivating the Feature (typically the Feature version number will be incremented). In some cases the file updates successfully, in others it doesn't but there are no errors.

So what's going on?

The answer is that the file will not be updated (and therefore changes will not be seen in the browser for example) if that particular file has been 'unghosted' into the database.

<StartUnghostingExplanation>
[Without digressing too much, for anybody new to this concept it can be a confusing term which Microsoft are using less these days - the replacement term is 'customized' which is generally easier to understand. Depending on how they arrived in SharePoint, many files will initially only exist on the filesystem of the SharePoint front-end web servers. However, if such a file is customized e.g. it is checked out and edited (either through the UI, using SharePoint Designer etc.) SharePoint takes a copy of the file (now with modifications) and stores this in the content database instead of saving the changes back to the filesystem. This is the unghosting process. Whenever this file is requested in the future, the modified version from the database will be returned. This architecture allows SharePoint to scale to enterprise level, by only storing separate copies of files when absolutely necessary.]
<EndUnghostingExplanation>

When a file is provisioned in SharePoint using a Feature, SharePoint will copy the file to wherever your Feature specified, and will reference the file from this location. In many cases, this may actually be the 12\Template\Features directory. So in the case of master pages/page layouts etc. for example, when a web page is requested these resources will be retrieved from this location. Or at least, that's what happens if the file isn't customized.

If the file has been customized at some point, what happens when the Feature is updated is that the file on the filesystem is updated and reflects the changes, but SharePoint is now returning the copy in the database and so the update isn't reflected on the site!

So, if you want to use Features as your ongoing deployment strategy for your SharePoint document library (aka 'GhostableInLibrary') files, you should ensure the files can only be modified in this way. So files should not be modified either through the SharePoint UI or via SharePoint Designer. Typically, this won't be the case in the development environment but could be enforced for other environments. This means that unghosting is avoided, and SharePoint will continue to reference the copy on the filesystem.

Some other points of note:

  • Another advantage of this approach is that when a Solution is retracted, the files will also be removed.
  • Even a check-out/check-in operation without actual changes will cause SharePoint to see the file as customized, and will then be retrieved from the database rather than filesystem.
  • A file can be 'uncustomized' in SharePoint Designer by right-clicking on the file and selecting 'Revert to site definition'. However, this will obviously cause you to lose any changes you have made to the file from the original version!

So next time you are using a Feature and wondering why the file isn't being updated, consider if the file has been customized!