Wednesday 4 July 2007

Considerations when using Features to deploy SharePoint files - ghosting/unghosting

In some of my earlier articles I talk about how to deploy various SharePoint artifacts as a Feature. In particular, 'Deploying master pages and page layouts as a Feature' discusses the idea of deploying these types of file (used in Publishing sites), but the concept applies to any file which will appear somewhere in a SharePoint list of some kind. In addition to the Master Page Gallery, other examples could include CSS/XSLT files being deployed to the Style Library, images to a picture library, and many other similar scenarios.

Occasionally people using this approach find that file updates can be difficult to apply. The process involves updating the Feature with the new version of the file, and then reinstalling and reactivating the Feature (typically the Feature version number will be incremented). In some cases the file updates successfully, in others it doesn't but there are no errors.

So what's going on?

The answer is that the file will not be updated (and therefore changes will not be seen in the browser for example) if that particular file has been 'unghosted' into the database.

<StartUnghostingExplanation>
[Without digressing too much, for anybody new to this concept it can be a confusing term which Microsoft are using less these days - the replacement term is 'customized' which is generally easier to understand. Depending on how they arrived in SharePoint, many files will initially only exist on the filesystem of the SharePoint front-end web servers. However, if such a file is customized e.g. it is checked out and edited (either through the UI, using SharePoint Designer etc.) SharePoint takes a copy of the file (now with modifications) and stores this in the content database instead of saving the changes back to the filesystem. This is the unghosting process. Whenever this file is requested in the future, the modified version from the database will be returned. This architecture allows SharePoint to scale to enterprise level, by only storing separate copies of files when absolutely necessary.]
<EndUnghostingExplanation>

When a file is provisioned in SharePoint using a Feature, SharePoint will copy the file to wherever your Feature specified, and will reference the file from this location. In many cases, this may actually be the 12\Template\Features directory. So in the case of master pages/page layouts etc. for example, when a web page is requested these resources will be retrieved from this location. Or at least, that's what happens if the file isn't customized.

If the file has been customized at some point, what happens when the Feature is updated is that the file on the filesystem is updated and reflects the changes, but SharePoint is now returning the copy in the database and so the update isn't reflected on the site!

So, if you want to use Features as your ongoing deployment strategy for your SharePoint document library (aka 'GhostableInLibrary') files, you should ensure the files can only be modified in this way. So files should not be modified either through the SharePoint UI or via SharePoint Designer. Typically, this won't be the case in the development environment but could be enforced for other environments. This means that unghosting is avoided, and SharePoint will continue to reference the copy on the filesystem.

Some other points of note:

  • Another advantage of this approach is that when a Solution is retracted, the files will also be removed.
  • Even a check-out/check-in operation without actual changes will cause SharePoint to see the file as customized, and will then be retrieved from the database rather than filesystem.
  • A file can be 'uncustomized' in SharePoint Designer by right-clicking on the file and selecting 'Revert to site definition'. However, this will obviously cause you to lose any changes you have made to the file from the original version!

So next time you are using a Feature and wondering why the file isn't being updated, consider if the file has been customized!

20 comments:

Anonymous said...

Thanks for the post Chris. Makes a lot of sense. Gotta love customized pages!

Liza said...

We have migrated content from MCMS to MOSS which seems to have the effetct that page layouts are copied to the database. There is no 'Revert to site definition' functionality for page layouts - is there any other way 'uncostumize' page layouts?

Chris O'Brien said...

Liza,

I've not looked specifically at the CMS 2002 migration scenario (though expect to soon for a client).

I'd _expect_ the page layouts to be in the database though. The only way they wouldn't be is if the page layouts were deployed with a Feature, but this wouldn't be the case for a CMS migration.

You'll only ever be able to uncustomize a file if there's a corresponding version on the filesystem somewhere.

HTH,

Chris.

Liza said...

But there IS a corresponding file version in the 12 hive - we have both migrated AND deployed page layouts through features. The problem is to have Sharepoint look at the file version instead of the database version - "uncostumizing" the page layouts?

Chris O'Brien said...

Ah OK - in that case it should be possible, assuming the database record for the file has a link to the correct physical file.

I would try using SPD to 'revert' the file (right-click on file in left pane, there should be an option), or alternatively SPFile.RevertContentStream() should do it.

HTH,

Chris.

Anonymous said...

Chris -

I asked this in your feature article, as well:

How do you tell if a particular file has been unghosted? My intranet site, when opened in SharePoint designer does not show the customized icon for any of my custom pages, yet my deployment of the feature is behaving as if they were: the files in the feature folder are updated, but the files in the master page gallery on the site are not.

This is a content publishing site and there are a fair number of pages now using the layouts I'm trying to update, so I don't want to use my previous brute force method of deleting all pages using the layouts, then deleting the layouts, etc..

Any thoughts on this would be appreciated.

Chris O'Brien said...

Colin,

Hopefully you've seen this by now, but I've commented on this at master pages/page layouts deployed as Feature not updating.

HTH,

Chris.

Tyler Holmes said...

Chris,

When it comes time to deploying configuration files that I'd like to sit in the same directory as the web.config am I out of luck? All of the feature syntax for deploying unghosted files seem to want to put it in the \TEMPLATE\FEATURES\FeatureName directory. In addition they potentially provide URLs to the content (something I don't want to do). I've seen people set up timer jobs to copy around resource files but isn't there something simpler?

The file i'm copying is the entlib.config from the enterprise library.

I'd love to hear your opinion.

Best,
Tyler

Chris O'Brien said...

Hi Tyler,

Yes, unfortunately the Feature/Solution framework doesn't really cater for files in the web application directory. We only have the 'Module' element in a Feature (deploys to Feature folder as you say), or the 'TemplateFile' deploys to \12\Template directory) and 'RootFile' (deploys to \12\ directory) in a Solution.

One option could be to write code in a Feature receiver for this file, or otherwise use some other scripted approach where you can access this location. Annoying I know.

HTH,

Chris.

death roll said...

Hi Chris,

I have a doubt on SharePoint Designer and page customization.

I have created customized pages for a list’s NewForm.aspx, EditForm.aspx and DispForm.aspx. These are created using SharePoint Designer. Now my requirement is to pack my site as Feature along with these customized pages. Since the Data Form Webpart included in these pages have hard coded GUIDs of associated lists, these SPD generated pages are not functioning in the target machine where I install my feature since these IDs are dynamically assigned while provisioning site. Please let me know how I can make these SPD generated files working when deployed as a featue.

I’ve successfully packaged the following SharePoint artifacts into features; the only problem is with the SPD customized pages.

Content Types
List Definitions
List Instances
Feature Receivers

Please help

Thank you,
-Arun

Chris O'Brien said...

Hi Arun,

That's a good question. Personally I think the solution is to ensure your lists have the same GUIDs in all environments. This then removes the need to somehow 'fix-up' list GUIDs in DataForm/ListView web parts etc. If you used content deployment (either from the standard functionality or my SharePoint Content Deployment Wizard) to transfer the lists in the first place, they will have the same GUIDs (need to check 'retain object IDs' checkbox in my tool).

Personally this is my preferred approach for transferring files which are content items, rather than filesystem items. Features are great, but IMHO are suited more towards one-time provisioning rather than scenarios such as WCM where updatability is required.

HTH,

Chris.

Jason Apergis said...

Chris,

I am pushing out some publishing pages via a Feature. The problem I am running into is that properties in the master gallery are not being updated. So I change the aspx and push out the Feature I see all the changes.

However in the properties I try to change something like the Title or the Version and that change is NOT reflected in the master gallery list.

Any ideas why? I would like to be able to support versioning when I push out a new updated publishing page .aspx.

Thanks,
Jason

Chris O'Brien said...

@Jason,

Am guessing you're updating the Module/File element in your Feature - I wouldn't recommend trying to version things in this way, the framework isn't designed to support it. The only way I think you could get metadata to update in this way is if you deleted the files before reactivating the Feature, but obviously this isn't possible in real-life usage.

My suggestion would be to version your aspx files in your main source control system instead.

HTH,

Chris.

Nick Hadlee said...

Hi Chris,
First up cheers for the content deployment tool - always using it and it has been a real asset to the toolkit!

I'm hoping you might be able to shed light on a 'quirk' of files deployed via features.

When deploying pages via a feature I have found that updates to the page layouts referenced by that page do not cascade to these feature pages. Pages create via the UI or other API will reflect the changes but the feature deployed pages are locked to the original physical file that was used to deploy them. Updating this file also does not make any difference...

e.g.
<File Url="default.aspx" Name="summary.aspx" Type="GhostableInLibrary" IgnoreIfAlreadyExists="True">
<Property Name="PublishingPageLayout" Value="~SiteCollection/_catalogs/masterpage/somelayout.aspx, Layout Description" />
... rest of properties ...
</File>

However detaching and reattaching the page to the layout seems to 'fix' the page layout connection. This isn't much of a workaround if there are a lot of pages deployed like this so I was hoping you may be able to suggest the reason behind this behavior or a better method of deployment?

Chris O'Brien said...

Hi Nick,

Interested in what you're reporting here - so do you mean that your using a Feature to deploy the page layout but also a page instance, and that updates to the layout aren't reflected in the page instance until you do your workaround?

Not sure I have any explanation for this I'm afraid, but I am interested! In terms of your workaround, how are you "detaching and reattaching" the page to the layout? Do you mean by doing a "Revert to site definition"?

Cheers,

Chris.

Nick Hadlee said...

Hi Chris,

Yes to the first paragraph questions. Two features are being used, a site scoped one for the page layouts then a web scoped one for the page instance.

I actually did the test 'fix' using SharePoint designer to dettach/reattach the page from/to the layout. It also works if you RevertContentStream() via a reghost command like Gary's. Comparing the SPFile properties to see what changes wasn't very fruitful...

Chris O'Brien said...

Nick,

Thanks for following up - this is interesting. I can't think of any reasons why this would happen, but clearly it is.

My suggestion would be to ask Waldek if he ran into it when he was doing this - see http://blog.mastykarz.nl/provisioning-publishing-pages-features-declarative-markup/.

I'll try and follow the investigation there!

Cheers,

Chris.

Anonymous said...

Hi,

I think you mixed ghosted and unghosted. Customizing a file means to ghost a file. A file that is not changed after being deployed is unghosted.

The other option is that you write a littel command line tool to unghost all ghosted files before deploying your WSP package. That is what we are doing...

Kr, Peter.

Nick Hadlee said...

Cheers Chris. I hadn't read that post but that's the exact situation that seems to result in this behavior. I have posted a question to Waldek so we will see if he has come across this issue.

Neamat said...

I've just sent a question from a few minutes.. I just wanted to add and say that I'm using anonymous access too to the site in case this might make a difference in your response.

I'd really appreciate if you reply.. Thank you in advance