Wednesday, 25 August 2010

Feature upgrade (part 5) – using PowerShell to upgrade Features

In this article series:

  1. Feature upgrade (part 1) – fundamentals
  2. Feature upgrade (part 2) – a sample to play with
  3. Feature upgrade (part 3) – introducing SPFeatureUpgrade kit
  4. Feature upgrade (part 4) – advanced scenarios
  5. Feature upgrade (part 5) – using PowerShell to upgrade Features (this article)

OK, so I lied in article 4 about this series being finished – I’ve now added PowerShell support to my SPFeatureUpgrade kit, a set of tools designed to help manage the process of upgrading Features in SharePoint 2010. In this article I’ll talk about the PowerShell cmdlet I’m providing, but also some interesting things I learnt about PowerShell along the way – turns out writing a custom cmdlet is a great way to learn some aspects of PowerShell! These might be of interest if, like me, you haven’t done tons of PS to date.

Introduction

There are several reasons why you might prefer to use PowerShell to upgrade Features, rather than the administration pages also in the kit:

  • You have a large farm, and upgrading hundreds of site/web-scoped Features could time out in a web process
  • You’re scripting lots of other things
  • You want to use some advanced logic to upgrade some Feature instances and not others
  • You don’t want to write a C# console app (or similar) to do this, because that would be well, “a bit 2003” :)

As I mentioned in the earlier posts, SharePoint 2010 does not provide any handy cmdlets (or UI) out-of-the-box for upgrading Features, so this is the gap I’m hoping to fill somewhat.

How to upgrade using PowerShell

Install the supplied WSP to get the admin pages (see earlier articles) and the PowerShell cmdlet. Once installed, the command is:

Upgrade-SPFeatures -Scope <SPFeatureScope> [-WebApplication [<SPWebApplicationPipeBind>]] [-Site [<SPSitePipeBind>]] [<CommonParameters>]

Let’s consider some scenarios – the following table shows the command to use to depending on the scope of Features you are upgrading:

Scope

Command

Notes

Farm Upgrade-SPFeatures –Scope Farm All farm-scoped Features requiring upgrade are upgraded.
Web application Upgrade-SPFeatures –Scope WebApplication All web application-scoped Features requiring upgrade are upgraded.
Site

Upgrade-SPFeatures –Scope Site
-WebApplication “http://cob.collab.dev”
---------------------------------- OR -----------------------------------Get-SPWebapplication “http://cob.collab.dev” | Upgrade-SPFeatures –Scope Site

All site-scoped Features requiring upgrade within the specified web application are upgraded.
Web Upgrade-SPFeatures –Scope Web –Site “http://cob.collab.dev/sites/site1”
---------------------------------- OR -----------------------------------Get-SPSite “http://cob.collab.dev/sites/site1” | Upgrade-SPFeatures –Scope Web
All web-scoped Features requiring upgrade within the specified site collection are upgraded.

When the command is executed, if any Feature instances fail to upgrade you will see in the result:

PS_MixedSuccessAndFailure

In terms of the breakdown of commands/parameters, effectively this is the same model my admin pages use – farm and web app Features are upgraded without specifying a filter, but for site or web Features you must specify the parent object. As you can see from the alternate usages for these scopes, the –WebApplication and –Site and parameters can be specified inline or piped-in. Being able to pipe in is useful as it allows you to filter the parent object in order to, say, only upgrade the Features in certain web applications like so:

Get-SPWebApplication | Where-Object {$_.Url -like "http://cob*"} | Upgrade-SPFeatures -Scope Site

Thus this gives you more control over the parent object and therefore what gets upgraded. But what happens if what you’re doing just doesn’t match how my commands are broken down?

Using other logic to selectively upgrade Features

Examples of this might be “upgrade  all web-scoped Features where the web is in this list [‘foo’, ‘bar’]”, or perhaps “upgrade all site-scoped Features where the site has other Feature ‘foo’ activated” and so on. For now, my cmdlet won’t really help you too much in this case so you will need to write some PowerShell which goes straight to the API. For upgrading web-scoped Features, this might look something like:

Function shouldUpgrade([Microsoft.SharePoint.SPFeature]$aFeature)
{
    # add whatever testing logic here..
    if ($aFeature.Definition.Name -eq 'SomeName')
    {
        return $true
    }
    return $false
}
 
$site = Get-SPSite "http://cob.collab.dev/sites/featuretesting" 
$featuresRequiringUpgrade = $site.QueryFeatures([Microsoft.SharePoint.SPFeatureScope]::Web, $true)
 
foreach ($feature in $featuresRequiringUpgrade)
{
    if (shouldUpgrade($feature) -eq $true)
    {
        try
        {
            $feature.Upgrade($false)
            Write-Host ("Upgrade of Feature '" + $feature.DefinitionId + "' with parent '" + $site.Url + "' succeeded")
        }
        catch 
        {
            Write-Host ("Upgrade of Feature '" + $feature.DefinitionId + "' with parent '" + $site.Url + "' failed")
        }
    }
    else
    {
        Write-Host ("Not upgrading this Feature")
    }
}

Obviously this example would need amending for other scopes, proper error reporting etc. Clearly, unless you’re very comfortable cutting PowerShell, having a cmdlet which contains most of the logic does make things easier. So at some point I’ll extend my cmdlet to somehow allow you to filter on which Feature instances to upgrade rather than upgrading all of them - perhaps by specifying a list in an XML file or similar. I could easily get PowerShell to ask you to confirm for each item in the loop, but obviously that doesn’t scale when large numbers of Feature instances (e.g. web-scoped Features) are involved and is no good for unattended use, so that doesn’t feel like the way to go.

Using –WhatIf to see what *would* be upgraded

My cmdlet fully supports the PowerShell –WhatIf switch – this can be very useful as it allows you to see what Feature instances would be upgraded without actually doing anything. In the image below I can see all of the webs in my site collection where my Feature would be upgraded:

 Upgrade-SPFeatures_WhatIf

Future enhancements

There are a couple of shortcomings which spring to mind in the current implementation. Actually an experienced PowerShell eye would probably find lots (feedback welcome, help me learn!), but I do know of these:

  • No ability to filter which Feature instances are upgraded (as discussed earlier)
  • Doesn’t return an appropriate object to the pipeline for downstream processing
    • Actually I only gained a real understanding of this after I’d uploaded to Codeplex. I’m happy to leave it for now, but will improve this in the next version.

The code

The cmdlet and the admin pages I blogged about previously can be downloaded from http://SPFeatureUpgrade.codeplex.com. Here’s what my current code for the cmdlet looks like:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Management.Automation;
using System.Text;
using Microsoft.SharePoint;
using Microsoft.SharePoint.Administration;
using Microsoft.SharePoint.PowerShell;
 
using COB.SharePoint.Utilities.FeatureUpgradeKit.Core;
using COB.SharePoint.Utilities.FeatureUpgradeKit.Core.Exceptions;
using COB.SharePoint.Utilities.FeatureUpgradeKit.Entities;
 
namespace COB.SharePoint.Utilities.FeatureUpgradeKit.PowerShell
{
    /// <summary>
    /// Provides a PowerShell cmdlet for upgrading Features in SharePoint.
    /// </summary>
    /// <created by="Chris O'Brien" date="20 August 2010" />
    [Cmdlet("Upgrade", "SPFeatures", SupportsShouldProcess=true)]
    [SPCmdlet(RequireLocalFarmExist=true, RequireUserFarmAdmin=true)]
    public class SPCmdletUpgradeFeatures : SPCmdlet
    {
        [Parameter(
           Mandatory = true,
           HelpMessage = "Scope of the Features to upgrade.")]
        public SPFeatureScope Scope
        {
            get;
            set;
        }
 
        [Parameter(
           ValueFromPipeline=true,     
           HelpMessage = "Value of the SPWebApplication whose Site-scoped Features should be upgraded.")]
        public SPWebApplicationPipeBind WebApplication
        {
            get;
            set;
        }
        
        [Parameter(
           ValueFromPipeline = true,
           HelpMessage = "Value of the SPSite whose Web-scoped Features should be upgraded.")]
        public SPSitePipeBind Site
        {
            get;
            set;
        }
 
        protected override void InternalValidate()
        {
            if (((Scope == SPFeatureScope.Farm) || (Scope == SPFeatureScope.WebApplication)) && (Site != null || WebApplication != null))
            {
                ThrowTerminatingError(new InvalidFeatureUpgradeDetailsException("When the -Scope parameter is Farm or WebApplication, no parent object (e.g. Farm, Web application, Site or Web) should " +
                    "be passed. Alternatively, specify Site or Web for the -Scope parameter and pass the appropriate parent object."), ErrorCategory.InvalidArgument, null);
            }
            if (Scope == SPFeatureScope.Site && WebApplication == null)
            {
                ThrowTerminatingError(new InvalidFeatureUpgradeDetailsException("To upgrade Site-scoped Features, the parent web application must be piped in or specified using the -WebApplication parameter."), ErrorCategory.InvalidArgument, null);
            }
            if (Scope == SPFeatureScope.Web && Site == null)
            {
                ThrowTerminatingError(new InvalidFeatureUpgradeDetailsException("To upgrade Web-scoped Features, the parent site collection must be piped in or specified using the -Site parameter."), ErrorCategory.InvalidArgument, null);
            }
            base.InternalValidate();
        }
 
        protected override void InternalProcessRecord()
        {   
            SPFeatureQueryResultCollection featuresForUpgrade = null;
            List<SPFeature> featuresForUpgradeList = null;
            SPWebApplication webApp = null;
            SPSite site = null;
 
            switch(Scope)
            {
                case SPFeatureScope.Farm:
                    featuresForUpgrade = SPWebService.AdministrationService.QueryFeatures(Scope, true);
                    featuresForUpgradeList = featuresForUpgrade.ToList<SPFeature>();
                    if (featuresForUpgradeList == null || featuresForUpgradeList.Count() == 0)
                    {
                        WriteResult(string.Format("No Features found requiring upgrade at scope '{0}'.", Scope));
                    }
                    break;
                case SPFeatureScope.WebApplication:
                    featuresForUpgrade = SPWebService.QueryFeaturesInAllWebServices(Scope, true);
                    featuresForUpgradeList = featuresForUpgrade.ToList<SPFeature>();
                    if (featuresForUpgradeList == null || featuresForUpgradeList.Count() == 0)
                    {
                        WriteResult(string.Format("No Features found requiring upgrade at scope '{0}'.", Scope));
                    }
                    break;
                case SPFeatureScope.Site:
                    webApp = WebApplication.Read();
                    
                    featuresForUpgrade = webApp.QueryFeatures(Scope, true);
                    featuresForUpgradeList = featuresForUpgrade.ToList<SPFeature>();
 
                    if (featuresForUpgradeList == null || featuresForUpgradeList.Count() == 0)
                    {
                        WriteResult(string.Format("No Features found requiring upgrade at scope '{0}' with parent web application '{1}'.", 
                            Scope, webApp.GetResponseUri(SPUrlZone.Default)));
                    }
                    break;
                case SPFeatureScope.Web:
                    site = Site.Read();
                    featuresForUpgrade = site.QueryFeatures(Scope, true);
                    featuresForUpgradeList = featuresForUpgrade.ToList<SPFeature>();
 
                    if (featuresForUpgradeList == null || featuresForUpgradeList.Count() == 0)
                    {
                        WriteResult(string.Format("No Features found requiring upgrade at scope '{0}' with parent site collection '{1}'.", 
                            Scope, site.Url));
                    }
                    break;
            }
 
            if ((featuresForUpgradeList != null) && (featuresForUpgradeList.Count() > 0))
            {
                List<FeatureDetails> featureDetailsList = getFeatureDetails(featuresForUpgradeList);
 
                if (ShouldProcess(string.Format("Scope = {0}", Scope)))
                {
                    Dictionary<FeatureDetails, IEnumerable<Exception>> upgradeResults = FeatureManager.UpgradeFeatures(featureDetailsList);
                    presentResultMessages(upgradeResults);
                }
                else
                {
                    string msg = string.Format("Would upgrade the following Features:{0}{0}", Environment.NewLine);
                    foreach(FeatureDetails details in featureDetailsList)
                    {
                        msg += string.Format("Feature Name='{0}', Parent='{1}', ID='{2}'.{3}",
                            details.FeatureName, details.ParentString, details.FeatureID, Environment.NewLine);
                    }
                  
                    WriteResult(msg);
                }
            }
            
            base.InternalProcessRecord();
        }
 
        private void presentResultMessages(Dictionary<FeatureDetails, IEnumerable<Exception>> upgradeResults)
        {
            int errorCount = 0;
            StringBuilder sbSuccessResults = new StringBuilder();
            StringBuilder sbFailedResults = new StringBuilder();
 
            Dictionary<FeatureDetails, IEnumerable<string>> resultOutput = new Dictionary<FeatureDetails, IEnumerable<string>>();
 
            foreach (KeyValuePair<FeatureDetails, IEnumerable<Exception>> kvp in upgradeResults)
            {
                List<Exception> exceptions = null;
                List<string> exceptionMessages = new List<string>();
 
                if (kvp.Value != null)
                {
                    exceptions = kvp.Value.ToList();
                }
 
                if (exceptions != null && exceptions.Count > 0)
                {
                    sbFailedResults.AppendFormat("{2}ERROR: Feature '{0}' with parent '{1}' failed to upgrade with the following errors:{2}{2}",
                        kvp.Key.FeatureName, kvp.Key.ParentString, Environment.NewLine);
                    foreach (Exception exception in exceptions)
                    {
                        sbFailedResults.AppendFormat("  - {0}{1}", exception.ToString(), Environment.NewLine);
                    }
                }
                else
                {
                    sbSuccessResults.AppendFormat("{2}SUCCESS: Feature '{0}' with parent '{1}' upgraded successfully.{2}",
                        kvp.Key.FeatureName, kvp.Key.ParentString, Environment.NewLine);
                }
            }
 
            string failures = sbFailedResults.ToString();
            string successes = sbSuccessResults.ToString();
 
            if (!string.IsNullOrEmpty(failures))
            {
                WriteResult(failures);
            }
            if (!string.IsNullOrEmpty(successes))
            {
                WriteResult(successes);
            }
        }
 
        private List<FeatureDetails> getFeatureDetails(List<SPFeature> featuresForUpgradeList)
        {
            List<FeatureDetails> featureDetails = new List<FeatureDetails>();
            foreach (var featureForUpgrade in featuresForUpgradeList)
            {
                featureDetails.Add(featureForUpgrade.ConvertToFeatureDetails());
            }
 
            return featureDetails;
        }
    }
}

Appendix - some interesting things I learnt about PowerShell

  • The pipeline concept – much like piping elsewhere (UNIX, jQuery spring to mind).
    • This basically allows multiple commands to be chained together into very powerful one-liners. Made possible by the fact that each cmdlet returns an object to the pipeline for the parent script or downstream cmdlets to process.
  • Passing parameters to PS functions – the syntax doesn’t separate parameters with a comma, which no doubt always fools newbies!
    • So an example from a script I wrote yesterday: 
      SetPropertyOnWebApp $webApplication "portalsuperuseraccount" $objectCachePortalSuperUser
  • The SharePoint PowerShell framework comes with custom ‘PipeBind’ objects for many SharePoint objects – using these for your parameters allows the SharePoint object to be identified in different ways (e.g. SPSite by URL or GUID)
  • Implementing Should-Process/-WhatIf in a cmdlet – thanks to Shay Levy (PowerShell MVP) for helping me with this :)
    • If your cmdlet has a property of ‘SupportsShouldProcess=true’ on the Cmdlet attribute, callers can use the –WhatIf switch. You can test for this by calling the ShouldProcess method on the base Cmdlet class – it will be false if –WhatIf was used.
    • A related approach is used when your cmdlet should ask for confirmation (-Confirm) before making a change/writing data
  • Parameter sets in a cmdlet
    • So long as the caller specifies at least one unique optional parameter, you can identify which ‘set’ of parameters was used and do conditional logic on this. This is akin to method overloading in C#.

A great source of learning for SharePoint/PowerShell development is of course, Gary Lapointe – the source code for all his cmdlets is published on his blog.

Monday, 2 August 2010

Feature upgrade (part 4) – advanced scenarios

In this article series:

  1. Feature upgrade (part 1) – fundamentals
  2. Feature upgrade (part 2) – a sample to play with
  3. Feature upgrade (part 3) – introducing SPFeatureUpgrade kit
  4. Feature upgrade (part 4) – advanced scenarios (this article)
  5. Feature upgrade (part 5) – using PowerShell to upgrade Features

If you’ve been following this series you should hopefully be familiar with SP2010 Feature upgrade by now, conceptually at least. As you start to use this capability for real, it will quickly become clear that there are some interesting scenarios where it’s difficult to know for sure what would happen. I’ll document some examples that I’ve become aware of already in this post, but will also come back and add further cases as I come across them – kinda like a ‘Feature upgrade FAQ’ of sorts. I don’t have too many of these at the moment, but I wanted to share what I know now rather than wait for a longer list to materialize – if you have a scenario in mind that you’d like checked out, by all means leave a comment and I’ll look into it.

Here’s what I have so far:

Question: What happens if multiple VersionRange elements match my Feature instance version?

Consider the case where we have a Feature instance (N.B. this means a particular instance of a Feature on a particular web, site or web app - depending on the scope in the Feature definition) at version 1.0.0.0, and we apply the following upgrade instructions to it:

<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/" Version="3.0.0.0">
  <UpgradeActions>
    <VersionRange BeginVersion="1.0.0.0" EndVersion="1.9.9.9">
        ...
    </VersionRange>
    <VersionRange BeginVersion="1.0.0.0" EndVersion="2.9.9.9">
        ...
    </VersionRange>
  </UpgradeActions>
</Feature>

Answer: Since both of the VersionRange elements match, what happens? The answer is that both VersionRange elements will be processed against the Feature, in the order they are defined in the file. Any instructions within the VersionRange elements will be processed, be they declarative or imperative via CustomUpgradeAction. This is probably the behavior you’d expect, but it’s good to know. [P.S. Special thanks to Gary Yeoman (Combined Knowledge instructor) for winning the bet and bottoming this one out before I managed to get to it :)]

Impact: What this means is that if you have ‘overlapping’ VersionRange elements like this, you might need to plan for the fact that both sets of upgrade actions will execute for Feature instances caught by the ranges. This could be in contrast to other Feature instances across the farm which are at say, 2.0.0.0, and therefore would only be caught by the second VersionRange. Would both sets of Feature instances be in the right ‘state’ to upgrade successfully? Are we removing anything in the first VersionRange (e.g. a content type) which is used in the second VersionRange? Clearly this depends on what you’re doing, but is worth considering if you do overlap.


Question: What happens if a Feature which has been upgraded several times is activated for the first time on a site/web?

Consider the case where you have a web-scoped Feature which has been upgraded a few times to (for example) version 3.0.0.0 across all of the webs which currently exist. What happens if you create a brand new web at this point? Are all the upgrade steps executed when the Feature is activated there?

Answer: No – since activation and upgrade are different actions. However, note that some declarative provisioning will happen, since Visual Studio adds element manifest links in two places in the Feature manifest.xml file. Specifically, if you add a new element manifest during a Feature upgrade (e.g. to provision some new fields/content types/modules etc.), Visual Studio adds this into the UpgradeActions/VersionRange/ApplyElementManifests section but also into the good old-fashioned ElementManifests section too – you can see this by looking at the preview of the generated file. Of course, the latter section is what SharePoint Features have always used in Feature activation, meaning that any artifacts in the referenced elements file will be provisioned on both initial activation and subsequent upgrade. Note this only applies to elements files - other declarative steps such as AddContentTypeField and MapFile are specific only to upgrade, and the same applies to imperative steps linked with CustomUpgradeAction.

Impact: Features must be designed carefully to deal with the fact that Feature instances can have varying lifecycles. Since a new Feature is given a version number of 0.0.0.0, one option may be to use overlapping VersionRange elements (as described above) which always start at 0.0.0.0, and then immediately upgrade the Feature(s) when a new site/web is created in our scenario. Otherwise, if you are upgrading Features using AddContentTypeField or CustomUpgradeAction, it may be necessary to ensure the same steps happen on activation also – for example by ensuring the same code is called in FeatureUpgrading and FeatureActivated. This is a good illustration of some of the complexities around Feature lifecycles.
   

Question:  Do Feature activation dependencies affect the order in which Feature instances are upgraded?

Answer: Yes – any depended on Features will be upgraded before dependent Features.

Impact: It should be possible to reference ‘parent’ artifacts which are provisioned in the same Feature upgrade cycle in dependent Features. For example, if a Feature upgrade provisions a field, dependent Features would be able to reference that same field because it would exist by the time their upgrade executes.

Summary

This post outlines some considerations around Feature upgrade and ALM in SharePoint, and (probably!) concludes my series on this topic. If you’re interested in hearing more on this I have a chapter in Real World SharePoint 2010 (“the SharePoint MVP book”), where I discuss this and related topics - coverage there includes things like versioning assemblies in SharePoint 2010 development and recommendations for assembly/versioning strategies.

P.S. I’d love to hear any interesting Feature upgrade scenarios for me to test out to add to the list!

Sunday, 25 July 2010

Feature upgrade (part 3) – introducing SPFeatureUpgrade kit

In this article series:

  1. Feature upgrade (part 1) – fundamentals
  2. Feature upgrade (part 2) – a sample to play with
  3. Feature upgrade (part 3) – introducing SPFeatureUpgrade kit (this article)
  4. Feature upgrade (part 4) – advanced scenarios
  5. Feature upgrade (part 5) – using PowerShell to upgrade Features

This is the third article in my Feature upgrade series. I wanted to post this earlier but (typical developer) I got caught up in doing some more work on the tool I’m introducing here. Back in the ‘Feature upgrade fundamentals’ post, I mentioned that although we now have this great framework for versioning and upgrading Features (thus providing a great vehicle to roll out new/modified functionality to our SharePoint sites), there are no tools out-of-the-box to help manage this process. This often confuses people when they start to look at Feature upgrade – no amount of redeploying/upgrading solutions or deactivating/reactivating Features will trigger the upgrade event (because reactivation is *not* upgrade, by design!), and SharePoint 2010 does not ship with any user interface, STSADM command or PowerShell cmdlet to do this. In other words, a situation also known as a “partner opportunity”! What you do get is a series of methods in the API – although it doesn’t take too much to throw together a console app which will fill most of this gap, there are a few permutations and there is some complexity around handling failed upgrades. And anyway, in my world it’s always nice to have the option of doing things visually.

So, just like the Content Deployment Wizard plugged the gap of SharePoint’s content deployment framework not having a rich UI, my SPFeatureUpgrade kit on Codeplex might be useful if you start to version/upgrade Features in your environment. The kit installs as a WSP and all source code is published on the Codeplex site.

The following is a slightly expanded version of the Codeplex blurb – any of the images can be clicked on to enlarge.

What's in the kit

The kit is comprised of 3 things:
  • An application page in Central Admin - administrators can use this to upgrade Features of any scope across the entire farm
  • An OPTIONAL page in each site collection's Site Settings area - this can be used to devolve responsibility for upgrading Features (and therefore updating functionality provided by developers) to local site administrators. A Feature must be enabled on each site to enable this.
  • [Coming soon] - a PowerShell cmdlet which allows Feature upgrade to be run from script. (This is the final piece and will be ready shortly, since I’ve got the core logic implemented already.)

One useful thing that the application pages allow you to do is selectively upgrade Features – it could be that you don’t want all sites or webs to receive the new functionality in the upgraded Feature. These pages display checkboxes next to each Feature instance so you can control what gets upgraded where. This is also useful if you plan to ‘phase’ the rollout of new functionality deployed using Feature upgrade.

[Sidenote – Feature upgrade is typically a developer-led thing, since it’s developers who build Features. If developers do not have permissions to to use the admin pages (or PowerShell) then clearly some communication will be required with the administrators who do. Note also that, as I discussed in Upgrading sandboxed solutions, the Feature upgrade concept does not apply in the sandbox since all Features instances there are automatically upgraded when a sandboxed solution is updated.]

Usage

Central Admin page

After deploying updates to a Feature, this page can be used to find and upgrade instances of the Feature across the farm. The tool is accessed from the ‘System Settings’ area with Central Admin:

CentralAdmin_FeatureUpgradeLink

Farm administrators can then select the scope (Farm, WebApplication, Site or Web) and then find Feature instances which require upgrade (this uses a set of methods named QueryFeatures() in the SharePoint API). In the image below I've selected 'WebApplication' as the scope and clicked 'Search' to display the results – I get a table of Features in different web applications which need upgrade (i.e. an updated version has been deployed but not yet upgraded):

(click any image to see larger/clearer version)
CentralAdmin_UpgradeWebAppFeatures 
I can use the checkboxes to selectively upgrade some instances and not others (by default all will be upgraded, but this can be controlled by deselecting checkboxes). In the image below I've upgraded 1 of the 2 web application Features found previously - the success/failure result of all feature instances are shown on the page:

CentralAdmin_CompletedUpgradeWebAppFeatures

In the example above, the selected Feature has now been upgraded.

To help the experience when used in larger farms, when the Central Admin page is used to upgrade Site or Web scoped Features, additional filters appear to allow you to restrict the scope of the query. [Sidenote - this uses some gratuitous jQuery to slide the filter controls in – I can practically hear a couple of server admins (or Spence :)) complaining about things sliding around, but hey I’ve got to have somewhere to practice!]. So for upgrading Site-scoped Features, a selector appears allowing to select the parent web application:

CentralAdmin_UpgradeSiteFeatures

Similarly for Web-scoped Features, you filter on parent site collection:

CentralAdmin_UpgradeWebFeaturesFiltering

Usage

Site Settings page
The Site Settings page works in a very similar way, except only Site and Web-scoped Features can be upgraded (in the current site collection). The ‘COB.SharePoint.Utilities.SiteSettingFeatureUpgrade’ Feature must be activated against each site collection which should show a link to this page. When enabled, the tool is accessed from the Site Settings area for the site collection:

SiteSettings_FeatureUpgradeLink

Site collection administrators can then select the scope (e.g. Site or Web) and then find the Features which require upgrade. In the image below I've selected 'Web' as the scope and clicked 'Search' to display the results:

CentralAdmin_UpgradeWebFeatures

As before, I can use the checkboxes to selectively upgrade some instances and not others (by default all will be upgraded, but this can be controlled by deselecting checkboxes). In the image below I've upgraded 2 of the 4 web Features found previously - the success/failure result of all feature instances are shown on the page:

FeatureUpgradeResult

Summary

SharePoint 2010 introduces the ability to version and upgrade Features, but doesn’t provide any high-level tools to manage this process. I’m providing tools for farm administrators and (optionally) site collection administrators to do this. At the time of writing this article, a final piece in the form of a PowerShell cmdlet is still in progress but should be available soon.

The kit can be downloaded from http://spfeatureupgrade.codeplex.com/

Thursday, 8 July 2010

Renewed again as SharePoint MVP

Received the mail from Microsoft last week to say I’m a SharePoint MVP for another year (my third). Looking back over the last 12 months, it’s been fun being an MVP in the run-up and aftermath of a new release and I definitely feel privileged to have been in that position. I had some opportunities to do interesting things such as help write the 2010 exams, produce MSDN content and co-author a book, which was great. Ultimately I had to decline several of them due to the combination of full-time job, existing community commitments and the arrival of our twins, but then you can’t have everything :)

This list is mainly for me rather than you, but the main things I did were:

One thing that does nag at me is that I know that some blog comments slipped through the net during the super-busy periods. I feel bad about this, but sometimes they just come in too thick and fast and I don’t seem to have a good system for finding unanswered ones! Apologies to those affected – by all means try again and I’ll do my best to answer.

Many thanks to those who left feedback, it’s always appreciated :)

Tuesday, 6 July 2010

Feature upgrade (part 2) – a sample to play with

In this article series:

  1. Feature upgrade (part 1) – fundamentals
  2. Feature upgrade (part 2) – a sample to play with (this article)
  3. Feature upgrade (part 3) – introducing SPFeatureUpgrade kit
  4. Feature upgrade (part 4) – advanced scenarios
  5. Feature upgrade (part 5) – using PowerShell to upgrade Features

In my last post I covered some fundamental aspects of SharePoint 2010’s Feature upgrade framework, and covered concepts such as upgrading Features using pure XML (ApplyElementManifests, AddContentTypeField) or shelling out to code (CustomUpgradeAction) to perform the necessary steps. For some reason it feels like Feature upgrade is one of those areas which is slightly harder to get up and running with, possibly because some XML will need to be cranked for some steps, or maybe because coming up with life-like upgrade scenarios with realistic artifacts takes some effort in itself. So, I thought it might be useful if I made available a sample I’ve been using (e.g. in my ALM talks and book chapter) for folks to download and use, in the hope that developers interested in seeing this stuff for themselves might have less work to do.

The sample

In addition to providing the source/XML files, I spent some time documenting the exact steps to go through when using the files to ‘execute’ my Feature upgrade scenarios. There’s a 3 page Word document which walks you through the upgrade steps, meaning the sample is effectively a “lab” of the type you might do on a training course:

 FeatureUpgradeSample_Doc

Similarly, the files provided cover all the different “states” in the process so you can just copy and paste the bits as you go:

FeatureUpgradeSample_Files

The upgrade examples

I came up with the following upgrade scenarios for the sample:

  • Upgrade iteration 1 - using ApplyElementManifests and AddContentTypeField to add new fields to a content type and push down to lists using it.
  • Upgrade iteration 2 – using CustomUpgradeAction/FeatureUpgrading to pseudo-upgrade a Text field to a Note field, by provisioning a new Note field/running code to iterate list items/copy data from old field to new/mark old field as hidden. (This scenario illustrates that many upgrades require code.)

The link

Download the files from http://sites.google.com/site/sharepointchris/sp2010_featureupgradesample

Summary

Feature upgrade is a slightly complex area, and I’m providing a sample that might help if this is an area of SharePoint 2010 you want to look more closely at. I’ve tested the documentation/files, but if anyone uses the sample and hits an issue, feel free to leave a comment here and I’ll do my best to help. The book chapter I wrote contains a rendition of this sample (along with much wider Application Lifecycle Management coverage, as you’d expect), but I thought it would be good to publish it to a wider audience.

Wednesday, 23 June 2010

Feature upgrade (part 1) - fundamentals

In this article series:

  1. Feature upgrade (part 1) – fundamentals (this article)
  2. Feature upgrade (part 2) – a sample to play with
  3. Feature upgrade (part 3) – introducing SPFeatureUpgrade kit
  4. Feature upgrade (part 4) – advanced scenarios
  5. Feature upgrade (part 5) – using PowerShell to upgrade Features

In recent articles I’ve touched on the introduction of versioning and upgradability to the Features framework in SharePoint 2010. I’ve covered this topic recently in conference talks (SharePoint Evolutions) and as a chapter in the forthcoming Real World SharePoint 2010 book – however I also want to cover it to a certain level here, as I’m mindful that not everyone came to the talks or will buy the book. I also want to introduce a tool I’ve published on Codeplex which might be useful if you decide to use Feature upgrade - more on this in the next article.

When would I use Feature upgrade?

Feature upgrade is useful in the following scenarios (though there may be more!):

  • To make changes to existing site collections or sites, or something inside – for example:
    • Making changes to existing items e.g. adding a new column to a content type/list
    • Adding new elements to existing sites e.g. a new list
    • Any change which involves doing something to a site using the API e.g. using code to modify the navigation settings for many sites
  • To add new functionality into an existing Feature, rather than create a new one – perhaps because that’s the most logical factoring
  • Where some functionality will be upgraded several times during it’s lifecycle, possibly in a situation where the changes are not rolled out to every site, or are rolled out at different times

If you’re modifying or expanding on functionality developed using Features (across many sites or just one), then Feature upgrade is likely to be a good vehicle to roll out your changes. This is thanks (in part) to the new QueryFeatures() methods in the API, which provide a convenient collection to iterate over to apply the changes. When the changes themselves are also implemented in code, in the past developers may have put their implementation in the FeatureActivating event and then ensured the Feature is deactivated/reactivated - however this came with baggage, since you might have had other code in there which was never intended to be re-run. Feature upgrade is designed to solve such problems.

What does Feature upgrade look like?

Microsoft added some new XML to the Features framework to support Feature upgrade. When a new version of a Feature is created, this may involve:

  • Incrementing the version number of an existing Feature (this is mandatory for Feature upgrade to happen)
  • Adding some new XML to define new items for the Feature
  • Writing some code to execute in the new FeatureUpgrading event in the Feature receiver
  • All of the above

Here’s an example of an upgraded Feature highlighting some of the possibilities:

<?xml version="1.0" encoding="utf-8" ?>
<Feature xmlns="http://schemas.microsoft.com/sharepoint/" Version="1.0.0.0">
  <UpgradeActions>
    <VersionRange BeginVersion="0.0.0.0" EndVersion="0.9.9.9">
      <ApplyElementManifests>
        <ElementManifest Location="SomeFunctionality_Iteration2\Elements.xml" />
      </ApplyElementManifests>
      
      <AddContentTypeField ContentTypeId="0x010073f25e2ac37846bb8e884770fb7307c7"
          FieldId="{536DC46C-DC26-4DB0-A97C-7C21E4362A85}" PushDown="TRUE"/>
      <AddContentTypeField ContentTypeId="0x010073f25e2ac37846bb8e884770fb7307c7"
          FieldId="{4E7A6719-011A-47EA-B983-A4941D688CA6}" PushDown="TRUE"/>
 
      <CustomUpgradeAction Name="UpdateSomething">
        <Parameters>
          <Parameter Name="PassSomeValue">This is a string</Parameter>
        </Parameters>
      </CustomUpgradeAction>
    </VersionRange>
</Feature>

[Sidenote] Note that the above XML is actually a subset of the full feature.xml file – when working with Feature upgrade, it is necessary to step outside of the Feature Designer in Visual Studio 2010 and edit the XML files directly (the old-fashioned way!). When doing this the best choice is to allow VS to merge your changes with the XML it is managing. The XML being managed by VS gets merged with your custom XML when the WSP is packaged – if you could this bit of the XML isolated (you can’t, since you never need to), this might look like this:

<Feature Title="Some functionality" Id="cae1f65d-0365-42e9-9907-356c7983e902" Scope="Site">
  <ElementManifests>
    <ElementManifest Location="SomeFunctionality\Elements.xml" />
    <ElementManifest Location="SomeMoreFunctionality\Elements.xml" />
  </ElementManifests>
</Feature>

Essentially Visual Studio will still manage our element manifests, but any XML around Feature upgrade needs to be edited by hand. Walking through the box containing the main XML, we can see:

  • The Feature has a Version attribute (to be incremented each time the Feature is upgraded)
  • A VersionRange element defining the upgrade steps to process for a particular upgrade i.e. when an existing Feature instance within the BeginVersion and EndVersion is upgraded with an updated Feature definition
  • An ApplyElementManifests element – this is used to add new elements to an existing Feature. When the Feature is upgraded, any items (e.g. content types, modules etc.) will be provisioned according to the element manifest(s)
  • AddContentTypeField element – this is a convenience mechanism for the common task of adding a field to an existing content type (a very common upgrade scenario). Note the PushDown attribute – this is hugely useful as it does the work of pushing down the change from the site content type to all list content types within the site (and therefore, all lists), all without any code.
  • CustomUpgradeAction element – this allows the developer to point to some code to run to perform the upgrade actions. It will be common to need this approach, given the vast array of things you might want to do in an upgrade. In terms of the ‘pointing’, in fact the code will always be the FeatureUpgrading method in the receiver, but the value passed in the Name attribute is passed to this method to identify which code to run (along with any parameters). Hence your FeatureUpgrading method is likely to contain a switch statement and would look something like this if it was to match up with the above XML:
    public override void FeatureUpgrading(SPFeatureReceiverProperties properties, string upgradeActionName, System.Collections.Generic.IDictionary<string, string> parameters)
    {
        SPWeb parentWeb = (SPWeb)properties.Feature.Parent;
     
        switch (upgradeActionName)
        {
            case "UpdateSomething":
                string someValue = parameters["PassSomeValue"];
                // do some stuff.. 
                break;
            default:
                break;
        }
    }

In addition to these possibilities there is one further declarative element – MapFile. MapFile allows you to repoint the location of an uncustomized file, this will literally update the pointer in the database. The precise scenarios where you’d want to use this (as opposed to simply deploying an updated version of the original file) escape my tiny mind unfortunately – the only thing I can think of is that if it allows the repointing to be done at different scopes (e.g. web), in a DelegateControl kind of way, that could be very useful. I’m unable to verify this however as I just cannot get MapFile to work, and neither can others (@jthake) that I know have tried. Oh well.

Taking a step back to look at these tools, it’s easy to think that if you don’t happen to be adding a field to a content type then realistically you’re looking at code. However ApplyElementManifests is often all you need for some scenarios e.g. a set of new fields + a new content type + a new publishing page layout.

Notes

These are some ‘fundamental’ things to know – I’ll discuss some more advanced aspects in a future article:-

  • Feature upgrade does NOT happen automatically (including when the Feature is deactivated/reactivated)! The only way to upgrade a Feature is to call SPFeature.Upgrade(), typically in conjunction with one of the QueryFeatures() methods. My tool which I’ll go on to talk about is a custom application page which helps you with this part – note there is no STSADM command, PowerShell cmdlet or user interface to do this out-of-the-box.
  • On the VersionRange element, BeginVersion is inclusive but EndVersion is not. In other words, a Feature instance will be upgraded if the current version number is equal to or greater than BeginVersion , and less than  EndVersion.
  • Upgrade instructions are executed in the order they are defined in the file.
  • If a Feature does not have a Version attribute, the version is 0.0.0.0.
  • Enabling logging can help diagnose any issues. In the ULS settings, under the ‘SharePoint Foundation’ category, set the following sub-categories to Verbose to see more info:
    • Feature Infrastructure
    • Fields
    • General

Summary

SharePoint 2010 introduces additional lifecycle management capabilities with the ability to version and upgrade Features. There are some declarative elements such as ApplyElementManifests and AddContentTypeField, but using the CustomUpgradeAction element allows you to shell out to code where necessary. The only way to actually perform upgrade on a Feature once it has been updated is to call SPFeature.Upgrade() on each instance of the Feature (e.g. in each web) which should be upgraded – new QueryFeatures() methods help you locate Feature instances which can be upgraded. I’ve written a custom application page which helps manage this process, to be discussed next time.

Thursday, 3 June 2010

Upgrading sandboxed solutions

I’ve had a big focus on Application Lifecycle Management and SharePoint 2010 recently, covering things such as the changes in the Solutions and Features frameworks, in particular to allow for ‘upgrading’ Features. I’ve not posted much info here (yet), but delivered the material over two sessions at the recent SharePoint Evolutions conference (you can get my slide decks here) and have a chapter in ‘Real World SharePoint 2010’ (aka the MVP book) on the topic. Today I want to briefly cover some info which I didn’t manage to talk about at the conference – specifically ALM and the sandbox. Just like farm solutions, most sandboxed solutions will most likely evolve over the course of their lifetime, with new functionality added and existing functionality refined - changes will be made to both code and artifacts (e.g. content types, list definitions etc.). Vendors will release new versions of their products, and clients will demand enhancements to other solutions they use which are delivered as sandboxed solutions. Importantly, the process for implementing such changes to existing apps is different between farm and sandboxed solutions – this post explores these differences.

Sidenote: the best starting point for this info is having an understanding of SP2010 ALM concepts such as Feature upgrade, better support for assembly versioning etc. - my slide decks linked earlier cover these, but I’ll try to provide some background with each point.

Difference 1 – upgrading Solution packages

Most devs and admins are familiar with upgrading WSPs using STSADM –o upgradesolution or the ‘retract/dedeploy’ cycle. In SharePoint 2010 much stays the same here aside from the addition of PowerShell commands such as Update-SPSolution. The key difference when upgrading sandboxed solutions is that a new filename for the WSP must be provided each time it is upgraded – the solution ID is then used internally to tie things together. I’m assuming this is because all versions of a sandboxed solution are stored in the Solution Gallery (unlike farm solutions, which only store the latest version once deployment/upgrade is complete) and of course, the Solution Gallery is fundamentally a SharePoint list – which, like any other, doesn’t allow multiple items with the same title. 

No doubt Microsoft could have chosen other options such as using list item versioning or making use of separate fields for name and title, but clearly these approaches were not preferred, so renaming WSPs it is. The logical approach will probably be to add version numbers into WSP filenames for sandboxed solutions.

Difference 2 – Feature upgrade and QueryFeatures()

The Feature upgrade framework is often a great way to make changes to existing SP2010 sites – in a nutshell, it is typically used in conjunction with the new QueryFeatures() method in the API, which helps you find Feature instances at different scopes which need upgrading. A Feature instance needs upgrading when the corresponding Feature.xml file in the SharePoint root has an incremented version number (e.g. 2.0.0.0) but SPFeature.Upgrade() has not yet been called for that instance. What happens is that you run QueryFeatures() on the appropriate object (e.g. SPSite to find Site or Web-scoped Feature instances), and then call SPFeature.Upgrade() for all items returned, assuming you wanted to upgrade the Feature everywhere. As an example, if you had a site collection with 50 webs and called SPSite.QueryFeatures(SPFeatureScope.Site, true), you would then get a collection of 50 Feature instances at Web scope and call SPFeature.Upgrade() on each one. Alternatively, you might choose to only upgrade selected webs – e.g. when only certain web should receive the updated functionality.

The key difference in the sandbox is that it is not possible to upgrade Feature instances selectively – upgrading the WSP has the effect of calling SPFeature.Upgrade() automatically for all instances. This means there is perhaps less flexibility in this area which could be important in some scenarios.

Difference 3 – issues with assembly versioning in the sandbox

A key improvement in ALM in SharePoint 2010 is increased support for versioning assemblies. WSP packages can now add BindingRedirect elements to web.config, meaning a reference to a 1.0.0.0 assembly can be redirected to (e.g.) 2.0.0.0. In some cases this works equally well in sandboxed and farm solutions, but my testing shows issues with sandboxed assemblies which contain web parts or other controls. Aside from the main issue, there is currently a minor gotcha with both sandboxed and farm solutions and SafeControl entries - the WSP deployment removes the 1.0.0.0 entry, and a 2.0.0.0 SafeControl entry alone isn’t sufficient so a “type not registered as safe” error occurs. However, note this can be mitigated by manually editing the solution manifest to add a 1.0.0.0 entry which will be merged with the automatically-generated entry for 2.0.0.0. Something seemingly more serious is that when versioning assemblies with web parts in the sandbox, even performing this mitigation step to ensure all BindingRedirect and SafeControl entries are correct results in a broken web part (error “Unhandled exception was thrown by the sandboxed code wrapper's Execute method in the partial trust app domain: An unexpected error has occurred”).

Since many web part vendors are expected to use the sandbox (in addition to the rest of us), this would mean that many are unable to leverage the versioning and release management improvements in the sandbox (for web parts at least). Other strategies such as using the FileAssemblyVersion instead for versioning may be necessary (doesn’t affect binding/SafeControls). Some folks in the Product Group are looping me in with the team who implemented versioning in the sandbox, so I’ll hopefully find out more and update this post. Right now I’m hoping there was a flaw in my (many) tests - if not, I’m hoping it’s something we might see fixed fairly soon. I’ll keep you posted.

Friday, 14 May 2010

Tips for building SharePoint 2010 base VM images

Now that SharePoint 2010 has RTM’d, lots of new virtual machines are emerging chrysalis-like from their proud host machines (hmm, that analogy might need some work) - whilst it’s mainly the early adopters at the moment, this will be a common process for most folks involved with SharePoint 2010 during it’s lifespan. Of course if you build many VMs, it’s worth spending time on some base images which you will use as templates for others. In this post, I’ll run through what I’ve been using as my VM “tree” which saves me time when creating new machines – if you don’t have images like this at the moment, you might be wasting time when it comes to spinning up a new environment. Even worse is being reluctant to create a new VM because it entails lots of hassle – having a barrier like this is not conducive to accurate development/testing/research because frankly, developer VMs tend to grow “baggage” which isn’t always cleaned up. My setup is developer-focused and is actually what I’m running at home on my laptop (8GB RAM) rather than at work, but you should be able to easily identify if you are working with different constraints/requirements.

The goals of creating decent base images are:

  • Reduce the effort across different configurations (e.g. Foundation/Server, standalone/farm etc.) by installing/configuring common elements once
  • Reduce the time it takes to create a new VM
  • Reduce human error in install process (since we do it fewer times)

Overall configuration

When creating SharePoint 2010 VMs, one of the early decisions is how to deal with platform services such as Active Directory, DNS and SQL. Briefly the choices are (focusing on AD here, but many also apply to SQL):

  • Have each VM host AD Directory Services, so each VM is in it’s own domain
  • Have a separate VM to host AD, other VMs are members of the same domain
  • Don’t run under AD, instead use the “create config DB using PowerShell” trick to run under local accounts. Without AD you won’t be able to do certain things (such as user profile sync, obviously) so you need to be happy with these trade-offs.

I’ve opted for:

  • One VM running Windows 2008 Server Core for AD DS, DNS etc. (I allocate just 256MB RAM and less than 5GB of disk space to this!)
  • One VM running ‘full’ SQL 2008 + SP2010 (I peg SQL to use a maximum of 1GB RAM – my feeling is that SharePoint needs it more in a developer VM)

However I wouldn’t necessarily recommend this factoring for everyone. I do it this way because I think if I have a hardware failure shortly before say, a big talk (as you can probably tell, it’s happened), I could get up and running on someone else’s borrowed laptop with a backup of the SP2010 VM pretty quickly. Sure I’d have some domain/account issues to deal with (if I didn’t have a backup of the AD VM), but it feels more self-contained and easier to restore if I had to. I might be slightly crazy on some of the logic here, and at some point I might cut over to the approach I’d generally recommend:

  • One VM running AD, DNS and SQL
  • One VM running SP2010

If you ever have other VMs running at the same time (e.g. an SP2007 VM or another SP2010 VM if you have the RAM), you will definitely see more performance with this option. You really don’t want more instances of SQL (in particular) running on the same physical hardware than you need.

My base image “tree”

In terms of a set of base images for creating new VMs from, this is what I currently have – the blue boxes represent base images, the gold ones are VMs which I actually run and use:

SharePointVMTree

Hopefully the names are descriptive enough to get a quick idea of each machine’s purpose. As you can see my “instance” VMs aren’t imaginatively named, and the “-1” suffix is simply because I’ll accumulate ‘n’ instance VMs over the next few years. In the unlikely event it’s not clear, SP is SharePoint Server, whilst SPF is SharePoint Foundation!

2010-BASE

  • Win 2008 R2
  • Roles
    • Application Server
    • File Services
      • Web Server (IIS)
  • Features
    • Remote admin tools e.g. AD tools
  • SQL 2008 SP1 CU2 (at some point I’ll upgrade this to R2) – note there is an element of risk in having SQL in a base image, and it’s important to know about this. For example I’ve not verified *everything* SharePoint needs works – I discuss the issue issue and how to deal with it further on.
  • WCF hotfix for R2 (KB976462)
  • Conveniences

Then sysprep’d.

2010-DEVBASE

As above with:

  • SharePoint 2010 pre-requisites installed
  • Visual Studio 2010 (RC)
  • SharePoint Designer 2010
    • Dev tools such as Reflector Pro (with the SharePoint assemblies decompiled and ready for debugging), I’ll probably add ReSharper 5.0 here soon etc.
  • CKS:DEV

Then sysprep’d.

SP2010-BASE

As 2010-DEVBASE but with SharePoint 2010 installed – * Config Wizard/psconfig not run.

SPF2010-BASE

As 2010-DEVBASE but with SharePoint Foundation installed – * Config Wizard/psconfig not run.

Benefit

This allows me quickly create new VMs as most of the install pain went into the base images. Even things like having Reflector Pro decompile the SharePoint assemblies (shh!) in the base image means I don’t have to repeat that step on each VM I end up using – if I’m knee-deep debugging SharePoint code I can guarantee I won’t be happy waiting 15 mins for that step to complete. It would be great if we could do more configuration than the above in base images, but I don’t think you reliably can. Also, I’d love to hear if you do things differently with your base images.

General notes

The following are some notes around the images and the process I use to clone:

  • The base image VMs are not joined to any domain
  • The images are all sysprep’d – if you’re not doing this with your base images, you will run into issues if you ever run 2 machines at the same time on the same domain
    • Note that sysprep resets lots of things, including Windows activation and the Windows Update status (Windows will report that updates have never been installed, though whatever updates the base image had will actually be there).
  • Hopefully everyone knows this, but in SharePoint 2010 (as with 2007), it’s not possible to have base images which have SharePoint installed and configured. Or rather, it’s possible but if VMs are created from such images they cannot reliably exist on the same network/domain, let alone the same SharePoint farm. This is because SharePoint stores the machine name in the config database, and if you ever want two instances running on the same network/domain you’ll need to rename one instance, which will then not match up with it’s database entries. The latest point you can get to is with the SP installation done but the Config Wizard not run – kind of inconvenient, but…
  • …of course PowerShell can help with making the rest of the configuration (e.g. provisioning/configuring service applications) as repeatable and quick as possible. Gary Lapointe (a man that should know if ever there was one) suggests that a 100% PowerShell scripted configuration may be out of reach in the current release due to some config steps which can’t really be done with PowerShell, but at some point I’ll likely try and stitch together some scripts which suit most of my needs.

Process to create a new VM

To create a new VM, I follow this process – note there are some hooky SQL steps here due to it being included in a sysprep’d base image, keep reading for why and how this can now be avoided:

  • Do a VMware (or Hyper-V) “clone” from a sysprep’d image (this will usually be my SP2010-DEVBASE image)
  • Start VM and then:
    • Change machine name
    • Join to COB domain
    • Do Windows updates, reboot
  • Grant the account which will be used to run the SharePoint config wizard rights in SQL. Since sysprep has changed the SIDs of the accounts, no user has permissions to SQL Server at this point. To fix, we need to start SQL in single-user mode and add logins/permissions:
    • Any any accounts which need SQL permissions outside of what the SharePoint config wizard will grant when it runs. The main example here is the domain account used to install/configure SharePoint, in my case “COB\sp_setup” (which should also be a local admin):
      1. Open SQL Server Configuration Manager
      2. Stop the main SQL service.
      3. Go into the properties of this service, and on the Advanced tab find the Startup Parameters property. Add “;-m” to the end (without the quotes, ensure you have no spaces) and click ‘OK’.
      4. Restart the service (you’re now in single-user mode).
      5. Create a SQL login for your install account, and add to the following server roles:
        1. securityadmin
        2. dbcreator
      6. Note that if you had SQL service accounts running as built-in system accounts, there could be further work to do there. I have a domain account for this purpose and didn’t run into issues with SQL service accounts.
      7. I’ve also seen references that the server name in SQL needs to be updated, but that’s not my experience.
  • Run SharePoint configuration wizard/psconfig
  • Do other configuration e.g. service apps

Sysprep and SQL – a better way

So that’s a bit of a smell – what’s the deal? Briefly, it is not supported to run sysprep on a machine running SQL Server, something I’m not sure lots of SharePoint folks know. There is no way in hell I’d think about doing things this way with images used for a client’s environment – some research suggests things like Reporting Services and DTC transactions are examples of SQL areas which do not play well with sysprep. However, like me, you might find sysprep’d images with SQL work just fine for SharePoint developer VMs, depending on what you’re doing. The alternative is somewhat more complex, but I’ll likely cut over to it fairly soon even for my home stuff – SQL Server 2008 R2 (released a couple of weeks ago) adds support for sysprep! Basically it’s a two-step installation process, the first half of which is done in the base image (“prepare”), the second in each instance (“complete”).

Josh Meyer has a great walkthrough of the overall process for creating SharePoint 2010 VMs.

Some background reading on sysprep and SQL:

Tuesday, 4 May 2010

SharePoint 2010 ALM – my slide decks from SPEvo

It took a while due to some other commitments, but I finally got round to making my slide decks from the SharePoint Evolutions Conference available. I delivered 3 sessions in the end, 2 of my own and one on behalf of Andrew Connell, one of the many travel casualties due to the ash cloud. I had a lot of fun talking – it was also interesting to see the room seemed to agree with my theory that 90% of the custom assemblies in SharePoint 2007 environments are still at version 1.0.0.0 :) As I hopefully conveyed, SharePoint 2010 has made some useful progress in this area, and previous pain points such as versioning assemblies and uncertainty about how to go about upgrading apps should be easier to deal with.

The details of my talks are below - as always, a lot of the goodness was in the demos, but I also tried to get a lot of the info into the slides:

Managing the SharePoint 2010 Application Lifecycle – beyond version 1.0.0.0 (part 1)

  • Provisioning with C# or XML
  • Changes to the Features framework
  • Leveraging WSP import when building Features
  • How to version/upgrade a Feature
    • XML upgrade actions
    • Code upgrade actions
  • Triggering upgrade
  • Feature upgrade gotchas

Demo 1 – using ‘Save as site template’/WSP import
Demo 2 - using ApplyElementManifests and AddContentTypeField to add fields to a content type and push down.
Demo 3 – using CustomUpgradeAction/FeatureUpgrading to pseudo-upgrade a Text field to a Note field, by provisioning a new Note field and running code to iterate list items/copy data from old field to new/mark old field as hidden.

Link - http://slidesha.re/9N7i1H

Managing the SharePoint 2010 Application Lifecycle – beyond version 1.0.0.0 (part 2)

  • Feature upgrade recap
  • How to upgrade different artifacts
    • Provisioned
    • Declarative
  • Upgrading workflows
  • Changes to the Solutions framework
    • Solution dependencies
    • Binding redirects
  • Dealing with shared assemblies
  • Versioning strategies

Demo 1 – recap of Feature upgrade demo
Demo 2 – incrementing the version number of a shared assembly, using BindingRedirect element deployed to web.config via WSP framework

Link - http://slidesha.re/ddE7RX

Going forward, I’ll be posting the following related stuff, hopefully very soon:

  • My custom application page for upgrading SharePoint 2010 Features
  • Sample Visual Studio projects containing the Features I used to demo upgrade