SharePoint dev strategies - it's not all about Features!

Something I've been meaning to discuss for a long time is the decision to develop SharePoint artifacts using Features or some other approach. I actually discussed this back in May 2007 in my post SharePoint deployment options : Features or Content Deployment?, but feel it's a topic worth revisiting/expanding on as I often see teams developing with Features without fully working out what exactly they are getting out of this approach. As you can guess by the article title, I'm not sold on the idea of using Features all the time (readers who have followed this blog from the start might find this surprising given I wrote many articles on how to work with Features) and want to put forward some points to consider when working out whether you need them or not.

Let's first consider some (selected) characteristics of Features:

• Provide a means of deploying SharePoint artifacts such as list templates/site columns/content types to multiple environments (e.g dev, test, production)
• Currently the only way to deploy such artifacts across multiple site collections
• Require some extra overhead to create, even with the community tools available (in comparison with creating artifacts directly in the SharePoint UI)
• Little/no support for certain key updates (e.g. updating a content type which has already been deployed and is in use) - updates must be done through the user interface or the API, since modifying the original Feature files to make changes is unsupported.

Given these points, one scenario I really can't see the benefit of Features for is when the solution consists of just one site collection - which is often the case for WCM sites. Why go through the extra hassle of packaging up artifacts into Features and be faced with difficulties managing updates when the artifacts will only ever exist in one site collection anyway? Sure, they may need to be deployed between environments but we have other ways of doing that.

N.B. The same applies to site definitions - why go to the trouble of creating a custom site definition when only one site will ever be created from it?

The alternative

If you aren't forced into using Features to deal with multiple site collections, not using them could be the 'most valid' choice. In my recent WCM projects, I haven't used Features for anything which doesn't require a Feature (e.g. a VS workflow, a CustomAction etc.) for a long time now, including the project I discussed recently in SharePoint WCM in the finance sector and Developer lessons learnt - SharePoint WCM in the finance sector. Certainly given the extremely tight timescale on that project, I actually feel we could have failed to deliver on time if we had used Features.

Instead, my approach is to create a blank site in the dev environment, and do all the list/site column/content type/master page development there using the SharePoint UI and SPD. My next step (perhaps not surprising to regular readers) is to use my Content Deployment Wizard tool to move all the SharePoint artifacts to the other environments when ready. Equally, you could choose to write your own code which does the same thing using the now well-documented Content Deployment API. You'll need to deal with any filesystem and .Net assets separately (generally before you import the SharePoint content on the destination), but in my view we've at least drastically simplified the SharePoint side of things. This seems to work well for many reasons:

• More efficient since no development time lost to building Features
• The update problem described earlier is taken care of for you (by the underlying Content Deployment API) - as an example, add a field to a content type in dev, deploy the content which uses it and the field will be added on the import site
• Concept of a 'package' is maintained, so .cmp files produced by the Wizard can be handed to a hosting company for them to import using the Wizard at their end. I hear of quite a few people doing this.
• We can store the .cmp files in source control and use them as part of a 'Software Development Lifecycle' approach. My approach (and I'd guess that of others using the tool in this way) is to store the .cmp file alongside the filesytem files such as .ascx files for the current 'release', and import them as part of the deployment process of moving the release to the next environment.

As an aside, when I decided to write a tool which would simplify dealing with dev/QA/UAT/production environments on SharePoint projects, I was initially torn between 'solving the content type update problem' and something based around the Content Deployment API. One reason why I decided on the latter was because the CD API already seemed to have solved the other issue!

Now I'm certainly not saying it works perfectly every time (it doesn't, though is much improved following SP1 and infrastructure update), but in my experience I seem to spend less time over the course of a project resolving deployment issues than I would do building/troubleshooting Features. Additionally, using Content Deployment allows deployment of, well, content - if your solution relies on pre-created publishing pages or you have a scenario such as your client creating some content in UAT which needs to be moved to production before go-live, Features won't help you here. The Content Deployment mechanism however, is designed for just that.

Where do Solutions (.wsp) fit in all this?

So to summarize the above, my rule of thumb for projects which aren't built around multiple site collections is don't use Features for things which don't absolutely require them. So where does that leave Solution packages (.wsp files) - should they be abandoned too? Well no, definitely not in my view. Solutions solve a slightly different problem set:-

• Deploying files to SharePoint web servers such that each server in a farm is a mirror of another. Ensuring all web front-ends have the same files used by SharePoint is, of course, a key requirement for SharePoint farms - this applies to Feature files when using them, but also to assemblies, 12 hive files etc.
• Web.config modifications e.g. the ‘SafeControls’ entry required for custom web parts/controls
• Code Access Security config modifications e.g. those required for controls not running from the GAC
• Some other tasks, such as deployment of web part definition files (.webpart)

Really, there's nothing stopping you from doing all this manually if you wanted to (especially if you're always deploying to a single server, so there are less things to keep in sync). But the point here is that Solutions genuinely do make your life easier for comparatively little effort, so the 'cost/benefit' ratio is perhaps different to Features for me - the key is using one of the automated build approaches such as WSP Builder. So, my recommendation would generally be to always use Solutions for assemblies, 12 hive files etc., particularly in multiple server farm environments.

Conclusion

My rules of thumb then, are:

• Consider not using Features (and site definitions) if your site isn't based around multiple site collections - using the Wizard or some other solution based on Content Deployment can be the alternative
• Use Solutions if you have multiple servers/environments, unless you're happy to have more work to do to keep them in sync
• If you are using Features, plan an approach for dealing with updates such as content type updates

My message here possibly goes against some of the guidance you might see other folks recommend, but I'm just going on the experience I've had delivering projects using different approaches. As always, the key is to consider deployment approach before you actually come to do it!

P.S. Also remember, deploying using backup and restore is a bad idea ;-)

Source code for SP Content Deployment Wizard now released!

Back from holiday now, and hopefully with something useful for those of you who use my Content Deployment Wizard tool to move SharePoint content around. I've now released the source code for the tool onto Codeplex, so if you have an interest in the Content Deployment API or just want to see how the wizard works, it's now all there for you. This is kind of a big step for me as I saw the tool more as a "free product" (built over many late nights after Suzanne was asleep!) rather than an open source project - I figured I would release the code at some point, but intended to wait a little longer until the next release, since I'm actually halfway through changing the code over for the new functionality. However, my hand was slightly forced because the Codeplex administrators suspended the project due to me not having supplied source code - apparently this is a rule of the site which I had missed. I wasn't happy with the tool not being available for people who wanted it, so wanted to rectify this as soon as possible. So this explains why anybody attempting to download the tool over the last 3 or 4 days was unable to - sincerely apologize for the inconvenience people!

In any case, it had been on my mind because there seems to be an increase recently in people building their own tools and functionality around the Content Deployment API - it's certainly an area with a lot of scope for custom solutions, and folks occasionally drop me a line with a request for a help. So it seems right to stop being precious about it and release the code sooner rather than later. I also fixed an annoying bug which caused an unhandled exception if a user with insufficient SharePoint permissions used the tool.

Anyway, hope it's useful to someone!

Content Deployment Wizard - Codeplex site homepage
Content Deployment Wizard - Release 1.1 with source code

P.S. I have a couple of things temporarily ahead of Wizard development in the queue, but as mentioned previously the focus for next version will be to add support for command-line use/scripting. This means the Wizard can be used in scheduled/automated deployments, with the benefit of the granular selection of content to deploy which standard Content Deployment does not provide.

Developer lessons learnt - SharePoint WCM in the finance sector

So in my recent SharePoint WCM in the finance sector post, I talked about what we built and why I think the result is kind of interesting. What I want to do today is share some of the technical lessons learnt, and give a sense of what worked and what didn't. As I mentioned last time, UK-based folks will hopefully be able to gain more than I can provide here when the site gets presented at the SharePoint UK user group, meaning we'll answer any question you care to come up with, not just some of the developer stuff I want to discuss today.

Now to frame all this, it's important to consider the type of project this was - the terms mean slightly different things to different people, but to me the emphasis was on 'development', as opposed to 'implementation' or 'customization'. In code terms, we ended up with the following:

• 17 Visual Studio projects in total
• 4 Windows services
• 5 nightly batch processes
• 5 supplementary SQL tables (outside of the SharePoint db)

Not bad for 8 weeks work. As an aside, although the first number seems surprisingly high, in the technical washup I did with the team nobody thought this wasn't the right way to factor the code. This is partly explained by the fact this single project is actually part of a bigger program of projects being done for the client, and also partly by the complexity of the Endeca search implementation and batch processes we needed.

What worked well (in no particular order)

• Using a 'development farm' amongst the developers - this means the content database is shared, and thus no effort is required for one developer to see the lists, site columns, content types, master pages/layouts etc. created by others. This is actually the only way to do team development in SharePoint for me, but worthwhile mentioning it as I know not all SharePoint shops do things this way.
• Proper use of tracing - this is the idea of writing log statements throughout code to easily diagnose problems once the code has been deployed to other environments (e.g. QA/UAT/production). We used the standard .Net System.Diagnostics trace framework with levels of Verbose, Info, Warning and Error - this has been familiar to me for a long time but a couple of the devs were new to it and agreed it was invaluable. In particular, we had a lot of library code and it's often difficult to find logic bugs when you can't directly see the result of something on a screen. For me, tracing essentially gives you the power to find certain bugs in seconds or minutes which could otherwise take hours to resolve. Although adding the tracing code can slow down coding, to mitigate this we used..
• ReSharper - at the start of the project I created several ReSharper templates to call our common code e.g. for tracing, and got all the team to download trial versions of ReSharper. This meant we could add trace statements in just a few keystrokes, meaning the 'I didn't have time to add trace!' excuse couldn't be used :-)
• My Config Store framework based on a SharePoint list * - we stored over 130 'configuration items', from 'True'/'False' config switches such as 'enforce password change for users first logon,' to known URLs, to certain strings displayed throughout the site. We also found a couple of areas for improvement (e.g. field not big enough to store XML fragments!) which will hopefully make it into the next release.
• Implementing logging/notifications for unhandled exceptions - I know the MS Enterprise Library a component for this, but we developed our own using a HTTP handler which sits in front of SharePoint's SPRequest handler. This means that whenever something happens in the code which we're not expecting, we get to find out about it immediately and can see the stack trace and other debug info in the e-mail. This was invaluable when the testers got to work, as it meant we could proactively deal with bugs before they even got reported. As soon as we noticed a mail with a new exception, we shout over to the particular test guy (identified by the user ID) "What exactly did you just do?" (which impressed them greatly!), so we nail the exact set of circumstances/data which caused the bug right there and then. 
• My Content Deployment Wizard tool * - I also played the 'deployment/release manager' role on this project, so I was probably the guy who benefited from this the most but I've actually used it somewhere on every single project I've done since building it. For releases when the team had updated x page layouts, x lookup lists and x Config Store items, the tool is invaluable for picking out just the changed items and deploying them to the other environments. For Config Store items in particular it was useful as some config is different between environments (similar to web.config keys) so you don't want to overwrite the entire list. For early releases when the team had made lots of of complex 'schema' updates (such as lots of intricate changes to site columns/lookups/content types), due to the time pressures I elected to take the 'everything will definitely work this way' route and drop the site collection on the target and import the whole thing (since no valuable data to preserve) so there are some complex deployment scenarios I still haven't fully tested personally, but with 3 environments on top of the dev environment to deploy to the Wizard was prettes tilitiy much a lifesaver.
• Cross-site lookup field by SharePoint Solutions - this solves the problem that a lookup column can only lookup data in the current web. We use this for several key sets of data so we get to have one copy and one copy only. Damn useful.
• LINQ to SQL - we use this for CRUD operations in our supplementary SQL tables, and the guys who used it agreed they saved significant time over the standard approach of writing ADO.Net code.

* hopefully it doesn't come across as shameless self-promotion to include these - the very reason I built them was to solve recurring problems I saw on SharePoint dev projects, and both utilities really did help us here.

Project challenges (in no particular order)

• Team Foundation Server weirdness - for reasons we still haven't established, we found the .csproj file for the web project (i.e. the most critical VS project!) would be checked out whenever a developer compiled the solution. With multiple checkout enabled, this means that pretty much every developer had the project file checked out all the time, regardless of whether he/she was making any project changes (e.g. adding new classes). This meant we had many more merge issues than normal - not fun.
• VM issues - for a while we thought ReSharper was the culprit here, but a VS hotfix brought more stability. A hunch says at least some of the issues are 64-bit related (our dev environment was matched to production in this respect), since often the problem would manifest itself via Visual Studio (a 32-bit application remember). Frequent VS crashes, "attempted to read or write protected memory" messages in the event log - oh joy.
• Failure to identify shared code soon enough - often a concern on complex development projects when the team is working at high speed. We did daily standup meetings (similar to scrum) but I suspect we may have focused too much on issues rather than what was being 'successfully' developed. So we lost some time to refactoring to bring things back in line, but this is why I like to think of the approach as 'Dangerously Rapid Application Development' (for those who remember the term ;-))
• Issues arising from sharing IP addresses on SSL - in several of our environments, we attempted to use the technique documented by Adrian Spear in To setup SSL on multiple Sharepoint 2007 web applications using host headers under IIS 6.0. I've used this successfully in the past but had some problems this time round - despite working fine in our QA environment, we had problems in other places. After carefully analyzing the differences, I worked out that this technique will only work if the SSL certificate being used is a wildcard certificate or is matched on the machine name rather than the site URL. This might be obvious to other people but wasn't to me!

Hope someone finds this useful!

SharePoint WCM in the finance sector

You might have been wondering where I've been for the last few weeks, so I thought I'd show rather than just tell. I rarely talk about specific 'daytime' projects I work on, but I guess I'm quite proud of this one and you might be interested in what we've done. I've been working with LBi (Europe's biggest design/technology agency) recently, leading a team of 5-7 developers on a WCM (Web Content Management) project for one of the global investment banks. You can see some aspects of what we've done with SharePoint in this post, and I'll follow up with some developer-to-developer info about techniques we used in the next one. Additionally, for UK-based folks I think Riaz Ahmed (my current boss and Mr SharePoint at LBi) is hoping to do a real show and tell at the UK SharePoint user group in a couple of months, where there'll be the opportunity for Q&A and to pick up far more info/lessons learnt (both technical and non-technical) than I can provide here.

The site is targeted at a closed audience over the internet (our client's clients), so unfortunately you can't see it for yourself - the site provides analysts with high-value information on financial markets, primarily in the form of market reports and newsflashes, though there are other content types. The user experience we were tasked with implementing (created by LBi's design team) means the user gets to have some pretty impressive functionality around all this (IMHO) which differentiates the service from others.

The centrepiece is the market report data, which is held for many countries and topics:

This is imported nightly from a master datasource (thus lending itself nicely to caching), and our client can also add comments to the imported data via SharePoint WCM page editing. Perhaps the slickest piece of functionality is the facility for the user to create personalized 'smart reports' with specific markets and topics which interest him/her. These are saved against the user's profile for quick access:

At runtime we build the personalized page from the same cache layer used by the standard pages in the first shot.

Since financial markets change quickly, our client's authoring teams (distributed around the world), can quickly create newsflashes via SharePoint publishing functionality for display on the site - this is controlled via SharePoint workflow. Users can search and filter by country, date, high priority flag and can also store in a 'My newsflashes' bucket for quick retrieval:

Users also have the option of receiving e-mail alerts when newsflashes are added to the site (immediate for high priority newsflashes, nightly batch for others), and can tailor their e-mail preferences by country...

..and topic:

The site also has a ton of other good stuff, such as:

• single sign-on with a sister site
• integration with a higher-level portal which can authenticate users itself (meaning users do not log-in separately on our site)
• WCAG 'A' -level accessibility - accessibility wasn't particularly a focus of the client for this site, but (happily) is just how things are done here anyhow. We use a lot of JavaScript to enhance the user experience but it always provides equivalent functionality when disabled - I have to say the LBi front-end developers use some incredibly innovative techniques to achieve this, I've rarely seen anything like it (see below for some examples).
• integration with Endeca for search (including the newsflash landing page you see above - this is powered by search)
• custom admin functionality for user creation/management
• many many smaller but 'interesting' requirements such as newsflashes having user-selected 'related items', attachments etc etc!

Rich, accessible controls

In addition to all the personalization, one of the things that I love about the user experience is the funky controls we have. So why have a dull dropdown like this:

...when you can have one like this:

 

This allows us to present only countries the user has told us they are interested in in the dropdown (thus avoiding clutter), whilst allowing them to easily see data for other countries if they have the occasional need to. To do this, clicking the 'More...' link expands the dropdown to show a second section containing the other countries (or more specifically, the other countries they are permitted to see):

Similarly for multiple selections, why have this:

...when you can have this:

 

Again, clicking the 'More...' link allows the user to select a country not in their 'preferred' countries:

Personally I think this makes for a great user experience, and it's all accessibility-compliant.

Project plan : caffeine

The development time for everything you see here (and everything you don't) was around 7-8 weeks, though this excludes some of the front-end development (HTML, CSS, JS). We're not quite over the finishing line yet, but are in the final stages of user acceptance testing and security testing. To say it's been tough would be something of an understatement, but I'm incredibly proud of my guys and it's been a great team effort - fortunately for me they're very talented :-)

Next time I'll detail some of the technical decisions we took and go through some developer bits which we felt worked well when building/deploying the site.

SPDataSource - every developer's friend (part 2)

SPDataSource - a refresher

So last time in part 1 of this 2-part series, we saw how SPDataSource is a great option for fetching data from lists since not only does it do the actual work of fetching the items for you, it also does this without any C#/VB.Net code. We can retrieve all the items from the list, or optionally supply a CAML query along with the list details if we wish to filter/sort the items. So as I showed in part 1, we can easily display a dropdown containing the items from a list with the following markup:

<SPWebControls:SPDataSource runat="server" ID="dsPersonTitles" DataSourceMode="List" 
  SelectCommand="<Query><OrderBy><FieldRef Name='SortOrder' Ascending='true' /></OrderBy></Query>"
  <SelectParameters>
      <asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
      <asp:Parameter Name="ListName" DefaultValue="PersonTitles" />
  </SelectParameters>
</SPWebControls:SPDataSource> 

<asp:DropDownList runat="server" ID="ddlPersonTitles" CssClass="title" DataSourceID="dsPersonTitles" DataTextField="Title" DataValueField="ID">
</asp:DropDownList>

We also saw how SPDataSource offers more than just retrieving items from a list, with the following other 'modes' (see part 1 for more details on these):

• CrossList - similar to doing a query with SPSiteDataQuery across all lists in a site collection
• ListItem - show field values from a single list item


• Webs - lists all webs in a site collection


• ListOfLists - lists all lists in a web

What I want to focus on this time is how to use parameters with SPDataSource, since when using the control for real you'll often want to do this.

Parameters with SPDataSource

SPDataSource works like other .Net data source controls, so the key to passing parameters is using one of the .Net 2.0 parameter classes, either declaratively (in markup) or in code. The example above shows using the basic Parameter class by setting the DefaultValue property to a known string, but the following parameter types can also be used (though note I haven't tried them all and the SPD Team Blog article I mentioned says somewhat vaguely that "most" of them can be used!):

• ControlParameter


• CookieParameter


• FormParameter


• ProfileParameter


• QueryStringParameter


• SessionParameter

Effectively using one of these saves you writing code to read the value from the respective location and passing it to SPDataSource yourself. I think that ControlParameter and QueryStringParameter are possibly of the most value, though all could be interesting depending on your requirements. As an example, to get the value a user selected earlier from a dropdown containing a list of lists and pass it to a SPDataSource control I would need:

<SelectParameters>
    <asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
    <asp:ControlParameter Name="ListName" ControlID="ddlLists"
      PropertyName="SelectedValue"/>
</SelectParameters>

I seemed to have some control execution lifecycle fun with ControlParameter but it did work in the end. To pull the value from a 'userID' querystring parameter I would need:

<SelectParameters>
    <asp:QueryStringParameter Name="userId" QueryStringField="userId" />
</SelectParameters>

And that's not all - I can also drop a parameter value into a string such as a CAML query used in the earlier example of 'List' mode:

<SPWebControls:SPDataSource runat="server" ID="dsPersonTitles" DataSourceMode="List"
  SelectCommand="<Query><OrderBy><FieldRef Name='{SortField}' Ascending='{SortAsc}' /></OrderBy></Query>">
  <SelectParameters>
      <asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
      <asp:Parameter Name="ListName" DefaultValue="PersonTitles" />
      <asp:QueryStringParameter Name="SortField" QueryStringField="SortField" DefaultValue="SortOrder" />
      <asp:QueryStringParameter Name="SortAsc" QueryStringField="SortAsc" DefaultValue="true" />
  </SelectParameters>
</SPWebControls:SPDataSource> 

Notice here I'm dynamically specifying the sort field and direction from querystring parameters. Cool stuff.

Even cooler is the idea of building your own parameter control - one that strikes me is one for retrieving values from a user's SharePoint profile (since the ProfileParameter listed above refers to ASP.Net profiles). Scott Mitchell has a great guide to this at Accessing and Updating Data in ASP.NET 2.0: Creating Custom Parameter Controls.

Summary

So we've covered a lot of ground there, so here's a recap of why the SPDataSource is your friend:

• Can bind to any control which can do data-binding - DropdownList, ListBox, CheckBoxList are some obvious candidates, but consider also Repeater, DataGrid and SharePoint DataView/SPGridView controls


• Can easily get parameters from other controls, querystring, session, form values etc. (or write some code to fetch from another location)


• Can use a variety of modes to make different queries e.g. items in a list, properties of a list item, webs in a site etc.

If you want to see more I recommend reading SPDataSource and Rollups with the Data View - this emphasises using SPDataSource with SharePoint DataViews but also has some good examples I haven't covered, such as using the 'CrossList' mode.

SPDataSource - every SharePoint developer's friend (part 1)

Sooner or later, nearly every SharePoint developer needs to write code to retrieve all the items from a list and display them - either on the page, or often in a control like a dropdown list control. An example could be retrieving a list of countries, or perhaps person titles for a form:

Clearly there are benefits from storing such values in a list, since we (or the client) can then add/edit/delete items easily. To implement this, the developer's first thought might be to write code like this:

private void bindPersonTitles()
{
    // the name of the DropDownList control we are populating is ddlPersonTitles..
    using (SPWeb configWeb = SPContext.Current.Site.AllWebs["configuration"])
    {
        SPList titlesList = configWeb.Lists["PersonTitles"];

        foreach (SPListItem titleItem in titlesList.Items)
        {
            ddlPersonTitles.Items.Add(new ListItem(titleItem.Title, titleItem.ID.ToString()));
        }
    }
}

Which is fine. Except if we want to filter or sort the list items, we really should use a CAML query instead of iterating through all the items - the code below shows this (sorting on a column called 'SortOrder'), and also has a slight twist on the previous example in that I'm using data-binding rather than looping through the items 'manually':

private void bindPersonTitlesByCamlQuery()
{
    using (SPWeb configWeb = SPContext.Current.Site.AllWebs["configuration"])
    {
        SPList titlesList = configWeb.Lists["PersonTitles"];
        SPQuery query = new SPQuery();
        query.Query = "<OrderBy><FieldRef Name=\"SortOrder\" /></OrderBy>";
        SPListItemCollection titlesItems = titlesList.GetItems(query);

        // showing alternative of using data-binding rather iterating through items..
        ddlPersonTitles.DataSource = titlesItems;
        ddlPersonTitles.DataTextField = "Title";
        ddlPersonTitles.DataValueField = "ID";
        ddlPersonTitles.DataBind();
    }
}

Fine again. Except I can't help thinking "all this just to take the items from a list and put them in a dropdown??" Surely there must be a better way. And what did Microsoft do every time they needed to do this, surely they didn't repeat the above code in every place in SharePoint? The answer is no, they used this:

SPDataSource

In short, SPDataSource is a web control which implements IDataSource and saves you writing code like the above. The great thing is that it is extremely flexible, and as we'll see, can be used for more than you might think. The best thing though, is that being a control it can be used declaratively, so I can bind my dropdown to the list without writing a single line of C# or VB.Net code - all I have to do is set properties correctly. We'll go through the detail in a second, but the markup to replace the last code sample would look like:

<SPWebControls:SPDataSource runat="server" ID="dsPersonTitles" DataSourceMode="List" 
  SelectCommand="<Query><OrderBy><FieldRef Name='SortOrder' Ascending='true' /></OrderBy></Query>"
  <SelectParameters>
      <asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
      <asp:Parameter Name="ListName" DefaultValue="PersonTitles" />
  </SelectParameters>
</SPWebControls:SPDataSource> 

<asp:DropDownList runat="server" ID="ddlPersonTitles" CssClass="title" DataSourceID="dsPersonTitles" DataTextField="Title" DataValueField="ID">
</asp:DropDownList>

So we're doing the following:

• setting the 'DataSourceMode' for the SPDataSource is set to 'List' - we'll examine other possible values shortly


• setting the 'SelectCommand' to the CAML query we want to use - here we're just sorting on the 'SortOrder' field once more


• telling the SPDataSource which list we want to use by supplying ASP.Net 2.0 parameter objects named 'WebUrl' and 'ListName' - we'll dive more into this later


• finally we bind the dropdown to the data by specifying the DataSourceID to the ID we gave our SPDataSource, and also say which fields in the resultset we want to use for the 'Text' and 'Value' of the dropdown. Incidentally I recommend using the ID for the value and the Title for the text (as shown above) so that it's easy to create an SPLookupValue to update a list item - more on this in part 2

We're only just starting to see the power of SPDataSource, but I love this approach for a couple of reasons:

• Details of my query aren't specified in compiled code, so if anything about the list changes (e.g. we restructure our site) I don't have to recompile and redeploy assemblies


• Less custom code, less bugs!


• Setting properties is arguably simpler than writing code

So let's dive deeper into what we can do - we'll cover 'modes' of SPDataSource in this article and how to dynamically pass parameters to control the query in part 2.

The different 'modes' of SPDataSource

SPDataSource isn't just limited to fetching the items from a list (DataSourceMode = 'List'). Other possibilities are:

• CrossList - similar to doing a query with SPSiteDataQuery across all lists in a site collection (for a sample of this see the SharePoint Designer Team blog post linked at the end of this article)


• ListItem - show field values from a single list item


• Webs - lists all webs in a site collection


• ListOfLists - lists all lists in a web

For the last 2 modes I was able to bind but had some difficulty working out values to use for the 'DataTextField'/'DataValueField' of my control. I was trying to simply get the web or list name, but none of the obvious values such as 'Title', 'ListName' etc. were in the resultset. I was unable to find any other information on this but I'm sure some more trial and error would solve it. The 'ListItem' mode can be interesting - in the following sample I'm binding a single list item to a DataGrid to show selected fields from the item, which itself is selected by using the 'ListItemID' property:

<SPWebControls:SPDataSource runat="server" ID="dsPeople" DataSourceMode="ListItem" UseInternalName="true"> 
  <SelectParameters>
      <asp:Parameter Name="WebUrl" DefaultValue="/configuration/" />
      <asp:Parameter Name="ListID" DefaultValue="34F91B0C-FCF2-455A-ABBA-67724FB4024A" />
      <asp:Parameter Name="ListItemID" DefaultValue="1" />
  </SelectParameters>
</SPWebControls:SPDataSource> 

<asp:GridView ID="grdPeople" runat="server" DataSourceID="dsPeople" 
    AutoGenerateColumns="False">
    <Columns>
        <asp:BoundField DataField="FullName" HeaderText="Blogger name" />
        <asp:BoundField DataField="WorkCity" HeaderText="City" />
        <asp:BoundField DataField="Blog_x0020_URL" HeaderText="Blog URL" />
    </Columns>
</asp:GridView>

This would give something like this, based on an underlying list item which has these fields (no formatting yet applied):

I found I had to specify UseInternalName = "true" and use the ListID rather than ListName parameter in this mode.

Next time - passing parameters to SPDataSource

So far we've looked at supplying parameters by using a standard ASP.Net parameter control and specifying the value in the DefaultValue' property. In fact, ASP.Net has a whole range of parameter controls which can do the work of retrieving a parameter from somewhere and passing it to the SPDataSource, so that's what we'll look at next time. Additionally, since we're often using SPDataSource to bind data to form controls, we'll look at the common scenario of getting the selected item out of the form control to save back to a lookup field in SharePoint.

<updated>That link to the SPD blog article I mentioned but forgot to link to is http://blogs.msdn.com/sharepointdesigner/archive/2007/04/24/spdatasource-and-rollups-with-the-data-view.aspx - the focus here is mainly on using SPDataSource with a DataView, but there's some great info and samples.</updated>

Received MVP award for SharePoint!

Had notification from Microsoft today that I'm now a SharePoint MVP :-)

This is great news, I'm really thrilled to be recognized for the community stuff I've done over the past year, and personally I know I've been impressed by the work of so many existing SharePoint MVPs, so it's a big honor to be part of that group. I'm particularly grateful to the people who nominated me, thanks guys. It was weird having to look back to fill in the spreadsheet MS ask you to complete to be considered for MVP - I guess some of the key highlights for me were:

• building the SharePoint Content Deployment Wizard
• other Codeplex tools such as the SharePoint Config Store and my solution to deploying lookup fields as a Feature
• blogging in general - not sure how scientific the process was, but nice to appear in the top 25 (just!) of Joel Oleson's top 100 SharePoint blogs list
• the speaking gigs e.g. my VS workflow deep-dive

I guess I'll enjoy putting the MVP image on my blog etc., but the biggest reward is without doubt when someone leaves a positive comment or mentions in person that they've found the articles or tools useful - I'm never sure if people realize how much this is listened to. Anyway, I'm busy on the next version of the Wizard and some other articles, so you'll hear a lot more from me in the future.

Still a long way to go until I'm Andrew Connell or any number of the others though ;-)

P.S. Great to hear my mate Vince received it as well, definitely well deserved..

 

Content Deployment Wizard updated - Release 1.0

I've released a new version of the Content Deployment Wizard today, labelled as 'Release 1.0'. This release adds only minor changes to beta 2, and is mainly intended to pave the way for a future beta release which will contain new functionality. However I also want to communicate that the current codebase is now stable, since I heard feedback from some folks saying the 'beta' label was putting some people off (e.g. non-technical clients of SharePoint consultants).

That said, the following minor updates have been incorporated:

• ability to modify which lists are filtered out of the treeview:
  
  Certain 'system'-type lists are not shown in the treeview as they are generally not relevant to import/export operations, but on occasion you may want to 'unhide' some lists e.g. 'Variation Labels' and 'Relationships List' lists if you are using variations. See the readme.txt file for instructions on how to do this
  
  
• minor bug fixes e.g. .cmp file now correctly saved with extension if a period (.) is present in your chosen filename, validation to prevent child objects being added from treeview when parent site collection already added

The new version can be downloaded from www.codeplex.com/SPDeploymentWizard.

Future development

Work is underway on the next release of the Wizard, with the main emphasis being on:

• allowing import/export operations to be scripted
• allowing settings used in an operation to be saved to a file for easy reuse
• support for incremental deployments (hopefully!)

If you have other suggestions, I can't guarantee I can include them in the next release but I'd certainly be interested to hear.

Disposing SharePoint objects - what they don't tell you

If you're a SharePoint developer and find yourself regularly writing code to work with SPListItem objects, here's some info which might help you factor your code in a slightly neater way in some scenarios.  Interestingly this doesn't get mentioned in any of the key papers/articles - I can think of a good reason for this (which we'll look at), but in my mind this is a safe technique. If you have other ideas however, I'd love to hear from you.

Most developers are now fairly well-versed in the techniques needed to write efficient SharePoint code (key resources are listed at the end of the article), specifically in terms of disposing of any SPSite/SPWeb objects your code creates. The thing is, having to always be mindful of disposing these objects can sometimes restrict us in the way we want to write our code - consider the following simplified example:

public void DoSomething()
{
    SPList list = getList();

    // do something with list..
    foreach (SPListItem item in list.Items)
    {
        processItem(item);
    }

    // ** PROBLEM - how do we now dispose of the SPSite/SPWeb objects we created earlier? **
}

private SPList getList()
{
    // can't dispose of these objects here if we're returning a list - we'll be attempting to use 
    // objects which have already been disposed of..
    SPSite site = new SPSite("http://cob.blog.dev");
    SPWeb web = site.OpenWeb("/MyWeb");
    SPList list = web.Lists["MyList"];

    return list;
}

The problem here is we created our SPSite/SPWeb objects in a different method to where we need to dispose them - this can happen a lot when writing library code. To extend this example, here's a pattern I regularly use to ensure I'm getting my objects the most efficient way - using SPContext if it's present, but instantiating them myself if it isn't (i.e. the code is called from an event receiver, console app etc.):

public void DoSomething()
{
    bool bDisposalsRequired = false;

    // get list from SPContext if we have one..
    SPList list = getListFromContext();
    if (list == null)
    {
        // otherwise get list from objects we create..
        list = getInstantiatedList();
        bDisposalsRequired = true;
    }
    
    // do something with list..
    foreach (SPListItem item in list.Items)
    {
        processItem(item);
    }

    if (bDisposalsRequired)
    {
        // ** PROBLEM - how do we now dispose of the SPSite/SPWeb objects we created earlier? **
    }
}

private SPList getInstantiatedList()
{
    // can't dispose of these objects here if we're returning a list - we'll be attempting to use 
    // objects which have already been disposed of..
    SPSite site = new SPSite("http://cob.blog.dev");
    SPWeb web = site.OpenWeb("/MyWeb");
    SPList list = web.Lists["MyList"];

    return list;
}

private SPList getListFromContext()
{
    SPContext currentContext = SPContext.Current;
    SPList list = null;

    if (currentContext != null)
    {
        list = currentContext.Site.AllWebs["MyWeb"].Lists["MyList"];
    }

    return list;
}

When the code starts to become more complex like this, we start to really want a simple way of disposing the objects we created elsewhere. I'd probably caution against this, but what happens if you created the SPSite/SPWeb objects not only in another method, but another class? All of a sudden disposing of objects correctly is a lot more complex than the examples given in the white papers. 

Now, the savvy SharePoint developer might think "Hold on, the SPList has a ParentWeb object (and SPListItem has a ParentList object!), what happens if I go up the hierarchy and dispose that way?" - code to this would look like:

if (bDisposalsRequired)
{
    list.ParentWeb.Dispose();
    list.ParentWeb.Site.Dispose();
}

Now how would we know the correct objects have been disposed? Are these objects the same objects we were using earlier? Or has SharePoint populated these properties with references to different objects, meaning we've left our original objects alive and could run into scalability problems?

Well, the good news is these are the same objects, so disposing in this way will dispose of the objects you created. We can tell this by comparing the objects using System.Object's GetHashCode() method:

private void compareObjects()
{
    SPSite site = new SPSite("http://cob.blog.dev");
    SPWeb web = site.OpenWeb("/MyWeb");
    SPList list = web.Lists["MyList"];
    
    SPWeb parentWeb = list.ParentWeb;
    


    // no message displayed..
    Debug.Assert(parentWeb.GetHashCode() != web.GetHashCode(), "Objects are not same!");
    

    // create a *new* SPWeb object and compare it to our other reference..
    web = site.OpenWeb("/MyWeb");

    // message IS displayed..
    Debug.Assert(parentWeb.GetHashCode() != web.GetHashCode(), "Objects are not same!");
}

What we see from this is that the SPWeb object stored in the list.ParentWeb property has the same hash code as the original SPWeb object we created, but if we create a new instance of the same web, it has a different hash code. This indicates the objects in the first comparison are the same - so disposing of them through the ParentWeb reference will indeed dispose of the objects we created. So happy days - we've found we can factor our code in a way which may be more logical and readable.

The caveat

So why might this not be mentioned in any of the published guidance? Well, before using this technique throughout your code you should consider that in writing code in this way we are effectively relying on an internal implementation detail of the current SharePoint API. If Microsoft were to change this in a service pack (or the next version of SharePoint), our code may no longer dispose of objects correctly. I'd recommend giving this your own consideration (not least because you might conclude differently to me), but I'd suggest this change would be fairly unlikely. The reason for this is the SharePoint team themselves need to eliminate any unnecessary SPSite/SPWeb objects, so of course it makes sense that the the same SPWeb instance used to obtain the SPList is used to populate the ParentWeb variable. Creating an extra instance would make the API less efficient. On that basis, I personally am happy to write code in this way. If you think I've got this wrong (I can't guarantee I haven't), of course I'd love to hear from you.

In an upcoming article, I'll also discuss the idea of going beyond the MSDN examples, and trying to write SharePoint data access code in a more 'enterprise' or 'patterns' kind of way. Stay tuned for more on this.

Disposing SharePoint objects - required reading

• Best Practices: Using Disposable Windows SharePoint Services Objects


• Best Practices: Common Coding Issues When Using the Shareoint Object Model

Code/Feature samples for common SharePoint development tasks

There are lots of tools available to help developers, but I always think the single most useful thing is actually seeing a good example of the thing I'm trying to develop. This is why SharePoint blogs are so popular right? As well as checking my favorite blogs or past articles, I'm always going back to previous work - perhaps to see what values I used in a certain SharePoint file or to grab a chunk of code I wrote which I can amend to fit my current requirement. So today I want to point you in the direction of some useful samples which could be good reference material if you're a SharePoint developer - in actual fact I'm not releasing anything new here, but highlighting that my recent 'Config Store' solution provides a great "one-stop shop" of downloadable examples of several common SharePoint development tasks. Examples of the following dev tasks can all be found in the solution:

• Deploy site columns, content types as Feature
• Deploy a list (with default list items) as a Feature
• Using Feature receivers
• Using SPWebConfigModification to make web.config changes in code/as part of a Feature
• Using SPQuery to efficiently find items in a list
• Build a Solution package (.wsp) without WSP Builder
• Write a Solution deployment script to retract Solution, deactivate Feature, redeploy Solution, activate Feature etc.
• Using event receivers
• As a .Net bonus, how to use the HttpRuntime class to work with ASP.Net caching outside of a web context

The entire set of code/Feature files can be downloaded from Codeplex, making it a great reference project containing samples of all the tasks listed above. If this is useful to you, the list below details the file to open to find the code sample, and some notes which might help you to get to grips with the task:

Deploy site columns, content types as Feature:

File	Notes
ConfigStoreElements.xml	This is accomplished by use of the Field/ContentType Feature elements. Note these should be before any ListTemplate/ListInstance items in the same elements file - if using both sets of elements in different files, ensure the field/content type file is listed first in manifest.xml.

Deploy a list (with default list items) as a Feature:

File	Notes
ConfigStoreElements.xml	This is accomplished by use of the ListTemplate/ListInstance Feature elements.

Feature receivers:

File	Notes
ConfigStoreFeatureReceiver.cs	In my case, the Feature receiver makes modifications to web.config and programmatically adds list event receivers to a list.

web.config modifications

File	Notes
ConfigStoreFeatureReceiver.cs	This is accomplished by use of the SPWebConfigModification class. I add three entries to the web.config to support my solution.

 

Using SPQuery to find items in a list:

File	Notes
ConfigStore.cs	I have two methods in the ConfigStore class which perform queries in this way (also known as a CAML query).

 

Build a Solution package (.wsp) without WSPBuilder:

File	Notes
makecab.ddf/                                   manifest.xml	Whilst WSPBuilder offers a great automated way to build Solutions, occasionally there is a need to step outside the tool to build the package 'manually'. In my case, I had a problem where WSPBuilder was adding elements in the 'wrong' order to the generated manifest.xml file, and this was causing the Solution deployment to fail.

 

Solution deployment script to retract Solution, deactivate Feature etc:

File	Notes
COB.SharePoint.Utilities.ConfigStore_Install.bat	This is the script I've often used to deploy my Solutions. It has error checking at each stage so I can easily tell at what point the install fails for simpler problem diagnosis.

 

Using event receivers:

File	Notes
ConfigStoreListEventReceiver.cs	In the Config Store solution I use a list event receiver to add items to the cache when the list contents change. This code shows using the ItemAdded and ItemUpdated events on the list.

 

Using the HttpRuntime class to work with ASP.Net caching outside of ASP.Net:

File

Notes

ConfigStoreListEventReceiver.cs

Something to remember with list event receivers is that we actually don't have access to HttpContext.Current or SPContext.Current - both are null. If it wasn't for the HttpRuntime class, this would be a problem for my Config Store code since I wanted to add/remove items from the cache in the list event receiver (i.e. manage the cache as soon as list items are changed). Fortunately, HttpRuntime allows us access to the ASP.Net framework from any code location (e.g. console app).

 

The link to download all this stuff is at http://www.codeplex.com/SPConfigStore. Hope you find the samples useful, feel free to leave a comment if anything doesn't make sense.