Monday, 8 December 2008

Building multi-lingual SharePoint sites - introducing the Language Store

If you're ever asked to build a multi-lingual site in SharePoint, it quickly becomes apparent that there are a few extra considerations compared to a single language site. These could include:

  • Information architecture
  • Language/culture detection
  • Deciding whether to use variations or not
  • URL strategy

..and so on. Clearly these are decisions which will have a different 'answer' for every multi-lingual project, and typically your client's specific requirements will steer your approach. However one challenge which is likely to remain constant across most such projects is this one:

  • How to deal with the many small strings of text which are not part of authored page content which need to be translated and displayed in the appropriate language

This is the challenge I'm focusing on here. To illustrate, here's an example from the BBC site where I've highlighted all the strings which may need to be translated but which don't belong to a particular page:

BBCExample

..and that's just one page - it turns out a typical site will have many of these. If you have to translate additional strings shown to authors in edit mode only, you could easily find the total number stretching into the hundreds. So we start to need a framework for storage/retrieval of these 'page furniture' items. If we were dealing with a shrink-wrapped product, .Net resource files could be a good choice, but this approach is probably not flexible enough for a website and won't allow content authors/power users to enter translations. Clearly something based around a SharePoint list is called for, so enter the 'Language Store' - my solution to the problem which you can now download from Codeplex (link at the end).

Introducing the Language Store

The Language Store is an adaptation of my earlier Config Store solution and follows some of the same principles:-

  • values are stored in a SharePoint list
  • an API is provided to retrieve values with a single method call
  • a caching framework is used for optimum performance
  • easily deployed as a .wsp

Items in the list are categorized, and have a column for each language translation:

LanguageStoreListNarrow

Note that each translation column in the list is named with the convention 'LANG_<culture name>' (N.B. you might know a 'culture name' as a 'locale ID' or similar) - so when a new language needs to be added to the site, you simply create a new column with the appropriate name and add the translations. A list of culture names can be found in the MSDN documentation for the CultureInfo class.

UPDATED - the 'National Language Support' API reference page on MSDN is a better reference - think the CultureInfo page has changed since I linked to it.

Retrieving values

To retrieve a value, we simply call the GetValue() method and pass the category and title of the item to retrieve:

string sButtonText = LanguageStore.GetValue("Search", "SearchGoButtonText");

Also, since many of the items we might put in the Language Store are only used in the presentation of the page, it's often a shame to have to switch to the code-behind just to fetch these values and assign them to a control's 'Text' property. So I've provided a tokenized method similar to SPUrl, which allows you to simply drop Language Store values into your markup like this:


<asp:Button runat="server" id="btnSearch" Text="<%$ SPLangStore:Search|SearchGoButtonText %>" />

I like this because it means you don't end up cluttering your code-behind with lots of lines just for fetching values from the Language Store and assigning them to ASP.Net labels or controls. For those who don't know how this is done I'll write more about it in the next post as I think it's a cool, under-used facility in .Net.

How the Language Store determines which language to retrieve

In the current implementation, the regional settings of the SPWeb are used to determine which translations are retrieved. It's a single method in the code (a single line in fact!), so this scheme could easily be changed if you have a different requirement. We're using the Language Store on our current project, and using the SPWeb setting makes sense for us since we're building around 100 different sites in ~30 languages, as opposed to one site which displays in the local language (according to the user's thread culture or similar).

Note that if the Language Store doesn't contain a value for the requested culture, a fallback process is used similar to .Net's globalization framework:

  1. Check preferred culture  e.g. fr-CH for French (Switzerland)
  2. Check preferred culture's parent e.g. fr for French
  3. Check default language (determined by configuration) e.g. en for English

This is useful where some items might have a different version for say American English (EN-US) and British English (EN-GB), but other items don't require distinction so a single value can be entered into the parent column (EN).

By the way, if you're wondering where the SPWeb regional settings are because you've never needed to change them, they're here:

RegionalSettingsLink

RegionalSettingsPage

A checkbox allows the regional settings you make on a given web cascade down to child webs, so we simply set this at the root of the site as a one time operation.


Other bits and pieces
  • All items are wrapped up in a Solution/Feature so there is no need to manually create site columns/content types/the Language Store list etc. There is also an install script for you to easily install the Solution.
  • Caching implementation is currently based around a CacheDependency on a file - this enables the cache on all servers in your farm to be invalidated when an item is updated, but does require that all WFEs can write to this location (e.g. firewalls are not in the way).
  • The Language Store can also be used where no SPContext is present e.g. a list event receiver. In this scenario, it will look for values in your SharePoint web application's web.config file to establish the URL for the site containing the Language Store (N.B. these web.config keys get automatically added when the Language Store is installed to your site). This also means it can be used outside your SharePoint application, e.g. a console app.
  • The Language Store can be moved from it's default location of the root web for your site (to do this, create a new list (in whatever child web you want) from the 'Language Store list' template (added during the install), and modify the 'LanguageStoreWebName'/'LanguageStoreListName' keys which were added to your web.config to point to the new location. As an alternative if you already added 100 items which you don't want to recreate, you could use my other tool, the SharePoint Content Deployment Wizard at http://www.codeplex.com/SPDeploymentWizard to move the list.)
  • All source code and Solution/Feature files are included, so if you want to change anything, you can.
  • Installation instructions are in the readme.txt in the download.

You can download the Language Store and all source code from www.codeplex.com/SPLanguageStore. All feedback welcome!

33 comments:

Soren said...

Hi Chris,

Great extension, thank you for sharing with the community!

One thought in regards to performance. How would you rate performance of the Language Store in comparison to .NET resource files?

Kind regards,
Soren

Chris O'Brien said...

Soren,

I've not done a direct comparison but I'm pretty sure I'd expect performance to be better if anything. Unless the app pool has just recycled, the Language Store will be retrieving from memory and that will always be quicker than disk or database.

Thanks,

Chris.

Niklas Östh said...

Relly like this aproach compare to changing the resx file, deploy the new one using a wsp and then run some type timerjob to copy it out to all web front ends inetpub directory for only a small change of one translation.

Matt Taylor said...

Hi Chris

I really liked your Config Store, a common sense solution executed in exactly the right way.

With the Language Store you've done it again, I particularly like the provision for a declarative as well as a programmatical implementation (which is exactly how I would want to use it).

Have a great Christmas, I think you've earned the rest!

Jamie McAllister said...

Chris,

Excellent bit of work! This really does tick all of the boxes.

Jamie

MOSSBUDDY said...

Chris again a GR8 Job, I had couple of doubts that I wanted your advice:
1. What should be the language column name in case if we are using the Sharepoint Japanese template (1041), should it be LANG_JA or LANG_JA-JP?

2. Do you think this approach you mentioned works better (due to the fact of caching) than the SPUtility.GetlocalizedString() and keeping the (custom app sepecific) resource files in the 12\resources folder?

3. If there are more than 30+ labels on a page, it would be efficient to call a single method and pass the array and getback all the localized strings in a collection\array than to call the GetValue() method 30+ times? Do you think creating another method to accept collection and return localized string collection make more sense than calling GetValue() 30+ times?

thanks for your great community effort.

regards
MOSSBUDDY

Chris O'Brien said...

MOSSBUDDY,

Thanks for your comments, some good questions. Taking them one by one:

1. Not 100% sure unfortunately, I wonder if there are two 'Japanese' options in the dropdown for the SPWeb settings? In any case, what you can do is turn tracing on by adding a trace switch, make a request in a web set to the Japanese locale, and you'll see whether JA or JA-JP is being requested.

2. I think the caching should mean the Language Store performs slightly better, but the way I see it the main reason to go with my solution is the fact that it's a SharePoint list which is used to store the values (rather than a file on the filesystem).

3. Yes, I agree this is a valid point, and would be a good enhancement to the current framework - I'll hopefully get round to adding this method soon. Remember though, that it's only the first hit (after an app pool recycle) which this is relevant for - since all others will fetch the value from memory.

Thanks,

Chris.

MOSSBUDDY said...

You are simply Awesome, thanks for your comments. You have a great Christmas and Happy Holidays. You are doing an awesome job for the community. thanks for all the gr8 works

Jaap said...

Great solution for multilanguage sites!

Small detail: I had to use <asp:Button runat="server" id="btnSearch" Text="<%$ SPLangStore:Search|SearchGoButtonText %>" />

The caching didn't work at first but thanks to the tracing I quickly found out that the CacheDependencyFile was not accessible.

Chris O'Brien said...

Jaap,

Oops, many thanks for that correction, I'll update the article.

And yes, the tracing around that section has helped me diagnose being unable to write to the file a few times now ;-)

Cheers,

Chris.

srikanth said...

Hi Chris

I did not find the source code at the url you mentioned. Please let me know from where i can download the code

Thanks,
sri

Chris O'Brien said...

Hi Srikanth,

You can find the source code on the 'Releases' tab on the Codeplex site.

HTH,

Chris.

Chris said...

Hi Chris,
Does it work with WSS as well? Or varition feature is a must to this solution?

regards,
CW

Chris O'Brien said...

Hi Chris,

I've not explicitly testing, but there's nothing in there which uses MOSS functionality. And I did test the related Config Store solution with WSS-only - so it really should be fine.

Thanks,

Chris.

dcarr said...

Chris,

This looks great. Potentially, this could help me out geatly. I have a question though. I have many site collections using hosheader urls in a single webapp. Basically seperate sites in seperate site collections. The default site collection for which the original list is not used. My question is, can I create a Language Store List for each site collection?

Thank you,

D.

Chris O'Brien said...

@dcarr,

You certainly could do this, but you'd need to change this code. Are you sure you don't want to just have a single centralized list though - we had 100 site collections (like you, host header site collections funnily enough), but still preferred the centralized model. The code deals with this fine.

If, however, you do prefer many Language Store lists, check our the 'sister' project, my Config Store framework. In the 2.0.0.0 release, I added the capability of using multiple lists (with 1 master list if the content couldn't be found in the 'local' list). Since the code is 90% the same, you should be able to port over the required bits fairly simply.

HTH,

Chris.

Marshal Nagpal said...

Hello Chris,

Lang store is throwing null exception when used in item reciever class, reason beign getCultureForRequest() method that uses SPContext class. Where SPContext is null

I hope you can fix this issue in next release.

Cheers
Marshal

Chris O'Brien said...

@Marshal,

Hmm, I hadn't considered that the Language Store would be used in an event receiver. I guess I won't say it's not supported but...how would you want the culture to be determined in an event receiver?

If you have an idea on what you'd like this logic to be, I'll try and find the time to implement it.

HTH,

Chris.

Marshal Nagpal said...

may be instead of SPCurrent object you need to create SPSite and SPWeb object in "getCultureForRequest()" method.

Chris O'Brien said...

@Marshal,

OK, I've now amended the code, tested and uploaded to Codeplex. Sorry it's taken a few days for me to get to this, but hopefully this will give you a way to use the Language Store in the way you want to. Since the Language Store code itself wouldn't know which SPSite/SPWeb to instantiate, I've added a new overload of the GetValue() method which allows you to pass an SPWeb. In a Feature receiver (your scenario), you would obtain this object through properties.Feature.Parent.

Version 1.0.1.0 on the Language Store Codeplex site has the updated code.

Hope that helps!

Chris.

Marshal Nagpal said...

Thanks chris, will use updated language store in new project. Current project is in UAT, and we have used .resx in feature to get tokens values.

Really appreciate your efforts for sharing and developing useful components. :)

Devendra said...

Hi Chris
Gr8 article.
I am using Arabic and English .
I want add the Arabic column to the Language store what r the changes i have to make to that code.

Devendra said...

Hi Chris,
I have implemented this one. very good article.
but i tried to implement the same thing on user control . then it is not displaying any thing.Could you please tell me how to implement the same in Usercontrol.

Chris O'Brien said...

Hi Devendra,

In general, adding columns for new languages is done by:

- adding a new column to Language Store list with the appropriate name e.g. 'LANG_ar-YE' for Armenian Arabic (as an example) or 'LANG_ar' for general Arabic
- setting the SPWeb.Locale setting to the appropriate language as described in the article

However, I've never implemented a right-to-left language so not sure if anything extra is required for these languages. If it is, this would be a .Net thing rather than anything specific to the Language Store.

HTH,

Chris.

Chris O'Brien said...

Devendra,

For your 2nd question - hmm, not sure what the problem is here. I've used user controls 90% of the time I've been rendering strings from the Language Store!

I'm guessing it's something else which is the problem (e.g. column name not being 'aligned' with SPWeb.Locale settings or similar). Suggest enabling tracing to help diagnose the problem (uses standard Systems.Diagnostics tracing).

HTH,

Chris.

Devendra said...

Hi Chris,

lblerror.Text = LanguageStore.GetValue("Record added successfully_c", "Record added successfully_t");

this code i added in my user control.
i am getting error
"The name 'LanguageStore' does not exist in the current context"

for this what are the name spaces i have add to my user control . or please suggest me what are the changes i have to do.

Chris O'Brien said...

Devendra,

Ah OK - you need to add a reference to my assembly (COB.SharePoint.Utilities.LanguageStore.dll) and add a "using" directive (or "Imports" in VB) to the namespace which contains the LanguageStore class - this is "COB.SharePoint.Utilities".

HTH,

Chris.

József said...

Hy Chris!

Your solution is great for the first look but I can't install it.

I downloaded the 1.1.0.0 version. You wrote in readme : 1. Find the 'COB.SharePoint.Utilities.LanguageStore_Install.bat' file in the 'InstallPackage' directory, and amend the 'url' parameter to the URL of your site (the web application URL).

I ran the bat with: "COB.SharePoint.Utilities.LanguageStore_Install.bat url http://myweb.com" order.

But it throws an exception:
Can't find the web application (http://cob.test.dev).

Please help me. How can I fix that? Am I misunderstood something?

Chris O'Brien said...

@József,

Ah - you need to edit the .bat file and change the URL in there, not pass it as a parameter. You're getting this error because http://cob.test.dev is still in the batch file.

HTH,

Chris.

shambon said...

Hi, i try o use the solution to create a new langagua knows as Malay(ms) which i try to use the malay(malaysia) language with code (ms-my). but the main problem to me is to add the title n category. is there any guide or article on the title n category. thanks

Chris O'Brien said...

@shambon,

Title and category are just for you to use to categorize the items you want to store. You'll need to use both when retrieving values from the Language Store in code - the GetValue() method takes parameters which correspond to the title and category of the item to retrieve.

HTH,

Chris.

Arash Alvandi said...

Hi Chris, can you please tell me if an updated version of your solution for SharePoint 2010 already exists. I have downloaded the solution from CodePlex and I'm not sure if only the Visual Studio conversion should be sufficient?

Thank you very much

Chris O'Brien said...

@Arash,

It's been a while now since I looked at this project - I haven't had the need to use it with SP2010 (so far), so I haven't created a SP2010-specific VS project.

I would try the VS upgrade wizard - I wouldn't particularly expect any issues to come out of that, and it's what I'd do if I needed to use the Language Store right now.

Thanks,

Chris.