Elasticon London 2017 (for a Sitecore developer)

Even though Elasticsearch is built on the same foundations (Apache Lucene) as Solr, we in the Sitecore community don’t see a lot of cases where Solr or Coveo have been replaced by Elasticsearch as the main search component.

Today I’ve been at Elasticon London 2017 and have been soaking up new product releases by Elastic. Here’s some notes I scribbled and points which would be interesting to the Sitecore world.

Elasticsearch 6.0

  • Elasticsearch 6.0 is the next major release – but no firm release date has been set just yet. The official answer right now is “coming soon”.
  • Elastic have recognised how painful the upgrade process is – particularly between 2.x and 5.x. When performing the upgrade, you must re-index all of your data to make it 5.x compatible – a huge job on large clusters!
  • While the 2.x to 5.x upgrade path isn’t going to get much easier, Elastic are making sure that the 5.x to 6.x upgrade path doesn’t require you to re-index all data, or take nodes offline.
  • The .NET client creates REST requests with a JSON payload which proxies requests and responses between your code and the Elasticsearch cluster. This is the client model they’re sticking with, and are actually rewriting the native Java client (which currently doesn’t generate REST requests) to be more in line with the .NET client.
  • When it comes to swapping out Solr and Coveo for Elasticsearch, I think this would be a very individual decision based on the needs of your project.

IMG_3921

IMG_3925

 

Anomaly Detection

  • You can now create and set off unsupervised machine learning jobs to continually parse any data and highlight anomalies.
  • The engineer I spoke to said usually “around three weeks” of learning will be enough to begin pulling out anomalies.
  • This could have applications ranging from security (detecting usual IP or DNS activity), to marketing: spotting if a ‘suspicious’ user journey is taking place (whatever this may be!) or perhaps highlighting if a user is stuck or lost.
  • The Elastic stack uses Beats – agents which can monitor a set of files, database, or network packets and streams the data into an Elasticsearch instance / cluster.

IMG_3915

 

Opcode

  • Elastic founder and CEO Shay Banon announced during his keynote that Elastic have acquired Opbeat, a Copenhagen based company whose product adds monitoring and profiling to JavaScript applications (think Node.js, React and Angular).
  • While this might have limited applicability for a lot of Sitecore solutions (where React and Angular might not be the norm), the interesting thing here will be to wait and see how Elastic fit Opbeat into their stack. My guess would be that they’ll extend the product and make it a more general-purpose monitoring and profiling tool.

IMG_3923

 

Machine Learning

  • Elastic’s Steve Dodson (who heads up the Machine Learning product) showed us the current offering, which again is centred around anomaly detection.
  • Most of Elastic’s demo use-cases for anomaly detection are for ops-level indicators like 404’s, 500s, response time, DDOS detection, and so on. Machine Learning kicks in to ignore ‘regular’ surges such as an increase in page response time during weekly batch jobs – but still alerting if the surge is stronger than usual.
  • With some fiddling, you could set up an anomaly detection profiler which tolerates a certain amount of server errors after a code release (and assumes you’ll fix them), but alerts you if it looks like you’ve broken something really big.
  • There was a preview of forecasting, a feature of the upcoming Elasticsearch 6.0. Forecasting does exactly as you’d expect – look at historical data and predict statistics for a future window.

IMG_3931

 

Elasticsearch SQL

Being an ex-database nerd, I loved this session. I’m not sure how the wider search community are going to feel about writing SQL again, but here’s what Elastic have in development:

  • A SQL-like DSL which is 50% ANSI SQL and 50% Elasticsearch-specific syntax additions.
  • You can run queries like SHOW TABLES to list all indexes, DESCRIBE my_index to show fields (columns) and datatypes. You can run search queries like this:
SELECT * FROM my_index WHERE QUERY('+chris perks')
  • All SQL queries translate into the same old Elasticsearch QueryDSL
  • Other constructs they’re including are GROUP BY, HAVING, even JOINs are in there – all translating to their equivalent Elasticsearch QueryDSL commands.
  • You can even wrap SQL in JSON and use it via a REST call *confused face emoji*
  • This is still heavily in development and won’t be released for a while.

 

Summary

There’s plenty of overlap with what the Elastic stack offers, and what you’ll already have set up with Sitecore, Solr, and xDB. There’ll be a fair amount of plumbing work to get Elasticsearch set up properly with Sitecore, whereas you get this out of the box for Solr and/or Coveo.

As Elastic expand, they’re adding many new tools and capabilities to their stack, so it’s definitely not correct to see Elasticsearch as a like-for-like replacement for Solr or Coveo or parts of xDB.

I can see use cases for engaging Elasticsearch alongside your current setup, if you either need a particular Elastic capability which Sitecore / Solr doesn’t give you (such as Machine Learning, or streaming data). Or, you have more faith in the scaling capability of Elasticsearch than you do Sitecore and Solr.

 

Visualising Sitecore Analyzers

When Sitecore indexes your content, Lucene analyzers work to break down your text into a series of individual tokens. For instance, a simple analyzer might convert input text to lowercase, split into separate words, and remove punctuation:

  • input: Hi there! My name is Chris.
  • output tokens: “hi”, “there”, “my”, “name”, “is”, “chris”

While this happens behind the scenes, and is usually not of too much interest outside of diagnostics or curiosity, there’s a way we can view the output of the analyzers bundled with Sitecore.

Let’s get some input text to analyze, in both English and French:

var text = "Did the Quick Brown Fox jump over the Lazy Dog?";
var text_fr = "Le Fox brune rapide a-t-il sauté sur le chien paresseux?";

Next, let’s write a generic method which takes some text and a Lucene analyzer, and runs the text through the analyzer:

private static void displayTokens(Analyzer analyzer, string text)
{
    var stream = analyzer.TokenStream("content", new StringReader(text));
    var term = stream.AddAttribute();
    while (stream.IncrementToken())
    {
      Console.Write("'" + term.Term + "', ");
    }
}

Now, let’s try this out on some Sitecore analyzers!

  • CaseSensitiveStandardAnalyzer retains case, but removes punctuation and stop words (common words which offer no real value when searching)
displayTokens(new CaseSensitiveStandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30), text);
> 'Did', 'Quick', 'Brown', 'Fox', 'jump', 'over', 'Lazy', 'Dog'
  • LowerCaseKeywordAnalyzer convers the input to lowercase, but retains the punctuation and doesn’t split the input into separate words.
displayTokens(new LowerCaseKeywordAnalyzer(), text);
> 'did the quick brown fox jump over the lazy dog?
  • NGramAnalyzer breaks text up into trigrams which are useful for autocomplete. See more here.
displayTokens(new NGramAnalyzer(), text);
> 'did_the_quick', 'the_quick_brown', 'quick_brown_fox', 'brown_fox_jump', 'fox_jump_over', 'jump_over_the', 'over_the_lazy', 'the_lazy_dog
  • StandardAnalyzerWithStemming introduces stemming, which finds a common root for similar words (lazy, lazily, laze -> lazi)
displayTokens(new StandardAnalyzerWithStemming(Lucene.Net.Util.Version.LUCENE_30), text);
> 'Did', 'the', 'Quick', 'Brown', 'Fox', 'jump', 'over', 'the', 'Lazi', 'Dog'
displayTokens(new SynonymAnalyzer(new XmlSynonymEngine("synonyms.xml")), text);
> 'did', 'quick', 'fast', 'rapid', 'brown', 'fox', 'jump', 'over', 'lazy', 'dog
  • Lastly, we try a FrenchAnalyzer. Stop words are language specific, and so the community often contributes analyzers which will remove stop words in languages other than English. In the example below, we remove common French words.
displayTokens(new FrenchAnalyzer(Lucene.Net.Util.Version.LUCENE_30), text_fr);
> 'le', 'fox', 'brun', 'rapid', 't', 'saut', 'chien', 'pares'

The full code is here: (https://gist.github.com/christofur/e2ea406c21bccd3b032c9b861df0749b)


using Lucene.Net.Analysis;
using Lucene.Net.Analysis.Fr;
using Lucene.Net.Analysis.Tokenattributes;
using Sitecore.ContentSearch.LuceneProvider.Analyzers;
using System;
using System.IO;
namespace SitecoreAnalyzers
{
class Program
{
static void Main(string[] args)
{
var text = "Did the Quick Brown Fox jump over the Lazy Dog?";
var text_fr = "Le Fox brune rapide a-t-il sauté sur le chien paresseux?";
Console.WriteLine("** CaseSensitiveStandardAnalyzer **");
displayTokens(new CaseSensitiveStandardAnalyzer(Lucene.Net.Util.Version.LUCENE_30), text);
Console.WriteLine("** LowerCaseKeywordAnalyzer **");
displayTokens(new LowerCaseKeywordAnalyzer(), text);
Console.WriteLine("** NGramAnalyzer **");
displayTokens(new NGramAnalyzer(), text);
Console.WriteLine("** StandardAnalyzerWithStemming **");
displayTokens(new StandardAnalyzerWithStemming(Lucene.Net.Util.Version.LUCENE_30), text);
Console.WriteLine("** SynonymAnalyzer – see http://firebreaksice.com/sitecore-synonym-search-with-lucene/ **");
displayTokens(new SynonymAnalyzer(new XmlSynonymEngine("synonyms.xml")), text);
Console.WriteLine("** FrenchAnalyzer **");
displayTokens(new FrenchAnalyzer(Lucene.Net.Util.Version.LUCENE_30), text_fr);
Console.ReadKey();
}
private static void displayTokens(Analyzer analyzer, string text)
{
var stream = analyzer.TokenStream("content", new StringReader(text));
var term = stream.AddAttribute<ITermAttribute>();
while (stream.IncrementToken())
{
Console.Write("'" + term.Term + "', ");
}
Console.Write("\n\n");
}
}
}

Explaining Lucene explain

Each time you perform a search using Lucene, a score is applied to the results returned by your query.

--------------------------------------
| #   | Score | Tags                 |
--------------------------------------
| 343 | 2.319 | Movies, Action       |
| 201 | 2.011 | Music, Classical     |
| 454 | 1.424 | Movies, Kids         |
| 012 | 0.003 | Music, Kids          |
 --------------------------------------

In our index, # is the unique document number, score is the the closeness of each hit to our query, and tags is a text field belonging to a document.

There are many methods Lucene can use to calculate scoring. By default, we use the DefaultSimilarity implementation of the Similarity abstract class. This class implements the commonly referenced TfIdf scoring formula:


(more: https://lucene.apache.org/core/3_0_3/api/core/org/apache/lucene/search/Similarity.html)

If you’re new to Lucene (or even if you’re not!) this formula can be a bit to get your head around. To get inside the formula for a given search result, Lucene provides an explanation feature, which we can call from code (c# example in Lucene.Net):

public List GetExplainerByRawQuery(string rawQuery, int doc = 0)
{
    using (var searcher = new IndexSearcher(_directory, false))
    {
        // Create a parser, and parse a plain-text query which searches for items tagged with 'movies' or 'kids' (or hopefully, both)
        var parser = new QueryParser(Lucene.Net.Util.Version.LUCENE_30, "id", analyzer);
        var query = parser.Parse("tags:(movies OR kids)");

        // Get references to the top 25 results
        var hits = searcher.Search(query, 25).ScoreDocs;

        // For each hit, get the accompanying explanation plan. We now have a List
        var explains = hits.Select((x, i) => searcher.Explain(query, i)).ToList();

        //Clean up and return
        analyzer.Close();
        searcher.Dispose();
        return explains;
    }
}

Calling searcher.Explain(query, match.doc) gives us a text output explanation of how the matched document scores against the query:

query: tags:movies|kids
----------------------------------------------------
| #   | Score  | Tags                              |
----------------------------------------------------
| 127 | 2.4824 | Movies, Kids, Animation, Movies   |
----------------------------------------------------
2.4824  sum of:
  1.4570  weight(tags:movies in 127) [DefaultSimilarity], result of:
    1.4570  score(doc=127,freq=2.0 = termFreq=2.0), product of:
      0.7079  queryWeight, product of:
        2.9105  idf(docFreq=147, maxDocs=1000)
        0.2432  queryNorm
      2.0581  fieldWeight in 127, product of:
        1.4142  tf(freq=2.0), with freq of:
          2.0000  termFreq=2.0
        2.9105  idf(docFreq=147, maxDocs=1000)
        0.5000  fieldNorm(doc=127)
  1.0255  weight(tags:kids in 127) [DefaultSimilarity], result of:
    1.0255  score(doc=127,freq=1.0 = termFreq=1.0), product of:
      0.7063  queryWeight, product of:
        2.9038  idf(docFreq=148, maxDocs=1000)
        0.2432  queryNorm
      1.4519  fieldWeight in 127, product of:
        1.0000  tf(freq=1.0), with freq of:
          1.0000  termFreq=1.0
        2.9038  idf(docFreq=148, maxDocs=1000)
        0.5000  fieldNorm(doc=127)

Ok! But still, there’s a lot going on in there. Let’s try and break it down.

  • 2.4824 is the total score for this single search result. As our query contained two terms, ‘movies’ and ‘kids’, Lucene breaks the overall query down into two subqueries.
  • The sum of the two subqueries (1.4570 for ‘movies’ and 1.0255 for ‘kids’) are added to arrive at our total score.

For our first subquery, the ‘movies’ part, we arrive at the score of 1.4570 by multiplying queryWeight (0.709) by fieldWeight (2.0581). Let’s go line by line:

  1.4570  weight(tags:movies in 127) [DefaultSimilarity], result of:
The total score for the ‘movies’ subquery is 1.4570. ‘tags:movies‘ is the raw query, 127 is the individual document number we’re examining, and DefaultSimilarity is the scoring mecahsnism we’re using.
1.4570 score(doc=127,freq=2.0 = termFreq=2.0), product of:
The term (‘movies‘) appears twice in the ‘tags‘ field for document 127, so we get a term frequency of 2.0
0.7079 queryWeight, product of:
queryWeight (0.7079) is how rare the search term is within the whole index – in our case, ‘movies‘ appears in 147 out of the 1000 documents in our index.   This normalization factor is the same for all results returned by our query and just stops the queryWeight scores from becoming too exaggerated for any single result.
2.9105 idf(docFreq=147, maxDocs=1000)
  This rarity is called inverse document frequency (idf)
0.2432 queryNorm
  .. and is itself multiplied by a normalization factor (0.2432) called queryNorm.

This normalization factor is the same for all results returned by our query and just stops the queryWeight scores from becoming too exaggerated for any single result.

2.0581 fieldWeight in 127, product of:
  fieldWeight (2.0581) is how often the search term (‘movies‘) appears in the field we searched on ‘tags’.
1.4142 tf(freq=2.0), with freq of:
  2.0000 termFreq=2.0
  We take the square root of the termFreq (2.0) = 1.4142
2.9105 idf(docFreq=147, maxDocs=1000)
  This is multiplied by the idf which we calculated above (2.9105)
0.5000 fieldNorm(doc=127)
   and finally by a field normalization factor (0.5000), which tells us how many overall terms were in the field.

This ‘boost‘ value will be higher for shorter fields – meaning the more promenant your search term was in a field, the more relevant the result.

Further reading:

Happy Lucene hacking!

Returning JSON errors from Sitecore MVC controllers

ASP.NET MVC gives us IExceptionFilter, with which we can create custom, global exception handlers to apply to controller actions.

public class ExceptionLoggingFilter : FilterAttribute, IExceptionFilter
{
	public void OnException(ExceptionContext filterContext)
	{
		// filterContext now contains lots of information about our exception, controller, action, etc
		filterContext.Exception.Message;
		filterContext.Exception.StackTrace;
		filterContext.Controller.GetType().Name;
		filterContext.Result.GetType().Name;
		UserAgent = filterContext.HttpContext.Request.UserAgent;
	}
}

 

We can apply this filter to all Action methods, by adding our filter to the list of global filters:

public class FilterConfig {
	public static void RegisterGlobalFilters(GlobalFilterCollection filters) {
		filters.Add(new ExceptionLoggingFilter());
	}
}

 

and wiring this up to our application in our Application_Start method:

FilterConfig.RegisterGlobalFilters(GlobalFilters.Filters);

 

In Sitecore

As you may expect, Sitecore exposes this functionality as pipeline processors. Sitecore defined a custom IExceptionFilter implementation (see our snippet above) which kicks off the mvc.exception pipeline, passing along the ExceptionContext object.

As client developers, it is our job to create an appropriate processor to accept the ExceptionContext and do something with it. Let’s run through an example where we want to return a JSON representation of the error, loaded with as much useful information as possible.

For more reading on Sitecore controller actions returning JSON, have a look at John West’s post here: https://community.sitecore.net/technical_blogs/b/sitecorejohn_blog/posts/use-json-and-mvc-to-retrieve-item-data-with-the-sitecore-asp-net-cms

So, first up, create an empty handler class, which inherits from ExceptionProcessor:

public class JSONExceptionHandler :
	Sitecore.Mvc.Pipelines.MvcEvents.Exception.ExceptionProcessor
{
	public override void Process(Sitecore.Mvc.Pipelines.MvcEvents.Exception.ExceptionArgs args)
	{

	}
}

 

Create a Web.config include, to add this processor to the mvc.exception pipeline:

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <pipelines>
      <mvc.exception>
        <processor type="Bleep.Handlers.JSONExceptionHandler, Bleep.Handlers"/>
      </mvc.exception>
    </pipelines>
  </sitecore>
</configuration>

 

Ok! Now our JSONExceptionHandler class will be called each time an exception occurs in MVC code. So, let’s grab all the detail we can from the ExceptionContext class and return it as JSON:

public override void Process(Sitecore.Mvc.Pipelines.MvcEvents.Exception.ExceptionArgs args)
{
	var filterContext = args.ExceptionContext;
 
	filterContext.Result = new JsonResult
	{
		JsonRequestBehavior = JsonRequestBehavior.AllowGet,
                  Data = new
		  {
    			Message = filterContext.Exception.Message,
    			StackTrace = filterContext.Exception.StackTrace,
    			Controller = filterContext.Controller.GetType().Name,
    			Result = filterContext.Result.GetType().Name,
    			UserAgent = filterContext.HttpContext.Request.UserAgent,
    			ItemName = args.PageContext.Item.Name,
    			Device = args.PageContext.Device.DeviceItem.Name,
    			User = filterContext.HttpContext.User.Identity.Name
		  }
	};
 
	filterContext.ExceptionHandled = true;
 
	// Log the error
	Sitecore.Diagnostics.Log.Error("MVC exception processing " 
                	+ Sitecore.Context.RawUrl, args.ExceptionContext.Exception, this);
}

 

This will produce a result such as:

ExceptionFilter2

Happy hacking!

A workaround for missing ViewData in Sitecore MVC

Passing data between Sitecore renderings can get tricky.

Sending messages between sibling renderings can lead us to worry about the order in which they render, and you may end up with renderings tightly coupled to other renderings. Jeremy Davis discusses ways to switch the order of rendering execution on his blog here: https://jermdavis.wordpress.com/2016/04/04/getting-mvc-components-to-communicate/

Share_Data_1

Pass data down, not across

My preferred approach is for renderings to be as isolated as possible and not need to talk to siblings. In a regular MVC site, we would instantiate a ViewModel, and pass it down to any child (or partial) views as needed. If a child view doesn’t change this ViewModel at all, we don’t have to worry about order of execution or changes of state.

In Sitecore, we can achieve this by wrapping child renderings in a parent Controller Rendering. This Controller Rendering creates and prepares the ViewModel, and then passes it down to one or more child renderings.

Share_Data_2

Let’s recap on the main points here:

  1. Our parent Controller Rendering creates and prepares a ViewModel. This parent specifies a view, which contains one or more placeholders.
  2. This ViewModel is passed along to any child renderings currently attached to the placeholders.
  3. During execution, child renderings do not modify the ViewModel. We may even consider the ViewModel immutable while rendering takes place.

Sitecore has a peculiarity here which makes our job difficult. Each rendering gets a new instance of ViewData – explained by Kern Herskind Nightingale here: http://stackoverflow.com/a/35210022/638064. This puts a stop to us using ViewData to pass our ViewModel down from the parent rendering to child renderings.

The Workaround

There’s a way you can ensure that ViewData is correctly passed down from parent to child renderings. Let’s go through how this is possible.

  1. In your top level controller, create a ViewModel, which will be passed down to all child renderings.
public ActionResult ParentContainer()
{
    var viewModel = new {PageSize = 3, CurrentPage = 2, Results = Sitecore.Context.Item.Fields["Results"].Value};
    return View();
}

  1. Add it to the ViewData collection in the current ViewContext
public ActionResult ParentContainer()
{
    var viewModel = new {PageSize = 3, CurrentPage = 2, Results = Sitecore.Context.Item.Fields["Results"].Value};
    ContextService.Get().GetCurrent().ViewData.Add("_SharedModel", viewModel);
    return View();
}
  1. In each child rendering, fetch the ViewModel and add it to the local ViewData for the current rendering (which will be empty at this point). View Renderings will do this step for you, so you don’t need to do anything special there
public ActionResult ChildRendering()
{
    // Get any ViewData previously added to this ViewContext
    var contextViewData = ContextService.Get().GetCurrent().ViewData;
    contextViewData.ToList().ForEach(x => ViewData.Add(x.Key, x.Value));
    return View();
}
  1. Et voila! You now have access to the same ViewModel for each of your child renderings.
@{
    Layout = null;
    var viewModel = ViewData["_SharedModel"];
}

Making it better

MVC offers us even better tools to remove code duplication. If you have a lot of child renderings needing access to your shared ViewModel, adding the code in step 3 will happen a lot. Let’s refactor that to an filter attribute.

public class RetrieveViewDataFilter : ActionFilterAttribute, IActionFilter
{
    public void OnActionExecuting(ActionExecutingContext filterContext)
    {
        //Merge ViewData from context
        var contextViewData = ContextService.Get().GetCurrent().ViewData;
        contextViewData.ToList().ForEach(x => filterContext.Controller.ViewData.Add(x.Key, x.Value));
    }

    public void OnActionExecuted(ActionExecutedContext filterContext)
    {
    }
}

Now, we just need to add this attribute to any Action Methods who may want to access shared ViewData from higher up in the stack

[RetrieveViewDataFilter]
public ActionResult ChildRendering()
{
    return View();
}

There we go. I’m sure Sitecore will amend their implementation at some point, but until then, we have an immutable, single direction ViewData flow.

Finding the current Action name from an MVC pipeline processor

Sitecore provides two pipeline hooks for tapping into an Action method at point of execution:

  • mvc.actionExecuting
  • mvc.actionExecuted

These follow standard MVC naming conventions – actionExecuting fires before your Action method executes, and actionExecuted fires afterwards.

A process hooking into actionExecuting looks something like this:

public class LogActionExecuting
{
    public void Process(ActionExecutingArgs args)
    {
        //Something here
    }
}

At this point, it might be useful to get some information about the Action we’re executing, or the Controller it belongs to. Sitecore allows us to access the MVC ActionDescriptor and ControllerDescriptor objects, which contain plenty of information about our Action and Controller.

public class LogActionExecuting
{
    public void Process(ActionExecutingArgs args)
    {
        //Some interesting items from the Action
        var actionName = args.Context.ActionDescriptor.ActionName;
        var actionAttributes = args.Context.ActionDescriptor.GetCustomAttributes(false);
           
        //Some interesting items from the Controller
        var controllerName = args.Context.ActionDescriptor.ControllerDescriptor.ControllerName;
        var controllerType = args.Context.ActionDescriptor.ControllerDescriptor.ControllerType;
        var controllerActions = args.Context.ActionDescriptor.ControllerDescriptor.GetCanonicalActions();
 
        //args.Context.ActionDescriptor.Execute(..);
 
    }
}

The last line is commented out, as executing the action from within itself may cause the universe to implode. Maybe.

Happy hacking!

Error: Could not find method: Process. Pipeline: /sitecore[database=”SqlServer” xmlns:patch=”http://www.sitecore.net/xmlconfig/”]/pipelines/mvc.resultExecuting[patch:source=”Sitecore.Mvc.config”]

When adding custom pipeline processors, you may come across the error:

Could not find method: Process. Pipeline: /sitecore[database="SqlServer" xmlns:patch="http://www.sitecore.net/xmlconfig/"]/pipelines/mvc.resultExecuting[patch:source="Sitecore.Mvc.config"]/processor[type="SitecoreCustom.Feature.Logging.LifecycleLogger.Pipelines.ActionExecuting.LogActionExecuting, SitecoreCustom.Feature.Logging.LifecycleLogger" patch:source="Feature.LifecycleLogger.config"]
Description: An unhandled exception occurred.

There are two usual culprits to check.

  1. Did you reference the correct class and assembly, in your pipeline configuration? A typo here will mean that the processor cannot be found.
  2. Are you sending in the correct typed arguments? Each pipeline requires a specific typed argument object, such as InitializeContainerArgs. If I expect the wrong args type for my chosen pipeline, Sitecore will consider the Process method invalid and will skip it.

 

These are both really easy mistakes to make, and hard to spot where you might be going wrong, as everything will compile fine.

Patching the Sitecore Commands.config file

A quick tip Mark Cassidy helped me with – patching the Commands.config is something you may need to do when adding custom Sitecore ribbon buttons.

The contents of Commands.config get wrapped into Web.config when Sitecore initializes, so actually, we can patch Commands.config just like we would Web.config. The element you’re aiming for is sitecore/commands/command.

<configuration xmlns:patch="http://www.sitecore.net/xmlconfig/">
  <sitecore>
    <commands>
      <command name="item:clone">
        <patch:attribute name="type">ChrisPerks.Features.Ticker.Commands.CustomButton, ChrisPerks.Features.Ticker</patch:attribute>
      </command>
    </commands>
  </sitecore>
</configuration>

And et voila! Your Commands.config is patched.

Sitecore MVC Internals: IControllerActivator

When we previously looked into IControllerFactory implementations in both Sitecore MVC and standard ASP.NET MVC, we noticed that Microsoft’s default IControllerFactory implementation, DefaultControllerFactory, doesn’t actually handle the instantiation of new IController instances. To enable dependency injection at the point of object creation, DefaultControllerFactory hands off instation of IController objects to another player – an object which implements IControllerActivator.

How we call an IControllerActivator implementation

Let’s look back at the code for the GetControllerInstance method of Microsoft’s DefaultControllerFactory class (see previous post for a more in-depth discussion)

protected internal virtual IController GetControllerInstance(RequestContext requestContext, Type controllerType)
{
    //...
    return this.ControllerActivator.Create(requestContext, controllerType);
}

This method crucially takes in the Type of a controller, which we have determined by looking carefully at the requested URL and matching it to a known route. We see that DefaultControllerFactory already has an implementation of IControllerActivator to work with, referenced by this.ControllerActivator.

What is Microsoft’s default implementation?

In standard ASP.NET MVC sites, this implementation is Microsoft’s DefaultControllerActivator. Let’s look at the DefaultControllerActivator.Create() method in full:

public IController Create(RequestContext requestContext, Type controllerType)
{
    try{
            return (IController) (this._resolverThunk().GetService(controllerType) ?? Activator.CreateInstance(controllerType));
    }
    catch (Exception ex)
    {
            throw new InvalidOperationException(string.Format((IFormatProvider) CultureInfo.CurrentCulture, MvcResources.DefaultControllerFactory_ErrorCreatingController, new object[1]{(object) controllerType}), ex);
    }
}

Ignoring boilerplate, the line we’re left with is:

(IController) (this._resolverThunk().GetService(controllerType) ?? Activator.CreateInstance(controllerType));

Don’t be discouraged by the odd looking _resolverThunk(). This is a reference to the chosen Dependency Resolver for your project. Remembering that ASP.NET MVC was built for extendability at every point, a Dependency Resolver lets us have full control over how any object is created – typically using a Dependency Injection framework. We’ll look at Dependency Resolvers in a future post.

Back to the line above. If you have a Dependency Resolver in place, the GetService() method will use this resolver to return an instance of the IController you need to fulfil the request. If not, we fall back to good old Activator.CreateInstance(), which is the .NET frameworks vanilla way of creating new objects at runtime.

Does Sitecore implement IControllerActivator?

No, Sitecore doesn’t utilist IControllerActivator. There’s a simple reason – once Sitecore has determined the type of Controller you would like to create, it hands off the job of creating the controller to the mvc.createController Pipeline:

IController controller = PipelineService.Get().RunPipeline<CreateControllerArgs, IController>(“mvc.createController”, 
        new CreateControllerArgs(requestContext, controllerName), 
        (Func<CreateControllerArgs, IController>) (args => args.Result));

That’s it for IControllerActivator, happy hacking!

Sitecore MVC internals: SitecoreControllerFactory

In this post, the first in a series of posts looking into the internals of Sitecore MVC, we’ll be looking at the SitecoreControllerFactory class in detail.

What does SitecoreControllerFactory do?

This class, provided by Sitecore in the Sitecore.Mvc.Controllers namespace, is responsible for creating an instance of an MVC controller, used by Sitecore for turning a HTTP request (for say, /products/12345) into a HTML response, rendered by the browser.

The concept of a Controller Factory is not specific to Sitecore, and is one of the mechanics of the wider ASP.NET MVC framework. Let’s look at the signature for this class.

public class SitecoreControllerFactory : IControllerFactory

Here we’re seeing a Sitecore class, SitecoreControllerFactory, implement an interface, IControllerFactory, which belongs to the wider ASP.NET MVC framework. IControllerFactory defines one method we’re interested in here:

public interface IControllerFactory
{
    IController CreateController(RequestContext requestContext, string controllerName);
    //..others
}

This means that SitecoreControllerFactory must implement a method called CreateController. Similarly, any non-Sitecore sites you build using ASP.NET MVC will likely use the built-in DefaultControllerFactory class that Microsoft provide for you. It’s happening behind the scenes, and unless you needed to do some heavy customisation to your project, you may not have noticed. Before we get back to Sitecore, let’s look at Microsoft’s implementation of CreateController, in their vanilla IControllerFactory implementation, DefaultControllerFactory:

public class DefaultControllerFactory : IControllerFactory
{
    //.. snipped
    public virtual IController CreateController(RequestContext requestContext, string controllerName)
    {
      //.. snipped
      Type controllerType = this.GetControllerType(requestContext, controllerName);
      return this.GetControllerInstance(requestContext, controllerType); 
    }
}

So, Microsoft’s default CreateController implementation returns a Controller object. How does this method know which Controller you want? Well, we break it up into two steps. First, we pass a controller name (such as ‘Products’) to the method. This name is taken from the route data extracted from our request to /products/12345. This controller name is passed to the GetControllerType method, which finds the appropriate Controller Type.

Secondly, we pass this Type to another method, GetControllerInstance, which handles the actual instantiation of the new Controller object, which we’ll eventually return. GetControllerInstance makes use of another MVC feature called Activators, which we’ll handle in another blog post.

So why do we need all of these steps, just to instantiate a Controller object?

ASP.NET MVC was built with a few design goals in mind – mainly testability and extensibility. One way of achieving these goals was to include wide support for dependency injection. In short – each of these steps can be replaced with our implementation, should we need to. And handily, that brings us back to Sitecore.

Back to Sitecore

Now we know that ASP.NET MVC makes it easy for us to replace any of the default behaviour for our own, bespoke implementations, we can look at one instance of where Sitecore have done exactly that. Sitecore’s SitecoreControllerFactory class implements IControllerFactory, and is used in place of Microsoft’s DefaultControllerFactory.

Let’s look at SitecoreControllerFactory’s CreateController method:

public virtual IController CreateController(RequestContext requestContext, string controllerName)
{
    //snipped
    IController controllerInstance = this.CreateControllerInstance(requestContext, controllerName);
    this.PrepareController(controllerInstance, controllerName);
    return controllerInstance;
}

This method doesn’t do much more than call two other methods – CreateControllerInstance and PrepareController. The main work we’re interested in here is in CreateControllerInstance, so let’s follow along that thread:

protected virtual IController CreateControllerInstance(RequestContext requestContext, string controllerName)
{
    if (controllerName.EqualsText(this.SitecoreControllerName))
    return this.CreateSitecoreController(requestContext, controllerName);

    if (TypeHelper.LooksLikeTypeName(controllerName))
    {
        Type type = TypeHelper.GetType(controllerName);
        if (type != (Type)null)
            return TypeHelper.CreateObject(type);
        }

    return this.InnerFactory.CreateController(requestContext, controllerName);
}

So, there are three ways in which Sitecore can choose to instantiate a Controller! This seems a little more involved than Microsoft’s default implementation, but this is Sitecore, so that seems reasonable. Let’s follow down the path of CreateSitecoreController.

protected virtual IController CreateSitecoreController(RequestContext requestContext, string controllerName)
{
    IController controller = PipelineService.Get().RunPipeline("mvc.createController", 
        new CreateControllerArgs(requestContext, controllerName), 
        (Func)(args => args.Result));
}

Ok! We’re now back firmly in Sitecore territory. Here’s where we branch from the way vanilla ASP.NET MVC does things, and how Sitecore handles things. Just like in standard ASP.NET MVC, we have a request context and a Controller name. However, instead of handing it off to GetControllerInstance as we did in DefaultControllerFactory, we call upon a Sitecore Pipeline, called mvc.createController.

This pipeline handles the instantiation of Controller objects by pulling some relevant details from Sitecore:

string controllerName = ((BaseItem) item).get_Item("__Controller");

string actionName = ((BaseItem) item).get_Item("__Controller Action");

Finally, we know which Controller should be used for the Sitecore item being requested (remember? /Products/12345). To handle the actual instantiation of the item, Sitecore gives us two ways to do this, both called from the mvc.createController pipeline:

IController CreateControllerUsingFactory()

IController CreateControllerUsingReflection()

The difference? Well, we can actually invoke another IControllerFactory instance to do the creation work for us. Think of this as Sitecore playing nicely – the SitecoreControllerFactory does only what it needs to, retrieving the details of which controller we need; it then relinquishes control of the actual instantiation of that controller to any other IControllerFactory you want to use. Sitecore wraps this detail up in a ControllerSpecification, however we’ll leave it there for now. This is something we’ll address in another post.

If no ControllerSpecification is given, then we can fall back to good old Reflection to create our instance.

I hope you’ve enjoyed this peek into the workings of Sitecore MVC. There’s plenty more to be discovered, so happy digging!