Search This Blog

2011-11-21

Controlling Ajax requests flow

While developing rich internet applications which contain a lot of Ajax requests, sooner or later you come to the idea of controlling requests flow. Users are continuously interacting with site(s) GUI that causes massive Ajax requests which are passed through load balances, firewalls, virtual servers etc. Problem might happen on any layer - from dead lock in database to exceptions in firewall; at the same time, user does not know and care about all that complex issues on the background. 
There are several ways to manage Ajax requests flow and I will show on of them: controlling request queue with JavaScript and jQuery.
The idea is not to bomb overloaded server with tons of requests. Instead, we assume if requests are processed to long - there is some issue on the server side and we should react somehow. With decribed approach we are organizing the queue of requests, but quick ones can still be processed simultaneously.
First, let's synthesize idea with tests. Let's imagine we have some web service that processes one request per two seconds. Our web client is continuously sending requests to the service.
module("Test.RequestController.js");

asyncTest("Test: sending requests", 6, function () {
  var url = "/TestWebService";
  var data = { "testData": "data" };

  var controller = new RequestController();
  controller.ajax({'url':url,'data':data}).done(function () {
    ok("done1");
  });
  controller.ajax({'url':url,'data':data}).done(function () {
    ok("done2");
  });
  setTimeout(function () {
    controller.ajax({'url':url,'data':data}).done(function () {
      ok("done3");
    });
    controller.ajax({'url':url,'data':data}).done(function () {
      ok("done4");
    });
  }, 1000);
  setTimeout(function () {
    controller.ajax({'url':url,'data':data}).done(function () {
      ok("done5");
    });
    controller.ajax({'url':url,'data':data}).done(function () {
      ok("done6");
      start();
    });
  }, 2000);
});
Our test service has two seconds delay. But, let's say after tree seconds delay, user should be warned about the problem and/or ajax requests should be redirected. We can wrap jQuery.ajax function with our Request controller which encapsulates required logic.
First, let's define constructor with preset data.
RequestController = function () {
  this.requestCounter = 0; // Number of too long requests
  this.timeout = 3000; // Default timeout for "long" requests - 3 sec
  this.maxLongRequests = 3; // Max amount of simultaneous long requests 

  this.requestPromise = jQuery(this).promise();
};
Then we should wrap jQuery ajax with our queue:
RequestController.prototype.ajax = function(ajaxParams) {
  var ajaxRequest = function() {
    var isLongRequest = false;

    var ajaxPromise = jQuery.ajax(ajaxParams).always(function() {
      if (isLongRequest) {
        this.requestCounter--;
      }
    }.bind(this));

    setTimeout(function() {
      if (ajaxPromise.state() == "pending") {
        this.requestCounter++;
        isLongRequest = true;
      }
    }.bind(this), this.timeout);

    return ajaxPromise;
  }.bind(this);


  if (this.requestCounter >= this.maxLongRequests) {
    // show warning to the user
    // and warn system admin or redirect request
    return this.requestPromise = this.requestPromise.pipe(ajaxRequest);
  } else {
    return this.requestPromise = ajaxRequest();
  }
};
The idea is simple enough: we have a wrapper function that calls jQuery.ajax; on request complete/fail we check: if request was too "long" we just decrease the pending requests counter. On setTimeout function we are checking whether request is finished; if not - request can be considered as long: we increase the counter and switch the flag of long request.
Finally, we check whether flow reached maximum allowed number of long requests: if not - just return request promise to the caller. If it reached - we can send a warning and put request to the queue described earlier.
The request flow look like this:


2011-11-17

Slavic mentality: laugh or cry


I would like to share an incredible story I have observed in my life. 
The story took place in the early 90s, during the high point of turmoil in Ukraine, and at that time I was studying in the elementary school. 
I lived in a long block of flats, together with my grandparents. Despite the large number of tenants, the people were generally polite and friendly except for some... strange neighbors. As you might guess, they are the heroes of the story.
My neighbors from the eighth floor were common workers on a confectionery. Those were difficult times so people were often paid in barter. Neighbors were lucky and received a lot of candies for their job and that is why I knew them :-). 
In order to protect the property they also had a German shepherd called Nayda. Nayda was a good and kind dog, but to the great misfortune, by breaking all rules, it was fed with chocolates earned by her hosts... Nayda was really a fat dog and weighed over 60 kg because of such a diet. Nayda lived 6 years of a sweet life. 
It was quiet and warm spring holiday, the grass was green, the birds were singing and people enjoyed with a nice weather. Nayda died that day. It was so sad... and neighbors laid it to rest according to our customs, accompanying the ritual with drinking vodka...
After the third bottle a “brilliant” idea was born in their heads - to bury the dog on our backyard. The problem is that the dog was heavy and the block house was so long that they would have to drag the body of about 400 meters to get around it. Their next brilliant idea was: "Well, Nayda is already dead so we can throw the body off the balcony and gravity will do everything for us!" Damn, how did they come up with this?!
Without any hesitations, they dragged the body onto the balcony and threw it through a window... Then they just got dressed, took a shovel and went out to dig a grave; but when they got to the point they discovered there was no dog! It just disappeared! So they started looking for a dog in the yard, swearing at the same time.
The real reason they realized a little later. 
Some balconies in apartment houses were equipped with special rails with stretched ropes that stick out two feet from the outside balcony; they are designed for drying clothes.
It turned out that the dog's body collided with ropes in flight in front of the third floor and jumped inside into the open window... OMG
The neighbor from the third floor was at home that day and he heard some strange noise, so he decided to find out the reason. When he opened the door of the balcony - he was stunned by what he saw...
When he collected his thoughts together and looked out of the window - he saw drunken neighbors, who were calling for their dead dog. 
After long abuses and battering, other neighbors called the police to investigate the situation. When police arrived, the house turned into a loony bin: people behaved inappropriately and evidences were incredible. However, finally, experienced police officers of the Soviet hardening dragged the body of the dog in the yard and buried. Drunken neighbors had been fined for disturbing public order, but the moral injury of the neighbors on the third floor has not healed to this day.

2011-10-31

jQuery Deferred queue

Since jQuery 1.5 had been released some very good feature called Deferred Object appeared for better callbacks handling and other kinds of synchronizing calls.
Deferred API has "jQuery.when()" which allows to multiplex deferreds (waiting on multiple deferreds at the same time)... and has "pipe()" to create a pipeline or chain of deferreds (since version 1.6). While this is very nice if all deferreds are available at the same point, it doesn't really convenient if there is a collections of deferreds coming from multiple sources (which may not be aware of one another). This plugin allows to solve the problem at some point. But it was based on previous version... before pipe() feature.
My simple solution also tries to handle multiplexing deferreds from different sources using shared Queue object.
Idea is to have a queue that is based on the principle of FIFO multiple-producers multiple-consumers tasks queues.
Let's start from unit test to describe how it works:
module("Test.Queue.js");

asyncTest("Test of appending to the queue ", 6, function () {
  var queue = new Queue();
  queue.append(function () {
   ok("func 1");

   setTimeout(function () {
      ok("func 1 nested");
      start(); // Continue async test. 
      this.resolve(1); // return some value for the subscribers.
    }.bind(this), 500); // Here I would like to have a control over Deferred through 'this'.
  })
  .done(function (arg) { // on done callback
    ok("func 1 done callback");
    equal(arg, 1);
  });

  queue.append(function (arg) {
    equal(arg, "test arg data");
    this.reject(); // operation was failed.
  }, "test arg data")
  .fail(function (funcArg) { // on fail callback
    ok("func 2 fail callback");
  });
});
So here we have a queue object and two calls of independent functions that might come from the multiple sources. Each function itself might have a callbacks or chain so "queue.append" should return a promise.
The assert trace should look like this: "func 1", "func 1 nested", "func 1 done callback", "func 2" and "func 2 fail callback".
And the code itself:
Queue = function () {
  this.promise = jQuery(this).promise(); // Stub promise
};

// Magic with arguments is needed to pass arguments to the called function
Queue.prototype.append = function () {
  var args = arguments;

  var fn = args[0];
  if (!fn || !jQuery.isFunction(fn)) {
    throw new TypeError('1st parameter should be a function');
  }

  var self = this;
  args = Array.prototype.slice.call(args, 1);
  return this.promise = this.promise.pipe(function () {
    return jQuery.Deferred(function () {
      try {
        return fn.apply(this, args);
      } catch (ex) {
        // log exception
        this.reject(ex);
        return self.promise = jQuery(self).promise();
      }
    }).promise();
  });
};
Implementation idea is very simple: Class just has a reference to the head of a queue and push new function to the Deferred pipeline.

2011-10-05

log4net appender for Raven DB

Recently I have started to look at noSQL databases and Document-Oriented DBs particularly for the .NET platform to solve some issues within our solution. One of the brilliant examples of such databases is Raven DB by Ayende. IMHO, one of the benefits of Raven DB is easy to start way. It has almost all I need out of the box: simple deploy, native .NET client with Linq, Map/Reduces indexes, RESTful API and so on and so forth.
Raven DB itself uses NLog by default and our solution uses log4net like thousands of other ones. So I decided to experiment with custom log4net appender for our needs because one of the first steps of moving to noSQL direction might be logging, due to simple implementation. Such an approach is far more flexible than logging to text files and might significantly reduce load on SQL servers in case of using ADO.NET appender.
So let's start.
First we need to decide what we are going to store. Since logging event  is simple and independent document  we can just wrap LoggingEvent.
// The log entry document entity that will be stored to the database.
public class Log : INamedDocument
{
  ...

  public Log(LoggingEvent logEvent)
  {
    if (logEvent == null)
    {
    throw new ArgumentNullException("logEvent");
    }

    this.LoggerName = logEvent.LoggerName;
    this.Domain = logEvent.Domain;
    this.Identity = logEvent.Identity;
    this.ThreadName = logEvent.ThreadName;
    this.UserName = logEvent.UserName;
    this.MessageObject = logEvent.MessageObject;
    this.TimeStamp = logEvent.TimeStamp;
    this.Exception = logEvent.ExceptionObject;
    this.Message = logEvent.RenderedMessage;
    this.Fix = logEvent.Fix.ToString();

    ...

    // Raven doesn't serialize unknown types like log4net's PropertiesDictionary
    this.Properties = logEvent.Properties.GetKeys().ToDictionary(key => key, key => logEvent.Properties[key].ToString());
  }

  public string LoggerName { get; set; }
  public string Domain { get; set; }
  public string Identity { get; set; }
  public string ThreadName { get; set; }
  public string UserName { get; set; }
  public object MessageObject { get; set; }
  public DateTime TimeStamp { get; set; }
  public object Exception { get; set; }
  public string Message { get; set; }
  public string Fix { get; set; }
  public IDictionary Properties { get; set; }
  ....
}
Next step will be choosing which appender template we need and which additional settings Raven requires. I think using of buffering appender makes sense in order not to drive database connection up the wall. As for additional setting of appender we might need connection string (or connection string name) and Raven's settings about max number of request per document session.
So our ".config" file would have something like this:


  
Ok, we have all the prerequisites and we can write some code.
Implementation is simple: on SendBuffer method it adds log events to a document session and saves changes to a DB server. On ActivateOptions it will initialize DB connection, i.e. create document store instance.
  public class RavenAppender : BufferingAppenderSkeleton
  {
    private readonly object lockObject = new object();

    private IDocumentStore documentStore;

    private IDocumentSession documentSession;

    #region Appender configuration properties

    /// 
    /// Gets or sets the connection string.
    /// 
    /// 
    /// The connection string.
    /// 
    public string ConnectionString { get; set; }

    /// 
    /// Gets or sets the max number of requests per session.
    /// By default the number of remote calls to the server per session is limited to 30.
    /// 
    /// 
    /// The max number of requests per session.
    /// 
    public int MaxNumberOfRequestsPerSession { get; set; }

    #endregion

    /// 
    /// Initializes a new instance of the  class.
    /// 
    public RavenAppender() {}

    /// 
    /// Initializes a new instance of the  class. 
 /// Can be used for unit testing
    /// 
    /// 
The document store.
    public RavenAppender(IDocumentStore documentStore)
    {
      if (documentStore == null)
      {
        throw new ArgumentNullException("documentStore");
      }

      this.documentStore = documentStore;
    }

    protected override void SendBuffer(LoggingEvent[] events)
    {
      if (events == null || !events.Any())
      {
        return;
      }

      this.CheckSession();

      foreach (var entry in events.Where(e => e != null).Select(e => new Log(e)))
      {
        this.documentSession.Store(entry);
      }

      this.Commit();
    }

    public override void ActivateOptions()
    {
      try
      {
        this.InitServer();
      }
      catch (Exception exception)
      {
        ErrorHandler.Error("Exception while initializing Raven Appender", exception, ErrorCode.GenericFailure);
        // throw;
      }
    }

    protected override void OnClose()
    {
      this.Flush();
      this.Commit();

      try
      {
        if (this.documentSession != null)
        {
          this.documentSession.Dispose();
        }

        if (this.documentStore != null && !this.documentStore.WasDisposed)
        {
          this.documentStore.Dispose();
        }
      }
      catch (Exception e)
      {
        ErrorHandler.Error("Exception while closing Raven Appender", e, ErrorCode.GenericFailure);
      }

      base.OnClose();
    }

    protected virtual void Commit()
    {
      if (this.documentSession == null)
      {
        return;
      }

      try
      {
        this.documentSession.SaveChanges();
      }
      catch (Exception e)
      {
        ErrorHandler.Error("Exception while commiting to the Raven DB", e, ErrorCode.GenericFailure);
      }
    }

    private void CheckSession()
    {
      if (this.documentSession != null)
      {
        return;
      }

      lock (this.lockObject)
      {
        if (this.documentSession != null)
        {
          if (this.documentSession.Advanced.NumberOfRequests >= this.documentSession.Advanced.MaxNumberOfRequestsPerSession)
          {
            this.Commit();
            this.documentSession.Dispose();
          }
          else
          {
            return;
          }
        }

        this.documentSession = this.documentStore.OpenSession();
        this.documentSession.Advanced.UseOptimisticConcurrency = true;

        if (this.MaxNumberOfRequestsPerSession > 0)
        {
          this.documentSession.Advanced.MaxNumberOfRequestsPerSession = this.MaxNumberOfRequestsPerSession;
        }
      }
    }

    private void InitServer()
    {
      if (this.documentStore != null)
      {
        return;
      }

      if (string.IsNullOrEmpty(this.ConnectionString))
      {
        var exception = new InvalidOperationException("Connection string is not specified.");
        ErrorHandler.Error("Connection string is not specified.", exception, ErrorCode.GenericFailure);

        return;
      }

      this.documentStore = new DocumentStore
      {
        ConnectionStringName = this.ConnectionString
      };

      this.documentStore.Initialize();
    }
  }
One detail we should care about is max number of requests per session. RavenDB's default limitation is 30 requests per session. On the one hand we have moved this option to the config file on the other hand we just recreate a session when max number of requests is reached.
So this is it: simple but effective. With RavenDB indexes it is possible to have extra-analysis or statistics of logged events, for instance, number of error per day / per module, new errors or warnings, etc.
Sources are available on GitHub.
NuGet package: http://nuget.org/List/Packages/log4net.Raven

Update:
The screenshot of the test Log documents



2011-08-30

Hard SVN branch switch for Git

I have been working with Git through git-svn bridge in SVN based organization together with Alexander Beletsky. We practice Git to solve lots of SVN issues.
There are several tricky moments with git-svn, for instance, SVN branch switching. If there is not quite big difference between your git repository and svn repo, you can use this advice.
But when I had returned from vacation, I found this approach was not working. I have missed one iteration so my Git repository should jump over huge list of changes. Saying in another words, I need just replace my "master" branch with SVN's latest iteration branch.
This solution helped me:

1. Create new git-svn branch.
git config --add svn-remote.SvnRemoteNewBranch.url https://svn/branches/NewIteration
git config --add svn-remote.SvnRemoteNewBranch.fetch :refs/remotes/SvnRemoteNewBranch
git svn fetch SvnRemoteNewBranch
git checkout -b LocalNewBranch -t SvnRemoteNewBranch
git svn rebase SvnRemoteNewBranch 

With it you will get the latest changes from svn to the "LocalNewBranch".
2. Replace "master" branch with svn's one.
git merge -s ours master
git checkout master
git merge LocalNewBranch  

This is it! Your "master" branch has the latest changes.


2011-05-17

JavaScript tip: on complete event

Recently I was fixing some issue in the JavaScipt event handler. Task was simple: to save user scroll position setting for the HTML control to the server.
Of course, it is possible to subscribe on the "scroll" event of the jQuery control and add a handler. But while user is scrolling it invokes event handler enormous number of times :-), so I need only "on scroll complete" event. And it took me some time to find one of the solutions.
If you need to have an "on complete" entry for some continuing event (like scrolling), described solution might help you.
Some event handler:

This code will write "scrolling..." to the console a lot of times while scrolling. In order to make it work just "on complete" we can add the timer function. Idea is simple: each time while scrolling it will cancel the timer function and then create it again. After the last scroll event (and after 1 sec) we will have an "on complete" function so it will write some message to the console.
EventHandler = function ($control) {
  if (!$control) {
    throw ("jQuery control is expected");
  }

  this.$control = $control;
  this.scrollCompleteHandling = null;
};

EventHandler.prototype.setupHandlers = function () {
  var self = this;
  this.$control.bind("scroll", function () {
    if (self.scrollCompleteHandling) {
      clearTimeout(self.scrollCompleteHandling);
    }
    self.scrollCompleteHandling = setTimeout(function () {
      console.log("scrolled...");
    }, 1000);
  });
};
It might not be the best option, but still it works :-)

2011-04-30

The gateway for an IoC container

Inversion of control and Dependency injection tools became an integral part of any enterprise application because they are great at reducing complex dependencies and increasing flexibility.
My first experience with IoC within enterprise applications was MS Unity that had been used  for decoupling parts of a system and adding extra-flexibility (e.g., to extend system with new implementations). As I discovered it is a great tool that helped me and my team a lot to meet the flexibility requirements of our solution. Nevertheless during research and development I found several issues of using IoC/DI tool in general. They might not be so obvious, but they became a bone in the throat for me.
1. If there is a necessity to implement extra-caching for some services (i.e., specific for our CMS) it would a hard time to create and extensions to do that, and our solution was to create a wrapper layer over Unity's container to intercept its calls and with the passage of time that layer had become pretty big and messy.
2. There is a negative dependency on API of the tool. Thus, Unity's xml container configuration had become our public interface for CMS developers to extend our solution and, as a result, our solution was dependent on reliability of Unity and more - it meant freezing of version Unity. Why? If customer invest in a solution based on our system he would definitely use our (actually, borrowed from Unity) xml structure and it would take time and money to modify it according to the new version of Unity. It did not take long to feel it in the hard way: Unity had been upgraded from version 1.2 to 2.0 so it took us a week to upgrade not released product!
3. IoC tools use reflection (emit is promised) to create and an instance(s) (even though using Singletons) so Resolving is always in the performance hot-sports (8-12% of CPU general usage! for our project). Solving the problem requires extra-monitoring and optimizing of using container. Basically such a problem happened because our team had become to use DI tool as a ServiceLocator instead of injecting dependencies that caused using Container.Resolve<T>() enormous amount of times. If you use 3-d party unmanaged tool is more complex to control resolving.
For my new project we have looked to some various DI tools like: Managed Extensibility Framework (it is actually a little bit different but, at some point, can be used as DI tool), autofac, Spring, Ninject, StructureMap and Windsor. The decision was to use Castle Windsor that, at first, disappointed me but than I realized that all tools basically able to do the same :-). So being inspired by Ritesh Rao's post and Davy Landman's one I realized that sometimes it makes sense to use a generic gateway wrapper for IoC container. It would allow to "mitigate" described issues plus benefits like tool version/vendor independence, more control for 3-d party tool as an "enemy". Of course there is a great tool called Common Service Locator library but, unfortunately, it had been released more than 2 years ago and it does not fit all the functionally we require from the DI tool.
So I have decided to develop some simple gateway that would fit major functionalities I faced for the IoC.
Such a wrapper has several disadvantages:
1. It cuts extra-functionalities of a tool beyond;
2. It takes time to develop and maintain it.
So, IMHO, for small projects it does not make any sense to use any wrappers, it will just increase the complexity. But for enterprise applications that are developed by many people for many years it will give distinct advantages.
Functionalities cuts is a very ambiguous thing. Many tools have extra-functionality like class interceptors, easy-registrations etc. IMHO, these bells and whistles might harm more than give benefits. Of course some of features are important so they should be considered to be used on enterprise approaches (like AOP and PostSharp as a tool).

Ok, let's look to the wrapping.
Basic solution is quite simple: there is container's core that has an instance(s) of the 3-d party tool to intercept all calls and static gateway for quick and convenient access to the IoC container. The first goal is completely isolate 3-d party tool for the our usage.
Okay, before we start on the IoC wrapper, I need some infrastructure work done. First we need is somehow to initialize the container i.e. to fulfill it with contract-implementation associations. One of the most simple solution is create a interface that allows to build the container.
public interface IIoCRegistration
{
  void BuildContainer(IoCContainerCore container);
}

[IoCRegistration]
public class Registrator : IIoCRegistration
{
  public void BuildContainer(IoCContainerCore container)
  {
    container.Register();
  }
}
The class that implements this interface will receive container instance and than it is possible to initialize it using API, either read from XML or anything else. So idea is to let user to implement the interface and mark classes with custom attribute to find registration in an assembly:
[AttributeUsage(AttributeTargets.Class)]
public sealed class IoCRegistrationAttribute : Attribute
{
}
Ok, but we also need to register that assemblies with container setup: to write their names to application config file. So let's create a section:
public sealed class IoCAssemblyRegistration : ConfigurationSection
{
  public const string ConfigurationSectionName = "iocAssemblyConfiguration";

  [ConfigurationProperty("assemblies")]
  public AssembliesElementCollection Assemblies
  {
    get
    {
      return (AssembliesElementCollection)this["assemblies"] ?? new AssembliesElementCollection();
    }
  }

  public static IoCAssemblyRegistration GetConfig()
  {
    return (IoCAssemblyRegistration)ConfigurationManager.GetSection(ConfigurationSectionName) ?? new IoCAssemblyRegistration();
  }
}

public class AssemblyElement : ConfigurationElement
{
  [ConfigurationProperty("assembly", IsRequired = true)]
  public string Assembly
  {
    get
    {
      return this["assembly"] as string;
    }
  }
}

public class AssembliesElementCollection : ConfigurationElementCollection
{
  public AssemblyElement this[int index]
  {
    get
    {
      return BaseGet(index) as AssemblyElement;
    }

    set
    {
      if (BaseGet(index) != null)
      {
        BaseRemoveAt(index);
      }

      BaseAdd(index, value);
    }
  }

  protected override ConfigurationElement CreateNewElement()
  {
    return new AssemblyElement();
  }

  protected override object GetElementKey(ConfigurationElement element)
  {
    return ((AssemblyElement)element).Assembly;
  }
In the .config file it will look like:

  
    
Ok, infrastructural work is done, let's define what basic functionality we would like to have:
1. Nested containers;
2. Registration of type and instance;
3. Resolve, resolve all;
4. Checking if registered;
5. Remove registrations;
6. Setup lifetime management.

Also we need to have a thread save container and, since we suppose that registering (writing to the container) would invoked rarely and resolving (reading) is a common usage, ReaderWriterLockSlim class can be used.

....
private readonly ReaderWriterLockSlim iocLocker;
....
private void InvokeWriteLock(Action action)
{
  this.iocLocker.EnterWriteLock();

  try
  {
    action();
  }
  catch (Exception exception)
  {
    Log.Error(exception.Message, exception, typeof(IoCContainerCore));
    throw new InvalidOperationException(exception.Message, exception);
  }
  finally
  {
    this.iocLocker.ExitWriteLock();
  }
}

private T InvokeReadLock(Func func)
{
  if (!this.iocLocker.IsReadLockHeld && !this.iocLocker.IsWriteLockHeld)
  {
    this.iocLocker.EnterReadLock();
  }

  try
  {
    return func();
  }
  catch (Exception exception)
  {
    Log.Error(exception.Message, exception, typeof(IoCContainerCore));
    throw new InvalidOperationException(exception.Message, exception);
  }
  finally
  {
    if (this.iocLocker.IsReadLockHeld)
    {
      this.iocLocker.ExitReadLock();
    }
  }
}
...

1. Nested containers. So let's allow to have a  hierarchy of containers:
public partial class IoCContainerCore : IDisposable
{
  private readonly ReaderWriterLockSlim childrenLocker;

  private IWindsorContainer container;

  private IoCContainerCore parent;

  private string name;

  private Dictionary childContainers;

  public IoCContainerCore()
  {
    this.container = new WindsorContainer();
    this.childrenLocker = new ReaderWriterLockSlim();
    this.childContainers = new Dictionary();
  }

  public IoCContainerCore Parent
  {
    get
    {
      return this.parent;
    }

    set
    {
      Assert.ArgumentNotNull(value, "value");
      this.parent = value;
      Log.Info(string.Format("Parent container named: '{0}' has been set.", value.Name), typeof(IoCContainerCore));
    }
   }

  public string Name
  {
    get
    {
      return this.name;
    }
    set
    {
      Assert.ArgumentNotNullOrEmpty(value, "value");
      this.name = value;
      Log.Info(string.Format("Container name: '{0}' has been set.", value), typeof(IoCContainerCore));
    }
 }

 public virtual void AddChildContainer(string childContainerKey, IoCContainerCore childContainer)
  {
    Assert.ArgumentNotNullOrEmpty(childContainerKey, "childContainerKey");
    Assert.ArgumentNotNull(childContainer, "childContainer");

    this.childrenLocker.EnterWriteLock();
    try
    {
      Assert.IsFalse(this.childContainers.ContainsKey(childContainerKey), string.Format("Child container named: '{0}' has already been added.", childContainerKey));

      this.childContainers.Add(childContainerKey, childContainer);

      Log.Info(string.Format("Child container named: '{0}' has been added.", childContainerKey), typeof(IoCContainerCore));
    }
    catch (Exception exception)
    {
      Log.Error(exception.Message, exception);
      throw;
    }
    finally
    {
      this.childrenLocker.ExitWriteLock();
    }
  }

  public virtual void RemoveChildContainer(string childContainerKey)
  {
    Assert.ArgumentNotNullOrEmpty(childContainerKey, "childContainerKey");

    this.childrenLocker.EnterWriteLock();
    try
    {
      Assert.IsFalse(this.childContainers.ContainsKey(childContainerKey), string.Format("Child container named: '{0}' has already been added.", childContainerKey));

      this.childContainers.Remove(childContainerKey);

      Log.Info(string.Format("Child container named: '{0}' has been removed.", childContainerKey), typeof(IoCContainerCore));
    }
    catch (Exception exception)
    {
      Log.Error(exception.Message, exception);
      throw;
    }
    finally
    {
      this.childrenLocker.ExitWriteLock();
    }
  }

  public virtual IoCContainerCore GetChildContainer(string childContainerKey)
  {
    Assert.ArgumentNotNullOrEmpty(childContainerKey, "childContainerKey");

    this.childrenLocker.EnterReadLock();
    try
    {
      Assert.IsTrue(this.childContainers.ContainsKey(childContainerKey), string.Format("Child container named: '{0}' does not excist.", childContainerKey));

      return this.childContainers[childContainerKey];
    }
    catch (Exception exception)
    {
      Log.Error(exception.Message, exception);
      throw;
    }
    finally
    {
      this.childrenLocker.ExitReadLock();
    }
  }
}
2. Registration. I have a list of registration functionality that I usually use.
1) Register by type: common use case;
2) Register by key: required when you have set of implementation ;
3) Register object instance: useful when object creation is not under your control;
4) Register with injecting parameters: setup additional parameters to the constructor;
5) Register with lifetime management: by default all registrations are singletons, but usually and especially for web application another management required like: per Request, per Session, per Thread. etc ;
6) Register with setting up dependencies: allow user to setup what exactly dependency should be used;
7) Register using factory method: useful when an object should be created in a special way (e.g. NHibernate data session).
There is a list a basic examples without combination of parameters.

public virtual void Register<TFrom, TTo>() where TTo : TFrom
{
  this.InvokeReadLock(() => this.container.Register(Component.For<TFrom>().ImplementedBy<TTo>()));
}

public virtual void RegisterInstance<TFrom>(TFrom fromInstance)
{
  Assert.ArgumentNotNull(fromInstance, "@from");

  this.InvokeReadLock(() => this.container.Register(Component.For<TFrom>().Instance(fromInstance)));
}

public virtual void Register<TFrom, TTo>(string key) where TTo : TFrom
{
  Assert.ArgumentNotNullOrEmpty(key, "key");

  this.InvokeReadLock(() => this.container.Register(Component.For<TFrom>().ImplementedBy<TTo>().Named(key)));
}

public virtual void Register<TFrom, TTo>(IEnumerable<KeyValuePair<string, string>> dependencies) where TTo : TFrom
{
  Assert.ArgumentNotNullOrEmpty(dependencies, "dependencies");

  this.InvokeReadLock(() =>
  {
    var windsorDepens = dependencies.Select(parameter => ServiceOverride.ForKey(parameter.Key).Eq(parameter.Value));

    return this.container.Register(Component.For<TFrom>().ImplementedBy<TTo>().DependsOn(windsorDepens.ToArray()));
  });
}

public virtual void Register<TFrom, TTo>(Action<IDictionary> setParameters) where TTo : TFrom
{
  Assert.ArgumentNotNull(setParameters, "setParameters");

  this.InvokeReadLock(() => this.container.Register(Component.For<TFrom>().ImplementedBy<TTo>().DynamicParameters((k, d) => setParameters(d))));
}

public virtual void Register<TFrom, TTo>(ILifeTimeDefinition lifeTimeDefinition) where TTo : TFrom
{
  Assert.ArgumentNotNull(lifeTimeDefinition, "lifeTimeDefinition");

  this.InvokeReadLock(() => this.container.Register(LifeTimeRegistration.GetManager(Component.For<TFrom>().ImplementedBy<TTo>(), lifeTimeDefinition)));
}

public virtual void Register<TFrom>(Func<TFrom> factoryMethod)
{
  Assert.ArgumentNotNull(factoryMethod, "factoryMethod");

  this.InvokeReadLock(() => this.container.Register(Component.For<TFrom>().UsingFactoryMethod(() => factoryMethod())));
}
...

So let's move to resolving.
1) Resolving by type
2) Resolving by type and key;
3) Resolve all;
4) Resolve injecting arguments;
5) Try resolve

public virtual T Resolve<T>()
{
  return this.InvokeReadLock(() =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.Resolve<T>();
  });
}

public virtual T Resolve<T>(string key)
{
  Assert.ArgumentNotNullOrEmpty(key, "key");

  return this.InvokeReadLock(() =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.Resolve<T>(key);
  });
}

public virtual T Resolve<T>(IDictionary arguments)
{
  Assert.ArgumentNotNull(arguments, "arguments");

  return this.InvokeReadLock(() =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.Resolve<T>(arguments);
  });
}

public virtual T Resolve<T>(object argumentsAsAnonymousType)
{
  Assert.ArgumentNotNull(argumentsAsAnonymousType, "argumentsAsAnonymousType");

  return this.InvokeReadLock(() =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.Resolve<T>(argumentsAsAnonymousType);
  });
}

public virtual object Resolve(Type service)
{
  Assert.ArgumentNotNull(service, "service");

  return this.InvokeReadLock(() =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.Resolve(service);
  });
}

public virtual bool TryResolve<T>(out T @object)
{
  return this.TryInvokeReadLock(
  () =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.Resolve<T>();
  },
  out @object);
}

public virtual T[] ResolveAll<T>()
{
  return this.InvokeReadLock(() =>
  {
    this.EnsureContainerIsInitialsied();
    return this.container.ResolveAll<T>();
  });
}

private void EnsureContainerIsInitialsied()
{
  if (!this.initialised)
  {
    this.InitializeContainer();
  }
}
...

As you can see an idea is simple but it's painstaking :-)
I'll skip blocks with checking if registered and removing registrations because it is quite trivial task.
And couple of words about lifetime management. In order to isolate DI tool classes I had to create own wrappers based on Lifetime definition and manager:

public interface ILifeTimeDefinition
{
  Type LifetimeManager { get; }
}
public static class LifeTimeRegistration
{
  public static ComponentRegistration<S> GetManager<S, T>(ComponentRegistration<S> componentRegistration,T definition) where T : ILifeTimeDefinition
  {
    return new LifestyleGroup<S>(componentRegistration).Custom(definition.LifetimeManager);
  }
}
public class PerWebRequest : ILifeTimeDefinition
{
  public Type LifetimeManager
  {
    get { return typeof(PerWebRequestLifestyleManager); }
  }
}
public class PerThread : ILifeTimeDefinition
{
  public Type LifetimeManager
  {
    get { return typeof(PerThreadLifestyleManager); }
  }
}
...

And now a static gateway example. It is made just for initialization, storing wrapper and convenience.

public static class IoCContainer
{
  private static readonly IoCContainerCore Container;

  static IoCContainer()
  {
    Container = new IoCContainerCore();
  }

  public static IoCContainerCore ContainerCore
  {
    get { return Container; }
  }

  public static T Resolve()
  {
    return Container.Resolve();
  }

  public static void Register() where TTo : TFrom
  {
    Container.Register();
  }
  ....
  public static void InitializeContainer()
  {
    if (Container.IsInitialised)
    {
      return;
    }

    IEnumerable configAssemblies = GetConfigAssemblies();
    if (configAssemblies.Any())
    {
      Container.InitializeContainer(configAssemblies, null);
    }
    else
    {
      Container.InitializeContainer();
    }
  }
   
  private static IEnumerable GetConfigAssemblies()
  {
    var section = ConfigurationManager.GetSection(IoCAssemblyRegistration.ConfigurationSectionName) as IoCAssemblyRegistration;
    if (section == null)
    {
      // throw new ConfigurationErrorsException("Configuration section was not found");
      yield break;
    }

    foreach (AssemblyElement element in section.Assemblies)
    {
      yield return element.Assembly;
    }
  }

}

Ok, and that works, but is it possible to change the DI tool? Yes, and it took my two and a half hours to replace Windsor with Unity and pass all unit tests.
IoC container usage is quite simple:
[TestFixture]
public class IoCContainerTest
{
  [SetUp]
  public void SetUp()
  {
    IoCContainer.InitializeContainer();
  }

  [TearDown]
  public void ResetContainer()
  {
    IoCContainer.ResetContainer();
  }
  
  [Test]
  public void ResolveTest()
  {
    var service = IoCContainer.Resolve();

    Assert.IsNotNull(service);
    Assert.IsTrue(service is Logger);
  }

  [TestCase("Implementation1")]
  [TestCase("Implementation1Child")]
  public void ResolveInvokeTest(string serviceName)
  {
    var service = IoCContainer.Resolve(serviceName);

    Assert.IsNotNull(service);
    service.Do("some data");
  }
 ...
}
As a summary, I want to underline what that messy stuff gave us. We have isolated 3-d party tool completely so we control IoC container usage now :-), we can do any manipulations to optimize it: caching, container storing, combining several tools if needed, lifetime managers manipulations.
Sources are available here for Windsor and here for Unity.

2011-03-13

PLINQ: is it worth to use?

Last month I had a speaker corner with my colleagues: "Are you still using loops? - We are coming to you!".
We always face the scalability issues in our work and one of the most evident scalability holes is processing great amount of data. One of the solutions could be the concurrent executing on multi-core CPUs. So the main point of the discussion was that we should not use loops any more because they are obsolete :-). Loops like: for, foreach, while, etc are not lead us to the atomic programming and as a result we are not really ready to develop effectively for the concurrent execution.
There were not only .Net developers participating the speaker corner, so they resented what they can use instead? Fortunately C# has LINQ and even more - Parallel LINQ. .NET 4.0 natively provides us with such a chance to use it without a necessity to be a concurrency guru.

Parallel LINQ (PLINQ) is a query execution engine that accepts any LINQ-to-Objects or LINQ-to-XML query and automatically utilizes multiple processors or cores for execution when they are available. PLINQ becomes increasingly crucial to ensuring the scalability of software on future parallel microprocessor architectures. In fact, threads and locks won't even come up unless you really want to dive under the hood to understand how it all works. PLINQ is a key component of Parallel FX, the next generation of concurrency support in the .NET.
Of course, using  concurrent processing one should think about external resource: if you actively work with it within the query, most time the threads are in synchronization state so you're apt to be quite disappointed about the performance.
I am not going to dive deep into the concurrent development theory, I am just want to share some observations weather PLINQ is really able to help with performance improvements or not. Let's write some simple application and profile it. Let's say we have some CollectionProcessor class that calculates a math expression for each element in the collection of numbers and that sums all elements.

public class CollectionProcessor
{
  private static double ItemProccess(int item)
  {
    return Math.Cos(10005000 * item) / item;
  }

  public static double ForEachSerial(IEnumerable<int> collection)
  {
    double sum = 0d;

    foreach (int i in collection)
    {
      sum += ItemProccess(i);
    }

    return sum;
  }

  public static double LinqSerial(IEnumerable<int> collection)
  {
    return collection.Select(ItemProccess).Sum();
  }

  public static double ForEachParallel(IEnumerable<int> collection)
  {
    double sum = 0d;

    foreach (int i in collection.AsParallel())
    {
      sum += ItemProccess(i);
    }

    return sum;
  }

  public static double LinqParallel(IEnumerable<int> collection)
  {
    return collection.AsParallel().Select(ItemProccess).Sum();
  }
}

Here we have four solutions: foreach loop, LINQ, foreach loop using parallel mode and PLINQ.
Let's test the results now and take 100050000 elements for the collection of numbers.

[TestFixture]
public class CollectionProcessorTest
{
  private IEnumerable Collection { get; set; }

  [SetUp]
  public void SetUp()
  {
    Collection = ParallelEnumerable.Range(0, 100050000);
  }

  [Test]
  public void ForEachSerialTest()
  {
    Stopwatch stopwatch = new Stopwatch();

    stopwatch.Start();
    CollectionProcessor.ForEachSerial(this.Collection);
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
  }

  [Test]
  public void LinqSerialTest()
  {
    Stopwatch stopwatch = new Stopwatch();

    stopwatch.Start();
    CollectionProcessor.LinqSerial(this.Collection);
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
  }

  [Test]
  public void ForEachParallelTest()
  {
    Stopwatch stopwatch = new Stopwatch();

    stopwatch.Start();
    CollectionProcessor.ForEachParallel(this.Collection);
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
  }

  [Test]
  public void LinqParallelTest()
  {
    Stopwatch stopwatch = new Stopwatch();

    stopwatch.Start();
    CollectionProcessor.LinqParallel(this.Collection);
    stopwatch.Stop();

    Console.WriteLine(stopwatch.ElapsedMilliseconds);
  }
}

I have Intel Core Quad CPU (4-cores) running on 3,3 GHz. The time of execution results are:
ForEachSerialTest : 16 sec;
LinqSerialTest : 16,3 sec;
ForEachParallelTest : 15,5 sec;
LinqParallelTest :  3,9 sec
Hey! Here we have such a great  time of execution improvements: up to 4 times! Also, it was very nice to see the Core-gadget while PLINQ was running: 

But foreach loop even in concurrent mode had not shown real improvements.
Ok, let's profile the results. 
There are not such a great performance improvements while profiler is working, but still it exists. The expanded tree is: 
Now we can see that MoveNext operation to iterate takes more time than item processing. So I don't see any sense to use .AsParallel() with foreach loop. 
ItemProcess takes the same time for sequential test (approx. 12 sec) but PLINQ requires only 8 sec. But it also creates more threads:
And that is the point! With PLINQ we have 4 threads working in parallel. 
The interesting thing is how PLINQ  works inside. I have tried to look at the systems calls more thoroughly. 
First, let's take a common LINQ query:

This looks simple: it creates expression tree: ItemProcess - WhereSelectEnumerableIterator - Sum.
But the PLINQ tree is a little bit different:-) I even had to flatten the picture.
The call tree is impressive! It uses lots of system commands to start and execute threads. It obviously requires a lot of resources to start the execution. So if you query is not really shared in the parallel tasks you would get serious performance issues with PLINQ.
In the conclusion, PLINQ allows LINQ developers to take advantage of parallel hardware — including achieving better performance and building more intrinsically scalable software — without needing to understand concurrency or data parallelism. But my own experience with PLINQ had shown that one should be very careful with it, test and profile the resulted query to insure it really gives benefits of running in parallel and does not wait for thread synchronization all the time.

P.S.
There are nice samples to start with PLINQ.

2011-02-27

The First Selenium Camp in Europe

I was lucky to visit the first Selenium Camp in Europe last Saturday. It was held in Kiev by XP Injection group.

Even though I am not a QA-or automation testing engineer it was quite useful to visit it. The first reason is we do use (or trying to use) Selenium for functional testing in our project, the second one it is quite nice to feel myself in the shoes of guys how test software all the time. As for my real experience it is quite small, I was just using IDE and RC to create some simple tests and even before I was using WatiN for some functional testing. But Selenium is a real bomb... just imagine, it becomes so popular that people organize big conferences about it!
Let's start from the basics, your Captain speaking:
Selenium is a portable software testing framework for web applications. Selenium provides a record/playback tool for authoring tests without learning a test scripting language. Selenium provides a test domain specific language (DSL) to write tests in a number of popular programming languages. Test playback is possible in most modern web browsers. Selenium deploys on Windows, Linux, and Macintosh platforms. 
Selenium contains several parts:
1. Core written in pure JavaScript, 
2. IDE as a Firefox extension that allows to record, edit, and debug tests, 
3. Remote Control (RC) that allows you to write automated web application UI tests in any programming language against any HTTP website using any mainstream JavaScript-enabled browser, 
4. Grid  allows to run tests on many servers at the same time, cutting down on the time it takes to test multiple browsers or operating systems.
5. Many extensions, integrations etc.

The kick-off meeting I visited was David Burns's "Selenium 2 : The future of Selenium is now!". As a result of a presentation I found that future of the Selenium is quite solid, many good thing like web drivers, fluent and simple API, caching elements collection are provided.
Web driver is a layer between Selenium Core and browser engines, created specially for each supported browser, that means written within the native environment for it, C++/COM for IE, extension mechanisms for FF and Chrome, etc. On the other hand web driver is ready to be used within high level languages like C#. Sometimes it is better to use WebDriver than Selenium and vice-versa. But just look at that:
[TestFixture]
public class GoogleIETest
{
  IWebDriver driver;
      
  [SetUp]
  public void Setup()
  {
    driver = new InternetExplorerDriver();
  }
     
  [TearDown]
  public void Teardown()
  {
    driver.Quit();
  }
      
  [Test]
  public void TestSearchGoogleForTheAutomatedTester()
  {
    //Navigate to the site
    driver.Navigate().GoToUrl("http://www.google.ua");

    //Find the Element and create an object so we can use it
    IWebElement queryBox = driver.FindElement(By.Name("q"));

    //Work with the Element that's on the page
    queryBox.SendKeys("The Automated Tester");
    queryBox.SendKeys(Keys.ArrowDown);
    queryBox.Submit();

    //Check that the Title is what we are expecting
    Assert.True(driver.Title.IndexOf("The Automated Tester") > -1);
  }
}
Yes, It looks just like in WatiN, but it is only a simple example, Selenium provides us with lots of advanced features. What about to be integrated with behavior driver development, run tests in concurrent mode for the different environments, file upload... Selenium is about that. But something is not so brilliant with new Selenium, i.e. Grid 2 future is not so clear. There is no support for Silverlight.
The second and third presentations I visited were about sharing some experience of using Selenium. Kirill Klimov and Mairbek Hadikov had shown us their way of progressing with Selenium from IDE to Grid. Kirill  described us the pros and cons of each Selenium tool form Core to Grid. As for me, I enjoyed RC the most. Mairbek had presented us some valuable advices:
1. Use automation testing
2. Integrate automation tests to Continuous Integration server and run test frequently.
3. Use code review for the test code
4. Use Selenium 2 :-)
5. Use CSS selectors, they are faster and more reliable that x-path
6. Use BDD or write test code that is clear to nontechnical people
7. Isolate test from the environment
8. Try to avoid side effects
9. Make assertions after selectors
10. Think about concurrent testing, different environment ...  thing about Grid
Of course, these advises are simple and common but some of them are regularly forgotten while writing tests.
The following topics I visited were devoted to integration of development process and Selenium.
For example, Aleksey Rezchikov shared his experience of using Selenium within Agile-oriented development. Some moments were quite interesting, i.e. there is a bunch of tools to bind a user story and functional test to make a story test: FitNesse,  Concordion, Cucumber, etc. Also he mentioned quite strong reasons to use PageObject and Feature Flags patterns.
One of the major Camp organizer Nikolay Alimenkov have shown as his way of creating acceptance tests using Selenium + Wiki = Live Requirements. This approach seems to be very cool but I am not sure I will be able to force my Product Owner to write acceptance tests even using wiki :-).
I was not sure that Selenium is able to handle rich Ajax application before the camp, but Sergey Shvets had shown his way of testing such things, and recipe was quite simple: use pauses, jQuery, atomic blocks of testing to manage Ajax requests: post backs. counts etc.

The presentations from Selenium Camp are here.

Remember, if developer is also a tester he is a Product Developer! :-)

2011-02-20

EmitMapper: customizing mapping tules

There is a need for data mapping while developing the enterprise applications with rich functionality, especially in Domain-driven design (DDD).
Martin Fowler says:
Sometimes you need to set up communications between two subsystems that still need to stay ignorant of each other. This may be because you can't modify them or you can but you don't want to create dependencies between the two or even between them and the isolating element.

Objects, relational databases, data transfer objects (DTO), etc. often have different mechanisms for structuring data. Many parts of an object, such as collections and inheritance, aren't present in relational databases, DTOs. When you build an object model with a lot of business logic it's valuable to use these mechanisms to better organize the data and the behavior that goes with it. Doing so leads to variant schemas; that is, the object schema and the relational schema don't match up.
You still need to transfer data between the two schemas, and this data transfer becomes a complexity in its own right. If the in-memory objects know about the relational database structure, changes in one tend to ripple to the other.
The Data Mapper is a layer of software that separates the in-memory objects from the database or other service. Its responsibility is to transfer data between the two and also to isolate them from each other.

So when there is a challenge of data mapping between two subsystem it becomes a bone in the throat! On the one hand it is possible to create handwritten mapping, use adapters. etc, it is fast and reliable, but on the other hand it takes much time to support and it is not flexible enough.
There is bunch of tools and solutions for data mapping, automatic: AutoMapper, BLToolkit, EmitMapper, semi-automatic: MapForce, Stylus Studio, BTS. Semi-automatic mapper are very advanced but they are usually not free and complex.
I was using automatic mappers, in particular EmitMapper as one of the fastest, because it uses Emit (unsurprisingly from its name :-)) But default configurations are also not so flexible, but there is a bunch of available customizations by default, like creating object for custom constructor, mapping rule definitions, etc. But the most interesting there is IMappingOperation that allow totally customize a mapping process.
Ok, let's look at the problem I recently had. On the one side of the system there is a container with data grouped by key-value collection (just like a table).
public class DataContainer
{
  public Dictionary<string, object> Fields;
}
On the other side there is strongly-typed entity (actually it can be tree of object, but let's simplify the task for this post):
public class Entity
{
  public Guid Id { get; set; }
  public string Name { get; set; }
  public int Number { get; set; }
  public decimal Price { get; set; }
  public string UserName { get; set; }
  public DateTime Time { get; set; }
}
And of-course there is a data container definition available (list of keys and filed type). Usually, if you are using an ORM there are some techniques to set-up mapping rules: xml, attributes, etc.
But if you are not using an ORM (either it is not a data-access layer object or it is not possible to use it), you have to find the way how to formalize the mapping rules. The most obvious way for me was to use custom attributes, just like in some ORMs. data serialization for services:
[AttributeUsage(AttributeTargets.Property | AttributeTargets.Field, AllowMultiple = true, Inherited = true)]
public class DataMemberAttribute : Attribute
{
  public virtual string FieldName { get; set; }
  public virtual Type FieldType { get; set; }
}
Ok, let's formalize the rules for our example:
public class Entity
{
  [DataMember(FieldName = "order_id", FieldType = typeof(string))]
  public Guid Id { get; set; }

  [DataMember(FieldName = "order_name")]
  public string Name { get; set; }

  [DataMember(FieldName = "order_number", FieldType = typeof(double))]
  [DataMember(FieldName = "order_number_2", FieldType = typeof(double))]
  public int Number { get; set; }

  [DataMember(FieldName = "order_price")]
  public decimal Price { get; set; }

  [DataMember]
  public string UserName { get; set; }

  [DataMember(FieldType = typeof(string))]
  public DateTime Time { get; set; }
}
So that means data container has these columns: order_id, order_name, order_number, order_number_2, order_price, UserName, Time. It is a simple example but it might be so in real situations. If field name or type is not set it will take the property name and type.
The task might be more clear after this test example:
[Test]
public void EntityToDataContainerMappingTest()
{
  var entity = new Entity
  {
    Id = Guid.NewGuid(),
    Name = "Entity Name",
    Number = 134567,
    Price = 100.500m,
    UserName = string.Empty,
    Time = DateTime.Now
  };

  var container = Mapper.Map<Entity, DataContainer>(entity);

  Assert.AreEqual(entity.Id.ToString(), container.Fields["order_id"]);
  Assert.AreEqual(entity.Number, container.Fields["order_number"]);
  Assert.AreEqual(entity.Number, container.Fields["order_number_2"]);
  Assert.AreEqual(entity.Name, container.Fields["order_name"]);
  Assert.AreEqual(entity.Price, container.Fields["order_price"]);
  Assert.AreEqual(entity.UserName, container.Fields["UserName"]);
}

The class diagram of the solution I want to create:
Mapper tool is wrapped with static facade (Mapper class) for convenient interface (to be protected against invariants, changing of a tool etc), that provides us both with mapping API and mapping customizations that implements EmitMapper's IMappingConfigurator interface.
Static facade will have an instance of mapping core (MapperCore class) that contains mappers (EmitMapper's ObjectsMapper classes) and configuration instances for the Emit. The lowest layer here is domain model mapping configurations that would allow us to map data container to the entity and in reverse order (ObjectToDataContainerConfigurator and DataContainerToObjectConfigurator classes).

Ok, where we should start? First, we need a logic to extract those fields descriptions for Reflection Utils class. Ok, let's go:
public static ConcurrentDictionary<string, Tuple<MemberInfo, Type>> GetTypeDataContainerDescription(Type type)
{
  Assert.ArgumentNotNull(type, "type");

  var fieldsDescription = from member in EmitMapper.Utils.ReflectionUtils.GetPublicFieldsAndProperties(type)
      let membersDefinitions = GetDataMemberDefinition(member)
      from fieldDescription in membersDefinitions
      let fieldName = fieldDescription.Item1
      let fieldType = fieldDescription.Item2
      select new KeyValuePair<string, Tuple<MemberInfo, Type>>(fieldName, Tuple.Create(member, fieldType));

  return new ConcurrentDictionary<string, Tuple<MemberInfo, Type>>(fieldsDescription);
}

public static IEnumerable<Tuple<string, Type>> GetDataMemberDefinition(MemberInfo memberInfo)
{
  Assert.ArgumentNotNull(memberInfo, "memberInfo");

  return from attribute in Attribute.GetCustomAttributes(memberInfo, typeof(DataMemberAttribute), true).Cast<DataMemberAttribute>()
      let fieldName = string.IsNullOrEmpty(attribute.FieldName) ? memberInfo.Name : attribute.FieldName
      let memberType = EmitMapper.Utils.ReflectionUtils.GetMemberType(memberInfo)
      let fieldType = attribute.FieldType ?? memberType
      select Tuple.Create(fieldName, fieldType);
}

This code looks messy, but one can always make a code refactoring :-) The purpose we had here is to get container schema for the type.
Let's create mapper core class, it would be nice to have the possibility of mapping objects, configuring a mapper and caching mappers.
public class MapperCore
{
  private static readonly IMappingConfigurator DefaultConfigurator;
  private static readonly ConcurrentBag<object> Mappers;
  private static readonly ConcurrentBag<Tuple<Type, Type, IMappingConfigurator>> MappingConfigurations;

  static MapperCore()
  {
    DefaultConfigurator = new DefaultMapConfig();
    Mappers = new ConcurrentBag<object>();
    MappingConfigurations = new ConcurrentBag<Tuple<Type, Type, IMappingConfigurator>>();
  }

  public virtual Tuple<Type, Type, IMappingConfigurator>[] Configurations
  {
    get { return MappingConfigurations.ToArray(); }
  }

  public void Initialize(IMapperInitializator mapperInitializator)
  {
    mapperInitializator.ConfigureMapper(this);
  }

  public virtual void AddConfiguration<TFrom, TTo>(IMappingConfigurator configurator)
  {
    Assert.IsNotNull(configurator, "configurator");

    MappingConfigurations.Add(new Tuple<Type, Type, IMappingConfigurator>(typeof(TFrom), typeof(TTo), configurator));
  }

  public virtual TTo Map<TFrom, TTo>(TFrom @from)
  {
    Assert.ArgumentNotNull(@from, "@from");

    var mapper = GetMapper<TFrom, TTo>();
    return mapper.Map(@from);
  }

  public virtual TTo Map<TFrom, TTo>(TFrom @from, TTo @to)
  {
    Assert.ArgumentNotNull(@from, "@from");
    Assert.ArgumentNotNull(@to, "@to");

    var mapper = GetMapper<TFrom, TTo>();
    return mapper.Map(@from, @to);
  }

  public virtual IEnumerable<TTo> MapCollection<TFrom, TTo>(IEnumerable<TFrom> @from)
  {
    Assert.ArgumentNotNullOrEmpty(@from, "@from");

    var mapper = GetMapper<TFrom, TTo>();
    return mapper.MapEnum(@from);
  }

  protected virtual ObjectsMapper<TFrom, TTo> GetMapper<TFrom, TTo>()
  {
    var mapper = Mappers.FirstOrDefault(m => m is ObjectsMapper<TFrom, TTo>) as ObjectsMapper<TFrom, TTo>;

    if (mapper == null)
    {
      var configuration = MappingConfigurations.Where(mp => mp.Item1.IsAssignableFrom(typeof(TFrom)) && mp.Item2.IsAssignableFrom(typeof(TTo))).FirstOrDefault();
      var config = configuration == null ? DefaultConfigurator : configuration.Item3;

      mapper = ObjectMapperManager.DefaultInstance.GetMapper<TFrom, TTo>(config);

      Mappers.Add(mapper);
    }

    return mapper;
  }
}
Ok, the service code has been written and now we can move to the most magic things .. configurations for mapping data containers to strongly-typed entities and vice versa. Two custom configurators had been written for EmitMapper for those purposes. The idea were also simple, just read the field descriptions from data member attributes and convert values walking through the entity's members:
public class DataContainerToObjectConfigurator : MapConfigBase<DataContainerToObjectConfigurator>
  {
    public override IMappingOperation[] GetMappingOperations(Type from, Type to)
    {
      return this.FilterOperations(from, to, ReflectionUtils.GetTypeDataContainerDescription(to)
.Select(fieldsDescription =>
        {
          var fieldName = fieldsDescription.Key;
          var destinationMember = fieldsDescription.Value.Item1;
          var fieldType = fieldsDescription.Value.Item2;
          return new DestWriteOperation
            {
              Destination = new MemberDescriptor(destinationMember),
              Getter = (ValueGetter<object>)((item, state) =>
                {
                  if (item == null || !(item is DataContainer))
                  {
                    return ValueToWrite<object>.Skip();
                  }
                  var container = item as DataContainer;

                  var destinationType = EmitMapper.Utils.ReflectionUtils.GetMemberType(destinationMember);
                  var destinationMemberValue = ReflectionUtils.ConvertValue(container.Fields[fieldName], fieldType, destinationType);

                  return ValueToWrite<object>.ReturnValue(destinationMemberValue);
          })
      };
    })).ToArray();
  }
}

public class ObjectToDataContainerConfigurator : MapConfigBase
{
  public ObjectToDataContainerConfigurator()
  {
    ConstructBy(() => new DataContainer { Fields = new Dictionary() });
  }

  public override IMappingOperation[] GetMappingOperations(Type from, Type to)
  {
    return this.FilterOperations(from, to, ReflectionUtils.GetTypeDataContainerDescription(from)
                .Select(fieldsDescription =>
                {
                  var fieldName = fieldsDescription.Key;
                  var sourceMember = fieldsDescription.Value.Item1;
                  var fieldType = fieldsDescription.Value.Item2;
                  return new SrcReadOperation
                    {
                      Source = new MemberDescriptor(sourceMember),
                      Setter = (destination, value, state) =>
                      {
                        if (destination == null || value == null || !(destination is DataContainer))
                        {
                          return;
                        }

                        var container = destination as DataContainer;

                        var sourceType = EmitMapper.Utils.ReflectionUtils.GetMemberType(sourceMember);
                        var destinationMemberValue = ReflectionUtils.ConvertValue(value, sourceType, fieldType);

                        container.Fields.Add(fieldName, destinationMemberValue);
                      }
                    };
                })).ToArray();
  }
}
EmitMapper utils provides us with fluent API of extracting and saving data without using System.Reflection API. But...not so simple here, actually we need a way to convert types for data container, because entity's property type may vary form data entry's type.
The .NET Framework provides several features that support type conversion. These include the following:
1. The Implicit operator, which defines the available widening conversions between types.
2. The Explicit operator, which defines the available narrowing conversions between types.
3. The IConvertible interface, which defines conversions to each of the base .NET Framework data types.
4. The Convert class, which provides a set of methods that implement the methods in the IConvertible interface.
5. The TypeConverter class, which is a base class that can be extended to support the conversion of a specified type to any other type.

In order to create generic type converter we should consider these type conversion stages. In the code below during converting different type first it tries to check whether types are the same or assailable. If so we can just assign value directly. If type implements IConvertable interface it make sense to use it to convert value because it does not use Reflection so works pretty fast, i.e. Guid can be converted to String. But if converting fails, we will cache the list of non-convertable types to avoid exceptional situation in future. For example, it tries to convert String to Guid but fails with exception (that cause performance issues).
The last echelon is using TypeConverter. Here we also need to cache the converters to increase execution time.
private static readonly ConcurrentDictionary<Type, TypeConverter> TypeConverters = new ConcurrentDictionary<Type, TypeConverter>();
private static readonly List<Tuple<Type, Type>> TypesNotIConvertible = new List<Tuple<Type, Type>>();

public static object ConvertValue(object sourceValue, Type sourceType, Type destinationType)
{
  if (sourceValue == null || sourceType == null || destinationType == null)
  {
    return null;
  }

  if (sourceType == destinationType || destinationType.IsAssignableFrom(sourceType))
  {
    return sourceValue;
  }

  if (sourceValue is IConvertible && !TypesNotIConvertible.Contains(Tuple.Create(sourceType, destinationType)))
  {
    try
    {
      return Convert.ChangeType(sourceValue, destinationType);
    }
    catch
    {
      TypesNotIConvertible.Add(Tuple.Create(sourceType, destinationType));
    }      
  }

  object result = null;

  var typeConverter = TypeConverters.GetOrAdd(destinationType, TypeDescriptor.GetConverter);
  if (typeConverter != null && typeConverter.CanConvertFrom(sourceType))
  {
    result = typeConverter.ConvertFrom(sourceValue);
  }
  else
  {
    typeConverter = TypeConverters.GetOrAdd(sourceType, TypeDescriptor.GetConverter);
    if (typeConverter != null && typeConverter.CanConvertTo(destinationType))
    {
      result = typeConverter.ConvertTo(sourceValue, destinationType);
    }
  }

  return result;
}
Ok, mapping seems to be working, but what is the performance of such a mapper?
Let's create some test to check it! Will play big and map 1 million objects:
[TestCase(1000000)]
public void EntityToDataContainerMappingCollectionTest(int capacity)
{
  var entities = Enumerable.Range(0, capacity).Select(i => new Entity
  {
    Id = Guid.NewGuid(),
    Number = i,
    Name = "Name_" + i,
    UserName = "UserName_" + i,
    Price = (decimal)Math.Sqrt(i),
    Time = DateTime.Now
  }).ToArray();

  Mapper.DataMapper.Initialize(new DomainMappingInitializator());

  // Cold mapping
  Mapper.MapCollection<Entity, DataContainer>(new List<Entity> { new Entity() });

  var stopWatch = new Stopwatch();
  stopWatch.Start();
  GC.Collect();

  var containers = Mapper.MapCollection<Entity, DataContainer>(entities).ToArray();

  stopWatch.Stop();
  Console.WriteLine(string.Format("Mapping of the collection with {0} elements took: {1} ms. Approx. mapping time per object: {2} sec.", capacity, stopWatch.ElapsedMilliseconds, ((float)stopWatch.ElapsedMilliseconds) / capacity));
}
For correct results we need to invoke cold start because emit needs to be compiled first and let the Garbage Collector to do it's work before testing.

The results are: approx. 6.2 seconds per 1 million entities and tables with 7 fields of various types. To compare, it takes 5.5 seconds to map the same collection for the direct handwritten mapping. Yeap, I believe,  the results are very good and solution is pretty scalable! Of course, I had to spent some time with dotTrace to optimize the performance but it is worth!

All sources of described EmitMapper customizations are available on GitHub.