If you do any work with xml you probably have come across scenarios where you are using an XmlWriter to produce an output stream of xml. Eventually this output stream is either persisted to disk via an XDocument, sent over the wire using a distributed technology such as WCF, Remoting etc., or possibly transformed with XSL/XSLT. A strong example is custom serialization classes that implement IXmlSerializable.  For example:

The class above is a simple data transfer class (DTO) that implements IXmlSerializable so that it can be serialized and/or deserialized from an objet to an xml stream and vice versa. Note: in most cases you would simple mark the class as [Serializable] and/or provide attributes from the System.Xml namespace to provide the same behavior, however in many cases the default implemention will not fit your particular scenario, hence you would implement IXmlSeriable and provide your own custom serialization.

Here is the 'custom' serialization implementation:

While the XmlWriter/XmlReader API's are pretty simple to use, they are also a bit verbose. If you happen to have a fairly large class with many fields, things start to get ugly pretty fast. Typically when I see large classes, I began to think about refactoring into smaller classes when applicable, but that not always the case. Since, most of them time when want serialization/deserialization you simple want to quickly (i.e. less keystrokes) turn the contents and structure of the class into its xml equivalent you are looking at reducing the amount of work needed. This is where extension methods really come in handy:

The result compared to above is a much cleaner, easier to read class:

While extension methods are not new, they do offer unique way of handling situations where you would like to simplify a set of operations without reaching for the traditional static xxxUtil class or creating a customized implementation or wrapper class. In this case,  XmlWriter is a class open for extension via basic inheritance, unlike a sealed class such as System.String, which is the intended purpose of extension methods: extended classes closed to inheritance (sealed).

Background

I am working on a project that requires a series of actions to be executed against an object and I immediatly thought: pipe and filters! Pipe and Filters is architectural pattern in which an event triggers a series of processing steps on a component, transforming it uniquely on each step. Each step is a called a filter component and the entire sequence of called the pipeline. The filter components take a message as input, do some sort of transformation on it and then send it to the next filter component for further processing. The most common implementations of a pipe and filter architecture are Unix programs where the output of one program can be linked to the output of another program, processing XML via a series of XSLT transformations, compilers (lexical analysis, parsing, semantic analysis, optomization and source code generation) and many, many other uses (think ASP.NET Http stack).

 

The diagram above depicts the actors and message flow. The pump is the event originator which pushes messages into the pipeline to be processed by the individual filter components. The messages flow through the pipeline and become inputs for each filter component. Each filter performs it's processing/transformation/whatever on the message and then pushes the message back into the pipeline for further processing on the next filter component. The sink is the destination of the message after it has been processed by each filter component.

Pipe and Filters Implemented

For me, the best way for me to implement a design pattern is to see how others have implemented it. A quick google search and I came up with the two good examples (actually there are many, many, many examples!):

• http://csharperimage.jeremylikness.com/2009/08/pipeline-and-yield-in-c.html
• http://ayende.com/Blog/archive/2008/01/05/Pipes-and-filters-The-IEnumerable-appraoch.aspx

Both examples take advantage of the newer features of C# such as the Yield keyword (the purpose behind their posts?), which did not apply exactly to my needs. A little meddling however, and I came up with the following:

Here is the final code for the FilterBase<T>:

And the code for the Pipeline<T> class:

Here is rather weak unit test illustrating the usage.

Note that I added a fluent-interface type behavior to the Pipeline<T> class so that you can chain together the registration of the filters and finally execute the chain.

References:

• http://www.dossier-andreas.net/software_architecture/pipe_and_filter.html
• http://www.eaipatterns.com/PipesAndFilters.html
• http://msdn.microsoft.com/en-us/library/ms978599.aspx
• http://wolfbyte-net.blogspot.com/2008/04/pipe-and-filters-fluent-apis-and-linq.html
• http://java.sun.com/blueprints/corej2eepatterns/Patterns/InterceptingFilter.html
• http://home.earthlink.net/~huston2/dp/ChainDecorator.html
  

 

This post nearly completes the API defined in my earlier posts on the DataMapper pattern and the Command Pattern that shows a solution for executing queries against a remote service and mapping the results to POCO objects. The Command pattern implementation gave us a means of creating client requests with various combinations of parameters and allowed the query to be executed against the remote service. The DataMapper allowed us explicitly map the results of the query as a response in the form of an XML stream (SOAP Body) to objects on the client via CLR Attributes. To succinctly complete the union between the data store, the mapping layer and the domain layer we use the Repository pattern: “A Repository mediates between the domain and data mapping layers…Client objects construct query specifications declaratively and submit them to the Repository for satisfaction.”  - Fowler [http://martinfowler.com/eaaCatalog/repository.html]

From an [earlier post], here is the whole, anemic API (with the exception of the WCF client used to communicate with the service):

From the model above, it’s pretty simple: commands are built by the client and executed against the service (the data store in this case), results are then mapped to POCO objects via .NET attributes and a little reflection. The repository becomes an intermediary between domain model, mapping and data store; it manages creation of the command object (and some parameter aspects) and then facilitates the mapping of the result set, hiding much of the heavy lifting (so, to speak) from the client. The client then gets a nice, clean typed object to use complete with intellisense and compiler support; much better than DataSets or XPath and XML.

Depending upon the implementation, the Repository lends itself to a rather simple structure; most of the moving parts involve the objects that it uses internally. Here is the code for the abstract IRepository class:

public abstract class Repository<T> : IRepository<T> where T : IEntit
    {
        protected Repository(IDataMapper<T> dataMapper, IContentCommand command)
        {
            DataMapper = dataMapper;
            Command = command;
        }

        #region IRepository<T> Members
 
        public virtual List<T> GetAll(IQueryRequest request)
        {
            IQueryResponse response = Command.Execute(request);
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.MapAll(reader);
            }
        }
 
        public virtual T Get(IQueryRequest request)
        {
            request.Start = 0; request.Limit = 1;
            IQueryResponse response = Command.Execute(request);
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.Map(reader);
            }
        }
 
        public T Get(IQueryRequest request, out int count)
        {
            request.Start = 0; request.Limit = 1;

            IQueryResponse response = Command.Execute(request);
            count = response.ResultCount;
 
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.Map(reader);
            }
        }
 
        public List<T> GetAll(IQueryRequest request, out int count)
        {
            IQueryResponse response = Command.Execute(request);
            count = response.ResultCount;
 
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.MapAll(reader);
            }
        }
 
        public Dictionary<string, List<IEntity>> GetAll(IBatchQueryRequest queries)
        {
            var results = new Dictionary<string, List<IEntity>>();
            IBatchQueryResponse response = Command.Execute(queries);
            for (int i = 0; i < response.Responses.Count(); i++)
            {
                IQueryRequest query = queries.Requests[i];
                using (var reader = new XmlTextReader(new StringReader(response.Responses[i].Xml)))
                {
                    var mapped = DataMapper.MapAll(reader) as List<IEntity>;
                    results.Add(query.Name, mapped);
                }
 
            }
            return null;
        }
 
        public IDataMapper<T> DataMapper { get; set; }
 
        public IContentCommand Command { get; set; }
 
        public bool EnableServiceLayerCaching { get; set; }
 
        #endregion
    }

Classes that derive from this are equally minimal in that they just override the constructor with appropriate IDataMapper<T> and IContentCommand implementions and they are ready to go. Note that the design supports Dependency Injection (DI) , which facilitates unit testing by providing hooks in which faked or mocked objects can be passed in by passing a direct dependency upon the service layer. My original implementation used a pure file based XML version independent of the remote service.

Command Pattern Background

One of the most ubiquitous software patterns in existence is the Command Pattern:  “Encapsulate a request as an object, thereby allowing for the parameterization of clients with different requests, queue or log requests, and support undoable operations” – GOF. It is generally used in situations where the all of the information necessary for a future call can be ‘built up” sequentially and finally executed at a point in time in the future – i.e. each action is state-full.

A common scenario is an object used to build up a database request taking in several different parameters such as the database name and location, the stored procedure name, any data that needs to be passed to the stored procedure and possibly a transaction object. Once the command has been constructed and all required parts are defined, it is executed, which delegates an action to call the stored procedure on the database and return a result set or perform some operation on an object, with the provided parameters. In the C# world, we are talking about System.Data.DbCommand and its concrete classes such as System.Data.SqlClient.SqlCommand or any other database specific implementation of DbCommand (Oracle, MySql, etc.).

Here is the basic UML diagram of the Command Pattern:

The Command Pattern typically utilizes the following actors:

1. Client – creates the command object providing the relevant information (parameters) needed to fulfill its obligations
2. Invoker – initiates the action on the command so that it’s obligations are fulfilled, making the decision of which of the commands actions should be called
3. Receiver – performs the command object’s obligations, whatever they might be – i.e. makes database call or some other action

The Command Pattern Implemented

The scenario is this: a service exists which takes as input a series of inputs (a query) and returns as output, the results of those inputs as an xml stream of attributes and values. Caching at the service level and several operations are also supported: filtering, sorting, paging, etc.

Each request varies by parameters and is encapsulated as a query object defined by the client. This query object is simply a data structure that contains the fields accepted optionally be the service. The query object is executed by a command object created by the client, who delegates control to the receiver encapsulated by it’s invoking one of its action methods.

The following diagram illustrates the implementation:

 

 

The Command implementation is very simple. It contains an ICommandReceiver reference and IQueryRequest reference which stores a reference to the current query being executed. The command delegates control to the receiver, which performs its work and returns back an IQueryResponse object containing the results of the query.

 

The ContentReceiver class implements ICommandReceiver interface and simply makes the call to the remote service via a WCF client. This happens to be a very simplistic implementation in that much of the heavy lifting (i.e. caching, query generation, etc.) occurs in the service. In this case, we are simply translating a client query into a SOAP request to access the services resources.

The client uses an IQueryRequest object to build up a set of parameters which are consumed by the service endpoint via the command object. The results are returned to the client with an IQueryResponse object. How the results are used is dependent upon the client.

Finally, a simple unit test illustrating the usage:

The DataMapper Pattern is a well documented and defined pattern for abstracting away an object’s storage from its in-memory representation. For the most part the pattern explicitly defines the domain of the pattern to be relational databases and object’s that map to there schema, however the pattern is adaptable to non-relational data-stores as well. In fact to applicable to any situation where the need to map records (related data units) to in memory objects; regardless of structure or underlying store.

Imagine, for instance, a service that is a façade for a complex matrix of unstructured key value pairs and a protocol for clients to query the service for tuples. For example:

• For the make ‘honda’, give me all ‘years’ 
• For the article ‘2311’, give me ‘title’, ‘description’, ‘author’, ‘publication_date’

This is not too much different from any relational database model, with the exception that there is no direct joining occurring to return back columns that occur in different tables and relate them to the input set. It’s basically just a giant hash-map of name/values pairs with an undefined structure…i.e. there is no way to directly map it to a domain model; what you supply as inputs, defines the structure of your output.

That’s where the DataMapper Pattern comes into play. According to Fowler, the DataMapper is a “a layer of Mappers that move data between object and a database while keeping them independent of each other and mapper itself”. The core motivation behind the pattern is separation of concerns; an abstraction between the data, the store, and entity objects themselves. By using a DataMapper we can associate the name/value pair scenario described above with our domain objects, providing structure to our model. This is important because we go from an untyped hash-map model to tangible objects (POCO) that are typed, thus allowing us the benefits of developer friendly features of our IDE, such as ‘intellisense’ and compile time type safety.

Implemented within an API, you get something that looks like the following:

Think of it as a minimalistic ORM. In this case such concerns as caching and transactions are the responsibility of the underlying data-store service. As a consumer we are only concerned with making a request for data, getting a result set, and mapping it to an entity or domain object. In the diagram above, there are actually three different patterns working to create the whole: the command pattern, the repository pattern, and the data mapper pattern. All three comprise the entire API.

A Simple DataMapper Implementation:
This implementation of a DataMapper really does just three things: a) it reads through an XML stream, b) it maps to the values of specific XML attributes to an associated property, and c) packages the results into either a collection or a single object of the Type provided defined in the generic argument:

Map maps a single result to a single object. MapAll maps multiple results to a list of objects, and Read is a private method that extracts all key value pairs (XML attributes and there associative values) to a dictionary.

All Read does is extract out the XML attribute names and there associative data and store them in a dictionary. The dictionary is then used by the Map and MapAll methods along with a System.Attribute class to map the values directly to the correct properties:

In the code snippet above, after the Read method has extracted the data for mapping we loop over each of the objects properties looking for a FieldMappingAttribute on each of the properties we have flagged for mapping. If a match is found the relevant data is mapped directly to the object’s property and then the object is returned.

 

MapAll does essentially the same as Map, but assumes that the stream contains more than one object definition and then returns the resulting collection.

A couple of things to note here, the code a) needs a little refactoring to improve performance and extensibility, b) we are assuming a relatively flat xml file streamed here, no nesting of elements for example and c) we are only supporting properties of the Type System.String . The simplicity is purposeful…when I need it I’ll add it.

One last thing to discuss is implementation of FieldMappingAttribute and its usage. FieldMappingAttribute is a simple class deriving from System.Attribute that allows us to decorate properties on our entity classes with attributes that define our mapping schema:

The usage is pretty simple and should be familiar to most .NET developers:

References:

• http://www.martinfowler.com/eaaCatalog/dataMapper.html
• http://msdn.microsoft.com/en-us/magazine/dd569757.aspx

I was trying to test a IEqualityComparer implementation today, when Resharper kept crashing:

 

I tried rebooting, rebuilding everything. I finally clicked 'Debug' and waited...nothing happened after a few minutes, so I tried re-installing Resharper to no avail. By this time I was bruning through  my day, running out of time (deadlines, blah).

I clicked 'Debug' again and once again nothing happened....for awhile. I got up and ran a few 'errands' and came back an viola, I found my problem:

 

It was just a StackOverFlowException caused by my poor implementation of IEqualityComparer. Once I saw this I knew immediatly what was going on and how to resolve it. Of course, Resharper stopped crashing after I fixed the bug.

The big issue was why Resharper didn't handle this and just display the stacktrace as usual. It might be related to the fact that the exception was generated in TypeMock.dll, AFAIK I was not using TypeMock for this Unit test and there are no references to it in my project...weird

I was writing some unit tests for a project I am currently working and kept getting the following non-useful exception:

I have seen the error before many times in the past, but I could not put my finger on what was causing it. The code is pretty simple, it just creates and instance of an object that implements IXmlSerializable passing in a stream which internally adds additional chicled object to the stream and then loads the result into a XmlReader. The the XmlReader then outputs the results to the standard console (note that a little later I add assertations to validate that the data was 'extracted' correctly):

 

Needless to say, this code failed every time on XDocument.Load with the 'Root element missing' exception.  I vaguely remember getting this error sometime in the past, but couldn't put my finger on it. I finally ran across this blog post and a light bulb went off.

After writer.Flush() is called the position of the stream is EOF. Before you can read the stream you must set the position of the stream to the begining. Note that calling writer.BaseStream.Position = 0 does _not_ work. You must call writer.BaseStream.Seek(0, SeekOrigin.Begin) to reset the streams position before creating the reader and loading the XDocument object. Also, note that the error only occurs because the XDocument object thknks the XML stream is invalid.

ASP.NET MVC introduces the notion of partial views, which either allow specific Actions to be called on a Controller and the resulting view outputted as HTML or allow Model Data to be passed from the current view to another, child view.

Calling Actions on a Controller and returning “sub” Views is incredible useful in situations were Ajax type behavior is required, since the resulting output is simply raw HTML. The HTML can then be appended to a DOM element via JQuery, some other JavaScript library or simply with straight JavaScript.

Unfortunately if you are still working on Web Forms projects or you have no need or desire to drink the kool-aid of MVC, you are pretty much out of luck for similar functionality built into Web Forms. However, it is fairly easy to implement this sort of behavior within the context of a Web Forms environment and then use a Ajax to render pure HTML from these “partial views.”

The code for doing this is surprising simple and was adapted from a Scott Guthrie posts a few years back:

In this case I allow a Dictionary containing the name value pairs of the properties on the View that will be loaded to be passed in as well. The key must match the name of a Property on the View and if it does the Property is set with the value of the key. This is useful in cases were you would like a parent View to pass data to a child View for initialization situations, etc.

I also have a static collection of Views defined which map a View’s name to a path in my applications root directory. RenderView looks up its path and loads the control before setting the Views data. In my case I load this from a table via a Data Access Object (DAO) in the Global.asax on Application_Start:



The PageUserControlDAO loads the View definitions from the database once when the application is started or the App Pool is restarted/refreshed.

Rendering the View involves calling the RenderView method and passing in the appropriate View name and a Dictionary containing the data to into the view:

In this case I used Page_Load to render the View into a panel (div) which isn’t too exciting. The really useful aspect however is when you combine JQuery and Web Services to render the View on demand from the client.

Rendering Views from the Client Using JQuery and Web Services

The Web Service:



The JQuery code to load the view and attach it to the DOM:

In the JavaScript above, renderView takes three parameters: the name of the View to render, the data to pass to the View, and the DOM container to load the view into. The $.ajax(..) method is a JQuery method that specifies the WebService to use, the method to call (RenderView) and the content type to specify in the request header.

When the method is called like so from the client:

 

The WebService method is invoked, which renders the view and returns pure HTML that is then appended to the DOM element $view (a simple div or span).