Simple Pipe and Filters Implementation in C# with Fluent Interface Behavior

by jmorris 16. September 2009 21:51

Background

I am working on a project that requires a series of actions to be executed against an object and I immediatly thought: pipe and filters! Pipe and Filters is architectural pattern in which an event triggers a series of processing steps on a component, transforming it uniquely on each step. Each step is a called a filter component and the entire sequence of called the pipeline. The filter components take a message as input, do some sort of transformation on it and then send it to the next filter component for further processing. The most common implementations of a pipe and filter architecture are Unix programs where the output of one program can be linked to the output of another program, processing XML via a series of XSLT transformations, compilers (lexical analysis, parsing, semantic analysis, optomization and source code generation) and many, many other uses (think ASP.NET Http stack).

 

 
 Pipe and Filters Conceptual

The diagram above depicts the actors and message flow. The pump is the event originator which pushes messages into the pipeline to be processed by the individual filter components. The messages flow through the pipeline and become inputs for each filter component. Each filter performs it's processing/transformation/whatever on the message and then pushes the message back into the pipeline for further processing on the next filter component. The sink is the destination of the message after it has been processed by each filter component.

Pipe and Filters Implemented

For me, the best way for me to implement a design pattern is to see how others have implemented it. A quick google search and I came up with the two good examples (actually there are many, many, many examples!):

Both examples take advantage of the newer features of C# such as the Yield keyword (the purpose behind their posts?), which did not apply exactly to my needs. A little meddling however, and I came up with the following:

 
Simple Pipe and Filters Implementation

Here is the final code for the FilterBase<T>:

And the code for the Pipeline<T> class:


Here is rather weak unit test illustrating the usage.

Note that I added a fluent-interface type behavior to the Pipeline<T> class so that you can chain together the registration of the filters and finally execute the chain.

References:

 

Tags: , , ,

Be Careful with MemberWiseClone and Reference Fields and ICloneable

by jmorris 8. September 2009 21:01

A common scenario most programmers have faced is the need to clone a cached object, so that when the object is modified on a per request basis - the cached 'orginal' object is not modified. For example, imagine that you have an XML file on disk and an in-memory object that contains the contents of the file (for lists, etc.). If the cost associated with opening the file (IO) and serialaizing is contents into the in-memory object are great and the cost of actually creating a copy of the in-memory object are small, then cloning the objt per request request, modifying it and then discarding per request is the way to go.

However, one small gotcha exists with Object.MemberWiseClone: while it actually creates a brand new object, it copies the references of non-primitive fields from the original object to the new object! So if you are not expecting changes to those fields, you are in for a big surprise:

 

After cloning you would expect that the original and cloned object would be disticnt objects with distinct object graphs, but that is not the case. Note that the Template object implements ICloneable and that templates.CloneTemplate(...) gets the orginal object from a dictionary and then makes a copy of it by calling Template.Clone() which uses MemberwiseClone() to create a copy of itself and return it to the caller.

Note that the MSDN documentation itself describes this behavior with reference type fields and MemberwiseClone():

"The MemberwiseClone method creates a shallow copy by creating a new object, and then copying the nonstatic fields of the current object to the new object. If a field is a value type, a bit-by-bit copy of the field is performed. If a field is a reference type, the reference is copied but the referred object is not; therefore, the original object and its clone refer to the same object." - MSDN

The solution is not use MemberwiseClone in your ICloneable implementation, unless you specifically want this kind of behavior and I am guessing that most folks want a real copy and not a shallow copy in which the object graph of the member variables points to the original objects roots. For example, replace this:

 

   

        #region ICloneable Members

 

        object ICloneable.Clone()

        {

            return MemberwiseClone();

        }

 

        public Template Clone()

        {

            return ((ICloneable)this).Clone() as Template;

        }

 

        #endregion

 

 

With the something like the following:

 

 

 

Tags:

Using the Repository Pattern with the Command and DataMapper Patterns

by jmorris 1. September 2009 22:00

This post nearly completes the API defined in my earlier posts on the DataMapper pattern and the Command Pattern that shows a solution for executing queries against a remote service and mapping the results to POCO objects. The Command pattern implementation gave us a means of creating client requests with various combinations of parameters and allowed the query to be executed against the remote service. The DataMapper allowed us explicitly map the results of the query as a response in the form of an XML stream (SOAP Body) to objects on the client via CLR Attributes. To succinctly complete the union between the data store, the mapping layer and the domain layer we use the Repository pattern: “A Repository mediates between the domain and data mapping layers…Client objects construct query specifications declaratively and submit them to the Repository for satisfaction.”  - Fowler [http://martinfowler.com/eaaCatalog/repository.html]

From an [earlier post], here is the whole, anemic API (with the exception of the WCF client used to communicate with the service):

From the model above, it’s pretty simple: commands are built by the client and executed against the service (the data store in this case), results are then mapped to POCO objects via .NET attributes and a little reflection. The repository becomes an intermediary between domain model, mapping and data store; it manages creation of the command object (and some parameter aspects) and then facilitates the mapping of the result set, hiding much of the heavy lifting (so, to speak) from the client. The client then gets a nice, clean typed object to use complete with intellisense and compiler support; much better than DataSets or XPath and XML.

Depending upon the implementation, the Repository lends itself to a rather simple structure; most of the moving parts involve the objects that it uses internally. Here is the code for the abstract IRepository class:

public abstract class Repository<T> : IRepository<T> where T : IEntit
    {
        protected Repository(IDataMapper<T> dataMapper, IContentCommand command)
        {
            DataMapper = dataMapper;
            Command = command;
        }

        #region IRepository<T> Members
 
        public virtual List<T> GetAll(IQueryRequest request)
        {
            IQueryResponse response = Command.Execute(request);
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.MapAll(reader);
            }
        }
 
        public virtual T Get(IQueryRequest request)
        {
            request.Start = 0; request.Limit = 1;
            IQueryResponse response = Command.Execute(request);
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.Map(reader);
            }
        }
 
        public T Get(IQueryRequest request, out int count)
        {
            request.Start = 0; request.Limit = 1;

            IQueryResponse response = Command.Execute(request);
            count = response.ResultCount;
 
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.Map(reader);
            }
        }
 
        public List<T> GetAll(IQueryRequest request, out int count)
        {
            IQueryResponse response = Command.Execute(request);
            count = response.ResultCount;
 
            using (var reader = new XmlTextReader(new StringReader(response.Xml)))
            {
                return DataMapper.MapAll(reader);
            }
        }
 
        public Dictionary<string, List<IEntity>> GetAll(IBatchQueryRequest queries)
        {
            var results = new Dictionary<string, List<IEntity>>();
            IBatchQueryResponse response = Command.Execute(queries);
            for (int i = 0; i < response.Responses.Count(); i++)
            {
                IQueryRequest query = queries.Requests[i];
                using (var reader = new XmlTextReader(new StringReader(response.Responses[i].Xml)))
                {
                    var mapped = DataMapper.MapAll(reader) as List<IEntity>;
                    results.Add(query.Name, mapped);
                }
 
            }
            return null;
        }
 
        public IDataMapper<T> DataMapper { get; set; }
 
        public IContentCommand Command { get; set; }
 
        public bool EnableServiceLayerCaching { get; set; }
 
        #endregion
    }

Classes that derive from this are equally minimal in that they just override the constructor with appropriate IDataMapper<T> and IContentCommand implementions and they are ready to go. Note that the design supports Dependency Injection (DI) , which facilitates unit testing by providing hooks in which faked or mocked objects can be passed in by passing a direct dependency upon the service layer. My original implementation used a pure file based XML version independent of the remote service.

Tags: , , , ,

Jeff Morris

Tag cloud

Month List

Page List