Simplicity Is A Virtue

Life is really simple, but we insist on making it complicated.
-Confucius

About a year ago, my team and I were working on a feature that turned out to be complex and unpleasant to implement. Multiple system dependencies, complex user interactions, unreliable third-party endpoints with which to integrate – the works. As we often do at Wetpaint, we had a junior and a senior members of the team working together on different parts of the feature. We were going through a code review round. Our senior engineer was visibly uncomfortable with some of the code “smells” caused by the overall complexity and wanted to simplify things, even if it took extra time to do. Being a team joker, he said: “Imagine that the next maintainer of this code is a serial killer who knows where you live. You don’t want him to get angry, right?” We fell over laughing, continued to listen to our instincts and worked to considerably simplify our code.

Looking at this funny incident now, I appreciate even more how undeniably prudent and forward-looking this advice is. Simplicity is a virtue. We should try to maximize it.

Simplicity is the ultimate sophistication

Simplicity is the ultimate sophistication

When I say “simplicity”, I realize how relative this term is.
Nowadays, in addition to “serial killer” test, I apply one more: can somebody new to this code but not new to the problem pick up the gist of what you built quickly? Over lunch? Over a day? When you have moved on to other things, teams, companies and cities?
This requires, in my mind at least, that I and my team try to follow these simple guidelines:

  • Be intentional. Naming things right is an art that must be mastered. Calling things for what they are has extraordinary importance. I make time to allow myself to think of a good name for that method or that class. Among other things, it reduces the need for comments I have to put in my code (more on that below).
  • Be brief but not obscure. If anything can be expressed in a few lines of code and still well-understood by a relative newbie, that’s a perfect balance. I don’t believe that stuffing 50 things into one line of amazing C++ is as useful.

  • Be fluent. I frequently consider returning self (or this) as method results, so that calls can be chained together into English-like sentences:

def service
  GoogleAnalytics.
    with_credentials(credentials).
    using_caching(interval).
    using_table(all_traffic)
end
  • Delegate responsibility to an Information Expert. I follow this pattern as much as I can, while deferring decisions about which component will handle each part of the execution until last possible moment.

  • Refactor. More often than not, I realize that initial class and method structure won’t work as I get deeper into development. As soon as this is discovered, I believe that trying to force the issue will cause more discomfort and waste more time than doing a quick refactory. Refactoring shipped code is 10-100 times more expensive than doing it during feature development.

  • Avoid comments. I know this is controversial. But consider the fact that most of the comments we see in code are either obvious and redundant (once you actually read the code) or, much worse, are placed in blocks of code that are so complicated and unreadable, comments don’t add value anyway. Therefore, over the years, I’ve grown accustomed to a style of programming when the only comments I should need are the ones from which we generate documentation. Everything else, most of the time, is a sign that naming is screwed up or complexity is too high. So I either refactor or rename things to signal intent better that a comment could.

  • Don’t build frameworks right away. Until I see the same code pattern repeat itself in the project multiple times, the simplest approach is the best. Once I begin noticing repetitious code, I DRY up my code up paying attention to varying requirements and giving my DRY pieces just enough flexibility to handle the needs of the project. Thus, a framework emerges organically, out of real code needs.

  • Keep learning experiments out of production code. The fact that I just learnt how to use, say, Backbone.js does not neccessarily mean that my entire project needs to be re-built with it just because I want to learn how to apply my new shiny knowledge. Best tool for the job principle still applies.

  • Be test-driven and test-obsessed. I strive to write tests not just to verify behavior of my code, but to explain to others intricate details of how my code functions. To do that, I try to be thorough with test coverage, mock or stub liberally, be very intentional in how I name my tests, and very careful with what expectations I assert at the end. This also helps with reducing amount of comments and instead replacing them with working test code.

I am not naive to think that every problem can be solved with these simple recipes. Some problems are naturally very hard and require big chunks of very complex code. But I am advocating going the extra mile and doing everything possible to make things simpler for people who will work with our code with us and after us.

Because overly complex, unreasonably “cool”, or utterly unmaintainable code quickly becomes legacy code. Legacy code that nobody wants to touch. Legacy code that does magical things nobody understands. Legacy code that is dead weight capable of sinking any software product.

Happy coding!

How Chaos Kills Engineering Team Productivity

We chose technology as our career, and technology, with all its beauty, is complicated, messy, and changes with neck-breaking speed. We have hectic lives.

Many of us deal with this constant change by compartmentalizing our time and reducing number of interruptions to our workflows. This works extremely well, to a point.

As Engineers and Product Managers, we are passionate about products we are building. We want them to be useful, beautiful, fast and technologically sound. In broad strokes, engineers trust PMs to lead “useful and beautiful” efforts, while PMs trust engineers to take care of “fast and technologically sound” bits. And as we know, quality of this partnership can make or break any product, and single most assured way to ruin a product is to introduce chaos into this arrangement.

Chaos is not the same thing as change, changing requirements, or occasional fire fights with systems being down. Change is OK. Change is good. There are well-known methodologies that allow us to deal with change.

Too much chaos kills team productivity

Too much chaos is poisonous, it kills team productivity
Image by T0mek | Stock Free Images

Chaos is something different – it is activity that does not contribute to product goals. Some of it comes from not having clearly defined goals in the first place, and some of it comes from not having enough constraints, allowing us to lose focus and waste time on things that aren’t important. Trying to second-guess others and work around them is contributing to chaos. Not relying on data for insights is chaos. Scope creep and feature favoritism is chaos. Untidy task management practices are chaos. Breaking build all the time is chaos. Not looking at bugs every morning and missing a critical one is chaos. And chaos is poison.

As technologists, we can’t help it – we constantly think about how to make our products better. Even outside of the office. Even as we sleep. Engineers think about their code, PMs think of new ways to make users’ lives better. We think of a right time to bring back something that was discarded a while ago into “not the right time” bucket. We look at data and have heated discussions about what it represents and how to move forward. We argue about design and other things. That’s how we build beautiful, inspiring products that people love to use. But this is only possible when fundamentals are solid and we don’t have to constantly worry about small things or take care of constant “Sky is falling, let’s get moving” situations.

We still deal with small, everyday problems that come up from time to time, this is unavoidable.  However, when they are relentless and omnipresent with increasing volume, they begin to occupy our mind, and forward progress on the project is lost: death by thousand cuts. All because of chaos.

So, how can we deal with this problem? By introducing culture of autonomy and building cohesive teams that like working with each other. We need to hire best people and let them work their magic, welcome and give honest feedback, don’t hesitate to ask for help, and above all, trust our team mates to have our back. Here are some good things that I’ve seen and practiced in many places, and things we live by at Wetpaint:

  • Our PMs take time to look at the data and say “no” to their understandable desires to cram as many features as possible into the product.
  • Our PMs don’t dictate how to implement features, but they are technically savvy and of course have an opinion. So they ask engineers to educate them about why certain things are implemented this way and what were the alternatives.
  • Our engineers watch builds like hawks and everybody obsesses about automation, unit tests and code coverage.
  • Our Test Team can openly ask others to help verify and close down open tasks as soon as sprint begins. Our Dev and PM team members gladly volunteer their time to help out.  When Test Team feels that our new features lack quality, they speak up and we, slightly embarrassed, fix things.
  • Our technical team can openly request for a feature to be cut from current sprint because interruptions came in or other features were under-estimated. PMs are happy to discuss alternatives and move things around. In return, Dev team becomes accountable for screw-ups with estimates.
  • Our prior decisions are reevaluated and bad ideas are killed in cold blood given new evidence in the form of data.

Our single most precious gift to our project partners is shielding them from chaos and letting them think. And work. We bet our careers on each other, so we need to help make each other successful. And we don’t have much hope for an emergence effect to arise from chaos, because our team is not an ant colony.

HealthCheck: Software Monitoring Inside Out

This post was originally published on Geekwire. I am republishing it here for the followers of this blog.

When we release software to customers, we immediately begin to consider how to monitor it. The goal is to find problems as early as possible, and always before our customers. We start with the basics: disk, memory, CPU load. Still, because of the complexity of the products we develop, there are countless examples of things going wrong: the system fails and we hear about it hours later.

System Components are like firing neurons

System components are like firing neurons – connected and hard to monitor. Photo credit: iStockphoto.com/ktsimage

Monitoring software behavior delivers serious value only when it captures deeper system metrics which can be analyzed to gain interesting operational and business insights. To measure these more advanced key performance indicators, we need to have access to internal components and complex data. External monitoring typically does not have sufficiently deep system access and therefore cannot provide answers to questions like these:

  • For an online shopping system, how many orders were received in the past hour? Is this significantly different compared to an average day?
  • For a publishing system, how much time elapses between submitting an article and that article going live? Does it have unusual spikes?
  • For a patient registration system, how many messages are sitting in its queues? What is an average wait time for a message?
  • For an online experimentation system, how long does it take to compute experiment results? Does this delay ever become unacceptable given traffic projections?
  • For a point of sales system, how many retries does it take to submit data to a payment gateway? Is that number unusual?

In summary, we need a way to boil down advanced system statistics to a definitive and actionable answer to one single question: Is our system healthy?

At Wetpaint, where we build a social platform for publishers, all our systems have a special /healthcheck URL. This endpoint is the main asset of our external monitoring system, Nagios, which we use for advanced checks. If it gets no response or an unacceptable response, the system is unhealthy.  The Ops person gets an alert and the Developer gets a call. Together, the team begins investigating what’s wrong. When they log into the system, the same /healthcheck endpoint produces a detailed report — and, if there are failures, it provides specific instructions to follow to troubleshoot the problem.

Wetpaint Health Status Report

Health Status Report for Wetpaint Social Distribution System.

We built our systems to self-monitor: we know how long it takes to submit a scheduled Facebook post or a tweet, how many of them failed to get submitted during the last 24 hours, whether content ingestion from our partners is working and up to date, how long it takes for data analysis and statistics jobs to finish computation, and countless other metrics. This approach shifted reliability worries entirely to the engineering team, where it belongs. It made the Ops job simpler, providing an answer to the ever important question: “I got a monitoring alert. Do I need to call a Dev or not?”

When building each of the checks, we asked ourselves:

  • Can this situation be systematically prevented?
  • In the future, can this situation be detected early? What data do we need to collect to ensure that?
  • Is there a clear, actionable set of steps that should be taken by a Support Engineer to fix the issue? Can these steps be automated?

These questions required deep design changes to our system. We ensured that everything leaves enough of a trace about its functionality, so /healthcheck can deterministically discover when something is broken. To do this, we focused on two crucial requirements:

  1. Precision to eliminate false positives and a “cry wolf” approach to alerts
  2. Actionable insights. The key is to give a specific roadmap to fix the problem at hand. After all, we woke somebody up in the middle of the night with an alert, the least we could do is to give them enough information to be successful in fixing the problem.

Over time, we arrived at a design that allows us to see the earliest signs of trouble long before anything becomes visible to our customers, and helps us focus on larger business questions. When we get a “Healthcheck Red” alert, we don’t stop working on the problem unless /healthcheck turns green, and we file it immediately as a severe interruption, which we track.

The journey of discovering where our systems can fail is long and tedious, but worthy and important. The trick is not to expect absolute perfection, but to protect ourselves from catastrophic failures by using deep, comprehensive monitoring and early detection. Just like in real medicine, discovering a problem early and systematically eliminating the root causes of ailments are the keys to successful treatment.

Why do Code Reviews?

Interestingly enough, this question came up in a conversation related to a recent event: severe bug escaped into the wild causing embarrassing data loss. The bug was a regression that was introduced and not noticed during a code review. Regression suite failed to catch it as well. Naturally, we were trying to understand what we can do better. One of the questions was related to code reviews: could a more thorough review have prevented this?

To our surprise, we discovered that not everybody agrees on goals of a code review as well as on his/her right and responsibilities as a code reviewer.

So, I came up with these code review goals that are now posted in our office:

Verify

  • Maintainability and best practices/coding guidelines
  • DRY, Design patterns, Readability, Testability
  • Test coverage
    • Very important for both new features and bugs
    • Evaluate exposure to missing tests
    • Tests should highlight edge cases + happy paths

Learn

  • Be able to take over the ownership
  • Understand requirements, context, and code

Help the author become better

  • Suggest alternative approaches, defend your decisions

Sign-off = Your Reputation on the Line

Creating Testable WCF Data Services

One of the effective ways to expose your data on the web is to use WCF Data Services and OData protocol.

In its simplest form, a WCF Data Service can be built by exposing your ADO.NET Entity Framework ObjectContext through the DataService<T> class.
However, when it comes to testing, especially in situations when you have to enhance data in the OData Feed with some additional information (through WCF Behaviors for example), or when you have to verify that your Query Interceptors work as designed, using Object Context that is bound to a live database is hard because:

  • Your tests that have little to do with the shape of your data become dependent on the database, which is a big problem
  • You have maintain test data in the database when you run your tests, which is a maintenance overhead

So there lies our problem: how to expose data using WCF Data Services in a way where the database dependency can be mocked out during unit testing? The short answer is that we need to achieve better testability by introducing persistence-ignorant domain layer and by making our existing Object Context implement an interface that can be mocked/substituted in testing.

This article describes the concrete incremental steps you can take to get you from a standard Data Service built on top of Entity Framework Object Context to a solution that will satisfy testability requirements above. I will demonstrate the technique on a sample service connected to the AdventureWorksLT database.

Initial State: Data Service Of Object Context

public class MyDataService : DataService
{
    public static void InitializeService(DataServiceConfiguration config)
    {
        config.SetEntitySetAccessRule("Products", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductCategories", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModelProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModels", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderHeaders", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderDetails", EntitySetRights.All);

        config.UseVerboseErrors = true;
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
    }
}

Step 1: Create an Interface that represents you Data Repository

This step is easy: For every entity type you expose in your Data Service, add a method that returns IQueryable<T>:

namespace MyDataService.DataAccess
{
    public interface IMyDataRepository
    {
        IQueryable</pre>
<address>Addresses { get; }
 IQueryable Customers { get; }
 IQueryable CustomerAddresses { get; }
 IQueryable Products { get; }
 IQueryable ProductCategories { get; }
 IQueryable ProductDescriptions { get; }
 IQueryable ProductModelProductDescriptions { get; }
 IQueryable ProductModels { get; }
 IQueryable SalesOrderHeaders { get; }
 IQueryable SalesOrderDetails { get; }
 }
}

After interface has been created, you can provide default/production implementation in the hand-written partial class that corresponds to your Object Context by just returning Object Sets already defined by you Object Context:

namespace MyDataService.DataAccess
{
    public partial class AdventureWorksLTEntities: IMyDataRepository
    {
        IQueryable IMyDataRepository.Products
        { get { return this.Products; } }

        IQueryable IMyDataRepository.ProductCategories
        { get { return this.ProductCategories; } }

        IQueryable IMyDataRepository.ProductDescriptions
        { get { return this.ProductDescriptions; } }

        IQueryable IMyDataRepository.ProductModelProductDescriptions
        { get { return this.ProductModelProductDescriptions; } }

        IQueryable IMyDataRepository.ProductModels
        { get { return this.ProductModels; } }

        IQueryable IMyDataRepository.SalesOrderHeaders
        { get { return this.SalesOrderHeaders; } }

        IQueryable IMyDataRepository.SalesOrderDetails
        { get { return this.SalesOrderDetails; } }

        IQueryable</pre>
<address>IMyDataRepository.Addresses
 { get { return this.Addresses; } }

 IQueryable IMyDataRepository.Customers
 { get { return this.Customers; } }

 IQueryable IMyDataRepository.CustomerAddresses
 { get { return this.CustomerAddresses; } }
 }
}

Now you have an interface in your system that defines data access methods already implemented by your Object Context.
This means that if you build the rest of the system to rely/depend on this interface, rather than use its implementation (i.e. the Object Context), you can achieve better flexibility and the testability without having to worry about test databases and policing test data in your build environment and on developers’ workstations.

This is a worthy and very achievable goal, but we have a few steps remaining to fully get us there.

Step 2: Change Code Generation Template to use Self-Tracking Entities

This step is needed because you typically do not want you entities to be database-bound, and instead you want a reusable, independent domain layer that can be used in multiple scenarios. This move will contribute to a better code flexibility and layering without jeopardizing data access functionality that Entity Framework provides out of the box. Full discussion on why separate persistence-ignorant domain layer is a good thing is out of scope for this article. If you want to research the subject further, I would start herehere and here.

Adding Self-Tracking Entities Code Generation

Having done that, your Project containing Data Context will look differently:

  • You will see AdventureWorks.Context.tt and code-generated Object Context in the AdventureWorks.Context.cs and AdventureWorks.Context.Extensions.cs
  • You will see AdventureWorks.tt and code-generated entity classes that match you Entity Framework Model.

Note that entity classes do not have any dependency on the EntityObject and therefore do not have to live in your Data Access layer.

Self-Tracking Entities: Generated Code

Step 3: Move Your POCO Entities into a separate “Domain” assembly

Now that we have a domain layer of self-tracking plain old CLR objects (POCOs), we want to make it reusable.
This means separating the entity classes into a separate assembly that does not have any dependencies on Entity Framework, System.Data or any other persistence technologies. The tricky part is that we want to still get the benefits of EF designer experience, i.e. if we added, removed or modified some entities, we want them to be re-generated the usual way.

To achieve that, follow these steps:

  • Create a new Project where you intend to host your Domain Layer. Reference System.Runtime.Serialization.
  • Right-Click on the Entity Code Generation template (AdventureWorks.tt) and select Exclude From Project.
  • In your Domain Layer project, add existing Entity Code Generation Template (located in EDMX folder) as a link.Visual Studio will regenerate Domain Classes in the correct namespace that matches your Domain Layer:Seperate Domain Layer
  • Reference your new Domain project from Data Layer and from the Web Project hosting your Data Service.
  • Add using statements for your Domain namespace to fix compiler errors.
  • You will notice that one of the places you had to add using statement is you code-generated AdventureWorks.Context.cs. Since this class is being generated every time, it is better to add additional using statement to the .Context.tt template:
    EntityContainer container = ItemCollection.GetItems().FirstOrDefault();
    if (container == null)
    {
        return "// No EntityContainer exists in the model, so no code was generated";
    }
    
    WriteHeader(fileManager, "MyDataService.Domain");
    BeginNamespace(namespaceName, code);
    

Step 4: Create your own Data Context that composes your Data Repository

In the Data Access project, add a class that will represent new context will use in the Data Service.
This class replaces Entity Context used before, and therefore it should implement all properties that return IQueryable instances for the Data Service. Since our Repository already implements all this, the implementation is trivial:

namespace MyDataService.DataAccess
{
    public class MyDataContext
    {
        public IMyDataRepository Repository { get; private set; }

        public MyDataContext(IMyDataRepository repository)
        {
            if (repository == null)
            {
                throw new ArgumentNullException("repository", "Expecting a valid Repository");
            }

            this.Repository = repository;
        }

        public MyDataContext(): this(new AdventureWorksLTEntities())
        {
        }

        public IQueryable</pre>
<address>Addresses
 {
 get { return this.Repository.Addresses; }
 }

 public IQueryable Customers
 {
 get { return this.Repository.Customers; }
 }

 public IQueryable CustomerAddresses
 {
 get { return this.Repository.CustomerAddresses; }
 }

 public IQueryable Products
 {
 get { return this.Repository.Products; }
 }

 public IQueryable SalesOrderDetails
 {
 get { return this.Repository.SalesOrderDetails; }
 }

 public IQueryable SalesOrderHeaders
 {
 get { return this.Repository.SalesOrderHeaders; }
 }

 public IQueryable ProductCategories
 {
 get { return this.Repository.ProductCategories; }
 }

 public IQueryable ProductDescriptions
 {
 get { return this.Repository.ProductDescriptions; }
 }

 public IQueryable ProductModelProductDescriptions
 {
 get { return this.Repository.ProductModelProductDescriptions; }
 }

 public IQueryable ProductModels
 {
 get { return this.Repository.ProductModels; }
 }
 }
}

Step 5: Update Data Service to use custom Data Context

[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class MyDataService : DataService
{
    public static void InitializeService(DataServiceConfiguration config)
    {
        config.SetEntitySetAccessRule("Products", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductCategories", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModelProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModels", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderHeaders", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderDetails", EntitySetRights.All);

        config.UseVerboseErrors = true;
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;
    }
}

One you do that, Data Services infrastructure will switch from Entity Framework provider to a Reflection Provider.
There are subtle but important differences between the two. One of the symptoms of those differences is that Reflection Provider does not understand POCO’s the same way Entity Provider understands Entities.
If you run you service as is, you will get a bunch of errors stating that there are some unsupported properties.
To fix those errors, you would have to create partial classes for your Domain entities that do two things:

  • Explicitly define data service keys
  • Exclude Change Tracker from list of properties available for the Data Services to pick up
[DataServiceKey("AddressID")]
[IgnoreProperties("ChangeTracker")]
public partial class Address
{
}

Finally, verify that your service still works by testing it in the browser.
You should be able to perform any operation you did before, but now you can also Unit Test your service without having to have a live database to talk to!

Step 6: For testing, inject Mock Data Repository

To do that, we will need to figure out how to “legally” let Data Service know that we want it to use our implementation of the Data Repository
instead of default one that connects to a real database using Entity Framework. One way to do that is to use Inversion of Control and Dependency Injection. However, making Data Service and WCF aware of IoC Containers is a tricky proposition – there is a simpler solution

What if we created a factory method and used it to create instances of the custom context whenever the Data Service needs one? Fortunately, WCF Data Service provides a way to do that by overriding CreateDataSource method in the Data Service class:

public static Func DataContextFactory;

protected override MyDataContext CreateDataSource()
{
    return DataContextFactory != null ? DataContextFactory() : base.CreateDataSource();
}

Now our test code can set the Factory to create Data Context already initialized with a mock Repository.

This is, however, is only part of the problem – we also want to be able to host the Data Service inside of the test process without having to deploy anything to a real web server.
Once again, this is solvable by creating in-process service host:

public static class TestServiceHost
{
    internal static DataServiceHost Host
    {
        get;
        set;
    }

    internal static Uri HostEndpointUri
    {
        get
        {
            return Host.Description.Endpoints[0].Address.Uri;
        }
    }

    ///
<summary> /// Initialize WCF testing by starting services in-process
 /// </summary>
    public static void StartServices()
    {
        var baseServiceAddress = new Uri("http://localhost:8888");
        Host = new DataServiceHost(typeof(Web.MyDataService), new[] { baseServiceAddress });
        Host.AddServiceEndpoint(typeof(IRequestHandler), new WebHttpBinding(), "Service.svc");
        Host.Open();
        Trace.WriteLine(String.Format(
            "Started In-Process Services Host on Base Address {0}, Endpoint Address {1}",
            baseServiceAddress,
            HostEndpointUri));
    }

    public static void StopServices()
    {
        if (Host != null)
        {
            Host.Close();
            Trace.WriteLine(String.Format("Stopped In-Process Services Host on {0}", HostEndpointUri));
        }
    }
}

Now we can start writing tests that do not need the hosting environment and a database to verify that your service works as designed, especially if you start adjusting its behavior to add security, custom metadata extensions, query interceptors,
exception handling, etc.:

[TestClass]
public class DataServiceTest
{
    [TestInitialize]
    public void SetUp()
    {
        TestServiceHost.StartServices();
    }

    [TestCleanup]
    public void TearDown()
    {
        TestServiceHost.StopServices();
    }

    [TestMethod]
    public void DataService_QueryProductss_ReturnsCorrectData()
    {
        var expected = new List()
                            {
                                new Product { Name = "Product A" },
                                new Product { Name = "Product B" }
                            };
        var mockRepository = new Mock();
        mockRepository.Setup(r => r.Products).Returns(expected.AsQueryable());
        Web.MyDataService.DataContextFactory = () => new MyDataContext(mockRepository.Object);

        var target = new ServiceClient.MyDataContext(TestServiceHost.Host.Description.Endpoints[0].Address.Uri);
        var actual = target.Products.ToList();
        Assert.AreEqual(expected.Count(), actual.Count());
        Assert.IsTrue(actual.All(p => expected.Any(x => x.Name == p.Name)));
    }
}

Summary

Building applications that expose WCF Data Service is relatively easy. What is more difficult is to make these applications testable.

This article demonstrated how to do that by switching code generation template to self-tracking POCOs,
introducing persistence-ignorant domain layer and by making our existing Object Context implement an interface that can be mocked/substituted in testing.
Every one of these improvements is for the better, but curiously, all of them happened in response to one goal to
make our system more testable.

Windows Azure Marketplace Expanded to Apps

Going beyond data: Windows Azure Marketplace just announced that it is expanding its capabilities to sell applications built on Windows Azure.  For now, apps are supported only in US market, but the team is planning to expand to other markets as well.

Feel free to explore Windows Azure Marketplace to find applications and data you can use for your business. Create your own app and become Windows Azure Marketplace Publisher!

Enforcing Code Coverage Targets is a Bad Idea

I recently observed an interesting conversation regarding code coverage standards. It boiled down to one simple question:

Should we enforce mandatory code coverage standards by failing any build with low code coverage?

Well, I think that the answer to this question is a resounding NO, and here is why:

Reason 1: Coverage does not mean Quality

Low code coverage tells you that you don’t have enough tests. High code coverage tells you that you have a lot of tests. Neither tells you anything particularly useful about quality of your production code or, for that matter, quality of your tests. I readily concede that low code coverage is a smell of bad quality, but it’s not a proof.  Absence of unit tests is alarming, and should be corrected, but what if your production code is actually pretty good? On the other hand, high code coverage may be achieved by bad test code – imagine battery of unit tests with no asserts in them because what your code returns in unpredictable. Yes, your code coverage is high, but your production code stinks.  At best, code coverage is an indirect indicator of overall quality and therefore by no means should be considered a build blocker (something that randomizes your entire team).

Reason 2: You get exactly what you measure

If you evaluate your team success based on the code coverage trends, you will get the positive trends. You might, however, eventually land in a situation when you have good code coverage but bad quality, because when pushed to meet the code coverage bar (thing that you measure), people will cut quality corners. Compare this situation with measuring high-severity bugs that escaped your system all the way to the customer. That is a direct indicator of quality and something that is being done by an independent party (your customer), which is even better!

Reason 3: More process won’t solve people problems

So, we have a fact: Your engineers (notice that I bundle both Developers and Testers in one group) do not write enough unit tests to exercise most of your system. But why? What could be the real story behind this behavior?

There could be multiple scenarios at play here, but all of them point to People problems, not to lack of Process. Breaking low coverage builds constitutes nothing but more process and will not correct one of these possible people problems long-term:

  • Our Engineers don’t understand the value of writing good tests.

Well, educate them! Explain that good test suites are not just burden, they are your safety net, your anti-regression armor, your documentation, you design driver, you early design verification and your sample code all in one!

  • Our Engineers don’t have time to write tests – they are busy coding the features.

Your estimates don’t include enough time to outfit your features with proper test automation, and your management is apprehensive about spending 10 to 30% more time on each feature. I suggest a simple experiment: pick two features roughly similar in complexity, build one without tests and one with tests. Follow up by calculating number of bugs per feature that will follow in the next release and keep track of total amount of time, including shipping service packs and deploying quick fixes, that was needed in both cases. The results will be self-explanatory. Now you have hard data to take to your management.

  • Our Developers don’t think that writing tests is in their job description.

Get another group of developers!

To summarize…

Measure your code coverage religiously and use it as one of the indicators of your overall quality. But don’t rely on it as your only indicator – don’t break your build over it, don’t yell at your engineers when code coverage drops 1%. Instead, make everybody aware why you think code coverage is important and review trends in your retrospective meetings. Encourage or even require your engineers to write tests to cover bugs and interesting usage scenarios, not your coverage targets. Enforce this rule using peer reviews. Correlate high-value bug escapes with components and use cases and adjust your efforts accordingly, so that if you have to improve things quickly, you can focus on unstable and buggy code first.