Category: Software Development

Simplicity Is A Virtue

Life is really simple, but we insist on making it complicated.

About a year ago, my team and I were working on a feature that turned out to be complex and unpleasant to implement. Multiple system dependencies, complex user interactions, unreliable third-party endpoints with which to integrate – the works. As we often do at Wetpaint, we had a junior and a senior members of the team working together on different parts of the feature. We were going through a code review round. Our senior engineer was visibly uncomfortable with some of the code “smells” caused by the overall complexity and wanted to simplify things, even if it took extra time to do. Being a team joker, he said: “Imagine that the next maintainer of this code is a serial killer who knows where you live. You don’t want him to get angry, right?” We fell over laughing, continued to listen to our instincts and worked to considerably simplify our code.

Looking at this funny incident now, I appreciate even more how undeniably prudent and forward-looking this advice is. Simplicity is a virtue. We should try to maximize it.

Simplicity is the ultimate sophistication

Simplicity is the ultimate sophistication

When I say “simplicity”, I realize how relative this term is.
Nowadays, in addition to “serial killer” test, I apply one more: can somebody new to this code but not new to the problem pick up the gist of what you built quickly? Over lunch? Over a day? When you have moved on to other things, teams, companies and cities?
This requires, in my mind at least, that I and my team try to follow these simple guidelines:

  • Be intentional. Naming things right is an art that must be mastered. Calling things for what they are has extraordinary importance. I make time to allow myself to think of a good name for that method or that class. Among other things, it reduces the need for comments I have to put in my code (more on that below).
  • Be brief but not obscure. If anything can be expressed in a few lines of code and still well-understood by a relative newbie, that’s a perfect balance. I don’t believe that stuffing 50 things into one line of amazing C++ is as useful.

  • Be fluent. I frequently consider returning self (or this) as method results, so that calls can be chained together into English-like sentences:

def service
  • Delegate responsibility to an Information Expert. I follow this pattern as much as I can, while deferring decisions about which component will handle each part of the execution until last possible moment.

  • Refactor. More often than not, I realize that initial class and method structure won’t work as I get deeper into development. As soon as this is discovered, I believe that trying to force the issue will cause more discomfort and waste more time than doing a quick refactory. Refactoring shipped code is 10-100 times more expensive than doing it during feature development.

  • Avoid comments. I know this is controversial. But consider the fact that most of the comments we see in code are either obvious and redundant (once you actually read the code) or, much worse, are placed in blocks of code that are so complicated and unreadable, comments don’t add value anyway. Therefore, over the years, I’ve grown accustomed to a style of programming when the only comments I should need are the ones from which we generate documentation. Everything else, most of the time, is a sign that naming is screwed up or complexity is too high. So I either refactor or rename things to signal intent better that a comment could.

  • Don’t build frameworks right away. Until I see the same code pattern repeat itself in the project multiple times, the simplest approach is the best. Once I begin noticing repetitious code, I DRY up my code up paying attention to varying requirements and giving my DRY pieces just enough flexibility to handle the needs of the project. Thus, a framework emerges organically, out of real code needs.

  • Keep learning experiments out of production code. The fact that I just learnt how to use, say, Backbone.js does not neccessarily mean that my entire project needs to be re-built with it just because I want to learn how to apply my new shiny knowledge. Best tool for the job principle still applies.

  • Be test-driven and test-obsessed. I strive to write tests not just to verify behavior of my code, but to explain to others intricate details of how my code functions. To do that, I try to be thorough with test coverage, mock or stub liberally, be very intentional in how I name my tests, and very careful with what expectations I assert at the end. This also helps with reducing amount of comments and instead replacing them with working test code.

I am not naive to think that every problem can be solved with these simple recipes. Some problems are naturally very hard and require big chunks of very complex code. But I am advocating going the extra mile and doing everything possible to make things simpler for people who will work with our code with us and after us.

Because overly complex, unreasonably “cool”, or utterly unmaintainable code quickly becomes legacy code. Legacy code that nobody wants to touch. Legacy code that does magical things nobody understands. Legacy code that is dead weight capable of sinking any software product.

Happy coding!


HealthCheck: Software Monitoring Inside Out

This post was originally published on Geekwire. I am republishing it here for the followers of this blog.

When we release software to customers, we immediately begin to consider how to monitor it. The goal is to find problems as early as possible, and always before our customers. We start with the basics: disk, memory, CPU load. Still, because of the complexity of the products we develop, there are countless examples of things going wrong: the system fails and we hear about it hours later.

System Components are like firing neurons

System components are like firing neurons – connected and hard to monitor. Photo credit:

Monitoring software behavior delivers serious value only when it captures deeper system metrics which can be analyzed to gain interesting operational and business insights. To measure these more advanced key performance indicators, we need to have access to internal components and complex data. External monitoring typically does not have sufficiently deep system access and therefore cannot provide answers to questions like these:

  • For an online shopping system, how many orders were received in the past hour? Is this significantly different compared to an average day?
  • For a publishing system, how much time elapses between submitting an article and that article going live? Does it have unusual spikes?
  • For a patient registration system, how many messages are sitting in its queues? What is an average wait time for a message?
  • For an online experimentation system, how long does it take to compute experiment results? Does this delay ever become unacceptable given traffic projections?
  • For a point of sales system, how many retries does it take to submit data to a payment gateway? Is that number unusual?

In summary, we need a way to boil down advanced system statistics to a definitive and actionable answer to one single question: Is our system healthy?

At Wetpaint, where we build a social platform for publishers, all our systems have a special /healthcheck URL. This endpoint is the main asset of our external monitoring system, Nagios, which we use for advanced checks. If it gets no response or an unacceptable response, the system is unhealthy.  The Ops person gets an alert and the Developer gets a call. Together, the team begins investigating what’s wrong. When they log into the system, the same /healthcheck endpoint produces a detailed report — and, if there are failures, it provides specific instructions to follow to troubleshoot the problem.

Wetpaint Health Status Report

Health Status Report for Wetpaint Social Distribution System.

We built our systems to self-monitor: we know how long it takes to submit a scheduled Facebook post or a tweet, how many of them failed to get submitted during the last 24 hours, whether content ingestion from our partners is working and up to date, how long it takes for data analysis and statistics jobs to finish computation, and countless other metrics. This approach shifted reliability worries entirely to the engineering team, where it belongs. It made the Ops job simpler, providing an answer to the ever important question: “I got a monitoring alert. Do I need to call a Dev or not?”

When building each of the checks, we asked ourselves:

  • Can this situation be systematically prevented?
  • In the future, can this situation be detected early? What data do we need to collect to ensure that?
  • Is there a clear, actionable set of steps that should be taken by a Support Engineer to fix the issue? Can these steps be automated?

These questions required deep design changes to our system. We ensured that everything leaves enough of a trace about its functionality, so /healthcheck can deterministically discover when something is broken. To do this, we focused on two crucial requirements:

  1. Precision to eliminate false positives and a “cry wolf” approach to alerts
  2. Actionable insights. The key is to give a specific roadmap to fix the problem at hand. After all, we woke somebody up in the middle of the night with an alert, the least we could do is to give them enough information to be successful in fixing the problem.

Over time, we arrived at a design that allows us to see the earliest signs of trouble long before anything becomes visible to our customers, and helps us focus on larger business questions. When we get a “Healthcheck Red” alert, we don’t stop working on the problem unless /healthcheck turns green, and we file it immediately as a severe interruption, which we track.

The journey of discovering where our systems can fail is long and tedious, but worthy and important. The trick is not to expect absolute perfection, but to protect ourselves from catastrophic failures by using deep, comprehensive monitoring and early detection. Just like in real medicine, discovering a problem early and systematically eliminating the root causes of ailments are the keys to successful treatment.

Why do Code Reviews?

Interestingly enough, this question came up in a conversation related to a recent event: severe bug escaped into the wild causing embarrassing data loss. The bug was a regression that was introduced and not noticed during a code review. Regression suite failed to catch it as well. Naturally, we were trying to understand what we can do better. One of the questions was related to code reviews: could a more thorough review have prevented this?

To our surprise, we discovered that not everybody agrees on goals of a code review as well as on his/her right and responsibilities as a code reviewer.

So, I came up with these code review goals that are now posted in our office:


  • Maintainability and best practices/coding guidelines
  • DRY, Design patterns, Readability, Testability
  • Test coverage
    • Very important for both new features and bugs
    • Evaluate exposure to missing tests
    • Tests should highlight edge cases + happy paths


  • Be able to take over the ownership
  • Understand requirements, context, and code

Help the author become better

  • Suggest alternative approaches, defend your decisions

Sign-off = Your Reputation on the Line

Creating Testable WCF Data Services

One of the effective ways to expose your data on the web is to use WCF Data Services and OData protocol.

In its simplest form, a WCF Data Service can be built by exposing your ADO.NET Entity Framework ObjectContext through the DataService<T> class.
However, when it comes to testing, especially in situations when you have to enhance data in the OData Feed with some additional information (through WCF Behaviors for example), or when you have to verify that your Query Interceptors work as designed, using Object Context that is bound to a live database is hard because:

  • Your tests that have little to do with the shape of your data become dependent on the database, which is a big problem
  • You have maintain test data in the database when you run your tests, which is a maintenance overhead

So there lies our problem: how to expose data using WCF Data Services in a way where the database dependency can be mocked out during unit testing? The short answer is that we need to achieve better testability by introducing persistence-ignorant domain layer and by making our existing Object Context implement an interface that can be mocked/substituted in testing.

This article describes the concrete incremental steps you can take to get you from a standard Data Service built on top of Entity Framework Object Context to a solution that will satisfy testability requirements above. I will demonstrate the technique on a sample service connected to the AdventureWorksLT database.

Initial State: Data Service Of Object Context

public class MyDataService : DataService
    public static void InitializeService(DataServiceConfiguration config)
        config.SetEntitySetAccessRule("Products", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductCategories", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModelProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModels", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderHeaders", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderDetails", EntitySetRights.All);

        config.UseVerboseErrors = true;
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;

Step 1: Create an Interface that represents you Data Repository

This step is easy: For every entity type you expose in your Data Service, add a method that returns IQueryable<T>:

namespace MyDataService.DataAccess
    public interface IMyDataRepository
<address>Addresses { get; }
 IQueryable Customers { get; }
 IQueryable CustomerAddresses { get; }
 IQueryable Products { get; }
 IQueryable ProductCategories { get; }
 IQueryable ProductDescriptions { get; }
 IQueryable ProductModelProductDescriptions { get; }
 IQueryable ProductModels { get; }
 IQueryable SalesOrderHeaders { get; }
 IQueryable SalesOrderDetails { get; }

After interface has been created, you can provide default/production implementation in the hand-written partial class that corresponds to your Object Context by just returning Object Sets already defined by you Object Context:

namespace MyDataService.DataAccess
    public partial class AdventureWorksLTEntities: IMyDataRepository
        IQueryable IMyDataRepository.Products
        { get { return this.Products; } }

        IQueryable IMyDataRepository.ProductCategories
        { get { return this.ProductCategories; } }

        IQueryable IMyDataRepository.ProductDescriptions
        { get { return this.ProductDescriptions; } }

        IQueryable IMyDataRepository.ProductModelProductDescriptions
        { get { return this.ProductModelProductDescriptions; } }

        IQueryable IMyDataRepository.ProductModels
        { get { return this.ProductModels; } }

        IQueryable IMyDataRepository.SalesOrderHeaders
        { get { return this.SalesOrderHeaders; } }

        IQueryable IMyDataRepository.SalesOrderDetails
        { get { return this.SalesOrderDetails; } }

 { get { return this.Addresses; } }

 IQueryable IMyDataRepository.Customers
 { get { return this.Customers; } }

 IQueryable IMyDataRepository.CustomerAddresses
 { get { return this.CustomerAddresses; } }

Now you have an interface in your system that defines data access methods already implemented by your Object Context.
This means that if you build the rest of the system to rely/depend on this interface, rather than use its implementation (i.e. the Object Context), you can achieve better flexibility and the testability without having to worry about test databases and policing test data in your build environment and on developers’ workstations.

This is a worthy and very achievable goal, but we have a few steps remaining to fully get us there.

Step 2: Change Code Generation Template to use Self-Tracking Entities

This step is needed because you typically do not want you entities to be database-bound, and instead you want a reusable, independent domain layer that can be used in multiple scenarios. This move will contribute to a better code flexibility and layering without jeopardizing data access functionality that Entity Framework provides out of the box. Full discussion on why separate persistence-ignorant domain layer is a good thing is out of scope for this article. If you want to research the subject further, I would start herehere and here.

Adding Self-Tracking Entities Code Generation

Having done that, your Project containing Data Context will look differently:

  • You will see and code-generated Object Context in the AdventureWorks.Context.cs and AdventureWorks.Context.Extensions.cs
  • You will see and code-generated entity classes that match you Entity Framework Model.

Note that entity classes do not have any dependency on the EntityObject and therefore do not have to live in your Data Access layer.

Self-Tracking Entities: Generated Code

Step 3: Move Your POCO Entities into a separate “Domain” assembly

Now that we have a domain layer of self-tracking plain old CLR objects (POCOs), we want to make it reusable.
This means separating the entity classes into a separate assembly that does not have any dependencies on Entity Framework, System.Data or any other persistence technologies. The tricky part is that we want to still get the benefits of EF designer experience, i.e. if we added, removed or modified some entities, we want them to be re-generated the usual way.

To achieve that, follow these steps:

  • Create a new Project where you intend to host your Domain Layer. Reference System.Runtime.Serialization.
  • Right-Click on the Entity Code Generation template ( and select Exclude From Project.
  • In your Domain Layer project, add existing Entity Code Generation Template (located in EDMX folder) as a link.Visual Studio will regenerate Domain Classes in the correct namespace that matches your Domain Layer:Seperate Domain Layer
  • Reference your new Domain project from Data Layer and from the Web Project hosting your Data Service.
  • Add using statements for your Domain namespace to fix compiler errors.
  • You will notice that one of the places you had to add using statement is you code-generated AdventureWorks.Context.cs. Since this class is being generated every time, it is better to add additional using statement to the template:
    EntityContainer container = ItemCollection.GetItems().FirstOrDefault();
    if (container == null)
        return "// No EntityContainer exists in the model, so no code was generated";
    WriteHeader(fileManager, "MyDataService.Domain");
    BeginNamespace(namespaceName, code);

Step 4: Create your own Data Context that composes your Data Repository

In the Data Access project, add a class that will represent new context will use in the Data Service.
This class replaces Entity Context used before, and therefore it should implement all properties that return IQueryable instances for the Data Service. Since our Repository already implements all this, the implementation is trivial:

namespace MyDataService.DataAccess
    public class MyDataContext
        public IMyDataRepository Repository { get; private set; }

        public MyDataContext(IMyDataRepository repository)
            if (repository == null)
                throw new ArgumentNullException("repository", "Expecting a valid Repository");

            this.Repository = repository;

        public MyDataContext(): this(new AdventureWorksLTEntities())

        public IQueryable</pre>
 get { return this.Repository.Addresses; }

 public IQueryable Customers
 get { return this.Repository.Customers; }

 public IQueryable CustomerAddresses
 get { return this.Repository.CustomerAddresses; }

 public IQueryable Products
 get { return this.Repository.Products; }

 public IQueryable SalesOrderDetails
 get { return this.Repository.SalesOrderDetails; }

 public IQueryable SalesOrderHeaders
 get { return this.Repository.SalesOrderHeaders; }

 public IQueryable ProductCategories
 get { return this.Repository.ProductCategories; }

 public IQueryable ProductDescriptions
 get { return this.Repository.ProductDescriptions; }

 public IQueryable ProductModelProductDescriptions
 get { return this.Repository.ProductModelProductDescriptions; }

 public IQueryable ProductModels
 get { return this.Repository.ProductModels; }

Step 5: Update Data Service to use custom Data Context

[ServiceBehavior(IncludeExceptionDetailInFaults = true)]
public class MyDataService : DataService
    public static void InitializeService(DataServiceConfiguration config)
        config.SetEntitySetAccessRule("Products", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductCategories", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModelProductDescriptions", EntitySetRights.All);
        config.SetEntitySetAccessRule("ProductModels", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderHeaders", EntitySetRights.All);
        config.SetEntitySetAccessRule("SalesOrderDetails", EntitySetRights.All);

        config.UseVerboseErrors = true;
        config.DataServiceBehavior.MaxProtocolVersion = DataServiceProtocolVersion.V2;

One you do that, Data Services infrastructure will switch from Entity Framework provider to a Reflection Provider.
There are subtle but important differences between the two. One of the symptoms of those differences is that Reflection Provider does not understand POCO’s the same way Entity Provider understands Entities.
If you run you service as is, you will get a bunch of errors stating that there are some unsupported properties.
To fix those errors, you would have to create partial classes for your Domain entities that do two things:

  • Explicitly define data service keys
  • Exclude Change Tracker from list of properties available for the Data Services to pick up
public partial class Address

Finally, verify that your service still works by testing it in the browser.
You should be able to perform any operation you did before, but now you can also Unit Test your service without having to have a live database to talk to!

Step 6: For testing, inject Mock Data Repository

To do that, we will need to figure out how to “legally” let Data Service know that we want it to use our implementation of the Data Repository
instead of default one that connects to a real database using Entity Framework. One way to do that is to use Inversion of Control and Dependency Injection. However, making Data Service and WCF aware of IoC Containers is a tricky proposition – there is a simpler solution

What if we created a factory method and used it to create instances of the custom context whenever the Data Service needs one? Fortunately, WCF Data Service provides a way to do that by overriding CreateDataSource method in the Data Service class:

public static Func DataContextFactory;

protected override MyDataContext CreateDataSource()
    return DataContextFactory != null ? DataContextFactory() : base.CreateDataSource();

Now our test code can set the Factory to create Data Context already initialized with a mock Repository.

This is, however, is only part of the problem – we also want to be able to host the Data Service inside of the test process without having to deploy anything to a real web server.
Once again, this is solvable by creating in-process service host:

public static class TestServiceHost
    internal static DataServiceHost Host

    internal static Uri HostEndpointUri
            return Host.Description.Endpoints[0].Address.Uri;

<summary> /// Initialize WCF testing by starting services in-process
 /// </summary>
    public static void StartServices()
        var baseServiceAddress = new Uri("http://localhost:8888");
        Host = new DataServiceHost(typeof(Web.MyDataService), new[] { baseServiceAddress });
        Host.AddServiceEndpoint(typeof(IRequestHandler), new WebHttpBinding(), "Service.svc");
            "Started In-Process Services Host on Base Address {0}, Endpoint Address {1}",

    public static void StopServices()
        if (Host != null)
            Trace.WriteLine(String.Format("Stopped In-Process Services Host on {0}", HostEndpointUri));

Now we can start writing tests that do not need the hosting environment and a database to verify that your service works as designed, especially if you start adjusting its behavior to add security, custom metadata extensions, query interceptors,
exception handling, etc.:

public class DataServiceTest
    public void SetUp()

    public void TearDown()

    public void DataService_QueryProductss_ReturnsCorrectData()
        var expected = new List()
                                new Product { Name = "Product A" },
                                new Product { Name = "Product B" }
        var mockRepository = new Mock();
        mockRepository.Setup(r => r.Products).Returns(expected.AsQueryable());
        Web.MyDataService.DataContextFactory = () => new MyDataContext(mockRepository.Object);

        var target = new ServiceClient.MyDataContext(TestServiceHost.Host.Description.Endpoints[0].Address.Uri);
        var actual = target.Products.ToList();
        Assert.AreEqual(expected.Count(), actual.Count());
        Assert.IsTrue(actual.All(p => expected.Any(x => x.Name == p.Name)));


Building applications that expose WCF Data Service is relatively easy. What is more difficult is to make these applications testable.

This article demonstrated how to do that by switching code generation template to self-tracking POCOs,
introducing persistence-ignorant domain layer and by making our existing Object Context implement an interface that can be mocked/substituted in testing.
Every one of these improvements is for the better, but curiously, all of them happened in response to one goal to
make our system more testable.

Enforcing Code Coverage Targets is a Bad Idea

I recently observed an interesting conversation regarding code coverage standards. It boiled down to one simple question:

Should we enforce mandatory code coverage standards by failing any build with low code coverage?

Well, I think that the answer to this question is a resounding NO, and here is why:

Reason 1: Coverage does not mean Quality

Low code coverage tells you that you don’t have enough tests. High code coverage tells you that you have a lot of tests. Neither tells you anything particularly useful about quality of your production code or, for that matter, quality of your tests. I readily concede that low code coverage is a smell of bad quality, but it’s not a proof.  Absence of unit tests is alarming, and should be corrected, but what if your production code is actually pretty good? On the other hand, high code coverage may be achieved by bad test code – imagine battery of unit tests with no asserts in them because what your code returns in unpredictable. Yes, your code coverage is high, but your production code stinks.  At best, code coverage is an indirect indicator of overall quality and therefore by no means should be considered a build blocker (something that randomizes your entire team).

Reason 2: You get exactly what you measure

If you evaluate your team success based on the code coverage trends, you will get the positive trends. You might, however, eventually land in a situation when you have good code coverage but bad quality, because when pushed to meet the code coverage bar (thing that you measure), people will cut quality corners. Compare this situation with measuring high-severity bugs that escaped your system all the way to the customer. That is a direct indicator of quality and something that is being done by an independent party (your customer), which is even better!

Reason 3: More process won’t solve people problems

So, we have a fact: Your engineers (notice that I bundle both Developers and Testers in one group) do not write enough unit tests to exercise most of your system. But why? What could be the real story behind this behavior?

There could be multiple scenarios at play here, but all of them point to People problems, not to lack of Process. Breaking low coverage builds constitutes nothing but more process and will not correct one of these possible people problems long-term:

  • Our Engineers don’t understand the value of writing good tests.

Well, educate them! Explain that good test suites are not just burden, they are your safety net, your anti-regression armor, your documentation, you design driver, you early design verification and your sample code all in one!

  • Our Engineers don’t have time to write tests – they are busy coding the features.

Your estimates don’t include enough time to outfit your features with proper test automation, and your management is apprehensive about spending 10 to 30% more time on each feature. I suggest a simple experiment: pick two features roughly similar in complexity, build one without tests and one with tests. Follow up by calculating number of bugs per feature that will follow in the next release and keep track of total amount of time, including shipping service packs and deploying quick fixes, that was needed in both cases. The results will be self-explanatory. Now you have hard data to take to your management.

  • Our Developers don’t think that writing tests is in their job description.

Get another group of developers!

To summarize…

Measure your code coverage religiously and use it as one of the indicators of your overall quality. But don’t rely on it as your only indicator – don’t break your build over it, don’t yell at your engineers when code coverage drops 1%. Instead, make everybody aware why you think code coverage is important and review trends in your retrospective meetings. Encourage or even require your engineers to write tests to cover bugs and interesting usage scenarios, not your coverage targets. Enforce this rule using peer reviews. Correlate high-value bug escapes with components and use cases and adjust your efforts accordingly, so that if you have to improve things quickly, you can focus on unstable and buggy code first.

Developer-Driven Is Not A Panacea

Idea of developer-driven or developer-centric organizations is being actively discuss lately thanks to the likes of Facebook. Some people, mostly developers, love it for its freedom and lack of heavy processes. I love it too – to a point. I believe in its usefulness while a product is being conceived, while it is in the prototype or even in a rapid growth stage, while creators of the product have neither rich user data nor significant market penetration. In these situations, product team does not have any data to guide their development direction, and therefore tries to probe with multiple features.  However, there are obvious downfalls even at this stage:

  • Developing cool new features before core functionality is in place is meaningless at best and distracting you from your mission at worst
  • Developers left to their own devices tend to over-engineer most of the features and create flexible frameworks for everything, even trivial things
  • Most developers suffer from NIH syndrome, which leads to everything being written at least twice if not more

Let’s assume now that the product is becoming successful. It is growing in size, gaining more and more fans and/or more and more revenue. At this stage blindly following Developer-Driven process can get you into even more trouble.


  • New features developers chose to implement may not be helping your product
  • New features developers chose to implement may not be what your users want most or want at all
  • New features developers chose to implement may be cannibalizing or hurting other important revenue-generating features
  • New features your developers chose to implement may not be the most profitable


  • Duplicated code grows all over the place and when new features are being introduced, raging debates start about what to use and what to retire and why. There is no consensus, so different factions continue to use and improve their code of choice ignoring others.
  • Set of home-grown tools emerges that mostly work, but nobody is completely happy with them, so from time to time somebody creates new tools that do almost the same thing but “better”. Maintenance costs continue to grow…
  • Documentation becomes so far outdated that knowledge about overall system architecture is priceless and can only be found with people who have been on the team since its inception.
  • Build takes forever because of the amount of code. Configuration is mind-boggling because it is hard to configure dozens of “data-driven” frameworks to actually do what users want.

I could continue this list for a while, you know…

So what’s the solution? In my mind, it is very simple – provide a system of lightweight checks and balances that will keep the good and eliminate the bad:

  1. Get a good PM on the team. Give them broad authority to influence and little authority to dictate. Make them work together with Dev and Test teams. Good PMs hate process just as much as Developers, and they will provide an excellent balance against cowboy development and over-engineering problems. More importantly, this move will compliment technology focus of Dev-driven product with razor-sharp customer focus.
  2. Get architects to actually do their job. Architects who design how features should work should also be capable enough to quickly code up reusable and complicated components and provide actionable feedback and oversight to other team members. However, architects who sit in an ivory tower are just as bad for the product as no architecture at all. Therefore, getting seasoned developers to become architects (i.e. experts) on the product is a better way to go, I think.
  3. Use experimentation in addition to (or instead of) traditional marketing research. Once a product get beyond initial rollout stages and starts gaining steam, the only way to grow effectively is to use data, quickly try out new features, measure, and if ideas fail (most of them do, by the way) – fail early and get out.
  4. Watch over the buildavoid having dedicated build team, keep number of branches to a minimum, make developers responsible for integrating code. This will serve as a forcing function to have a reasonable component architecture and inject enough engineering discipline.
  5. Do not subscribe to “good developers write bug free code” myth, invest in test infrastructure, measure test coverage. Once again, placing test responsibility with Developers can be used as a forcing function to inject stability and reason into your code base.

In short, maintain user-centric focus, keep it simple as much as possible, use real data, trust but verify.