You won’t find bugs you aren’t testing for

I bought a game recently called X Rebirth, a long-awaited sequel to a series of “X” games that have been released by Egosoft over the last decade. Fans of the “X” series have really been psyched about what the screenshots and videos promised from X Rebirth. A lot of people pre-ordered the game. I bought the collector’s edition because it came with the game soundtrack as well as a few other goodies.

Unfortunately the game launch in mid-November was a complete disaster. The game is so riddled with bugs that it has garnered a 34/100 rating on metacritic (that’s bad), and reviewers have not been kind. The devs have been working hard, releasing patches almost weekly. For the most recent update, this paragraph in the release notes really caught my eye (emphasis mine):

It turned out that the issues many of you are seeing with trade ships not working correctly is not one bug but a whole series of connected problems… I do not want to give silly excuses here. Of course you deserve a bug free game, but you may ask why these trade problems were not uncovered during testing: We had a group of testers dedicated to testing our economy over a long period of time. These guys used a special version of the game which allowed them to run the entire economy at a higher speed. It seems that most of these problems we now see did not happen with this time acceleration. Meanwhile these problems also do not happen early on in the game or if you (like most testers do) have to restart the game frequently.

Note that they made two fundamental testing errors that cost them big time:

  1. Their testers use a “special” version of the game, meaning a version that the end-user will not actually be playing. In their case it was necessary but still dangerous as it does not simulate the conditions under which a normal user will play.
  2. They did not take into account prolonged periods of play, where the state of the game can be compromised by small bugs compounding over time.

I wish I could say that I haven’t made these same mistakes in my own career, but that would be a lie.

I’ve learned the hard way, as the Egosoft devs have, that the best way to test software is to use it myself, as my customer will use it, under conditions that approximate, as close as possible, my customer’s environment and usage habits. It is especially easy as a web developer to “bounce the server”, or “clear browser cache”, or “fake a dependency” while running my application, but in production, when my app is serving my customers and their needs, they (likely) won’t be able to do these things. And even if they can they shouldn’t have to.

So even though Egosoft still owes me USD $60.00 worth of gaming, at least I have learned a good lesson: always wait for reviews before I buy, and deliver the software I actually did test to the customer.

Things on which I am stewing

I have about an hour before the November STL ALT.NET meetup begins, so I thought I would quickly jot down some things on which I have been stewing for the last few months, not because I think you will find them particularly interesting but because writing is therapeutic and often leads to greater clarity of thought.

Who I follow on Twitter, and why

I recently spent some time organizing my twitter feed into lists and browsing Twitter’s “follow” recommendations.  Before I follow someone, I browse their current tweets to see if I am interested in what they are saying.  I like people who are focused with their content.  When I look at a person’s tweets, there is some threshold that I have–some content litmus test that I use–to determine if the bulk of what they write on Twitter is valuable enough for me to follow them.  For developers, it boils down to: how much do you tweet about development, or development-related topics?  If I glance at a Twitter stream and the bulk of what I see is not related to development, I probably won’t follow the Tweeter. (Is that even a noun?)  Another criteria I have is the quality of content people retweet or link to.   Twitter can be a tremendous place to find great articles, blog posts, open source projects, etc., but not all content is created equal, and I consider trivial links or retweets to be a form of spam.  But again, I have a personal standard for judging what is “trivial” that other people might not agree with.

I would be interested to hear from others who also screen potential Twitter follows, and what methods or standards they use.

I’m not learning Rails

I attended two Ruby conferences this year: Ruby Hoedown and Ruby Midwest.  I thoroughly enjoyed myself at both conferences, and I learned a lot.  I enjoy Ruby as a language.  We use Rake at work, and I enjoy writing Ruby code to make my build tasks easier.  Rails is a powerful framework for RAD development, and the sheer volume of helpful gems that are available for Rails is quite impressive.  The reason I am not going to learn Rails, however, has nothing to do with the quality of the Ruby language or the Rails framework or the Ruby community (which is fantastic).  After stewing for quite some time on my dissatisfaction (or rather, disinterest) in Rails development, I came to a point of self-realization.

I don’t want to learn Rails because I don’t want to write CRUD applications.  And Rails is a framework targeted specifically at CRUD developers.

Can you use Rails in a non-CRUD oriented way?  Yes–but a significant portion of Rails gems, websites, articles, discussion boards, videos, books, etc. are all CRUD-centric; only a precious few breach convention.  And most Rails developers see this strong CRUD emphasis as a feature, and indeed they should, because Rails solves many problems quite elegantly; it is the right tool for many jobs.

So, Rails, it’s not you–it’s me.

I really want to become better at domain modeling

My real passion–one that has been developing for several years in the quiet recesses of my frontal lobe–is domain modeling.  The technologies and methods to which I find myself attracted (Domain Driven Design, CQRS, etc.) are all concerned primarily with encapsulating domain logic and separating it from implementation details like persistence, the user interface, remote services, etc.  This stands in direct contrast to common CRUD methodologies which often wed, for the sake of expediency and conceptual simplicity, “domain classes” and the UI, persistence, and/or service layers.  Both models have trade-offs.  But I have been programming in a (more-or-less) CRUD-oriented way for years now, and I want to grow beyond that.

The distinction between these two modes of development became very clear to me at Ruby Midwest, where I watched Uncle Bob give a presentation on this very topic.  I realized that (from what I have seen) this discipline is fairly neglected in general business development.  Maybe it is just assumed that developers have this knowledge already.  I suspect, however, that the familiar nature of CRUD, and the quick gains it achieves in the short-term are the real justifications reasons why most projects start–and remain–CRUD-centric.

As I work through this gradual shift in my development paradigm over the next few years, I will write more about the things I discover.

As always, I’m interested in feedback.  Give me your opinions.  Really, I can take it (with scotch, on the rocks).

Naming interfaces

Interfaces in .NET have a specific naming convention.  They begin with a capital “I”, followed by some text that indicates what the interface represents.  IEnumerable is a classic example; it means the implementing object can be enumerated.  This naming convention, however, is pretty unique to .NET.  In the Java world, interfaces do not have a special prefix to indicate that they are interfaces.  Java, unlike .NET, uses two keywords, “extends” and “implements”, to indicate whether a class is participating in an inheritance hierarchy and/or representing some behavior, respectively.  In .NET, we use a simple colon and rely on the ordinal position of the class/interface names that come after to indicate the same.

The prefix “I” used for interfaces is seen by many (myself included) as unnecessary.  It serves no useful purpose other than to provide a visual cues to developers, allowing them to identify interfaces quickly while coding.  Java IDEs typically provide these visual cues so that developers don’t have to rely on a naming convention for this purpose.

Eclipse example 1

Eclipse example 2

Visual Studio has visual aids as well, but they are not as obvious, in my opinion.

I do think a naming convention for interfaces is still a good idea, but I don’t think the prefix “I” is necessary.  Instead, I would like to propose that interfaces be named for the behavior that the represent, starting with a prefix like “Can”, or “Will”, or “Has”, followed by a description of that behavior.  For example:

  • IEnumerable becomes CanEnumerate
  • IDisposable becomes CanDispose
  • IQueryable becomes CanQuery
Not only is this naming convention more descriptive and less contrived (sometimes it is difficult to come up with an interface name like IXyzable), it forces developers to really think about the behavior they are trying to implement, and may dissuade them from creating interfaces that have nothing whatsoever to do with behavior.  (The sudden explosion of IOC mania has obscured the real purpose of interfaces, but that is another topic for another time.)
If you have thoughts on this topic, I’d be interested to hear them.

Linus Torvalds on Project Management

“The first thing [people get wrong about open source projects] is thinking that you can throw things out there and ask people to help… That’s not how it works. You make it public, and then you assume that you’ll have to do all the work, and ask people to come up with suggestions of what you should do, not what they should do. Maybe they’ll start helping eventually, but you should start off with the assumption that you’re going to be the one maintaining it and ready to do all the work… If you start off with some ‘kumba-ya feeling’ where you think people from all the world are going to come together to make a better world by working together on your project, you probably won’t be going very far.” — Linus Torvalds (full article)

CQRS and Event Sourcing with NCQRS

At the August 2011 STL ALT.NET meeting, Nicholas Cloud talks about the CQRS and Event Sourcing — architectural patterns for building highly scalable and reliable systems — using the open source .NET implementation, NCQRS.

CQRS and Event Sourcing with NCQRS from STL ALT.NET on Vimeo.

EDIT: mynkow posted a minor correction to my presentation on the NCQRS Google group:

“Around 1h and 15m is not true. The internal event handler is called when you call ApplyEvent(). That is with the current code base there is no need to create internal handlers.”

Typemock to sponsor STL ALT.NET

I am pleased to announce an exciting addition to the growing list of STL ALT.NET sponsors!  Unit test tool vendor Typemock will provide a single license of their TDD oriented testing product, Isolator, to be raffled each month.  To win, you must follow our eligibility rules; to increase your odds of winning, you can offer to present a lightning talk on a topic of your choice.

Special SharePoint is special

I read an article this morning about SharePoint development that is so unbelievably naive it scares me. It scares me because I know that intelligent, motivated business men and women around the world are probably reading this article, nodding their thoughtful heads, and forwarding it to their developers as an informal contract of expectations. So without further ado, I’m going to indulge my inner jadedness and give you my thoughts.


Software Development Times brings us an enlightening piece called Why SharePoint Development is so damn special!, a pretentious exposition on what differentiates SharePoint development from other development disciplines. In the opening paragraphs, the author (Bjorn Furuknap) outlines his argument:

  1. SharePoint developers need to master a range of skills that “make quantum mechanics seem like Spelling 101”
  2. traditional development methods are teh suck
  3. a SharePoint solution may be consumed or changed by *gasp* other developers
  4. SharePoint developers are shooting a moving target, blind-folded, “without knowing what the target looks like, where it is… what type of ammo… to use…, [or whether they] are in the right shooting range

The rest of Furuknap’s article explains each of these points in detail.

SharePoint developers should be NINJAS with l33t skillz

I know and have worked with SharePoint developers, and I realize that SharePoint development requires knowledge that other .NET development disciplines do not. Like any specialization, there are particular libraries, tool kits, SDKs, etc. that a developer must learn to be proficient. I expected Furuknap to enumerate these particular specifics–after all, he states quite clearly that:

“SharePoint development is completely different than other forms of development in that it covers such a wide range of tiers and technologies.” (emphasis mine)

So what is it that makes SharePoint completely different?

  1. “customizing a solution through the Web interface, or out-of-the-box customization”
  2. “understand extensibility through the middle tier”
  3. “utilize scripting and markup languages such as jQuery, JavaScript, XML, XSL, and so on, usually done through tools such as Microsoft’s SharePoint designer”
  4. “the third tier of development where the programmers reside… using tools such as Visual Studio and Notepad”
  5. “all the varieties of .NET”
  6. “XML… [and] the XML dialects that SharePoint uses”
  7. “[the] intricacies of the SharePoint object models, both the server object model and now the client object model”
  8. “[s]ecurity… that SharePoint does… completely differently from anyone else” (sic)
  9. “Web services, both the ASMX and the WCF type”
  10. “Web service standards and definitions”
  11. “non-SharePoint technologies… such as Active Directory and a number of other authentication mechanisms”
  12. “SQL Server”

A pretty impressive list. Except for the fact that most of these skills are requisite in pretty much any .NET web development job. Only the skills specifically billed as SharePoint skills, or SharePoint extensions to other technologies, would differentiate the SharePoint developer from the average ASP.NET developer. This is hardly the “pirates vs. ninjas” smackdown that Furuknap implies.

Martin Fowler, Bob Martin, and (probably) all other Martins should be ignored

There are some truly great minds in software development. Two of my favorites are Martin Fowler and Bob Martin.  They’ve been around.   They’ve seen software development grow as a discipline, and have observed, documented, lectured, and probably composed interpretive dances to convey objectively demonstrable best practices in the field. One of these practices, considered by many to be an excellent foundation for writing stable yet extensible code, is unit testing. I will not define it here; I assume you know that of which I speak.   But apparently, SharePoint developers are a breed apart, and for them, the Old Ways are not applicable.

“…traditional development methods and experience will only get you a fraction of the way. Unit testing, for example, is useful only for the first few hundred lines of code, after which it becomes more of a burden than an asset… Unit testing and mocking is all the rage these days, and for a very good reason. When you are the master of the code, producing perfect code makes all the sense in the world. After all, any code error goes directly back to you, and if you develop your tests right, your code should have no errors. However, to a SharePoint developer, building code that works is just a small part of the puzzle. As I mentioned earlier, we are not the only developers on a SharePoint solution. In fact, we should encourage our users to modify what we make in order to better utilize the adaptability that is SharePoint’s greatest feature. This presents the traditional developer with a nightmare. If you are not in control over the environment into which you deploy code, how can you test whether your code works? You may set up a mock environment that matches the production environment 100%, and within five minutes, some user may completely alter the entire solution, changing the target you try to hit. This is not the worst part, however. Even if your code runs perfectly at the time you deploy it, it also needs to endure the active modification of users throughout its entire lifespan. Your code may depend on a list having a certain set of columns, but users may change that at any time. They may even remove or replace the entire list, and how will your code run then? The difficulty for a SharePoint developer is not just in writing code that runs perfectly once, but that runs perfectly, no matter what happens to the assumptions you’ve made for your code. There’s no way you can design tests that will properly validate your code against every possible change that users can make, and traditional testing becomes rather pointless.”

Now, since you’re familiar with unit testing (right? RIGHT?) you should spot a few obvious fallacies right away.

First, how the hell did he determine that “unit testing… is only useful for the first hundred lines of code”? Seriously. Is there a metric for this? Anyone?

Second, unit tests are not about producing “perfect code”. Unit tests are about code isolation. They are about determining if code performs as it should ceteris paribus. Code is only “perfect” insofar as it performs according to expectations, e.g. according to the promises implied by its public API.

Third, unless you are the lone developer on a project, you are always building code that others will use and will possibly modify. This is in no way unique to SharePoint developers! Unit tests help establish API consistency, precisely so that others can program against, and/or modify code with the confidence that their changes will not cause unintended consequences.

Finally, the goal of unit testing and mocking is not to reproduce production environment conditions. Furuknap insists: “If you are not in control over the environment into which you deploy code, how can you test whether your code works?” This is what integration tests are for, Furuknap. This has nothing to do with unit tests. Or maybe we don’t need those either.

Lock ‘n Load

Furuknap’s last point is that, because “SharePoint is such a complex beast” (his words, not mine), coordinating a SharePoint project among several development teams is like “trying to coordinate firing a gun at a target nobody knows the location of, without knowing what gun to fire, and, of course, without even looking.” In his analogy, the target is equivalent to the “task they are trying to accomplish”.

Why is SharePoint development this difficult? According to Furuknap, it’s because of the AGILE AXIS of EVIL!

  • “deliver the first line of code before the ink on the contract is dry”
  • pushing the product to “production by the time the contract is put in the envelope”
  • and “by the time the contract is filed in your SharePoint document repository, your client should be on their way to request the next major project”

In other words:

“…because of the wonders of agile expectations (which are good, don’t get me wrong), we now don’t know what our target looks like.”

Instead, he suggests that:

“The solution to all this is thorough planning and design up front. If you know exactly what you want to build before you write your first line of code, you also know the exact output and can communicate that to other teams. Of course, this assumes that the other teams do the same. In short, plan everything ahead and you won’t be surprised… Well, that was the way one developed solutions before agile principles taught us otherwise, also known as the waterfall era.”

What is hilarious about this elevation of Waterfall is a revelatory statement in which Furuknap states the explicit reason that Agilists embrace iterative development over Big Design Up Front.

“Don’t forget: From the time you release your first code to the time the customer tells you what changes they want, everything about your assumptions has changed again, because users have already started working with and relying on your solution… And then they ask us if we can do this any faster…”

Waterfall methodologies are based on the twin premises that a) all business needs, assumptions, processes, etc. can be articulated at the beginning of a project, and b) they will not change for the duration of the project. These assumptions have been challenged by critics of esteemed ilk (Ward Cunningham, the Martins, Kent Beck, Andy Hunt, and others from Olympus) and have been proved wanting.  Agile development relies on a tight feedback loop between developers and customers, so that customer expectations can be met as they evolve, not after the fact.

The last third of this article is divided into two need-to-know SharePoint technologies with links to additional reading material.  The first is the client object model, which Furuknap listed early on as an essential skill for SharePoint development. The second is Microsoft Business Connectivity Services (formerly named Business Data Catalog — what would a Microsoft technology be without multiple names?), which “provides read/write access to external data from line-of-business systems, Web services, databases, and other external systems within Microsoft SharePoint 2010 and Microsoft Office 2010 applications.” This sounds really important, but it failed to make the essentials list, and so leaves me confused and mostly indifferent.


At this point you may imagine that I have something against SharePoint developers, but that is certainly not true. Although I can’t say the same for our friend Furuknap, who recommends “booking a session with a psychologiest, because you are quite possibly stark raving mad” if you’re an aspiring SharePoint developer. To be fair, although he has no problem arming crazy people and letting them shoot at moving targets, he does advocate a commensurate pay grade for these Wunderkinds.  You know, because they are so damn special. 🙂

SharePoint development is a growing discipline Mr. Furuknap.  Don’t spoil it by turning its uniqueness into a license for ignoring the thoughtful wisdom of the last half-century of development.


Using NDepend to beat your assemblies into submission

This is the first in a series of articles about using the .NET software metrics tool, NDepend.  Each article will focus on a particular use for NDepend, demonstrating how it helps developers “x-ray” their code to see strengths and weaknesses in design.

The Problem, or Why Does it Burn so Bad?

No one likes to experience pain, but the fact is that pain is often a very good metric for measuring the level of stupidity required for making certain decisions.  For example, I have a very distinct childhood memory of touching a hot iron to determine if it was in fact plugged in.  It dawned on me later, after much wailing and many tears, that perhaps looking at the outlet would have been a better approach, but if I had experienced no pain, I would not have learned.

Software development is really no different, though you might not know it by examining many software projects.  Many of the pain points in software development are actually indicators–signals that alert developers that there are design flaws, perhaps significant ones, in the software solution.  Often under tight deadlines, budget constraints, and a small army of Pointy Haired Bosses, developers spend hours agonizing over software bugs that seem to spring from the aether.  Changes in one module often break code in another, seemingly unrelated module, creating QA and maintenance nightmares that are only remedied by more duct tape, bailing wire, and bubble gum.  Problems persist until, eventually,  the code base becomes too fragile and tangled to change.

A common cause of pain is often bad coupling, or bad dependencies, between software modules.  In .NET, the unit of composition for software modules is the assembly.  In general, the following principles should be applied to assemblies:

  1. modular software should be built to reuse, and be reused by, other modules;
  2. software modules should be easy to change; and
  3. changes to a given software module should not affect dependent software modules

Most developers understand and agree with the first principle, but have experienced the pain and frustration of practically implementing the second and third.  If a .NET assembly violates the second principle, it is said to be rigid; if it violates the third, it is said to be fragile.  [Martin, Metrics]

Reuse is impossible to achieve without dependencies.  Assemblies hold references to other assemblies and reuse their types. Unfortunately, an assembly becomes difficult to change as its dependents increase, and as a consequence, changes to that assembly can have adverse consequences to dependents which may have no conceptual relationship to it.  It would seem that, to adhere to the first principle of modular software, the second and the third must necessarily be violated.  And it precisely this violation that causes so much turmoil in software projects.

The good news is that principles two and three can be adhered to if abstractions (in the form of actual abstract classes or interfaces) are introduced between dependent modules.  Consider the following simplified case of a dependency between two classes in two different assemblies:

In this scenario, consider that changes to Class2 could have potential ramifications for Class1, even though Class1 may not require change to fulfill a given requirement or change request.  The more classes that come to depend on Class2, the more difficult changes will be–the number of dependencies and possible side effects will cause developers to flee in panic whenever changes to Class2 are necessary.  In contrast, consider the following alternative scenario:

By introducing an abstraction between classes 1 and 2, the developer is free to change the implementation of Class2 as necessary without fear that it will cause other assemblies and classes to change as well.  By changing the nature of the dependency across assemblies, developers can easily adhere to all three principles of modular software.  It is also interesting to note that the rigid nature of the direct dependency between Class1 and AbstractClass2 also serves to crystallize the nature of abstract classes (or interfaces)–that is, the dependency itself dissuades developers from changing the public facing API of the assembly.  This is the Open/Closed principle applied to assemblies.  Assembly APIs should be open for extension, but closed to modification.

The Solution, or How Using NDepend is like Bringing a Bazooka to a Gun Fight

SO.  Now that the nature of the problem is clear, how does a developer go about identifying problem assemblies and types in an application?  The software metrics tool NDepend allows developers to examine their software projects and determine where pain points are so they can be eliminated.  NDepend applies a number of software metric algorithms to a given code base, and presents a host of useful information about the relationships of assemblies, namespaces, types, members, variables, etc. in a .NET project.

To illustrate how NDepend can help solve the assembly dependency problem, I resurrected an old .NET project from the mothballs to see exactly how bad my code violated the modular software principles.  I was delighted to find that NDepend has Visual Studio 2010 integration, which makes it very easy to analyze a solution without leaving the IDE.  I proceeded to upgrade my old project to VS2010 format, and attach a new NDepend project to begin my analysis.

Because NDepend is such a powerful tool, it is easy to drown in the sheer number of metrics available for a .NET solution.  I spent quite a bit of time reading and digesting the NDepend Code Metrics Definitions page on the NDepend website.  For this specific problem, I was interested in assembly metrics, specifically:

  • Afferent Coupling (Ca) — the number of types external to a given assembly that depend on types within that assembly (incoming dependencies)
  • Efferent Coupling (Ce) — the number of types within a given assembly that depend on types external to that assembly (outgoing dependencies)
  • Instability (I) — the ratio of efferent coupling to total coupling within an assembly: I = Ce / (Ce + Ca)
  • Abstractness (A) — the ratio of abstract types (and interfaces) in an assembly to the number of total types in the assembly (if A = 1, the assembly is completely abstract; if A = 0, the assembly is completely concrete)
  • Distance from the main sequence (D) — the degree to which an assembly’s abstractness and dependencies balance each other (a very abstract assembly should have many dependents, and a very concrete assembly should have few)

NDepend offers a number of ways to find data, but the most versatile is the CQL query editor.  CQL is a robust and, frankly, daunting domain-specific-language that enables developers to slice and dice a code base to identify solution items that meet specific criteria.  As a starting point, I was interested in assemblies in my project that had a high number of incoming dependencies (afferent coupling), but a low level of abstraction.  I opened the CQL editor (complete with intelisense!) and queried the code base.

Right away, I noticed that the Common assembly has a high number of dependencies, but a very low level of abstraction, which means assemblies that depend on it are directly using its concrete types.  If I made changes to Common, I would have a good change of breaking other assemblies, forcing me to change them as well.  Since this project is fairly trivial in size, my deployment strategy had always been to release all assemblies in each software release.  Non-trivial projects, often composed of dozens or hundreds of assemblies, fare far better if changes can be deployed on an assembly-by-assembly basis.  If my Common assembly were a member of a larger project, it would require all dependent assemblies to be released whenever its concrete types were changed.

It is clear that Common was an offensive assembly, but I needed to know what its dependent assemblies were to identify the exact points of Ca to be abstracted.  I could have monkeyed around with the CQL editor some more to find that information, but I have a general disdain for SQL-like languages (which makes me a favorite among DBAs!), and I wanted to try out other NDepend features, so I opened the Dependency Matrix to find the information I needed.

The Dependency Matrix is really as awesome as it sounds.  It allows a developer to view every assembly, type, member, etc. in terms of every other assembly, type, member, etc., on a matrix that shows the number of dependencies at a given drill-down level.  Since I was interested in the dependencies of types among assemblies, I adjusted the view option and got a nice breakdown.

As you can see, each assembly is listed on the X and Y axis.  The squares in the middle represent the number of types that have dependency relationships on the other assemblies.  The green boxes represent the efferent (outgoing) coupling of the horizontal assemblies; the blue boxes represent the afferent (incoming) coupling of the same.  I located Common on the horizontal axis and noticed that types from the root namespace, UI.Common, Cache, Data.Import, and Data directly used types in Common.  Since I wanted to identify those types in Common that were directly referenced, I expanded the horizontal assembly node to get a more detailed view.

The bulk of the incoming references are for business entities, which are used in many places in the application.  Now, recall that the measure of an assembly’s instability is the ratio of efferent (outgoing) coupling to total coupling.  I noticed that Common, lacking any green box on its horizontal axis, references no other assemblies (framework assemblies excluded), so its instability in terms of developer-maintained code is zero.  Common is a completely “stable” or rigid assembly, meaning that it is not easily changeable (like the foundation of a structure is “stable”), but it is also not very abstract, making it a very fragile assembly–both are conditions that I wanted to avoid.  To fix these problems, I could have used the dependency matrix to “drill down” deeper into actual types and methods to determine the exact locations of tightly coupled code and introduce abstractions that would allow Common to maintain stability, but also allow extensibility.

I was curious about other dependencies in my project, but wanted to get visual overview of the project as a whole.  I opened the Dependency Graph and changed the box size setting to adjust each node in the graph by afferent coupling.  The result confirmed my analysis of the Common assembly, but also highlighted the Logger assembly as another possible offender (click to enlarge).

The interesting thing about this diagram is that it shows the flow of dependencies from the left to the right, and it’s very obvious that the assemblies on the right are the ones that a) need to be most stable, since they have the most dependencies, and b) desperately need to be abstract to avoid violating the second and third principles of modular software.  The Dependency Graph fly-out lists detailed information about the Logger assembly, but the two items that immediately stand out are the abstractness rating (0.3, relatively close to 0), and the instability (0.2, very difficult to change).  So Common and Logger have the same problems.

I decided to run the NDepend Report last, which produces a comprehensive HTML file with detailed information about the entire project.  I was looking for a specific diagram which illustrates, in a nutshell, the reason why dependency management is important, and I found it (click to enlarge).

There is a lot of information in this graph, but notice the lower-left-hand corner: a special little hell known as the “Zone of Pain”.  On this graph, Logger and Common occupy a sherbert-orange colored slice of the graph, just next to it.  The Y axis of the graph indicates that the level of abstraction for these two assemblies is low (i.e., they contain mostly concrete classes), and the X axis indicates that each assembly has a low level of instability, i.e., a high level of stability, i.e., a large number of incoming dependencies which make it difficult for the assemblies to change.  If the “Zone of Pain” is hell, the “Zone of Uselessness” is Plato’s heaven: all forms, and no substance.  Classes that fall into this space have maximum abstractness (they have no concrete classes), and maximum instability, i.e., no incoming dependencies.  So they are, in reality, “useless” and unused.

The perpendicular position of each assembly on the graph relative to the center diagonal line is known as the “distance from the main sequence”.  Notice that there is a nice green area surrounding the main sequence.  This represents the ideal location that assemblies should occupy, which implies a special relationship between abstractness and afferent coupling.  According to the chart, the more “stable” an assembly is, the more abstract it should be, and likewise the more “instable” an assembly is, the more concrete it should be.  Assemblies that occupy the lower-right-hand corner have no dependent assemblies (think web project, for example), and can have as many concrete classes as they like, while assemblies that occupy the upper-left-hand corner should be completely abstract because their dependents are so many.  As long as assemblies remain in close proximity to the main sequence, they conform to the Open/Closed principle, and will be relatively painless to modify.

The Conclusion, or How Drinking Helped Me to Understand All of This

My first round with NDepend pretty much kicked my ass, but in a very good way.  I was able to look at a project that I was intimately familiar with in a new way, and learned that there is a very precise and quantifiable relationship between the quality of a design and the extensibility of a design.  NDepend takes complicated metrics and gives developers a way to visualize data about solutions, track down problem areas, and refactor bad designs with confidence.  Any software team that is serious about making robust, extensible software should consider adding NDepend to the toolbox.

For further reading on assembly dependencies, I recommend the following resources:

How not to unit test

I remember when I wrote my first unit test.  It happened at a point of realization — I had enough experience writing code that the pain of software bugs started to outweigh the joy of software development.  I’d heard about unit tests from friends, read about them on blogs, heard about them at conferences.  When I inevitably wrote my first test, it was pretty frightening.  I understood enough about unit tests to limit each test to a single class method or property, but I did not understand that dependencies would be my undoing.  My early tests traversed all layers “downstream” of the object under inspection, and even queried the database and live web services.  My tests tended to look like this:

Unit tests across layers - this is badSince those early days, I’ve learned a lot about unit testing, specifically that it helps you break your dependencies so that your code can be tested in isolation.  Testing across software “layers”, i.e., testing a unit of code and its dependencies can be considered anti-testing, and actually gives you false confidence in your system when, in reality, it is a fragile cacophony.

1. Tests across layers require an exponential number of tests.

Every developer who writes dependency-traversing unit tests eventually comes to the sickening realization that no matter how many tests he writes, the possible combination of parameters and possible code branches that could be executed in any given test is, well, staggering.  For every new layer, or every new dependency, the number of tests for a given member would be raised to the value of the number of tests for that dependency.  In contrast, if the dependencies of a given unit of code are mocked or stubbed for the purposes of unit testing, the developer only has to write the tests that would test the code immediately under inspection.  In other words, breaking dependencies = additive, keeping dependencies = exponential.

2. Tests across layers require simulated application state.

Any developer maintaining a large config file for his tests knows how much of a pain this is.  If the application relies on:

  • configuration values that are read by the application at startup;
  • configuration values that are provided by background threads;
  • configuration values that are retrieved from database calls;

then code that relies on this state, either directly or through dependencies, must have unit tests that recreate it for every test run.  Again, this can be very prohibitive, especially if the volume of configuration data is vast, or in relatively inaccessible places (e.g., other threads).

3. Tests across layers can give false positives.

A unit test is typically named for the unit of code being tested, for example, CalculateCartTotal_SomeState_SomeExpectedResult().  If this unit test spans application layers the developer who wrote it won’t immediately know if an error in the test is a result of a calculation failure, corrupt database data, missing session values, etc.  The test might fail for reasons that have absolutely nothing to do with calculating a shopping cart’s total price.  Precious development time is often spent debugging test paths to determine where the offending dependency lies.

4. Tests across layers can give false negatives.

Because of #1, it is possible that all tests for a given method will pass, even though the method itself is flawed.  This can occur because the chance of a developer missing a given combination of state and behavior in a unit test is significant.  This is possible in test suites where dependencies are removed, of course, but is far less likely given the reduced number of possible tests and the ability to simulate known state (by mocking and stubbing) that will produce exact, expected results.

5. Unit tests are for code, not data or connectivity validation.

A common justification I hear for encouraging unit tests that cut across application layers is: “It helps us know if our data is correct.”  This justification confuses the role of unit tests and integration tests.  Unit tests are designed to determine if code functions properly, not if a given system can connect or retrieve data from another system.  Integration tests help us to validate those boundaries, but unit tests are only concerned with the behavior of code.  The validation of data should occur as close to the data source as possible, even within the data source if possible.

6. Unit tests across layers are slow.

One of the primary benefits of dependency-ridden unit tests is that they are fast.   A developer can run a suite of unit tests in a matter of seconds, or minutes if the code base is particularly large.  This means that the developer gets instant feedback about changes made to the system.  If unit tests cross data access boundaries, or query live services, performance will drop dramatically.  In a high-pressure deadline-driven environment, this encourages developers to ignore unit tests altogether, or leave them for “later” (you know, the later that never comes).  Without unit tests as a feedback mechanism the developer will revert back to the days of design-by-debugging.

7. Remember: testing the sum of the parts = testing the whole!

So if dependencies are bad in unit tests, how does one test the system in tot0?  If all dependencies are tested in isolation, that is, without reference to, or reliance on, any other testable code, they will work when combined if all unit tests pass.  Software systems should be decoupled anyway — your code should have a sparse number of direct references to concrete types, and instead, rely on references to abstract classes or interfaces which can easily be mocked or stubbed with mocking frameworks.  Real unit tests help you see where these dependencies are, and eliminate them.

Dependency example - this is also bad
In this example, ClassA has a direct dependency on ClassB, as show by field _b.  When a unit test is written for SomeMethodThatUsesB(), ClassB will directly affect the outcome of the test. In contrast, If ClassA depends on an interface for ClassB:

Dependency example 2 - this is good

then the implementation of ClassB can be replaced with a mock object with known state and behavior, that will affect the unit tests for SomeMethodThatUsesB() in a known and controllable way.  Not only is this good unit test design, it is good object design as well (the implementation for ClassB can be extended in your application without breaking classes, like ClassA, that depend on the interface IClassB).

The good news is that writing correct and accurate unit tests will make you a better developer.  The bad news is that people often feel a warm fuzzy feeling if their tests span layers in an application, because they believe it offers better “coverage”.  Nothing is further from the truth.  Breaking applications down into discreet units and testing each unit in isolation, under controlled conditions, is the only way to achieve accurate test results and prevent those early AM calls from disturbing your beauty sleep.  Now go forth and test.