Should: Func’y Testing

When it comes to testing anything, testing methods (or functions) is probably the test performed most. I know that most of the time, I want to know if the function I just wrote is going to do what I want it to do:

  1. Will it return the correct value?
  2. Will it return the correct type?
  3. Will it throw an exception?
  4. Will it throw anything else? (like a tantrum)
  5. If it doesn’t return anything, does it perform it’s actions successfully?

These are a few of the things I constantly find myself testing, and I’m sure other people do too. After all, this is what Test-Driven Development (TDD) was created for. I have to admit, because of my previously-mentioned fussiness with current .NET testing frameworks and procedures, I’ve resorted to things like writing console apps to test output, creating obscure debugging classes to record information, and writing messy debug code that eventually just gets deleted.

Current testing procedure seems to follow along all the same lines. Create a class to hold test functions that test specific methods or parts of that class. Once you have all these functions set up, your usual next step would be to fill them with instances of your test class, a few parameters if any are needed, some test data and some assertions.

The problem here is the incredible amount of repetition. Clearly those who designed .NET testing frameworks had never heard of DRY (dont-repeat-yourself).

Now, what if all that could be done in one line? This is where Expectation Tests come in. I mentioned these in my previous post and went over their syntax, and I’m definitely still in the concept stages as I write this, but I’ve discovered that you can do just that:

this.Expects(Context.someFunction(), "should return true"); 

Note: here, this is a class that implements IShouldTest<T> where T is the class, or the “context”. Within this test class, there is a property called Context which obviously points to your passed context, which is instantiated automatically on every test run. You don’t have to do that yourself every time.

Now, this singular line hides a lot of workings behind the scenes, just like a good framework should, but it’s also incredibly powerful. What we’re saying here is that this test context expects the function Context.someFunction() to return true. In one fell swoop we’ve provided the code to test, the expected test parameters and made the test framework aware of the test. Easy peasy.

The concept behind it is pretty simple. Why should I have to specifically define every single element of my test, every single time? Why can’t I just say “I want to test this, this is what it should do”?

This is all made possible due to the improvements in how delegates are used in .NET. I must admit, for a while I just didn’t understand why you’d want to use a delegate, apart from async and event stuff, and the massive waste of time setting these things up was a huge put-off for me personally. Since Func<T> and it’s cousinAction<T> turned up, you can now leave all the annoying delegate work to the framework. It performs all the necessary creation of delegate types for you, and the best thing about this is that it isn’t doing anything more than it used to, so there’s no performance hit.

Another reason why testing using Func<T> and it’s friends is great is LINQ and Lambda. Have you ever wondered, ‘how would my function react if i gave it this?’:

this.Expects((hugeDictionary) => someSortingFunction(hugeDictionary), "should take < 1 minute"); 

Pretty cool, huh?

Advertisements

shoulda, woulda, coulda

I hate Testing Frameworks. Actually, let me be specific…I hate .NET Testing Frameworks. They make me angry and sad; so I’ve decided that because I have nothing better to do, I’ll write my own. But first, a little comparison…

VS2010 Testing Framework (MSTest?)

Microsoft’s built in testing system isn’t too bad. It’s integration is done well, but it’s so bloody overly complicated and slow, and messy. Every test you perform dumps a shit-load of messy files everywhere on your filesystem and inside your solution folder. It then has the CHEEK to complain that you should delete some when it gets “full”?

Poor effort Bill (or is it Steve now?)

NUnit

As far as I’ve ever been able to tell, this is simply a big, even more bloated version of the above. Some of the testing functionality is simpler and easier to understand, and it does have more assertions…that’s where the good things end. Like a lot of software, it’s probably the most used? Weird that.

xUnit

Made by the same blokes as the above, but apparently it’s better and easier to use. I went over their comparison table they’ve got up on their Codeplex site….yeah, no. If anything, it’s a massive excercise in how to complicate a piece of supposedly useful software even further. I mean…test methods are called “Facts”? Not only that, the method to actually get testing is beyond annoying.

Getting there…but not quite.

Should…

I’m tired. I like ruby, I like libraries like “shoulda” and “cucumber”. You just want to test whether that function does what you want it to; does it fulfil the requirements? Does it do what you expected it to do? I got tired of fumbling around the above libraries. I wanted a simple, easy to use, easy to understand testing framework so that I could test Nucleo quickly and effectively, so I created “Should”. At least that’s what It’s called at the moment.

Creating a test involves (at the moment) two things:

  1. ShouldTestContext<T> object. Think of it as the manager for all your tests. The generic type parameter <T> is the object you want to perform tests on, or the “context”.
  2. One or more Expectation objects. An expectation test is exactly that; it tests an expectation of something else.
  3. Once you’ve set up all your expectation tests, you make a call to theShouldTestContext<T>.RealiseExpectations() method and shazzam, it tests away!

Expectation Test Syntax

I wanted to make writing the actual tests as easy as possible; plain english if i could. Turns out I could:

Expectation ex = new Expectation(() = { return true; }, "should return true"); 

Here, I’ve created a new expectation test and fed it a lambda expression as a delegate that simply returns true (for demo purposes) and my expectation (logically) is that it “should return true”. Using a simple parser, an Expectation is then created by analysing the logic of the expectation string. The parser determines the type of test by what kind of type is passed to it (method, property, field etc.) and then runs a test against the object based on the given expectation. The kind of tests you can run are all your usual suspects, and I may add a few more in the future. Here are some examples of some expectation test strings:

"should contain 'Hello'" (test a collection, array or string) "shouldalways return true" (test a constant) "shouldnot have length > 10" (test a property or field) 

The important thing to note is that, underlying all of this is the same basic usage of assertions. It’s the way in which those assertions are performed which is the cool thing. I am also making it completely programmatic too, so if you want to write your tests in the standard way, you still can, and they’ll still be beautiful:

Should.BeEqual(1, 1); Should.ReturnTrue(() => { return true; }); Etc. etc. - more to come..