About 1,5 year ago I wrote a post about code testing in Haskell. Post was accompanied by haskell-testing-stub: a small project showing how to organize tests and benchmarks in Haskell. I used test-framework package to gather tests written in different testing frameworks (QuickCheck and HUnit in my case) in a coherent suite. Test-framework might have been a good choice in 2012 but that might not be the case today. The major issue is that it has been abandoned by and was unmaintained for several months. Recently the project has been taken up by the community and received some updates but for now it looks like there won’t be any major development.
As a response to the test-framework package being unmaintained Roman Cheplyaka has released tasty (original announcement here). Since its release in August 2013 tasty has received packages supporting integration with QuickCheck, HUnit, SmallCheck, hspec as well as support for golden testing and few others. I decided to give tasty a try and use it in my haskell-testing-stub project. Tasty turned out to be almost a drop-in replacement for test-framework. I had to update cabal file (quite obviously), change imports to point to tasty rather than test-framework and replace usage of
[Test] type with
TestTree. The only problem I encountered was adapting tests from HUnit. It turns out that tasty-hunit package does not have a function that allows to use an existing suite of HUnit tests. That feature was present in test-framework-hunit as
hUnitTestToTests function. I mailed Roman about this and his reply was that this was intentional as he does not “believe it adds anything useful to the API (i.e. the way to *write* code).” That’s not a big issue though as it was easy to adapt the missing function (although I think I’ll just put it in a separate package and release it so others don’t have to reinvent the wheel).
I admit that at this point I am not sure whether switching from test-framework to tasty is a good move. The fact that tasty is actively developed is a huge plus although test-framework has reached a mature state so perhaps active development is no longer of key importance. Also, test-framework still has more supporting libraries than tasty. Migrating them should be easy but up till now no one has done it. So I’m not arguing heavily for tasty. This is more like an experiment to see how it works.