Episode 40: Better Built By Burkhard
Doing TDD with I/O-Free Tests
I’ll change my publishing schedule slightly. In the future, you’ll receive my newsletter on the first and third Monday of the month - instead of the 1st and 15th day of the month. I will occasionally skip the newsletter on the third Monday of the month in favour of a blog post on my website. I have been struggling lately to write at least one blog post per month. Now enjoy the newsletter.
Doing TDD with I/O-Free Tests
In my post Applying TDD to Classes Accessing Files, I stated that “bad unit tests read from files”. In more general words, unit tests doing any kind of I/O are bad. I went on to define good unit tests as FIRST following the definition by Tim Ottinger and Jeff Langr: Fast, Isolated, Repeatable, Self-verifying and Timely.
This definition is still unwieldy and invites tedious discussions about what a unit test is and what not. I had too many of these over the years. In the spirit of KISS (Keep It Simple, Stupid), I should have defined unit tests as I/O-free tests. That’s exactly what Ted M. Young does in his insightful post I’m Done with Unit and Integration Tests.
When I saw the title in my Twitter feed, my first thought was: “Oh no, yet another diatribe against TDD” (see Around the Web: TDD Bashing from an earlier newsletter). Fortunately, it was the opposite.
The two main kinds of tests I write. The majority are I/O-Free tests, and when I hit the I/O boundary and need to test against some real external service, I use I/O-Dependent tests (or I/O-Based).
Ted M. Young, I’m Done with Unit and Integration Tests
In his talk More Testable Code with Hexagonal Architecture (from 11:10), Ted defines I/O as “anything outside the current process”. I would even narrow I/O down to anything outside the current thread. I/O includes
access to files, sockets, pipes and databases,
access to hardware like system clock, GPIO, serial port, CAN bus, Bluetooth, LTE-M, WiFi and Ethernet,
communication with cloud services via MQTT, CoAP, HTTP and WebSockets,
communication with other processes via DBUS, gRPC and Qt Remote Objects, and
communication between threads.
I/O-dependent tests are slow and unpredictable. They fail the FIR prefix of the FIRST properties for unit tests: fast, isolated and repeatable. For example, CAN communication has a noticeable latency of 10-20ms, fails for many different reasons like buffer overflow, loose connectors and bus congestion, and depends on the message order.
The ST suffix - self-verifying (automated) and timely - is always satisfied for tests used for TDD. It has nothing to do with I/O.
In contrast, I/O-free tests satisfy the FIRST attributes. Hence, they are ideally suited for TDD, although they comprise tests that would traditionally be called integration or acceptance tests. Is the distinction between unit and integration tests helpful? Here is Ted’s answer:
I actually don’t care about: Is it unit? Is it integration? What does that mean? What I care about is how easily are they to write and how fast do they run. And all those attributes I just mentioned [(isolated, automatic, concise, self-verifying, precise, predictable, written by developers)] .
Ted M. Young, More Testable Code with Hexagonal Architecture (from 6:03)
Indeed, who cares! As long as I/O-free tests make my job as a developer easier, I don’t care whether they are unit, integration, acceptance or any other tests. How about you?
My Content
Interview with Crossware about Embedded World 2023
Axel Fersen from Crossware I/O interviewed Yoann Lopes (The Qt Company), me, Frank Federking (Tuxera), Simon Hausmann (Slint) and Philip Privalov (Candera) about our most interesting takeaways from Embedded World 2023.
For me, it was that Toradex is interpreting SoM more and more as Solutions on Modules - as written in my last newsletter. They have a ready-made solution for OTA updates and are working on one for secure boot. This approach differentiates Toradex from their competitors.
Corrections in Episode 39
I corrected some mistakes in episode 39 of my newsletter.
The one-product license does not include unlimited seats. You must pay royalties per device (€3.50 or less). The number of developers working on a single product is unlimited. So, the total price is €5,900 per year plus royalties.
Slint’s lead over Qt with respect to memory consumption is much smaller than I thought. The runtime of Qt for MCU consumes 200-250 KB of RAM. That’s on par with Slint. And that’s a lot less than the 6-8 MB I claimed in my newsletter. Consequently, Qt for MCU can run on the same lower-end and less expensive microcontrollers as Slint.
KDAB: Comparing, Qt, Slint and Flutter
I apologise to Till Adam and KDAB that I didn’t give you the chance to review my post before I published it. That was not OK. I am sorry.
Till Adam is the CCO of KDAB and not the COO.
Around the Web
Denis Kranjcec: Still using feature branching? Try trunk-based development instead
Feature branching works best in zero-trust environments. […] When you have a team […] working on a project together, feature branching creates problems. Team members should trust each other and be able to collaborate without using [toll gates] for pull request reviews. Otherwise, that team has a bigger problem than their choice of a branching pattern.
Last year, I advised a software organisation with several teams how to introduce continuous delivery. The French and German teams didn’t trust each other a bit. Unsurprisingly, they used feature branching. As they only integrated their changes every three weeks, they inevitably ended up in merge hell. This lead to a lot of blaming about who broke what and to even less trust.
It took four months of convincing and cajoling to abandon feature branching in favour of trunk-based development (TBD). Mandating TDD, enforcing that every code change had to be covered by tests, granting developers extra time to write the tests, and breaking down the work in chunks of less than two days helped a lot. As the smaller and more frequent integrations worked most of the time, developers gained trust in the process and in each other.
Denis’ comes to similar conclusion. Trust in code creates trust in the people who write the code.
To be successful with trunk-based development, and continuous integration, an application must have automated tests you trust. […]
It turned out the biggest obstacle was changing our mindset. Test-driven development helped us to trust our code and work in small batches (dozens of commits daily per developer is typical). […]
Additionaly, pair and mob programming helped us further trust our code before we pushed it to the trunk. Continuous review works great too. Post-merge reviews also work well […]
Birgitta Böckeler, Nina Siessegger: On Pair Programming
I am very much in line with all eXtreme Programming (XP) practices except for one: pair programming. I use pair programming when coaching development teams or when bringing new developers up to speed. Birgitta and Nina call this strong-style pairing best suited for knowledge transfer (see the section about pairing styles). I can clearly see the value of pairing in this case.
I am less convinced when it comes to everyday programming. With driver-navigator pairing, I almost always end up as the navigator who “reviews the code on-the-go, gives directions and shares thoughts”. As the navigator, I had to watch developers ignore best practices like TDD, refactoring, simple design and coding standards over and over again. These pair-programming sessions are too often a waste of time.
Over the years, I have come to the conclusion that pair programming and all the other continuous-delivery (CD) practices - a superset of the XP practices - only work under certain preconditions.
All practices or nothing. A single practice like pair programming or TDD isn’t enough to write better software faster. The CD practices are a socio-technical system with many positive feedback loops. If you pick only some practices and ignore others, your benefits will quickly degrade. The virtuous cycle eventually turns into a vicious cycle.
Company culture. Company leaders must say that the software organisation will use CD principles and practices for development from now on. They must be clear that this not a short-lived fashion but a long-term effort. And then, they must walk their talk and get everyone else on board. Eventually, the company lives the same culture for software development. Discussions of why you should apply a practice are a thing of the past.
Personal attitude. The primary goal of pair programming is writing better software faster than a single programmer could do. Teaching your pair practices like TDD, refactoring or simple design, let alone convincing them to apply these practices, is not the responsibility of the more experienced pair. Refining these skills is, however.
Convincing is the responsibility of the company leadership, learning is the responsibility of each developer. It’s part of continuous improvement, which only works if people are curious to learn new things and open to change their behaviour. The company contributes the safe environment and the people the right attitude. Then, continuous improvement has a good chance.
If these preconditions are not satisfied, you will struggle or even fail to introduce CD practices in development teams. I think that this perspective is missing in the article.
Birgitta and Nina do an excellent job in listing the benefits and challenges of pair programming. Pairing is exhausting, because “working so closely with another person for long stretches of time is intense. You need to communicate constantly and it requires empathy and interpersonal skills.”
I am pretty bad at finding potential solutions for problems while talking someone else through my thinking. The authors address this fairly common problem in Pairing with lots of Unknowns.
When you work on a large topic where both of you don't have an idea how to solve a problem, the usual pairing styles often don't work as well. […] Researching and experimenting together works in some constellations, but it can also be frustrating because we all have different approaches to figuring out how things work, and we read and learn at different paces.
In such cases, the authors recommend to use pair development instead of pair programming. Pair development allows you to work alone for some time to search for potential solutions and try them out. After some time, the pair gets together and discusses their options. This is the classic inspect-and-adapt approach to cut through the complexity of problems.
I believe that neither solo nor pair programming alone yields the best results. Pair development seems to strike the right balance between the two extremes. Each pair can adjust it towards pair or solo programming - as they see fit.