Episode 27: Burkhard on Qt Embedded Systems
Welcome to Episode 27 of My Newsletter on Qt Embedded Systems
The war is back in Europe! In a criminal act, Russia is attacking and invading Ukraine - for the second time in 8 years. My heart goes out to the Ukrainian people suffering from Russian atrocities. I will happily help Ukrainians find jobs in Germany and the EU through my professional network.
My position towards dictatorships like Russia, China and Saudi Arabia has always been clear. I have never done business with these countries and I will never do.
Many companies now proudly declare that they stop their business with Russia. I consider this hypocritical. These companies should have stopped business with Russia in 2014, when Russia invaded the Ukraine for the first time. Even better, they should have never done business with Russia. I doubt that these companies will abstain from business with Russia once the war is "over".
European countries, especially Germany, should have reduced their dependency on Russian oil and gas over the last 20 years. Instead, they increased it to 40% on average.
These countries are surprised that rogue nations leverage this dependency to get away with a military invasion. I am not surprised. This happens when countries, companies and people trade liberal and democratic values for profit and greed.
Ukraine's fight for freedom and democracy is a strong reminder that freedom and democracy must not be taken for granted but must be fought for every day. I hope that people in Western democracies hear the wake-up call before it's too late.
It's hard to move from war back to mundane topics like software architecture and team topologies. I hope that the rest of my newsletter can take your mind off the heavy topics like war and COVID-19 - at least a little bit.
Enjoy reading and take care - Burkhard 🇺🇦💜
Within two days after publication, the post garnered more than 1000 views. That's by far the best performance of any of my posts ever. Many people seem to be interested in creating dependency graphs for (legacy) software.
The post looks at dependency graphs of CMake targets: executables and libraries. CMake can generate the graphs in dot format with its Graphviz options. These graphs cover the container and component level of Simon Brown's C4 diagrams. So, they give a good idea of the current software architecture.
You can use the dependency graphs to speed up the builds of a CI pipeline. If a library changes, the graph tells you which other libraries and executables need rebuilding and which not.
The commentators suggested some other tools to create dependency graphs. SourceTrail generates class diagrams. Unfortunately, it has been discontinued at the end of 2021. I evaluated CppDepend - one of SourceTrail's competitors - in August 2020. CppDepend didn't cut it.
Doxygen seems to be a good option to generate graphs and diagrams, even if you haven't commented your code properly. Like CMake, Doxygen generates .dot files.
In his book Test-Driven Development (p. 114), James Grenning defines fakes and mocks as follows:
A fake "provides a partial implementation for the replaced component".
A mock "is programmed to return specific values to the [class under test]". It deals "with a situation where multiple calls are made too, and each call and response are potentially different".
The MockCanBusDevice, I test-drive in the post, is a mix of a fake and a mock. It behaves like a SocketCAN device. It enables tests to control whether opening a CAN bus device fails or whether writing to a CAN bus device fails. The MockCanBusDevice makes it easy to test these error scenarios. I use the MockCanBusDevice in the excavator terminal I currently help building.
My Review of the Book "Team Topologies"
With Team Topologies, Matthew Skelton and Manuel Pais have written a brilliant book how to create a high-performing software development organisation with the best team structure and interaction. What is best today may not be best in a couple of weeks or months. Organisations are complex adaptive systems that change as the result of the interactions between their components (e.g., the teams) and the interactions with other systems (e.g., the rest of the organisation, competitors, suppliers, technology and market trends). The resulting changes are hard to predict and hard to control.
Chapter 2: Conway's Law and Why It Matters
Understanding Conway's law is the cornerstone for good team design. "Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations." In short: The software architecture always reflects the organisation structure.
You can use Conway's law to your advantage by shaping the organisation structure according to the desired software architecture. This is called the reverse Conway manoeuvre. A loosely coupled architecture with cohesive components translates into an organisation with cross-functional teams working in a self-dependent way. What makes for a good architecture also makes for a good organisation structure. In the words of Michael Nygard: "Team assignments are the first draft of the architecture."
Ruth Malan points out the implications of Conway's law: "If we have managers deciding which services will be built, by which teams, we implicitly have managers deciding on the system architecture." So, these managers better have the required technical expertise. This is why Apple requires its leaders to "know the details of their organizations three levels down" (see my post How Apple is Organized For Innovation). An architect is required to have both technical and social skills, as "organization design and software design are, in practice, two sides of the same coin".
As between software components, "not all communication and collaboration [between teams] is good". The team structure should "minimize the number of communication paths between teams" and "encourage teams to communicate who wouldn't otherwise do so". Unexpected intensive communication between two teams often points to problems in the software design (e.g., a bad interface).
Chapter 5: The Four Fundamental Team Topologies
The authors identify four fundamental team topologies: the stream-aligned team, the enabling team, the complicated-subsystem team and the platform team. "The stream-aligned team is the primary team in an organization, and the purpose of the other fundamental team topologies is to reduce the burden on the stream-aligned teams."
Feature or product teams are the prototypical stream-aligned teams, where product teams may be made of one or more feature teams. Streams can be defined by many criteria like customer types, business areas, geography and user personas. The stream for different teams must be clearly separated to minimise communication.
A stream-aligned team has the following characteristics.
It gets valuable features in the hands of customers at a steady pace - measured by Continuous Delivery metrics.
It adapts quickly to stakeholder feedback, that is, it is agile.
It has minimal communication with other teams.
It has enough time and the necessary skills to reduce technical debt or to avoid it in the first place.
It "proactively and regularly reaches out to [...] complicated-subsystem, enabling and platform [teams]" to ensure that these teams know its most pressing needs.
It feels in charge of its own fate.
The job of stream-aligned teams is to get valuable features into customers' hands as quickly as possible. Hence, they don't have the time to make components reusable for other teams, to improve the CI/CD pipeline or to get lots of legacy code under test. This is the job of enabling teams, which are composed of technical specialists. These specialists fill the knowledge gaps of the stream-aligned teams.
"The end goal of an enabling team is to increase the autonomy of stream-aligned teams by growing their capabilities [...] If an enabling team does its job well, the team that it is helping should no longer need the help from the enabling after a few weeks or months; there should not be a permanent dependency on an enabling team."
Enabling teams don't help with the execution but provide technical guidance. Members of an enabling team will eventually move to teams of other types.
The enabling team shows the following behaviours among others.
It anticipates the needs of stream-aligned teams and regularly checks "when more collaboration is needed".
It facilitates learning across all stream-aligned teams.
Imaging systems for recognising traffic on roads, the ripeness of fruit or the remaining cooking time are examples of complicated subsystems. Members of complicated-subsystem teams must be experts "to understand and make changes to the subsystem". Embedding such experts in the stream-aligned teams would be a waste of time and money. The complicated-subsystem team synchronises regularly with the stream-aligned teams to work on the right requirements.
Platform teams build services, libraries, Linux systems or CI/CD pipelines so that stream-aligned teams can "deliver product features at a higher pace, with reduced coordination". A platform team may consist of stream-aligned teams, complicated-subsystem teams and platform teams. A platform team
collaborates closely with stream-aligned teams to anticipate their needs,
produces "services" that are reliable and can be used with little to no communication, and
understands that stream-aligned teams may adopt its "services" with some delay.
Often, a good platforms is itself built on a platform. For example, the libraries and services of an embedded system could be built on Qt. A platform is managed like a product. It is "not simply driven by feature requests from Dev teams; instead, it is created and carefully shaped to meet their needs in the longer term".
Chapter 7: Team Interaction Modes
The authors identify three essential team interaction modes:
Collaboration: working closely together with another team
X-as-a-Service: consuming or providing something with minimal collaboration
Facilitating: helping (or being helped by) another team to clear impediments
"Poorly defined team interactions and responsibilities are a source of friction and ineffectiveness."
Collaboration: Driver of Innovation and Rapid Discovery but Boundary Blurring
Stream-aligned teams use collaboration mode in their interaction with platform and complicated-subsystem teams to align on the needs. Once the platform team has delivered a reusable component, the stream-aligned teams use it as a service. Heavy collaboration with many teams slows down a team. Therefore, a team should not use this mode with more than one team at a time.
Collaboration mode works best between two teams with high mutual respect. Teams located in different countries or time zones are typically ill-suited for Collaboration mode. Collaboration could be improved by "rewarding one team for the work of the other team".
X-as-a-Service: Clear Responsibilities with Predictable Delivery but Needs Good Product Management
Stream-aligned and complicated-subsystem teams use X-as-a-Service (XaaS) mode to consume "services" from platform and complicated-subsystem teams. One team can use XaaS mode with many other teams, as the communication overhead is minimal. Ownership of a "service" always lies with the team producing the service and never with the. team consuming it. There is never shared code ownership. In XaaS mode, the speed of innovation (e.g., adding new value to a system) is slower than in Collaboration mode, because it happens across team and software boundaries.
The team providing X-as-a-Service should "emphasise the user [or developer] experience". The value of X for the consuming team should be high compared to the effort to use X.
Facilitating: Sense and Reduce Gaps in Capabilities
Enabling teams use Facilitating mode to help other teams do their work better. "A team with a facilitating remit does not take part in building the main software systems, supporting components or platform, but, instead, focuses on the quality of interactions between other teams building and running the software." A team uses Facilitating mode only with a few other teams at the same time.
"People in the stream-aligned team need to be open to being helped by the enabling team; they need to have an open mind to new approaches and be aware that the enabling team has probably seen some better approaches."
In an upcoming post, I will use Simon Brown's C4 diagrams to visualise the architecture of Qt embedded systems. Here is the first section of my post and the Context or system diagram of a harvester terminal.
The C4 model looks at software on four different levels.
Context: An embedded system is part of a larger ecosystem: the system context. The Context diagram describes the interactions between the embedded system and other systems like users, machines, cloud services and operating conditions.
Containers: An embedded system consist of one or more executable units: the containers (not related to Docker containers). The containers of an infotainment system are the radio, media, phone and navigation applications and their services. The Container diagram shows the responsibilities of the executable units, their interactions and the technological choices for the interactions.
Components: Containers are decomposed into modules or components. The Component diagram shows the responsibilities of the components and their interactions. It also shows which technology (e.g., QML, Qt, Boost, Protobuf or C++) is used to implement a component.
Code: Code diagrams should be generated automatically. I use them rarely, because expressive unit tests from TDD are typically more illuminating than code diagrams. A good use could be the visualisation of influential functional requirements that shape an architecture.
The diagram shows the Context of a harvester terminal, which is much more instructive than my DIY diagram from the post Architecture of Qt Embedded Systems: Getting Started.
The notation is self-explaining and "works well on whiteboards, paper, sticky notes, index cards and a variety of diagramming tools". The brilliant thing about the notation is that we can use it on all four abstraction levels. There is no need to use different notations on different levels.
The Qt Company announced that Stellantis uses Qt in the infotainment systems of four of its brands: Peugeot, Citröen, DS (the luxury brand of Citröen that got its name from the Citröen DS - a.k.a. the "goddess"), and Opel. For Peugeot and Citröen, it's at least the third generation of infotainment systems with Qt. The first generation was built by Magneti Marelli and hit the market in 2009 or 2010. The second generation was built by Continental and was released to the market in 2012 or 2013. The third generation was released in 2017.
For Opel, it's the first generation with Qt to my knowledge. Well-informed sources tell me that other Stellantis brands like Chrysler and Fiat also use Qt.
Basyskom helped the WEINIG Group - the world's largest manufacturer of machines for solid wood and panel processing - create the Wood Working Companion Specification for OPC UA and deploy the open62541 OPC UA stack on their machines. open62541 comes under the Mozilla Public License v2.0 and is the free open-source implementation of Qt OPC UA.
Professional cooking appliances, food scales, fruit sorting machines, packaging machines and many other machines must talk OPC UA these days. Qt OPC UA is a good choice - and Basyskom a good partner (see all their posts about OPC UA).