Episode 37: Better Built By Burkhard

Consequences of Choosing the Wrong HMI Terminal

Consequences of Choosing the Wrong HMI Terminal

Topcon sells its Opus control panels (HMI terminals) in lower 6-digit volumes per year. This makes Topcon one of the biggest terminal manufacturers in the world. For the last 1.5 years, I have been helping a small service company build the HMI for the driver terminal of an excavator.

When I joined the project, the decision to use the Topcon B6 was made. I expressed my doubts about choosing the B6 back in autumn 2021. The hardware of the B6 is on par with its competition but the board-support package (BSP) isn’t. The BSP was and still is based on Yocto 2.7 (Warrior). Yocto 2.7 reached end of life in October 2019. So, it had already been unsupported for 2 years, when the project started.

In autumn 2021, I could only argue in general that working with an unsupported and outdated BSP is a bad idea. We would have to invest some extra effort to build Qt 6 for Yocto 2.7. If a bug occurred in the Linux system, we would have to search through 2 years of bug fixes and hope that we could apply all the necessary patches. We also hoped that Topcon would do the reasonable thing and upgrade to Yocto 3.1 (EOL: April 2024) or even Yocto 4.0 (EOL: April 2024) - before the release in December 2022.

As I feared, this turned out to be too much wishful thinking. It’s now March 2023. The excavators haven’t been delivered to the customers yet. The excavator manufacturer is losing money. The BSP is still on Yocto 2.7. Topcon has no plans to upgrade to an up-to-date Yocto version, as it doesn’t even see the need. The service company is too small to have leverage over Topcon and finds itself between a rock and a hard place.

By now, I can point exactly to the consequences of choosing the wrong terminal with the Topcon B6.

• As we wanted to write our applications in a future-proof way, we decided to use Qt 6. Yocto only supports Qt 6 builds from version 3.1. As a workaround, we built Qt 6 against the Yocto 2.7 SDK.

• In summer 2022, we found out that Qt 6.2 played the video from the rear-view camera with a 3-second delay. By default, Qt 6.2 and the old GStreamer libraries from Yocto 2.7 don’t set the latency at all. We fixed the 3-second delay by setting the latency in the C++ code of the Qt-GStreamer plugin to 50ms.

• Shortly before the BAUMA trade show in October 2022, we found out that the alternative bird-view camera system also showed a 3-second delay - despite our fix. We tried a couple of other pipeline configurations by changing the C++ code of the Qt-GStreamer plugin - without success.
  We found out that both cameras worked fine out-of-the-box with Qt 5.12 (the default Qt version for Yocto 2.7). We downgraded the Qt applications to Qt 5.15 and built Qt 5.15 against the SDK of Yocto 2.7. The cameras worked fine for BAUMA.

• By default, the Yocto layer meta-qt5 doesn’t support Qt 5.15 in Yocto 2.7. However, downgrading meta-qt5 to Yocto 2.7 is a fairly straightforward exercise. It took us less than a day. As Qt 5.15 is the only Qt 5 version still supported, every Topcon customer would spend a day or more on this downgrade.

• After two months of intensive testing, the excavator manufacturer found another bug related to the cameras. When the ignition is off for 10s, the terminal goes into sleep mode with microprocessor off. When the ignition is on again, the terminal wakes up and the Linux system resumes its work. The video-for-linux (V4L) subsystem doesn’t resume properly in some cases and makes the application crash.
  Our workaround was to introduce an application manager running in a separate process. The application manager would terminate the single HMI application safely on ignition-off and restart it on ignition-on. The HMI restarts easily under 2s, the requirement from the OEM. However, the camera takes 20s to restart. Hence, the driver doesn’t see the rear-view video during this time.
  The workaround makes the system architecture more complicated and less stable. I think it highly likely that the V4L wake-up issue is fixed in Linux systems built with newer Yocto versions like 3.1 or 4.0. If not, we would stand a a much better chance to fix the problem than with a 3-year old Linux system.
  The workaround seems to be good enough to get the excavators out to the customers - with a 2.5-month delay.

• There is a workaround for the workaround that causes a 20s blackout of the rear-view video. We could reduce the number of times the driver has to sit through this 20s blackout. The terminal could go into sleep mode only after 2, 3 or even 4 hours so that the camera restart happens only once a day, when the driver starts working in the morning.
  The terminal application polls a sys file to be notified about ignition changes. This covers the transition from ON to LOW_POWER mode. The transition from LOW_POWER to SLEEP mode requires the application to listen on a socket for these events. Topcon provides a library under GPL!
  If we used this library in our code (e.g., in the application manager), we would have to make our code publicly available. Moreover, we would have to introduce inter-process communication between the application manager and the HMI application. So far, they both read the ignition status from the sys file. That is extra effort and makes the architecture more complicated.
  I’d argue that it is unnecessary effort. There is no reason for Topcon to license its library under GPL. They could license it under LGPL-2.1-or-later as all the user-space libraries of Kernel modules are. Our workaround could be to rewrite the power management or find a properly licensed library that implements the power management for the iMX6 SoCs.

The outdated, unsupported and wrongly licensed BSP from Topcon forced us into more and more workarounds - and workarounds of workarounds. Topcon could have saved us a lot of time and money by providing an up-to-date, properly licensed and well-tested BSP. And not only us but all their customers building HMIs for agricultural and construction machines.

Topcon excuses these shortcomings with lower prices for their terminals. I doubt that for the B6. For the same volumes, the B6 costs pretty much the same as comparable terminals from CrossControl or Christ Electronics.

Even if the B6 was cheaper, it would have to be a lot cheaper. The lower price must compensate the OEM for the losses from delivering the excavators to their customers with a delay of several months. The OEM may get a bad reputation and may lose more business down the road. The same is true for the service company.

Topcon commits the same mistake as the SoC and SoM makers. Instead of adding value for their customers, they add costs.

The most important player of the system is still missing: the customer. [Their] core business […] is building the HMI or controller software for machines and devices [- and] certainly not building a custom Linux system with Yocto. They best buy the custom Linux system for a fixed price or for a premium on the [terminal] price, spend the time saved on their core business and get their products to the market faster.

Burkhard Stubert, The BSP is not enough!

At the start of an HMI development, the OEM and their supplier should evaluate several terminal options. They should have a special look at the BSP.

• With which Yocto version is the BSP built?

• How often did the terminal maker update the Yocto version in the past?

• What is terminal maker’s plan for BSP updates in the future? Is the plan published on their web site?

• What special value does the terminal maker add to the BSP? Where does the terminal maker reduce the costs for its customers? (See The BSP is not enough! for some ideas).

I’d strongly recommend to write down your decisions as architectural decision records (ADRs). ADRs give you the time to think about options and consequences and way them against each other. ADRs tend to lead to better decisions - especially when coupled with the Advice Process described by Andrew Harmel-Law.

In short, you’ll save a lot of money and trouble when you choose the right terminal at the start of your project. Think slow at the beginning of the project to move fast during the project.

My Content

KDAB Insights: Building a Smart User Interface - what’s really involved? (video)

Frances Tait from KDAB interviewed me on the KDAB Insights channel about my mission:

I want to free smart machines from dumb user interfaces, one machine at a time.

We start by defining dumb and smart user interfaces. User interfaces become smarter by making common tasks easier and eventually by automating tasks. AI will play an important role in this transition. However, fully autonomous machines are still far away. Humans are much better than AIs in taking decisions in new situations. In short: “Automate the boring stuff and let people do the creative stuff.”

Why don’t companies spend a bit more money at the beginning to earn a lot more money in the mid-term? My short answer: “Many companies are just focused on costs. They do not look at the opportunity. […] It’s a risk. But risk also means opportunity, more value, more return on investment.” Tesla showed this mindset, when they integrated the functionality of most of the 50-80 ECUs in a car into one supercomputer.

Yes, part of my mission statement is, indeed, to put myself out of work by teaching my customers how to do things on their own. I deeply believe that my customers should have control over their own fate. I help them get there. “If customers need me for the next 5 or 10 years, I have done something wrong. And the customers as well.”

You probably find some of my statements debatable. So, watch the video and let’s discuss my mission in the comment section.

Thank you to Frances and KDAB for having me as an expert on their insights channel 🙏