Attacks on Embedded Devices Are Too Easy
Episode 61: Better Built By Burkhard
Dear Reader,
My highlight from last month was the live discussion with Jon Oster, the security expert at Toradex, about the EU Cyber Resilience Act (CRA). You find the recording on the Torizon web site.
I think we hit home one point. The most important feature to comply with the EU CRA is a reliable secure OTA update. Even without the EU CRA, reliable OTA updates happen to be the one thing that must work when you first release embedded devices. They enable you to add the real features one by one later. If you break this rule, you might end up like VW: VW had to update roughly 30,000 ID.3 cars - manually!
Toradex offers an end-to-end solution for secure OTA updates. They achieve automotive-grade security with the Uptane framework. Jon showed a slide comparing Uptane’s threat model with that of other OTA update solutions. Unsurprisingly, Uptane is the clear winner, as it addresses threats (potential attacks) that other solutions don’t even have on their radar.
We didn’t even mention another extremely useful feature of Uptane: management of the private keys for signing the U-Boot, Linux kernel, rootfs and container images during the build. Uptane offers a way how to keep the private keys totally offline in a hardware security module (HSM). Yashovardhan Bapat, a Toradex intern working in Jon’s team, has written an excellent article about this topic.
I would have loved to have Yashovardhan’s article at my disposal last year, when I implemented a makeshift solution (basically, an encrypted USB drive containing the private keys) for a customer. Managing private keys is a tricky and often overlooked feature, which takes time to implement properly.
I freely admit that I called Toradex’s use of Uptane “over-engineering” for many embedded devices in one or maybe two meetings with Jon. The more I learn about Uptane and security the more appealing I find Uptane. It becomes even more appealing, as you essentially pay a little bit more per Toradex SoM but get secure OTA updates with Uptane in return. In contrast, you must implement secure OTA updates for the SoMs from other providers on your own or hire experts. From my own experience, I can tell you that such an implementation can easily take 3-6 months - if you roughly know what you do.
We touched briefly on the topic when the support period for a product starts according to the EU CRA. Jon corrected my slightly confused understanding and pointed to the definition of placing a product on the market in The ‘Blue Guide’ on the implementation of EU product rules 2022 (section 2.3). The support period starts, when the ownership of an individual product is transferred from one legal or natural person to another for the first time. The new owner must be based in the EU. Ownership transfer can happen, for example, by buying, renting, subscribing, donating or given as a gift.
Let me clarify the term placing a product on the market with an example. Tomra - a company from Norway, which is not in the EU - sells its reverse vending machines (RVM) to many REWE supermarkets in Germany. Once a supermarket pays for an RVM and becomes the new owner, the support period starts for this individual RVM. If a supermarket buys one RVM on 16 June 2025 and a second RVM on 21 May 2026, the support period for the first RVM starts on 16 June 2025 and the support period for the second RVM starts on 21 May 2026. The support period works similar to the warranty period.
One participant made a comment to this effect: Many service providers try to scare the bejesus out of product companies with the EU CRA to make higher profits. Unfortunately, there are always companies using FUD (fear, uncertainty and doubt) to make a quick buck. However, I am also aware of the many machines and devices on the market with glaring security holes. Even the security-conscious automotive industry makes attacks on cars far too easy, as the two examples below of the Subaru hack and the keyless car theft show. The damages are real, high and likely. You should, indeed, be a bit scared and use this as the motivation to make the life of the bad guys a bit more difficult.
Enjoy reading,
Burkhard 💜
Attacks on Embedded Devices Are Too Easy
Sam Curry: Hacking Subaru: Tracking and Controlling Cars via the STARLINK Admin Panel
Sam Curry is a white-hat hacker and has found severe vulnerabilities in many cars, modems and web sites. He helps the companies fix the vulnerabilities before he writes about them on his blog.
In the Subaru hack, Sam could lock, unlock, start and stop any Internet-connected Subaru in the US, Canada and Japan. For each car, he could retrieve the vehicle PIN, the location history for a full year, and personal data about the customer including address, authorised users and emergency contacts. He could do all this remotely from his comfy chair at home.
The alarming thing is how easy it is to break into the cars. Here is how Sam did it (see his excellent article for more details).
Sam’s first attempt to get into the car was through the MySubaru app, which allows users to remotely execute car commands. However, he could not get past the app’s authentication.
In his second attempt, Sam tried to find a “publicly accessible employee-facing application with broader permissions than the customer-facing apps”. A good example is a web site for Subaru’s support personnel that should only be accessible from inside the company but is publicly accessible by all and sundry. This approach has worked for Sam many times before.
Sam and a friend scanned the URL
subarucs.comfor subdomains and found the URL for the STARLINK Admin Portal, where STARLINK is the name for Subaru’s in-vehicle infotainment system. The admin portal is for authorised Subaru employees only and must not be in the public Internet.Looking at the source code of the portal’s login page gave the hackers a hint in which directories to look for interesting JavaScript files. They found the file
login.jswith code how to reset a forgotten password. They could put together aPOSTrequest with a fictional email address, password and confirmed password and send it to the URL endpoint/forgotPassword/resetPassword.json. The response was anHTTP 200message containing the string“error”.Sam and his friend also found an endpoint that would return the user’s security questions for a valid email. A security question is much easier to bypass than a code sent to the legal user’s phone.
A search for “STARLINK Subaru” on LinkedIn yielded a list of software engineers. The forgotten-password request worked for the fourth email. The admin portal showed the security question for this employee.
The dialog asking the security question was implemented in JavaScript on the client side. The hackers commented out the single line showing the dialog and reloaded the page. And bingo! They had full access to the admin portal. Of course, Sam tried out his newfound power - but always with consent of the car owner.
By entering the last name and ZIP code, he could retrieve the locations where the car had been the last year.
By entering the license plate, he could retrieve the customer’s last name, zip code, phone number, email and VIN.
Just with the license plate, Sam could add himself as an authorised user to a friend’s car and unlock the car remotely. He could have locked and unlocked, started and stopped any Subaru in the US, Canada and Japan remotely.
Subaru fixed the vulnerability within 24 hours after Sam reported it. The vulnerability was never exploited.
I find it scary how easy it is to take over the cars. My very basic knowledge of JavaScript and HTTP requests and a bit of googling would be enough to execute such an attack.
The Subaru hack shows that the attack surface is much bigger than the product manufactured by a company (e.g., car, harvester or excavator). The attack surface contains all the devices connected directly or indirectly to the product and the software running on them. Bad actors can use any of these devices to attack any product in this network. You might remember from my last newsletter that the NotPetya worm was initially planted in Ukrainian tax software and eventually paralysed the shipping giant Maersk.
If thieves had detected this vulnerability first, they could have easily stolen thousands of cars. The financial damage would have been enormous. Most likely, Subaru would have had to pay for the damages, as the open-door policy of their admin portal constituted gross negligence. If taken seriously, the EU Cyber Resilience Act (CRA) helps you avoid these damages, as I argued in my previous newsletter Why the EU CRA is Important.
Most agricultural, construction and industrial machines are now part of the Internet of Things (IoT). The manufacturers have admin portals similar to Subaru’s. Their customers can access their machines through apps and web sites on their phones, tablets and PCs. How high is the likelihood that these manufacturers do a better job with their admin and customer portals than Subaru, Kia and many other well-known OEMs? Very very very low, of course!
Ken Tindell: CAN Injection: keyless car theft
This CCTV footage (start watching at 1:30 minutes) shows how two thieves drive away with a car within two minutes. The attack works for most cars of the major brands including “Jeep, Maserati, Honda, Renault, Jaguar, Fiat, Peugeot, Nissan, Ford, BMW, Volkswagen, Chrysler, Cadillac, GMC - and Toyota”. It works for your and my car - despite our cars having immobilisers and other security features.
Although I have been well aware of the inherent insecurity of CAN buses for a long time, I was still shocked by the video. And I’d assume that you are too.
The CAN injection attack roughly works like this.
The thieves break into a front headlamp and tear out the CAN wires.
The thieves attach a CAN injector device to the CAN wires. The internal CAN bus has no security. The CAN injector is in the case of a Bluetooth speaker. It consists of a CAN transceiver, which is programmed to imitate the smart key ECU. The device is sold as an “emergency starter kit” in the dark web for up to 5,000 USD - a worthwhile investment for the thieves. Its components cost 10 USD.
When the CAN injector receives the “car ready” message, it renders mute the other ECUs on the same CAN bus. The other ECUs still receive all messages but cannot send any messages. The CAN injector is the only device on the CAN bus that can sill send messages.
The CAN injector sends the message “smart key is valid” 20 times per second. A gateway relays the messages from the control bus with the CAN injector, smart key and door control to the powertrain CAN bus with the engine control. In response, the engine control deactivates the immobiliser.
The real smart key ECU would normally send messages “smart key is invalid” many times per second. These messages would clash with the messages “smart key is valid” from the CAN injector. Any ECU could detect a serious problem and kill the car so that only an authorised technician can get it working again. However, the clash cannot happen, as all other ECUs are mute.
The thieves press the Play button on the device, which then sends a burst of “Unlock the door” messages. The car diligently unlocks the door.
The thieves get into the car, start the engine by pressing the Start/Stop button and drive away.
The attack is so easy, because the first two generations of the CAN protocol - CAN Classic and CAN FD - don’t have any built-in security. CAN Classic was introduced in the 1980s, when the Internet didn’t even exist or was just about to be created. Hence, there were no cyber threats. CAN security - CANsec, for short - was only added to the third-generation protocol CAN XL.
Almost all agricultural and construction machines, cars, trucks, motor cycles, e-bikes and boats use CAN Classic or CAN FD and will use it for many years to come. As soon as you have control over a device that can send and receive messages over the CAN bus, you can run the above attack. CAN buses are wide open for attacks.
The attackers don’t even have to attach a device physically to the CAN bus. They only need remote access to a device that is connected to a CAN bus. On agricultural and construction machines, that device could be any ECU with a 4G/5G Internet connection like the telematics unit. The following factors make it easier to run the CAN injection attack on such an ECU.
The telematics unit provides ssh access. Bad actors can easily guess the password or find out the password with a dictionary or brute-force attack. This gives attackers the full power of a Linux terminal.
All applications run with root privileges on the device - including command-line terminals.
The device has the CAN utilities installed. Hence, attackers can send any CAN commands with the utility
cansend.The telematics unit provides a remote terminal or a remote desktop via VNC or TeamViewer. Most open-source VNC solutions must run inside a VPN to be secure. Unfortunately, VPNs are major targets for bad actors, who find new vulnerabilities and ways to exploit them.
The online portal may have some debugging functionality to send CAN messages to the telematics unit, which forwards them to the CAN bus. Users and hence attackers may be able to upload a file with a CAN trace to the telematics unit, which replays the trace.
And these are just the factors I can think off the top of my head. Bad actors are much more creative than I am. Most of the time they will find an easier way to unlock and start a car, as the Subaru hack above shows.
Ken gives two possible fixes for the CAN injection attack.
Quick and dirty fix. When the CAN injector mutes the other ECUs on the control bus, the other ECUs will send error messages - so-called diagnostic trouble codes (DTCs) - over the control bus. As the error is extremely rare, the gateway could detect the error condition and reject to forward the message “smart key is valid” to the engine control ECU on the powertrain bus.
This fix is a small change in the firmware of the gateway. Attackers could still work around it, but it would take them deeper expertise and more time. Manufacturers could use this time to come up with a proper fix.Cryptographic messaging fix. The proper fix would be for the gateway or engine control ECU to check that high-risk messages like “smart key is valid”, “unlock the door” and “start the engine” are authentic. The receiver requires proof from the sender that it is not an imposter. This would quickly unmask the CAN injector.
Generating an authentication code (a digital signature) for the high-risk messages is best done by a hardware security module (HSM). If you can still modify your board, you can select a SoC with a built-in HSM (e.g., a TPM) or connect an HSM via SPI or I2C to your board. Otherwise, you can emulate an HSM in software. Ken’s company Canis Labs has implemented an HSM emulator, which needs 300 KB of code and 200 bytes of RAM.
Additionally, manufacturers must install secret keys in the ECUs at the factory. The keys must differ from car to car. Otherwise, attackers could figure out the keys for one car and use it for many other cars.
Ken is sceptical that car manufacturers will implement these fixes anytime soon.
Car makers have learned over the years to act carefully when making changes to vehicle systems: what appears to be quick and simple often turns out to not be, and even a simple fix requires extensive testing to make sure there are no unintended consequences. So it will take some time to implement this.
Ken Tindell, CAN Injection: keyless car theft
If you use CAN Classic or CAN FD in your products without extra security features, you will violate at least the essential requirements Confidentiality Protection and Integrity Protection of the EU CRA. Even with the built-in security of CAN XL, you must do some extra work to comply with the EU CRA.
The CANsec module of CAN XL (see Table 1) protects devices against spoofing attacks as performed by the CAN injection devices, against replay attacks and against some other common attacks. However, it doesn’t protect against denial of service (DoS) attacks, where a malicious device or application spams the CAN bus with zero-ID messages. The smaller the ID of a CAN message is the higher its priority is. Other devices must stop their transmission if they see a message with a higher-priority ID on the CAN bus. Zero-ID Messages will block all other traffic on a CAN bus.
If you want to learn more about how to protect CAN buses, you should read Ken’s excellent article CAN Bus Security: Attacks on CAN bus and their mitigations.