3 More Commonly Overlooked Techniques for Developing Reliable Firmware

In "3 Commonly Overlooked Techniques for Developing Reliable Firmware," we examined how the use of error-correcting codes (ECC), robust watchdog solutions, and assertions can help you improve the reliability of your embedded system. While we looked at just three techniques, there are so many more to discuss. In this post, we are going to examine three more commonly overlooked techniques for developing reliable firmware that I think will help you to elevate the quality of your embedded software. 

Tip #4: Use Trace Tools

One of my favorite techniques for improving reliability is to use trace tools throughout the development life cycle A lot of teams will ignore trace tools until there is a problem with their device. They’ll then trace their code to try and understand what is causing the issue. Unfortunately, if they have not been tracing their code during the rest of their development, they’ll have no clue what is going on! Is that 80% CPU utilization normal? The response time of task A is 10 ms. Has it always been that long?

Trace tools are the oscilloscope for software developers. You would not try to develop and debug a printed circuit board without having an oscilloscope. The oscilloscope allows you to visualize what the electrical signals on the board are doing. They provide critical insights to help you verify the design or debug an issue. Trace tools are no different for software! They help you to visualize how your system is behaving. You can even generate reports that help you understand how tasks interact, and their response and execution times along with a plethora of other data. 

Related:Abstracting Your Hardware With an AI-Generated HAL

I always recommend setting up a trace tool from the start of the project. You can then trace your code with every feature to not just understand how that feature affects your system, but to help you catch strange and unexpected behavior. (Some of these tools can even provide observability into your production devices, but that’s for another time). 

Tip #5: Leverage Test-Driven Development (TDD)

When I first encountered TDD, I was highly skeptical. The process of writing a failing test first before writing any production code was completely alien. Weren’t you supposed to write your code, get it working, and then develop your tests? TDD was completely flipping the idea of testing. The logic behind it was interesting, but would it improve reliability and code quality?

From my experiences using TDD over the last seven years or so, I’ve found it to be one of the best techniques for developing more reliable firmware. Writing tests first help to create verified tests that you know will fail. When we used to write tests after the fact, there was no guarantee those tests would catch what we were hoping they would. Seeing the test fail, and then only writing the code necessary to pass that test, helps to ensure that the test will catch any issues. 

Related:3 Techniques to Simulate Firmware

TDD is also interesting because it allows your tests to drive the development of your software. How often do we get ahead of ourselves and write a whole bunch of code only to discover that it isn’t needed? At this moment in time, if you don’t have a test for that piece of code, it doesn’t belong in the production code. Again, this is a little strange at first and can actually feel like you are too slow. But how expensive is it to track down bugs and issues later in the development cycle? I’ll give you a hint; exponentially more expensive! 

TDD has a lot of benefits that you might not immediately think about. First, the tests you write can be added to regression tests to ensure that new features in the future don’t break working features. Next, your tests can then be added to a DevOps pipeline that ensures you have software that is constantly integrated and tested. Finally, TDD helps guide you to write code that is more scalable and portable. To leverage TDD effectively, you must abstract your hardware from your application. This then opens up the possibility for the last technique we will discuss simulation.     

Related:3 Commonly Overlooked Techniques for Developing Reliable Firmware

Tip #6: Use Simulation

Simulation is a technique that allows you to run and test your application code off target. We all know that at the start of a project we won’t have our production hardware available. For most embedded developers, that means you run out and get a development board and create a franken-board. While this is fine, working on-target can be slow and tedious, even though it may seem only natural to a developer. The problem I often find with teams working with hardware first is that they often tightly couple their application code to the hardware and build their system from the ground up. 

The reliability of the application code can be dramatically improved by using simulation. Developers can not only get the application in front of key stakeholders early, they can also integrate their application into a CI/CD framework. Just like with TDD, this allows tests to be performed constantly, ensuring that the application is working as expected. Running the application in multiple environments can also help improve confidence that the firmware works.

In some cases, it may even be possible to simulate the behavior of the software system using modeling tools. These high-level tools can then be injected with various inputs and data to ensure that everything works as expected. Instead of hand coding these applications, one might use a code generation tool to keep the simulation and the code in-sync. Changes to the model can then automatically update the code, minimizing the opportunities for bugs and other issues. 

Conclusions

The ability to write reliable firmware is a critical skill for embedded software developers. Due to the IoT, more devices than ever before are being designed to run 24/7. You can’t have a smart light switch stop functioning. How funny is having to power cycle your house? I’d guess not very funny to the average person. The three additional techniques that we discussed today can help you to develop more reliable firmware. The path won’t necessarily be easy. You may have to realign your thinking about how to develop firmware. At the end of the day though, not only will your products be more reliable, but your customers will be happier with your products.