How to Debug Complex Embedded Systems in the Field
Debugging in Tight Confined (and really cool) Spaces
During a recent trip to a Dallas airfield, I utilized the Active-Pro Debugger and Logic Analyzer to debug a sophisticated multiprocessor embedded system used in aircraft identification. This device is essential for maintaining accurate and reliable aircraft tracking and communication. Here’s how the Active-Pro proved indispensable in achieving operational success within a remarkably short timeframe.
Inside the cockpit of the RC-135 Aircraft
Debugging Complex Systems that Require Personnel, Equipment and Absolutely No Breakpoints
Upon arrival, I immediately set up our embedded device within a rack of communication equipment in the cockpit of the aircraft. The tiny Active-Pro connected to the device and also connected to my laptop was all that was necessary for gathering all that data we needed. These tests were using equipment and personnel that had to be configured, coordinated and timed successfully for the system to operate. Firmware breakpoints were not an option, and capturing the data we needed had to be accurate, complete and contain all of the data necessary to solve any issues since rerunning the tests was a time consuming, expensive proposition. Connecting the Active-Pro Debugger and Logic Analyzer via the Active Debug Port, I initiated comprehensive 45-minute captures of all system activity. This included hardware signals, communication packets, and firmware operations while the system actively tracked aircraft.
Gathering All the Data Needed to Solve System Faults
During the length of the test cycle, the system logged several faults. I used the Active-Pro to capture all of the traffic, signals and variables before, during and after the faults. By examining the captured signals, communications, and internal firmware states, I pinpointed the errors along with what lead up to the errors, and how the errors rippled through the system signals, firmware and communication packets. I revised the firmware and FPGA Verilog code based on my findings.
I updated the firmware through the firmware download application that we designed for the device. The new version of firmware and FPGA code worked like a champ, and the system ran successfully. A subsequent 60-minute Active-Pro capture confirmed the system’s perfect operation across various modes.
This experience underscored the Active-Pro's critical role in debugging complex embedded systems. The device facilitated extended captures of communication packets, hardware signals, and internal firmware states—capabilities crucial for diagnosing and resolving issues in real-time systems without the need of breakpoints, or bulky test equipment. The success of this project, evidenced by precise, factual traces of system operations, demonstrated the reliability and effectiveness of our design in a live aviation communication system.
Active-Pro attached to our test jig that breaks out the important signals and the Active Debug Port to the microprocessors
In conclusion, the Active-Pro Debugger and Logic Analyzer was pivotal in ensuring the success of debugging this complex embedded system with incredibly tight size constraints and test setup requirement. Without it, debugging the intricate interactions of hardware and firmware in such a detailed and timely manner would have been unfeasible.
Tim Harvey
Principal Electrical Engineer
WebSec Corporation