Arduino Interrupts Suck, FPGA to the Rescue

It should be easy enough to read PWM signals using Arduino, right? At least that's what I thought, but I found that there's all sorts of gotchas and maybe even bugs in the Arduino libraries. I eventually gave up and built a custom FPGA to do the job.

Overview of PWM

PWM, or pulse width modulation, is just a single wire that alternates (pulses) between high and low voltage. It's up to the attached devices to decide how those highs and lows are interpreted. Variables include the pulse frequency and how long the signal is held high or low. Some devices care about the duty cycle, which is just the percent of time that the signal is high. Others, like an RPM sensor reader, only care about the frequency of pulses. Hobby servos care about the pulse width, or the time duration that the signal is high.

For hobby servos, the pulse width varies between 1 and 2 milliseconds +/- a few hundred microseconds depending on the servo. The servo translates the pulse width to rotational angles, so for example 1 millisecond may be full left, 2 millisecond full right, and 1.5 milliseconds is centered. The pulses are generated by the RC receiver, usually at 50 or 100Hz. The frequency has no effect on the servo position - only the pulse width matters. There's some good YouTube videos on the topic, and I think this one does a good job of explaining it.

A sample PWM signal (the yellow line) from a RC receiver. On the Y axis, voltage is pulsing between low voltage (close to 0V) and high voltage (close to 3.3V). The X axis is time, where each pulse is 2 milliseconds and the entire graph represents 70 milliseconds.

Since hobby servos work on fairly small timescales, timing is very important. Holding the signal high for just one millisecond too long could mean the difference between "full left" and "full right", or "full throttle" and "full brake". However, in terms of modern microcontrollers, 1 millisecond is a long time. Even the slowest of modern microcontrollers that you'll find on an Arduino board will run at 8MHz, which is one clock cycle every 1/8th of a microsecond. It may take a few dozen clock cycles worth of CPU instructions to change the signal from low to high, but that's still fast enough to have about a 4 microsecond response time (about 0.4% of a full rotation, or about 0.7° of precision assuming 180° full rotation).

Some microcontroller boards have specialized hardware to generate PWM signals, offloading much of the work from the CPU, and ensuring high accuracy. But what about reading PWM signals?

Reading PWM with Arduino

There's many examples of reading PWM using the Arduino API (here's a good overview). Your choices are:

• Use the pulseIn() function, which is a blocking call, making it impractical for all but the most basic examples.
• Use separate RISING and FALLING interrupt handlers. This would be great, except for the fact that Arduino limits you to a single interrupt handler attached to a pin at any given time. If you want two different interrupt handlers that alternate then you have to attach a new interrupt handler within each interrupt handler. That's gross, and slow.
• Use a pin CHANGE interrupt handler. This would be great if the handler was told if it's a rising or falling edge, but it's not. This means you have to do a digitalRead() inside the interrupt handler, but this approach is flawed if the signal changes between the interrupt firing and reading the pin.

I tried both approaches with interrupts, but mostly settled on using a CHANGE interrupt with a digitalRead() in the handler. The idea is to record the current time (in microseconds) when you see a rising edge, then at the falling edge subtract that recorded time from the current time, which yields the pulse width. It looks something like this:

volatile unsigned long pulse_begin_micros = 0;

volatile unsigned long pulse_width_micros = 0;

void changeISR() {

  unsigned long now = micros();

  bool high = digitalRead(PWM_PIN);

  if (high) {

    pulse_begin_micros = now;

  } else {

    unsigned long pw = now - pulse_begin_micros;

    pulse_width_micros = pw;

  }

}

void setup() {

  pinMode(PWM_PIN, INPUT);

  attachInterrupt(digitalPinToInterrupt(PWM_PIN), 

                  changeISR, CHANGE);

}

That does work, but only if your microcontroller has consistent pin change interrupt latency, and even then it may only work well most of the time.

Notably, this does not work well on any board I've tried that uses the Nordic nRF52840 (I tried the Arduino Nano 33 BLE Sense and the Adafruit nRF52840 Feather and ItsyBitsy). That chip is plagued with massive amounts of interrupt latency. My pulse width readings were rarely spot-on, usually varied by 50 microseconds, and often varied by over 500 microseconds. This is flat-out useless for reading hobby servo PWM signals. I never conclusively determined what was causing this, but I suspect it was higher priority interrupts associated with the BLE radio that preempted or delayed my pin change interrupts.

I have had better luck with boards based on a basic ARM Cortex-M0 (I tried the Adafruit Trinket M0 and Seeeduino XIAO). For the most part, reading PWM signals with this method is accurate to a few microseconds or better. However, I found that it occasionally "glitches" +/- 1 millisecond for one or maybe two consecutive pulses at 100Hz (so 10 to 20 millisecond glitches). Sometimes it's stable for 60 seconds, then it'll glitch once, then be stable again for a long time.

Is this really a problem?

A glitch like that probably isn't going to cause a big problem for some applications. As a test, I setup my components as a simple relay placed between the RC receiver and the steering servo and speed controller - read the steering and throttle PWM channels sent by the RC receiver, then generate new signals with the same pulse width which are sent to the steering servo and speed controller. I was able to drive the RC car for a few minutes without detecting any kind of glitch. Maybe a 10ms glitch is too short for me to detect, or maybe the servos have some kind of noise filter, or maybe the 50Hz output from the Arduino happened to skip over the glitches that I read from the 100Hz input.

However, I'm using a 3rd PWM channel to act as my kill switch, which also signals the navigation microcontroller that it has been killed or enabled. This signal would reset some internal state every time it glitches, which is a problem. This 3rd channel is just an on/off switch on the RC transmitter, which switches between 1ms and 2ms pulses. I set a 1.5ms threshold on my kill switch microcontroller - if the latest pulse is over the threshold then it's switched off, if it's under the threshold then it's on. When it's on, I turned on a LED and asserted a pin to HIGH. This pin drives a multiplexer select pin and also an input pin on the navigation microcontroller. The PWM read glitches were visible on both the LED and the navigation microcontroller.

I tried doing some sort of averaging technique, or requiring stable reads for a few pulses before confirming that the signal had in fact changed. This worked, and honestly I probably could have just accepted this and moved on, but I was wearing my perfectionist hat and this made me unsatisfied.

Requiring a stable read of 3 pulses before confirming that the signal has changed. The PWM signal (yellow) changes widths 3 pulses before the kill switch indicator (blue) changes. The top graph shows a longer timescale, and the part of the top graph that is not greyed out is zoomed in on the bottom graph. The zoomed in view clearly shows how the pulse width changed from 2ms to 1ms.

Investigating the glitch

This is definitely a glitch in my microcontroller, not a glitch in the PWM signal. I connected the PWM signal and the kill switch indicator pin to an oscilloscope, and set a trigger for changes in the kill switch indicator. The pulse width definitely did not cross my threshold of 1500 microseconds (it was holding steady at the expected 1000 or 2000 microseconds), and yet the kill switch indicator pin would toggle for usually one pulse, and sometimes two pulses, then return to the expected value.

Glitch #1: PWM (yellow) width of 2ms should result in the kill switch indicator (blue) being low, but as shown here it occasionally glitches high.

Glitch #2: PWM (yellow) width of 1ms should result in the kill switch indicator (blue) being high, but as shown here it occasionally glitches low. 

I also logged the toggle events to the serial port, and included the pulse width that caused the toggle. It was always 1 millisecond over or under the expected range, and it would quickly toggle right back to a pulse width within the expected range. Aligning this with the oscilloscope readings, I could see where the kill switch indicator pin was toggled, but the pulse width had not changed.

What causes the glitch?

I still don't know what causes the glitch, but I have a theory: I suspect there is a bug in Arduino's micros() function, at least on ARM Cortex-M0.

The way I understand it, the microcontroller CPU tracks "time" (probably just the number of clock cycles since it was powered on) in a register that counts system clock ticks, but this counter is subject to frequent overflow. The Arduino framework installs an interrupt handler that fires once per millisecond, in which a software counter is incremented to track milliseconds. On the microcontrollers that I'm using, this millisecond counter is 32-bits wide, which only overflows every 49.7 days. I figure that the per-microsecond interrupt handler must also record the current microseconds register value. millis() would just return the value in the millisecond counter, but micros() probably returns something like:

millis() * 1000 + ([current microsecond counter] - [last recorded microsecond counter])

I am suspicious of this per-millisecond interrupt because that's the exact amount that it was off by in the PWM read. But correlation is not causation, so I'm not sure. I could go on a hours-long debugging session on the Arduino codebase (with which I have no familiarity, and I'm certainly no expert at microcontroller interfaces), but I'm already frustrated with the Arduino ecosystem for unrelated reasons, so I decided against that for now.

Now that I'm looking at my guess of what the code looks like, if that is how it's done then there is a race between millis() and [last recorded microsecond counter], which would result in exactly +/- 1 millisecond away from the expected value (+/- depends on which load/store wins the race). Hmmm...

Still, I had a problem without a solution.

FPGA to the rescue

A few months ago I had discussed my project with my neighbor who works extensively with FPGAs. When I discussed the PWM interrupt handlers he hinted that I could use a FPGA to deal with PWM signals. At the time - this was after I found the nRF52840 interrupt delay but before I found the glitch - I considered it for about 3 seconds but decided against it. After fighting with the glitch for a while, I decided to give it a try. If nothing else, maybe I'd learn something new. So I ordered a few boards from TinyFPGA.

Verilog (or any kind of HDL) was completely new to me, but you can learn most anything with some determination and the internet. Within a few hours of focus time, I had a working FPGA implementation that did PWM input and output, with one of the PWM inputs acting as a multiplexer switch. Except...there's that stupid glitch again! WTF???

A few more hours of research and digging around the internet later, I finally found these slides, specifically slides 8-10. I was driving multiple logic gates from one external signal (the PWM input), and I was occasionally seeing very strange results. I simply stored the external signal in a register, then one clock cycle later I read the register into the logic gates. Problem solved! I later went back and fed it through a few more stages of registers so that I avoid metastability problems (metastability is yet another totally new concept to me).

Communicating with the other microcontrollers

My original project intentions require that the navigation microcontroller is able to read throttle and steering input from the RC receiver (if nothing else, so that I can calibrate the PWM output signals). So I needed some way to get this information from the FPGA to the microcontroller - some way which doesn't involve PWM, because Arduino sucks at that. I2C should work nicely.

My FPGA board uses a Lattice MachXO2-1200. This board has some hardened logic which simplifies the development of I2C implementations. The cost of this is that you now have to understand how to interface with this onboard hardened logic (called the Embedded Function Block, or EFB) as well as understanding your I2C interface. Still, I think it is worth the effort. Lattice provides a reference implementation in Verilog, but I found it really difficult to follow. The combination of sequential and combinational logic just hurts my brain, and I find it much easier to understand sequential logic.

I discussed with my neighbor (FPGA guy), and thankfully he is of the opinion that combinational logic has many downsides, and encourages people to avoid it. I went back to my desk, threw away the garbage that I made based on the reference implementation, printed out the reference guide for using I2C on the EFB and just designed it how I thought it could be done in sequential logic only. Two hours later, I had an I2C slave on my FPGA!

My Verilog code can be found at github.com/ScottDuckworth/PulseController (direct link to the interesting part). I now only have maybe 10-15 hours of experience with Verilog that was spread over about a week, but I'm pretty happy with what I was able to accomplish mostly on my own (with some help from my neighbor). Effectively it's just another piece of hardware that I can put on my I2C bus that lets me read the pulse widths of 5 channels from my RC receiver, and write 2 channels (steering and throttle) which it will send to the steering servo and speed controller unless I've flipped the kill switch, in which case it just relays the physical PWM inputs (so that I have manual control of the car).

Integrating the FPGA into my project

This is a much more integrated solution than my previous design of using a microcontroller and a 4 channel 2x1 mux IC. The FPGA eliminates the pin change interrupt handlers for 5 channels of PWM inputs @ 100Hz and maybe (depending on if this is offloaded to hardware) the timer interrupt handlers for 2 channels of PWM outputs - so that's something like 1000 to 1400 fewer interrupts per second on my microcontrollers. The FPGA also consumes less power (about 11mA) than the Trinket M0 that it replaced (about 17mA), and I'm doing more with it. Win-win!

Along the way, I discovered with my new oscilloscope that the PWM signals sent by the RC receiver is actually just 3.3V, not 6V as I had originally thought (the power voltage is 6V, but the signal wire is just 3.3V). Apparently this is pretty standard for RC receivers, which means that I don't need the level shifters for PWM inputs or outputs. This is great news because I get to remove components and greatly reduce the amount of wiring. Less wiring hopefully means less signal interference from the radios, which I was seeing some evidence of with the oscilloscope.

This does mean that I need to build another board. To the soldering station!