Test the device: the value of EOL and self-tests

It’s essential to test the manufactured devices. It’s usually forgotten if you make a one-of-a-kind gadget at home, because each and every functionality is thoroughly checked during the process. But everything changes after you try to scale production – even a small batch of 5-10 devices are made in a different way!

Prototype and mass production

You can make a single device with much love and care, but if you have to deliver a few dozen of them next day, time can be critical. Manufacturing processes are constantly optimized and an assembly line is designed to do things as fast as possible. For example, a lot of time can be spared if similar steps are done at the same time – if one 10k resistor must be soldered, solder all of them! If you still do parts one by one, try batch soldering. But can you guess the problem with this method?

If you said “a lot of half-finished workpieces”, you guessed right! If your PCB is populated with only 10k resistors, it’s probably not functional yet, and can’t even test! Yes, you can do visual inspection and if you have enough test points, electrical tests can be done too, but is it really necessary?

End of Line test (EOL)

Inspections and tests can be conducted to find defects and manufacturing errors early to improve yield. However, an End of Line test’s goal is to prove the correctness of the whole device. If we talk about electric devices (especially the ones with a microcontroller), complex functional tests can be done. Making a testing strategy (and firmware) may take some time, but it pays off on the long run. Manual testing is time-consuming, error-prone, requires concentration, try to avoid it!

There is one more thing to consider: calibration. Manufacturing tolerances may (and will) lead to differences, and depending on your product, calibration can be one of the last manufacturing steps. Precalibrated digital sensors are cool, but expensive: a famous digital sensor, DS18B20 costs $4.10-$2.30, while an NTC plus one resistor costs ¢5.8 – ¢3.9. Now make just 100 of these devices! ~$200 difference in price. Does it worth?

Self-test

Long story short, there are devices which must run reliably. We don’t talk about your phone or computer: if those crash, maybe your work is lost. I had power outage back when automatic backups weren’t a thing in Office and forgot to save my essay. I was furious because of hours of lost work, but that’s it, the only thing died that day was my motivation to write the perfect document.

Not suprisingly, it’s a common experience. Save often!

However if a pacemaker, insulin pump, ABS, airbag or a railway signal fails, that can easily lead to something catastrophic and/or fatal. The shinkansen in Japan is frequently checked. It’s probably an expensive process but a crash over 300 km/h – well, I’m not gonna show disturbing scenes, but Googling “Santiago de Compostela derailment” can give you a hint.

But what about the mentioned insulin pump? Will any user go to a hospital every week to check if that device is still OK? Probably some would. What about doing this daily? Maybe if you live in a hospital. But doing it hourly? It’s easier to manually measure your blood sugar.

Just to put it in perspective: one of the devices I worked with does a full self-test every 10 seconds. It’s just impossible to do it manually.

The goal of self-tests

While tests interfere with normal operation, they are necessary interruptions. The level, length and frequency vary. A Power On Self-Test (or POST) is done every time a PC starts, can take seconds, and doesn’t run again, the goal is to check if the computer can start. An ECU in a car may run a full self-test every minute to ensure reliable operation, while the process takes only a few milliseconds. Self-tests come in all shapes and sizes, but the common thing is to find defects/errors after the product is sold. Your computer just refuses to boot if an error is found, and a Mars rover must correct if a fault is found. Really diverse!

A practical example: rental Arduino shields

Buying secondhand or renting things is always risky, for both parties. If there was a way to ensure everything is fine! Oh wait, there can be.

When the first Covid lockdowns happened, I was forced to send my students home. We still had several classes left, we had to finish the course, so they got a package with everything needed. The result was disappointing, not only because everyone was just learning this “remote lifestyle”, but because it took an hour to assemble the devices – every time! (what a waste) It’s not a big deal in person, you can inspect, help and correct wiring mistakes in seconds. Now try this while many of the students had 640*480 cameras or no webcam at all!

arduino education kit shield red blue
My minions! Help the poor students!

So, the idea of having an easy to use kit came up. First I tried to put everything on a single shield, but ended up making 2 separate ones. Most of the parts are soldered, so no jumper wires, and all the features are queezed in this pack: basic GPIO, interrupts, PWM, motor control, sound generation, analog sensors, I²C, SPI – a one-stop embedded course! The students even learn how to write their own driver for the integrated I²C EEPROM or the LED matrix!

It’s great and all, but everything can break, right? Despite the ESD protected shipping, the end user can fry the parts at home, solder joints can break, and the distributor can get counterfeit parts too! (it’s more than embarrassing when the motors stop almost immediately)

So I made a test suit. Not two, just one! In this case, the EOL and self-test can be the same, 100% code reuse! (every developer’s dream) It literally takes only a few minutes to test practically every part of the kit. I can test the device after manufacturing, before shipping, and after receiving. The students can test before and after the course.

So consider writing tests, even for your embedded devices – it may save someone’s life or precious time some day!