If things were systematic and clean, the process of going from an idea to a working prototype may be:
- Based on idea, decide on candidate hardware and algorithm
- Log data with the candidate hardware
- Process and work out algorithm of interest
- Iterate between steps 2 and 3.
- Develop prototype using choice of hardware and algorithm
Offline Analysis
Steps 2 and 3 can be described as "offline analysis". I used to think this step was universal: whether developing a new feature on existing machinery, a drone or robotics application, or a new phone app.
If it doesn't work from the leisure of a personal computer, why attempt it with all the complexities of the real system?
Turns out this step is not universal. Many a sensor-based application gets developed without this step of offline analysis. Does skipping this step save time? Or does it create a space for illusive bugs to hide?
Here are four reasons why one may skip this step and thoughts on the implications.
1. It is simple enough
Some applications or features are simple enough that one should go straight for the real system. One thing to consider, however, is the broader project and the rest of the system. Will the next feature be more complex? If so, it may make sense to set up the tools to enable offline analysis. Or, sometimes the interconnection of several simple systems can add complexity worth capturing with offline analysis.
Otherwise, if it is simple enough, let's keep it that way!
2. No time
There is only time to develop this once, and so I'll do it on the real system.
Having the time and team support to do offline analysis is a benefit not all people have. This may be one of many features and systems that need to be designed, implemented, and tested.
However, in the case that development hits unexpected complexities, then this question arises: "If you don't have time to do it right, when will you have time to do it over?"
3. Too difficult to simulate system
This can be a real challenge. Simulating the sub-systems of interest may not be trivial and actually can turn out to be a project in and of itself. Two thoughts on this:
- Develop the appropriate simulation and offline analysis tools in parallel to the system.
- Find commercial or open source software that may serve to simulate the system for you.
The first thought is little relief for a person trying to add a complex feature to an existing or mature project where this idea was already ignored. If anything, it is a lesson for future projects. The second thought is to leverage other simulation tools so that you can focus on the core design problem.
4. More comfortable with real system
The actual system may be a more comfortable environment to work and develop than the offline environment. For example, developing algorithms for an embedded system using the physical sensor, microprocessor, and C/C++ code may be more comfortable than working with analysis tools like MATLAB, Python, Excel, or Calc. Likewise, some physical hardware and controllers (e.g. from National Instruments, Danfoss, or Programmable Logic Controllers (PLC)) are designed for ease of design iteration with the real system.
One should work with the tools with which they are comfortable. However, do consider that this project may be handed off. Supporting scaffolding like test cases or simulation tools which can be used to replicate bugs and resolve issues will be valuable. Otherwise, you may become the scaffolding and have to personally investigate and resolve all future issues!
Conclusion
The tools to enable offline analysis can be valuable both in the initial system development as well as replicating and resolving issues observed in the field. There are valid reasons to forgo this step, but this should be a decision rather than a matter of neglect.