Guideline advice. Adjust your algorithm to the underlining architecture. Not all algorithms are out of the box suitable for all kinds of accelerators (e.g. FPGA or GPU). Algorithms that allow stream pro- cessing are more appropriate for FPGAs, while algorithms that require random access to memory should preferably be ported onto a GPU. Optimize and tailor your algorithm to meet the strengths of the chosen architecture.
Guideline advice. The RTOS should be chosen based on the application requirement. You need to choose the RTOS to be used based on the requirements regarding performance, safety/reliability, timing, resources utilization and required devices and services. If your design requires high performance, meaning that you need to use a multicore CPU and/or hardware accelerators (e.g. FPGAs), it is recommended to use an RTOS that supports those devices. If your design needs a high degree of reliability, you have to choose an RTOS that can provide that level of reliability with features such as time & space isolation, watchdogs or self-repair. If you need to satisfy some industrial safety norm, you need to make sure that the RTOS is certifiable at the required level. Otherwise, a more common, non-safety certifiable RTOS will be enough. If your design requires hard real-time behaviour, then you need to choose an RTOS that can warranty on time execution by design and never rely on having a performance that is “good enough on average”. Average timings are anecdotic for a real-time system: it is the worst case execution time that must be met. It is also recommended to estimate the effect of latencies and jitter on system performance. The choice of RTOS must be made according to the hardware constraints of the target plat- form regarding memory, power and other limited resources. In particular, the footprint of the RTOS and middleware can have a significant impact if memory is limited, or even when not limited the fact that the RTOS kernel is small means it could be cache resident so that OS overheads is minimal. This can be a major different when compared to large general pur- pose Oss. Regarding power, support for low power features (e.g. DVFS) at the RTOS level is recommended for low power designs. The chosen RTOS must be able to run on the target platform and drivers for the required devices and services should be either available or foreseen as part of the development. It is recommended to limit device and service support to the strict necessary and not to clutter the system with support for unnecessary devices. This is the reason to recommend an RTOS based on modular micro kernel architecture.
Guideline advice. Move data processing as close to the sensor as possible. (Pre-) Processing a video stream with an embedded CPU consumes too much processing power, time and energy. KPIs in your real-time enabled low-power image processing platform will not be met.
Guideline advice. The choice of an FPGA for an Image Processing system has benefits in terms of flexible Input/Output, as can implement the any camera standard found in Table 1 of the TULIPP D1.1 Reference Platform3. An FPGA can also provide pre/-post processing on images. It’s actually possible to add 32- bit and/or 64-bit CPUs inside a bare-bone FPGA and that typically comes as an IP-Core, but it uses a lot of the FPGA resources. This is a route to take if your aim to develop a system for eventual volume production, as then possible to create an “Application-Specific-Integrated-Circuit” [ASIC]
