There are many options for endoscopes:
The processor has special ASIC, such as Japanese, Taiwan and Korean camera ASIC; Useful dedicated CPU, such as the sea
Si, Ruixingwei and Junzheng, etc. Xiong Maojun's low-cost solution is based on Xilinx ZYNQ.
Looking at the image sensors used for endoscopes at the middle and low end, the OV is mainly used Currently, OV6946+OV426, OV6948+OV426, OV9734, OH02A1S (with near red External reinforcement), OH0TA10 (OCHTA10)+OV426 (this material is said to be the smallest in the industry, 400 × 400 phase.
Plain, package size is only 550 μ m*550 μ m) , DH01A10, etc. The interfaces include analog, MIPI and LVDS； In addition, NanEyeM of Ames, 320 × 320 phase element, embedded code interface LVDS requires high-speed clock oversampling to receive, which is troublesome. For the sensor of OV, simulation.
After ADC sampling, the output can generally be converted to LVDS or DVP interface.
And frame rate are low, and the rate is not too high (lane rate ＜ 800Mbps). You can use ZYNQ to power up Block the network to receive images.
1、 Hardware scheme
Let's take the OV's DH01A10 as an example. In fact, DH01A10 is the OV6946's Upgrade version, also 400 × 400 phase, the difference is that the process is more advanced:
(1)Image enhancement; (2) Sensitivity up to 3600mV/lux sec (OV6946 only has 1000mV/lux sec 3.6 times); (3) Package size is only 550 μ m × five hundred and fifty μ m. 1/36 "(the smallest in the industry, almost 1/4 of OV6946); (4) The drive capacity of output interface is 4m, which can be received by OV426 or other ADC schemes.
The quality is higher, the driving ability is stronger, and there is no vertical stripe like non-uniformity. So, for ordinary hands
The hardware block diagram of the desktop computer for surgery or examination is shown in Figure 1 below.
OV426 (or self grounding circuit)
OV6946
OV6948
DH01A10
model
Proposed
OV9734
OH02A1S
OH01A10
OH02A10
OV02C, etc
MIPI
SCCB
DVP
SCCB
XC7Z010-2CLG400I
QSPI
Flash （128Mb）
eMMC （ 8GB）
DDR-1066 （32bit/1GB）
GigE
PHY
USB1
USB2
USB3
USB
HUB
USB
PHY
RGMII RJ45
ULPI
HDMI
Driver
TMDS_ 33 Type A
LVDS or eDP
LVDS local screen
Note: If it is an eDP screen, LVDS needs to be converted to eDP bridge piece RTC, key input, etc
Figure 1 Hardware Block Diagram of Medical Endoscope Based on ZYNQ
As shown in Figure 1 above, the medical handheld or console has no high requirements for volume and power consumption, and the interface requirements
It is relatively rich. For example, recorded videos and photos can be exported through gigabit networks and USB interfaces,
The parameters of the machine can be configured through the network. There are keys and touch screens to control the machine, and HDMI can be connected
Enter the general display and so on. Generally, photos are stored in JPEG format and videos are stored in mp4 or avi
Format, stored in the internal high-capacity eMMC. Generally, the camera module will be modularized to
Dismantle the unit and appear as a consumable.
2、 Software solution
In this system, ZYNQ implements hardware and software co processing, and the software scheme is shown in Figure 2 below.
On PL, it mainly realizes Sensor driver (image acquisition and register configuration), ISP processing, JPEG
Video coding and eDP or LVDS screen drive. On the PS, the upper framework is based on QT and realizes the product
UI, touch screen, control interface, video encoding and decoding, video recording, storage, preview and playback, photos
And configure FPGA for initialization, auto exposure and auto leveling
And. In terms of device performance, the ARM of ZYNQ-7000 series is a dual core Cortex-A9,
-2 speed can only reach 766MHz at most, so optimize this system to make video recording and display reach
Real time requires a lot of work to be done, and requires a lot of cooperation between Linux and fpga
Processing can be said to maximize the performance of the chip. Even so, the arm side implements h.264 encoding
It can only guarantee the real-time performance of 400 * 400/30fps. Motion JPEG is performed by FPGA if the specification is higher than this specification
Encoding, and finally packaging for AVI video storage. The decoding is relatively simple during playback, and ARM operation can meet the requirements
Ask. The system supports up to 1080P/30fps stream processing. On the ISP, the system also needs to target medical and
The sensor does some special processing. For example, most of the OV6946 have vertical stripes, which should be uneven
Uniformity correction algorithm. The dynamic range of such sensor is generally low, and some contrast enhancement or sharpening is required
The chemical processing makes it look more sharp, and some will also be targeted for color mixing processing and so on.
Fig. 2 Endoscope software scheme based on ZYNQ
In general, the low-end endoscopic front-end equipment will not involve the automatic imaging of professional medical software
Analysis, it is mainly to manually view and analyze these images on site during the examination and operation. Even with automated software
The analysis requirements are generally handled by professional software after the image data is transmitted to the PC through the interface.
For more high-end laparoscopes and the like, the solution is available from HiSilicon and ZYNQ UltraScale+MPSoC
The Codec model and the overseas head also have high-end FPGA+high-end own ASIC chips
The whole technical difficulty will go up a big step.
In addition, the capsule endoscope, which is very popular now, has taken another low power consumption way: sensor
The performance is not necessarily high. It requires wireless image transmission. The frame rate may be 1~2Hz, or it may be triggered, but
There is a requirement of extremely low power consumption and a requirement of very small size. It is generally required to be able to work (sleep)
Hours. At present, the existing products on the market also use low-power FPGA (actel) plus MSP430 single chip
The scheme of the machine (Xiong Maojun has taken apart and studied one or two existing products, and also had the idea of tossing about one).
3、 Other
In the current situation, Xiong Maojun's low-end endoscope solutions are mainly based on
There are also HiSilicon and ZU4EV on ZYNQ-7010 and Rising Micro RV1126 platforms
One is that it is not easy to get the chip, the other is that it is too expensive, the cost is too high, and the risk is too high.
It's almost something. Either the performance here is not enough, or there are fewer interfaces. In a word, we hope that other countries can rise up to achieve a small goal: to reach the Hi3559 level of HiSilicon.
Xiong Maojun has a mature complete scheme and machine based on ZYNQ-7000 and HiSilicon, which is also welcome Interested friends came to the scene for guidance. If you need it and I have it, we may be happy.
Our cooperation:
(1) Based on ZYNQ-7000, Lattice Crosslink NX and Elix Ti60 solutions:
Sensors for OV6946, OV6948 and OH0TA10 soft mirrors;
(2) Based on HiSilicon (Hi3516, Hi3519 and Hi3559): mainly for OS08A10 and OH08A40
Such 4K hard mirror or binocular surgical robot applications;
(3) Based on Xilinx ZYNQ MPSoC (ZU4EV/ZU5EV and ZU7EV): mainly for
Low delay application of 4K hard mirror or binocular surgical robot such as OS08A10 and OH08A40.