Employees of ccc are currently developing methods and algorithms for automatic object recognition and 3D pose recognition as part of a funded project. Currently, the technology is used in particular for the recognition of ski jumpers in order to combine several camera shots into one video.
In the future, the developed algorithms will serve as a basis for the new and further development of ccc's video analysis solutions. Thus, modules are to be developed that can realize different tasks (AI-based object recognition and tracking as well as 3D pose recognition).
For the development of the AI, training data sets for ski jumper recognition were created. This was used to train neural networks that will eventually be connected to the video analysis system utilius kiwano.
So far, image processing algorithms have been used to try to detect fast movements in the image and thus ski jumpers. However, these algorithms can only be used under certain conditions and are susceptible to not recognizing the ski jumper if, for example, there is a lot of snow or other people are visible on the ski jumps.
One approach to develop more robust algorithms is to train neural networks. In the field of object recognition, classification and object tracking, many groundbreaking successes have been achieved in recent years. It is now possible to analyze images very quickly and with a high degree of quality.
Methods are used in which an image is divided into different areas. For each area the probability is calculated that an object of the class is located in an area. Areas with the same class and high probability are grouped together.
The trained neural networks were to be used to improve the editing of the videos at the Oberstdorf and Oberhof sites. In these systems, the approach, jump and flight phase of the ski jumpers are recorded laterally. There is also a PTZ camera that tracks the ski jumper from behind. After the ski jumper completes his jump, the sequences in which the ski jumper is seen in a video are cut together and synchronized with the video from the PTZ camera. The entry and exit of a ski jumper into the image section must therefore be detected.
In order to recognize the ski jumper, a neural network had to be trained and for this it was necessary to create a training data set. This consists of 300 images of several ski jumps, each with different ski jumpers, seasons and weather conditions.
In all images, the area where the ski jumper is located was marked and thus the image coordinates were captured. The network then learns to recognize these areas and assigns them the label ski jumper.
Subsequently, a new post-processing process was implemented in utilius kiwano. This uses the trained model to determine times in the video where the ski jumper is seen. A post-processing process is executed as soon as a recording is finished. After that, the videos are cut together by another post-processing process.
For the user, this creates a single video in which the jumper is constantly visible. And this despite the fact that the jump was recorded by about 10 cameras, in which the jumper can only be seen very briefly in the picture.
This project is funded by the European Regional Development Fund (ERDF).