In many research areas, such as automated driving, more and more learned methods based on neural networks are used for visual environment perception. In our group, we investigate semantic and instance segmentation, as well as methods that allow adaptation of the segmentation to unseen data or new classes.
Semantic segmentation is defined as the pixel-wise classification of a camera image. The goal is to obtain an understanding of the objects present in the image. Using neural networks, each pixel in the image is assigned to one of for example 19 predefined classes. Sometimes, however, this information is not sufficient because overlapping objects of the same class are not distinguished. The solution to this problem is instance segmentation, in which each individual instance of a certain class in the image also receives its own identifier. This makes it possible, for example, to count or track these objects.
Training these aforementioned segmentation networks requires a large amount of labeled data. However, if the data during operation deviates from the training data, for example because they are recorded with a different camera, then the pixel-wise classification may no longer function properly. This is referred to as a so-called domain gap between the data during operation (target data) and the training data. To overcome this domain gap, we develop methods for unsupervised domain adaptation [1] that do not require labels from the intended target domain.
Many domain adaptation methods, while not requiring labels of the target domain, depend on training data and (unlabeled) images of the target domain, being available simultaneously during the training and adaptation process. However, this is not always possible, as privacy regulations or storage limitations may prevent the original training data from being made available to the adaptation process. Therefore, our group performs research on methods that allow continual adaptation without training domain data. To achieve this, we are exploring methods such as adapting the network weights [2] or applying a style transfer to the images [3]. Another use case that we are working on is class-incremental learning. This research aims at teaching a network new classes without forgetting the previously learned information [4].
[1] J.-A. Bolte, M. Kamp, A. Breuer, S. Homoceanu, P. Schlicht, F. Hüger, D. Lipinski, and T. Fingscheidt, “Unsupervised Domain Adaptation to Improve Image Segmentation Quality Both in the Source and Target Domain,” in Proc. of CVPR - Workshops, Long Beach, CA, USA, Jun. 2019, pp. 1404-1413
[2] M. Klingner, J.-A. Termöhlen, J. Ritterbach, and T. Fingscheidt, “Unsupervised BatchNorm Adaptation (UBNA): A Domain Adaptation Method for Semantic Segmentation Without Using Source Domain Representations,” arXiv 2011.08502, Nov. 2020, pp. 1-11
[3] J.-A. Termöhlen, M. Klingner, L. J. Brettin, N. M. Schmidt, and T. Fingscheidt, „Continual Unsupervised Domain Adaptation for Semantic Segmentation by Online Frequency Domain Style Transfer,” in Proc. of ITSC, Indianapolis, IN, USA, Sep. 2021, pp. 1-8
[4] M. Klingner, A. Bär, P. Donn, and T. Fingscheidt, “Class-Incremental Learning for Semantic Segmentation Re-Using Neither Old Data Nor Old Labels,” in Proc. of ITSC, Rhodes, Greece, Sep. 2020, pp. 1-8.