Publications by Michael Felsberg

Channel-coded feature maps (CCFMs) represent arbitrary image features using multi-dimensional histograms with soft and overlapping bins. This representation can be seen as a generalization of the SIFT descriptor, where one advantage is that it is better suited for computing derivatives with respect to image transformations. Using these derivatives, a local optimization of image scale, rotation and position relative to a reference view can be computed. If piecewise polynomial bin functions are used, e.g. B-splines, these histograms can be computed by first encoding the data set into a histogram-like representation with non-overlapping multi-dimensional monomials as bin functions. This representation can then be processed using multi-dimensional convolutions to obtain the desired representation. This allows to reuse much of the computations for the derivatives. By comparing the complexity of this method to direct encoding, it is found that the piecewise method is preferable for large images and smaller patches with few channels, which makes it useful, e.g. in early steps of coarse-to-fine approaches.

Intrinsic dimensionality is a concept introduced by statistics and later used in image processing to measure the dimensionality of a data set. In this paper, we introduce a continuous representation of the intrinsic dimension of an image patch in terms of its local spectrum or, equivalently, its gradient field. By making use of a cone structure and barycentric co-ordinates, we can associate three confidences to the three different ideal cases of intrinsic dimensions corresponding to homogeneous image patches, edge-like structures and junctions. The main novelty of our approach is the representation of confidences as prior probabilities which can be used within a probabilistic framework. To show the potential of our continuous representation, we highlight applications in various contexts such as image structure classification, feature detection and localisation, visual scene statistics and optic flow evaluation.

In this paper we propose a new approach to real-time view-based pose recognition and interpolation. Pose recognition is particularly useful for identifying camera views in databases, video sequences, video streams, and live recordings. All of these applications require a fast pose recognition process, in many cases video real-time. It should further be possible to extend the database with new material, i.e., to update the recognition system online. The method that we propose is based on P-channels, a special kind of information representation which combines advantages of histograms and local linear models. Our approach is motivated by its similarity to information representation in biological systems but its main advantage is its robustness against common distortions such as clutter and occlusion. The recognition algorithm consists of three steps: (1) low-level image features for color and local orientation are extracted in each point of the image; (2) these features are encoded into P-channels by combining similar features within local image regions; (3) the query P-channels are compared to a set of prototype P-channels in a database using a least-squares approach. The algorithm is applied in two scene registration experiments with fisheye camera data, one for pose interpolation from synthetic images and one for finding the nearest view in a set of real images. The method compares favorable to SIFT-based methods, in particular concerning interpolation. The method can be used for initializing pose-tracking systems, either when starting the tracking or when the tracking has failed and the system needs to re-initialize. Due to its real-time performance, the method can also be embedded directly into the tracking system, allowing a sensor fusion unit choosing dynamically between the frame-by-frame tracking and the pose recognition.

In this paper, we present a new and efficient method to implement robust smoothing of low-level signal features: B-spline channel smoothing. This method consists of three steps: encoding of the signal features into channels, averaging of the channels, and decoding of the channels. We show that linear smoothing of channels is equivalent to robust smoothing of the signal features if we make use of quadratic B-splines to generate the channels. The linear decoding from B-spline channels allows the derivation of a robust error norm, which is very similar to Tukey's biweight error norm. We compare channel smoothing with three other robust smoothing techniques: nonlinear diffusion, bilateral filtering, and mean-shift filtering, both theoretically and on a 2D orientation-data smoothing task. Channel smoothing is found to be superior in four respects: It has a lower computational complexity, it is easy to implement, it chooses the global minimum error instead of the nearest local minimum, and it can also be used on nonlinear spaces, such as orientation space. © 2006 IEEE.

In this paper we address the topics of scale-space and phase-based image processing in a unifying framework. In contrast to the common opinion, the Gaussian kernel is not the unique choice for a linear scale-space. Instead, we chose the Poisson kernel since it is closely related to the monogenic signal, a 2D generalization of the analytic signal, where the Riesz transform replaces the Hilbert transform. The Riesz transform itself yields the flux of the Poisson scale-space and the combination of flux and scale-space, the monogenic scale-space, provides the local features phase-vector and attenuation in scale-space. Under certain assumptions, the latter two again form a monogenic scale-space which gives deeper insight to low-level image processing. In particular, we discuss edge detection by a new approach to phase congruency and its relation to amplitude based methods, reconstruction from local amplitude and local phase, and the evaluation of the local frequency.

We introduce a compact coding of image information in terms of local multi-modal image descriptors. This coding allows for an explicit separation of the local image information into different visual sub-modalities: geometric information (orientation) and structural image information (contrast transition and colour). Based on this image representation, we derive a similarity function that compares visual information in each of these sub-modalities. This allows for an investigation of the importance of the different factors for stereo matching on a large data set. From this investigation we conclude that it is the combination of visual modalities that gives the best results. Concrete weights for their relative importance are measured. In addition to these quantitative results, we can demonstrate by our simulations that although our image representation reduces image information by 97% we achieve a matching performance which is comparable to block matching techniques. This shows that our very condensed representation preserves the relevant visual information. © 2004 Elsevier B.V. All rights reserved.

In this paper we address the problem of appropriately representing the intrinsic dimensionality of image neighborhoods. This dimensionality describes the degrees of freedom of a local image patch and it gives rise to some of the most often applied corner and edge detectors. It is common to categorize the intrinsic dimensionality (iD) to three distinct cases: i0D, i1D, and i2D. Real images however contain combinations of all three dimensionalities which has to be taken into account by a continuous representation. Based on considerations of the structure tensor, we derive a cone-shaped iD-space which leads to a probabilistic point of view to the estimation of intrinsic dimensionality.

We consider alpha scale spaces, a parameterized class (alpha is an element of (0, 1]) of scale space representations beyond the well-established Gaussian scale space, which are generated by the alpha-th power of the minus Laplace operator on a bounded domain using the Neumann boundary condition. The Neumann boundary condition ensures that there is no grey-value flux through the boundary. Thereby no artificial grey-values from outside the image affect the evolution proces, which is the case for the alpha scale spaces on an unbounded domain. Moreover, the connection between the a scale spaces which is not trivial in the unbounded domain case, becomes straightforward: The generator of the Gaussian semigroup extends to a compact, self-adjoint operator on the Hilbert space L-2(Omega) and therefore it has a complete countable set of eigen functions. Taking the alpha-th power of the Gaussian generator simply boils down to taking the alpha-th power of the corresponding eigenvalues. Consequently, all alpha scale spaces have exactly the same eigen-modes and can be implemented simultaneously as scale dependent Fourier series. The only difference between them is the (relative) contribution of each eigen-mode to the evolution proces. By introducing the notion of (non-dimensional) relative scale in each a scale space, we are able to compare the various alpha scale spaces. The case alpha = 0.5, where the generator equals the square root of the minus Laplace operator leads to Poisson scale space, which is at least as interesting as Gaussian scale space and can be extended to a (Clifford) analytic scale space.

This paper introduces a two-dimensional generalization of the analytic signal. This novel approach is based on the Riesz transform, which is used instead of the Hilbert transform. The combination of a 2D signal with the Riesz transformed one yields a sophisticated 2D analytic signal, the monogenic signal. The approach is derived analytically from irrotational and solenoidal vector fields. Based on local amplitude and local phase, an appropriate local signal representation is presented which preserves the split of identity, i.e., the invariance – equivariance property of signal decomposition. This is one of the central properties of the 1D analytic signal that decomposes a signal into structural and energetic information. We show that further properties of the analytic signal concerning symmetry, energy, allpass transfer function, and orthogonality are also preserved, and we compare this to the behavior of other approaches for a 2D analytic signal. As a central topic of this paper, a geometric phase interpretation is introduced which is based on the relation between the 1D analytic signal and the 2D monogenic signal established by the Radon transform. Possible applications of this relationship are sketched and references to other applications of the monogenic signal are given. This report is a revised version of the technical report 2009 [7], and therefore supercedes it.

Linear scale-space theory is the fundamental building block for many approaches to image processing like pyramids or scale-selection. However, linear smoothing does not preserve image structures very well and thus non-linear techniques are mostly applied for image enhancement. A different perspective is given in the framework of channel-smoothing, where the feature domain is not considered as a linear space, but it is decomposed into local basis functions. One major drawback is the larger memory requirement for this type of representation, which is avoided if the channel representation is subsampled in the spatial domain. This general type of feature representation is called channel-coded feature map (CCFM) in the literature and a special case using linear channels is the SIFT descriptor. For computing CCFMs the spatial resolution and the feature resolution need to be selected. In this paper, we focus on the spatio-featural scale-space from a scale-selection perspective. We propose a coupled scheme for selecting the spatial and the featural scales. The scheme is based on an analysis of lower bounds for the product of uncertainties, which is summarized in a theorem about a spatio-featural uncertainty relation. As a practical application of the derived theory, we reconstruct images from CCFMs with resolutions according to our theory. The results are very similar to the results of non-linear evolution schemes, but our algorithm has the fundamental advantage of being non-iterative. Any level of smoothing can be achieved with about the same computational effort.

This work presents a novel object tracking approach, where the motion model is learned from sets of frame-wise detections with unknown associations. We employ a higher-order Markov model on position space instead of a first-order Markov model on a high-dimensional state-space of object dynamics. Compared to the latter, our approach allows the use of marginal rather than joint distributions, which results in a significant reduction of computation complexity. Densities are represented using a grid-based approach, where the rectangular windows are replaced with estimated smooth Parzen windows sampled at the grid points. This method performs as accurately as particle filter methods with the additional advantage that the prediction and update steps can be learned from empirical data. Our method is compared against standard techniques on image sequences obtained from an RC car following scenario. We show that our approach performs best in most of the sequences. Other potential applications are surveillance from cheap or uncalibrated cameras and image sequence analysis.

In this paper we propose a new approach to real-time view-based object recognition and scene registration. Object recognition is an important sub-task in many applications, as e.g., robotics, retrieval, and surveillance. Scene registration is particularly useful for identifying camera views in databases or video sequences. All of these applications require a fast recognition process and the possibility to extend the database with new material, i.e., to update the recognition system online. The method that we propose is based on P-channels, a special kind of information representation which combines advantages of histograms and local linear models. Our approach is motivated by its similarity to information representation in biological systems but its main advantage is its robustness against common distortions as clutter and occlusion. The recognition algorithm extracts a number of basic, intensity invariant image features, encodes them into P-channels, and compares the query P-channels to a set of prototype P-channels in a database. The algorithm is applied in a cross-validation experiment on the COIL database, resulting in nearly ideal ROC curves. Furthermore, results from scene registration with a fish-eye camera are presented.

This paper explores the possibility to use a single low-resolution FIR camera for detection of pedestrians in the near zone in front of a vehicle. A low resolution sensor reduces the cost of the system, as well as the amount of data that needs to be processed in each frame.

We present a system that makes use of hot-spots and image positions of a near constant bearing to detect potential pedestrians. These detections provide seeds for an energy minimization algorithm that fits a pedestrian model to the detection. Since false alarms are hard to tolerate, the pedestrian model is then tracked, and the distance-to-collision (DTC) is measured by integrating size change measurements at sub-pixel accuracy, and the car velocity. The system should only engage braking for detections on a collision course, with a reliably measured DTC.

Preliminary experiments on a number of recorded near collision sequences indicate that our method may be useful for ranges up to about 10m using an 80x60 sensor, and somewhat more using a 160x120 sensor. We also analyze the robustness of the evaluated algorithm with respect to dead pixels, a potential problem for low-resolution sensors.

In this paper, we combine the well-established technique of Wiener filtering with an efficient method for robust smoothing: channel smoothing. The main parameters to choose in channel smoothing are the number of channels and the averaging filter. Whereas the number of channels has a natural lower bound given by the noise level and should for the sake of speed be as small as possible, the averaging filter is a less obvious choice. Based on the linear behavior of channel smoothing for inlier noise, we derive a Wiener filter applicable for averaging the channels of an image. We show in some experiments that our method compares favorable with established methods.

In this paper we propose a new operator which combines advantages of monogenic scale-space and Gaussian scale-space, of the monogenic signal and the structure tensor. The gradient energy tensor (GET) defined in this paper is based on Gaussian derivatives up to third order using different scales. These filters are commonly available, separable, and have an optimal uncertainty. The response of this new operator can be used like the monogenic signal to estimate the local amplitude, the local phase, and the local orientation of an image, but it also allows to measure the coherence of image regions as in the case of the structure tensor. Both theoretically and in experiments the new approach compares favourably with existing methods.

 To program a robot to solve a simple shape-sorter puzzle is trivial. To devise a Cognitive System Architecture, which allows the system to find out by itself how to go about a solution, is less than trivial.The development of such an architecture is one of the aims of the COSPAL project, leading to new techniques in vision based Artificial Cognitive Systems, which allow the development of robust systems for real dynamic environments. The systems developed under the project itself remain however in simplified scenarios, likewise the shape-sorter problem described in the present paper.The key property of the described system is its robustness. Since we apply association strategies of local features, the system behaves robustly under a wide range of distortions, as occlusion, colour and intensity changes. The segmentation step which is applied in many systems known from literature is replaced with local associations and view-based hypothesis validation. The hypotheses used in our system are based on the anticipated state of the visual percepts. This state replaces explicit modeling of shapes. The current state is chosen by a voting system and verified against the true visual percepts. The anticipated state is obtained from the association to the manipulator actions, where reinforcement learning replaces the explicit calculation of actions. These three differences to classical schemes allow the design of a much more generic and flexible system with a high level of robustness.On the technical side, the channel representation of information and associative learning in terms of the channel learning architecture are essential ingredients for the system. It is the properties of locality, smoothness, and non-negativity which make these techniques suitable for this kind of application. The paper gives brief descriptions of how different system parts have been implemented and show some examples from our tests.

Traditionally,quadrature filters and derivatives have been considered asalternative approaches to low-level image analysis. In this paperwe show that there actually exist close connections: We define thequadrature-based boundary tensor and the derivative-based gradientenergy tensor which exhibit very similar behavior. We analyse thereason for this and determine how to minimize the difference. Theseinsights lead to a simple and very efficient integrated featuredetection algorithm. 

 A robust mean value is often a good alternative to the standard mean value when dealing with data containing many outliers. An efficient method for samples of one-dimensional features and the truncated quadratic error norm is presented and compared to the method of channel averaging (soft histograms).

 Different kinds of visual substructures (such as homogeneous, edge-like and junction-like patches) can be distinguished by the intrinsic dimensionality of the local signals. The concept of intrinsic dimensionality has been mostly exercised using discrete formulations. A recent work [FK03,KF03] introduced a continuous definition and showed that the inherent structure of the intrinsic dimensionality has essentially the form of a triangle. The current study work analyzes the distribution of signals according to the continuous interpretation of intrinsic dimensionality and the relation to orientation and optic flow features of image patches. Among other things, we give a quantitative interpretation of the distribution of signals according to their intrinsic dimensionality that reveals specific patterns associated to established sub-structures in computer vision. Furthermore, we link quantitative and qualitative properties of the distribution of optic-flow error estimates to these patterns.

 In this paper we address one of the standard problems of image processing and computer vision: The detection of points of interest (POI). We propose two new approaches for improving the detection results. First, we define an energy tensor which can be considered as a phase invariant extension of the structure tensor. Second, we use the channel representation for robustly clustering the POIinformation from the first step resulting in sub-pixel accuracy for the localisation of POI. We compare our method to several related approaches on a theoretical level and show a brief experimental comparison to theHarrisdetector.

We consider ? scale spaces, a parameterized class (? ? (0,1]) of scale space representations beyond the well-established Gaussian scale space, which are generated by the ?-th power of the minus Laplace operator on a bounded domain using the Neumann boundary condition. The Neumann boundary condition ensures that there is no grey-value flux through the boundary. Thereby no artificial grey-values from outside the image affect the evolution proces, which is the case for the ? scale spaces on an unbounded domain. Moreover, the connection between the ? scale spaces which is not trivial in the unbounded domain case, becomes straightforward: The generator of the Gaussian semigroup extends to a compact, self-adjoint operator on the Hilbert space L2(?) and therefore it has a complete countable set of eigen functions. Taking the ?-th power of the Gaussian generator simply boils down to taking the ?-th power of the corresponding eigenvalues. Consequently, all ? scale spaces have exactly the same eigen-modes and can be implemented simultaneously as scale dependent Fourier series. The only difference between them is the (relative) contribution of each eigen-mode to the evolution proces. By introducing the notion of (non-dimensional) relative scale in each ? scale space, we are able to compare the various ? scale spaces. The case ?=0.5, where the generator equals the square root of the minus Laplace operator leads to Poisson scale space, which is at least as interesting as Gaussian scale space and can be extended to a (Clifford) analytic scale space.

In this paper we present a method to implement the monogenic scale space on a bounded domain and show some applications. The monogenic scale space is a vector valued scale space based on the Poisson scale space, which establishes a sophisticated alternative to the Gaussian scale space. The features of the monogenic scale space, including local amplitude, local phase, local orientation, local frequency, and phase congruency, are much easier to interpret in terms of image features evolving through scale than in the Gaussian case. Furthermore, applying results from harmonic analysis, relations between the features are obtained which improve the understanding of image analysis. As applications, we present a very simple but still accurate approach to image reconstruction from local amplitude and local phase and a method for extracting the evolution of lines and edges through scale.

Channel smoothing is an alternative to diffusion filtering for robust estimation of image features. Its main advantages are speed, stability with respect to parameter changes, and a simple implementation. However, channel smoothing becomes instable in certain situations, typically for elongated, periodic patterns like for instance fingerprints. As for the diffusion filtering an anisotropic extension is required in these cases. In this paper we introduce a new method for anisotropic channel smoothing which is comparable to coherence enhancing diffusion, but faster and easier to implement. Anisotropic channel smoothing implements an orientation adaptive non-linear filtering scheme as a special case of adaptive channel filtering. The smoothing algorithm is applied to several fingerprint images and the results are compared to those of coherence enhancing diffusion.

We introduce a compact coding of image information which explicitely separates geometric information (orientation) and structural information (phase and color). We investigate the importance of these factors for stereo matching on a large data set. From these investigation we can conclude that it is their combination that gives the best results. Concrete weights for their relative importance are measured.

Quadrature filters are a well known method of low-level computer vision for estimating certain properties of the signal, as there are local amplitude and local phase. However, 2D quadrature filters suffer from being not rotation invariant. Furthermore, they do not allow to detect truly 2D features as corners and junctions unless they are combined to form the structure tensor. The present paper deals with a new 2D generalization of quadrature filters which is rotation invariant and allows to analyze intrinsically 2D signals. Hence, the new approach can be considered as the union of properties of quadrature filters and of the structure tensor. The proposed method first estimates the local orientation of the signal which is then used for steering some basis filter responses. Certain linear combination of these filter responses are derived which allow to estimate the local isotropy and two perpendicular phases of the signal. The phase model is based on the assumption of an angular band-limitation in the signal. As an application, a simple and efficient point-of-interest operator is presented and it is compared to the Plessey detector.

Augmented reality is a growing field, with many diverse applications ranging from TV and film production, to industrial maintenance, medicine, education, entertainment and games. The central idea is to add virtual objects into a real scene, either by displaying them in a see-through head-mounted display, or by superimposing them on an image of the scene captured by a camera. Depending on the application, the added objects might be virtual characters in a TV or film production, instructions for repairing a car engine, or a reconstruction of an archaeological site. For the effect to be believable, the virtual objects must appear rigidly fixed to the real world, which requires the accurate measurement in real-time of the position of the camera or the user-s head. Present technology cannot achieve this without resorting to systems that require a significant infrastructure in the operating environment, severely restricting the range of possible applications.

In this paper we present a new method to implement a robust estimator: B-spline channel smoothing. We show that linear smoothing of channels is equivalent to a robust estimator, where we make use of the channel representation based upon quadratic B-splines. The linear decoding from B-spline channels allows to derive a robust error norm which is very similar to Tukey's biweight error norm. Using channel smoothing instead of iterative robust estimator implementations like non-linear diffusion, bilateral filtering, and mean-shift approaches is advantageous since channel smoothing is faster, it is easy to implement, it chooses the global minimum error instead of the nearest local minimum, and it can also be used on non-linear spaces, such as orientation space. As an application, we implemented orientation smoothing and compared it to the other three approaches.

In this paper we propose a new operator which combines advantages of monogenic scale-space and Gaussian scale-space, of the monogenic signal and the structure tensor. The gradient energy tensor (GET) defined in this paper is based on Gaussian derivatives up to third order using different scales. These filters are commonly available, separable, and have an optimal uncertainty. The response of this new operator can be used like the monogenic signal to estimate the local amplitude, the local phase, and the local orientation of an image, but it also allows to measure the coherence of image regions as in the case of the structure tensor

In this paper we address the topics of scale-space and phase-based signal processing in a common framework. The involved linear scale-space is no longer based on the Gaussian kernel but on the Poisson kernel. The resulting scale-space representation is directly related to the monogenic signal, a 2D generalization of the analytic signal. Hence, the local phase arises as a natural concept in this framework which results in several advanced relationships that can be used in image processing.