Publications by Johan Wiklund

This paper presents a method that combines shadow detection and a 3D box model including shadow simulation, for estimation of size and position of vehicles. We define a similarity measure between a simulated image of a 3D box, including the box shadow, and a captured image that is classified into background/foreground/shadow. The similarity Measure is used in all optimization procedure to find the optimal box state. It is shown in a number of experiments and examples how the combination shadow detection/simulation improves the estimation compared to just using detection or simulation, especially when the shadow detection or the simulation is inaccurate. We also describe a tracking system that utilizes the estimated 3D boxes, including highlight detection, spatial window instead of a time based window for predicting heading, and refined box size estimates by weighting accumulated estimates depending oil view. Finally, we show example results.

This paper describes a method for vision-based unmanned aerial vehicle (UAV) motion estimation from multiple planar homographies. The paper also describes the determination of the relative displacement between different UAVs employing techniques for blob feature extraction and matching. It then presents and shows experimental results of the application of the proposed technique to multi-UAV detection of forest fires.  

The management of environmental and industrial disasters, search and rescue operations, surveillance of natural scenarios, environmental monitoring, and many other field robotics applications require high mobility and the need to reach locations that are difficult to access with ground vehicles. In many cases, the use of aerial vehicles is the best way to approach the objective to get information or to deploy instrumentation. Unmanned air vehicles (UAVs) have significantly increased their flight performance and autonomous onboard processing capabilities in the last ten years. But a single aerial vehicle equipped with a large array of different sensors of various modalities is limited at any time to a single viewpoint. A team of aerial vehicles, however, can simultaneously collect information from multiple locations and exploit the information derived from multiple disparate points. Furthermore, having a team with multiple heterogeneous aerial vehicles offers additional advantages due to the possibility of beneficial complementarities of the vehicles.

The problem of detection of orientation in finite dimensional Euclidean spaces is solved in the least squares sense. In particular, the theory is developed for the case when such orientation computations are necessary at all local neighborhoods of the n-dimensional Euclidean space. Detection of orientation is shown to correspond to fitting an axis or a plane to the Fourier transform of an n-dimensional structure. The solution of this problem is related to the solution of a well-known matrix eigenvalue problem. Moreover, it is shown that the necessary computations can be performed in the spatial domain without actually doing a Fourier transformation. Along with the orientation estimate, a certainty measure, based on the error of the fit, is proposed. Two applications in image analysis are considered: texture segmentation and optical flow. An implementation for 2-D (texture features) as well as 3-D (optical flow) is presented. In the case of 2-D, the method exploits the properties of the complex number field to by-pass the eigenvalue analysis, improving the speed and the numerical stability of the method. The theory is verified by experiments which confirm accurate orientation estimates and reliable certainty measures in the presence of noise. The comparative results indicate that the proposed theory produces algorithms computing robust texture features as well as optical flow. The computations are highly parallelizable and can be used in realtime image analysis since they utilize only elementary functions in a closed form (up to dimension 4) and Cartesian separable convolutions.

Tensor valued data are frequently used in medical imaging. For a  3-dimensional second order tensor such data imply at least six  degrees of freedom for each voxel. The operators ability to perceive  this information is of outmost importance and in many cases a  limiting factor for the interpretation of the data. In this paper we  propose a decomposition of such tensor fields using the Tflash  tensor glyphs that intuitively conveys important tensor features to  a human observer. A matlab implementation for visualization of  single tensors are described in detail and a VTK/ITK implementation  for visualization of tensor fields have been developed as a Medical  Studio component.

This Chapter is devoted to the cooperation of multiple UAVs for environment perception. First, probabilistic methods for multi-UAV cooperative perception are analyzed. Then, the problem of multi-UAV detection, localization and tracking is described, and local image processing techniques are presented. Then, the Chapter shows two approaches based on the Information Filter and on evidence grid representations.

The Application Visualization System software from Advanced Visual Systems Inc is an interactive visualization environment for scientists, engineers and technical professionals. This report contains a short overview of the AVS software packages and a discussion about its general performance. The software package has actively been used at the Computer Vision Laboratory, Linköping University, during the last three years. The AVS package has been used in many applications. Examples are generating images from a virtual environment, simulation of a controllable robot with a stereo camera head and visualization of multidimensional data structures. Lately we also have used AVS for handling communication between different processes which may be distributed on different machines. AVS was primarily developed as a tool for visualization of complex data sets. However, another important aspect of the software is that it can be used as an advanced workbench for controlling networks of Unix processes (including external ones on different machine types) using simple visual programming.

This paper explores the possibility to use a single low-resolution FIR camera for detection of pedestrians in the near zone in front of a vehicle. A low resolution sensor reduces the cost of the system, as well as the amount of data that needs to be processed in each frame.

We present a system that makes use of hot-spots and image positions of a near constant bearing to detect potential pedestrians. These detections provide seeds for an energy minimization algorithm that fits a pedestrian model to the detection. Since false alarms are hard to tolerate, the pedestrian model is then tracked, and the distance-to-collision (DTC) is measured by integrating size change measurements at sub-pixel accuracy, and the car velocity. The system should only engage braking for detections on a collision course, with a reliably measured DTC.

Preliminary experiments on a number of recorded near collision sequences indicate that our method may be useful for ranges up to about 10m using an 80x60 sensor, and somewhat more using a 160x120 sensor. We also analyze the robustness of the evaluated algorithm with respect to dead pixels, a potential problem for low-resolution sensors.

This paper presents and overview of the basic and applied research carried out by the Computer Vision Laboratory, Linköping University, in the WITAS UAV Project. This work includes customizing and redesigning vision methods to fit the particular needs and restrictions imposed by the UAV platform, e.g., for low-level vision, motion estimation, navigation, and tracking. It also includes a new learning structure for association of perception-action activations, and a runtime system for implementation and execution of vision algorithms. The paper contains also a brief introduction to the WITAS UAV Project.

This paper presents a general strategy for automated generation of efficient representations in vision. The approach is highly task oriented and what constitutes the relevant information is defined by a set of examples. The examples are pairs of situations that are dependent through the chosen feature but are otherwise independent. Particularly important concepts in the work are mutual information and canonical correlation. How visual operators and representations can be generated from examples are presented for a number of features, e.g. local orientation, disparity and motion. Interesting similarities to biological vision functions are observed. The results clearly demonstrates the potential of combining advanced filtering techniques and learning strategies based on canonical correlation analysis (CCA).

This paper presents a general approach for obtaining optimal filters as well as filter sequences. A filter is termed optimal when it minimizes a chosen distance measure with respect to an ideal filter. The method allows specification of the metric via simultaneous weighting functions in multiple domains, e.g. the spatio-temporal space and the Fourier space. Metric classes suitable for optimization of localized filters for multidimensional signal processing are suggested and discussed.

It is shown how convolution kernels for efficient spatio-temporal filtering can be implemented in practical situations. The method is based on applying a set of jointly optimized filter kernels in sequence. The optimization of sequential filters is performed using a novel recursive optimization technique. A number of optimization examples are given that demonstrate the role of key parameters such as: number of kernel coefficients, number of filters in sequence, spatio-temporal and Fourier space metrics.

The sequential filtering method enables filtering using only a small fraction of the number of filter coefficients required using conventional filtering. In multidimensional filtering applications the method potentially outperforms both standard convolution and FFT based approaches by two-digit numbers.

This paper presents a new and efficient approach for optimization and implementation of filter banks e.g. velocity channels, orientation channels and scale spaces. The multi layered structure of a filter network enable a powerful decomposition of complex filters into simple filter components and the intermediary results may contribute to several output nodes. Compared to a direct implementation a filter network uses only a fraction of the coefficients to provide the same result. The optimization procedure is recursive and all filters on each level are optimized simultaneously. The individual filters of the network, in general, contain very few non-zero coefficients, but there are are no restrictions on the spatial position of the coefficients, they may e.g. be concentrated on a line or be sparsely scattered. An efficient implementation of a quadrature filter hierarchy for generic purposes using sparse filter components is presented.

A scheme for performing generalized convolutions is presented. A flexible convolver, which runs on standard workstations, has been implemented. It is designed for maximum throughput and flexibility. The implementation incorporates spatio-temporal convolutions with configurable vector combinations. It can handle general multi-linear operations, i.e. tensor operations on multidimensional data of any order. The input data and the kernel coefficients can be of arbitrary vector length. The convolver is configurable for IIR filters in the time dimension. Other features of the implemented convolver are scattered kernel data, region of interest and subsampling. The implementation is done as a C-library and a graphical user interface in AVS (Application Visualization System).

There are a number of different algorithms for object motion estimation in image sequences. Almost every algorithm is based on one of three different mathematical methods. An overview of these methods is given, together with some published application examples on object tracking.

A method for tracking of multiple moving objects has been developed on a GOP 300 image processing system. This method works on image sequences with a stationary background, and can be divided into the following steps:

1. Find the positions for all objects that have moved.
2. Predict the new positions for all known objects.
3. Match these two sets of points.
4. Produce the required output.

These steps are repeated for every sample of the sequence. As an output from every processed sample in a test sequence, both a resulting image and a record in a datafile have been generated. The resulting image is a copy of the actual sample with the active object identities overlayed at the corresponding positions. The resulting images have been stored on a video tape.

One major goal of the COSPAL project is to develop an artificial cognitive system architecture with the capability of exploratory learning. Exploratory learning is a strategy that allows to apply generalization on a conceptual level, resulting in an extension of competences. Whereas classical learning methods aim at best possible generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class. Incremental or online learning is an inherent requirement to perform exploratory learning.

Exploratory learning requires new theoretic tools and new algorithms. In the COSPAL project, we mainly investigate reinforcement-type learning methods for exploratory learning and in this paper we focus on its algorithmic aspect. Learning is performed in terms of four nested loops, where the outermost loop reflects the user-reinforcement-feedback loop, the intermediate two loops switch between different solution modes at symbolic respectively sub-symbolic level, and the innermost loop performs the acquired competences in terms of perception-action cycles. We present a system diagram which explains this process in more detail.

We discuss the learning strategy in terms of learning scenarios provided by the user. This interaction between user (’teacher’) and system is a major difference to most existing systems where the system designer places his world model into the system. We believe that this is the key to extendable robust system behavior and successful interaction of humans and artificial cognitive systems.

We furthermore address the issue of bootstrapping the system, and, in particular, the visual recognition module.We give some more in-depth details about our recognition method and how feedback from higher levels is implemented. The described system is however work in progress and no final results are available yet. The available preliminary results that we have achieved so far, clearly point towards a successful proof of the architecture concept.

A recursive method to condense general multidimensional FIR-filters into a sequence of simple kernels with mainly one dimensional extent has been worked out. Convolver networks adopted for 2, 3 and 4D signals is presented and the performance is illustrated for spherically separable quadrature filters. The resulting filter responses are mapped to a non biased tensor representation where the local tensor constitutes a robust estimate of both the shape and the orientation (velocity) of the neighbourhood. A qualitative evaluation of this General Sequential Filter concept results in no detectable loss in accuracy when compared to conventional FIR (Finite Impulse Response) filters but the computational complexity is reduced several orders in magnitude. For the examples presented in this paper the attained speed-up is 5, 25 and 300 times for 2D, 3D and 4D data respectively The magnitude of the attained speed-up implies that complex spatio-temporal analysis can be performed using standard hardware, such as a powerful workstation, in close to real time. Due to the soft implementation of the convolver and the tree structure of the sequential filtering approach the processing is simple to reconfigure for the outer as well as the inner (vector length) dimensionality of the signal. The implementation was made in AVS (Application Visualization System) using modules written in C.