- Description of the Research Group +
The VisLab group was founded in 1992 and conducts research on computer and robot vision, focussing on video analysis, surveillance, learning, humanoid robotics and cognition. Our mission is twofold: (i) the development of new methodologies and tools for computer and robot vision and (ii) the demonstration of such methodologies in challenging applications.
We take a highly multidisciplinary perspective, combining disciplines like engineering, neuroscience and psychology, with the twin goals of drawing inspiration from biology to develop advanced artificial systems as well as modeling biological systems with computational and robotic tools.
Over the years we have designed various experimental testbeds for acquiring real sensor data and experience realistic scenarios. Today, the lab is equipped with cutting-edge research infrastructures, including one iCub humanoid robot, that allows us to develop frontier research in computer and robot Vision.
Internationalization has also been an important strategic direction of the lab, not only for recruiting talented international students and researchers but also concerning the participation in a large number of EU projects with more than 100 different partners, academic and corporate, from the EU, USA and Japan.
In the first years we focused on two main lines of research: (i) computer vision systems and (ii) vision based navigation and control.
In the framework of computer vision, we developed methodologies for video surveillance 3D reconstruction (both from stereo and from monocular images with geometric priors), motion analysis (segmentation, filtering, egomotion estimation), camera design (omnidirectional images, non-central cameras), space-variant images (log-polar) and panoramic vision.
The research line on vision based navigation and control was based on a multitude of robotic systems where inspiration from biology was often drawn. One such example was the use of optic flow for navigation (similar) to insects, reconstruction and usage of topological maps, docking systems, vision control of aerial or swimming vehicles and active binocular heads.
We made important contributions to all these areas of research that are well documented on a large set of ISI papers with a large number of citations and impact in the field, 15 PhD theses and several tens of MSc dissertations. For more than a decade, we have been working with neuroscientists, psychologists and linguists in multidisciplinary projects, addressing aspects like artificial learning and cognition, taking advantage of the massive amounts of memory and processing capacity that are currently available.
Since 2008, the lab has been involved in the IST-EPFL Joint Doctoral Program with the Lab coordinator acting as Director of this International Doctoral Program, where robotics vision is one of the most active domains. Two VisLab doctoral students have already graduated from this program, and were awarded a joint diploma by IST and EPFL. More recently, in 2013, the VisLab director led a proposal to the FCT Doctoral Schools call, entitled "Robotics, Brain and Cognition" where, once again, the Vislab research areas are central in the program.
A few spin off companies emerged from the research team, led by alumni with both entrepreneurial and technical skills in the areas of video surveillance, 3D modeling and robotics. Collaborations with these companies continues through contractual projects that allow technology transfer to the market and provide employment alternatives for MSc and PhD students graduates trained in the laboratory.
- Main achievements +
The work at VisLab aims to develop new methodologies for computer and robot vision and to use those tools to address several applications with societal impact. The approach followed is strongly multidisciplinary with close links to biology, neuroscience and psychology. The group is regularly involved in large-scale, ambitious projects with international partners (e.g. EU).
Some of the main achievements are listed below:
- Structure of the Research Group +
Currently, the lab is composed of 4 permanent members (faculty), 5 postdoctoral researchers, 14 doctoral students, 8 research assistants and 1 lab technician.
The organization structure consists of the coordinator, the management board and the scientific board. The coordinator is in charge of the overall scientific coordination and communication to the LARSYS top management and external stakeholders. The management board is composed by all (4) permanent members and the scientific board is composed of all the PhD holders (permanent or temporary).
The research is organized in four main pillars, each coordinated by one PI (faculty). While the topics are often intersecting, this organization has proven to be quite effective to ensure the adequate strategic development of the various areas. The current research areas and coordinators is listed below:
Coordinator: José Santos-Victor
Deputy coordinator: Alexandre Bernardino
The group runs a weekly meeting since 1993. The lab meeting has proved to be instrumental for accelerating the learning and knowledge transfer amongst the group members as well as for discussing all issues related to the lab organization, deadlines, purchases, problems, new ideas, etc.
The group also hosts several research associates, mainly consisting of research engineers and computer scientists that contribute to the development and integration of subsystems and research results, often in the context of international projects. VisLab members represent a mix of nationalities and cultures, from Europe and international. The working language is English.
The VisLab is equipped with sophisticated research platforms. One lab technician is employed with the main job of maintaining those platforms and keeping them operational. A mechanical engineer is also hired by the group with the responsibility of designing parts and mechanisms for the different robots and platforms.
- Objectives of the Research Group +
Through more than two decades of activity, we have built a solid research team and operate a cutting edge research infrastructure. The following paragraphs describe the research plan for the VisLab four main research areas:
a1 - Image analysis and surveillance
Video surveillance plays a major role in modern societies. We will extend our work on the analysis of pedestrian activities using static cameras, in two directions: (i) the analysis of multiple pedestrians and their interactions; (ii) large scale problems (e.g., surveillance of sea or forest) using unmanned aerial vehicles (UAV) that allow the observation of wide areas, raising a set of challenging problems: image stabilization, background modeling, event detection. This activity will be done in the scope of project Seagull involving the Portuguese Air Force and Navy.
We will also consider other applications dealing with the interpretation of visual data (e.g., image retrieval, medical diagnosis, space science). The analysis of such images raises several challenging problems: i) how to extract valuable information? ii) how to process huge amounts of data? iii) how to incorporate human knowledge? We will address these problems in a Machine Learning framework trying to develop probabilistic models for the visual information. Recent advances in hierarchical Bayesian models are a direction to be explored.
a2 - Visual Navigation and Calibration;
Navigation of wheeled, legged or humanoid robots requires keeping some sort of continuous self-calibration based on proprioceptive data combined with vision or inertial sensors. Most approaches rely on exteroception for unknown or dynamic environments or on full environment reconstruction (e.g. SLAM), at the cost of prohibitive computational resources. Instead, we will explore the use of customized vision sensors, topological, appearance-based representations and attention mechanisms to develop efficient navigation systems, with a parsimonious use of hardware and computational resources.
We will further develop the use and design of novel, non-conventional cameras, combining different reflective surfaces with lenses, space-variant imaging, different sensitivity spectra, depth sensing, aggregation of multiple cameras, and active light projection.
a3 - Bio-inspired Vision and Learning;
We continue research in biologically inspired visual systems, focusing on the role of motor control and learning on the efficiency and performance of visual processes for robots. Currently we are investigating the coupling between the visual, attentional, prediction and motor systems of biological agents to devise robots with matched sensorimotor systems learned by self-exploration in the target environment. Based on the results obtained up to now, and supported by a recently approved national project, we will target the self-organization of visual sensorimotor systems with high efficiency (energy, computation) and performance (reaction time bandwidth) that can foster a new generation of highly optimized embedded visual chips for low-cost purposive robots.
a4 - Cognitive Robots
We will conduct research on the recognition of human gestures with applications in programming by demonstration of humanoid robot companions. Learning how to interact with objects, learning how to grasp and reasoning about the world in terms of language, will also be one key research direction in the next few years.
Particular emphasis will be given to all aspects of sensorimotor coordination, affordance representation and vision.
These research efforts will substantiate the collaboration of the VisLab in the LARSYS thematics areas, notably in the interaction with life sciences, cognitive robots and systems and, possibly, in the areas of aerospace and urban systems.
Finally, we will pursue the strategy of internationalization through the active participation and leadership in international projects and partnerships (e.g the IST-EPFL Doc. program or the FET Flagships)