Multi-Target Prediction
(tutorial lecture for DS 2013)

Author: Krzysztof Dembczyński

Affiliation: Laboratory of Intelligent Decision Support Systems
Politechnika Poznańska, Poznań Poland

Abstract. Traditional methods in machine learning and statistics provide data-driven models for predicting one-dimensional targets, such as binary outputs in classification and real-valued outputs in regression. In recent years, novel application domains have triggered fundamental research on more complicated problems where multi-target predictions are required. Such problems arise in diverse application domains, such as document categorization, tag recommendation of images, videos and music, information retrieval, natural language processing, drug discovery, biology, etc. Specific multi-target prediction problems have been studied in a variety of subfields of machine learning and statistics, such as multi-label classification, multivariate regression, sequence learning, structured output prediction, preference learning, multi-task learning and collective learning. Despite their commonalities, work on solving problems in the above domains has typically been performed in isolation, without much interaction between the different sub-communities. The main goal of the tutorial is to present a unifying overview of the above-mentioned subfields of machine learning, by focusing on the simultaneous prediction of multiple, mutually dependent output variables. We will distinguish two different views on these problems. The individual-target view concerns improving a prediction quality of a single target by using information from other targets. The joint-target view concerns minimization of complex loss functions that cannot be decomposed to single targets.