Title:Multi-stage Multi-task feature learning via adaptive threshold

Abstract:Multi-task feature learning aims to learn the shared features among tasks to improve the generalization. In [1], it was shown that minimizing non-convex optimization models, for example, one based on the capped-l1, l1 regularization, can obtain a better solution than the convex alternatives. In addition, an efficient multi-stage algorithm was proposed solve these non-convex models. However, they use a fixed threshold in the definition of the capped-l1, l1 regularization. In this paper we propose to employ an adaptive threshold in the capped-l1, l1 regularized formulation, and the corresponding multi-stage multi-task feature learning algorithm (MSMTFL) will incorporate a component to adaptively determine the threshold. The resulted variant of the original MSMTFL algorithm is expected to improve the ability of the original MSMTFL algorithm in terms of better feature selection performance. In particular, the adaptive threshold approach comes from iterative support detection (ISD, for short) method [2], which use the "first significant jump" rule to obtain the threshold adaptively. The rule aims to refine the threshold in each stage. Empirical studies on both synthetic and real-world data sets demonstrate the effectiveness of the new MSMTFL algorithm using an adaptive threshold approach (MSMTFL-AT, for short) over the original version.