The balancing principle for parameter choice in distance-regularized domain adaptation
Borrowing the balancing principle from ill-posed inverse problems to justify the regularization parameter when the target domain has no labels.
Abstract
We address the unsolved algorithm design problem of choosing a justified regularization parameter in unsupervised domain adaptation. This problem is intriguing as no labels are available in the target domain. Our approach starts with the observation that the widely-used method of minimizing the source error, penalized by a distance measure between source and target feature representations, shares characteristics with regularized ill-posed inverse problems. Regularization parameters in inverse problems are optimally chosen by the fundamental principle of balancing approximation and sampling errors. We use this principle to balance learning errors and domain distance in a target error bound. As a result, we obtain a theoretically justified rule for the choice of the regularization parameter. In contrast to the state of the art, our approach allows source and target distributions with disjoint supports. An empirical comparative study on benchmark datasets underpins the performance of our approach.
The paper appeared at NeurIPS 2021 and is in the proceedings.
Read next
- Parameter choice and neuro-symbolic approaches for deep domain-invariant learning My doctoral thesis at JKU Linz: how to pick hyperparameters for domain adaptation when the target has no labels, and what to do when you cannot train at all.
- Addressing parameter choice issues in unsupervised domain adaptation by aggregation Extending weighted least squares to vector-valued functions, with a target error asymptotically no worse than twice the unknown optimal aggregation.