| Paper ID | MLSP-25.5 |
| Paper Title |
ON THE MARGINAL BENEFIT OF ACTIVE LEARNING: DOES SELF-SUPERVISION EAT ITS CAKE? |
| Authors |
Yao-Chun Chan, Mingchen Li, Samet Oymak, University of California, Riverside, United States |
| Session | MLSP-25: Reinforcement Learning 1 |
| Location | Gather.Town |
| Session Time: | Thursday, 10 June, 13:00 - 13:45 |
| Presentation Time: | Thursday, 10 June, 13:00 - 13:45 |
| Presentation |
Poster
|
| Topic |
Machine Learning for Signal Processing: [MLR-SSUP] Self-supervised and semi-supervised learning |
| IEEE Xplore Open Preview |
Click here to view in IEEE Xplore |
| Virtual Presentation |
Click here to watch in the Virtual Conference |
| Abstract |
Active learning is the set of techniques for intelligently labeling large unlabeled datasets to reduce the labeling effort. In parallel, recent developments in self-supervised and semi-supervised learning (S4L) provide powerful techniques, based on data-augmentation, contrastive learning, and self-training, that enable superior utilization of unlabeled data which led to a significant reduction in required labeling in the standard machine learning benchmarks. A natural question is whether these paradigms can be unified to obtain superior results. To this aim, this paper provides a novel algorithmic framework integrating self-supervised pretraining, active learning, and consistency-regularized self-training. We conduct extensive experiments with our framework on CIFAR10 and CIFAR100 datasets. These experiments enable us to isolate and assess the benefits of individual components which are evaluated using state-of-the-art methods (e.g.~Core-Set, VAAL, simCLR, FixMatch). Our experiments reveal two key insights: (i) Self-supervised pre-training significantly improves semi-supervised learning, especially in the few-label regime, (ii) The benefit of active learning is undermined and subsumed by S4L techniques. Specifically, we fail to observe any additional benefit of state-of-the-art active learning algorithms when combined with state-of-the-art S4L techniques. |
