Fully Parallel Hyperparameter Search: Reshaped Space-Filling - Université du Littoral Côte d'Opale
Communication Dans Un Congrès Année : 2020

Fully Parallel Hyperparameter Search: Reshaped Space-Filling

Résumé

Space-filling designs such as Low Discrepancy Sequence (LDS), Latin Hypercube Sampling (LHS) and Jittered Sampling (JS) were proposed for fully parallel hyperparameter search, and were shown to be more effective than random and grid search. We prove that LHS and JS outperform random search only by a constant factor. Consequently, we introduce a new sampling approach based on the reshaping of the search distribution, and we show both theoretically and numerically that it leads to significant gains over random search. Two methods are proposed for the reshaping: Recentering (when the distribution of the optimum is known), and Cauchy transformation (when the distribution of the optimum is unknown). The proposed methods are first validated on artificial experiments and simple real-world tests on clustering and Salmon mappings. Then we demonstrate that they bring performance improvement in a wide range of expensive artificial intelligence tasks, namely attend/infer/repeat, video next frame segmentation forecasting and progressive generative adversarial networks.
Fichier principal
Vignette du fichier
cauwet20a.pdf (2.36 Mo) Télécharger le fichier
Origine Fichiers produits par l'(les) auteur(s)

Dates et versions

hal-03091435 , version 1 (31-12-2020)

Identifiants

  • HAL Id : hal-03091435 , version 1

Citer

M.-L Cauwet, C Couprie, Julien Dehos, P Luc, J Rapin, et al.. Fully Parallel Hyperparameter Search: Reshaped Space-Filling. 37th International Conference on Machine Learning (ICML), Jul 2020, Virtual Event, France. ⟨hal-03091435⟩
56 Consultations
52 Téléchargements

Partager

More