A memory constrained bayesian optimization via robust online memory estimation

Résumé

Abstract Bayesian optimization (BO) is a memory-intensive algorithm that requires training and evaluating an expensive objective function. In contrast to previous works that use an offline memory estimation to make BO memory-efficient, we propose a robust and simple online memory estimation method that requires training a model only for the first two iterations of the first epoch. Our memory estimation method is then integrated with a simple, performance-based surrogate model of BO in a seamless (or in sync) mode that enforces memory efficiency even if it does not bypass a preset threshold. The online memory estimation method has been evaluated on two different datasets, showing that it is more accurate than the existing offline method ( $$2.19\times $$ 2.19 × for MNIST and $$3.51\times $$ 3.51 × for CIFAR datasets). Furthermore, compared to a memory-unaware baseline, the enhanced BO has no loss of accuracy and is $$11.31\times $$ 11.31 × memory-efficient for a simple CNN-based image classification, and $$5.03\times $$ 5.03 × memory efficient but $$9.27\times $$ 9.27 × slower for a more complex LSTM-based text classification (useful for a resource-constrained environment where delay is tolerable but memory is scarce), while it is $$2.6\times $$ 2.6 × memory efficient but $$1.23\times $$ 1.23 × slower on a pretrained ResNet50 model.

Citer ce document

Accès au document

Texte intégral en lecture en ligne, réservé aux abonnés SPHAERO et aux membres de l'institution. Se connecter

Auteur(s)

Statistiques

Consultations : 1

Téléchargements : 0