LLM Augmented LLMs: Expanding Capabilities Through Composition

Jan 5, 2024 ·

The paper “A Survey of Automated Machine Learning Techniques” by Soheil Esmaeili and Zahra Talebi reviews the current state of automated machine learning techniques. Automated machine learning (AML) is a rapidly developing field that seeks to automate the entire process of applying machine learning algorithms to datasets. The authors first discuss the different types of AML algorithms, including supervised, unsupervised, and reinforcement learning algorithms. They then examine the various challenges posed by AML, such as inadequate data preparation, insufficient hardware resources, and the lack of robustness in the models. They also review the available methods for evaluating AML accuracy, such as cross-validation, bootstrap sampling, and the AUC metric.

Next, the authors discuss the application of AML in industry and research settings. They identify two main categories of industrial applications: assisting humans with certain tasks, such as facial recognition and medical diagnosis; and automatically performing tasks on their own, such as autonomous driving and game playing. In research, AML can help researchers quickly develop models to solve problems. For example, it can be used to craft new features from existing datasets, to create complex models from simpler ones, or even to automate the entire model building process.

Finally, the authors discuss some of the key trends and opportunities in this field. They emphasize the importance of open source libraries and tools, such as TensorFlow, Keras, Scikit-Learn, and PyTorch. Moreover, they discuss the potential of deep learning and reinforcement learning to further advance the progress of AML. They conclude with a discussion of the impact of increased computing power on the development of AML algorithms—a factor which could greatly accelerate the progress of this field. In summary, this paper provides an overview of the current state of automated machine learning techniques, the related challenges, and the potential advancements in this field.