ReadingResearchRadarInvestment framework
Sign in / Sign up
← Back to reading
Long-term reading3 min readDisconfirmAI Editorial

Neural Architecture Search: Automatically Designing Neural Network Structures with Reinforcement Learning

In the early days of deep learning, network architecture design was highly dependent on expert experience, as seen in the evolution from AlexNet to ResNet.

Verified 2026-10-25 00:00 GMT+8

In the early days of deep learning, network architecture design was highly dependent on expert experience, as seen in the evolution from AlexNet to ResNet. The 2016 paper "Neural Architecture Search with Reinforcement Learning" by Barret Zoph and Quoc V. Le pioneered the use of a recurrent neural network (RNN) as a controller to generate architectural descriptions of other neural networks, optimizing this process through reinforcement learning.

The core idea lies in treating architecture design as a sequence generation problem. The controller RNN gradually predicts hyperparameters of subnetworks (such as number of layers, filter sizes, etc.), forming a variable-length string description. Once a subnetwork is constructed and trained, its accuracy on the validation set is used as a reward signal. Since accuracy is non-differentiable, the authors employed the REINFORCE policy gradient algorithm to update the controller's parameters, enabling the controller to learn over time how to generate higher-accuracy architectures. To reduce the high variance in gradient estimation, an exponential moving average was introduced as a baseline function.

Experimental results showed that this method achieved a test error rate of 3.65% on the CIFAR-10 dataset when designing architectures from scratch—outperforming many manually designed architectures at the time. On the Penn Treebank language modeling task, it discovered a new type of recursive unit with a perplexity of 62.4, surpassing the widely used LSTM unit. This demonstrates that automated search can produce competitive model structures.

However, the method has extremely high computational cost. Each controller gradient update requires training a complete subnetwork to convergence, typically taking several hours. Although distributed training and asynchronous parameter updates were used to accelerate the process, the overall search remained slow and resource-intensive. Additionally, the design of the search space still required human definition—for example, setting maximum layer counts or specifying possible operation types—meaning that "fully" automated search remained constrained by predefined rule frameworks.

Sign in / Sign up
ReadingResearchRadarInvestment framework

For today's readers, understanding the historical context of NAS helps grasp current trends in AI engineering. Suppose we face a new dataset in an emerging domain; traditionally, one might spend weeks experimenting with different CNN variants. Based on the ideas from this paper, we could instead define a search space and let the algorithm run in the background for several days to find an optimal architecture. Although modern NAS methods have significantly reduced computational demands, Zoph and Le's work established the fundamental paradigm of "using machine learning to optimize machine learning," i.e., meta-learning applied at the architecture level.

We recommend Barret Zoph and Quoc V. Le's 2016 paper "Neural Architecture Search with Reinforcement Learning" to researchers and engineers seeking to deeply understand the origins of automated machine learning. This paper not only demonstrates how reinforcement learning can solve discrete combinatorial optimization problems but also laid the foundation for subsequent efficient NAS algorithms. Suggested reading starts with the introduction and methodology sections, focusing on the interaction mechanism between controller and subnetwork, and the application details of the REINFORCE algorithm.

Sources

  • arxiv.org《Neural Architecture Search with Reinforcement Learning》（2016）↗
Continue researching →