Exploring Augmented Random Search: A Faster and Simpler Alternative to Deep Learning

Augmented Random Search (ARS) is a powerful technique in the field of artificial intelligence that has been gaining attention due to its ability to train models efficiently, especially in benchmark tasks such as teaching an AI to walk. ARS has been shown to outperform Google Deep Mind and other models in this regard, being up to 100 times faster. In this blog post, we will discuss the intuition and practical aspects of the ARS algorithm, as well as its comparison with other AI techniques.

Intuition Behind ARS

The intuition behind ARS lies in its philosophical approach and the main concepts that govern its functioning. At the core of ARS is the method of finite differences, which enables the algorithm to learn and adapt quickly. ARS controls 22 degrees of freedom in the MuJoCo engine and constantly receives feedback from the input to control decisions.

A perceptron, a fundamental element of ARS, is essentially one or more sums of weighted inputs. The weight matrix is adjusted after each episode based on the estimated reward provided by the environment. ARS uses Finite Differences Methods to calculate sets of matrices with positively and negatively perturbed weights and runs several episodes with these perturbed weights to find the adjustment for the actual weights.

Three Key Aspects of ARS

1. Scaled update steps by the standard deviation of rewards

2. Online normalization of states

3. Discarding directions that yield the lowest rewards

These three aspects help make ARS an efficient and powerful learning algorithm. The second aspect, online normalization of states, occurs in real-time, ensuring that changes in weights do not have an extreme impact on the perceptron's sum. The third aspect involves removing summands with low-magnitude rewards in the delta calculation.

Comparison with Other AI Techniques

While ARS has proven to be effective in certain applications, it is important to note the differences between ARS and other AI techniques like deep learning. ARS uses the cumulative reward of an episode rather than an action space for each individual action, making it more similar to eligibility traces. Additionally, ARS utilizes the Method of Finite Differences as an approximation of Gradient Descent instead of backpropagation through the network. Lastly, ARS is considered shallow learning, as opposed to deep learning with multiple hidden layers.

Despite these differences, ARS has been shown to be up to 15 times faster than other algorithms, exploiting weaknesses in alternative techniques. However, deep learning is generally more versatile and adaptable, making it more suitable for certain applications.

Conclusion

Augmented Random Search (ARS) is a fascinating and powerful technique that has shown great potential in various benchmark tasks, outperforming other AI models in terms of speed and efficiency. While it may not be as versatile as deep learning, ARS's simpler and faster approach makes it an attractive option for certain applications. Further reading and research on ARS, as well as other techniques like evolution strategies, can help expand our understanding of AI and its potential applications. Stay tuned for a follow-up post on the practical aspects of ARS.

Augmented random search superdatascience/ars MuJoCo engine alexis-jacq numpy ARS Finite Differences Methods open ai evolution strategies evolution strategies as a scalable alternative to reinforcement learning