Gabriela Kadlecová, Lukasik, J., Pilát, M., Petra Vidnerová, Safari, M., Roman Neruda, Hutter, F.
Surprisingly Strong Performance Prediction with Neural Graph Features.
Recent advances in AI and ML demand the leveraging of increasingly more complex neural network architectures that should be fast to train, easy to interpret, and as accurate as possible. The ideal network architecture clearly depends on the task at hand. Recently, neural architecture search (NAS) techniques have attracted broad awareness due to an automatic exploration of viable network models. Various performance prediction characteristics have been investigated to ameliorate the high computational costs necessary to train the probed networks, among others, the zero-cost proxies. Motivated by their limits, Kadlecová et al. introduced new innovative neural graph features (GRAF) that allow for higher stability and interpretability of the found results. In addition to being straightforward and fast to evaluate, they outperform most other predictors across multiple benchmarks, particularly when combined with other zero-cost proxies.
Scientists from multiple international institutions collaborated on the research. The paper was presented at the prestigious ICML conference in July 2024; since then, it has been cited twice. Its first author, Ms. Kadlecová, is a young author pursuing her PhD.