Volltext-Downloads (blau) und Frontdoor-Views (grau)

Evolving Parsimonious Networks by Mixing Activation Functions

  • Neuroevolution methods evolve the weights of a neural network, and in some cases the topology, but little work has been done to analyze the effect of evolving the activation functions of individual nodes on network size, an important factor when training networks with a small number of samples. In this work we extend the neuroevolution algorithm NEAT to evolve the activation function of neurons in addition to the topology and weights of the network. The size and performance of networks produced using NEAT with uniform activation in all nodes, or homogenous networks, is compared to networks which contain a mixture of activation functions, or heterogenous networks. For a number of regression and classification benchmarks it is shown that, (1) qualitatively different activation functions lead to different results in homogeneous networks, (2) the heterogeneous version of NEAT is able to select well performing activation functions, (3) the produced heterogeneous networks are significantly smaller than homogeneous networks.

Export metadata

Additional Services

Search Google Scholar Check availability

Statistics

Show usage statistics
Metadaten
Document Type:Conference Object
Language:English
Author:Alexander Hagg, Maximilian Mensing, Alexander Asteroth
Parent Title (German):GECCO '17: Proceedings of the Genetic and Evolutionary Computation Conference. Berlin, Germany, July 15-19, 2017
First Page:425
Last Page:432
ISBN:978-1-4503-4920-8
DOI:https://doi.org/10.1145/3071178.3071275
ArXiv Id:http://arxiv.org/abs/1703.07122
Publisher:Association for Computing Machinery
Place of publication:New York, NY, United States
Date of first publication:2017/07/01
Copyright:© 2017 Copyright held by the owner/author(s). Publication rights licensed to ACM. Abstracting with credit is permitted.
Funding:This work received funding from the German Federal Ministry of Education and Research (BMBF) under the Forschung an Fachhochschulen mit Unternehmen programme (grant agreement number 03FH012PX5 project "Aeromat").
Keyword:activation function; bloat; heterogeneous networks; neuroevolution; regression
Departments, institutes and facilities:Fachbereich Informatik
Institut für Technik, Ressourcenschonung und Energieeffizienz (TREE)
Projects:AErOMAt - Automatisiertes Entwickeln aerodynamischer Strukturen und Fahrzeuge mithilfe evolutionärer Optimierung und Surrogatmodellierung (DE/BMBF/03FH012PX5,13FH012PX5)
Dewey Decimal Classification (DDC):0 Informatik, Informationswissenschaft, allgemeine Werke / 00 Informatik, Wissen, Systeme / 004 Datenverarbeitung; Informatik
Entry in this database:2017/04/26