Generative Feature Training of Thin 2-Layer Networks

This repository contains the implementations for the paper "Generative Feature Training of Thin 2-Layer Networks". The code is written in PyTorch (version 2.4).

Do not hesitate to contact us (contact details can be found here and here), if you have any questions.

Usage

Below, we describe the usage of the code for reproducing the results from the paper.

Generative Feature Transforms

The script run_experiments.py starts the generative feature training. It takes three input arguments for selecting the experiment:

• the argument functions specifies whether to run the examples for function approximation (True) or for regression on the UCI datasets (False). Default is False.

• the argument setting specifies which exact setting. For function approximation, the settings 0 to 5 refer to the settings from Table 1, the settings 6 to 8 refer to the visualizations from Section 4.2 and settings 9 to 11 refer to the settings from Table 2. For regression the settings 0 to 5 refer to the different dataset (in the same order as in Table 3).

• the argument activation selects the Phi (choices Fourier and sigmoid). Default is Fourier.

• the argument n_features selects the number of features. Default is 100.

Some examples:

python run_experiments.py --functions True --setting 0 --activation Fourier

for reproducing the results from the first column Table 1 for GFT and GFT-p with Fourier activation

python run_experiments.py --setting 0 --activation sigmoid

for reproducing the results from Table 2 for GFT and GFT-p with propulsion dataset and sigmoid activation

Neural Network Comparison

The script neural_network.py starts the comparison for neural networks with the same input arguments as the generative feature transforms. The script run_experiments_gd.py does the same for the F-Opt line.

Citation

@article{HN2025,
  title={Generative Feature Training of Thin 2-Layer Networks},
  author={Hertrich, Johannes and Neumayer, Sebastian},
  journal={Transactions of Machine Learning Research (accepted)},
  url={https://openreview.net/forum?id=6oXNpKuBDK},
  year={2024}
}