"An adaptive algorithm for unsupervised learning"

• paper info : https://laurentperrinet.github.io/publication/perrinet-19-hulk/
• PDF
• code for paper: https://github.com/SpikeAI/HULK including the LaTeX source
• code for framework: https://github.com/bicv/SparseHebbianLearning/
• code for figures https://github.com/SpikeAI/HULK/blob/master/Annex.ipynb (which is rendered @ https://spikeai.github.io/HULK/ )
• video abstract (and the code for generating it)
• LICENSE

---

This supplementary information presents :

• first, the code to generate the figures from the paper,
• second, some control experiments that were mentionned in the paper,
• finally, some perspectives for future work inspired by the algorithms presented in the paper.

%matplotlib inline

%load_ext autoreload
%autoreload 2

A convenience script model.py allows to run and cache most learning items in this notebooks:

%run model.py

tag = HULK
n_jobs = 0

from shl_scripts.shl_experiments import SHL
shl = SHL(**opts)
data = shl.get_data(matname=tag)

shl?

Type:        SHL
String form: <shl_scripts.shl_experiments.SHL object at 0x1134c3910>
File:        ~/science/HULK/SparseHebbianLearning/shl_scripts/shl_experiments.py
Docstring:  
Base class to define SHL experiments:
    - initialization
    - coding and learning
    - visualization
    - quantitative analysis

print('# of pixels per patch =', shl.patch_width**2)

# of pixels per patch = 441

print('number of patches, size of patches = ', data.shape)
print('average of patches = ', data.mean(), ' +/- ', data.mean(axis=1).std())
SE = np.sqrt(np.mean(data**2, axis=1))
print('average energy of data = ', SE.mean(), '+/-', SE.std())

number of patches, size of patches =  (65520, 441)
average of patches =  -4.1888600727021664e-05  +/-  0.006270387629074682
average energy of data =  0.26082782604823146 +/- 0.07415089441760706

#!ls -l {shl.cache_dir}/
#!ls -l {shl.cache_dir}/{tag}*
#!ls -ltr {shl.cache_dir}/{tag}*lock*
#!rm {shl.cache_dir}/{tag}*lock*
#!rm {shl.cache_dir}/{tag}*
#!rm {shl.cache_dir}/{tag}*HAP_seed*
#!ls -l {shl.cache_dir}/{tag}*
#!ls -ltr {shl.cache_dir}/{tag}*lock*

figure 1: Role of homeostasis in learning sparse representations

fname = 'figure_map'
# we cross-validate with 10 different learnings
one_cv = 8 # and pick one at random to display intermediate results

learning

The actual learning is done in a second object (here dico) from which we can access another set of properties and functions (see the shl_learn.py script):

homeo_methods = ['None', 'OLS', 'HEH']

list_figures = ['show_dico', 'time_plot_error', 'time_plot_logL', 'time_plot_MC', 'show_Pcum']
list_figures = []
dico = {}
for i_cv in range(N_cv):
    dico[i_cv] = {}
    for homeo_method in homeo_methods:
        shl = SHL(homeo_method=homeo_method, seed=seed+i_cv, **opts)
        dico[i_cv][homeo_method] = shl.learn_dico(data=data, list_figures=list_figures, matname=tag + '_' + homeo_method + '_seed=' + str(seed+i_cv))

list_figures = ['show_dico']
for i_cv in [one_cv]:
    for homeo_method in homeo_methods:
        print(hl + hs + homeo_method[:3] + hs + hl)
        shl = SHL(homeo_method=homeo_method, seed=seed+i_cv, **opts)
        shl.learn_dico(data=data, list_figures=list_figures, matname=tag + '_' + homeo_method + '_seed=' + str(seed+i_cv))

        print('size of dictionary = (number of filters, size of imagelets) = ', dico[i_cv][homeo_method].dictionary.shape)
        print('average of filters = ',  dico[i_cv][homeo_method].dictionary.mean(axis=1).mean(), 
              '+/-',  dico[i_cv][homeo_method].dictionary.mean(axis=1).std())
        SE = np.sqrt(np.sum(dico[i_cv][homeo_method].dictionary**2, axis=1))
        print('average energy of filters = ', SE.mean(), '+/-', SE.std())
        plt.show()

----------          Non          ----------
size of dictionary = (number of filters, size of imagelets) =  (676, 441)
average of filters =  -7.038410893224767e-06 +/- 0.0008419021277793694
average energy of filters =  1.0 +/- 3.866729645080236e-17