Description

Training an MLP regressor (or classifier) using l-bfgs currently cannot run for more than (approx) 15000 iterations.
This artificial limit is caused by the call site to l-bfgs passing the MLP argument value "max_iters" to the argument for "maxfun" (maximum number of function calls), but not for "maxiter" (maximum number of iterations), so that no matter how large a number you pass as "max_iters" to train for MLP, the iterations are capped by the default value for maxiter (15000).

Steps/Code to Reproduce

Fit an MLP for a problem that requires > 15000 iterations

Here is an example (tested in python 2.7):
https://gist.github.com/daniel-perry/d9e356a03936673e58e0ce47d5fc70ef

(you will need data.npy from the gist linked to above)

from __future__ import print_function
import numpy as np
from sklearn.neural_network import MLPRegressor

train = np.load("data.npy").tolist()

max_iter = 18000
clf = MLPRegressor(max_iter=max_iter, activation='relu', solver='lbfgs', verbose=True)

clf.fit(train["train_x"],train["train_y"])

print("score: ", clf.score(train["train_x"],train["train_y"]))
print("iters: ", clf.n_iter_, " / ", max_iter)

Expected Results

The training should run for 18000 iterations.

Actual Results

The training runs for 15000 iterations.

Versions

Here are my local version details, though the problem appears to exist on the current head, and so should exist for any python/sklearn versions.

'Python', '2.7.12 (default, Jul 1 2016, 15:12:24) \n[GCC 5.4.0 20160609]'
'NumPy', '1.13.0'
'SciPy', '0.19.1'
'Scikit-Learn', '0.18'