Just in case you didn't saw, It seem like your PR is already conflicting ... :-(
Hope it's not too bad...

That's a bummer, because this one is absolutely miserable to manage/rebase. So far I don't quite know how to handle it without having to manually mess with the internals of the repo, maybe it's possible. I'm at a conference now, will deal with it in a couple of days.

If people can at least provide feedback directly from the branch, it would be useful. Because of the submodule move, managing this one is nasty and I'd like to have it in flight as little as possible.

I thought we were going to wait until Zack has merged his multi directory support to do this. He has Already done two difficult rebases in this one would be another tough one. Also there maybe other pull requests that will conflict with this one as well

Sent from my iPhone

On Jun 3, 2013, at 6:05 PM, Fernando Perez [email protected] wrote:

That's a bummer, because this one is absolutely miserable to manage/rebase. So far I don't quite know how to handle it without having to manually mess with the internals of the repo, maybe it's possible. I'm at a conference now, will deal with it in a couple of days.

If people can at least provide feedback directly from the branch, it would be useful. Because of the submodule move, managing this one is nasty and I'd like to have it in flight as little as possible.

—
Reply to this email directly or view it on GitHub.

And this would most probably conflict with @minrk 's bootstrapify...

@ellisonbg as I said, this isn't ready for merge yet... I just wanted to see if it can be done. @minrk and I were talking about it in detail, and couldn't quite convince ourselves that it would work. So I decided to just try and implement it to see if it was indeed possible.

What I would love to know, however, is how to switch in and out of this specific PR without a complete clusterfuck with git submodules. Because right now that makes this PR extremely unwieldy.

But we should sort out whether this can work at all or not: given that it's likely to conflict with other PRs, it means we'll need to do it on a 'flag day' after as many of them are merged, and without any others in the queue. For that reason, we need to finish testing it in isolation until it's working reasonably well, so that when the day comes, the job is only a (likely nasty) rebase + merge.

Because if we don't work on it in parallel (even if it's unmergeable), then when we have the window with other conflicting work out of the way, we'd have to rush to do all the actual coding work quickly, and that's just not a good idea (we don't even know if there are going to be other difficulties).

So the only solution is to keep working on this until it's ready in isolation (testing it as an unmerged branch). And then, pulling a single flag day merge for it when we have a good window where the other PRs that can conflict have gone in.

Testing in isolation, and only rebasing once when it's ready for merge makes sense, since it's really just a big rename. The only new code here should be the proxy module for deprecating IPython.frontend.

One test fails:

dreload(sys.modules[get_ipython().__module__])

Run it once, and you will get an ImportError, a second time and IPython will crash all the way.

Weird... I'll look into it when I get back. Thanks for the feedback!

reload is always my first test when messing with imports, since almost every magical thing breaks under reload unless you pay special attention.

Notebook broken with this, Jinja is probably looking in the wrong folder.

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/tornado/web.py", line 1042, in _execute
    getattr(self, self.request.method.lower())(*args, **kwargs)
  File "/Users/matthiasbussonnier/ipython/IPython/frontend/html/notebook/auth/login.py", line 43, in get
  File "/Users/matthiasbussonnier/ipython/IPython/frontend/html/notebook/auth/login.py", line 36, in _render
  File "/Users/matthiasbussonnier/ipython/IPython/frontend/html/notebook/base/handlers.py", line 264, in render_template
  File "/Users/matthiasbussonnier/ipython/IPython/frontend/html/notebook/base/handlers.py", line 260, in get_template
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Jinja2-2.6-py2.7.egg/jinja2/environment.py", line 719, in get_template
    return self._load_template(name, self.make_globals(globals))
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Jinja2-2.6-py2.7.egg/jinja2/environment.py", line 693, in _load_template
    template = self.loader.load(self, name, globals)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Jinja2-2.6-py2.7.egg/jinja2/loaders.py", line 115, in load
    source, filename, uptodate = self.get_source(environment, name)
  File "/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/Jinja2-2.6-py2.7.egg/jinja2/loaders.py", line 180, in get_source
    raise TemplateNotFound(template)
TemplateNotFound: login.html

@Carreau, are you sure you moved the submodule correctly? The nb works for me, but I had to mess very badly with my repo to make this work. It might be easiest for you if you just test this in a separate clone, so that you can mangle its submodule struccture without damaging the one you work on normally...

@fperez I'll retry. Do you have 5 min on hipchat ?

Ok, stalled *.pyc plus setup.py develop that check component still in frontend. A few warning messages, but seem to work for now.

OK, I could use some extra testing on this now... I have the test suite passing on my desktop except for two weird errors, but which I don't think are related to this:

======================================================================
ERROR: test suite for <module 'IPython.core.tests.test_compilerop' from '/home/fperez/usr/lib/python2.7/site-packages/IPython/core/tests/test_compilerop.pyc'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 314, in setupContext
    try_run(context, names)
  File "/usr/lib/python2.7/dist-packages/nose/util.py", line 478, in try_run
    return func()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/tests/test_compilerop.py", line 55, in setUp
    nt.assert_equal(sys.getdefaultencoding(), "utf-8" if py3compat.PY3 else "ascii")
AssertionError: 'utf-8' != 'ascii'

======================================================================
ERROR: test suite for <module 'IPython.core.tests.test_history' from '/home/fperez/usr/lib/python2.7/site-packages/IPython/core/tests/test_history.pyc'>
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 208, in run
    self.setUp()
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 291, in setUp
    self.setupContext(ancestor)
  File "/usr/lib/python2.7/dist-packages/nose/suite.py", line 314, in setupContext
    try_run(context, names)
  File "/usr/lib/python2.7/dist-packages/nose/util.py", line 478, in try_run
    return func()
  File "/home/fperez/usr/lib/python2.7/site-packages/IPython/core/tests/test_history.py", line 26, in setUp
    nt.assert_equal(sys.getdefaultencoding(), "utf-8" if py3compat.PY3 else "ascii")
AssertionError: 'utf-8' != 'ascii'

Is anyone else seeing those? Any idea what to do with them?

I also know it needs a rebase, will try to finish that tomorrow. But if you could see how it goes in the meantime, that would be great.

@takluyver, you're our encoding/unicode guru; any thoughts on the errors above?

That's a sanity check we added - it fails if something is changing the default encoding, which is generally a bad idea. I think pygtk used to do that - could the tests be loading pygtk? Or have you set anything up, like in site.py, to call sys.setdefaultencoding()?

I have seen this before #2035. In that case it was caused by the test suite importing sympy which in turn does import matplotlib.pyplot fixing your backend to the default one. If that backend happens to be GTK this happens.

Mmh, that might be it... In core we do pull in pylab at some point, and I'm not seeing the error on a different machine, so that might be it. Thanks!

OK @minrk branch #3393 (bootstrap) has been merged. Once this has been rebased, we can start final tests of this PR.

On my desktop at work, the only error I'm seeing is:

======================================================================
ERROR: Test that namespace cleanup is not too aggressive GH-238
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/nose/case.py", line 197, in runTest
    self.test(*self.arg)
  File "/usr/lib/python2.7/dist-packages/numpy/testing/decorators.py", line 213, in knownfailer
    raise KnownFailureTest(msg)
KnownFailureTest: This test is known to fail

I don't really know why this is appearing as an error instead of K, since it's a proper KnownFailure. I'd never seen this problem, but that's a nose issue (I think) and I doubt anything specific to this PR.

Anyone seen this?

Didn't that happen at one point due to interference from a matplotlib known
failure plugin? I think it was you who tracked that down.

Ah, yes @takluyver, you're right! I'd totally forgotten, thanks. I now remember it was a royal PITA to hunt it down.

BTW, you were also right on the pygtk backend being the issue. I switched it to qt and the failures are gone. Maybe we should force that in our tests by hand to protect against this.

In any case, now the suite passes for me cleanly!

Hey folks, this rebase is looking pretty awful: I'm getting a huge amount of conflicts, partly coming from changes to the static html dirs, and partly from inside the JS files.

I need to crash now, and it may be that a safer solution is to cherry-pick all the commits that constitute the actual new work by hand, re-doing the moving around of submodules manually.

I might do that tomorrow if I can find the time for it, but help would be appreciated, as I'll be offline Weds/Thursday.

Other than getting through the rebase, I think the code itself in the PR is solid (obviously open to review, but there's actually not much real new code).

I just rebased this as #3443, if you want to take that.

Awesome I will test later this morning

Sent from my iPhone

On Jun 18, 2013, at 7:08 AM, Min RK [email protected] wrote:

I just rebased this as #3443, if you want to take that.

—
Reply to this email directly or view it on GitHub.

I've done a forced push after resetting to @minrk's nice rebase (thanks again!).

On the KnownFailure I was seeing, it appears to be caused by matplotlib from master. Back in #823 we struggled with that problem, though in that case the issue was in how mpl was packaged in Ubuntu, and the workaround was to install from source.

It appears that now the situation is the opposite: ubuntu's mpl is OK, but git master is playing this nasty nose trick somehow, breaking recognition of KnownFailure.

But since working with the system mpl is fine, I'll ignore the issue for us, and will follow up with matplotlib.

Once I revert back to the system mpl, everything looks good on my system;

**********************************************************************
Test suite completed for system with the following information:
{'commit_hash': 'e24836f',
 'commit_source': 'repository',
 'default_encoding': 'UTF-8',
 'ipython_path': '/home/fperez/usr/lib/python2.7/site-packages/IPython',
 'ipython_version': '1.0.dev',
 'os_name': 'posix',
 'platform': 'Linux-3.8.0-23-generic-x86_64-with-Ubuntu-13.04-raring',
 'sys_executable': '/usr/bin/python',
 'sys_platform': 'linux2',
 'sys_version': '2.7.4 (default, Apr 19 2013, 18:28:01) \n[GCC 4.7.3]'}

Tools and libraries available at test time:
   curses cython jinja2 matplotlib numpy oct2py pexpect pygments pymongo qt rpy2 sqlite3 tornado wx wx.aui zmq

Tools and libraries NOT available at test time:
   azure

Ran 12 test groups in 85.602s

Status:
OK

The Travis build is failing b/c it gets confused with the changes to submodles, so I suggest we ignore travis on this one and do the testing ourselves.

As far as I'm concerned, this is ready to review/merge. I can do the merge today, but after today I'll have near-zero bandwidth until Friday, and then it's not clear how much I'll have. So if you folks can do a bit of testing today, that would be awesome. That way we get this flushed and don't block any other work.

The travis error is a real bug, I've fixed it in a PR against your branch.

I also found a few frontends hanging around in setup, gitignore, and MANIFEST.in, fixed in fperez#2

A few minor tweaks, and this should pass travis after merging my PR.

Ahh, but I am confused. I thought the plan was not only to flatten frontend, but to also flatten html/notebook into just html. Did that plan change?

I should mention that I still think we should do that. We could do that in a separate PR, but I don't think that makes sense...

yes, I forgot about that. We should do it here.

Hey guys, I just got home now and have to be up early again for another day of meetings. So I suggest that you close this one and pick it up on a team branch you can also write to from here.

Honestly, I just forgot about the flattening of html/notebook, sorry about that! It never registered in my brain; in fact I was thinking we should do that too and wanted to ask if you had anything against the idea :)

While I prefer to keep things in one PR so the discussion is more readable, given my travel/schedule constraints this week, my vote is to close this one so you can finish it off in a different branch.