Add multiline coding in ipython 5 with --simple-prompt

As emacs user I often use emacs repl with ipython, but after update to ipython 5 I need to use --simple-prompt flag for my work. But when I try to create new def or class I get this error message

In [1]: def new():
  File "<ipython-input-1-ce93a60072ff>", line 1
    def new():
              ^
SyntaxError: unexpected EOF while parsing

@tacaswell

The simple-prompt option doesn't support multi-line input, but I thought Emacs 'inferior-shell' magic was working round that. @tacaswell ?

In ansi-term ipython work well, but I need REPL. Standart python interactive shell works fine, but I really miss %magic%

FYI: prompt_toolkit 1.0.4 has been released today, it has one improvement that should make prompt_toolkit render at least slightly better in emacs-term. Can you have a try?

@tacaswell

In the context of IPython, that means try without --simple-prompt, so that it uses prompt_toolkit.

@tacaswell might also be interested in testing.

Hello, I have updated to prompt_toolkit 1.0.4 with emacs 25.0.95.1 and ipython 5.0.0. For some reason, running just ipython defaults to the simple-prompt and I am having trouble testing with prompt_toolkit - which is weird and I cannot figure out why.

However, running jupyter console still returns the same error it was returning before updating prompt_toolkit. Not sure if this helps or just confuses things further.

Here is the traceback

Traceback (most recent call last):
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\Scripts\jupyter-console-script.py", line 5, in <module>
    sys.exit(jupyter_console.app.main())
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\jupyter_core\application.py", line 267, in launch_instance
    return super(JupyterApp, cls).launch_instance(argv=argv, **kwargs)
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\traitlets\config\application.py", line 595, in launch_instance
    app.initialize(argv)
  File "<decorator-gen-114>", line 2, in initialize
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\traitlets\config\application.py", line 74, in catch_config_error
    return method(app, *args, **kwargs)
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\jupyter_console\app.py", line 137, in initialize
    self.init_shell()
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\jupyter_console\app.py", line 110, in init_shell
    client=self.kernel_client,
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\traitlets\config\configurable.py", line 412, in instance
    inst = cls(*args, **kwargs)
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\jupyter_console\ptshell.py", line 252, in __init__
    self.init_prompt_toolkit_cli()
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\jupyter_console\ptshell.py", line 404, in init_prompt_toolkit_cli
    self.pt_cli = CommandLineInterface(app, eventloop=self._eventloop)
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\prompt_toolkit\interface.py", line 80, in __init__
    self.output = output or create_output()
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\prompt_toolkit\shortcuts.py", line 114, in create_output
    return Win32Output(stdout)
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\prompt_toolkit\terminal\win32_output.py", line 80, in __init__
    info = self.get_win32_screen_buffer_info()
  File "C:\Users\A6419643\AppData\Local\Continuum\Miniconda3\lib\site-packages\prompt_toolkit\terminal\win32_output.py", line 150, in get_win32_screen_buffer_info
    raise NoConsoleScreenBufferError
prompt_toolkit.terminal.win32_output.NoConsoleScreenBufferError: No Windows console found. Are you running cmd.exe?

Process Python exited abnormally with code 1

@jonathanslenders

I'd guess that maybe our checks for a terminal fail, so it falls back to simple-prompt automatically.

@jonathanslenders - I take it that traceback is not what you were expecting! Is there anything else the Windows+Emacs users should try?

Any progress on this issue?

Sorry, I missed the ping on this in July.

In comint mode you only get one line, but you can use one of the send-buffer-to-shell commands in python-mode / elpy to send a buffer to the interpreter. That plus an org-mode source region or just a /tmp/scratch.py file is how I do multi-line editing these days.

This is pretty huge for us Emacs users. :(

Why does it work on iPython 4.x? What changed exactly?

@AndreaOrru

@AndreaOrru Pretty sure the change is moving from readline to prompt toolkit which lots some of the special-cased work arounds on both sides.

I strongly suggest opening /tmp/scratch.py, writing your multi-line stuff there (or using org-mode + source blocks), and shipping it to the shell via send-buffer-to-shell (bound to C-c C-c by elpy).

Hi guys,

I have the following problem using Emacs25 with ipython 5, as it uses some strange colours. I have this as walk-around

(setq python-shell-interpreter "ipython" python-shell-interpreter-args "--simple-prompt -i")

but then I get this error as in reported bug

SyntaxError: unexpected EOF while parsing

whenever I try to send multiline code ending with ':' character. Sending whole buffer works just fine but individual lines are problematic, which makes testing code very difficult.

Please help.

@mashu

hi @mashu,

the work around you use above makes the ipython prompt work in emacs, but it does not allow multiline editing. There is currently no solution that allows multiline editing in an ipython shell in emacs.

If you want multiline editing, I would follow @tacaswell's advice above using python buffers, etc.

So why you couldn`t have both readline and promt_toolkit and switching them using some flags? Even if you won`t support readline you can at least leave it as it is. This is huge regression for many users and I don`t think promt_toolkit is much better then readline. It isn`t work with ipdb also.

@ajsteven130

@ajsteven130 Thanks for great advice
@ALL Using python instead of ipython5 with emacs seems like a better solution for now.

What works for me is that I can copy now multiple lines into the python buffer and they get executed.
That wasn't the case when using ipython. However, line-by-line sending does not work in Emacs(elpy) at the moment of writing I encounter this bug #907 and suggested alternative C-M-x does not work as expected as well. Kind of stucked now with copying blocks of code, but better than nothing.

iPython 5.1 in Windows (7) Emacs 25.1.2 run-python.

The trick to use an elpy buffer to send to the *Python* buffer works well - thanks.

None of the %magic% functions work and, in particular, I'd like %recall. Backed up briefly to iPython 4.2.1 however this put me back to no Output at all so that's not an option.

What programming changes would be involved to make %recall work?

@ivanov

I'm curious if @ivanov might have some ideas on the vim front (I know the Emacs issues are separate)?

nope, no clue, unfortunately. I still have an old IPython installed when I need to get a lot of interactive work done because it respects .inputrc and I haven't made the time to make prompt_toolkit act the ways readline does. The functionality provided by --TerminalInteractiveShell.editing_mode=vi is inconsistent enough with muscle memory that it makes it more frustrating than just using the default mode, so I stick with that when I'm in IPython 5 these days.

Ipython6 is released, is this issue fixed?

I upgraded to Ipython5 and downgraded to Ipython4 with same Emacs issue. If everything works same, I could just stick with Ipython4 but something is changed and work weird now.
When I define simple function and execute it twice in ipython, it shows like below:

Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
Type "copyright", "credits" or "license" for more information.

IPython 4.2.1 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: def temp():
   ....: return 0
   ....: 

In [12]: temp()
%recall;reduce;%rehashx;reload;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed
%recall;reduce;%rehashx;reload;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed
%recall;reduce;%rehashx;reload;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed
raise;range;raw_input;%recall;reduce;%rehashx;reload;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed;%rm;%rmdir;round;%%ruby;%run
return
return
return
raise;range;raw_input;%recall;reduce;%rehashx;reload;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed;%rm;%rmdir;round;%%ruby;%run
return
return
return
raise;range;raw_input;%recall;reduce;%rehashx;reload;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed;%rm;%rmdir;round;%%ruby;%run
Out[16]: 0

In [17]: temp()
---------------------------------------------------------------------------
IOError                                   Traceback (most recent call last)
<ipython-input-21-9a3c6ba04cf8> in <module>()
----> 1 temp()

<ipython-input-11-059b6888777a> in temp()
      1 def temp():
----> 2     import codecs, os;__pyfile = codecs.open('''/tmp/py42419kY''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py42419kY''');exec(compile(__code, '''/tmp/py42419kY''', 'exec'));
      3     import codecs, os;__pyfile = codecs.open('''/tmp/py4241Kve''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py4241Kve''');exec(compile(__code, '''/tmp/py4241Kve''', 'exec'));
      4     import codecs, os;__pyfile = codecs.open('''/tmp/py4241X5k''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py4241X5k''');exec(compile(__code, '''/tmp/py4241X5k''', 'exec'));
      5     import codecs, os;__pyfile = codecs.open('''/tmp/py4241kDr''', encoding='''utf-8''');__code = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py4241kDr''');exec(compile(__code, '''/tmp/py4241kDr''', 'exec'));

/usr/lib/python2.7/codecs.pyc in open(filename, mode, encoding, errors, buffering)
    894             # Force opening of the file in binary mode
    895             mode = mode + 'b'
--> 896     file = __builtin__.open(filename, mode, buffering)
    897     if encoding is None:
    898         return file

IOError: [Errno 2] No such file or directory: '/tmp/py49784HG'

It works fine if I execute ipython in terminal.

I just want to mention that if you would like the old (IPython <= 4.x) readline interface, it is now available separately as rlipython.

pip install rlipython
python -c "import rlipython as r; r.install()"

Right now, it only works in IPython 6.0 (Python 3), but it will also work in Python 2 with the next IPython 5 release (5.4).

I suspect, but have not verified, that the style of interaction with emacs that used to work in IPython <= 4 should work again with rlipython. If not, file an issue there and I can take a look.

@ivanov

Thanks for your suggestion @ivanov but unfortunately I have a same issue even after rlipython installed.
Error messages are shown as below:

Python 3.5.2 (default, Nov 17 2016, 17:05:23)
Type 'copyright', 'credits' or 'license' for more information
IPython 6.0.0 -- An enhanced Interactive Python. Type '?' for help.

In [1]: def temp():
   ...: return 0
   ...: 

In [9]: temp()
%recall;%rehashx;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed
%recall;%rehashx;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed
%recall;%rehashx;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed
raise;range;%recall;%rehashx;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed;%rm;%r\
mdir;round;%%ruby;%run
return
return
return
raise;range;%recall;%rehashx;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed;%rm;%r\
mdir;round;%%ruby;%run
return
return
return
raise;range;%recall;%rehashx;%reload_ext;%rep;repr;%rerun;%reset;%reset_selective;return;reversed;%rm;%r\
mdir;round;%%ruby;%run
Out[12]: 0

In [13]: temp()
---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-17-9a3c6ba04cf8> in <module>()
----> 1 temp()

<ipython-input-8-3cf4da01de46> in temp()
      1 def temp():
----> 2     import codecs, os;__pyfile = codecs.open('''/tmp/py30478b2j''', encoding='''utf-8''');__code\
 = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py30478b2j''');exec(compile(__\
code, '''/tmp/py30478b2j''', 'exec'));
      3     import codecs, os;__pyfile = codecs.open('''/tmp/py30478oAq''', encoding='''utf-8''');__code\
 = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py30478oAq''');exec(compile(__\
code, '''/tmp/py30478oAq''', 'exec'));
      4     import codecs, os;__pyfile = codecs.open('''/tmp/py304781Kw''', encoding='''utf-8''');__code\
 = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py304781Kw''');exec(compile(__\
code, '''/tmp/py304781Kw''', 'exec'));
      5     import codecs, os;__pyfile = codecs.open('''/tmp/py30478CV2''', encoding='''utf-8''');__code\
 = __pyfile.read().encode('''utf-8''');__pyfile.close();os.remove('''/tmp/py30478CV2''');exec(compile(__\
code, '''/tmp/py30478CV2''', 'exec'));

/usr/lib/python3.5/codecs.py in open(filename, mode, encoding, errors, buffering)
    893         # Force opening of the file in binary mode
    894         mode = mode + 'b'
--> 895     file = builtins.open(filename, mode, buffering)
    896     if encoding is None:
    897         return file

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/py30478b2j'

In [18]:

Looks like something is broken on the process to upgrade and downgrade ipython.
I found simple-prompt option does not support mulitline coding too late, so I had to downgrade ipython on my home PC, company PC and server PC. Now all ipython from those computers shows same error.
I hope anyhow this issue will be solved soon. Thanks a lot.

In case of my emacs + ipython error, It is solved when I try python-mode package instead of built-in emacs python package. It works as usual when I start ipython using python-mode package.(w/ Ipython 4.2.1 version) I hope this would be helpful for anybody having same problem with me.

Thanks again for all contributing to ipython.

+) One more update, my error case has to do with company package. It works clearly if I start ipython with global-company-mode disabled.

This is preventing the cantor sage backend from working.

You most likely want to have a look at rlipython, it allows to re-use readline as a backend and should work way better with emacs.

Force ipython3 REPL.

@ivanov

@ivanov @Carreau I've been using --simple-prompt which currently allows some kind of primitive multiline editing in emacs although I've always missed the superior pt capabilities since I started running my repls inside emacs. So I googled for alternatives and ended up here. After testing rlipython I'm unable to find any relevant differences vs plain --simple-prompt, at least for multiline editing. What would be rlipython selling points for emacs users? Do you know?

@memeplex

@memeplex - does the emacs repl allow you to use up arrows to cycle in history? --simple-prompt does not support that, but that's the only thing that comes immediately to mind. I guess the lack of that feature also implies that you do not get history across sessions.

Up/down arrows work for moving between lines in a multiline input but not for history cycling.

Navigation Menu

Repository navigation

Add multiline coding in ipython 5 with --simple-prompt #9816

17 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Add multiline coding in ipython 5 with --simple-prompt #9816

Description

Activity

takluyver commented on Jul 28, 2016

Crandel commented on Jul 28, 2016

jonathanslenders commented on Aug 3, 2016

takluyver commented on Aug 3, 2016

ajsteven130 commented on Aug 4, 2016

takluyver commented on Aug 4, 2016

osdf commented on Aug 29, 2016

tacaswell commented on Aug 29, 2016

AndreaOrru commented on Oct 16, 2016

tacaswell commented on Oct 16, 2016

mashu commented on Oct 26, 2016

ajsteven130 commented on Oct 26, 2016

Crandel commented on Oct 26, 2016

mashu commented on Oct 26, 2016

17 remaining items

patfla commented on Jan 26, 2017

fperez commented on Feb 21, 2017

ivanov commented on Feb 26, 2017

ramsayleung commented on Apr 26, 2017

tttuuu888 commented on May 8, 2017

ivanov commented on May 17, 2017

tttuuu888 commented on May 18, 2017

tttuuu888 commented on May 18, 2017

infinity0 commented on May 22, 2017

Carreau commented on May 25, 2017

memeplex commented on Jul 30, 2018

ivanov commented on Jul 30, 2018

memeplex commented on Jul 30, 2018

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions