Scribd
Upload
9 views6 pages

Asyncio Subprocess

Uploaded by

vijayarajunnikrishnan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
9 views6 pages

Asyncio Subprocess

Uploaded by

vijayarajunnikrishnan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as TXT, PDF, TXT or read online on Scribd
You are on page 1/ 6

Subprocesses

************

**Source code:** Lib/asyncio/subprocess.py,


Lib/asyncio/base_subprocess.py

======================================================================

This section describes high-level async/await asyncio APIs to create


and manage subprocesses.

Here's an example of how asyncio can run a shell command and obtain
its result:

import asyncio

async def run(cmd):


proc = await asyncio.create_subprocess_shell(
cmd,
stdout=asyncio.subprocess.PIPE,
stderr=asyncio.subprocess.PIPE)

stdout, stderr = await proc.communicate()

print(f'[{cmd!r} exited with {proc.returncode}]')


if stdout:
print(f'[stdout]\n{stdout.decode()}')
if stderr:
print(f'[stderr]\n{stderr.decode()}')

asyncio.run(run('ls /zzz'))

will print:

['ls /zzz' exited with 1]


[stderr]
ls: /zzz: No such file or directory

Because all asyncio subprocess functions are asynchronous and asyncio


provides many tools to work with such functions, it is easy to execute
and monitor multiple subprocesses in parallel. It is indeed trivial
to modify the above example to run several commands simultaneously:

async def main():


await asyncio.gather(
run('ls /zzz'),
run('sleep 1; echo "hello"'))

asyncio.run(main())

See also the Examples subsection.

Creating Subprocesses
=====================

async asyncio.create_subprocess_exec(program, *args, stdin=None, stdout=None,


stderr=None, limit=None, **kwds)
Create a subprocess.

The *limit* argument sets the buffer limit for "StreamReader"


wrappers for "stdout" and "stderr" (if "subprocess.PIPE" is passed
to *stdout* and *stderr* arguments).

Return a "Process" instance.

See the documentation of "loop.subprocess_exec()" for other


parameters.

Changed in version 3.10: Removed the *loop* parameter.

async asyncio.create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None,


limit=None, **kwds)

Run the *cmd* shell command.

The *limit* argument sets the buffer limit for "StreamReader"


wrappers for "stdout" and "stderr" (if "subprocess.PIPE" is passed
to *stdout* and *stderr* arguments).

Return a "Process" instance.

See the documentation of "loop.subprocess_shell()" for other


parameters.

Important:

It is the application's responsibility to ensure that all


whitespace and special characters are quoted appropriately to
avoid shell injection vulnerabilities. The "shlex.quote()"
function can be used to properly escape whitespace and special
shell characters in strings that are going to be used to
construct shell commands.

Changed in version 3.10: Removed the *loop* parameter.

Note:

Subprocesses are available for Windows if a "ProactorEventLoop" is


used. See Subprocess Support on Windows for details.

See also:

asyncio also has the following *low-level* APIs to work with


subprocesses: "loop.subprocess_exec()", "loop.subprocess_shell()",
"loop.connect_read_pipe()", "loop.connect_write_pipe()", as well as
the Subprocess Transports and Subprocess Protocols.

Constants
=========

asyncio.subprocess.PIPE

Can be passed to the *stdin*, *stdout* or *stderr* parameters.

If *PIPE* is passed to *stdin* argument, the "Process.stdin"


attribute will point to a "StreamWriter" instance.

If *PIPE* is passed to *stdout* or *stderr* arguments, the


"Process.stdout" and "Process.stderr" attributes will point to
"StreamReader" instances.

asyncio.subprocess.STDOUT

Special value that can be used as the *stderr* argument and


indicates that standard error should be redirected into standard
output.

asyncio.subprocess.DEVNULL

Special value that can be used as the *stdin*, *stdout* or *stderr*


argument to process creation functions. It indicates that the
special file "os.devnull" will be used for the corresponding
subprocess stream.

Interacting with Subprocesses


=============================

Both "create_subprocess_exec()" and "create_subprocess_shell()"


functions return instances of the *Process* class. *Process* is a
high-level wrapper that allows communicating with subprocesses and
watching for their completion.

class asyncio.subprocess.Process

An object that wraps OS processes created by the


"create_subprocess_exec()" and "create_subprocess_shell()"
functions.

This class is designed to have a similar API to the


"subprocess.Popen" class, but there are some notable differences:

* unlike Popen, Process instances do not have an equivalent to the


"poll()" method;

* the "communicate()" and "wait()" methods don't have a *timeout*


parameter: use the "wait_for()" function;

* the "Process.wait()" method is asynchronous, whereas


"subprocess.Popen.wait()" method is implemented as a blocking
busy loop;

* the *universal_newlines* parameter is not supported.

This class is not thread safe.

See also the Subprocess and Threads section.

async wait()

Wait for the child process to terminate.

Set and return the "returncode" attribute.


Note:

This method can deadlock when using "stdout=PIPE" or


"stderr=PIPE" and the child process generates so much output
that it blocks waiting for the OS pipe buffer to accept more
data. Use the "communicate()" method when using pipes to avoid
this condition.

async communicate(input=None)

Interact with process:

1. send data to *stdin* (if *input* is not "None");

2. closes *stdin*;

3. read data from *stdout* and *stderr*, until EOF is reached;

4. wait for process to terminate.

The optional *input* argument is the data ("bytes" object) that


will be sent to the child process.

Return a tuple "(stdout_data, stderr_data)".

If either "BrokenPipeError" or "ConnectionResetError" exception


is raised when writing *input* into *stdin*, the exception is
ignored. This condition occurs when the process exits before
all data are written into *stdin*.

If it is desired to send data to the process' *stdin*, the


process needs to be created with "stdin=PIPE". Similarly, to
get anything other than "None" in the result tuple, the process
has to be created with "stdout=PIPE" and/or "stderr=PIPE"
arguments.

Note, that the data read is buffered in memory, so do not use


this method if the data size is large or unlimited.

Changed in version 3.12: *stdin* gets closed when "input=None"


too.

send_signal(signal)

Sends the signal *signal* to the child process.

Note:

On Windows, "SIGTERM" is an alias for "terminate()".


"CTRL_C_EVENT" and "CTRL_BREAK_EVENT" can be sent to processes
started with a *creationflags* parameter which includes
"CREATE_NEW_PROCESS_GROUP".

terminate()

Stop the child process.

On POSIX systems this method sends "SIGTERM" to the child


process.
On Windows the Win32 API function "TerminateProcess()" is called
to stop the child process.

kill()

Kill the child process.

On POSIX systems this method sends "SIGKILL" to the child


process.

On Windows this method is an alias for "terminate()".

stdin

Standard input stream ("StreamWriter") or "None" if the process


was created with "stdin=None".

stdout

Standard output stream ("StreamReader") or "None" if the process


was created with "stdout=None".

stderr

Standard error stream ("StreamReader") or "None" if the process


was created with "stderr=None".

Warning:

Use the "communicate()" method rather than


"process.stdin.write()", "await process.stdout.read()" or "await
process.stderr.read()". This avoids deadlocks due to streams
pausing reading or writing and blocking the child process.

pid

Process identification number (PID).

Note that for processes created by the


"create_subprocess_shell()" function, this attribute is the PID
of the spawned shell.

returncode

Return code of the process when it exits.

A "None" value indicates that the process has not terminated


yet.

A negative value "-N" indicates that the child was terminated by


signal "N" (POSIX only).

Subprocess and Threads


----------------------

Standard asyncio event loop supports running subprocesses from


different threads by default.
On Windows subprocesses are provided by "ProactorEventLoop" only
(default), "SelectorEventLoop" has no subprocess support.

Note that alternative event loop implementations might have own


limitations; please refer to their documentation.

See also: The Concurrency and multithreading in asyncio section.

Examples
--------

An example using the "Process" class to control a subprocess and the


"StreamReader" class to read from its standard output.

The subprocess is created by the "create_subprocess_exec()" function:

import asyncio
import sys

async def get_date():


code = 'import datetime; print(datetime.datetime.now())'

# Create the subprocess; redirect the standard output


# into a pipe.
proc = await asyncio.create_subprocess_exec(
sys.executable, '-c', code,
stdout=asyncio.subprocess.PIPE)

# Read one line of output.


data = await proc.stdout.readline()
line = data.decode('ascii').rstrip()

# Wait for the subprocess exit.


await proc.wait()
return line

date = asyncio.run(get_date())
print(f"Current date: {date}")

See also the same example written using low-level APIs.

You might also like