Skip to content

Commit 50c808c

Browse files
authored
Merge pull request astropy#416 from astrofrog/simple-docs
Added initial docs about what the package does
2 parents 9f5f3f4 + 427f56e commit 50c808c

2 files changed

Lines changed: 328 additions & 22 deletions

File tree

‎CHANGES.rst‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,9 @@ astropy-helpers Changelog
99
3.1 (unreleased)
1010
----------------
1111

12-
- Fixed the compability of the build_docs command with Sphinx 1.8 and above. [#413]
12+
- Added extensive documentation about astropy-helpers to the README.rst file. [#416]
13+
14+
- Fixed the compatibility of the build_docs command with Sphinx 1.8 and above. [#413]
1315

1416
- Removing deprecated test_helpers.py file. [#369]
1517

‎README.rst‎

Lines changed: 325 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,24 @@
11
astropy-helpers
22
===============
33

4-
* Stable versions: https://pypi.org/project/astropy-helpers/
5-
* Development version, issue tracker: https://github.com/astropy/astropy-helpers
4+
.. image:: https://travis-ci.org/astropy/astropy-helpers.svg
5+
:target: https://travis-ci.org/astropy/astropy-helpers
6+
7+
.. image:: https://ci.appveyor.com/api/projects/status/rt9161t9mhx02xp7/branch/master?svg=true
8+
:target: https://ci.appveyor.com/project/Astropy/astropy-helpers
9+
10+
.. image:: https://codecov.io/gh/astropy/astropy-helpers/branch/master/graph/badge.svg
11+
:target: https://codecov.io/gh/astropy/astropy-helpers
612

7-
.. warning::
13+
**Warning:** Please note that version ``v3.0`` and later of ``astropy-helpers``
14+
requires Python 3.5 or later. If you wish to maintain Python 2 support
15+
for your package that uses ``astropy-helpers``, then do not upgrade the
16+
helpers to ``v3.0+``. We will still provide Python 2.7 compatible
17+
releases on the ``v2.0.x`` branch during the lifetime of the ``astropy``
18+
core package LTS of ``v2.0.x``.
819

9-
Please note that version ``v3.0`` and later of ``astropy-helpers`` does
10-
require Python 3.5 or later. If you wish to maintain Python 2 support
11-
for your package that uses ``astropy-helpers``, then do not upgrade the
12-
helpers to ``v3.0+``. We will still provide Python 2.7 compatible
13-
releases on the ``v2.0.x`` branch during the lifetime of the ``astropy``
14-
core package LTS of ``v2.0.x``.
20+
About
21+
-----
1522

1623
This project provides a Python package, ``astropy_helpers``, which includes
1724
many build, installation, and documentation-related tools used by the Astropy
@@ -23,22 +30,319 @@ implementation are in the accepted
2330
``astropy_helpers`` includes a special "bootstrap" module called
2431
``ah_bootstrap.py`` which is intended to be used by a project's setup.py in
2532
order to ensure that the ``astropy_helpers`` package is available for
26-
build/installation. This is similar to the ``ez_setup.py`` module that is
27-
shipped with some projects to bootstrap `setuptools
28-
<https://bitbucket.org/pypa/setuptools>`_.
33+
build/installation.
2934

30-
As described in APE4, the version numbers for ``astropy_helpers`` follow the
31-
corresponding major/minor version of the `astropy core package
32-
<http://www.astropy.org/>`_, but with an independent sequence of micro (bugfix)
33-
version numbers. Hence, the initial release is 0.4, in parallel with Astropy
34-
v0.4, which will be the first version of Astropy to use ``astropy-helpers``.
35+
As described in `APE4 <https://github.com/astropy/astropy-APEs/blob/master/APE4.rst>`_, the version
36+
numbers for ``astropy_helpers`` follow the corresponding major/minor version of
37+
the `astropy core package <http://www.astropy.org/>`_, but with an independent
38+
sequence of micro (bugfix) version numbers. Hence, the initial release is 0.4,
39+
in parallel with Astropy v0.4, which will be the first version of Astropy to
40+
use ``astropy-helpers``.
3541

3642
For examples of how to implement ``astropy-helpers`` in a project,
3743
see the ``setup.py`` and ``setup.cfg`` files of the
3844
`Affiliated package template <https://github.com/astropy/package-template>`_.
3945

40-
.. image:: https://travis-ci.org/astropy/astropy-helpers.svg
41-
:target: https://travis-ci.org/astropy/astropy-helpers
46+
What does astropy-helpers provide?
47+
----------------------------------
48+
49+
Astropy-helpers' big-picture purpose is to provide customization to Python's
50+
packaging infrastructure process in ways that the Astropy Project has found to
51+
help simplifying the developint and releasing packages. This is primarily
52+
build around ``setup.py`` commands, as outlined below, as well as code to help
53+
manage version numbers and better control the build process of larger packages.
54+
55+
Custom setup.py commands
56+
^^^^^^^^^^^^^^^^^^^^^^^^
57+
58+
The main part of astropy-helpers is to provide customized setuptools commands.
59+
For example, in a package that uses astropy-helpers, the following command
60+
will be available::
61+
62+
python setup.py build_docs
63+
64+
and this command is implemented in astropy-helpers. To use the custom
65+
commands described here, add::
66+
67+
from astropy_helpers.setup_helpers import register_commands
68+
69+
to your ``setup.py`` file, then do::
70+
71+
cmdclassd = register_commands(NAME, VERSION, RELEASE)
72+
73+
where ``NAME`` is the name of your package, ``VERSION`` is the full version
74+
string, and ``RELEASE`` is a boolean value indicating whether the version is
75+
a stable released version (``True``) or a developer version (``False``).
76+
Finally, pass ``cmdclassd`` to the ``setup`` function::
77+
78+
setup(...,
79+
cmdclass=cmdclassd)
80+
81+
The commands we provide or customize are:
82+
83+
**python setup.py test**
84+
85+
This command will automatically build the package, install it to a temporary
86+
directory, and run the tests using `pytest <http://pytest.org/>`_ on this
87+
installed version. Note that the bulk of this command is actually defined
88+
in ``astropy.tests.command.AstropyTest``, because that allows the test
89+
machinery to operate outside a setuptools command. This, here we
90+
simply define the custom
91+
setuptools command.
92+
93+
**python setup.py sdist**
94+
95+
We redefine ``sdist`` to use the version from distutils rather than from
96+
setuptools, as the setuptools version requires duplication of information
97+
in ``MANIFEST.in``.
98+
99+
**python setup.py build_docs**
100+
101+
This command will automatically build the package, then run sphinx to build
102+
the documentation. This makes development much easier because it ensures
103+
sphinx extensions that use the package's code to make documentation are
104+
actually using the in-development version of the code. Sphinx itself
105+
provides a custom setuptools command, which we
106+
expand with the following options:
107+
108+
* ``-w``: set the return code to 1 if there are any warnings during the build
109+
process.
110+
111+
* ``-l``: completely clean previous builds, including files generated by
112+
the sphinx-automodapi package (which creates API pages for different
113+
functions/classes).
114+
115+
* ``-n``: disable the intersphinx option.
116+
117+
* ``-o``: open the documentation in a browser if a build finishes successfully.
118+
119+
In addition, ``build_docs`` will automatically download and temporarily install
120+
sphinx-astropy (which is a meta-package that
121+
provides standardized configuration and documentation dependencies for astropy
122+
packages) if it isn't already installed. Temporary installation means that the
123+
package will be installed into an ``.eggs`` directory in the current working
124+
directory, and it will only be available for the duration of the call to
125+
``build_docs``.
126+
127+
**python setup.py build_ext**
128+
129+
This is also used when running ``build`` or ``install``. We add several features
130+
compared to the default ``build_ext`` command:
131+
132+
* For packages with C/Cython extensions, we create a ``packagename._compiler``
133+
submodule that contains information about the compilers used.
134+
135+
* Packages that need to build C extensions using the Numpy C API, we allow
136+
those packages to define the include path as ``'numpy'`` as opposed to having
137+
to import Numpy and call ``get_include``. The goal is to solve the issue that
138+
if one has to import Numpy to define extensions, then Numpy has to be
139+
installed/available before the package is installed, which means that one
140+
needs to install Numpy in a separate installation step.
141+
142+
* We detect broken compilers and replace them with other compilers on-the-fly
143+
unless the compiler is explicitly specified with the ``CC`` environment
144+
variable.
145+
146+
* If Cython is not installed, then we automatically check for generated C files
147+
(which are normally present in the stable releases) and give a nice error
148+
if these are not found.
149+
150+
Version helpers
151+
^^^^^^^^^^^^^^^^
152+
153+
Another piece of functionality we provide in astropy-helpers is the ability
154+
to generate a ``packagename.version`` file that includes functions that
155+
automatically set the version string for developer versions, to e.g.
156+
``3.2.dev22213`` so that each developer version has a unique number (although
157+
note that branches an equal number of commits away from the master branch will
158+
share the same version number). To use this, import::
159+
160+
from astropy_helpers.git_helpers import get_git_devstr
161+
162+
in your ``setup.py`` file, and you will then be able to use::
163+
164+
VERSION += get_git_devstr()
165+
166+
where ``VERSION`` is a version string without any developer version suffix.
167+
168+
We then also provide a function that generates a ``version.py`` file inside your
169+
package (which can then be imported as ``packagename.version``) that contains
170+
variables such as ``major``, ``minor``, and ``bugfix``, as well as
171+
``version_info`` (a tuple of the previous three values), a ``release`` flag that
172+
indicates whether we are using a stable release, and several other complementary
173+
variables. To use this, import::
174+
175+
from astropy_helpers.version_helpers import generate_version_py
176+
177+
in your ``setup.py`` file, and call::
178+
179+
generate_version_py(NAME, VERSION, RELEASE, uses_git=not RELEASE)
180+
181+
where ``NAME`` is the name of your package, ``VERSION`` is the full version string
182+
(including any developer suffix), ``RELEASE`` indicates whether the version is a
183+
stable or developer version, and ``uses_git`` indicates whether we are in a git
184+
repository (using ``not RELEASE`` is sensible since git is not available in a
185+
stable release).
186+
187+
Collecting package information
188+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
189+
190+
The ``setup`` function from setuptools can take a number of options that indicate
191+
for example what extensions to build, and what package data to include. However,
192+
for large packages this can become cumbersome. We therefore provide a mechanism
193+
for defining extensions and package data inside individual sub-packages. To do
194+
this, you can create ``setup_package.py`` files anywhere in your package, and
195+
these files can include one or more of the following functions:
196+
197+
* ``get_package_data``:
198+
This function, if defined, should return a dictionary mapping the name of
199+
the subpackage(s) that need package data to a list of data file paths
200+
(possibly including wildcards) relative to the path of the package's source
201+
code. e.g. if the source distribution has a needed data file
202+
``astropy/wcs/tests/data/3d_cd.hdr``, this function should return
203+
``{'astropy.wcs.tests':['data/3d_cd.hdr']}``. See the ``package_data``
204+
option of the :func:`distutils.core.setup` function.
205+
206+
It is recommended that all such data be in a directory named ``data`` inside
207+
the package within which it is supposed to be used. This package data
208+
should be accessed via the ``astropy.utils.data.get_pkg_data_filename`` and
209+
``astropy.utils.data.get_pkg_data_fileobj`` functions.
210+
211+
* ``get_extensions``:
212+
This provides information for building C or Cython extensions. If defined,
213+
it should return a list of ``distutils.core.Extension`` objects.
214+
215+
* ``get_build_options``:
216+
This function allows a package to add extra build options. It
217+
should return a list of tuples, where each element has:
218+
219+
- *name*: The name of the option as it would appear on the
220+
commandline or in the ``setup.cfg`` file.
221+
222+
- *doc*: A short doc string for the option, displayed by
223+
``setup.py build --help``.
224+
225+
- *is_bool* (optional): When `True`, the option is a boolean
226+
option and doesn't have an associated value.
227+
228+
Once an option has been added, its value can be looked up using
229+
``astropy_helpers.setup_helpers.get_distutils_build_option``.
230+
231+
* ``get_external_libraries``:
232+
This function declares that the package uses libraries that are
233+
included in the astropy distribution that may also be distributed
234+
elsewhere on the users system. It should return a list of library
235+
names. For each library, a new build option is created,
236+
``'--use-system-X'`` which allows the user to request to use the
237+
system's copy of the library. The package would typically call
238+
``astropy_helpers.setup_helpers.use_system_library`` from its
239+
``get_extensions`` function to determine if the package should use
240+
the system library or the included one.
241+
242+
* ``get_entry_points()``:
243+
This function can returns a dict formatted as required by
244+
the ``entry_points`` argument to ``setup()``.
245+
246+
With these files in place, you can then add the following to your ``setup.py``
247+
file::
248+
249+
from astropy_helpers.setup_helpers import get_package_info
250+
251+
...
252+
253+
package_info = get_package_info()
254+
255+
...
256+
257+
setup(..., **package_info)
258+
259+
OpenMP helpers
260+
^^^^^^^^^^^^^^
261+
262+
We provide a helper function ``add_openmp_flags_if_available`` that can be used
263+
to automatically add OpenMP flags for C/Cython extensions, based on whether
264+
OpenMP is available and produces executable code. To use this, edit the
265+
``setup_package.py`` file where you define a C extension, import the helper
266+
function::
267+
268+
from astropy_helpers.openmp_helpers import add_openmp_flags_if_available
269+
270+
then once you have defined the extension and before returning it, use it as::
271+
272+
extension = Extension(...)
273+
274+
add_openmp_flags_if_available(extension)
275+
276+
return [extension]
277+
278+
Using astropy-helpers
279+
---------------------
280+
281+
The easiest way to get set up with astropy-helpers in a new package is to use
282+
the `package-template <http://docs.astropy.org/projects/package-template>`_
283+
that we provide. This template is specifically designed for use with the helpers,
284+
so using it avoids some of the tedium of setting up the heleprs.
285+
286+
However, we now go through the steps of adding astropy-helpers
287+
as a submodule to a package in case you wish to do so manually. First, add
288+
astropy-helpers as a submodule at the root of your repository::
289+
290+
git submodule add git://github.com/astropy/astropy-helpers astropy_helpers
291+
292+
Then go inside the submodule and check out a stable version of astropy-helpers.
293+
You can see the available versions by running::
294+
295+
$ cd astropy_helpers
296+
$ git tag
297+
...
298+
v2.0.6
299+
v2.0.7
300+
...
301+
v3.0.1
302+
v3.0.2
303+
304+
If you want to support Python 2, pick the latest v2.0.x version (in the above
305+
case ``v2.0.7``) and if you don't need to support Python 2, just pick the latest
306+
stable version (in the above case ``v3.0.2``). Check out this version with e.g.::
307+
308+
$ git checkout v3.0.2
309+
310+
Then go back up to the root of your repository and copy the ``ah_bootstrap.py``
311+
file from the submodule to the root of your repository::
312+
313+
$ cd ..
314+
$ cp astropy_helpers/ah_bootstrap.py .
315+
316+
Finally, add::
317+
318+
import ah_bootstrap
319+
320+
at the top of your ``setup.py`` file. This will ensure that ``astropy_helpers``
321+
is now available to use in your ``setup.py`` file. Finally, add then commit your
322+
changes::
323+
324+
git add astropy_helpers ah_bootstrap.py setup.py
325+
git commit -m "Added astropy-helpers"
326+
327+
Updating astropy-helpers
328+
------------------------
329+
330+
If you would like the Astropy team to automatically open pull requests to update
331+
astropy-helpers in your package, then see the instructions `here
332+
<https://github.com/astropy/astropy-procedures/blob/master/update-packages/README.md>`_.
333+
334+
To instead update astropy-helpers manually, go inside the submodule and do::
335+
336+
cd astropy_helpers
337+
git fetch origin
338+
339+
Then checkout the version you want to use, e.g.::
340+
341+
git checkout v3.0.3
342+
343+
Go back up to the root of the repository and update the ``ah_bootstap.py`` file
344+
too, then add your changes::
42345

43-
.. image:: https://coveralls.io/repos/astropy/astropy-helpers/badge.svg
44-
:target: https://coveralls.io/r/astropy/astropy-helpers
346+
cp astropy_helpers/ah_bootstrap.py .
347+
git add astropy_helpers ah_bootstrap.py
348+
git commit ...

0 commit comments

Comments
 (0)