Skip to content

Commit 98ec165

Browse files
ferdnycptmcg
authored andcommitted
Docs: Add doctest-wrtiting documentation
- Add "Some documentation points" to `CONTRIBUTING.md` - Add separate `docs/Writing_Doctests.md` deep-dive document - Configure Sphinx to include `Writing_Doctests.md` in docs
1 parent 0b46bcd commit 98ec165

File tree

4 files changed

+545
-34
lines changed

4 files changed

+545
-34
lines changed

CONTRIBUTING.md

Lines changed: 93 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -38,41 +38,49 @@ If you have a question on using pyparsing, there are a number of resources avail
3838

3939
If you have an example you wish to submit, please follow these guidelines.
4040

41-
- **License - Submitted example code must be available for distribution with the rest of pyparsing under the MIT
41+
- **License - Submitted example code must be available for distribution with the rest of pyparsing under the MIT
4242
open source license.**
4343

4444
- Please follow PEP8 name and coding guidelines, and use the black formatter
45-
to auto-format code.
45+
to auto-format code.
4646

4747
- Examples should import pyparsing and the common namespace classes as:
4848

49-
import pyparsing as pp
50-
# if necessary
51-
ppc = pp.pyparsing_common
52-
ppu = pp.pyparsing_unicode
49+
```python
50+
import pyparsing as pp
51+
# if necessary
52+
ppc = pp.pyparsing_common
53+
ppu = pp.pyparsing_unicode
54+
```
5355

54-
- Submitted examples *must* be Python 3.6.8 or later compatible. (It is acceptable if examples use Python
55-
features added after 3.6)
56+
- Submitted examples _must_ be Python 3.6.8 or later compatible.
57+
(It is acceptable if examples use Python features added after 3.6)
5658

5759
- Where possible use operators to create composite parse expressions:
5860

59-
expr = expr_a + expr_b | expr_c
61+
```python
62+
expr = expr_a + expr_b | expr_c
63+
```
6064

6165
instead of:
6266

63-
expr = pp.MatchFirst([pp.And([expr_a, expr_b]), expr_c])
67+
```python
68+
expr = pp.MatchFirst([pp.And([expr_a, expr_b]), expr_c])
69+
```
6470

6571
Exception: if using a generator to create an expression:
6672

67-
import keyword
68-
python_keywords = keyword.kwlist
69-
any_keyword = pp.MatchFirst(pp.Keyword(kw)
70-
for kw in python_keywords))
73+
```python
74+
import keyword
75+
python_keywords = keyword.kwlist
76+
any_keyword = pp.MatchFirst(pp.Keyword(kw)
77+
for kw in python_keywords))
78+
```
7179

72-
- Learn [Common Pitfalls When Writing Parsers](https://github.com/pyparsing/pyparsing/wiki/Common-Pitfalls-When-Writing-Parsers) and
80+
- Learn [Common Pitfalls When Writing Parsers][pitfalls] and
7381
how to avoid them when developing new examples.
7482

75-
- See additional notes under [Some Coding Points](#some-coding-points).
83+
- See additional notes under [Some coding points](#some-coding-points).
7684

7785
## Submitting changes
7886

@@ -88,11 +96,11 @@ intended on prior versions of Python (currently back to Python 3.6.8).
8896

8997
## Some design points
9098

91-
- Minimize additions to the module namespace. Over time, pyparsing's namespace has acquired a *lot* of names.
99+
- Minimize additions to the module namespace. Over time, pyparsing's namespace has acquired a _lot_ of names.
92100
New features have been encapsulated into namespace classes to try to hold back the name flooding when importing
93101
pyparsing.
94102

95-
- New operator overloads for ParserElement will need to show broad applicability, and should be related to
103+
- New operator overloads for ParserElement will need to show broad applicability, and should be related to
96104
parser construction.
97105

98106
- Performance tuning should focus on parse time performance. Optimizing parser definition performance is secondary.
@@ -108,28 +116,87 @@ These coding styles are encouraged whether submitting code for core pyparsing or
108116
name casing. I had just finished several years of Java and Smalltalk development, and camel case seemed to be the
109117
future trend in coding styles. As of version 3.0.0, pyparsing is moving over to PEP8 naming, while maintaining
110118
compatibility with existing parser code by defining synonyms using the legacy names. These names will be
111-
retained until a future release (probably 4.0), to provide a migration path for current pyparsing-dependent
119+
retained until a future release (probably 4.0), to provide a migration path for current pyparsing-dependent
112120
applications - DO NOT MODIFY OR REMOVE THESE NAMES.
113121
See more information at the [PEP8 wiki page](https://github.com/pyparsing/pyparsing/wiki/PEP-8-planning).
114122

115123
- No backslashes for line continuations.
116-
Continuation lines for expressions in ()'s should start with the continuing operator:
124+
Continuation lines for expressions in `()`'s should start with the continuing operator:
117125

118-
really_long_line = (something
119-
+ some_other_long_thing
120-
+ even_another_long_thing)
126+
```python
127+
really_long_line = (something
128+
+ some_other_long_thing
129+
+ even_another_long_thing)
130+
```
121131

122132
- Maximum line length is 120 characters. (Black will override this.)
123133

124134
- Changes to core pyparsing must be compatible back to Py3.6 without conditionalizing. Later Py3 features may be
125135
used in examples by way of illustration.
126136

127-
- str.format() statements should use named format arguments (unless this proves to be a slowdown at parse time).
137+
- `str.format()` statements should use named format arguments (unless this proves to be a slowdown at parse time).
128138

129139
- List, tuple, and dict literals should include a trailing comma after the last element, which reduces changeset
130140
clutter when another element gets added to the end.
131141

132-
- New features should be accompanied by updates to unitTests.py and a bullet in the CHANGES file.
142+
- New features should be accompanied by updates to `unitTests.py` and a bullet in the CHANGES file.
133143

134-
- Do not modify pyparsing_archive.py. This file is kept as a reference artifact from when pyparsing was distributed
144+
- Do not modify `pyparsing_archive.py`. This file is kept as a reference artifact from when pyparsing was distributed
135145
as a single source file.
146+
147+
## Some documentation points
148+
149+
- The docstrings in pyparsing (which are generated into the package's
150+
API documentation by Sphinx) make heavy use of doctests for their
151+
example code. This allows examples to be tested and verified as
152+
working, and ensures that any changes to the code which affect
153+
output are accompanied by corresponding changes in the examples.
154+
155+
- The codebase's docstring tests can be verified by running the
156+
command `make doctest` from the `docs/` directory. The output
157+
should ideally look something like this:
158+
159+
```console
160+
$ make doctest
161+
[...documentation build...]
162+
running tests...
163+
164+
Document: pyparsing
165+
-------------------
166+
1 item passed all tests:
167+
204 tests in default
168+
204 tests in 1 item.
169+
204 passed.
170+
Test passed.
171+
172+
Document: whats_new_in_3_1
173+
--------------------------
174+
1 item passed all tests:
175+
15 tests in default
176+
15 tests in 1 item.
177+
15 passed.
178+
Test passed.
179+
180+
Doctest summary
181+
===============
182+
219 tests
183+
0 failures in tests
184+
0 failures in setup code
185+
0 failures in cleanup code
186+
```
187+
188+
Any failed tests will be displayed in detail.
189+
190+
- Much more information about doctests can be found in the
191+
[Pyparsing documentation][pyparsing-docs], in the chapter titled
192+
"Writing doctest examples". Even if you have never worked with them
193+
before, it should guide you through everything you need to know in
194+
order to write Pyparsing doctest examples. If you are already familiar
195+
with doctests and with `sphinx.ext.doctest` in general, you may wish
196+
to skip over the introductory content and go straight to the section
197+
on "Doctests in Pyparsing" which covers some issues specific to the
198+
project.
199+
200+
<!-- Named hyperlink targets, used in the preceding text -->
201+
[pitfalls]: https://github.com/pyparsing/pyparsing/wiki/Common-Pitfalls-When-Writing-Parsers
202+
[pyparsing-docs]: https://pyparsing-docs.readthedocs.io/en/latest/

0 commit comments

Comments
 (0)