Skip to content

Commit 15e547b

Browse files
authored
Merge pull request #4850 from Liam-DeVoe/rename-internal-example-terminology
Rename remaining internal example terminology to test case
2 parents 265fcc3 + d7fcdfa commit 15e547b

36 files changed

Lines changed: 312 additions & 309 deletions

‎hypothesis/RELEASE.rst‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
RELEASE_TYPE: patch
2+
3+
This release continues the terminology change in :ref:`version 6.159.0 <v6.159.0>`, updating more internal wording and names from "example" to "test case".

‎hypothesis/docs/reference/schema_observations.json‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -20,15 +20,15 @@
2020
},
2121
"representation": {
2222
"type": "string",
23-
"description": "The string representation of the input. In Hypothesis, this includes the property name and arguments (like ``test_a(a=1)``), any interactive draws from |st.data|, and additionally some comments from |Phase.explain| for failing examples."
23+
"description": "The string representation of the input. In Hypothesis, this includes the property name and arguments (like ``test_a(a=1)``), any interactive draws from |st.data|, and additionally some comments from |Phase.explain| for failing test cases."
2424
},
2525
"arguments": {
2626
"type": "object",
2727
"description": "A structured json-encoded representation of the input. Hypothesis provides a dictionary of argument names to json-ified values, including interactive draws from the |st.data| strategy. If 'status' is 'gave_up', this may be absent or incomplete. In other libraries this can be any object."
2828
},
2929
"how_generated": {
3030
"type": ["string", "null"],
31-
"description": "How the input was generated, if known. In Hypothesis this might be an explicit example, generated during a particular phase with some backend, or by replaying the minimal failing example."
31+
"description": "How the input was generated, if known. In Hypothesis this might be an explicit example, generated during a particular phase with some backend, or by replaying the minimal failing test case."
3232
},
3333
"features": {
3434
"type": "object",

‎hypothesis/docs/tutorial/adapting-strategies.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -75,7 +75,7 @@ Where possible, you should use |.filter|. Hypothesis can often rewrite simple fi
7575

7676
For more complex relationships that can't be expressed with |.filter|, use |assume|.
7777

78-
Here's an example of a test where we want to filter out two different types of examples:
78+
Here's an example of a test where we want to filter out two different types of inputs:
7979

8080
.. code-block:: python
8181

‎hypothesis/docs/tutorial/replaying-failures.rst‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -76,8 +76,8 @@ Prefer |@example| over the database for correctness
7676
7777
While the database is useful for quick local iteration, Hypothesis may invalidate it when upgrading (because e.g. the internal format may have changed). Changes to the source code of a test function may also change its database key, invalidating its stored entries. We therefore recommend against relying on the database for the correctness of your tests. If you want to ensure an input is run every time, use |@example|.
7878

79-
Replaying examples with |@reproduce_failure|
80-
--------------------------------------------
79+
Replaying test cases with |@reproduce_failure|
80+
----------------------------------------------
8181

8282
If |settings.print_blob| is set to ``True`` (the default in the ``ci`` settings profile), and a test fails, Hypothesis will print an |@reproduce_failure| decorator containing an opaque blob as part of the error message:
8383

‎hypothesis/src/hypothesis/_settings.py‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -974,7 +974,8 @@ def report_multiple_bugs(self):
974974
Because Hypothesis runs the test many times, it can sometimes find multiple
975975
bugs in a single run. Reporting all of them at once is usually very useful,
976976
but replacing the exceptions can occasionally clash with debuggers.
977-
If disabled, only the exception with the smallest minimal example is raised.
977+
If disabled, only the exception with the smallest
978+
|minimal failing test case| is raised.
978979
979980
The default value is ``True``.
980981
"""

‎hypothesis/src/hypothesis/core.py‎

Lines changed: 54 additions & 53 deletions
Original file line numberDiff line numberDiff line change
@@ -594,7 +594,7 @@ def execute_explicit_examples(state, wrapped_test, arguments, kwargs, original_s
594594
state.execute_once,
595595
empty_data,
596596
is_final=True,
597-
print_example=True,
597+
print_test_case=True,
598598
example_kwargs=example_kwargs,
599599
)
600600
with with_reporter(fragments_reported.append):
@@ -947,7 +947,7 @@ def __init__(
947947
}
948948

949949
self.last_exception = None
950-
self.falsifying_examples = ()
950+
self.falsifying_test_cases = ()
951951
self.ever_executed = False
952952
self.xfail_example_reprs: set[str] = set()
953953
self.failed_normally = False
@@ -985,7 +985,7 @@ def execute_once(
985985
self,
986986
data,
987987
*,
988-
print_example=False,
988+
print_test_case=False,
989989
is_final=False,
990990
expected_failure=None,
991991
example_kwargs=None,
@@ -1071,9 +1071,9 @@ def run(data: ConjectureData) -> None:
10711071
nonlocal text_repr
10721072
text_repr = repr_call(test, args, kwargs)
10731073

1074-
if print_example or current_verbosity() >= Verbosity.verbose:
1074+
if print_test_case or current_verbosity() >= Verbosity.verbose:
10751075
printer = RepresentationPrinter(context=context)
1076-
if print_example:
1076+
if print_test_case:
10771077
printer.text("Failing test case:")
10781078
else:
10791079
printer.text("Test case:")
@@ -1194,16 +1194,16 @@ def _flaky_replay_to_failure(
11941194
self, err: FlakyReplay, context: BaseException
11951195
) -> FlakyFailure:
11961196
assert self._runner is not None
1197-
# Note that in the mark_interesting case, _context_ itself
1198-
# is part of err._interesting_examples - but it's not in
1199-
# _runner.interesting_examples - this is fine, as the context
1197+
# Note that in the mark_interesting case, _context_
1198+
# is part of err._interesting_origins - but it's not in
1199+
# _runner.interesting_test_cases - this is fine, as the context
12001200
# (i.e., immediate exception) is appended.
1201-
interesting_examples = [
1202-
self._runner.interesting_examples[origin]
1201+
interesting_test_cases = [
1202+
self._runner.interesting_test_cases[origin]
12031203
for origin in err._interesting_origins
1204-
if origin in self._runner.interesting_examples
1204+
if origin in self._runner.interesting_test_cases
12051205
]
1206-
exceptions = [result.expected_exception for result in interesting_examples]
1206+
exceptions = [result.expected_exception for result in interesting_test_cases]
12071207
exceptions.append(context) # the immediate exception
12081208
return FlakyFailure(err.reason, exceptions)
12091209

@@ -1438,14 +1438,14 @@ def run_engine(self):
14381438

14391439
if runner.call_count == 0:
14401440
return
1441-
if runner.interesting_examples:
1442-
self.falsifying_examples = sorted(
1443-
runner.interesting_examples.values(),
1441+
if runner.interesting_test_cases:
1442+
self.falsifying_test_cases = sorted(
1443+
runner.interesting_test_cases.values(),
14441444
key=lambda d: sort_key(d.nodes),
14451445
reverse=True,
14461446
)
14471447
else:
1448-
if runner.valid_examples == 0:
1448+
if runner.valid_test_cases == 0:
14491449
explanations = []
14501450
if runner.exit_reason is ExitReason.finished:
14511451
explanations.append(
@@ -1456,23 +1456,23 @@ def run_engine(self):
14561456
)
14571457
# use a somewhat arbitrary cutoff to avoid recommending spurious
14581458
# fixes.
1459-
# eg, a few invalid examples from internal filters when the
1459+
# eg, a few invalid test cases from internal filters when the
14601460
# problem is the user generating large inputs, or a
14611461
# few overruns during internal mutation when the problem is
14621462
# impossible user filters/assumes.
1463-
if runner.invalid_examples > min(20, runner.call_count // 5):
1463+
if runner.invalid_test_cases > min(20, runner.call_count // 5):
14641464
explanations.append(
1465-
f"{runner.invalid_examples} of {runner.call_count} "
1466-
"examples failed a .filter() or assume() condition. Try "
1465+
f"{runner.invalid_test_cases} of {runner.call_count} "
1466+
"test cases failed a .filter() or assume() condition. Try "
14671467
"making your filters or assumes less strict, or rewrite "
14681468
"using strategy parameters: "
14691469
"st.integers().filter(lambda x: x > 0) fails less often "
14701470
"(that is, never) when rewritten as st.integers(min_value=1)."
14711471
)
1472-
if runner.overrun_examples > min(20, runner.call_count // 5):
1472+
if runner.overrun_test_cases > min(20, runner.call_count // 5):
14731473
explanations.append(
1474-
f"{runner.overrun_examples} of {runner.call_count} "
1475-
"examples were too large to finish generating; try "
1474+
f"{runner.overrun_test_cases} of {runner.call_count} "
1475+
"test cases were too large to finish generating; try "
14761476
"reducing the typical size of your inputs?"
14771477
)
14781478
rep = get_pretty_function_description(self.test)
@@ -1497,11 +1497,11 @@ def run_engine(self):
14971497
stacklevel=3,
14981498
)
14991499

1500-
if not self.falsifying_examples:
1500+
if not self.falsifying_test_cases:
15011501
return
15021502
elif not (self.settings.report_multiple_bugs and pytest_shows_exceptiongroups):
15031503
# Pretend that we only found one failure, by discarding the others.
1504-
del self.falsifying_examples[:-1]
1504+
del self.falsifying_test_cases[:-1]
15051505

15061506
# The engine found one or more failures, so we need to reproduce and
15071507
# report them.
@@ -1513,55 +1513,56 @@ def run_engine(self):
15131513
report_lines.append("")
15141514

15151515
explanations = explanatory_lines(self.explain_traces, self.settings)
1516-
for falsifying_example in self.falsifying_examples:
1516+
for falsifying_test_case in self.falsifying_test_cases:
15171517
fragments = []
15181518

1519-
ran_example = runner.new_conjecture_data(
1520-
falsifying_example.choices, max_choices=len(falsifying_example.choices)
1519+
ran_test_case = runner.new_conjecture_data(
1520+
falsifying_test_case.choices,
1521+
max_choices=len(falsifying_test_case.choices),
15211522
)
1522-
ran_example.span_comments = falsifying_example.span_comments
1523+
ran_test_case.span_comments = falsifying_test_case.span_comments
15231524
tb = None
15241525
origin = None
1525-
assert falsifying_example.expected_exception is not None
1526-
assert falsifying_example.expected_traceback is not None
1526+
assert falsifying_test_case.expected_exception is not None
1527+
assert falsifying_test_case.expected_traceback is not None
15271528
try:
15281529
with with_reporter(fragments.append):
15291530
self.execute_once(
1530-
ran_example,
1531-
print_example=True,
1531+
ran_test_case,
1532+
print_test_case=True,
15321533
is_final=True,
15331534
expected_failure=(
1534-
falsifying_example.expected_exception,
1535-
falsifying_example.expected_traceback,
1535+
falsifying_test_case.expected_exception,
1536+
falsifying_test_case.expected_traceback,
15361537
),
15371538
)
15381539
except StopTest as e:
15391540
# Link the expected exception from the first run. Not sure
15401541
# how to access the current exception, if it failed
15411542
# differently on this run. In fact, in the only known
15421543
# reproducer, the StopTest is caused by OVERRUN before the
1543-
# test is even executed. Possibly because all initial examples
1544+
# test is even executed. Possibly because all initial test cases
15441545
# failed until the final non-traced replay, and something was
15451546
# exhausted? Possibly a FIXME, but sufficiently weird to
15461547
# ignore for now.
15471548
err = FlakyFailure(
1548-
"Inconsistent results: An example failed on the "
1549+
"Inconsistent results: A test case failed on the "
15491550
"first run but now succeeds (or fails with another "
15501551
"error, or is for some reason not runnable).",
15511552
# (note: e is a BaseException)
1552-
[falsifying_example.expected_exception or e],
1553+
[falsifying_test_case.expected_exception or e],
15531554
)
15541555
errors_to_report.append(ReportableError(fragments, err))
15551556
except UnsatisfiedAssumption as e: # pragma: no cover # ironically flaky
15561557
err = FlakyFailure(
1557-
"Unreliable assumption: An example which satisfied "
1558+
"Unreliable assumption: A test case which satisfied "
15581559
"assumptions on the first run now fails it.",
15591560
[e],
15601561
)
15611562
errors_to_report.append(ReportableError(fragments, err))
15621563
except BaseException as e:
15631564
# If we have anything for explain-mode, this is the time to report.
1564-
fragments.extend(explanations[falsifying_example.interesting_origin])
1565+
fragments.extend(explanations[falsifying_test_case.interesting_origin])
15651566
error_with_tb = e.with_traceback(get_trimmed_traceback())
15661567
errors_to_report.append(ReportableError(fragments, error_with_tb))
15671568
tb = format_exception(e, get_trimmed_traceback(e))
@@ -1570,16 +1571,16 @@ def run_engine(self):
15701571
# execute_once() will always raise either the expected error, or Flaky.
15711572
raise NotImplementedError("This should be unreachable")
15721573
finally:
1573-
ran_example.freeze()
1574+
ran_test_case.freeze()
15741575
if observability_enabled():
15751576
# log our observability line for the final failing test case
15761577
tc = make_testcase(
15771578
run_start=self._start_timestamp,
15781579
property=self.test_identifier,
1579-
data=ran_example,
1580+
data=ran_test_case,
15801581
how_generated="minimal failing test case",
15811582
representation=self._string_repr,
1582-
arguments=ran_example._observability_args,
1583+
arguments=ran_test_case._observability_args,
15831584
timing=self._timing_features,
15841585
coverage=None, # Not recorded when we're replaying the MFE
15851586
status="passed" if sys.exc_info()[0] else "failed",
@@ -1593,7 +1594,7 @@ def run_engine(self):
15931594
if self.settings.print_blob:
15941595
fragments.append(
15951596
"\nYou can reproduce this test case by temporarily adding "
1596-
f"{reproduction_decorator(falsifying_example.choices)} "
1597+
f"{reproduction_decorator(falsifying_test_case.choices)} "
15971598
"as a decorator on your test function"
15981599
)
15991600

@@ -2104,7 +2105,7 @@ def wrapped_test(*arguments, **kwargs):
21042105
f"{type(runner).__qualname__}, but this "
21052106
"class does not inherit from the supported versions in "
21062107
"`hypothesis.extra.django`. Use the Hypothesis variants "
2107-
"to ensure that each example is run in a separate "
2108+
"to ensure that each test case is run in a separate "
21082109
"database transaction."
21092110
)
21102111

@@ -2163,7 +2164,7 @@ def wrapped_test(*arguments, **kwargs):
21632164
try:
21642165
state.execute_once(
21652166
ConjectureData.for_choices(decode_failure(failure)),
2166-
print_example=True,
2167+
print_test_case=True,
21672168
is_final=True,
21682169
)
21692170
raise DidNotReproduce(
@@ -2213,14 +2214,14 @@ def wrapped_test(*arguments, **kwargs):
22132214
Phase.explicit in state.settings.phases
22142215
and getattr(wrapped_test, "hypothesis_explicit_examples", ())
22152216
)
2216-
SKIP_BECAUSE_NO_EXAMPLES = unittest.SkipTest(
2217+
SKIP_BECAUSE_NO_TEST_CASES = unittest.SkipTest(
22172218
"Hypothesis has been told to run no test cases for this test."
22182219
)
22192220
if not (
22202221
Phase.reuse in settings.phases or Phase.generate in settings.phases
22212222
):
22222223
if not ran_explicit_examples:
2223-
raise SKIP_BECAUSE_NO_EXAMPLES
2224+
raise SKIP_BECAUSE_NO_TEST_CASES
22242225
return
22252226

22262227
try:
@@ -2287,7 +2288,7 @@ def wrapped_test(*arguments, **kwargs):
22872288
raise the_error_hypothesis_found
22882289

22892290
if not (ran_explicit_examples or state.ever_executed):
2290-
raise SKIP_BECAUSE_NO_EXAMPLES
2291+
raise SKIP_BECAUSE_NO_TEST_CASES
22912292
finally:
22922293
with thread_overlap_lock:
22932294
del thread_overlap[threadid]
@@ -2323,8 +2324,8 @@ def _get_fuzz_target() -> (
23232324
thread_overlap=thread_overlap,
23242325
)
23252326
database_key = function_digest(test) + b".secondary"
2326-
# We track the minimal-so-far example for each distinct origin, so
2327-
# that we track log-n instead of n examples for long runs. In particular
2327+
# We track the minimal-so-far test case for each distinct origin, so
2328+
# that we track log-n instead of n test cases for long runs. In particular
23282329
# it means that we saturate for common errors in long runs instead of
23292330
# storing huge volumes of low-value data.
23302331
minimal_failures: dict = {}
@@ -2423,7 +2424,7 @@ def find(
24232424
random: Random | None = None,
24242425
database_key: bytes | None = None,
24252426
) -> Ex:
2426-
"""Returns the minimal example from the given strategy ``specifier`` that
2427+
"""Returns the minimal value from the given strategy ``specifier`` that
24272428
matches the predicate function ``condition``."""
24282429
if settings is None:
24292430
settings = Settings(max_examples=2000)
@@ -2433,7 +2434,7 @@ def find(
24332434

24342435
if database_key is None and settings.database is not None:
24352436
# Note: The database key is not guaranteed to be unique. If not, replaying
2436-
# of database examples may fail to reproduce due to being replayed on the
2437+
# of database test cases may fail to reproduce due to being replayed on the
24372438
# wrong condition.
24382439
database_key = function_digest(condition)
24392440

‎hypothesis/src/hypothesis/extra/ghostwriter.py‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1812,7 +1812,7 @@ def maker(
18121812
_write_call(func, "b", "a", assign="right"),
18131813
)
18141814
if identity is not None:
1815-
# Guess that the identity element is the minimal example from our operands
1815+
# Guess that the identity element is the minimal value from our operands
18161816
# strategy. This is correct often enough to be worthwhile, and close enough
18171817
# that it's a good starting point to edit much of the rest.
18181818
if identity is ...:

0 commit comments

Comments
 (0)