Skip to content

Commit a024e60

Browse files
johanneskoesterjohanneskoester
authored
feat: provide mechanism to link between report items (snakemake.report_href, see docs) (#3224)
<!--Add a description of your PR here--> ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes - **New Features** - Enhanced reporting capabilities with new directives and methods for generating comprehensive HTML reports. - Introduced dynamic linking in reports and the ability to include file labels and categories. - Added new HTML report generation functionality with example workflows. - New functionality to manage report hyperlinks, improving navigation within reports. - **Bug Fixes** - Improved error handling and user feedback during report generation. - **Tests** - Added new tests for report generation functionality to ensure reliability. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
1 parent 0b541cc commit a024e60

File tree

12 files changed

+277
-21
lines changed

12 files changed

+277
-21
lines changed

‎docs/snakefiles/reporting.rst‎

Lines changed: 104 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,8 +1,8 @@
11
.. _snakefiles-reports:
22

3-
-------
3+
=======
44
Reports
5-
-------
5+
=======
66

77
From Snakemake 5.1 on, it is possible to automatically generate detailed self-contained HTML reports that encompass runtime statistics, provenance information, workflow topology and results.
88
**As an example, the report of the Snakemake rolling paper can be found** `here <https://snakemake.github.io/resources/report.html>`__.
@@ -94,7 +94,7 @@ This works as follows:
9494
"""
9595
9696
Defining file labels
97-
~~~~~~~~~~~~~~~~~~~~~
97+
--------------------
9898

9999
In addition to category, and subcategory, it is possible to define a dictionary of labels for each report item.
100100
By that, the actual filename will be hidden in the report and instead a table with the label keys as columns and the values in the respective row for the file will be displayed.
@@ -122,16 +122,19 @@ Consider the following modification of rule ``b`` from above:
122122
123123
124124
Determining category, subcategory, and labels dynamically via functions
125-
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
125+
-----------------------------------------------------------------------
126126

127127
Similar to e.g. with input file and parameter definition (see :ref:`snakefiles-input_functions`), ``category`` and a ``subcategory`` and ``labels`` can be specified by pointing to a function that takes ``wildcards`` as the first argument (and optionally in addition ``input``, ``output``, ``params`` in any order).
128128
The function is expected to return a string or number (int, float, numpy types), or, in case of labels, a dict with strings as keys and strings or numbers as values.
129129

130130

131131
Linking between items
132-
~~~~~~~~~~~~~~~~~~~~~
132+
---------------------
133133

134-
In every ``.rst`` document, you can link to
134+
From captions
135+
^^^^^^^^^^^^^
136+
137+
In every ``.rst`` document (i.e. in the captions), you can link to
135138

136139
* the **Workflow** panel (with ``Rules_``),
137140
* the **Statistics** panel (with ``Statistics_``),
@@ -140,8 +143,102 @@ In every ``.rst`` document, you can link to
140143

141144
For details about the hyperlink mechanism of restructured text see `here <https://docutils.sourceforge.io/docs/user/rst/quickref.html#hyperlink-targets>`__.
142145

146+
From results
147+
^^^^^^^^^^^^
148+
149+
From within results that are included into the report, you can link to other report items.
150+
This works by using the ``snakemake.report_href()`` method that is available from within :ref:`python scripts <snakefiles-external_scripts>`.
151+
The method takes the path to the target report item in exactly the same form as it is given in the Snakefile,
152+
and optionally can be extended to target child paths or by URL arguments.
153+
For example, consider the following Snakefile:
154+
155+
.. code-block:: python
156+
157+
rule a:
158+
input:
159+
report("test.html"),
160+
report(
161+
"subdir",
162+
patterns=["{name}.html"],
163+
)
164+
output:
165+
report(
166+
"test2.html",
167+
)
168+
script:
169+
"test_script.py"
170+
171+
Inside of the script, we can now use ``snakemake.report_href()`` to create a link to the file ``test.html`` such that it can be accessed from the file ``test2.html``:
172+
173+
.. code-block:: python
174+
175+
import textwrap
176+
177+
with open(snakemake.output[0], "w") as f:
178+
print(
179+
textwrap.dedent(f"""
180+
<html>
181+
<head>
182+
<title>Report</title>
183+
</head>
184+
<body>
185+
<a href={snakemake.report_href("test.html")}>Link to test.html</a>
186+
</body>
187+
</html>
188+
"""
189+
),
190+
file=f,
191+
)
192+
193+
Note that you will rarely directly generate HTML like this in a Python script within a Snakemake workflow.
194+
Rather, you might want to access ``snakemake.report_href()`` when e.g. generating a table which is later rendered into HTML by e.g. `Datavzrd <https://datavzrd.github.io>`__ (also see :ref:`interaction_visualization_reporting_tutorial`).
195+
196+
In case you want to refer to a file that is inside of a directory that is included into the Snakemake report, you can do so using the ``child_path`` method:
197+
198+
.. code-block:: python
199+
200+
import textwrap
201+
202+
with open(snakemake.output[0], "w") as f:
203+
print(
204+
textwrap.dedent(f"""
205+
<html>
206+
<head>
207+
<title>Report</title>
208+
</head>
209+
<body>
210+
<a href={snakemake.report_href("subdir").child_path("foo.html")}>Link to test.html</a>
211+
</body>
212+
</html>
213+
"""
214+
),
215+
file=f,
216+
)
217+
218+
Further, using ``url_args()`` you can add URL arguments and using ``anchor()`` you can add a target anchor to the link, e.g. to scroll to a specific section of the target document:
219+
220+
.. code-block:: python
221+
222+
import textwrap
223+
224+
with open(snakemake.output[0], "w") as f:
225+
print(
226+
textwrap.dedent(f"""
227+
<html>
228+
<head>
229+
<title>Report</title>
230+
</head>
231+
<body>
232+
<a href={snakemake.report_href("subdir").child_path("foo.html").url_args(someparam=5).anchor("mysection")}>Link to test.html</a>
233+
</body>
234+
</html>
235+
"""
236+
),
237+
file=f,
238+
)
239+
143240
Rendering reports
144-
~~~~~~~~~~~~~~~~~
241+
-----------------
145242

146243
To create the report simply run
147244

‎snakemake/common/__init__.py‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@
1818
import asyncio
1919
import collections
2020
from pathlib import Path
21+
from typing import Union
2122

2223
from snakemake import __version__
2324
from snakemake_interface_common.exceptions import WorkflowError
@@ -53,6 +54,13 @@ def get_snakemake_searchpaths():
5354
return list(unique_justseen(paths))
5455

5556

57+
def get_report_id(path: Union[str, Path]) -> str:
58+
h = hashlib.sha256()
59+
h.update(str(path).encode())
60+
61+
return h.hexdigest()
62+
63+
5664
def mb_to_mib(mb):
5765
return int(math.ceil(mb * 0.95367431640625))
5866

‎snakemake/report/__init__.py‎

Lines changed: 27 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -58,8 +58,7 @@
5858
JobRecordInterface,
5959
FileRecordInterface,
6060
)
61-
from pathlib import Path
62-
from snakemake.common import is_local_file
61+
from snakemake.common import get_report_id
6362
from snakemake.exceptions import WorkflowError
6463

6564

@@ -358,6 +357,7 @@ class JobRecord(JobRecordInterface):
358357
class FileRecord(FileRecordInterface):
359358
path: Path
360359
job: Job
360+
parent_path: Optional[Path] = None
361361
category: Optional[str] = None
362362
wildcards_overwrite: Optional[Wildcards] = None
363363
labels: Optional[dict] = None
@@ -378,10 +378,7 @@ def __post_init__(self):
378378
logger.info(f"Adding {self.name} ({format_size(self.size)}).")
379379
self.mime, _ = mime_from_file(self.path)
380380

381-
h = hashlib.sha256()
382-
h.update(str(self.path).encode())
383-
384-
self.id = h.hexdigest()
381+
self.id = get_report_id(self.parent_path or self.path)
385382
self.wildcards = logging.format_wildcards(self.raw_wildcards)
386383
self.params = (
387384
logging.format_dict(self.job.params)
@@ -440,7 +437,10 @@ def name(self):
440437

441438
@property
442439
def filename(self):
443-
return os.path.basename(self.path)
440+
if self.parent_path is None:
441+
return os.path.basename(self.path)
442+
else:
443+
return str(self.path.relative_to(self.parent_path))
444444

445445
@property
446446
def workflow(self):
@@ -542,7 +542,11 @@ def get_time(rectime, metatime, sel_func):
542542
report_obj = get_flag_value(f, "report")
543543

544544
def register_file(
545-
f, wildcards_overwrite=None, aux_files=None, name_overwrite=None
545+
f,
546+
parent_path=None,
547+
wildcards_overwrite=None,
548+
aux_files=None,
549+
name_overwrite=None,
546550
):
547551
wildcards = wildcards_overwrite or job.wildcards
548552
category = Category(
@@ -556,6 +560,9 @@ def register_file(
556560
results[category][subcategory].append(
557561
FileRecord(
558562
path=Path(f),
563+
parent_path=(
564+
Path(parent_path) if parent_path is not None else None
565+
),
559566
job=job,
560567
category=category,
561568
raw_caption=report_obj.caption,
@@ -603,16 +610,26 @@ def register_file(
603610
rule=job.rule,
604611
)
605612

613+
found_something = False
606614
for pattern in report_obj.patterns:
607615
pattern = os.path.join(f, pattern)
608616
wildcards = glob_wildcards(pattern)._asdict()
617+
found_something |= len(wildcards) > 0
609618
names = wildcards.keys()
610619
for w in zip(*wildcards.values()):
611620
w = dict(zip(names, w))
612621
w.update(job.wildcards_dict)
613622
w = Wildcards(fromdict=w)
614-
f = apply_wildcards(pattern, w)
615-
register_file(f, wildcards_overwrite=w)
623+
subfile = apply_wildcards(pattern, w)
624+
register_file(
625+
subfile, parent_path=f, wildcards_overwrite=w
626+
)
627+
if not found_something:
628+
logger.warning(
629+
"No files found for patterns given to report marker "
630+
"in rule {job.rule.name} for output {f}. Make sure "
631+
"that the patterns are correctly specified."
632+
)
616633
else:
617634
raise WorkflowError(
618635
"Directory marked for report but neither file patterns "

‎snakemake/report/html_reporter/template/components/app.js‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,7 @@ let app;
77
class App extends React.Component {
88
constructor(props) {
99
super(props);
10-
this.state = { hideNavbar: false, navbarMode: "menu", content: "rulegraph", ruleinfo: undefined, category: undefined, subcategory: undefined, searchTerm: undefined, resultPath: undefined, contentPath: undefined };
10+
this.state = { hideNavbar: false, navbarMode: "menu", content: "rulegraph", ruleinfo: undefined, category: undefined, subcategory: undefined, searchTerm: undefined, resultPath: undefined, contentPath: undefined, renderTrigger: undefined };
1111
this.setView = this.setView.bind(this);
1212
this.showCategory = this.showCategory.bind(this);
1313
this.showResultInfo = this.showResultInfo.bind(this);
@@ -39,7 +39,8 @@ class App extends React.Component {
3939
resultPath: view.resultPath || this.state.resultPath,
4040
contentPath: view.contentPath || this.state.contentPath,
4141
contentText: view.contentText || this.state.contentText,
42-
})
42+
renderTrigger: view.content !== undefined ? Math.random() : this.state.renderTrigger,
43+
});
4344
}
4445

4546
showCategory(category) {

‎snakemake/report/html_reporter/template/components/content.js‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -65,7 +65,7 @@ class ContentDisplay extends React.Component {
6565
case "pdf":
6666
return e(
6767
"iframe",
68-
{ src: this.props.app.state.contentPath, className: "w-full h-screen" }
68+
{ src: this.props.app.state.contentPath, className: "w-full h-screen", key: `${this.props.app.state.renderTrigger}` }
6969
);
7070
case "text":
7171
return e(

‎snakemake/script/__init__.py‎

Lines changed: 65 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,8 +20,10 @@
2020
from abc import ABC, abstractmethod
2121
from collections.abc import Iterable
2222
from pathlib import Path
23-
from typing import List, Optional, Pattern, Tuple, Union
23+
from typing import List, Optional, Pattern, Tuple, Union, Dict
2424
from urllib.error import URLError
25+
import urllib.parse
26+
from typing import TypeVar
2527

2628
from snakemake import io as io_
2729
from snakemake import sourcecache
@@ -37,6 +39,7 @@
3739
infer_source_file,
3840
)
3941
from snakemake.utils import format
42+
from snakemake.common import get_report_id
4043

4144
# TODO use this to find the right place for inserting the preamble
4245
PY_PREAMBLE_RE = re.compile(r"from( )+__future__( )+import.*?(?P<end>[;\n])")
@@ -46,6 +49,58 @@
4649
snakemake: "Snakemake"
4750

4851

52+
# For compatibility with Python <3.11 where typing.Self is not available.
53+
ReportHrefType = TypeVar("ReportHrefType", bound="ReportHref")
54+
55+
56+
class ReportHref:
57+
def __init__(
58+
self,
59+
path: Union[str, Path],
60+
parent: Optional[ReportHrefType] = None,
61+
url_args: Optional[Dict[str, str]] = None,
62+
anchor: Optional[str] = None,
63+
):
64+
self._parent = parent
65+
if parent is None:
66+
self._id = get_report_id(path)
67+
else:
68+
self._id = parent._id
69+
# ensure that path is a url compatible string
70+
self._path = path if isinstance(path, str) else str(path.as_posix())
71+
self._url_args = (
72+
{key: value for key, value in url_args.items()} if url_args else {}
73+
)
74+
self._anchor = anchor
75+
76+
def child_path(self, path: Union[str, Path]) -> ReportHrefType:
77+
return ReportHref(path, parent=self)
78+
79+
def url_args(self, **args: str) -> ReportHrefType:
80+
return ReportHref(path=self._path, parent=self._parent, url_args=args)
81+
82+
def anchor(self, anchor: str) -> ReportHrefType:
83+
return ReportHref(
84+
path=self._path, parent=self._parent, url_args=self._url_args, anchor=anchor
85+
)
86+
87+
def __str__(self) -> str:
88+
path = os.path.basename(self._path) if self._parent is None else self._path
89+
if self._url_args:
90+
91+
def fmt_arg(key, value):
92+
return f"{key}={urllib.parse.quote(str(value))}"
93+
94+
args = f"?{'&'.join(fmt_arg(key, value) for key, value in self._url_args.items())}"
95+
else:
96+
args = ""
97+
if self._anchor:
98+
anchor = f"#{urllib.parse.quote(self._anchor)}"
99+
else:
100+
anchor = ""
101+
return f"../{self._id}/{path}{args}{anchor}"
102+
103+
49104
class Snakemake:
50105
def __init__(
51106
self,
@@ -75,6 +130,15 @@ def __init__(
75130
self.bench_iteration = bench_iteration
76131
self.scriptdir = scriptdir
77132

133+
def report_href(self, path: Union[str, Path]) -> ReportHref:
134+
"""Return an href to the given path in the report context, assuming that the
135+
path is given as it is given to the report marker in the workflow.
136+
137+
The returned object can be extended to child paths using the `child_path(path)`
138+
method. This is useful if the referred item is a directory.
139+
"""
140+
return ReportHref(path)
141+
78142
def log_fmt_shell(
79143
self, stdout: bool = True, stderr: bool = True, append: bool = False
80144
) -> str:

‎tests/test_report_href/Snakefile‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
rule a:
2+
input:
3+
report("test.html"),
4+
report(
5+
"subdir",
6+
patterns=["subdir/{name}.html"],
7+
)
8+
output:
9+
report(
10+
"test2.html",
11+
)
12+
script:
13+
"test_script.py"

0 commit comments

Comments
 (0)