Skip to content

It would be great to monitor the PID of a Job with --wms-monitor option! #3620

@fsysy

Description

@fsysy

Hello,
I would like to express my sincere gratitude for the incredible dedication of the Snakemake development team.

I understand that the progress of Snakemake can be monitored using Panoptes. While it is possible to track the progress, I believe it is not yet linked to actual resource usage (please correct me if I am mistaken).

To optimize resource allocation for each rule, I need to monitor the real-time discrepancy between the planned resources (like threads and memory specified in the .smk file) and the actual usage. As far as I know, Snakemake itself does not support this functionality, and even when using virtualization tools like Docker, monitoring actual usage is quite tricky and requires separate tools for each application.

I looked into the wms-monitor option and reviewed logging.py, but after inspecting the request data coming into the server, I found that while information like rule names, inputs, and outputs is available, there is not much related to actual process information. For example, there is no way to find out which PID is executing each rule. This is crucial information when monitoring real-time resource usage.

Therefore, I kindly request that you consider adding the process.pid related to each rule in the info output.

Thank you very much!

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions