Hello,
I would like to express my sincere gratitude for the incredible dedication of the Snakemake development team.
I understand that the progress of Snakemake can be monitored using Panoptes. While it is possible to track the progress, I believe it is not yet linked to actual resource usage (please correct me if I am mistaken).
To optimize resource allocation for each rule, I need to monitor the real-time discrepancy between the planned resources (like threads and memory specified in the .smk file) and the actual usage. As far as I know, Snakemake itself does not support this functionality, and even when using virtualization tools like Docker, monitoring actual usage is quite tricky and requires separate tools for each application.
I looked into the wms-monitor option and reviewed logging.py, but after inspecting the request data coming into the server, I found that while information like rule names, inputs, and outputs is available, there is not much related to actual process information. For example, there is no way to find out which PID is executing each rule. This is crucial information when monitoring real-time resource usage.
Therefore, I kindly request that you consider adding the process.pid related to each rule in the info output.
Thank you very much!
Hello,
I would like to express my sincere gratitude for the incredible dedication of the Snakemake development team.
I understand that the progress of Snakemake can be monitored using Panoptes. While it is possible to track the progress, I believe it is not yet linked to actual resource usage (please correct me if I am mistaken).
To optimize resource allocation for each rule, I need to monitor the real-time discrepancy between the planned resources (like threads and memory specified in the .smk file) and the actual usage. As far as I know, Snakemake itself does not support this functionality, and even when using virtualization tools like Docker, monitoring actual usage is quite tricky and requires separate tools for each application.
I looked into the wms-monitor option and reviewed logging.py, but after inspecting the request data coming into the server, I found that while information like rule names, inputs, and outputs is available, there is not much related to actual process information. For example, there is no way to find out which PID is executing each rule. This is crucial information when monitoring real-time resource usage.
Therefore, I kindly request that you consider adding the process.pid related to each rule in the info output.
Thank you very much!