Adding unique_string support#129
Conversation
|
Hmm, worth thinking about if should be in inputs too. I'm not mad about having it there, necessarily? |
|
Also, can we have this target main and then rebase the cp_plus_stardist branch after? We want to get back into the habit of small PRs to main, rather than mega-PRs, if we can. TY! |
|
Agree on aiming for smaller PRs to main going forward! |
Might be! I can may-be think of a case where an algorithm could take both “raw images” and “mask images” as inputs, both .tif. Schema says type: image, format: tif for both, but without unique_string we can’t automatically tell them apart if they’re in the same folder. |
|
Other than also implementing it in inputs, anything else left needed to convert from draft to ready? |
|
I need to test this passing a folder of images (for all algorithms) & inputs that's all |
|
It's not ready for review yet - I am trying to resolve rebase conflcits |
…ration Now loads a single image as ndarray and multiple outputs as list of ndarrays
|
Great, Single and multiple output handling works correctly for algorithms whose Docker images contain an importable Python module, but fails for algorithms like classical_segmentation, where the entry point is a standalone script |
|
Is it failing for a particular interface(s)? Feel free to add more details if you want help chasing it down. |
- Replaced per-type output blocks with single dynamic loader - Added dynamic workspace + display_data registration for all outputs - Improved measurement saving and logging consistency
[fix] wildcard * support for outputs
|
Nope, Now everything works fine across all interfaces, but the output mount path must be Why? In Bilayers, we take a slightly different approach - each output can go to a user-defined path (e.g. /bilayers/output, /bilayers/models, etc.). That’s fine in general, and it means multiple directories might be mounted. However, if the user sets both input and output to the same container path (for example, --save_dir /bilayers/input_images), Docker ends up getting two mounts for the same location: When this happens, the second mount overrides the first one, so the input files become invisible inside the container. Now, if we mount only unique folders (to avoid duplicate mounts), the container only sees what’s explicitly mounted - for eg. In this case, only /bilayers/input_images exists inside the container. P.S.: I've the draft-code that fails at this point for mounting (haven't pushed) |
|
But otherwise, help me break the current plugin just at save_dir in any algo define path as |
|
Sorry, I don't follow one small point here- why isn't the conclusion " |
|
Well, algorithms may allow to write in the same directory as input (eg. cellpose does) - to advocate that case |
separated input/output mounts, added fallbacks, deduplication, and safe image saving
|
Okay, works fine now - when we give unique input & output directories! Failed for Instanseg this time, (before commit e88849d), it was generating some outputs for the same set of images. But the failure is mostly related to the algorithm not plugin |
|
Outputs aren’t written to the default mounted folder because, like standard cp plugins, this module only produces data in memory - saving them to disk is handled separately via the SaveImages module if the user chooses. |
* Moving existing src folder in src/bilayers * Changed type hints like str | None to Optional[str] to support py=3.9 * [gradio] Improve mandatory parameter/input validation and error handling - Validate both None and empty string inputs for mandatory parameters - Update error messaging to clearly flag missing required values - Refactor logic to skip optional parameters when no input is provided * [jupyter] optional bug resolution * unify top-level text creation * linkml schema rule respective changes for modifying citations * ruff fixes for the modified code * removed script,module fields from spec file * [docs] updation * [mod] test config file * [mod] docker-build workflow on self-hosted runners * [mod] workflow on push branch changed to main * [del] extra -f flag in build_interface nox session * resolved most of warnings * Fixed Documentation for now by removing LinkML docs * CLI Usage according to updated folder structure * Added tests * [add] readme * [add] stardist stuff * basic skeleton, need to change * [mod] category, module_name in cp_plugin * [mod] cp plugin * [updated] changes * Measurer changed to Measurement * [resolved] podman and docker in one logic and some small tweaks * [update] temp folder preserve setting * [mod] cp changes on new reviews * [try] docs fix * [basic] workflow changes * [fix] imports * Don't use label images as input for segmentation * typos * fix module name format * build stardist_inference too * Non "none with float/int" modules run * Split numeric parameters that use "None" as a "donotset" * [fix] handle booleans better * [fix] use tmp_path helper for consistent temp file * Adding unique_string support (#129) * [add] unique_string at parsing level * [mod] use glob-based matching for outputs * [fix] ' * [fix] normalize bool to avoid string/True confusion * [readd] somehow deleted stardist_inference.py * chore: trigger GitHub recheck after merge * [fix] correctly handle single vs multiple output files in plugin generation Now loads a single image as ndarray and multiple outputs as list of ndarrays * [fix] unify output handling - Replaced per-type output blocks with single dynamic loader - Added dynamic workspace + display_data registration for all outputs - Improved measurement saving and logging consistency * [add] unique_string support for inputs [fix] wildcard * support for outputs * [add] dynamic mount handling separated input/output mounts, added fallbacks, deduplication, and safe image saving --------- Co-authored-by: Nodar Gogoberidze <[email protected]> Co-authored-by: bethac07 <[email protected]> Co-authored-by: Beth Cimini <[email protected]>
No description provided.