Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 83.14%. Comparing base (765123c) to head (d7af431).
⚠️ Report is 301 commits behind head on main.

@@           Coverage Diff           @@
##             main    #2873   +/-   ##
=======================================
  Coverage   83.14%   83.14%           
=======================================
  Files         126      126           
  Lines       13994    13994           
  Branches     2900     2900           
=======================================
  Hits        11636    11636           
  Misses       1727     1727           
  Partials      631      631           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Using this document to check common use cases in the repo to test and make sure the linkml-runtime merge is working as expected: https://docs.google.com/document/d/1Z5Ln3Lm3hlRhcu2Chpn-JKl_mkE9sXUJVtbuxMt11Jw/edit?usp=sharing

The merging accomplished by this PR is really needed to make the developer experience much better! But it is probably tough as hell to be reviewed...
@iQuxLE any hints on which strategy any reviewer should follow?

@Silvanoc thanks for your comments and apologies for the late answer.
@sujaypatil96 and @sierra-moxon are testing these changes with current implementations and have a testing agenda.

We most likely need to redo parts of the PR as in the process we are rewriting history of runtime so that it looks as it was always inside the main repo. Otherwise the history can't be transferred.

Due to changes in runtime - as long as there is no general stop on PRs - we can't necessarily merge without running this again.

However there is a pipeline we can go through pretty fast and as soon as we know everything works on biolink etc this (the history merge) can be repeated and we should be able to go forward.

I know it's going a bit slowly but everyone needs to make sure we are not missing something.

@iQuxLE don't worry, I just wanted to avoid this effort stalling because nobody is looking at this PR. When someone invests the huge effort that you've probably invested here in the thought that it will be highly welcomed, it's hard the feeling that nobody cares.

I think that this merging is very important to improve the developers experience and therefore I wanted to give you the attention that this effort is worth and help moving forward.

I'm glad to learn that this topic is moving on, even if we don't have any observable manifestations of that movement.

Thanks for reviewing this @Silvanoc. We reviewed this PR on call last Friday, 10/3. There is an internal document that describes the procedure that @iQuxLE followed to make this PR, and we tried to replicate the same in #2928 and I think we were able to do it successfully. @kevinschaper made a bash script that has all the git commands needed to replicate this monorepo merge process, and he will be putting the script in it's own repo in the linkml org soon, so others can replicate too. We have a testing document for this PR as well: https://docs.google.com/document/d/1Z5Ln3Lm3hlRhcu2Chpn-JKl_mkE9sXUJVtbuxMt11Jw/edit?usp=sharing. If you have any ideas, or you think we've missed something while testing this PR, please feel free to make suggestions on this document.

However, the plan for us is to resolve all the merge conflicts on this PR and make a release candidate from this branch. Then, some of us have volunteered to test this release candidate in downstream schema repos and then once we've confirmed that everything is working as expected, we can make an official release.

A crazy question: Would it make sense to include also the linkml-model repository into the monorepo? - If so, now would be the time. 😉

I'll add the consideration of adding the model to the mono-repo to the LinkML developers call. I do want to raise the concern that doing so might slow us down in getting this done. I think it is too important to wait much longer. I'd like to resolve the conflicts, re-run the workflow, and get this merge finished.

I extended the bash script from @kevinschaper to also include linkml_model linkml-runtime-model-migration.sh

This does not yet work well. There are several conflicts. The script needs more work.

After thinking and experimenting more with merging linkml-model as well, I think that it should be done later and as a separate step. It has other challenges for example handling of the documentation which is not relevant for linkml-runtime.

I wonder about the final structure a bit. Why is it like it is (which somewhat deviates from uv-managed monorepo recommendations)? It would be great to have an issue for the monorepo plan. --> Made one: #2936

I would strongly recommend structuring these as PEP-420 namespace packages.

packaging docs: https://packaging.python.org/en/latest/guides/packaging-namespace-packages/
example: https://github.com/pypa/sample-namespace-packages/tree/master/native

so then they could be distributed as separable packages like linkml-runtime, linkml-pydantic, linkml-core, and so on (i would propose linkml as a distribution becomes just an alias package that refers to all the packages in the namespace, and that's what namespace packages imply) but then still share the same linkml namespace at import time: from linkml.runtime import ..., from linkml.generators.pydantic import...

the uv workspaces-only route may work for linkml_runtime, but it would make it more challenging to split off the generators and the other parts of linkml, since each of those would have to become from linkml_pydantic import .... uv workspaces are also not a standard, but if this project is already locked in to uv then that bullet has already been bitten. it looks like they can be used together: https://github.com/libranet/demo-uv-workspace/tree/main (from this issue: astral-sh/uv#10628 )

This task was finally completed in #2987