Thanks for looking into this!
Should we check that clang-13 exists in the first place? After the first runs of this script it shouldn't exist anymore.
Alternatively, we could also clean our machines manually from clang-13 instead of adding this to the script IIUC.

Alternatively, we could also clean our machines manually from clang-13 instead of adding this to the script IIUC.

this llvm.sh stuff doesn't exist on the squid/reef branches. would those builds keep installing v19 packages from the repos added by llvm.sh? or would they keep reinstalling v13 packages?

from https://tracker.ceph.com/issues/70792#note-2:

The right solution is to fix the 'update clang' build script snippet to remove any other clang versions. This may happen again as we change the required version; removing 13 (or any other versions) now won't fix the problem, if there is one, when we move from 19 to 20, for example.

@dmick doesn't want to manage the builder dependencies manually. when we eventually want to bump this llvm version to 20+, we're likely to hit the same conflicts again

removing any llvm packages is probably the most reliable approach here (ie purging llvm python3-lldb without specifying version), though it would uninstall/reinstall v19 packages for each build

ubuntu 24.04 builders probably won't default to clang-13 anyway, so we may have conflicts with other versions too

Looking at some of the latest makecheck, v14 is actually being installed (not 13) for main/reef:

CI_DEBUG: Running clean_boost_on_ubuntu() in install-deps.sh
Reading package lists...
Building dependency tree...
Reading state information...
clang is already the newest version (1:14.0-55~exp2).

Do we know whether 14 would also conflict with 19? If it won't, it should explain why only some machines are not happy.

this llvm.sh stuff doesn't exist on the squid/reef branches. would those builds keep installing v19 packages from the repos added by llvm.sh? or would they keep reinstalling v13 packages?

IIUC older branches and main would keep reinstalling v14 as it's part of install-deps. But as long as we only remove 13 here it should be ok (no one should reinstall it).

If 14 would also conflict (hopefully not), then we would keep installing/reinstalling it unless we stop getting 14 as part of install-deps when running mackecheck.

FWIW, I still think it would be better to be done manually if no script would keep reinstalling it back since after the first run this line would be irrelevant. However, this concern is not a blocker and I'm ok with merging as is if I'm missing something else.

It seems to me that we want to purge any/all earlier versions of llvm-and-friends, not just 13

updated to remove the -13s from the apt purge command

Do we think it's safer to leave the other llvm packages intact, or remove them as well?

hmm looking at llvm.sh, this is the list of packages it installs:

PKG="clang-$LLVM_VERSION lldb-$LLVM_VERSION lld-$LLVM_VERSION clangd-$LLVM_VERSION"

so i could copy those for the purge command. the conflicting python3-lldb isn't mentioned there, but i assume it would be removed along with 'lldb'?

and, by the way, the llvm.sh on download.ceph.com is already out of date

could include python3-lldb anyway to be safe, like this?

$DRY_RUN $SUDO apt-get purge --auto-remove clang lldb lld clangd python3-lldb -y

We will purge possibly conflicting packages only under the if ! type clang-19 check above - Just to make sure: the machines which had 13/19 conflicts were not able to install 19 successfully and therefore would fail the !clang-19 check and are expected to enter this branch, right?

that sounds right

does seem like the wrong approach though; it'd be much less likely to break in future if we made it "uninstall everything that's not the version we're asking for" unconditionally

happy to move the 'apt purge' command out of the condition, but is there a way to say "uninstall all clang stuff except v19"?

or would we have to uninstall all clang packages, then run llvm.sh unconditionally?

"for i in 13..current-1 do"... ?