pyproject: Don't use the namesquatting preventing bs4 pypi package by doronbehar · Pull Request #3706 · scipp/scipp

doronbehar · 2025-05-23T09:35:50Z

See:

https://pypi.org/project/bs4/

jl-wynen · 2025-05-23T09:43:48Z

Thanks! I think we can actually remove the dependency completely. @nvaytet do you know why we depend on it?

SimonHeybrock · 2025-05-23T10:31:11Z

Thanks! I think we can actually remove the dependency completely. @nvaytet do you know why we depend on it?

I think there are some tests of the _repl_html_ that use it?

jl-wynen · 2025-05-23T10:57:26Z

Grepping for it doesn't show anything. So if it is used, then indirectly.

nvaytet · 2025-05-23T11:01:19Z

I think there are some tests of the repl_html that use it?

I think those test were removed a long time ago.
There is a simple way to check. We can remove it and see if tests pass?

SimonHeybrock · 2025-05-23T11:01:34Z

@jl-wynen

scipp/tests/html/html_repr_test.py

Line 126 in 0d221d1

from bs4 import BeautifulSoup

jl-wynen · 2025-05-23T11:04:20Z

@jl-wynen

scipp/tests/html/html_repr_test.py

Line 126 in 0d221d1

from bs4 import BeautifulSoup

Huh, my grep tool seems to be glitching out, it doesn't find this.

doronbehar · 2025-05-23T11:05:55Z

I ran the tests with this patch and they passed.

MridulS · 2025-05-23T11:22:38Z

We should also update https://github.com/scipp/scipp/blob/main/requirements/test.in and then run tox -e deps in the root directory to lock it. We can do it in a new PR too :)

doronbehar · 2025-05-23T11:51:34Z

We should also update main/requirements/test.in and then run tox -e deps in the root directory to lock it. We can do it in a new PR too :)

I removed other references to bs4 python dependency, and I ran tox -e deps and I got a lot of updates I'm hesitant to push here, as I barely know this tooling:

diff --git i/requirements/asv.txt w/requirements/asv.txt
index 825c3d2b5..65043050e 100644
--- i/requirements/asv.txt
+++ w/requirements/asv.txt
@@ -1,9 +1,9 @@
 # SHA1:1bade1c6e89445fcf3f060d8ca122826d55d4fc8
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 -r base.txt
 -r extra.txt
@@ -17,9 +17,9 @@ distlib==0.3.9
     # via virtualenv
 filelock==3.18.0
     # via virtualenv
-importlib-metadata==8.6.1
+importlib-metadata==8.7.0
     # via asv-runner
-json5==0.10.0
+json5==0.12.0
     # via asv
 pympler==1.1
     # via asv
@@ -29,11 +29,7 @@ pyyaml==6.0.2
     # via asv
 tabulate==0.9.0
     # via asv
-tomli==2.2.1
-    # via
-    #   asv
-    #   build
-virtualenv==20.29.3
+virtualenv==20.31.2
     # via asv
 zipp==3.21.0
     # via importlib-metadata
diff --git i/requirements/base.txt w/requirements/base.txt
index 600f2f418..5643c2df1 100644
--- i/requirements/base.txt
+++ w/requirements/base.txt
@@ -1,9 +1,9 @@
 # SHA1:0bc619b1257b8664bf5447ca8b9cc95742c1db60
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 numpy==2.0.2
     # via -r base.in
diff --git i/requirements/build.txt w/requirements/build.txt
index ca9300c22..851fb68c6 100644
--- i/requirements/build.txt
+++ w/requirements/build.txt
@@ -1,17 +1,17 @@
 # SHA1:3cd35e519e45950f4891894c79763dbcc5a20bf7
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 bottle==0.12.25
     # via conan
-certifi==2025.1.31
+certifi==2025.4.26
     # via requests
-charset-normalizer==3.4.1
+charset-normalizer==3.4.2
     # via requests
-cmake==3.31.6
+cmake==4.0.2
     # via -r build.in
 colorama==0.4.6
     # via conan
@@ -27,7 +27,7 @@ jinja2==3.1.6
     # via conan
 markupsafe==3.0.2
     # via jinja2
-ninja==1.11.1.3
+ninja==1.11.1.4
     # via -r build.in
 node-semver==0.6.1
     # via conan
diff --git i/requirements/ci.txt w/requirements/ci.txt
index 230929a76..6a75cbc45 100644
--- i/requirements/ci.txt
+++ w/requirements/ci.txt
@@ -1,17 +1,17 @@
 # SHA1:6344d52635ea11dca331a3bc6eb1833c4c64d585
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 cachetools==5.5.2
     # via tox
-certifi==2025.1.31
+certifi==2025.4.26
     # via requests
 chardet==5.2.0
     # via tox
-charset-normalizer==3.4.1
+charset-normalizer==3.4.2
     # via requests
 colorama==0.4.6
     # via tox
@@ -27,32 +27,26 @@ gitpython==3.1.44
     # via -r ci.in
 idna==3.10
     # via requests
-packaging==24.2
+packaging==25.0
     # via
     #   -r ci.in
     #   pyproject-api
     #   tox
-platformdirs==4.3.7
+platformdirs==4.3.8
     # via
     #   tox
     #   virtualenv
-pluggy==1.5.0
+pluggy==1.6.0
     # via tox
-pyproject-api==1.9.0
+pyproject-api==1.9.1
     # via tox
 requests==2.32.3
     # via -r ci.in
 smmap==5.0.2
     # via gitdb
-tomli==2.2.1
-    # via
-    #   pyproject-api
-    #   tox
-tox==4.24.2
+tox==4.26.0
     # via -r ci.in
-typing-extensions==4.12.2
-    # via tox
-urllib3==2.3.0
+urllib3==2.4.0
     # via requests
-virtualenv==20.29.3
+virtualenv==20.31.2
     # via tox
diff --git i/requirements/coverage.txt w/requirements/coverage.txt
index 5298b7262..e3cdeb216 100644
--- i/requirements/coverage.txt
+++ w/requirements/coverage.txt
@@ -1,12 +1,12 @@
 # SHA1:38651cf74695bf5b62507f06df0d32302b6ca62e
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 -r test.txt
-coverage[toml]==7.7.0
+coverage[toml]==7.8.2
     # via pytest-cov
-pytest-cov==6.0.0
+pytest-cov==6.1.1
     # via -r coverage.in
diff --git i/requirements/docs.txt w/requirements/docs.txt
index 69ca9738a..3bb19c220 100644
--- i/requirements/docs.txt
+++ w/requirements/docs.txt
@@ -1,9 +1,9 @@
 # SHA1:eece35641da8575d06bfbf50895c57b350ead703
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 -r base.txt
 -r extra.txt
@@ -20,13 +20,13 @@ babel==2.17.0
     # via
     #   pydata-sphinx-theme
     #   sphinx
-beautifulsoup4==4.13.3
+beautifulsoup4==4.13.4
     # via
     #   nbconvert
     #   pydata-sphinx-theme
 bleach[css]==6.2.0
     # via nbconvert
-contourpy==1.3.1
+contourpy==1.3.2
     # via matplotlib
 cycler==0.12.1
     # via matplotlib
@@ -40,13 +40,13 @@ docutils==0.21.2
     #   sphinx
 fastjsonschema==2.21.1
     # via nbformat
-fonttools==4.56.0
+fonttools==4.58.0
     # via matplotlib
 gitdb==4.0.12
     # via gitpython
 gitpython==3.1.44
     # via -r docs.in
-hypothesis==6.129.4
+hypothesis==6.131.23
     # via -r docs.in
 imagesize==1.4.1
     # via sphinx
@@ -54,7 +54,7 @@ ipydatawidgets==4.3.5
     # via pythreejs
 ipympl==0.9.7
     # via -r docs.in
-ipywidgets==8.1.5
+ipywidgets==8.1.7
     # via
     #   -r docs.in
     #   ipydatawidgets
@@ -68,11 +68,11 @@ jinja2==3.1.6
     #   sphinx
 jsonschema==4.23.0
     # via nbformat
-jsonschema-specifications==2024.10.1
+jsonschema-specifications==2025.4.1
     # via jsonschema
 jupyterlab-pygments==0.3.0
     # via nbconvert
-jupyterlab-widgets==3.0.13
+jupyterlab-widgets==3.0.15
     # via ipywidgets
 kiwisolver==1.4.8
     # via matplotlib
@@ -86,7 +86,7 @@ markupsafe==3.0.2
     # via
     #   jinja2
     #   nbconvert
-matplotlib==3.10.1
+matplotlib==3.10.3
     # via
     #   ipympl
     #   plopp
@@ -111,15 +111,15 @@ nbsphinx==0.9.7
     # via -r docs.in
 pandocfilters==1.5.1
     # via nbconvert
-pillow==11.1.0
+pillow==11.2.1
     # via
     #   ipympl
     #   matplotlib
-plopp==25.3.0
+plopp==25.5.0
     # via -r docs.in
 pydata-sphinx-theme==0.16.1
     # via -r docs.in
-pyparsing==3.2.1
+pyparsing==3.2.3
     # via matplotlib
 pythreejs==2.4.2
     # via -r docs.in
@@ -129,17 +129,17 @@ referencing==0.36.2
     # via
     #   jsonschema
     #   jsonschema-specifications
-rpds-py==0.23.1
+rpds-py==0.25.1
     # via
     #   jsonschema
     #   referencing
 smmap==5.0.2
     # via gitdb
-snowballstemmer==2.2.0
+snowballstemmer==3.0.1
     # via sphinx
 sortedcontainers==2.4.0
     # via hypothesis
-soupsieve==2.6
+soupsieve==2.7
     # via beautifulsoup4
 sphinx==8.1.3
     # via
@@ -167,13 +167,16 @@ sphinxcontrib-serializinghtml==2.0.0
     # via sphinx
 tinycss2==1.4.0
     # via bleach
-tomli==2.2.1
-    # via sphinx
 traittypes==0.2.1
     # via ipydatawidgets
+typing-extensions==4.13.2
+    # via
+    #   beautifulsoup4
+    #   pydata-sphinx-theme
+    #   referencing
 webencodings==0.5.1
     # via
     #   bleach
     #   tinycss2
-widgetsnbextension==4.0.13
+widgetsnbextension==4.0.14
     # via ipywidgets
diff --git i/requirements/extra.txt w/requirements/extra.txt
index 31d72d513..9b6938339 100644
--- i/requirements/extra.txt
+++ w/requirements/extra.txt
@@ -1,25 +1,23 @@
 # SHA1:86bc4a1850680b70c0e1929f7fa566f413f57b06
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 -r base.txt
 asttokens==3.0.0
     # via stack-data
-certifi==2025.1.31
+certifi==2025.4.26
     # via requests
-charset-normalizer==3.4.1
+charset-normalizer==3.4.2
     # via requests
 comm==0.2.2
     # via ipykernel
-debugpy==1.8.13
+debugpy==1.8.14
     # via ipykernel
 decorator==5.2.1
     # via ipython
-exceptiongroup==1.2.2
-    # via ipython
 executing==2.2.0
     # via stack-data
 graphviz==0.20.3
@@ -30,8 +28,10 @@ idna==3.10
     # via requests
 ipykernel==6.29.5
     # via -r extra.in
-ipython==8.34.0
+ipython==9.2.0
     # via ipykernel
+ipython-pygments-lexers==1.1.1
+    # via ipython
 jedi==0.19.2
     # via ipython
 jupyter-client==8.6.3
@@ -48,9 +48,9 @@ matplotlib-inline==0.1.7
     #   ipython
 nest-asyncio==1.6.0
     # via ipykernel
-numba==0.61.0
+numba==0.61.2
     # via -r extra.in
-packaging==24.2
+packaging==25.0
     # via
     #   ipykernel
     #   pooch
@@ -63,39 +63,41 @@ parso==0.8.4
     # via jedi
 pexpect==4.9.0
     # via ipython
-platformdirs==4.3.7
+platformdirs==4.3.8
     # via
     #   jupyter-core
     #   pooch
 pooch==1.8.2
     # via -r extra.in
-prompt-toolkit==3.0.50
+prompt-toolkit==3.0.51
     # via ipython
 ptyprocess==0.7.0
     # via pexpect
 pure-eval==0.2.3
     # via stack-data
 pygments==2.19.1
-    # via ipython
+    # via
+    #   ipython
+    #   ipython-pygments-lexers
 python-dateutil==2.9.0.post0
     # via
     #   jupyter-client
     #   pandas
-pytz==2025.1
+pytz==2025.2
     # via pandas
-pyzmq==26.3.0
+pyzmq==26.4.0
     # via
     #   ipykernel
     #   jupyter-client
 requests==2.32.3
     # via pooch
-scipy==1.15.2
+scipy==1.15.3
     # via -r extra.in
 six==1.17.0
     # via python-dateutil
 stack-data==0.6.3
     # via ipython
-tornado==6.4.2
+tornado==6.5.1
     # via
     #   ipykernel
     #   jupyter-client
@@ -107,13 +109,11 @@ traitlets==5.14.3
     #   jupyter-client
     #   jupyter-core
     #   matplotlib-inline
-typing-extensions==4.12.2
-    # via ipython
-tzdata==2025.1
+tzdata==2025.2
     # via pandas
-urllib3==2.3.0
+urllib3==2.4.0
     # via requests
 wcwidth==0.2.13
     # via prompt-toolkit
-xarray==2025.3.0
+xarray==2025.4.0
     # via -r extra.in
diff --git i/requirements/mypy.txt w/requirements/mypy.txt
index 074171aae..1273d0546 100644
--- i/requirements/mypy.txt
+++ w/requirements/mypy.txt
@@ -1,13 +1,13 @@
 # SHA1:e528cdd6a92a8eccd7060a9cdcab0560c1a0ab50
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 -r docs.txt
 -r test.txt
 mypy==1.15.0
     # via -r mypy.in
-mypy-extensions==1.0.0
+mypy-extensions==1.1.0
     # via mypy
diff --git i/requirements/static.txt w/requirements/static.txt
index bf84eda07..b2c56f97e 100644
--- i/requirements/static.txt
+++ w/requirements/static.txt
@@ -1,9 +1,9 @@
 # SHA1:5a0b1bb22ae805d8aebba0f3bf05ab91aceae0d8
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 cfgv==3.4.0
     # via pre-commit
@@ -11,15 +11,15 @@ distlib==0.3.9
     # via virtualenv
 filelock==3.18.0
     # via virtualenv
-identify==2.6.9
+identify==2.6.10
     # via pre-commit
 nodeenv==1.9.1
     # via pre-commit
-platformdirs==4.3.7
+platformdirs==4.3.8
     # via virtualenv
 pre-commit==4.2.0
     # via -r static.in
 pyyaml==6.0.2
     # via pre-commit
-virtualenv==20.29.3
+virtualenv==20.31.2
     # via pre-commit
diff --git i/requirements/test.txt w/requirements/test.txt
index df0a479c3..cedfd3b8b 100644
--- i/requirements/test.txt
+++ w/requirements/test.txt
@@ -1,23 +1,23 @@
-# SHA1:3aee726967b9135c2c45566b757cee9fa193f83a
+# SHA1:ce98b825493523025587d8adbf776bc8c4eec838
 #
-# This file is autogenerated by pip-compile-multi
+# This file was generated by pip-compile-multi.
 # To update, run:
 #
-#    pip-compile-multi
+#    requirements upgrade
 #
 -r base.txt
 -r extra.txt
 attrs==25.3.0
     # via hypothesis
-beautifulsoup4==4.13.3
+beautifulsoup4==4.13.4
     # via -r test.in
 execnet==2.1.1
     # via pytest-xdist
-hypothesis==6.129.4
+hypothesis==6.131.23
     # via -r test.in
 iniconfig==2.1.0
     # via pytest
-pluggy==1.5.0
+pluggy==1.6.0
     # via pytest
 pytest==8.3.5
     # via
@@ -27,7 +27,7 @@ pytest-xdist==3.6.1
     # via -r test.in
 sortedcontainers==2.4.0
     # via hypothesis
-soupsieve==2.6
+soupsieve==2.7
+    # via beautifulsoup4
+typing-extensions==4.13.2
     # via beautifulsoup4
-tomli==2.2.1
-    # via pytest

MridulS · 2025-05-23T12:01:15Z

@doronbehar no worries! the diff looks fine to me and this was part of #3697 anyway :)

doronbehar · 2025-05-28T11:57:32Z

It'd be nice to get this PR too before the next release 🙏.

Stop using the namesquat preventing bs4 pypi package

37563d2

doronbehar force-pushed the beautifulsoup4 branch from 44a6066 to 37563d2 Compare May 23, 2025 11:41

Merge branch 'main' into beautifulsoup4

d79dc77

MridulS enabled auto-merge (squash) May 28, 2025 11:53

MridulS approved these changes May 28, 2025

View reviewed changes

MridulS merged commit 474a0d5 into scipp:main May 28, 2025
4 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pyproject: Don't use the namesquatting preventing bs4 pypi package#3706

pyproject: Don't use the namesquatting preventing bs4 pypi package#3706
MridulS merged 2 commits intoscipp:mainfrom
doronbehar:beautifulsoup4

doronbehar commented May 23, 2025

Uh oh!

jl-wynen commented May 23, 2025

Uh oh!

SimonHeybrock commented May 23, 2025

Uh oh!

jl-wynen commented May 23, 2025

Uh oh!

nvaytet commented May 23, 2025

Uh oh!

SimonHeybrock commented May 23, 2025

Uh oh!

jl-wynen commented May 23, 2025

Uh oh!

doronbehar commented May 23, 2025

Uh oh!

MridulS commented May 23, 2025

Uh oh!

doronbehar commented May 23, 2025

Uh oh!

MridulS commented May 23, 2025

Uh oh!

doronbehar commented May 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

doronbehar commented May 23, 2025

Uh oh!

jl-wynen commented May 23, 2025

Uh oh!

SimonHeybrock commented May 23, 2025

Uh oh!

jl-wynen commented May 23, 2025

Uh oh!

nvaytet commented May 23, 2025

Uh oh!

SimonHeybrock commented May 23, 2025

Uh oh!

jl-wynen commented May 23, 2025

Uh oh!

doronbehar commented May 23, 2025

Uh oh!

MridulS commented May 23, 2025

Uh oh!

doronbehar commented May 23, 2025

Uh oh!

MridulS commented May 23, 2025

Uh oh!

doronbehar commented May 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants