pythonPackages.deeplabcut: init at 2.1.6 by tbenst · Pull Request #61253 · NixOS/nixpkgs

tbenst · 2019-05-10T21:21:46Z

Motivation for this change

add deeplabcut, and fix derivation for wxPython40, which is broken and unlisted: #54235. Now working thanks to help from Phoenix team wxWidgets/Phoenix#1162 and @deliciouslytyped

Things done

Resolves: #54235

deliciouslytyped · 2019-05-10T21:44:25Z

It would probably be more appropriate to separate wxPython 4 changes into a separate PR. (but I'm not sure)

tbenst · 2019-05-16T04:29:00Z

@deliciouslytyped hmm not sure either, I couldn't find relevant guidance in the contribution docs. I put as one pull request as DeepLabCut needs WxPython4 to work, and separating could lead to a broken package if merged before WxPython, but my NixOS / open source experience is limited so happy for advice!

pkgs/development/python-modules/deeplabcut/default.nix

tbenst · 2019-05-27T23:18:11Z

Thanks, @risicle, made the suggested change!

veprbl · 2019-06-01T20:48:35Z

@tbenst Could you separate wxPython fix into a separate PR? We generally prefer orthogonal changes to be in separate commits. The changes appear to be significant enough to be reviewed separately, so doing it in two separate PR's should help.

deliciouslytyped · 2019-06-01T20:50:52Z

@tbenst On that note, since you're worried about breakage on separation you could probably wait for the wxPython4 to get merged and then do the deeplabcut.

tbenst · 2019-06-05T05:53:52Z

@veprbl @deliciouslytyped sure can do. will have to wait a bit due to looming deadlines

veprbl

Reviewed wxPython part

pkgs/development/python-modules/wxPython/4.0.nix

tbenst · 2019-10-30T07:05:45Z

updated now that wxPython4 is merged. Blocked until #71282

Also need to fix the my_X pattern by instead using something like the following from pytorch derivation:

# confirm that cudatoolkits are sync'd across dependencies
assert !(openMPISupport && cudaSupport) || matchesCudatoolkit openmpi;
assert !cudaSupport || matchesCudatoolkit magma;

# confirm that mkl is sync'd across dependencies
assert !mklSupport || mkl != null;
assert !(mklSupport && cudaSupport) || matchesMkl magma;
assert !mklSupport || (numpy.blasImplementation == "mkl" && numpy.blas == mkl);

pkgs/development/python-modules/deeplabcut/default.nix

tbenst · 2020-05-15T06:58:17Z

Ok, I fixed the imgaug issues. I think this is finally ready to merge! Verified working if cherry-picked onto 20.03.

Note that numpy is currently broken on master. But fixing that is outside the scope of this PR (I hope...).

Edit: forgot to mention, I figured out the hang. It's caused by an unknown bug in OpenBLAS. Hang does not occur with MKL. It's occurs in a call to np.dot, and cannot be replicated with the inputs alone, and most tests work fine, so I don't want to force users not to use openblas.

Edit2: found a small issue, need to add wrapGapps, will push...

tbenst · 2020-06-17T06:12:47Z

ok, all the gapps & numpy issues are resolved, but now boto3 is broken on master.

ERROR: Could not find a version that satisfies the requirement botocore<1.17.0,>=1.16.23 (from boto3==1.13.23) (from versions: none)
ERROR: No matching distribution found for botocore<1.17.0,>=1.16.23 (from boto3==1.13.23)

pkgs/development/python-modules/intel-openmp/default.nix

jonringer · 2020-06-17T16:59:37Z

ok, all the gapps & numpy issues are resolved, but now boto3 is broken on master.

This is fixed in staging-next, please allow a few days for it to land in master

stale · 2021-06-04T23:55:50Z

I marked this as stale due to inactivity. → More info

SuperSandro2000

Please rebase this and familiarize yourself with the updated python docs.
If this PR gets no reaction in the next months, I am suggesting to close it.

SuperSandro2000 · 2022-11-07T23:19:25Z

pkgs/applications/science/machine-learning/deeplabcut/default.nix

+
+  script = writeText "deeplabcut"
+    ''
+      #!/usr/bin/env python


Please substitute this right away.

SuperSandro2000 · 2022-11-07T23:19:37Z

pkgs/applications/science/machine-learning/deeplabcut/default.nix

+  postInstall = ''
+    ${old.postInstall}


Suggested change

postInstall = ''

${old.postInstall}

postInstall = old.postInstall + ''

SuperSandro2000 · 2022-11-07T23:19:48Z

pkgs/applications/science/machine-learning/deeplabcut/default.nix

+  postInstall = ''
+    ${old.postInstall}
+    mkdir -p $out/bin
+    cp ${script} $out/bin/deeplabcut


Suggested change

cp ${script} $out/bin/deeplabcut

ln -s ${script} $out/bin/deeplabcut

SuperSandro2000 · 2022-11-07T23:20:05Z

pkgs/data/machine-learning/resnet/default.nix

+    meta = with stdenvNoCC.lib; {
+      description = "Pretrained weights for ResNet";
+      longDescription = ''
+        Deep residual networks, or ResNets for short, provided the breakthrough idea of identity mappings in order to enable training of very deep convolutional neural networks on ImageNet. See: Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Dec 2015.


Suggested change

Deep residual networks, or ResNets for short, provided the breakthrough idea of identity mappings in order to enable training of very deep convolutional neural networks on ImageNet. See: Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Dec 2015.

Deep residual networks, or ResNets for short, provided the breakthrough idea of identity mappings in order to enable training of very deep convolutional neural networks on ImageNet.

See: Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Dec 2015.

SuperSandro2000 · 2022-11-07T23:20:26Z

pkgs/applications/science/machine-learning/deeplabcut/default.nix

+    });
+  };
+
+  python = python37.override {


We no longer support 3.7 and version numbers should not be hardcoded like this

SuperSandro2000 · 2022-11-07T23:20:56Z

pkgs/development/python-modules/imageio-ffmpeg/default.nix

+  '';
+
+  checkPhase = ''
+    for f in tests/test*.py; do echo ========== $f ==========; python $f; done


Suggested change

for f in tests/test*.py; do echo ========== $f ==========; python $f; done

for f in tests/test*.py; do echo ========== $f ==========; ${python.interpreter} $f; done

SuperSandro2000 · 2022-11-07T23:21:06Z

pkgs/development/python-modules/imgaug/default.nix

+  # checkPhase = ''
+  #    pytest
+  # '';


We want to run them

tbenst requested a review from FRidh as a code owner May 10, 2019 21:21

ofborg bot added the 6.topic: python Python is a high-level, general-purpose programming language. label May 10, 2019

tbenst force-pushed the deeplabcut branch from 18f771e to 98bd234 Compare May 10, 2019 22:29

risicle reviewed May 25, 2019

View reviewed changes

pkgs/development/python-modules/deeplabcut/default.nix Outdated Show resolved Hide resolved

tbenst force-pushed the deeplabcut branch from 8c2864f to 34cb037 Compare May 27, 2019 22:07

deliciouslytyped mentioned this pull request Jun 1, 2019

Link failure in wxWidgets component when building wxPython 4.0 #55426

Closed

veprbl reviewed Jun 6, 2019

View reviewed changes

tbenst mentioned this pull request Jul 1, 2019

python: wxPython: init at 4.0.6 #64047

Closed

10 tasks

FRidh assigned FRidh and unassigned FRidh Jul 29, 2019

tbenst force-pushed the deeplabcut branch from 34cb037 to d21226f Compare October 30, 2019 06:56

tbenst requested a review from jonringer as a code owner October 30, 2019 06:56

veprbl reviewed Oct 30, 2019

View reviewed changes

jonringer reviewed Nov 5, 2019

View reviewed changes

pkgs/development/python-modules/deeplabcut/default.nix Outdated Show resolved Hide resolved

pkgs/development/python-modules/deeplabcut/default.nix Outdated Show resolved Hide resolved

tbenst force-pushed the deeplabcut branch from d21226f to 6351a70 Compare November 6, 2019 19:39

jonringer reviewed Nov 6, 2019

View reviewed changes

pkgs/development/python-modules/deeplabcut/default.nix Outdated Show resolved Hide resolved

pkgs/development/python-modules/deeplabcut/default.nix Outdated Show resolved Hide resolved

veprbl reviewed Nov 6, 2019

View reviewed changes

pkgs/development/python-modules/deeplabcut/default.nix Outdated Show resolved Hide resolved

tbenst force-pushed the deeplabcut branch from 6351a70 to eb67abc Compare November 6, 2019 23:37

tbenst changed the title ~~pythonPackages.deeplabcut: init at 2.0.5~~ pythonPackages.deeplabcut: init at 2.1.3 Nov 6, 2019

tbenst force-pushed the deeplabcut branch from 3965bc1 to bfa5866 Compare May 15, 2020 06:55

tensorpack: init at 0.9.8

6c1154f

tbenst force-pushed the deeplabcut branch 2 times, most recently from 2b35fc4 to 794626b Compare June 17, 2020 06:09

ofborg bot requested review from CMCDragonkai and Rakesh4G June 17, 2020 06:17

ofborg bot added 10.rebuild-linux: 11-100 This PR causes between 11 and 100 packages to rebuild on Linux. and removed 10.rebuild-linux: 101-500 This PR causes between 101 and 500 packages to rebuild on Linux. labels Jun 17, 2020

veprbl reviewed Jun 17, 2020

View reviewed changes

pkgs/development/python-modules/intel-openmp/default.nix Outdated Show resolved Hide resolved

tbenst and others added 7 commits June 17, 2020 10:18

pythonPackages.intel-openmp: init at 2020.0.133

12c781b

python3Packages.imageio-ffmpeg: 0.3.0 -> 0.4.1

c2fb88c

python3Packages.imageio: patch tests for imageio-ffmpeg version bump

34aeb26

resnet: init at 2016-08-28

91f21c6

pythonPackages.deeplabcut: init at 2.1.6.4

5a5d24b

deeplabcut: init at 2.1.6.4

2672fb3

pythonPackages.imgaug: 0.3.0 -> 0.4.0

0a108d0

tbenst force-pushed the deeplabcut branch from 794626b to 0a108d0 Compare June 17, 2020 17:18

veprbl added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 20, 2020

ryantm added 2.status: merge conflict This PR has merge conflicts with the target branch and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Oct 3, 2020

stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 4, 2021

SuperSandro2000 requested changes Nov 7, 2022

View reviewed changes

stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Nov 7, 2022

Artturin closed this Feb 2, 2023

	postInstall = ''
	${old.postInstall}
	postInstall = old.postInstall + ''

	cp ${script} $out/bin/deeplabcut
	ln -s ${script} $out/bin/deeplabcut

	Deep residual networks, or ResNets for short, provided the breakthrough idea of identity mappings in order to enable training of very deep convolutional neural networks on ImageNet. See: Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Dec 2015.
	Deep residual networks, or ResNets for short, provided the breakthrough idea of identity mappings in order to enable training of very deep convolutional neural networks on ImageNet.
	See: Deep Residual Learning for Image Recognition by Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun, Dec 2015.

	for f in tests/test*.py; do echo ========== $f ==========; python $f; done
	for f in tests/test*.py; do echo ========== $f ==========; ${python.interpreter} $f; done

Uh oh!

Conversation

tbenst commented May 10, 2019 • edited by veprbl Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation for this change

Things done

Uh oh!

deliciouslytyped commented May 10, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tbenst commented May 16, 2019

Uh oh!

Uh oh!

tbenst commented May 27, 2019

Uh oh!

veprbl commented Jun 1, 2019

Uh oh!

deliciouslytyped commented Jun 1, 2019

Uh oh!

tbenst commented Jun 5, 2019

Uh oh!

veprbl left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tbenst commented Oct 30, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

tbenst commented May 15, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tbenst commented Jun 17, 2020

Uh oh!

Uh oh!

jonringer commented Jun 17, 2020

Uh oh!

stale bot commented Jun 4, 2021

Uh oh!

SuperSandro2000 left a comment

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

Uh oh!

SuperSandro2000 Nov 7, 2022

Choose a reason for hiding this comment

tbenst commented May 10, 2019 •

edited by veprbl

Loading

deliciouslytyped commented May 10, 2019 •

edited

Loading

tbenst commented Oct 30, 2019 •

edited

Loading

tbenst commented May 15, 2020 •

edited

Loading