flake 8 : formatting issues by alalazo · Pull Request #1 · adamjstewart/spack

alalazo · 2016-05-11T06:43:34Z

This PR is just a convenient way to let you know what I do locally to solve flake8 issues:

$ pwd
/home/mculpo/PycharmProjects/spack

$ git remote -v
...
upstream    https://github.com/LLNL/spack.git (fetch)
upstream    https://github.com/LLNL/spack.git (push)

$ git branch -vv
...
* features/elk                             3a37074 [adamjstewart/features/elk: ahead 1] flake 8 : formatting issues

$ git diff upstream/develop... --name-only | perl -ne 'print if /\.py$/'
var/spack/repos/builtin/packages/elk/package.py

$ yapf -i var/spack/repos/builtin/packages/elk/package.py

$ python2 -m flake8 --format pylint --config flake8.ini var/spack/repos/builtin/packages/elk/package.py

yapf is a tool to format automatically python code (see discussion in spack#337). We have a configuration in the root directory of spack that should satisfy the formatting request of flake8.ini.

davydden · 2016-05-11T06:52:44Z

should this be wrapped in a spack command? Similar to what brew audit <package> does?

alalazo · 2016-05-11T07:06:41Z

@davydden I think @tgamblin is already considering your suggestion

adamjstewart · 2016-05-11T15:17:39Z

I just discovered the following Vim setting for Vim 7.3+

set colorcolumn=80  " highlight column 80

Definitely useful for finding long lines

adamjstewart · 2016-05-11T17:10:07Z

@alalazo: I just installed yapf and flake8 and used them on spack#558 and it worked wonderfully. Thanks!

adamjstewart · 2016-05-11T17:10:48Z

I'm going to close this, but I would agree with some kind of spack audit or spack autoformat command.

tgamblin · 2016-05-11T17:27:56Z

@adamjstewart: I reworked the Travis support last night so that it calls share/spack/qa/run-flake8. I think working that into a spack audit command would be great though I'm not sure when I can do it. The audit command could also check for things like FIXME in modified packages.

tgamblin · 2016-05-11T17:28:30Z

@adamjstewart: oh and in case it wasn't clear, for now you should be able to just run share/spack/qa/run-flake8 from the repo root and it'll do the same thing as Travis.

PEP8 Goodness

Plumed

Verification build [email protected]:pr.ucx.spack $ spack install ucx % gcc @ 4.8.5 ==> Installing libsigsegv ==> Installing m4 ==> Installing pkgconf ==> Installing ncurses ==> Installing readline ==> Installing gdbm ==> Installing perl ==> Installing autoconf ==> Installing automake ==> Installing libtool ==> Installing numactl ==> Installing zlib ==> Installing openssl ==> Installing diffutils ==> Installing bzip2 ==> Installing xz ==> Installing libxml2 ==> Installing tar ==> Installing gettext ==> Installing help2man ==> Installing bison ==> Installing flex ==> Installing libnl ==> Installing rdma-core ==> Installing ucx ==> Successfully installed ucx Fetch: 1.17s. Build: 40.59s. Total: 41.76s. [+] /scratch/users/dantopa/new-spack/pr.ucx.spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/ucx-1.4.0-ilfonxj2f5my4ur65li3sr7kuvfg3kim [email protected]:pr.ucx.spack $ spack spec ucx Input spec -------------------------------- ucx Concretized -------------------------------- [email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] patches=592f30f7f5f757dfc239ad0ffd39a9a048487ad803c26b419e0f96b8cda08c1a arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] patches=3877ab548f88597ab2327a2230ee048d2d07ace1062efe81fc92e91b7f39cd00,c0a408fbffb7255fcc75e26bd8edab116fc81d216bfd18b473668b7739a4158e,fc9b61654a3ba1a8d6cd78ce087e7c96366c290bc8d2c299f09828d793b853c8 +sigsegv arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected]+cpanm patches=0eac10ed90aeb0459ad8851f88081d439a4e41978e586ec743069e8b059370ac +shared+threads arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected]~symlinks~termlib arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^rdma-core@20%[email protected] build_type=RelWithDebInfo arch=linux-centos7-x86_64 ^[email protected]%[email protected]~doc+ncurses+openssl+ownlibs patches=dd3a40d4d92f6b2158b87d6fb354c277947c776424aa03f6dc8096cf3135f5d0 ~qt arch=linux-centos7-x86_64 ^[email protected]%[email protected]+systemcerts arch=linux-centos7-x86_64 ^[email protected]%[email protected]+optimize+pic+shared arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected]+bzip2+curses+git~libunistring+libxml2 patches=9acdb4e73f67c241b5ef32505c9ddf7cf6884ca8ea661692f21dca28483b04b8 +tar+xz arch=linux-centos7-x86_64 ^[email protected]%[email protected]+shared arch=linux-centos7-x86_64 ^[email protected]%[email protected]~python arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected] arch=linux-centos7-x86_64 ^[email protected]%[email protected]+lex arch=linux-centos7-x86_64 Environment: [email protected]:pr.ucx.spack $ echo $HOSTNAME darwin-fe2.lanl.gov [email protected]:pr.ucx.spack $ grep -i 'model name' /proc/cpuinfo | sort | uniq model name : Intel(R) Xeon(R) CPU E5-2695 v3 @ 2.30GHz [email protected]:pr.ucx.spack $ lsb_release -d -bash: lsb_release: command not found [email protected]:pr.ucx.spack $ uname -a Linux darwin-fe2.lanl.gov 4.18.8-1.el7.elrepo.x86_64 #1 SMP Sat Sep 15 10:10:09 EDT 2018 x86_64 x86_64 x86_64 GNU/Linux [email protected]:pr.ucx.spack $ date Mon Nov 19 10:47:47 MST 2018 [email protected]:pr.ucx.spack $ pwd /scratch/users/dantopa/new-spack/pr.ucx.spack Signed-off-by: Daniel Topa <[email protected]>

Whenever attempting to use any ncurses functionality within cscope, a page fault would result within the ncurses library. Program received signal SIGSEGV, Segmentation fault. 0x00007ffff7fad3cf in termattrs_sp () from .../lib/libncursesw.so.6 (gdb) bt #0 0x00007ffff7fad3cf in termattrs_sp () from .../lib/libncursesw.so.6 #1 0x00007ffff7faa794 in _nc_setupscreen_sp () from .../lib/libncursesw.so.6 #2 0x00007ffff7fa614c in newterm_sp () from .../lib/libncursesw.so.6 #3 0x00007ffff7fa65b9 in newterm () from .../lib/libncursesw.so.6 #4 0x00007ffff7fa2970 in initscr () from .../lib/libncursesw.so.6 #5 0x0000000000403dc2 in main (argc=<optimized out>, argv=0x7fffffffcea8) at main.c:574 This is due to a conflict between libtinfo.so and libtinfow.so. Both are linked into cscope: $ ldd $(which cscope) /bin/bash: .../lib/libtinfo.so.6: no version information available (required by /bin/bash) linux-vdso.so.1 (0x00007fff5dbcb000) libncursesw.so.6 => .../lib/libncursesw.so.6 (0x00007f435cc69000) libtinfo.so.6 => .../lib/libtinfo.so.6 (0x00007f435cc2c000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f435ca29000) libtinfow.so.6 => .../lib/libtinfow.so.6 (0x00007f435c9e8000) /lib64/ld-linux-x86-64.so.2 (0x00007f435cca7000) Specifically linking libtinfow.so instead of libtinfo.so resolves the issue. All instances of '...' above represent the path to the installed ncurses for Spack.

The r-devtools package was not installable due to a few issues. - The rstudioapi spec was for 0.11.0 but the rstudioapi version is actually 0.11. This caused an error during concretization. - Set r-usethis to depend on [email protected]: rather than [email protected]. - Set r-usethis to depend on [email protected]: rather than [email protected]. - Added version r-gh-1.1.0 as it is not currently present in spack.

Before: ``` $ hyperfine '~/spack/bin/spack -e . build-env rocfft' Benchmark #1: ~/spack/bin/spack -e . build-env rocfft Time (mean ± σ): 1.593 s ± 0.016 s [User: 1.468 s, System: 0.126 s] Range (min … max): 1.575 s … 1.628 s 10 runs ``` After: ``` $ hyperfine '~/spack/bin/spack -e . build-env rocfft' Benchmark #1: ~/spack/bin/spack -e . build-env rocfft Time (mean ± σ): 1.407 s ± 0.020 s [User: 1.280 s, System: 0.127 s] Range (min … max): 1.393 s … 1.455 s 10 runs ```

Git 2.24 introduced a feature flag for repositories with many files, see: https://github.blog/2019-11-03-highlights-from-git-2-24/#feature-macros Since Spack's Git repository contains roughly 8,500 files, it can be worthwhile to enable this, especially on slow file systems such as NFS: ``` $ hyperfine --warmup 3 'cd spack-default; git status' 'cd spack-manyfiles; git status' Benchmark #1: cd spack-default; git status Time (mean ± σ): 3.388 s ± 0.095 s [User: 256.2 ms, System: 625.8 ms] Range (min … max): 3.168 s … 3.535 s 10 runs Benchmark #2: cd spack-manyfiles; git status Time (mean ± σ): 168.7 ms ± 10.9 ms [User: 98.6 ms, System: 126.1 ms] Range (min … max): 144.8 ms … 188.0 ms 19 runs Summary 'cd spack-manyfiles; git status' ran 20.09 ± 1.42 times faster than 'cd spack-default; git status' ```

Merge after a few months

@omsai

1. support version 3.1.3, which now depends on sundials@6 2. support version 3.1.2:, which broke the two patch files and therefore the two patch files have been replaced by more flexible filter_file() commands inside a patch() function. 3. rename the variant for python extension from using the package name "+pyuqtk" to the more standard "+python" 4. add maintainers @omsai and the upstream developer @bjdebus who offered to help with the spack packaging. 5. swig should only be a build-time dependency. swig is only necessary until @:3.1.0 6. confirmed python dependencies are correct by inspecting imports, subset python dependencies type to build, run, and confirmed all 31 build-time tests pass including the 9 python tests: ```console $ spack env create uqtk-dev $ spack add [email protected] $ spack install --test root && cat $(spack location -i uqtk)/.spack/install-time-test-log.txt ==> Testing package uqtk-3.1.3-nok6fut ==> [2023-04-19-14:56:25.005361] Running build-time tests ==> [2023-04-19-14:56:25.005536] RUN-TESTS: build-time tests [check] ==> [2023-04-19-14:56:25.009543] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'test' ==> [2023-04-19-14:56:25.014903] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' 'test' Running tests... /home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/cmake-3.26.3-zjmsfz23j5l4ytniz26uzvxonlu5qebr/bin/ctest --force-new-ctest-process Test project /tmp/omsai/spack-stage/spack-stage-uqtk-3.1.3-nok6fut47h42cnaau7wkoohgqy5f2qqa/spack-build-nok6fut Start 1: ArrayReadAndWrite Start 2: ArrayDelColumn Start 3: Array1DMiscTest Start 4: Array2DMiscTest 1/31 Test #1: ArrayReadAndWrite ................ Passed 0.01 sec Start 5: ArraySortTest 2/31 Test #2: ArrayDelColumn ................... Passed 0.01 sec Start 6: MultiIndexTest 3/31 Test #3: Array1DMiscTest .................. Passed 0.01 sec Start 7: CorrTest 4/31 Test #4: Array2DMiscTest .................. Passed 0.01 sec Start 8: QuadLUTest 5/31 Test #5: ArraySortTest .................... Passed 0.02 sec Start 9: MCMC2dTest 6/31 Test #6: MultiIndexTest ................... Passed 0.01 sec Start 10: MCMCRandomTest 7/31 Test #8: QuadLUTest ....................... Passed 0.02 sec Start 11: MCMCNestedTest 8/31 Test #10: MCMCRandomTest ................... Passed 0.02 sec Start 12: Deriv1dTest 9/31 Test #12: Deriv1dTest ...................... Passed 0.01 sec Start 13: SecondDeriv1dTest 10/31 Test #13: SecondDeriv1dTest ................ Passed 0.01 sec Start 14: GradHessianTest 11/31 Test #11: MCMCNestedTest ................... Passed 0.03 sec Start 15: GradientPCETest 12/31 Test #14: GradHessianTest .................. Passed 0.01 sec Start 16: PCE1dTest 13/31 Test #15: GradientPCETest .................. Passed 0.01 sec Start 17: PCEImplTest 14/31 Test #16: PCE1dTest ........................ Passed 0.01 sec Start 18: PCELogTest 15/31 Test #18: PCELogTest ....................... Passed 0.01 sec Start 19: Hessian2dTest 16/31 Test #19: Hessian2dTest .................... Passed 0.01 sec Start 20: BCS1dTest 17/31 Test #20: BCS1dTest ........................ Passed 0.01 sec Start 21: BCS2dTest 18/31 Test #21: BCS2dTest ........................ Passed 0.01 sec Start 22: LowRankRegrTest 19/31 Test #22: LowRankRegrTest .................. Passed 0.01 sec Start 23: PyModTest 20/31 Test #17: PCEImplTest ...................... Passed 0.07 sec Start 24: PyArrayTest 21/31 Test #23: PyModTest ........................ Passed 0.08 sec Start 25: PyArrayTest2 22/31 Test #25: PyArrayTest2 ..................... Passed 0.30 sec Start 26: PyQuadTest 23/31 Test #24: PyArrayTest ...................... Passed 1.44 sec Start 27: PyBCSTest1D 24/31 Test #26: PyQuadTest ....................... Passed 1.68 sec Start 28: PyBCSTest2D 25/31 Test #27: PyBCSTest1D ...................... Passed 1.66 sec Start 29: PyBADPTest 26/31 Test #7: CorrTest ......................... Passed 3.43 sec Start 30: PyRegressionTest 27/31 Test #28: PyBCSTest2D ...................... Passed 1.50 sec Start 31: PyGalerkinTest 28/31 Test #9: MCMC2dTest ....................... Passed 3.90 sec 29/31 Test #29: PyBADPTest ....................... Passed 1.66 sec 30/31 Test #30: PyRegressionTest ................. Passed 1.72 sec 31/31 Test #31: PyGalerkinTest ................... Passed 1.63 sec 100% tests passed, 0 tests failed out of 31 Total Test time (real) = 5.35 sec ==> [2023-04-19-14:56:30.382797] '/home/omsai/src/spack/opt/spack/linux-pureos10-skylake/gcc-10.2.1/gmake-4.4.1-b6g4apmfvxz3bn4eabh37dehcrg65fj7/bin/make' '-j4' '-n' 'check' ==> [2023-04-19-14:56:30.385983] Target 'check' not found in Makefile ```

flake 8 : formatting issues

3a37074

alalazo mentioned this pull request May 11, 2016

Elk: Use new openmp_flag and mpi attributes spack/spack#932

Merged

adamjstewart closed this May 11, 2016

adamjstewart mentioned this pull request May 11, 2016

Provide new package: MOAB spack/spack#938

Merged

alalazo deleted the features/elk branch May 12, 2016 06:20

adamjstewart pushed a commit that referenced this pull request Aug 2, 2016

Merge pull request #1 from robertdfrench/olcf/unified_cray_platform

07f2dd5

PEP8 Goodness

adamjstewart pushed a commit that referenced this pull request Aug 30, 2016

Merge pull request #1 from ARCCA/plumed

4b354cb

Plumed

adamjstewart pushed a commit that referenced this pull request Nov 15, 2022

Merge pull request #1 from spack/develop

02c7e51

Merge after a few months

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

flake 8 : formatting issues#1

flake 8 : formatting issues#1
alalazo wants to merge 1 commit intoadamjstewart:features/elkfrom
alalazo:features/elk

alalazo commented May 11, 2016

Uh oh!

davydden commented May 11, 2016

Uh oh!

alalazo commented May 11, 2016

Uh oh!

adamjstewart commented May 11, 2016

Uh oh!

adamjstewart commented May 11, 2016

Uh oh!

adamjstewart commented May 11, 2016

Uh oh!

tgamblin commented May 11, 2016

Uh oh!

tgamblin commented May 11, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

alalazo commented May 11, 2016

Uh oh!

davydden commented May 11, 2016

Uh oh!

alalazo commented May 11, 2016

Uh oh!

adamjstewart commented May 11, 2016

Uh oh!

adamjstewart commented May 11, 2016

Uh oh!

adamjstewart commented May 11, 2016

Uh oh!

tgamblin commented May 11, 2016

Uh oh!

tgamblin commented May 11, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants