Making PATH modification optional #64

Mzack9999 · 2023-02-03T11:55:49Z

Optional path modification
Supports set/unset via -sp/-up

tarunKoyalwar

@Mzack9999 , Printing PATH env variable values does not seem necessary/good idea.

./pdtm 

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.3

		projectdiscovery.io

[INF] [OS: DARWIN] [ARCH: ARM64] [GO: 1.19.4]
[INF] Path to download project binary: /Users/tarun/.pdtm/go/bin

[INF] Path /Users/tarun/.pdtm/go/bin configured in environment variable $PATH: /Users/tarun/.pdtm/go/bin,/opt/homebrew/opt/libpcap/bin,/Users/tarun/.projectdiscovery,/opt/homebrew/opt/openjdk/bin,/Users/tarun/go/bin,/opt/local/bin,/opt/local/sbin,/opt/homebrew/sbin,/opt/homebrew/bin
1. subfinder (latest) (2.5.5)
2. dnsx (latest) (1.1.1)
3. naabu (latest) (2.1.1)
4. httpx (latest) (1.2.6)
5. nuclei (latest) (2.8.8)
6. uncover (latest) (1.0.2)
7. cloudlist (latest) (1.0.2)
8. proxify (latest) (0.0.8)
9. tlsx (latest) (1.0.5)
10. notify (latest) (1.0.4)
11. chaos-client (latest) (0.4.0)
12. shuffledns (latest) (1.0.8)
13. mapcidr (latest) (1.0.3)
14. interactsh-client (latest) (1.0.7)
15. katana (latest) (0.0.3)

tarunKoyalwar · 2023-02-09T10:55:18Z

@Mzack9999 , I was thinking we could something similar to this

./pdtm 

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.3

		projectdiscovery.io

[INF] [OS: DARWIN] [ARCH: ARM64] [GO: 1.19.4]
[INF] Path to download project binary: /Users/tarun/.pdtm/go/bin
[INF] Path /Users/tarun/.pdtm/go/bin configured in environment variable $PATH: 

1. subfinder (latest) (2.5.5)
2. dnsx (latest) (1.1.1)
3. naabu (latest) (2.1.1)
4. httpx (latest) (1.2.6)
5. nuclei (latest) (2.8.8)
6. uncover (latest) (1.0.2)
7. cloudlist (latest) (1.0.2)
8. proxify (latest) (0.0.8)
9. tlsx (latest) (1.0.5)
10. notify (latest) (1.0.4)
11. chaos-client (latest) (0.4.0)
12. shuffledns (latest) (1.0.8)
13. mapcidr (latest) (1.0.3)
14. interactsh-client (latest) (1.0.7)
15. katana (latest) (0.0.3)

tarunKoyalwar

lgtm !

$ ./pdtm 

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.3

		projectdiscovery.io

[INF] [OS: DARWIN] [ARCH: ARM64] [GO: 1.19.4]
[INF] Path to download project binary: /Users/tarun/.pdtm/go/bin
[INF] Path /Users/tarun/.pdtm/go/bin configured in environment variable $PATH
1. subfinder version not found
2. dnsx version not found
3. naabu version not found
4. httpx (latest) (1.2.7)
5. nuclei version not found
6. uncover version not found
7. cloudlist version not found
8. proxify version not found
9. tlsx (latest) (1.0.5)
10. notify version not found
11. chaos-client version not found
12. shuffledns version not found
13. mapcidr version not found
14. interactsh-client version not found
15. katana (latest) (0.0.3)

@Mzack9999 since we only track/show/update tools installed via pdtm (i.e $HOME/.pdtm) Instead of printing nuclei version not found what do you think if we did something like this

$ ./pdtm 

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.3

		projectdiscovery.io

[INF] [OS: DARWIN] [ARCH: ARM64] [GO: 1.19.4]
[INF] Path to download project binary: /Users/tarun/.pdtm/go/bin
[INF] Path /Users/tarun/.pdtm/go/bin configured in environment variable $PATH

1. httpx (latest) (1.2.7)
2. tlsx (latest) (1.0.5)
3. katana (latest) (0.0.3)

[INF] Tools not tracked[installed] by pdtm

1. nuclei version not found
2. uncover version not found
3. cloudlist version not found
4. notify version not found
5. chaos-client version not found
6. shuffledns version not found
7. mapcidr version not found
8. interactsh-client version not found
9. subfinder version not found
10. dnsx version not found
11. naabu version not found

tarunKoyalwar · 2023-03-03T07:20:38Z

since there were some issues with pdtm many users installed tools via go tool adding such details/description would help and also avoid conflicts with old installations etc

tarunKoyalwar · 2023-03-08T06:48:39Z

@Mzack9999 , since path modification is disabled by default what do you think of alias ?
i.e when we install say nuclei we alias full path with with name this way there won't be any direct path modification or related concerns .

alias nuclei="/Users/tarun/.pdtm/go/bin/nuclei"

What do you think ?

Mzack9999 · 2023-03-08T12:35:08Z

Still, to make it persistent, it's necessary to store it somewhere on the system. I think we can suggest to the user the command to execute (pdtm or system commands). As a reference, homebrew indicates to the user the commands to copy/paste and execute.

Mzack9999 · 2023-03-12T00:00:31Z

Todo: (cc @tarunKoyalwar )

Path modification should be enabled (actual default behavior and expected one by users with no setup effort - even if a bit invasive)
CRUD operations for tools within $home directory
Read-only operations for tools outside $home directory

tarunKoyalwar

$  go run . -u nuclei

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

		projectdiscovery.io

parent is /Users/tarun
sub is /Users/tarun/.pdtm/go/bin
[INF] nuclei: not found in path /Users/tarun/.pdtm/go/bin/nuclei
got error here
[ERR] error while updating nuclei: tool nuclei not found in path /Users/tarun/.pdtm/go/bin/nuclei: skipping

Revisting CRUD operations other than (default PDTM PATH)

Now after trying this out , I think we don't need all CRUD operations supported for cases when path is different from default PATH

As a User, I think pdtm should detect all pd tools , install them if explicitly mentioned and update my tools (installed with pdtm and without pdtm [Keeping HOME FOLDER restriction HERE]) also I don't want pdtm to take complete ownership (DELETE PERM) of tools that were not installed via pdtm

so I think we should only support READ UPDATE for tools outside of pdtm default path . this way we are less invasive keeping User experience same . What do you think @Mzack9999

internal/runner/runner.go

Mzack9999 · 2023-03-13T23:26:15Z

@tarunKoyalwar I'm unable to reproduce it:

$ ls /Users/marcornvh/.pdtm/go/bin/nuclei 
/Users/marcornvh/.pdtm/go/bin/nuclei
$  go run . -u nuclei

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

                projectdiscovery.io

[INF] nuclei: already up to date

Testing on OSX latest

I'm not sure we should update tools arbitrarily outside of scoped directories. Eventually one improvement we might try, is to detect if they are within the go bin folder, and in such case trigger the install via go install. What do you think?

tarunKoyalwar · 2023-03-17T18:36:22Z

@Mzack9999

currently if I try to update tool not installed by pdtm (or not installed at all) it tries to update it and fails . we probably should add a check here before updating
Another change related to update that if user wants to update nuclei but it is not installed via pdtm (maybe installed via package manager or go install) we can just install a new version in .pdtm/go/bin and since PATH is added by default nuclei at .pdtm/go/bin is picked everytime and this won't be invasive at all

Mzack9999 · 2023-03-20T13:07:05Z

@tarunKoyalwar do you mean installing the tool if not present? Actually if the tool is not installed we skip all I/O operations and just return an error

tarunKoyalwar · 2023-03-20T18:45:25Z

@Mzack9999 , not installing but a additional check before updating a tool when we try to update any not existent tool

$ which dnsx
dnsx not found

 $ ./pdtm -u dnsx                           

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

		projectdiscovery.io

[INF] Current pdtm version v0.0.4 (latest)
[INF] dnsx: not found in path /Users/tarun/.pdtm/go/bin/dnsx
[ERR] error while updating dnsx: tool dnsx not found in path /Users/tarun/.pdtm/go/bin/dnsx: skipping

as you can see in above example . pdtm first tries to find if dnsx is in path . if not it should fail silently (or skip the next step). instead currently even tries to update a not existent tool and returns [Err]

I think the proper output for this should be

 $ ./pdtm -u dnsx                           

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

		projectdiscovery.io

[INF] Current pdtm version v0.0.4 (latest)
[INF] dnsx: not found in path /Users/tarun/.pdtm/go/bin/dnsx: skipping update

ehsandeep

pdtm $./pdtm -ua

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

		projectdiscovery.io

[INF] Current pdtm version v0.0.4 (latest)
[INF] subfinder: already up to date
[INF] dnsx: already up to date
[INF] naabu: already up to date
[INF] httpx: already up to date
[INF] nuclei: already up to date
[INF] uncover: already up to date
[INF] cloudlist: already up to date
[INF] proxify: already up to date
[INF] tlsx: already up to date
[INF] notify: already up to date
[INF] chaos-client: already up to date
[INF] shuffledns: already up to date
[INF] mapcidr: already up to date
[INF] interactsh-client: already up to date
[INF] katana: already up to date
[INF] pdtm: already up to date
[INF] asnmap: already up to date


pdtm $./pdtm -ia

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

		projectdiscovery.io

[INF] Current pdtm version v0.0.4 (latest)
[INF] subfinder: already installed
[INF] dnsx: already installed
[INF] naabu: already installed
[INF] httpx: already installed
[INF] nuclei: already installed
[INF] uncover: already installed
[INF] cloudlist: already installed
[INF] proxify: already installed
[INF] tlsx: already installed
[INF] notify: already installed
[INF] chaos-client: already installed
[INF] shuffledns: already installed
[INF] mapcidr: already installed
[INF] interactsh-client: already installed
[INF] katana: already installed
[INF] pdtm: already installed
[INF] asnmap: already installed

tarunKoyalwar

lgtm !

$ ./pdtm 

                ____          
     ____  ____/ / /_____ ___ 
    / __ \/ __  / __/ __ __  \
   / /_/ / /_/ / /_/ / / / / /
  / .___/\__,_/\__/_/ /_/ /_/ 
 /_/                          v0.0.4

		projectdiscovery.io

[INF] Current pdtm version v0.0.4 (latest)
[INF] [OS: DARWIN] [ARCH: ARM64] [GO: 1.20.2]
[INF] Path to download project binary: /Users/tarun/.pdtm/go/bin
[INF] Path /Users/tarun/.pdtm/go/bin configured in environment variable $PATH
1. subfinder (latest) (2.5.7)
2. dnsx (latest) (1.1.3)
3. naabu (latest) (2.1.4)
4. httpx (latest) (1.2.9)
5. nuclei (not installed)
6. uncover (latest) (1.0.3)
7. cloudlist (latest) (1.0.2)
8. proxify (latest) (0.0.8)
9. tlsx (latest) (1.0.6)
10. notify (latest) (1.0.4)
11. chaos-client (latest) (0.5.1)
12. shuffledns (latest) (1.0.8)
13. mapcidr (latest) (1.1.1)
14. interactsh-client (latest) (1.1.2)
15. katana (latest) (1.0.0)
16. pdtm (latest) (0.0.4)
17. asnmap (latest) (1.0.2)

making env path optional + add remove

cf3567c

Mzack9999 added the Type: Enhancement Most issues will probably ask for additions or changes. label Feb 3, 2023

Mzack9999 self-assigned this Feb 3, 2023

Mzack9999 marked this pull request as draft February 3, 2023 11:56

Mzack9999 added 5 commits February 3, 2023 13:25

redefining flags

1e6b8d8

add race test

294f165

removing outdated flag

04260cd

adding env var check

1c76aba

system paths handling

a4d5c1a

Mzack9999 linked an issue Feb 3, 2023 that may be closed by this pull request

Env Path variable modification should be optional #54

Closed

Mzack9999 marked this pull request as ready for review February 3, 2023 13:58

ehsandeep requested review from ehsandeep and tarunKoyalwar February 7, 2023 15:22

tarunKoyalwar requested changes Feb 9, 2023

View reviewed changes

Mzack9999 added 5 commits February 13, 2023 13:19

Merge branch 'dev' into issue-54-env-var

e1b7a1b

Merge branch 'dev' into issue-54-env-var

a6b96fd

removing path listing

094f1a2

removing double new line

d0ea9a7

fixing path lookup

be67597

Mzack9999 mentioned this pull request Mar 1, 2023

chore(deps): bump github.com/stretchr/testify from 1.8.1 to 1.8.2 #83

Merged

Mzack9999 requested a review from tarunKoyalwar March 1, 2023 14:04

tarunKoyalwar approved these changes Mar 3, 2023

View reviewed changes

Merge branch 'dev' into issue-54-env-var

e965567

Mzack9999 added 2 commits March 13, 2023 12:20

always set default path

862d842

scoping CRUD to home folder

57a4ea0

Mzack9999 requested a review from tarunKoyalwar March 13, 2023 11:45

tarunKoyalwar requested changes Mar 13, 2023

View reviewed changes

internal/runner/runner.go Show resolved Hide resolved

internal/runner/runner.go Show resolved Hide resolved

improving logic + error messages

2e5b484

ehsandeep and others added 2 commits March 19, 2023 11:30

Merge branch 'dev' into issue-54-env-var

ec807e6

misc update

51ef7d5

Mzack9999 requested a review from tarunKoyalwar March 20, 2023 13:07

better scoping path

6143324

Mzack9999 added 6 commits March 20, 2023 21:18

reducing error msg noise

049532e

fixing -install verbosity

af8edcb

fixing version comparison

703751b

correcting error msg

3b4e0e5

fixing error msg

281ba90

fixing idx offset

7cc4160

ehsandeep approved these changes Mar 20, 2023

View reviewed changes

ehsandeep modified the milestone: pdtm v0.0.5 Mar 20, 2023

update usage in README.md

3e02f6c

tarunKoyalwar approved these changes Mar 21, 2023

View reviewed changes

tarunKoyalwar requested a review from ehsandeep March 21, 2023 06:44

Update README.md

ddb3d05

ehsandeep approved these changes Mar 21, 2023

View reviewed changes

ehsandeep merged commit 181facc into dev Mar 21, 2023

ehsandeep deleted the issue-54-env-var branch March 21, 2023 06:52

dogancanbakir mentioned this pull request Feb 9, 2024

Allow tools to be installed outside $HOME in /usr/local/bin/ for example #248

Closed

Making PATH modification optional #64

Making PATH modification optional #64

Uh oh!

Conversation

Mzack9999 commented Feb 3, 2023

Uh oh!

tarunKoyalwar left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tarunKoyalwar commented Feb 9, 2023

Uh oh!

tarunKoyalwar left a comment

Choose a reason for hiding this comment

Uh oh!

tarunKoyalwar commented Mar 3, 2023

Uh oh!

tarunKoyalwar commented Mar 8, 2023

Uh oh!

Mzack9999 commented Mar 8, 2023

Uh oh!

Mzack9999 commented Mar 12, 2023

Uh oh!

tarunKoyalwar left a comment

Choose a reason for hiding this comment

Revisting CRUD operations other than (default PDTM PATH)

Uh oh!

Uh oh!

Uh oh!

Mzack9999 commented Mar 13, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

tarunKoyalwar commented Mar 17, 2023

Uh oh!

Mzack9999 commented Mar 20, 2023

Uh oh!

tarunKoyalwar commented Mar 20, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ehsandeep left a comment

Choose a reason for hiding this comment

Uh oh!

tarunKoyalwar left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

tarunKoyalwar left a comment •

edited

Loading

Mzack9999 commented Mar 13, 2023 •

edited

Loading

tarunKoyalwar commented Mar 20, 2023 •

edited

Loading