Skip to content

Comments

code refactoring to provide abstracted library#182

Merged
ehsandeep merged 14 commits intodevfrom
issue-168-uncover-refactoring
Apr 11, 2023
Merged

code refactoring to provide abstracted library#182
ehsandeep merged 14 commits intodevfrom
issue-168-uncover-refactoring

Conversation

@xm1k3
Copy link
Contributor

@xm1k3 xm1k3 commented Mar 20, 2023 •
edited
Loading 

func main() {
	// Parse the command line flags and read config files
	options := runner.ParseOptions()

	agentFactory, err := runner.New(options)
	if err != nil {
		gologger.Fatal().Msgf("Could not create runner: %s\n", err)
	}
	newRunner, err := runner.NewRunner(options, agentFactory)
	if err != nil {
		gologger.Fatal().Msgf("Could not create runner: %s\n", err)
	}

	err = newRunner.Run(context.Background())
	if err != nil {
		gologger.Fatal().Msgf("Could not run enumeration: %s\n", err)
	}
}

@xm1k3 xm1k3 self-assigned this Mar 20, 2023
@xm1k3 xm1k3 added the Type: Enhancement Most issues will probably ask for additions or changes. label Mar 20, 2023
@xm1k3 xm1k3 linked an issue Mar 20, 2023 that may be closed by this pull request
Refactor uncover and provide abstracted library (similar to tlsx) #168
Closed
@xm1k3
Copy link
Contributor Author

xm1k3 commented Mar 22, 2023 

agentFactory, err := runner.New(options)
if err != nil {
	gologger.Fatal().Msgf("Could not create runner: %s\n", err)
}

@xm1k3 xm1k3 marked this pull request as ready for review March 22, 2023 14:30
@ehsandeep ehsandeep requested a review from tarunKoyalwar March 22, 2023 16:53
@tarunKoyalwar tarunKoyalwar marked this pull request as draft March 28, 2023 20:58
@tarunKoyalwar tarunKoyalwar added the Status: In Progress This issue is being worked on, and has someone assigned. label Mar 28, 2023
@github-advanced-security
Copy link

github-advanced-security bot commented Mar 29, 2023

You have successfully added a new CodeQL configuration .github/workflows/codeql-analysis.yml:analyze/language:go. As part of the setup process, we have scanned this repository and found 1 existing alert. Please check the repository Security tab to see all alerts.

github-advanced-security[bot]
github-advanced-security bot found potential problems Mar 29, 2023
View reviewed changes
sources/session.go
MaxIdleConns: 100,
MaxIdleConnsPerHost: 100,
TLSClientConfig: &tls.Config{
InsecureSkipVerify: true,

Check failure

Code scanning / CodeQL

Disabled TLS certificate check

InsecureSkipVerify should not be used in production code.
@tarunKoyalwar
Copy link
Member

tarunKoyalwar commented Mar 29, 2023 

$ go run . -s "ssl:'hackerone.com'" -v

  __  ______  _________ _   _____  _____
 / / / / __ \/ ___/ __ \ | / / _ \/ ___/
/ /_/ / / / / /__/ /_/ / |/ /  __/ /    
\__,_/_/ /_/\___/\____/|___/\___/_/

		projectdiscovery.io

[INF] Current uncover version v1.0.3 (latest)
[shodan] 2606:4700::6810:6434:2083
[shodan] 104.16.100.52:8443
[shodan] 104.16.99.52:8443
[shodan] 2606:4700::6810:6434:2087
[shodan] 2606:4700::6810:6334:443
[shodan] 2606:4700::6810:6334:8443
[shodan] 2606:4700::6810:6334:2087
[shodan] 104.16.99.52:2083
[shodan] 104.16.99.52:443
[shodan] 104.16.100.52:443
[shodan] 104.16.100.52:2083
[shodan] 2606:4700::6810:6434:8443
[shodan] 2606:4700::6810:6434:443
[shodan] 2606:4700::6810:6334:2083
[shodan] 164.92.231.194:443

@tarunKoyalwar tarunKoyalwar marked this pull request as ready for review March 29, 2023 13:19
@tarunKoyalwar tarunKoyalwar requested review from Mzack9999 and ehsandeep and removed request for tarunKoyalwar March 29, 2023 13:34
@tarunKoyalwar
Copy link
Member

tarunKoyalwar commented Mar 29, 2023 •
edited
Loading

Integration / Library Code

  • Now uncover is completely abstracted i.e logic/code does not need to be changed downstream (nuclei etc) if we add new/sources

uncover/examples/main.go

Lines 1 to 44 in fe9e479

package main
import (
"context"
"fmt"
"github.com/projectdiscovery/gologger"
"github.com/projectdiscovery/uncover"
"github.com/projectdiscovery/uncover/sources"
)
func main() {
opts := uncover.Options{
Agents: []string{"shodan"},
Queries: []string{"ssl:'hackerone.com'"},
Limit: 50,
MaxRetry: 2,
Timeout: 20,
}
u, err := uncover.New(&opts)
if err != nil {
panic(err)
}
allagents := u.AllAgents()
gologger.Info().Msgf("Available uncover agents/sources :")
for _, v := range allagents {
fmt.Println(v)
}
fmt.Println("\n\n- Uncover Results:")
result := func(result sources.Result) {
fmt.Println(result.IpPort())
}
// Execute executes and returns a channel with all results
// ch , err := u.Execute(context.Background())
// Execute with Callback calls u.Execute() internally and abstracts channel handling logic
if err := u.ExecuteWithCallback(context.TODO(), result); err != nil {
panic(err)
}
}

@tarunKoyalwar
Copy link
Member

tarunKoyalwar commented Mar 29, 2023

Follow up Issue

Mzack9999
Mzack9999 requested changes Mar 29, 2023
View reviewed changes
Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation: lgtm
Notes:

  • suggesting minor optional refactor
  • I think we should make it clear in the readme that now the runner is not reusable anymore with different queries, but a new instance needs to be created with new options

@xm1k3 xm1k3 requested a review from Mzack9999 March 29, 2023 17:18
Mzack9999
Mzack9999 approved these changes Mar 30, 2023
View reviewed changes
Copy link
Member

@Mzack9999 Mzack9999 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

ehsandeep
ehsandeep requested changes Apr 2, 2023
View reviewed changes
Copy link
Member

@ehsandeep ehsandeep left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This works

echo jira | ./uncover -v

but not this:

echo jira | ./uncover

@xm1k3 xm1k3 requested a review from ehsandeep April 3, 2023 13:13
@xm1k3 xm1k3 mentioned this pull request Apr 5, 2023
Merged
@tarunKoyalwar
Copy link
Member

tarunKoyalwar commented Apr 11, 2023 

$ ./uncover -q "ssl:hackerone.com" -e shodan  -v

  __  ______  _________ _   _____  _____
 / / / / __ \/ ___/ __ \ | / / _ \/ ___/
/ /_/ / / / / /__/ /_/ / |/ /  __/ /    
\__,_/_/ /_/\___/\____/|___/\___/_/

		projectdiscovery.io

[INF] Current uncover version v1.0.4-dev (dev)
[shodan] 104.16.100.52:443
[shodan] 104.16.99.52:2083
[shodan] 104.16.100.52:2083
[shodan] 2606:4700::6810:6434:8443
[shodan] 2606:4700::6810:6434:2083
[shodan] 104.16.99.52:8443
[shodan] 104.16.100.52:8443
[shodan] 2606:4700::6810:6434:443
[shodan] 2606:4700::6810:6334:2083
[shodan] 2606:4700::6810:6334:443
[shodan] 104.16.99.52:443
[shodan] 2606:4700::6810:6334:8443
[shodan] 164.92.231.194:443
$ ./uncover -q "ssl:hackerone.com" -e shodan 

  __  ______  _________ _   _____  _____
 / / / / __ \/ ___/ __ \ | / / _ \/ ___/
/ /_/ / / / / /__/ /_/ / |/ /  __/ /    
\__,_/_/ /_/\___/\____/|___/\___/_/

		projectdiscovery.io

[INF] Current uncover version v1.0.4-dev (dev)
104.16.100.52:443
104.16.99.52:2083
104.16.100.52:2083
2606:4700::6810:6434:8443
2606:4700::6810:6434:2083
104.16.99.52:8443
104.16.100.52:8443
2606:4700::6810:6434:443
2606:4700::6810:6334:2083
2606:4700::6810:6334:443
104.16.99.52:443
2606:4700::6810:6334:8443
164.92.231.194:443

@ehsandeep ehsandeep merged commit 804fb0f into dev Apr 11, 2023
@ehsandeep ehsandeep deleted the issue-168-uncover-refactoring branch April 11, 2023 13:15
@ehsandeep ehsandeep removed the Status: In Progress This issue is being worked on, and has someone assigned. label Apr 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@ehsandeep ehsandeep ehsandeep approved these changes

@Mzack9999 Mzack9999 Mzack9999 approved these changes

Assignees

@tarunKoyalwar tarunKoyalwar

@xm1k3 xm1k3

Labels

Type: Enhancement Most issues will probably ask for additions or changes.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Refactor uncover and provide abstracted library (similar to tlsx)

4 participants

@xm1k3 @tarunKoyalwar @ehsandeep @Mzack9999