Fix Aggregator, TypeAssigner, LocalNameStandardizer by swissiety · Pull Request #741 · soot-oss/SootUp

swissiety · 2023-11-03T13:50:04Z

TypeAssigner: 'found' Bottomtype to UnknownType
LocalNameStanrdizer: dont handle Local Types where no standard Type is given
fix Aggregator doing changes on Stmts like:
from (AggregatorTest.testIssue739()):

    $l0 := @parameter0: java.lang.String[];
    $stack4 = $l0[0];
    $stack5 = staticinvoke <java.lang.Integer: java.lang.Integer valueOf(java.lang.String)>($stack4);
    $l1 = virtualinvoke $stack5.<java.lang.Integer: int intValue()>();

to (which is wrong; missing staticinvoke, wrong cast to Integer insetad of String)

    $l0 := @parameter0: java.lang.String[];
    $stack5 = $l0[0];
    $#l0 = (java.lang.Integer) $stack5;
    $l1 = virtualinvoke $#l0.<java.lang.Integer: int intValue()>();

fix Aggregator to not remove a needed Stmt in AggregatorTest.testIssue739():

    $stack6 = $l1 + $l1;   // removed
    $stack7 = $stack6 + $l1;

closes #739

…simple Local assignment

TypeAssigner: change augmentation in cases of BottomType to unknownType

…Value"

…method finding ;)

swissiety · 2023-11-06T16:12:29Z

@JonasKlauke SourcteType already works in finding the entryMethod ;-)

codecov · 2023-11-06T16:17:25Z

Codecov Report

Attention: 90 lines in your changes are missing coverage. Please review.

Comparison is base (f2e1465) 63.74% compared to head (cc8a35a) 63.64%.

Additional details and impacted files

@@              Coverage Diff              @@
##             develop     #741      +/-   ##
=============================================
- Coverage      63.74%   63.64%   -0.11%     
+ Complexity      3367     3360       -7     
=============================================
  Files            317      317              
  Lines          15145    15124      -21     
  Branches        2555     2551       -4     
=============================================
- Hits            9654     9625      -29     
- Misses          4601     4608       +7     
- Partials         890      891       +1

Files	Coverage Δ
...a/sootup/callgraph/AbstractCallGraphAlgorithm.java	`89.63% <100.00%> (ø)`
...jimple/common/expr/AbstractInstanceInvokeExpr.java	`92.30% <100.00%> (+5.64%)`	⬆️
...ore/jimple/common/stmt/AbstractDefinitionStmt.java	`81.25% <100.00%> (ø)`
...tup/core/typehierarchy/MethodDispatchResolver.java	`86.36% <100.00%> (-0.13%)`	⬇️
...utlocation/JavaClassPathAnalysisInputLocation.java	`55.55% <100.00%> (+0.71%)`	⬆️
...ytecode/interceptors/BytecodeBodyInterceptors.java	`0.00% <ø> (ø)`
...ootup/java/bytecode/interceptors/TypeAssigner.java	`100.00% <100.00%> (+40.00%)`	⬆️
...tecode/interceptors/UnreachableCodeEliminator.java	`100.00% <100.00%> (ø)`
...tecode/interceptors/typeresolving/CastCounter.java	`82.47% <100.00%> (ø)`
.../java/sootup/core/typehierarchy/TypeHierarchy.java	`78.57% <0.00%> (-2.08%)`	⬇️
... and 10 more

... and 3 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

virusdefender · 2023-11-07T15:02:42Z

Run my code with this branch ( and with BytecodeClassLoadingOptions.Default ), if some type class is not found, Could not find xxx in View Exception will be thrown.

Target_jsp.class.zip

java.lang.IllegalStateException: Failed to apply sootup.java.bytecode.interceptors.TypeAssigner@16eccb2e to <Target_jsp: org.apache.tomcat.InstanceManager _jsp_getInstanceManager()>
        at sootup.java.bytecode.frontend.AsmMethodSource.resolveBody(AsmMethodSource.java:229)
        at sootup.core.model.SootMethod.lazyBodyInitializer(SootMethod.java:98)
        at com.google.common.base.Suppliers$NonSerializableMemoizingSupplier.get(Suppliers.java:183)
        at sootup.core.model.SootMethod.getBody(SootMethod.java:177)
        at sootup.core.util.printer.JimplePrinter.printMethods(JimplePrinter.java:242)
        at sootup.core.util.printer.JimplePrinter.printTo(JimplePrinter.java:210)
        at com.cloudwalker.javawebshell.main.SootUpCommand.run(Main.kt:311)
        at picocli.CommandLine.executeUserObject(CommandLine.java:1939)
        at picocli.CommandLine.access$1300(CommandLine.java:145)
        at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2358)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2352)
        at picocli.CommandLine$RunLast.handle(CommandLine.java:2314)
        at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
        at picocli.CommandLine$RunLast.execute(CommandLine.java:2316)
        at picocli.CommandLine.execute(CommandLine.java:2078)
       
Caused by: java.lang.IllegalArgumentException: Could not find org.apache.jasper.runtime.HttpJspBase in View.
        at sootup.core.views.View.lambda$getClassOrThrow$0(View.java:90)
        at java.base/java.util.Optional.orElseThrow(Optional.java:403)
        at sootup.core.views.View.getClassOrThrow(View.java:89)
        at sootup.core.typehierarchy.ViewTypeHierarchy.sootClassFor(ViewTypeHierarchy.java:421)
        at sootup.core.typehierarchy.ViewTypeHierarchy.superClassOf(ViewTypeHierarchy.java:302)
        at sootup.java.bytecode.interceptors.typeresolving.BytecodeHierarchy.buildAncestryPaths(BytecodeHierarchy.java:236)
        at sootup.java.bytecode.interceptors.typeresolving.BytecodeHierarchy.getLeastCommonAncestor(BytecodeHierarchy.java:170)
        at sootup.java.bytecode.interceptors.typeresolving.TypeResolver.applyAssignmentConstraint(TypeResolver.java:209)
        at sootup.java.bytecode.interceptors.typeresolving.TypeResolver.resolve(TypeResolver.java:65)
        at sootup.java.bytecode.interceptors.TypeAssigner.interceptBody(TypeAssigner.java:42)
        at sootup.java.bytecode.frontend.AsmMethodSource.resolveBody(AsmMethodSource.java:226)
        ... 15 more

While I indeed didn't add the corresponding jar of this class to the classpath, isn't it sufficient to construct the type information in Jimple only relying on the information in the class file?

…ierarchy.contains() <!=> view.getClass().isPresent() as signatures could be referenced from other classes which then end up in the underlying graph of the typehierarchy

swissiety · 2023-11-07T17:08:08Z

@virusdefender thx, this should be fixed in #b270b08

virusdefender · 2023-11-07T23:29:51Z

thanks, it's much better now than before, but there are still a few small issues

method return value is not assigned to variable

unknown $stack0;

staticinvoke <javax.servlet.jsp.JspFactory: javax.servlet.jsp.JspFactory getDefaultFactory()>();
<$package.Target_jsp: javax.servlet.jsp.JspFactory _jspxFactory> = $stack0;

the return value of getDefaultFactory is not assigned to stack0

staticinvoke <java.lang.Runtime: java.lang.Runtime getRuntime()>();
...
$r4 = virtualinvoke $stack24.<java.lang.Runtime: java.lang.Process exec(java.lang.String[])>($r17);

the same case, stack24 should be the return value of getRuntime()

`SomeType varName = null`

unknown $_jspx_out, $_jspx_page_context, $stack24;

_jspx_out, _jspx_page_context are assigned to type and value null at first

I try to rebase this pr with #742 but the problem still exists

…g. an Expr is a use as well

…is used in *another* essential stmt

…eed to filter for essential stmts again in the second step

swissiety added 4 commits November 3, 2023 14:41

cleanup srcType..

1007485

add test input and enable interceptors

3d7a38c

handle only operands that we can handle

d0c977d

check if a Local is used *somewhere* not just on the rigth side of a …

1a1b66f

…simple Local assignment

swissiety changed the title ~~Fix aggregator~~ Fix Aggregator, TypeAssigner, LocalNameStandardizer Nov 6, 2023

swissiety added 3 commits November 6, 2023 12:02

LocalNameStandardizer: dont rename Local if Type is not handled;

5489b09

TypeAssigner: change augmentation in cases of BottomType to unknownType

fix: dont replace the rhs of a non-invoke completely by the aggregatee

6a94685

fix value replacement in aggregation via a hacky "ValueBox.canContain…

7b5dc03

…Value"

swissiety requested a review from JonasKlauke November 6, 2023 13:03

swissiety added 3 commits November 6, 2023 14:04

cleanup

12dcf42

adapting test to casts with non-field local names /add $ prefix

894fbf9

fix test: adding name standardizer

3e5e94b

swissiety marked this pull request as ready for review November 6, 2023 13:50

SourceType.Application vs SourceType.Libarary works already with main…

316e537

…method finding ;)

swissiety added 2 commits November 7, 2023 17:00

tame another class not in the view exception throwing; it seems typeh…

b270b08

…ierarchy.contains() <!=> view.getClass().isPresent() as signatures could be referenced from other classes which then end up in the underlying graph of the typehierarchy

try if test is not executed because of naming convention

fadcf43

ignore - previously ignored testcase

c4da4d4

swissiety added 8 commits November 8, 2023 11:38

fix Body.collectUses() signature to use value instead of LValue as e.…

05afa28

…g. an Expr is a use as well

Merge branch 'develop' into fix/739_Aggregator

62fa162

fix porting bug in DeadAssignmentEliminator: checks now if a def/use …

f557de2

…is used in *another* essential stmt

cleanup

080bfd7

fix early execution end conditions

d462a66

work directly on essentialStmts while collecting uses - removes the n…

0a10b59

…eed to filter for essential stmts again in the second step

break early

edbcc4b

cleanup

59fef4d

swissiety added 3 commits November 9, 2023 13:16

correct visibility

94feb00

dont print in testcase

f684a98

don't distinguish explicitly between incomplete/complete superclassesof

cc8a35a

kadirayk approved these changes Nov 9, 2023

View reviewed changes

kadirayk merged commit 23503d7 into develop Nov 9, 2023

swissiety deleted the fix/739_Aggregator branch November 9, 2023 14:06

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix Aggregator, TypeAssigner, LocalNameStandardizer#741

Fix Aggregator, TypeAssigner, LocalNameStandardizer#741
kadirayk merged 25 commits intodevelopfrom
fix/739_Aggregator

swissiety commented Nov 3, 2023 •

edited

Loading

Uh oh!

swissiety commented Nov 6, 2023

Uh oh!

codecov bot commented Nov 6, 2023 •

edited

Loading

Uh oh!

virusdefender commented Nov 7, 2023

Uh oh!

swissiety commented Nov 7, 2023

Uh oh!

virusdefender commented Nov 7, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

swissiety commented Nov 3, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

swissiety commented Nov 6, 2023

Uh oh!

codecov bot commented Nov 6, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

virusdefender commented Nov 7, 2023

Uh oh!

swissiety commented Nov 7, 2023

Uh oh!

virusdefender commented Nov 7, 2023

method return value is not assigned to variable

SomeType varName = null

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

swissiety commented Nov 3, 2023 •

edited

Loading

codecov bot commented Nov 6, 2023 •

edited

Loading

`SomeType varName = null`