-
Notifications
You must be signed in to change notification settings - Fork 6.1k
Buggy import path resolution in corner cases with remappings and base path #11036
Description
Inspired by #9353.
The way the compiler resolves imports into actual filesystem paths is intuitive in simple cases but gets convoluted quickly when you take into account all the factors that can affect it: remappings, base path, various flavors of relative paths, special file names, etc. I checked how it works in corner cases and in a lot of them the beahavior is either buggy or at least does not work the way I would expect it to.
Issues
This section lists problems I have identified. See the reference at the end of the post for detailed examples.
- Base path is sometimes prepended to absolute paths.
- Base path is prepended to source directory path in imports starting with
.or... - Base path is prepended to remapped absolute imports.
- Base path is prepended to source directory path in imports starting with
- Imports starting with
.or..are not always relative to the source directory- Absolute imports can be remapped into imports starting with
.or..but they then become relative to the current working directory. - Base path replaces source directory in absolute imports remapped to relative imports starting with
.or...- It should behave the same way as relative imports starting with
..or...
- It should behave the same way as relative imports starting with
- Paths that become relative after stripping of
file://are always relative to the current working directory.- This does not happen when base paths is used.
- Absolute imports can be remapped into imports starting with
- Using a relative path to the source file on the command line sometimes makes relative imports resolve to different paths than when using an absolute path.
- Remapping of
.and..works differently when the path to the source file given to the compiler is relative. - Remappings ignore any number of leading
.and..segments when the path to the source file given to the compiler is relative.
- Remapping of
- Relative paths with
..segments that go beyond filesystem root are silently replaced with valid ones.- Remappings ignore any number of leading
.and..segments in imports going beyond the root directory. - Remappings ignore any number of leading
.and..segments when the path to the source file given to the compiler is relative. - Relative imports going beyond root directory ignore
..and are relative to the current working directory (or to the base directory when base path is used).
- Remappings ignore any number of leading
- It's possible to remap some of the paths added implicitly to relative imports.
- Parts of the path that
.and..get resolved into can get remapped. - Source directory path can be remapped.
- Parts of the path that
- Not all prefixes can be remapped.
- Leading
.and..in imports cannot be remapped. file://prefix as a whole cannot be remapped but its fragments can.
- Leading
- Special behavior of
<stdin>.- A file called
<stdin>from current working directory can only be imported if use of the standard input is not requested.- If a file called
<stdin>exists and standard input was requested, importing it should be an error.
- If a file called
- Behavior of
<stdin>in general might be unintended though there's probably no harm in allowing it to be remapped.
- A file called
- Trailing slashes in imports are replaced with
/.(unless they start with.or..).- They should either be stripped or replaced with a single slash.
- Multiple remappings of the same prefix are ignored and the last one takes effect.
Current path resolution behavior
This section is meant mostly as a reference. There's no need to read it all to understand the issue. Skimming the headers should give you a good picture of what is happening.
Use it to see examples of the problems I listed above or just the current behavior in specific cases.
Common setup for all examples below
- File containing the imports:
/project/contract.sol. - Working directory:
/home/user/work/.
Simple imports
Absolute
import "/tmp/code/token.sol"; // -> /tmp/code/token.sol
import "/tmp/code/../code/token.sol"; // -> /tmp/code/token.solRelative imports are relative to current working directory
import "token/token.sol"; // -> /home/user/work/token/token.sol
import ".../token/token.sol"; // -> /home/user/work/.../token/token.solRelative imports starting with . or .. are relative to the source directory
import "./token/token.sol"; // -> /project/token/token.sol
import "../project/token/token.sol"; // -> /project/token/token.solRelative imports going beyond root directory ignore .. and are relative to the current working directory
import "token/token.sol"; // -> /home/user/work/token/token.sol
import "../token/token.sol"; // -> /home/user/token/token.sol
import "../../token/token.sol"; // -> /home/token/token.sol
import "../../../token/token.sol"; // -> /token/token.sol
import "../../../../token/token.sol"; // -> /home/user/work/token/token.sol
import "../../../../../token/token.sol"; // -> /home/user/work/token/token.sol
import "../../.././../../token/token.sol"; // -> /home/user/work/token/token.solPath Remapping
Absolute imports can be remapped
// Remappings: /tmp/code=/usr/lib
import "/tmp/code/token.sol"; // -> /usr/lib/token.solRelative imports can be remapped
// Remappings: token=contract
import "token/token.sol"; // -> /home/user/work/contract/token.solRelative imports can be remapped into absolute imports
// Remappings: token=/usr/lib
import "token/token.sol"; // -> /usr/lib/token.solAbsolute imports can be remapped into imports relative to the current working directory
// Remappings: /usr/lib=contract
import "/usr/lib/token.sol"; // -> /home/user/work/contract/token.solAbsolute imports can be remapped into imports starting with . or .. but they are also relative to the current working directory
Normally imports starting with . or .. are relative to the source directory.
// Remappings: /usr/lib=./contract
import "/usr/lib/token.sol"; // -> /home/user/work/contract/token.sol// Remappings: /usr/lib=../contract
import "/usr/lib/token.sol"; // -> /home/user/contract/token.solOnly prefix can be remapped
// Remappings: /usr=/tmp /lib=/bin contracts=token token.sol=contract.sol
import "/usr/usr/lib/token.sol"; // -> /tmp/usr/lib/token.sol
import "contracts/contracts/usr/lib/token.sol"; // -> /home/user/work/token/contracts/usr/lib/token.sol
import "token.sol"; // -> /home/user/work/contract.solThe remapping with the longest matching prefix is used
// Remappings: /usr=/project/dex /usr/lib=/project/token contracts=tokens contracts/token.sol=dex.sol
import "/usr/lib/contracts/token.sol"; // -> /project/token/contracts/token.sol
import "contracts/token.sol"; // -> /home/user/work/dex.solRemapping is not recursive
// Remappings: /a=/b /b=/c /c=/a
import "/a/token.sol"; // -> /b/token.solThe last remapping takes precedence
// Remappings: /a=/b /a=/c /a=/d
import "/a/token.sol"; // -> /d/token.solRemappings do not have to match whole path segments
// Remappings: /c=/k c=k
import "/contracts/contract.sol"; // -> /kontracts/contract.sol
import "contracts/contract.sol"; // -> /home/user/work/kontracts/contract.solLeading . and .. in imports cannot be remapped
// Remappings: .=/tmp
import "./token/token.sol"; // -> /project/token/token.sol
import "../token/token.sol"; // -> /project/token.sol
import ".../token/token.sol"; // -> /tmp../token/token.sol// Remappings: ./token=contract ../token=contract .../token=contract
import "./token/token.sol"; // -> /project/token/token.sol
import "../token/token.sol"; // -> /project/token.sol
import ".../token/token.sol"; // -> /home/user/work/contract/token.solParts of the path that . and .. get resolved into can get remapped
// Remappings: /project=/tmp /token=/tmp /project/token.sol=/tmp/dex.sol
import "./token/token.sol"; // -> /tmp/token/token.sol
import "../token/token.sol"; // -> /tmp/token.sol
import "./token.sol"; // -> /tmp/dex.solRemapping of . and .. works differently when the path to the source file given to the compiler is relative
This is one of the cases where the path to the file containing imports (specified on the compiler's command line) affects how the imports are resolved.
The effect is different depending on the locations of the source directory and the current working directory relative to each other.
Example: Current working directory is a prefix of the source directory
- Source file passed to the compiler:
../contract.sol. - Working directory:
/project/subdir/.
// Remappings: ./token=contract ../token=contract
import "./token/token.sol"; // -> /home/user/work/contract/token.sol
import "../token/token.sol"; // -> /home/user/work/token.solPaths are not canonicalized before remapping
// Remappings: /tmp/code=/usr/lib token=contract
import "/tmp/../tmp/code/token.sol"; // -> /tmp/code/token.sol
import "code/../token/token.sol"; // -> /home/user/work/token/token.solRemappings ignore any number of leading . and .. segments in imports going beyond the root directory
// Remappings: token=contract
import "token/token.sol"; // -> /home/user/work/contract/token.sol
import "../token/token.sol"; // -> /home/user/token/token.sol
import "../../token/token.sol"; // -> /home/token/token.sol
import "../../../token/token.sol"; // -> /token/token.sol
import "../../../../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../../../../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../.././../../token/token.sol"; // -> /home/user/work/contract/token.solRemappings ignore any number of leading . and .. segments when the path to the source file given to the compiler is relative
The effect is different depending on the locations of the source directory and the current working directory relative to each other.
Example: Current working directory is a prefix of the source directory
- Source file passed to the compiler:
../contract.sol. - Working directory:
/project/subdir/.
// Remappings: token=contract
import "token/token.sol"; // -> /home/user/work/contract/token.sol
import "../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../../../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../../../../token/token.sol"; // -> /home/user/work/contract/token.sol
import "../../.././../../token/token.sol"; // -> /home/user/work/contract/token.solCurrent working directory path cannot be remapped
// Remappings: /home/user/work=/workdir
import "token/token.sol"; // -> /home/user/work/token/token.solSource directory path can be remapped
// Remappings: /project=/tmp
import "./token/token.sol"; // -> /tmp/token/token.sol
import "../project/token/token.sol"; // -> /tmp/token/token.solBase Path
Base path does not affect absolute imports
// Base path: /base
import "/tmp/code/token.sol"; // -> /tmp/code/token.sol
import "/tmp/code/../code/token.sol"; // -> /tmp/code/token.solBase path replaces current working directory in relative imports
// Base path: /base
import "token/token.sol"; // -> /base/token/token.solBase path is prepended to source directory path in imports starting with . or ..
// Base path: /base
import "./token/token.sol"; // -> /base/project/token/token.sol
import "../token/token.sol"; // -> /base/token/token.solRelative imports going beyond root directory ignore .. and are relative to the base directory
// Base path: /base
import "token/token.sol"; // -> /base/token/token.sol
import "../token/token.sol"; // -> /base/token/token.sol
import "../../token/token.sol"; // -> /base/token/token.sol
import "../../../token/token.sol"; // -> /base/token/token.sol
import "../../../../token/token.sol"; // -> /base/token/token.sol
import "../../../../../token/token.sol"; // -> /base/token/token.sol
import "../../.././../../token/token.sol"; // -> /base/token/token.solBase path is prepended to remapped absolute imports
// Base path: /base
// Remappings: /tmp/code=/usr/lib
import "/tmp/code/token.sol"; // -> /base/usr/lib/token.solBase path replaces current working directory in remapped relative imports
// Base path: /base
// Remappings: token=contract
import "token/token.sol"; // -> /base/contract/token.solBase path is prepended to relative imports remapped into absolute imports
// Base path: /base
// Remappings: token=/usr/lib
import "token/token.sol"; // -> /base/usr/lib/token.solBase path replaces current working directory in absolute imports remapped to relative imports
// Base path: /base
// Remappings: /usr/lib=contract
import "/usr/lib/token.sol"; // -> /base/contract/token.solBase path replaces source directory in absolute imports remapped to relative imports starting with . or ..
Without remapping such imports are relative to the source directory, not the current working directory.
// Base path: /base
// Remappings: /usr/lib=./contract
import "/usr/lib/token.sol"; // -> /base/contract/token.sol// Base path: /base
// Remappings: /usr/lib=../contract
import "/usr/lib/token.sol"; // -> /contract/token.solParts of the path that . and .. get resolved into can get remapped when base path is used and such remappings don't override the base path
// Base path: /base
// Remappings: /project=/tmp /token=/tmp /project/token.sol=/tmp/dex.sol
import "./token/token.sol"; // -> /base/tmp/token/token.sol
import "../token/token.sol"; // -> /base/tmp/token.sol
import "./token.sol"; // -> /base/tmp/dex.solBase path cannot be remapped
// Base path: /base
// Remappings: /base=/tmp
import "token/token.sol"; // -> /base/token/token.solSource directory path can be remapped even when base path is used
// Base path: /base
// Remappings: /project=/tmp /base=/usr/lib
import "./token/token.sol"; // -> /base/tmp/token/token.sol
import "../project/token/token.sol"; // -> /base/tmp/token/token.solRelative base path is relative to the current working directory
// Base path: base
import "token/token.sol"; // -> /home/user/work/base/token/token.sol
import "./token/token.sol"; // -> /home/user/work/base/project/token/token.sol
import "../token/token.sol"; // -> /home/user/work/base/token/token.sol// Base path: .
import "token/token.sol"; // -> /home/user/work/token/token.sol
import "./token/token.sol"; // -> /home/user/work/project/token/token.sol
import "../token/token.sol"; // -> /home/user/work/token/token.sol// Base path: ..
import "token/token.sol"; // -> /home/user/token/token.sol
import "./token/token.sol"; // -> /home/user/project/token/token.sol
import "../token/token.sol"; // -> /home/user/token/token.solStandard input
Standard input can be used as if it was an actual file called <stdin> if requested
// Command line: contains `-`
import "<stdin>"; // -> stdinIf there is an actual file called in the current directory, it's ignored.
A file called <stdin> from current working directory can only be imported if use of the standard input is not requested.
// Command line: does not contain `-`
import "<stdin>"; // -> /home/user/work/<stdin><stdin> can be remapped to a path when it represents standard input
// Command line: contains `-`
// Remappings: <stdin>=/tmp/code/token.sol
import "<stdin>"; // -> /tmp/code/token.sol<stdin> can be remapped to a path when it represents a file
// Command line: does not contain `-`
// Remappings: <stdin>=/tmp/code/token.sol
import "<stdin>"; // -> /tmp/code/token.solPaths can be remapped to <stdin> when it represents standard input
// Command line: contains `-`
// Remappings: /tmp/code/token.sol=<stdin>
import "/tmp/code/token.sol"; // -> stdinPaths can be remapped to <stdin> when it represents a file
// Command line: does not contain `-`
// Remappings: /tmp/code/token.sol=<stdin>
import "/tmp/code/token.sol"; // -> /home/user/work/<stdin>Base path does not affect <stdin> when it represents standard input
// Command line: contains `-`
// Base path: /base
import "<stdin>"; // -> stdinBase path affects <stdin> when it does not represent standard input
// Command line: does not contain `-`
// Base path: /base
import "<stdin>"; // -> /base/<stdin>Extra Slashes
Multiple slashes between path segments in imports are squashed into one
import "/tmp//code/token.sol"; // -> /tmp/code/token.sol
import "/tmp////code/token.sol"; // -> /tmp/code/token.solTrailing slashes in imports are replaced with /.
import "/tmp/code/token.sol/"; // -> /tmp/code/token.sol/.
import "/tmp/code/token.sol//"; // -> /tmp/code/token.sol/.
import "/tmp/code/token.sol////"; // -> /tmp/code/token.sol/.Trailing slashes in relative imports that start with . or .. imports are stripped
import "./token/token.sol/"; // -> /project/token/token.sol
import "./token/token.sol//"; // -> /project/token/token.sol
import "./token/token.sol////"; // -> /project/token/token.sol
import "../token/token.sol/"; // -> /project/token.sol
import "../token/token.sol//"; // -> /project/token.sol
import "../token/token.sol////"; // -> /project/token.solTwo Leading slashes in imports are preserved
import "//tmp/code/token.sol"; // -> //tmp/code/token.solMore than two Leading slashes in imports are squashed into one
import "///tmp/code/token.sol"; // -> /tmp/code/token.sol
import "////tmp/code/token.sol"; // -> /tmp/code/token.sol
import "/////tmp/code/token.sol"; // -> /tmp/code/token.solSlashes after the leading . or .. in relative imports are squashed into one
import "./token/token.sol"; // -> /project/token/token.sol
import ".//token/token.sol"; // -> /project/token/token.sol
import ".///token/token.sol"; // -> /project/token/token.sol
import ".////token/token.sol"; // -> /project/token/token.sol
import "..//token/token.sol"; // -> /token/token.sol
import "..///token/token.sol"; // -> /token/token.sol
import "..////token/token.sol"; // -> /token/token.sol
import "../////token/token.sol"; // -> /token/token.solSlashes in base path follow the same rules as slashes in imports
// Base path: ////base1////base2/////base3////
import "token/token.sol"; // -> /base1/base2/base3/token/token.solSlashes in remappings must match exactly
// Remappings: /tmp/code=/usr/lib
import "/tmp//code/token.sol"; // -> /tmp/code/token.sol
import "/tmp/code/token.sol/"; // -> /usr/lib/token.sol/
import "/tmp/code/token.sol//"; // -> /usr/lib/token.sol/
import "//tmp/code/token.sol"; // -> //tmp/code/token.sol
import "///tmp/code/token.sol"; // -> /tmp/code/token.sol// Remappings: /tmp/code/token.sol//=/usr/lib/token.sol
import "/tmp/code/token.sol"; // -> /usr/lib/token.sol
import "/tmp/code/token.sol/"; // -> /usr/lib/token.sol/.
import "/tmp/code/token.sol//"; // -> /usr/lib/token.sol
import "/tmp/code/token.sol///"; // -> /usr/lib/token.sol/.Multiple trailing slashes in remapping targets are squashed into one
// Remappings: /tmp/code=////usr////lib////
import "/tmp/code/token.sol"; // -> /usr/lib/token.solURLs
file:// prefix is stripped from paths
import "file:///tmp/code/token.sol"; // -> /tmp/code/token.sol
import "/tmp/code/file://token.sol"; // -> /tmp/code/file:/token.solOther protocol prefixes are not stripped
import "http:///tmp/code/token.sol"; // -> /home/user/work/http:/tmp/code/token.sol
import "https:///tmp/code/token.sol"; // -> /home/user/work/https:/tmp/code/token.sol
import "ftp:///tmp/code/token.sol"; // -> /home/user/work/ftp:/tmp/code/token.solPaths that become relative after stripping of file:// are always relative to the current working directory
import "file://token.sol"; // -> /home/user/work/token.sol
import "file://./token.sol"; // -> /home/user/work/token.sol
import "file://../token.sol"; // -> /home/user/token.solBase path is prepended to paths that become relative after stripping of file://
// Base path: /base
import "file://token.sol"; // -> /base/home/user/work/token.solPrepending base path to paths that start with . or .. after stripping of file:// makes them relative to the source directory
// Base path: /base
import "file://./token.sol"; // -> /base/project/token.sol
import "file://../token.sol"; // -> /base/token.solfile:// prefix as a whole cannot be remapped
// Remappings: file://=/usr/lib
import "file:///tmp/code/token.sol"; // -> /tmp/code/token.sol// Remappings: file:/=/usr/lib
import "file:///tmp/code/token.sol"; // -> /tmp/code/token.solFragments of the file:// prefix can be remapped
// Remappings: file=/usr/lib
import "file:///tmp/code/token.sol"; // -> /usr/lib:/tmp/code/token.sol// Remappings: f=/usr/lib
import "file:///tmp/code/token.sol"; // -> /usr/libile:/tmp/code/token.sol