Skip to content

Commit bc3ebe0

Browse files
Kateryna Prokopenkov8-scoped@luci-project-accounts.iam.gserviceaccount.com
authored andcommitted
Revert "[parser] Create debug magic comment values as source substrings"
This reverts commit 8dcf14f. Reason for revert: - Parser::SetDebugMagicCommentFields now creates script->source_url via factory()->NewSubString(...). For any //# sourceURL=... with length >= 13 (SlicedString::kMinLength), source_url is now a SlicedString instead of an internalized SeqString. - In V8HeapExplorer::AddEntry (src/profiler/heap-snapshot-generator.cc), SlicedString nodes are emitted into the heap snapshot with the literal name "(sliced string)" (and ExtractStringReferences only emits parent and length, not offset). - As a result, ContextAnalyzer in DevTools reads node.findInternalEdgeTarget('source_url')?.name() as "(sliced string)" instead of the script's sourceURL for eval scripts (like //# sourceURL=inside-direct-eval.js) Failure Link: https://ci.chromium.org/b/8667891195092908785/test-results?q=ExactID%3Afront_end%2Fentrypoints%2Fheap_snapshot_worker%2FHeapSnapshot.test.api.ts%3Aheapsnapshot_analyze_context_fields_api_test%3Aanalyzes_context_fields+VHash%3Ae3b0c44298fc1c14&clean= Original change's description: > [parser] Create debug magic comment values as source substrings > > The values of the //# sourceURL, sourceMappingURL and debugId magic > comments were copied into LiteralBuffers while scanning and then > internalized. An inline source map (sourceMappingURL=data:...) can be > tens of MiB, so this allocated a string of that size on the heap in a > non-throwing allocation, e.g. aborting a Node.js worker with a small > old-generation limit even though the (external) source fits. > > The scanner now only records the source locations of these values, and > the values are created as substrings of the script source: > - On the main thread, when parsing finishes. > - For background and streaming compiles, where the source is not > available while parsing, the locations are kept in the > BackgroundCompileTask and the substrings are created in > FinalizeScript after the source is set. Locations outside of the > source provided by the embedder are ignored. > > The semantics are unchanged: the last comment wins, trailing garbage or > an empty value removes the value, and a source map URL provided through > the API takes precedence. > > Since these values are now usually SlicedStrings of the script source, > the code serializer serializes all sliced strings as sequential > strings (one-byte if their contents are, even if their parent is > two-byte). The script source is an attached object, and off-thread > deserialization attaches a placeholder for it, so a serialized slice > of it would not be valid. > > Consuming a code cache still materializes these values as sequential > strings, e.g. on Node.js's NODE_COMPILE_CACHE path, so a huge cached > value can still be large on the heap there. Keeping them as substrings > of the source when deserializing requires a code cache format change > and is left for a follow-up. > > This also removes the LiteralBuffer copies of the values. > > [email protected], [email protected] > > Bug: 566467883 > Change-Id: I07d8d0c36f56c7aee40793b1db272657832ab36f > Refs: nodejs/node#64155 > Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/8472768 > Reviewed-by: Leszek Swirski <[email protected]> > Reviewed-by: Patrick Thier <[email protected]> > Auto-Submit: eliau elkouby (‫אליהו אלקובי‬‎) <[email protected]> > Commit-Queue: Patrick Thier <[email protected]> > Cr-Commit-Position: refs/heads/main@{#110612} Bug: 566467883 Change-Id: Ie6326e508e0243c26a1c958e0556f622c283d144 Reviewed-on: https://chromium-review.googlesource.com/c/v8/v8/+/8574549 Reviewed-by: Patrick Thier <[email protected]> Auto-Submit: Kateryna Prokopenko <[email protected]> Commit-Queue: Kateryna Prokopenko <[email protected]> Reviewed-by: Leszek Swirski <[email protected]> Cr-Commit-Position: refs/heads/main@{#110803}
1 parent b01c7d6 commit bc3ebe0

16 files changed

Lines changed: 119 additions & 746 deletions

‎src/codegen/compiler.cc‎

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2047,9 +2047,6 @@ void BackgroundCompileTask::Run(
20472047
parser.ParseOnBackground(isolate, &info, script_, start_position_,
20482048
end_position_, function_literal_id_);
20492049
parser.UpdateStatistics(script_, &use_counts_, &total_preparse_skipped_);
2050-
if (toplevel_script_compilation) {
2051-
debug_magic_comment_locations_ = parser.debug_magic_comment_locations();
2052-
}
20532050

20542051
TRACE_EVENT(TRACE_DISABLED_BY_DEFAULT("v8.compile"),
20552052
"V8.CompileCodeBackground");
@@ -2793,14 +2790,8 @@ MaybeHandle<SharedFunctionInfo> BackgroundCompileTask::FinalizeScript(
27932790
{
27942791
DisallowGarbageCollection no_gc;
27952792
SetScriptFieldsFromDetails(isolate, *script, script_details, &no_gc);
2793+
LOG(isolate, ScriptDetails(*script));
27962794
}
2797-
// After SetScriptFieldsFromDetails, so that a source map URL provided
2798-
// through the API takes precedence. The locations come from the streamed
2799-
// source; the embedder is expected to pass the same source here, but an
2800-
// out-of-bounds location is ignored.
2801-
Parser::SetDebugMagicCommentFields(isolate, script, source,
2802-
debug_magic_comment_locations_);
2803-
LOG(isolate, ScriptDetails(*script));
28042795
}
28052796

28062797
ReportStatistics(isolate);

‎src/codegen/compiler.h‎

Lines changed: 0 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,6 @@
2424
#include "src/objects/debug-objects.h"
2525
#include "src/parsing/parse-info.h"
2626
#include "src/parsing/pending-compilation-error-handler.h"
27-
#include "src/parsing/scanner.h" // For Scanner::DebugMagicCommentLocations.
2827
#include "src/snapshot/code-serializer.h"
2928
#include "src/utils/allocation.h"
3029
#include "src/zone/zone.h"
@@ -654,9 +653,6 @@ class V8_EXPORT_PRIVATE BackgroundCompileTask {
654653
DeferredFinalizationJobDataList jobs_to_retry_finalization_on_main_thread_;
655654
base::SmallVector<v8::Isolate::UseCounterFeature, 8> use_counts_;
656655
int total_preparse_skipped_ = 0;
657-
// The source is not available on the background thread, so the debug magic
658-
// comment values are created from these locations when finalizing.
659-
Scanner::DebugMagicCommentLocations debug_magic_comment_locations_;
660656

661657
// Single function data for top-level function compilation.
662658
MaybeIndirectHandle<SharedFunctionInfo> input_shared_info_;

‎src/parsing/literal-buffer.cc‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,11 +6,29 @@
66

77
#include "src/base/strings.h"
88
#include "src/base/unique-array.h"
9+
#include "src/execution/isolate.h"
10+
#include "src/execution/local-isolate.h"
11+
#include "src/heap/factory.h"
12+
#include "src/objects/objects.h"
13+
#include "src/objects/string.h"
914
#include "src/utils/memcopy.h"
1015

1116
namespace v8 {
1217
namespace internal {
1318

19+
template <typename IsolateT>
20+
DirectHandle<String> LiteralBuffer::Internalize(IsolateT* isolate) const {
21+
if (is_one_byte()) {
22+
return isolate->factory()->InternalizeString(one_byte_literal());
23+
}
24+
return isolate->factory()->InternalizeString(two_byte_literal());
25+
}
26+
27+
template DirectHandle<String> LiteralBuffer::Internalize(
28+
Isolate* isolate) const;
29+
template DirectHandle<String> LiteralBuffer::Internalize(
30+
LocalIsolate* isolate) const;
31+
1432
size_t LiteralBuffer::NewCapacity(size_t min_capacity) {
1533
return min_capacity < (kMaxGrowth / (kGrowthFactor - 1))
1634
? min_capacity * kGrowthFactor

‎src/parsing/literal-buffer.h‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -99,6 +99,9 @@ class LiteralBuffer final {
9999
is_one_byte_ = true;
100100
}
101101

102+
template <typename IsolateT>
103+
DirectHandle<String> Internalize(IsolateT* isolate) const;
104+
102105
private:
103106
static constexpr size_t kInitialCapacity = 256;
104107
static constexpr size_t kGrowthFactor = 4;

‎src/parsing/parser.cc‎

Lines changed: 17 additions & 30 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
#include "src/common/message-template.h"
2121
#include "src/common/scoped-modification.h"
2222
#include "src/compiler-dispatcher/lazy-compile-dispatcher.h"
23-
#include "src/heap/factory-inl.h"
2423
#include "src/heap/local-heap-inl.h"
2524
#include "src/heap/parked-scope.h"
2625
#include "src/logging/counters.h"
@@ -3587,47 +3586,32 @@ void Parser::InsertSloppyBlockFunctionVarBindings(DeclarationScope* scope) {
35873586
// ----------------------------------------------------------------------------
35883587
// Parser support
35893588

3590-
void Parser::HandleDebugMagicComments(Isolate* isolate,
3589+
template <typename IsolateT>
3590+
void Parser::HandleDebugMagicComments(IsolateT* isolate,
35913591
DirectHandle<Script> script) {
3592-
// On the main thread, the scanned source is the script source.
3593-
DirectHandle<String> source(Cast<String>(script->source()), isolate);
3594-
SetDebugMagicCommentFields(isolate, script, source,
3595-
scanner_.debug_magic_comment_locations());
3596-
}
3597-
3598-
// static
3599-
void Parser::SetDebugMagicCommentFields(
3600-
Isolate* isolate, DirectHandle<Script> script, DirectHandle<String> source,
3601-
const Scanner::DebugMagicCommentLocations& locations) {
3602-
// The values can be arbitrarily large (e.g. inline source maps), so they are
3603-
// substrings of the source rather than copies.
3604-
auto value = [&](Scanner::Location location) -> DirectHandle<String> {
3605-
if (!location.IsValid() || location.length() == 0 ||
3606-
static_cast<uint32_t>(location.end_pos) > source->length()) {
3607-
return {};
3608-
}
3609-
return isolate->factory()->NewSubString(source, location.beg_pos,
3610-
location.end_pos);
3611-
};
3612-
DirectHandle<String> source_url = value(locations.source_url);
3592+
DirectHandle<String> source_url = scanner_.SourceUrl(isolate);
36133593
if (!source_url.is_null()) {
36143594
script->set_source_url(*source_url);
36153595
}
3596+
DirectHandle<String> source_mapping_url = scanner_.SourceMappingUrl(isolate);
36163597
// The API can provide a source map URL and the API should take precedence.
36173598
// Let's make sure we do not override the API with the magic comment.
3618-
if (IsUndefined(script->source_mapping_url())) {
3619-
DirectHandle<String> source_mapping_url =
3620-
value(locations.source_mapping_url);
3621-
if (!source_mapping_url.is_null()) {
3622-
script->set_source_mapping_url(*source_mapping_url);
3623-
}
3599+
if (!source_mapping_url.is_null() &&
3600+
IsUndefined(script->source_mapping_url())) {
3601+
script->set_source_mapping_url(*source_mapping_url);
36243602
}
3625-
DirectHandle<String> debug_id = value(locations.debug_id);
3603+
3604+
DirectHandle<String> debug_id = scanner_.DebugId(isolate);
36263605
if (!debug_id.is_null()) {
36273606
script->set_debug_id(*debug_id);
36283607
}
36293608
}
36303609

3610+
template void Parser::HandleDebugMagicComments(Isolate* isolate,
3611+
DirectHandle<Script> script);
3612+
template void Parser::HandleDebugMagicComments(LocalIsolate* isolate,
3613+
DirectHandle<Script> script);
3614+
36313615
void Parser::UpdateStatistics(Isolate* isolate, DirectHandle<Script> script) {
36323616
CHECK_NOT_NULL(isolate);
36333617

@@ -3730,6 +3714,9 @@ void Parser::ParseOnBackground(LocalIsolate* isolate, ParseInfo* info,
37303714
MaybeProcessSourceRanges(info, result, stack_limit_);
37313715
});
37323716
// We need to unpark by now though, to be able to internalize.
3717+
if (flags().is_toplevel()) {
3718+
HandleDebugMagicComments(isolate, script);
3719+
}
37333720
if (result == nullptr) return;
37343721
PostProcessParseResult(isolate, info, result);
37353722
}

‎src/parsing/parser.h‎

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -169,18 +169,8 @@ class V8_EXPORT_PRIVATE Parser : public NON_EXPORTED_BASE(ParserBase<Parser>) {
169169
DirectHandle<Script> script,
170170
base::SmallVector<v8::Isolate::UseCounterFeature, 8>* use_counters,
171171
int* preparse_skipped);
172-
void HandleDebugMagicComments(Isolate* isolate, DirectHandle<Script> script);
173-
const Scanner::DebugMagicCommentLocations& debug_magic_comment_locations()
174-
const {
175-
return scanner_.debug_magic_comment_locations();
176-
}
177-
// Sets the debug magic comment fields of {script} to substrings of {source}
178-
// at the given locations. Locations that are empty, invalid or out of bounds
179-
// are ignored.
180-
static void SetDebugMagicCommentFields(
181-
Isolate* isolate, DirectHandle<Script> script,
182-
DirectHandle<String> source,
183-
const Scanner::DebugMagicCommentLocations& locations);
172+
template <typename IsolateT>
173+
void HandleDebugMagicComments(IsolateT* isolate, DirectHandle<Script> script);
184174

185175
private:
186176
friend class ParserBase<Parser>;

‎src/parsing/scanner.cc‎

Lines changed: 56 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -314,14 +314,12 @@ void Scanner::TryToParseMagicComment(base::uc32 hash_or_at_sign) {
314314
}
315315
if (!name.is_one_byte()) return;
316316
base::Vector<const uint8_t> name_literal = name.one_byte_literal();
317-
// The debug magic comment values are recorded as source locations, so that
318-
// they can be created as substrings of the source instead of being copied.
319-
Location* value_location = nullptr;
320-
bool is_compile_hints = false;
317+
LiteralBuffer* value = nullptr;
318+
LiteralBuffer per_function_compile_hints_value;
321319
if (name_literal == base::StaticOneByteVector("sourceURL")) {
322-
value_location = &debug_magic_comment_locations_.source_url;
320+
value = &source_url_;
323321
} else if (name_literal == base::StaticOneByteVector("sourceMappingURL")) {
324-
value_location = &debug_magic_comment_locations_.source_mapping_url;
322+
value = &source_mapping_url_;
325323
DCHECK(hash_or_at_sign == '#' || hash_or_at_sign == '@');
326324
saw_source_mapping_url_magic_comment_at_sign_ = hash_or_at_sign == '@';
327325
} else if (!saw_non_comment_ &&
@@ -332,33 +330,30 @@ void Scanner::TryToParseMagicComment(base::uc32 hash_or_at_sign) {
332330
} else if (name_literal ==
333331
base::StaticOneByteVector("functionsCalledOnLoad") &&
334332
hash_or_at_sign == '#') {
335-
is_compile_hints = true;
333+
value = &per_function_compile_hints_value;
336334
} else if (name_literal == base::StaticOneByteVector("debugId") &&
337335
hash_or_at_sign == '#') {
338-
value_location = &debug_magic_comment_locations_.debug_id;
336+
value = &debug_id_;
339337
} else {
340338
return;
341339
}
342340
if (c0_ != '=') {
343341
return;
344342
}
345-
if (!is_compile_hints) {
346-
DCHECK_NOT_NULL(value_location);
347-
*value_location = ScanMagicCommentValue([](base::uc32) {});
348-
return;
343+
value->Start();
344+
if (!ScanMagicCommentValue([value](base::uc32 c) { value->AddChar(c); })
345+
.IsValid()) {
346+
value->Start();
347+
}
348+
if (value == &per_function_compile_hints_value &&
349+
per_function_compile_hints_value.is_one_byte()) {
350+
base::Vector<const uint8_t> value_literal =
351+
per_function_compile_hints_value.one_byte_literal();
352+
per_function_compile_hint_positions_.clear();
353+
per_function_compile_hint_positions_idx_ = 0;
354+
ProcessPerFunctionCompileHints(value_literal, source_pos(),
355+
per_function_compile_hint_positions_);
349356
}
350-
LiteralBuffer value;
351-
value.Start();
352-
Location value_location_in_source =
353-
ScanMagicCommentValue([&value](base::uc32 c) { value.AddChar(c); });
354-
// A value followed by trailing garbage is treated as empty, which clears
355-
// the previously recorded compile hints.
356-
if (!value_location_in_source.IsValid()) value.Start();
357-
if (!value.is_one_byte()) return;
358-
per_function_compile_hint_positions_.clear();
359-
per_function_compile_hint_positions_idx_ = 0;
360-
ProcessPerFunctionCompileHints(value.one_byte_literal(), source_pos(),
361-
per_function_compile_hint_positions_);
362357
}
363358

364359
bool Scanner::HasPerFunctionCompileHint(int position) {
@@ -700,6 +695,43 @@ Token::Value Scanner::ScanTemplateSpan() {
700695
return result;
701696
}
702697

698+
template <typename IsolateT>
699+
DirectHandle<String> Scanner::SourceUrl(IsolateT* isolate) const {
700+
DirectHandle<String> tmp;
701+
if (source_url_.length() > 0) {
702+
tmp = source_url_.Internalize(isolate);
703+
}
704+
return tmp;
705+
}
706+
707+
template DirectHandle<String> Scanner::SourceUrl(Isolate* isolate) const;
708+
template DirectHandle<String> Scanner::SourceUrl(LocalIsolate* isolate) const;
709+
710+
template <typename IsolateT>
711+
DirectHandle<String> Scanner::SourceMappingUrl(IsolateT* isolate) const {
712+
DirectHandle<String> tmp;
713+
if (source_mapping_url_.length() > 0) {
714+
tmp = source_mapping_url_.Internalize(isolate);
715+
}
716+
return tmp;
717+
}
718+
719+
template DirectHandle<String> Scanner::SourceMappingUrl(Isolate* isolate) const;
720+
template DirectHandle<String> Scanner::SourceMappingUrl(
721+
LocalIsolate* isolate) const;
722+
723+
template <typename IsolateT>
724+
DirectHandle<String> Scanner::DebugId(IsolateT* isolate) const {
725+
DirectHandle<String> tmp;
726+
if (debug_id_.length() > 0) {
727+
tmp = debug_id_.Internalize(isolate);
728+
}
729+
return tmp;
730+
}
731+
732+
template DirectHandle<String> Scanner::DebugId(Isolate* isolate) const;
733+
template DirectHandle<String> Scanner::DebugId(LocalIsolate* isolate) const;
734+
703735
bool Scanner::ScanDigitsWithNumericSeparators(bool (*predicate)(base::uc32 ch),
704736
bool is_check_first_digit) {
705737
// we must have at least one digit after 'x'/'b'/'o'

‎src/parsing/scanner.h‎

Lines changed: 10 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -464,18 +464,12 @@ class V8_EXPORT_PRIVATE Scanner {
464464
return ScanTemplateSpan();
465465
}
466466

467-
// Source locations of the values of the debug magic comments. A location is
468-
// invalid if there was no such comment, or if the last one was malformed; it
469-
// is empty if the last one had an empty value. In both cases there is no
470-
// value.
471-
struct DebugMagicCommentLocations {
472-
Location source_url = Location::invalid();
473-
Location source_mapping_url = Location::invalid();
474-
Location debug_id = Location::invalid();
475-
};
476-
const DebugMagicCommentLocations& debug_magic_comment_locations() const {
477-
return debug_magic_comment_locations_;
478-
}
467+
template <typename IsolateT>
468+
DirectHandle<String> SourceUrl(IsolateT* isolate) const;
469+
template <typename IsolateT>
470+
DirectHandle<String> SourceMappingUrl(IsolateT* isolate) const;
471+
template <typename IsolateT>
472+
DirectHandle<String> DebugId(IsolateT* isolate) const;
479473

480474
bool SawSourceMappingUrlMagicCommentAtSign() const {
481475
return saw_source_mapping_url_magic_comment_at_sign_;
@@ -819,7 +813,10 @@ class V8_EXPORT_PRIVATE Scanner {
819813
// Whether this scanner encountered an HTML comment.
820814
bool found_html_comment_;
821815

822-
DebugMagicCommentLocations debug_magic_comment_locations_;
816+
// Values parsed from magic comments.
817+
LiteralBuffer source_url_;
818+
LiteralBuffer source_mapping_url_;
819+
LiteralBuffer debug_id_;
823820
bool saw_source_mapping_url_magic_comment_at_sign_ = false;
824821
bool saw_magic_comment_compile_hints_all_ = false;
825822
bool saw_non_comment_ = false;

‎src/snapshot/code-serializer.h‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -257,12 +257,6 @@ class CodeSerializer : public Serializer {
257257
sink_.Put(kSynchronize, "EndOfUntrustedSection");
258258
Pad();
259259
}
260-
// Slices usually refer to the script source (e.g. debug magic comment
261-
// values), which is an attached object: off-thread deserialization attaches
262-
// a placeholder instead of the source.
263-
bool SerializeSlicedStringsAsSequentialStrings() const override {
264-
return true;
265-
}
266260

267261
DISALLOW_GARBAGE_COLLECTION(no_gc_)
268262
SourceHash source_hash_;

0 commit comments

Comments
 (0)