Skip to content

Conversation

@stof
Copy link
Member

@stof stof commented Mar 11, 2021

This reworks the CompilationResult result object introduced in #270. This has to be merged before the next release (otherwise CompilationResult will be covered by BC and prevent the changes)

  • Scssphp\Scssphp\CompilationResult is now an immutable value object instead of a mutable one
  • Scssphp\Scssphp\CompilationResult now longer exposes some internal API publicly
  • metadata used only by the caching layer are not exposed in the CompilationResult anymore. They are kept internal (the object stored in the cache is an internal CachedResult object wrapping the CompilationResult)
  • metadata collected during the compilation are stored in properties of the Compiler (that are reset at the end of the compilation to free memory, except for parsedFiles which has a deprecated getter)
  • the Compiler no longer retains a reference to the CompilationResult after returning it (which kept reference to much more memory than just parsedFiles as there is the CSS and sourcemap in there)
  • getParsedFiles is not exposed in CompilationResult, in favor of getIncludedFiles, which returns a list of file paths. When looking at various open-source packages using scssphp, any usage of $compiler->getParsedFiles() I found were actually usages of array_keys($compiler->getParsedFiles()

/cc @Cerdic as you implement the initial version.

stof added 5 commits March 11, 2021 00:52
Making the CompilationResult object aware of the Compiler to check for
cache freshness is weird, as the Compiler is the one with knowledge
about the logic. The CompilationResult should be a value object.
a5f8410 said that this was making it
consistent with the way dart-sass builds the list. But this was wrong,
due to a bad analysis of the dart-sass behavior. Code generating a CSS
import rule des not consider the URL of that import as an included file
for the sass rendering.
@stof stof requested a review from Cerdic March 11, 2021 13:46
@stof stof added this to the 1.5 milestone Mar 11, 2021
@Cerdic
Copy link
Collaborator

Cerdic commented Mar 11, 2021

I think I'm a bit lost.

The idea of the CompilationResult was (for me) exactly to group and expose all informations needed to manage cache and/or cache invalidation logic to avoid in the future any new problem about consumers having issue with the cache invalidation : if the default (improved) logic was not fitting their need it was possible to manage it your own way.

That said, I can fully see your logic of trying to lower as much as possible the public and exposed side of the lib, to allow a easier evolution in the future.

But here I have the feeling we're almost back to the start, the CompilationResult beeing just a container for some additional data to the plain css, and not giving any more information than before (ie the fact that the result is a cached result or not, and the list of resolvedImports which is the keypoint if you want to make your own check on invalidation mecanism)

I have to say I'm not available enough to spend sufficient time and dig on this subject to have a fully clear view.

public function getIncludedFiles()
{
return array_column($this->importedFiles, 'filePath');
return $this->getIncludedFiles();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mistake here I think. Should be $this->includedFiles;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed

@stof
Copy link
Member Author

stof commented Mar 11, 2021

@Cerdic using the resolved import actually relies on being able to use Compiler::findImport, which is pretty much something I'd like to avoid having as part of the public API of the Compiler in 2.0. That's my reasoning for keeping it out of the public API.
Thus, the structure of this resolvedImports array is tied to the current implementation of the import resolution algorithm. But I know for sure that we would have to change it when we implement modules (the resolution rules for @import and @use have some differences due to the import fallback for migration paths). Making this structure public would prevent implementing modules.

#270 claimed to solve 2 issues at once:

the fact that the result is a cached result or not

why should you care whether the Compiler was able to reuse its cache or had to do the compilation again ? You still get the same result in both cases. I don't see a need for that.

But here I have the feeling we're almost back to the start, the CompilationResult beeing just a container for some additional data to the plain css

That was precisely the request in #222: being a container against the extra additional data that was accessible through a stateful API of the Compiler before (which is an awful way to access it as the usability of Compiler::getParsedFiles depends on the previous usage of the compiler and the freeing of memory depends on the next usage)

The idea of the CompilationResult was (for me) exactly to group and expose all informations needed to manage cache and/or cache invalidation logic to avoid in the future any new problem about consumers having issue with the cache invalidation : if the default (improved) logic was not fitting their need it was possible to manage it your own way.

AFAICT, your implementation still does not allow any hooking inside the cache invalidation. You only get your hand on the CompilationResult object after the Compiler is done with it, and so caching is already done by that time.

If the goal is to allow disabling the built-in compiler cache to handle caching in the caller while still having a proper cache invalidation, I can suggest an alternative API. We could have CompilationResult::getCacheInfo(): CacheInfo (name to determined, I'm not satisfied with that one) returning an internal object, and Compiler::isFreshCache(CacheInfo $cacheInfo, $checkImports = true): bool checking whether the cache is still fresh. Everything inside this CacheInfo object would be internal. Its only public API would be that you can pass it to Compiler::isFreshCache to get freshness info (and that you can serialize it for storage, so we would not put any Closure in it for instance). That way, we can do any refactoring we want in the cache invalidation logic without BC breaks (as the metadata would be internal).
That would basically replace my CachedResult wrapper with an object available in CompilationResult

Tracking the data during the compilation is now done again internally in
the Compiler. this gives more flexibility into refactoring the logic if
necessary. It also avoids storing the CompilationResult after the end of
the compilation for the getParsedFiles deprecated getter, which would
keep a reference to the generated CSS and sourcemap as well, potentially
increasing memory usage.
Data used only by our caching layer for freshness checks is now kept
outside the CompilationResult object, to avoid exposing it in the public
API.
@stof stof force-pushed the rework_compilation_result branch from f7ef795 to 72cb387 Compare March 11, 2021 16:01
@Cerdic
Copy link
Collaborator

Cerdic commented Mar 11, 2021

Feel free to go, as I said my mind is not able to get the big picture right now and this is not helping to challenge you

@stof
Copy link
Member Author

stof commented Mar 11, 2021

@Cerdic the fact that you want to be able to check for cache invalidation in the embedder (see #316) is actually a good challenge for that suggested API. But do you need the embedded to be able to use the data of resolvedImports by themselves or would it be OK for you to have the blackbox implementation that I suggested in my previous comment (that allows us to make the logic itself evolve) ?

@Cerdic
Copy link
Collaborator

Cerdic commented Mar 11, 2021

It depends of the level of consistency you need and if you can deal with some edge case. I was dealing with the previous invalidation implementation relying only on the importedFiles but this was an issue for some users.

If you want a reliable invalidation, the acces to the resolvedImports data is necessary, but as you said you also need to access to the resolveImport() method.

I think the implementation of the invalidation check in scssphp is now reliable enough to avoid the need of this api for now and we could postpone this work as there is already a lot to do besides that - except if we decide to get rid of the cache mecanism in the lib.

@mahagr
Copy link

mahagr commented Mar 12, 2021

My biggest issue with CompilationResult is that it saves the source map (if not embedded) into the filesystem. What if I wanted to upload the file into CDN instead of saving it locally? Or use FlySystem to save both the CSS and map?

The class should only do what it is supposed to.

@stof
Copy link
Member Author

stof commented Mar 12, 2021

@mahagr saving the sourcemap in the filesystem is indeed the point that still bothers me. But as long as we keep returning it from compile() instead of creating a new API (see my comment in #310 (comment)), we cannot avoid it to avoid too much BC breaks.

@Cerdic for now, I think I will merge this PR without the external access to the cache invalidation logic. Exposing that publicly later (if requested) is easy to do in a backward compatible way (while removing it from the public API is not)

@Cerdic
Copy link
Collaborator

Cerdic commented Mar 12, 2021

@stof let's go for that

@mahagr
Copy link

mahagr commented Mar 12, 2021

@stof Can you at least provide the alternate method to return CSS and map separately without saving anything to a file? There must be some way to do that without breaking things. :)

@stof stof merged commit 9bb1831 into scssphp:master Mar 12, 2021
@stof stof deleted the rework_compilation_result branch March 12, 2021 13:57
@stof
Copy link
Member Author

stof commented Mar 12, 2021

@mahagr see #320 where I discuss the remaining points about that API.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants