Skip to content

Conversation

@ioweb-gr
Copy link
Contributor

@ioweb-gr ioweb-gr commented Jul 28, 2024

Description (*)

Currently when running magento 2 in developer mode with server-side less compilation it's very hard to detect where a style is coming from. The only workaround is to use Grunt.
However the underlying library for less compilation supports sourcemap generation with a flag that can be toggled based on the mode magento is in.

Related Pull Requests

Fixed Issues (if relevant)

Manual testing scenarios (*)

  1. Switch magento 2 in developer mode
  2. Let it regenerate css from the less files
  3. Check inspector to see the source file responsible for the code

image
image

Questions or comments

Contribution checklist (*)

  • Pull request has a meaningful description of its purpose
  • All commits are accompanied by meaningful commit messages
  • All new or changed code is covered with unit/integration tests (if applicable)
  • README.md files for modified modules are updated and included in the pull request if any README.md predefined sections require an update
  • All automated tests passed successfully (all builds are green)

Resolved issues:

  1. resolves [Issue] add sourceMap generation for less files in developer mode #38982: add sourceMap generation for less files in developer mode

@m2-assistant
Copy link

m2-assistant bot commented Jul 28, 2024

Hi @ioweb-gr. Thank you for your contribution!
Here are some useful tips on how you can test your changes using Magento test environment.

Add the comment under your pull request to deploy test or vanilla Magento instance:
  • @magento give me test instance - deploy test instance based on PR changes
  • @magento give me 2.4-develop instance - deploy vanilla Magento instance

❗ Automated tests can be triggered manually with an appropriate comment:

  • @magento run all tests - run or re-run all required tests against the PR changes
  • @magento run <test-build(s)> - run or re-run specific test build(s)
    For example: @magento run Unit Tests

<test-build(s)> is a comma-separated list of build names.

Allowed build names are:
  1. Database Compare
  2. Functional Tests CE
  3. Functional Tests EE
  4. Functional Tests B2B
  5. Integration Tests
  6. Magento Health Index
  7. Sample Data Tests CE
  8. Sample Data Tests EE
  9. Sample Data Tests B2B
  10. Static Tests
  11. Unit Tests
  12. WebAPI Tests
  13. Semantic Version Checker

You can find more information about the builds here
ℹ️ Run only required test builds during development. Run all test builds before sending your pull request for review.


For more details, review the Code Contributions documentation.
Join Magento Community Engineering Slack and ask your questions in #github channel.

@magento-automated-testing
Copy link

Failed to run the builds. Please try to re-run them later.

@ioweb-gr
Copy link
Contributor Author

@magento run all tests

@engcom-Hotel
Copy link
Contributor

@magento create issue

@engcom-Hotel engcom-Hotel added Priority: P2 A defect with this priority could have functionality issues which are not to expectations. and removed Progress: pending review labels Jul 29, 2024
Copy link

@Eddcapone Eddcapone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not working.
I tried your change and then called grunt refresh to regenerate the less files, but it still does not generate source maps. I enabled source maps in the browser dev tools.

EDIT: Hold up! It works after I deleted my whole static files first with

rm -rf pub/static/frontend/* pub/static/adminhtml/*

and then running: php bin/magento setup:static-content:deploy -f

I need to check if it works with and without your solution applied.

@ioweb-gr
Copy link
Contributor Author

ioweb-gr commented Aug 2, 2024

Not working.
I tried your change and then called grunt refresh to regenerate the less files, but it still does not generate source maps. I enabled source maps in the browser dev tools.

EDIT: Hold up! It works after I deleted my whole static files first with

rm -rf pub/static/frontend/* pub/static/adminhtml/*

and then running: php bin/magento setup:static-content:deploy -f

I need to check if it works with and without your solution applies.

Actually my patch was just for server side compilation not for grunt.

@Eddcapone
Copy link

Eddcapone commented Aug 2, 2024

@ioweb-gr Grunt is also server side compilation.

@ioweb-gr
Copy link
Contributor Author

ioweb-gr commented Aug 2, 2024

True it's in server side compilation mode but uses external tool to process the less files and not the library https://github.com/wikimedia/less.php/ that Magento uses in the default server side compilation mode

Forgive my wrong use of the server side compilation term, I mean the Magento's internal server side compilation tool and not grunt

@Eddcapone
Copy link

Ok, I can confirm that the source map generation does work with your change, and it does not work without your change!

But the problem is, that in magento 2 you would not develop with php bin/magento setup:static-content:deploy -f, this would take way too much time. We would rather need a solution to fix grunt not generating any source files anymore.

But I would add your change nevertheless. It is better than status quo. At the moment we have no source maps at all...

Good job!

@ioweb-gr
Copy link
Contributor Author

ioweb-gr commented Aug 2, 2024

With some third party themes in the market you can't use grunt as they override the default generation of static files to optimize things. You can't avoid it some times.

Moreover in developer mode, the static files get generated dynamically so on a good server it's not that much of an issue.

Grunt sometimes takes the same time to generate a new build. If it was webpack it would be a different story but with grunt it's not so much faster in most cases :(

@engcom-Bravo engcom-Bravo added the Triage: Dev.Experience Issue related to Developer Experience and needs help with Triage to Confirm or Reject it label Aug 5, 2024
@Eddcapone
Copy link

Eddcapone commented Aug 5, 2024

@ioweb-gr I would never buy such a theme, because developing with setup:static-content:deploy would take a huge amount of time. You would need to call it each time you make a tiny change and want to test it.

Moreover in developer mode, the static files get generated dynamically so on a good server it's not that much of an issue.

Initially the files are getting generated, but it does not detect changes in the code. Let's say you want to change the background color of a container and need to change code in a less file, then it wont be enough to just reload the site. You need to recompile everything, which takes alot of time. Thats were grunt comes into play. Grunt will detect the change in that file and recompiles that file only and it only takes a few seconds.

Update: I found a fix for grunt .

@ioweb-gr
Copy link
Contributor Author

ioweb-gr commented Aug 5, 2024

Nonetheless it's very common in the themes especially those that provide some custom css sections inside of the configuration files.

Also for grunt I've seen some other preventive cases

  1. Some times it's handing //magento_import directive differently
  2. Some times live reload isn't able to connect especially in some hosting environments preventing effective use

All I'm saying is sometimes it's not worth the trouble when you can simply run n98-magerun2 d:a:c and hit reload other than that I totally agree with you.

Personally I think that M2 needs a better system overall for frontend processing other than require and grunt / less which will help with extracting critical css, chunk properly and deliver in a proper way only the necessary bundles to the client optimizing it overall

@ioweb-gr
Copy link
Contributor Author

I'd also like to say that even with Luma and a child theme, sometimes grunt doesn't provide the same result as just clearing all static files and letting the system regenerate them, especially if you've enabled minification and critical path.

Generally I see grunt producing inconsistent results to what appears in production when running a full deployment therefore sometimes I feel it's actually making me slower than just deleting the generated files on demand in developer mode and letting the system regenerate them.

Frankly I wish the system would use one tool for compilation for both development and production and stick to that (preferrably outside of the internal php framework)

@engcom-November engcom-November self-assigned this Sep 3, 2024
@engcom-November
Copy link
Contributor

Hello @ioweb-gr,

Thank you for the collaboration and contribution!

✔️ QA Passed

Steps to reproduce:

  1. Switch magento 2 in developer mode
  2. Let it regenerate css from the less files
  3. Check inspector to see the source file responsible for the code

Before: ✖️

Cannot see the source map of the files.
image

After: ✔️

Able to see the sorce map of the files.
image

@ioweb-gr
Copy link
Contributor Author

ioweb-gr commented Oct 1, 2024

Hi @hostep I've applied the two patches you provided.

Thanks for providing the diffs so that I can apply them directly.

@engcom-November can you follow up on this?

@hostep
Copy link
Contributor

hostep commented Oct 1, 2024

Thanks @ioweb-gr, let's see how the automated tests run this time.

@magento run all tests

@hostep
Copy link
Contributor

hostep commented Oct 1, 2024

Okay nice, the unit tests work perfect even on another machine then my own, very good!

However, I didn't check the unit test itself for coding standards checks, so those still fail.
Here's my suggestion to fix those:

diff --git a/lib/internal/Magento/Framework/Css/Test/Unit/PreProcessor/Adapter/Less/ProcessorTest.php b/lib/internal/Magento/Framework/Css/Test/Unit/PreProcessor/Adapter/Less/ProcessorTest.php
index 26ffa78abdd..0afa14ea86a 100644
--- a/lib/internal/Magento/Framework/Css/Test/Unit/PreProcessor/Adapter/Less/ProcessorTest.php
+++ b/lib/internal/Magento/Framework/Css/Test/Unit/PreProcessor/Adapter/Less/ProcessorTest.php
@@ -19,15 +19,15 @@ use Psr\Log\LoggerInterface;
 
 class ProcessorTest extends TestCase
 {
-    const TEST_CONTENT = 'test-content';
+    private const TEST_CONTENT = 'test-content';
 
-    const ASSET_PATH = 'test-path';
+    private const ASSET_PATH = 'test-path';
 
-    const TMP_PATH_LESS = '_file/test.less';
-    const TMP_PATH_CSS_PRODUCTION = '_file/test-production.css';
-    const TMP_PATH_CSS_DEVELOPER = '_file/test-developer.css';
+    private const TMP_PATH_LESS = '_file/test.less';
+    private const TMP_PATH_CSS_PRODUCTION = '_file/test-production.css';
+    private const TMP_PATH_CSS_DEVELOPER = '_file/test-developer.css';
 
-    const ERROR_MESSAGE = 'Test exception';
+    private const ERROR_MESSAGE = 'Test exception';
 
     /**
      * @var Processor
@@ -179,8 +179,16 @@ class ProcessorTest extends TestCase
 
         $clearSymbol = ["\n", "\r", "\t", ' '];
         self::assertEquals(
-            trim(str_replace($clearSymbol, '', file_get_contents(__DIR__ . '/' . self::TMP_PATH_CSS_PRODUCTION))),
-            trim(str_replace($clearSymbol, '', $this->processor->processContent($assetMock)))
+            trim(str_replace(
+                $clearSymbol,
+                '',
+                file_get_contents(__DIR__ . '/' . self::TMP_PATH_CSS_PRODUCTION)
+            )),
+            trim(str_replace(
+                $clearSymbol,
+                '',
+                $this->processor->processContent($assetMock)
+            ))
         );
     }
 
@@ -214,8 +222,16 @@ class ProcessorTest extends TestCase
 
         $clearSymbol = ["\n", "\r", "\t", ' '];
         self::assertEquals(
-            trim(str_replace($clearSymbol, '', file_get_contents(__DIR__ . '/' . self::TMP_PATH_CSS_DEVELOPER))),
-            trim(str_replace($clearSymbol, '', $this->normalizeInlineSourceMap($this->processor->processContent($assetMock))))
+            trim(str_replace(
+                $clearSymbol,
+                '',
+                file_get_contents(__DIR__ . '/' . self::TMP_PATH_CSS_DEVELOPER)
+            )),
+            trim(str_replace(
+                $clearSymbol,
+                '',
+                $this->normalizeInlineSourceMap($this->processor->processContent($assetMock))
+            ))
         );
     }
 

The failing functional tests don't seem related to the changes proposed here and are probably flaky.

@ioweb-gr
Copy link
Contributor Author

@hostep I applied your latest patch :) Let's see how it goes now

@hostep
Copy link
Contributor

hostep commented Oct 10, 2024

@magento run all tests

@hostep
Copy link
Contributor

hostep commented Oct 10, 2024

Looks good, failing functional tests don't seem related to the changes from this PR.

@hostep hostep removed their assignment Oct 10, 2024
@engcom-November
Copy link
Contributor

@magento run Functional Tests B2B, Functional Tests CE, Functional Tests EE

@engcom-November
Copy link
Contributor

The repeating failure in Functional Tests B2B is flaky and known issue.

Functional Tests B2B:
Run 1:
image

Run 2:
image

Known issue:
ACQE-5677 - StorefrontQuoteCanBeRenamedUntilLockedTest

Hence moving this to Merge in progress.

@engcom-Dash
Copy link
Contributor

We have seen static failures(related to copyright tag) in mainline PR after merging this PR into that. Hence for further analysis; for now moving this PR to Extended testing.

@engcom-Dash
Copy link
Contributor

Hello @ioweb-gr,

Thanks for the contributions!

It seems that we don't have permission to commit fixes to the repository. Could you please address the static failures listed below in your PR? We are trying to merge this PR into the mainline PR but noticed the following failures. Please take the necessary action so we can proceed further.

image

To fix these errors, could you please replace the following copyright tag:

 /**
 * Copyright © Magento, Inc. All rights reserved.
 * See COPYING.txt for license details.
 */

With this one:

/**
 * Copyright 2015 Adobe
 * All Rights Reserved.
 */ 

Thanks!

@engcom-Dash
Copy link
Contributor

@magento run Static Tests

@ioweb-gr
Copy link
Contributor Author

ioweb-gr commented Nov 8, 2024

How about now? :)

@engcom-Dash
Copy link
Contributor

@magento run all tests

@engcom-Dash
Copy link
Contributor

How about now? :)

Thanks @ioweb-gr. Changes look good. Let's wait for the build result.

@engcom-Dash
Copy link
Contributor

@magento run Unit Tests

@engcom-Dash
Copy link
Contributor

Got a green build so moving it to merge in progress.
image

@hostep
Copy link
Contributor

hostep commented Nov 12, 2024

The changes from this PR got merged in 2.4-develop branch today via 651928c together with a bunch of other stuff, but this PR isn't marked as merged. @engcom-Dash can you investigate how that happened?

@engcom-Dash
Copy link
Contributor

The changes from this PR got merged in 2.4-develop branch today via 651928c together with a bunch of other stuff, but this PR isn't marked as merged. @engcom-Dash can you investigate how that happened?

Hello @hostep, Thank you for pointing that out. It looks like the changes from this PR were merged into the 2.4-develop branch today via 651928c, but we missed one of the merge commits into our mainline PR, which is why it's not showing as merged.

We will make sure to add the missing commit 7fc0a30 in our next mainline PR. Apologies for the oversight, and thank you for your understanding.

@magento-devops-reposync-svc magento-devops-reposync-svc merged commit 657075b into magento:2.4-develop Dec 24, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Priority: P2 A defect with this priority could have functionality issues which are not to expectations. Progress: accept Project: Community Picked PRs upvoted by the community Triage: Dev.Experience Issue related to Developer Experience and needs help with Triage to Confirm or Reject it

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Issue] add sourceMap generation for less files in developer mode

9 participants