Skip to content

Comments

[Fix]: Ignore hidden files during Astro IDE project export#1946

Merged
pritt20 merged 6 commits intomainfrom
ide_ignore_hidden_files
Oct 6, 2025
Merged

[Fix]: Ignore hidden files during Astro IDE project export#1946
pritt20 merged 6 commits intomainfrom
ide_ignore_hidden_files

Conversation

@pritt20
Copy link
Contributor

@pritt20 pritt20 commented Sep 26, 2025

Description

This PR changes update Astro IDE export command to ignore hidden files i.e. (.)files to prevent exporting local virtual environment, .git files and other unwanted files to Astro IDE.

🎟 Issue(s)

Related #XXX

🧪 Functional Testing

Tested the change locally:

~ ls -ltra

drwxr-xr-x pritt staff  96 B  Fri Sep 19 02:27:23 2025  ..
drwxr-xr-x pritt staff 160 B  Fri Sep 19 02:27:28 2025  .astro
.rw-r--r-- pritt staff  73 B  Fri Sep 19 02:27:28 2025  .dockerignore
.rw-r--r-- pritt staff 132 B  Fri Sep 19 02:27:28 2025  .gitignore
.rw-r--r-- pritt staff  44 B  Fri Sep 19 02:27:28 2025  Dockerfile
.rw-r--r-- pritt staff 4.4 KB Fri Sep 19 02:27:28 2025  README.md
.rw-r--r-- pritt staff 111 B  Fri Sep 19 02:27:28 2025  blueprint.toml
drwxr-xr-x pritt staff 160 B  Fri Sep 19 02:27:28 2025  dags
.rw-r--r-- pritt staff   0 B  Fri Sep 19 02:27:28 2025  packages.txt
.rw-r--r-- pritt staff 173 B  Fri Sep 19 02:27:28 2025  requirements.txt
drwxr-xr-x pritt staff  96 B  Fri Sep 19 02:27:28 2025  tests
drwxr-xr-x pritt staff 448 B  Fri Sep 19 02:49:11 2025  .
drwxr-xr-x pritt staff 288 B  Fri Sep 19 02:49:48 2025  .venv
drwxr-xr-x pritt staff 384 B  Sat Sep 27 04:35:20 2025  .git
~ astro ide project export --force

Please select the project from the list below:
 #     PROJECT NAME     ID                            
 1     blueprint        cmfpw8yjl01yi01n0zrrgfh4o     
 2     test             cmfnttshq0e5201kirizai7zz     

> 1
Successfully exported project to cmfpw8yjl01yi01n0zrrgfh4o

📸 Screenshots

image

📋 Checklist

  • Rebased from the main (or release if patching) branch (before testing)
  • Ran make test before taking out of draft
  • Ran make lint before taking out of draft
  • Added/updated applicable tests
  • Tested against Astro-API (if necessary).
  • Tested against Houston-API and Astronomer (if necessary).
  • Communicated to/tagged owners of respective clients potentially impacted by these changes.
  • Updated any related documentation

return nil
}

// Skip all hidden dotfiles and dot-directories, except allow the root ".astro" file
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.astro is a dir rather than a file. Astro projects also include .gitignore and .dockerignore which we shouldn't be dropping.

Maybe we could do this a different way. The default Astro project .gitignore file excludes .git, .venv, and other files we shouldn't include, so perhaps we could use the go-git package to figure out which paths are allowed by .gitignore and only include those? (I've done something similar for an internal component of Astro IDE)

@coveralls-official
Copy link

coveralls-official bot commented Oct 3, 2025

Pull Request Test Coverage Report for Build 4f7399df-91ef-4c62-b46a-22beea9a00ba

Details

  • 30 of 35 (85.71%) changed or added relevant lines in 1 file are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.1%) to 38.296%

Changes Missing Coverage Covered Lines Changed/Added Lines %
cloud/ide/project.go 30 35 85.71%
Totals Coverage Status
Change from base Build c1d2bd57-f612-47ce-a451-e9f2f5990f3f: 0.1%
Covered Lines: 23883
Relevant Lines: 62365

💛 - Coveralls

// gitignoreMatcher creates a matcher from .gitignore patterns in the given root.
func gitignoreMatcher(root string) gitignore.Matcher {
fs := osfs.New(root)
patterns, _ := gitignore.ReadPatterns(fs, []string{})
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We shouldn't ignore the error. If it returns one can we print a warning that it couldn't be parsed (including the error we're given) and continue without doing the gitignore filtering?

}

// decideArchiveEntry determines whether to include a path, skip it, or skip its subtree.
func decideArchiveEntry(relPath string, info os.FileInfo, matcher gitignore.Matcher) (skipEntry, skipDir bool) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we name it shouldSkipArchiveEntry and return a single bool? The caller can use info.IsDir() to determine whether to return nil or SkipDir

@@ -351,6 +355,8 @@ func ExportProject(client astrocore.CoreClient, projectID, organizationID, works

// createTarGzArchive creates a tar.gz archive of the given directory
func createTarGzArchive(sourceDir, targetFile string) error {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should add unit tests for this and the extract function, which covers the existing archive activity and the new gitignore skipping.

@pritt20 pritt20 merged commit c332756 into main Oct 6, 2025
7 checks passed
@pritt20 pritt20 deleted the ide_ignore_hidden_files branch October 6, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants