Skip to content

Conversation

@Wyverald
Copy link
Member

@Wyverald Wyverald commented Feb 14, 2024

  • Added a new parameter watch to path.readdir() that allows one to watch for changes in entries under a given directory.
    • only names are watched; 'entry types' are not. This somewhat matches the return value of path.readdir(), which only contains entry names
    • same restrictions as rctx.watch() in terms of which paths can be watched and which cannot; similarly for mctx.
  • Made a big-ish refactor that eliminated the two RepoRecordedInput.File subclasses, and pulled the difference into a separate RepoRecordedInput.RepoCacheFriendlyPath instead. This new path class is used by the new RepoRecordedInput.Dirents as well.

Work towards #20952.

@Wyverald Wyverald requested a review from lberki as a code owner February 14, 2024 02:20
@github-actions github-actions bot added team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file. awaiting-review PR is awaiting review from an assigned reviewer labels Feb 14, 2024
Copy link
Contributor

@ahumesky ahumesky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval for the android-related changes

@Wyverald
Copy link
Member Author

sorry about the noise -- there's literally no android-related changes in this PR. GitHub really doesn't handle stacked PRs well. I'll get it sorted eventually, promise...

@Wyverald Wyverald removed the request for review from ted-xie February 14, 2024 22:18
@Wyverald Wyverald changed the title rctx.watch_dir() Make path.readdir() watch the dirents by default Feb 15, 2024
@Wyverald Wyverald force-pushed the wyv-watch-dir branch 3 times, most recently from 650dc3e to 8874358 Compare February 15, 2024 02:25
}

@StarlarkMethod(
name = "watch_dir",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

WDYT about updating watch() instead to watch the list of directory entries if its target is a directory and the contents of a file if its target is a file? That would spare you a method on the API and I can't immediately identify any case that would be impossible with that setup.

Alternatively, WDYT about extending watch() to have arguments parallel with glob(): watch([<file name>]) would watch a list of specified files, watch(["*.txt"]) would watch every text file and watch(["**"]) (if implemented later), would be a recursive glob. It would also provide a convenient place to add exclude= and other accoutrements as needed. You might even be able to piggyback upon the implementation of glob().

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Doing my first review round of the day, I'll assume that you are still working on this change and that's why you haven't replied on this thread)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad -- it look like the changes were more extensive than I thought. Reviewed (mind answering the question above?)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fwiw i didn't reply to this thread yesterday because it didn't exist yesterday :P looks like you drafted it but never sent it until today.

but anyway, putting a glob pattern directly in rctx.watch is a bit awkward because you're presumably globbing from the working directory (ie. where the repo is being fetched into). so rctx.watch(['*.txt']) wouldn't make sense, but path.glob(...) probably would.

in any case i don't think we could realistically piggyback onto GlobFunction as it just works too differently: worrying about package boundaries, returning labels instead of paths, not digesting file contents, etc etc.

regarding folding watch_dir into watch: i ended up going with path.readdir instead, which i think it's a more natural api. we could actually still choose to make rctx.watch on a dir be sensitive to dirents in the future, but i don't think it's as important a question now. let me know if you disagree.

Copy link
Contributor

@lberki lberki Feb 16, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad it is then. I'm not the most relaxed and well rested I have ever been...

I have only one concern with the path.readdir() idea: we currently have the read() / watch() dichotomy for files and the "just use readdir() for everything" idea makes it so that we don't have it for directories. I'm not sure it matters, though. At the very least, this decision allows you to postpone the decision about whether you want the recursive watching syntax to be the same as that of globbing (I mean, that's probably like a day, but still...) and teaching readdir() to watch the file system is a pretty obvious win.

- Added a new parameter `watch` to `path.readdir()` that allows one to watch for changes in entries under a given directory.
  - only names are watched; 'entry types' are not. This somewhat matches the return value of `path.readdir()`, which only contains entry names
  - same restrictions as `rctx.watch()` in terms of which paths can be watched and which cannot; similarly for `mctx`.
- Made a big-ish refactor that eliminated the two `RepoRecordedInput.File` subclasses, and pulled the difference into a separate `RepoRecordedInput.RepoCacheFriendlyPath` instead. This new path class is used by the new `RepoRecordedInput.Dirents` as well.

Work towards #20952.
@Wyverald Wyverald requested review from a team as code owners February 15, 2024 20:46
@Wyverald Wyverald requested review from aranguyen and removed request for a team February 15, 2024 20:46
@Wyverald Wyverald changed the base branch from wyv-watch-exists to master February 15, 2024 20:47
@Wyverald Wyverald removed request for a team and aranguyen February 15, 2024 20:47
@github-actions github-actions bot removed the awaiting-review PR is awaiting review from an assigned reviewer label Feb 16, 2024
@Wyverald Wyverald deleted the wyv-watch-dir branch February 16, 2024 20:56
Wyverald added a commit that referenced this pull request Feb 20, 2024
- Added a new parameter `watch` to `path.readdir()` that allows one to watch for changes in entries under a given directory.
  - only names are watched; 'entry types' are not. This somewhat matches the return value of `path.readdir()`, which only contains entry names
  - same restrictions as `rctx.watch()` in terms of which paths can be watched and which cannot; similarly for `mctx`.
- Made a big-ish refactor that eliminated the two `RepoRecordedInput.File` subclasses, and pulled the difference into a separate `RepoRecordedInput.RepoCacheFriendlyPath` instead. This new path class is used by the new `RepoRecordedInput.Dirents` as well.

Work towards #20952.

Closes #21341.

PiperOrigin-RevId: 607772207
Change-Id: Ibba2b3389acd23e0a703818fec2cd58321a9b896
Wyverald added a commit that referenced this pull request Feb 20, 2024
- Added a new parameter `watch` to `path.readdir()` that allows one to watch for changes in entries under a given directory.
  - only names are watched; 'entry types' are not. This somewhat matches the return value of `path.readdir()`, which only contains entry names
  - same restrictions as `rctx.watch()` in terms of which paths can be watched and which cannot; similarly for `mctx`.
- Made a big-ish refactor that eliminated the two `RepoRecordedInput.File` subclasses, and pulled the difference into a separate `RepoRecordedInput.RepoCacheFriendlyPath` instead. This new path class is used by the new `RepoRecordedInput.Dirents` as well.

Work towards #20952.

Closes #21341.

PiperOrigin-RevId: 607772207
Change-Id: Ibba2b3389acd23e0a703818fec2cd58321a9b896
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

team-ExternalDeps External dependency handling, remote repositiories, WORKSPACE file.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants