Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support for caching multiple directories #63

Open
smsunarto opened this issue Apr 9, 2024 · 4 comments
Open

Support for caching multiple directories #63

smsunarto opened this issue Apr 9, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@smsunarto
Copy link

Currently, the plugin only takes in a single directory/path for caching. It would be great to have support for caching multiple directories so you don't have to create multiple entries of cache.

@toote toote added the enhancement New feature or request label Apr 22, 2024
@toote
Copy link
Contributor

toote commented Aug 3, 2024

@smsunarto this is an interesting idea but I can't think of a good reason to do this that wouldn't overcomplicate the code of the plugin without a lot of clarifications in the documentation. In particular, when specifying multiple paths my brain automatically assumes that they are to be treated as a unit so they should all be part of the the same cache entry. Unfortunately, that would mean that the cache entry would only work if the same paths are specified and in the same order (and/or further code will be necessary to make the entry calculation stable) which would be counter-intuitive.

On the other hand, if we are talking about each path having its own cache entry, the code changes to the plugin itself is not too complicated. But then you would be assuming that they share the same configuration for every other setting, or we would need to complicate the settings for the plugin A LOT to handle all possible cases for very little gain when using multiple plugin entries in the step makes a lot more sense and is already flexible enough.

Or is there anything I am missing here?

@smsunarto
Copy link
Author

On the other hand, if we are talking about each path having its own cache entry, the code changes to the plugin itself is not too complicated. But then you would be assuming that they share the same configuration for every other setting,

I think this is perfectly fine for starters; I'd imagine this covers the bases for a lot of people who would need to cache multiple directories.

For example: one of our use case is for Go linting and we want to cache both the Go linter artifacts (to make subsequent runs faster) and the golangci-lint binary, which is in separate paths.

I'd imagine people who needs to have different cache configurations would be fine with creating a separate cache entry.

@ianwremmel
Copy link

seconding, I have a similar need for two caches. I have a node monorepo orchestrated by nx. I need to cache /root/.npm to speed up npm ci and I need to cache the checkout directory's .nx folder to avoid a full rebuild in each step.

@thiagoboa
Copy link

Or is there anything I am missing here?

In our case, we have a node monorepo where multiple workers can be added to the workers folder. Each with its dependencies and, therefore their own node_modules. I would be a pain to edit our pipeline file each time a new worker is added just to include its node_modules path to the cache config. Whereas a simple workers/*/node_modules would be much more desirable.

On the other hand, if we are talking about each path having its own cache entry, the code changes to the plugin itself is not too complicated. But then you would be assuming that they share the same configuration for every other setting,

That would be perfectly fine to have all of the node_modules under workers to share the same config.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

4 participants