Summary

Fix #8071

Implements a two-layer in-memory caching system to reduce redundant API calls during pipeline execution, as proposed in #8071.

Layer 1: HTTP Response Cache (pkg/core/httpclient/)

A cachingTransport wrapping the existing HTTP transport chain caches successful (2xx) GET responses in memory. All plugins benefit automatically since they go through the centralized httpclient package — zero plugin changes required.

• Enabled/disabled via EnableHTTPCache()/DisableHTTPCache() in Engine.Run()
• Only caches GET requests with 2xx status codes
• Thread-safe via sync.RWMutex

Layer 2: Source Result Cache (pkg/core/cache/)

A SourceCache shared across pipelines deduplicates source executions when the same Kind+Spec appears in multiple pipelines (or multiple times within one). The cache key is a SHA256 of only Kind + sanitized Spec (via ReportConfig()), so sources with the same plugin config but different names, transformers, or dependsOn share a cache entry.

• Transformers are not cached — they are re-applied on cache hit
• Only successful results are cached
• Injected from Engine into each Pipeline before execution

How they complement each other

# Single manifest, three sources, one network call:

source: golang-1.23
  http cache store: https://go.dev/dl/?mode=json&include=all   # fresh API call

source: golang-1.22
  http cache hit: https://go.dev/dl/?mode=json&include=all     # different filter, same URL

source: golang-1.23-dup
  source cache hit for "Get Golang 1.23 duplicate"             # same Kind+Spec

Test

To test this pull request, you can run the following commands:

go test -race ./pkg/core/cache/ ./pkg/core/httpclient/ ./pkg/core/pipeline/ ./pkg/core/engine/

Additional Information

Checklist

• I have updated the documentation via pull request in website repository.

Tradeoff

• The cache.Key() function calls resource.New() to obtain ReportConfig(), which means the plugin is instantiated twice on a cache miss (once for key, once for execution). This is acceptable for now as plugin constructors are lightweight.
• The shell resource kind is cached like any other resource. For sources with side effects, this may need an opt-out mechanism in a future iteration — pending maintainer input.
• HTTP cache keys use the URL string only. Requests to the same URL with different auth headers would share a cache entry, which is correct for updatecli's usage patterns (consistent auth per endpoint within a run).

Potential improvement

• Add a singleflight.Group to coalesce concurrent cache misses for the same key if pipeline execution is ever parallelized.
• Add an opt-out config flag (nocache: true) per source for resources with side effects.
• Optimize double resource.New() by threading the instantiated resource through the cache key computation.
• Extend caching to conditions (second cache map as described in Feature Request: Cache result resource #8071).