fix(ci): migrate benchmarks to benchmarking-platform trigger#607
Open
jbachorik wants to merge 29 commits into
Open
fix(ci): migrate benchmarks to benchmarking-platform trigger#607jbachorik wants to merge 29 commits into
jbachorik wants to merge 29 commits into
Conversation
This comment has been minimized.
This comment has been minimized.
…ost-benchmark jobs
Contributor
CI Test ResultsRun: #27865993044 | Commit:
Status Overview
Legend: ✅ passed | ❌ failed | ⚪ skipped | 🚫 cancelled Summary: Total: 32 | Passed: 32 | Failed: 0 Updated: 2026-06-20 09:00:35 UTC |
Contributor
Benchmark ResultsPipeline: https://gitlab.ddbuild.io/DataDog/apm-reliability/benchmarking-platform/-/pipelines/120034731 Commit: ✅ Within expected boundariesNo significant runtime deltas (all within run-to-run noise) and no internal-counter outliers. Runtime details (per benchmark × JDK)
Internal counter details (ddprof)ddprof internal counters, latest / dev (✅ = 0, · = unavailable):
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bc5c810c9d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?:
Replaces the broken inline benchmark jobs with a benchmarking-platform bridge trigger. The old jobs ran the profiler inline via shell scripts and uploaded results to S3 themselves; the new approach delegates entirely to the BP pipeline at
DataDog/apm-reliability/benchmarking-platform@java-profiler.Motivation:
The existing benchmark CI was broken and unmaintained. The benchmarking-platform provides a reliable, standardised infra for running and tracking reliability benchmarks.
Additional Notes:
benchmarks-triggeris a GitLab bridge job — it cannot appear inneeds:of downstream jobs, sopost-benchmarks-pr-commentandpublish-benchmark-gh-pagesnow run in apost-benchmarksstage ordered afterbenchmarks.download-s3-reports.shfetches reports uploaded by BP unders3://relenv-benchmarking-data/java-profiler/${CI_PIPELINE_ID}/.images.ymlinclude is removed — the BP side manages its own Docker image build.jb/bench-memory-limit-fixis superseded by this PR (the memory-heavy aarch64 jobs are gone).DataDog/benchmarking-platformPR Run tests on musl aarch64 #190.How to test the change?:
Push to a branch and verify the GitLab pipeline shows a
benchmarks-triggerbridge job that fires the BP downstream pipeline. Thepost-benchmarks-pr-commentandpublish-benchmark-gh-pagesjobs should run in thepost-benchmarksstage after the trigger completes.For Datadog employees: