base on 📈 Summarise your test failures as a build annotation # JUnit Annotate Buildkite Plugin [](https://buildkite.com/buildkite/plugins-junit-annotate) A [Buildkite plugin](https://buildkite.com/docs/agent/v3/plugins) that parses junit.xml artifacts (generated across any number of parallel steps) and creates a [build annotation](https://buildkite.com/docs/agent/v3/cli-annotate) listing the individual tests that failed. ## Example The following pipeline will run `test.sh` jobs in parallel, and then process all the resulting JUnit XML files to create a summary build annotation. ```yml steps: - command: test.sh parallelism: 50 artifact_paths: tmp/junit-*.xml - wait: ~ continue_on_failure: true - plugins: - junit-annotate#v2.6.0: artifacts: tmp/junit-*.xml ``` For scenarios where you have different artifact paths that you want to add as annotation then call the plugin multiple times in the pipeline with different contexts as shown below: ```yml steps: - command: test.sh parallelism: 50 artifact_paths: tmp/junit-*.xml - command: anothertest.sh artifact_paths: artifacts/junit-*.xml - wait: ~ continue_on_failure: true - plugins: - junit-annotate#v2.6.0: artifacts: tmp/junit-*.xml - plugins: - junit-annotate#v2.6.0: artifacts: artifacts/junit-*.xml context: junit-artifacts ``` ## Configuration ### `artifacts` (required) The artifact glob path to find the JUnit XML files. Example: `tmp/junit-*.xml` ### `always-annotate` (optional, boolean) Forces the creation of the annotation even when no failures or errors are found ### `context` (optional) Default: `junit` The buildkite annotation context to use. Useful to differentiate multiple runs of this plugin in a single pipeline. ### `job-uuid-file-pattern` (optional) Default: `-(.*).xml` The regular expression (with capture group) that matches the job UUID in the junit file names. This is used to create the job links in the annotation. To use this, configure your test reporter to embed the `$BUILDKITE_JOB_ID` environment variable into your junit file names. For example `"junit-buildkite-job-$BUILDKITE_JOB_ID.xml"`. ### `failure-format` (optional) This setting controls the format of your failed test in the main annotation summary. There are two options for this: * `classname` (the default) * displays: `MyClass::UnderTest text of the failed expectation in path.to.my_class.under_test` * `file` * displays: `MyClass::UnderTest text of the failed expectation in path/to/my_class/under_test.file_ext` ### `fail-build-on-error` (optional) Default: `false` If this setting is true and any errors are found in the JUnit XML files during parsing, the annotation step will exit with a non-zero value, which should cause the build to fail. ### `failed-download-exit-code` (optional, integer) Default: `2` Exit code of the plugin if the call to `buildkite-agent artifact download` fails. ### `min-tests` (optional, integer) Minimum amount of run tests that need to be analyzed or a failure will be reported. It is useful to ensure that tests are actually run and report files to analyze do contain information. ### `report-skipped` (optional, boolean) Default: `false` Will add a list of skipped tests at the end of the annotation. Note that even if there are skipped tests, the annotation may not be added unless other options or results of the processing forces it to. ### `report-slowest` (optional) Default: `0` Include the specified number of slowest tests in the annotation. The annotation will always be shown. ### `ruby-image` (optional) The docker image to use for running the analysis code. Must be a valid image reference that can run the corresponding ruby code and the agent running the step must be able to pull it if not already present. Default: `ruby:3.1-alpine@sha256:a39e26d0598837f08c75a42c8b0886d9ed5cc862c4b535662922ee1d05272fca` ### `run-in-docker` (optional, boolean) Default: `true` Controls whether the JUnit processing should run inside a Docker container. When set to `false`, the processing will run directly on the host using the system's Ruby installation. ## Developing To run testing, shellchecks and plugin linting use use `bk run` with the [Buildkite CLI](https://github.com/buildkite/cli). ```bash bk run ``` Or if you want to run just the plugin tests, you can use the docker [Plugin Tester](https://github.com/buildkite-plugins/buildkite-plugin-tester): ```bash docker run --rm -ti -v "${PWD}":/plugin buildkite/plugin-tester:latest ``` To test the Ruby code with `rake` in docker: ```bash docker-compose run --rm ruby ``` To test your plugin in your builds prior to opening a pull request, you can refer to your fork and SHA from a branch in your `pipeline.yml`. ``` steps: - label: Annotate plugins: - YourGithubHandle/junit-annotate#v2.6.0: ... ``` ## License MIT (see [LICENSE](LICENSE)) ", Assign "at most 3 tags" to the expected json: {"id":"1664","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"