base on OSV-SCALIBR: A library for Software Composition Analysis # OSV-SCALIBR [](https://pkg.go.dev/github.com/google/osv-scalibr) OSV-SCALIBR (Software Composition Analysis Library) is an extensible library providing: - File system scanner used to extract software inventory data (e.g. installed language packages) and detect known vulnerabilities or generate SBOMs. See the [list of currently supported software inventory types](docs/supported_inventory_types.md). - Container analysis functionality (e.g. layer-based extraction) - Guided Remediation (generating upgrade patches for transitive vulnerabilities) - And more! This can be used as a library with a custom wrapper to perform scans on e.g. container images or remote hosts, or via the [OSV-Scanner CLI](https://github.com/google/osv-scanner). It comes with built-in plugins for inventory extraction and vulnerability detection and it also allows users to run their custom plugins. ## Prerequisites To build OSV-SCALIBR, you'll need to have `go` installed. Follow https://go.dev/doc/install. ## How to use ### Via the OSV-Scanner CLI If your use case is known vulnerability scanning and extraction in a CLI context, check out the [OSV-Scanner usage guide](https://google.github.io/osv-scanner/usage/). **Note:** Note all OSV-SCALIBR functionality is available via OSV-Scanner yet. Check out [this issue](https://github.com/google/osv-scanner/issues/1701) for the current status. ### Via the OSV-SCALIBR wrapper binary 1. `go install github.com/google/osv-scalibr/binary/scalibr@latest` 1. `scalibr --result=result.textproto` See the [result proto definition](/binary/proto/scan_result.proto) for details about the scan result format. Run `scalibr --help` for a list of additional CLI args. ### As a library: 1. Import `github.com/google/osv-scalibr` into your Go project 1. Create a new [scalibr.ScanConfig](/scalibr.go#L36) struct, configure the extraction and detection plugins to run 1. Call `scalibr.New().Scan()` with the config 1. Parse the returned [scalibr.ScanResults](/scalibr.go#L50) See below for an example code snippet. ### On a container image Add the `--remote-image` flag to scan a remote container image. Example: ``` scalibr --result=result.textproto --remote-image=alpine@sha256:0a4eaa0eecf5f8c050e5bba433f58c052be7587ee8af3e8b3910ef9ab5fbe9f5 ``` Or the `--image-tarball` flag to scan a locally saved image tarball like ones produced with `docker save my-image > my-image.tar`. Example: ``` scalibr --result=result.textproto --image-tarball=my-image.tar ``` ### SPDX generation OSV-SCALIBR supports generating the result of inventory extraction as an SPDX v2.3 file in json, yaml or tag-value format. Example usage: ``` scalibr -o spdx23-json=result.spdx.json ``` Some fields in the generated SPDX can be overwritten: ``` scalibr -spdx-document-name="Custom name" --spdx-document-namespace="Custom-namespace" --spdx-creators=Organization:Google -o spdx23-json=result.spdx.json ``` ## Running built-in plugins ### With the standalone binary The binary runs SCALIBR's "recommended" internal plugins by default. You can enable more plugins with the `--extractors=` and `--detectors=` flags. See the definition files for a list of all built-in plugins and their CLI flags ([extractors (fs)](/extractor/filesystem/list/list.go#L26), [detectors](/detector/list/list.go#L26)). ### With the library A collection of all built-in plugin modules can be found in the definition files ([extractors](/extractor/filesystem/list/list.go#L26), [detectors](/detector/list/list.go#L26)). To enable them, just import the module and add the appropriate plugin names to the scan config, e.g. ``` import ( "context" scalibr "github.com/google/osv-scalibr" el "github.com/google/osv-scalibr/extractor/filesystem/list" dl "github.com/google/osv-scalibr/detector/list" scalibrfs "github.com/google/osv-scalibr/fs" ) exs, _ := el.ExtractorsFromNames([]string{"language"}) dets, _ := dl.DetectorsFromNames([]string{"cis"}) cfg := &scalibr.ScanConfig{ ScanRoots: scalibrfs.RealFSScanRoots("/"), FilesystemExtractors: exs, Detectors: dets, } results := scalibr.New().Scan(context.Background(), cfg) ``` You can also specify your scanning host's capabilities to only enable plugins whose requirements are satisfied (e.g. network access, OS-specific plugins): ``` import ( ... "github.com/google/osv-scalibr/plugin" ) capab := &plugin.Capabilities{ OS: plugin.OSLinux, Network: plugin.NetworkOnline, DirectFS: true, RunningSystem: true, } ... cfg := &scalibr.ScanConfig{ ScanRoots: scalibrfs.RealFSScanRoots("/"), FilesystemExtractors: el.FilterByCapabilities(exs, capab), Detectors: dl.FilterByCapabilities(dets, capab), } ... ``` ## Creating + running custom plugins Custom plugins can only be run when using OSV-SCALIBR as a library. 1. Create an implementation of the OSV-SCALIBR [Extractor](/extractor/filesystem/extractor.go#L30) or [Detector](/detector/detector.go#L28) interface. 2. Add the newly created struct to the scan config and run the scan, e.g. ``` import ( "github.com/google/osv-scalibr/extractor/filesystem" scalibr "github.com/google/osv-scalibr" ) cfg := &scalibr.ScanConfig{ Root: "/", FilesystemExtractors: []extractor.Extractor{&myExtractor{}}, } results := scalibr.New().Scan(context.Background(), cfg) ``` ### A note on cross-platform OSV-SCALIBR is compatible with Linux and has experimental support for Windows and Mac. When a new plugin is implemented for OSV-SCALIBR, we need to ensure that it will not break other platforms. Our runners will generally catch compatibility issue, but to ensure everything is easy when implementing a plugin, here are a few recommendations to keep in mind: * Ensure you work with file paths using the `filepath` library. For example, avoid using `/my/path` but prefer `filepath.Join('my', 'path')` instead. * If the plugin can only support one system (e.g. a windows-specific detector), the layout will generally be to have two versions of the file: * `file_system.go`: where `system` is the targeted system (e.g. `file_windows.go`) that contains the code specific to the target system. It must also contain the adequate go build constraint. * `file_dummy.go`: contains the code for every other system. It generally does nothing and just ensures that the code compiles on that system; * Because of the way our internal automation works, we generally require unit tests to be defined for every platform and be filtered out dynamically if not compatible. In other words, a test should be filtered in/out using `if runtime.GOOS` rather than a `//go:build` constraint. Here is an [example](https://github.com/google/osv-scalibr/commit/7a87679f5c688e7bac4527d29c1823597a52bb40#diff-72efad005e0fbfe34c60e496dfb55ec15fc50f4b12be0934f08a3acaf7733616L79). ## Custom logging You can make the OSV-SCALIBR library log using your own custom logger by passing an implementation of the [`log.Logger`](/log/log.go#L22) interface to `log.SetLogger()`: ``` import ( customlog "path/to/custom/log" "github.com/google/osv-scalibr/log" scalibr "github.com/google/osv-scalibr" ) cfg := &scalibr.ScanConfig{ScanRoot: "/"} log.SetLogger(&customlog.Logger{}) results := scalibr.New().Scan(context.Background(), cfg) log.Info(results) ``` ## Contributing Read how to [contribute to OSV-SCALIBR](CONTRIBUTING.md). To build and test your local changes, run `make` and `make test`. A local `scalibr` binary will be generated in the repo base. Some of your code contributions might require regenerating protos. This can happen when, say, you want to contribute a new inventory type. For such cases, you'll need install a few dependencies * `protoc`: Install the appropriate [precompiled protoc binary](https://grpc.io/docs/protoc-installation/#install-pre-compiled-binaries-any-os). * For Mac, you can also [install via HomeBrew](https://grpc.io/docs/protoc-installation/#install-using-a-package-manager). * `protoc-gen-go`: Run `go install google.golang.org/protobuf/cmd/protoc-gen-go` and then run `make protos` or `./build_protos.sh`. ## Disclaimers OSV-SCALIBR is not an official Google product. ", Assign "at most 3 tags" to the expected json: {"id":"12780","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"