base on A high performance caching library for Go <p align="center"> <img src="docs/assets/logo.png" width="40%" height="auto" > <h2 align="center">In-memory caching library</h2> </p> <p align="center"> <a href="https://pkg.go.dev/github.com/maypok86/otter/v2"><img src="https://pkg.go.dev/badge/github.com/maypok86/otter/v2.svg" alt="Go Reference"></a> <img src="https://github.com/maypok86/otter/actions/workflows/test.yml/badge.svg" /> <a href="https://github.com/maypok86/otter/actions?query=branch%3Amain+workflow%3ATest" > <img src="https://gist.githubusercontent.com/maypok86/2aae2cd39836dc7c258df7ffec602d1c/raw/coverage.svg"/></a> <a href="https://github.com/maypok86/otter/releases"><img alt="GitHub Release" src="https://img.shields.io/github/v/release/maypok86/otter"></a> <a href="https://github.com/avelino/awesome-go"><img src="https://awesome.re/mentioned-badge.svg" alt="Mentioned in Awesome Go"></a> </p> Otter is designed to provide an excellent developer experience while maintaining high performance. It aims to address the shortcomings of its predecessors and incorporates design principles from high-performance libraries in other languages (such as [Caffeine](https://github.com/ben-manes/caffeine)). ## 📖 Contents - [Features](#features) - [Usage](#usage) - [Requirements](#requirements) - [Installation](#installation) - [Examples](#examples) - [Performance](#performance) - [Throughput](#throughput) - [Hit ratio](#hit-ratio) - [Memory consumption](#memory-consumption) - [Projects using Otter](#projects) - [Related works](#related-works) - [Contribute](#contribute) - [License](#license) ## ✨ Features <a id="features" /> Performance-wise, Otter provides: - [High hit rates](https://maypok86.github.io/otter/performance/hit-ratio/) across all workload types via [adaptive W-TinyLFU](https://dl.acm.org/citation.cfm?id=3274816) - [Excellent throughput](https://maypok86.github.io/otter/performance/throughput/) under high contention on most workload types - Among the lowest [memory overheads](https://maypok86.github.io/otter/performance/memory-consumption/) across all cache capacities - Automatic data structures configuration based on contention/parallelism and workload patterns Otter also provides a highly configurable caching API, enabling any combination of these optional features: - Size-based [eviction](https://maypok86.github.io/otter/user-guide/v2/features/eviction/#size-based) when a maximum is exceeded - Time-based [expiration](https://maypok86.github.io/otter/user-guide/v2/features/eviction/#time-based) of entries, measured since last access or last write - [Automatic loading](https://maypok86.github.io/otter/user-guide/v2/features/loading/) of entries into the cache - [Asynchronously refresh](https://maypok86.github.io/otter/user-guide/v2/features/refresh/) when the first stale request for an entry occurs - [Writes propagated](https://maypok86.github.io/otter/user-guide/v2/features/compute/) to an external resource - Accumulation of cache access [statistics](https://maypok86.github.io/otter/user-guide/v2/features/statistics/) - [Saving cache](https://maypok86.github.io/otter/user-guide/v2/features/persistence/) to a file and loading cache from a file ## 📚 Usage <a id="usage" /> For more details, see our [user's guide](https://maypok86.github.io/otter/user-guide/v2/getting-started/) and browse the [API docs](https://pkg.go.dev/github.com/maypok86/otter) for the latest release. ### 📋 Requirements <a id="requirements" /> Otter requires [Go](https://go.dev/) version [1.24](https://go.dev/doc/devel/release#go1.24.0) or above. ### 🛠️ Installation <a id="installation" /> #### With v1 ```shell go get -u github.com/maypok86/otter ``` #### With v2 ```shell go get -u github.com/maypok86/otter/v2 ``` See the [release notes](https://github.com/maypok86/otter/releases) for details of the changes. Note that otter only supports the two most recent minor versions of Go. Otter follows semantic versioning for the documented public API on stable releases. `v2` is the latest stable major version. ### ✏️ Examples <a id="examples" /> Otter uses a plain `Options` struct for cache configuration. Check out [otter.Options](https://pkg.go.dev/github.com/maypok86/otter/v2#Options) for more details. Note that all features are optional. You can create a cache that acts as a simple hash table wrapper, with near-zero memory overhead for unused features — thanks to [node code generation](https://github.com/maypok86/otter/blob/main/cmd/generator/main.go). **API Usage Example** ```go package main import ( "context" "time" "github.com/maypok86/otter/v2" "github.com/maypok86/otter/v2/stats" ) func main() { ctx := context.Background() // Create statistics counter to track cache operations counter := stats.NewCounter() // Configure cache with: // - Capacity: 10,000 entries // - 1 second expiration after last access // - 500ms refresh interval after writes // - Stats collection enabled cache := otter.Must(&otter.Options[string, string]{ MaximumSize: 10_000, ExpiryCalculator: otter.ExpiryAccessing[string, string](time.Second), // Reset timer on reads/writes RefreshCalculator: otter.RefreshWriting[string, string](500 * time.Millisecond), // Refresh after writes StatsRecorder: counter, // Attach stats collector }) // Phase 1: Test basic expiration // ----------------------------- cache.Set("key", "value") // Add initial value // Wait for expiration (1 second) time.Sleep(time.Second) // Verify entry expired if _, ok := cache.GetIfPresent("key"); ok { panic("key shouldn't be found") // Should be expired } // Phase 2: Test cache stampede protection // -------------------------------------- loader := func(ctx context.Context, key string) (string, error) { time.Sleep(200 * time.Millisecond) // Simulate slow load return "value1", nil // Return new value } // Concurrent Gets would deduplicate loader calls value, err := cache.Get(ctx, "key", otter.LoaderFunc[string, string](loader)) if err != nil { panic(err) } if value != "value1" { panic("incorrect value") // Should get newly loaded value } // Phase 3: Test background refresh // -------------------------------- time.Sleep(500 * time.Millisecond) // Wait until refresh needed // New loader that returns updated value loader = func(ctx context.Context, key string) (string, error) { time.Sleep(100 * time.Millisecond) // Simulate refresh return "value2", nil // Return refreshed value } // This triggers async refresh but returns current value value, err = cache.Get(ctx, "key", otter.LoaderFunc[string, string](loader)) if err != nil { panic(err) } if value != "value1" { // Should get old value while refreshing panic("loader shouldn't be called during Get") } // Wait for refresh to complete time.Sleep(110 * time.Millisecond) // Verify refreshed value v, ok := cache.GetIfPresent("key") if !ok { panic("key should be found") // Should still be cached } if v != "value2" { // Should now have refreshed value panic("refresh should be completed") } } ``` You can find more usage examples [here](https://maypok86.github.io/otter/user-guide/v2/examples/). ## 📊 Performance <a id="performance" /> The benchmark code can be found [here](./benchmarks). ### 🚀 Throughput <a id="throughput" /> Throughput benchmarks are a Go port of the caffeine [benchmarks](https://github.com/ben-manes/caffeine/blob/master/caffeine/src/jmh/java/com/github/benmanes/caffeine/cache/GetPutBenchmark.java). This microbenchmark compares the throughput of caches on a zipf distribution, which allows to show various inefficient places in implementations. You can find results [here](https://maypok86.github.io/otter/performance/throughput/). ### 🎯 Hit ratio <a id="hit-ratio" /> The hit ratio simulator tests caches on various traces: 1. Synthetic (Zipf distribution) 2. Traditional (widely known and used in various projects and papers) You can find results [here](https://maypok86.github.io/otter/performance/hit-ratio/). ### 💾 Memory consumption <a id="memory-consumption" /> This benchmark quantifies the additional memory consumption across varying cache capacities. You can find results [here](https://maypok86.github.io/otter/performance/memory-consumption/). ## 🏗️ Projects using Otter <a id="projects" /> Below is a list of known projects that use Otter: - [Grafana](https://github.com/grafana/grafana): The open and composable observability and data visualization platform. - [Centrifugo](https://github.com/centrifugal/centrifugo): Scalable real-time messaging server in a language-agnostic way. - [FrankenPHP](https://github.com/php/frankenphp): The modern PHP app server - [Unkey](https://github.com/unkeyed/unkey): Open source API management platform ## 🗃 Related works <a id="related-works" /> Otter is based on the following papers: - [BP-Wrapper: A Framework Making Any Replacement Algorithms (Almost) Lock Contention Free](https://www.researchgate.net/publication/220966845_BP-Wrapper_A_System_Framework_Making_Any_Replacement_Algorithms_Almost_Lock_Contention_Free) - [TinyLFU: A Highly Efficient Cache Admission Policy](https://dl.acm.org/citation.cfm?id=3149371) - [Adaptive Software Cache Management](https://dl.acm.org/citation.cfm?id=3274816) - [Denial of Service via Algorithmic Complexity Attack](https://www.usenix.org/legacy/events/sec03/tech/full_papers/crosby/crosby.pdf) - [Hashed and Hierarchical Timing Wheels](https://ieeexplore.ieee.org/document/650142) - [A large scale analysis of hundreds of in-memory cache clusters at Twitter](https://www.usenix.org/system/files/osdi20-yang.pdf) ## 👏 Contribute <a id="contribute" /> Contributions are welcome as always, before submitting a new PR please make sure to open a new issue so community members can discuss it. For more information please see [contribution guidelines](./CONTRIBUTING.md). Additionally, you might find existing open issues which can help with improvements. This project follows a standard [code of conduct](./CODE_OF_CONDUCT.md) so that you can understand what actions will and will not be tolerated. ## 📄 License <a id="license" /> This project is Apache 2.0 licensed, as found in the [LICENSE](./LICENSE). ", Assign "at most 3 tags" to the expected json: {"id":"6347","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"