base on Python-based research interface for blackbox and hyperparameter optimization, based on the internal Google Vizier Service. <figure> <img src="docs/assets/vizier_logo2.png" width=20% align="right"/> </figure> # Open Source Vizier: Reliable and Flexible Black-Box Optimization. [](https://badge.fury.io/py/google-vizier) [](https://github.com/google/vizier/actions/workflows/ci.yml?query=branch%3Amain)  [**Google AI Blog**](https://ai.googleblog.com/2023/02/open-source-vizier-towards-reliable-and.html) | [**Getting Started**](#getting_started) | [**Documentation**](#documentation) | [**Installation**](#installation) | [**Citing and Highlights**](#citing_vizier) ## What is Open Source (OSS) Vizier? [OSS Vizier](https://arxiv.org/abs/2207.13676) is a Python-based service for black-box optimization and research, based on [Google Vizier](https://dl.acm.org/doi/10.1145/3097983.3098043), one of the first hyperparameter tuning services designed to work at scale. <figure> <p align="center" width=65%> <img src="docs/assets/oss_vizier_service.gif"/> <br> <em><b>OSS Vizier's distributed client-server system. Animation by Tom Small.</b></em> </p> </figure> ## Getting Started <a name="getting_started"></a> As a basic example for users, below shows how to tune a simple objective using all flat search space types: ```python from vizier.service import clients from vizier.service import pyvizier as vz # Objective function to maximize. def evaluate(w: float, x: int, y: float, z: str) -> float: return w**2 - y**2 + x * ord(z) # Algorithm, search space, and metrics. study_config = vz.StudyConfig(algorithm='DEFAULT') study_config.search_space.root.add_float_param('w', 0.0, 5.0) study_config.search_space.root.add_int_param('x', -2, 2) study_config.search_space.root.add_discrete_param('y', [0.3, 7.2]) study_config.search_space.root.add_categorical_param('z', ['a', 'g', 'k']) study_config.metric_information.append(vz.MetricInformation('metric_name', goal=vz.ObjectiveMetricGoal.MAXIMIZE)) # Setup client and begin optimization. Vizier Service will be implicitly created. study = clients.Study.from_study_config(study_config, owner='my_name', study_id='example') for i in range(10): suggestions = study.suggest(count=2) for suggestion in suggestions: params = suggestion.parameters objective = evaluate(params['w'], params['x'], params['y'], params['z']) suggestion.complete(vz.Measurement({'metric_name': objective})) ``` ## Documentation <a name="documentation"></a> OSS Vizier's interface consists of [three main APIs](https://oss-vizier.readthedocs.io/en/latest/guides/index.html): * [**User API:**](https://oss-vizier.readthedocs.io/en/latest/guides/index.html#for-users) Allows a user to optimize their blackbox objective and optionally setup a server for distributed multi-client settings. * [**Developer API:**](https://oss-vizier.readthedocs.io/en/latest/guides/index.html#for-developers) Defines abstractions and utilities for implementing new optimization algorithms for research and to be hosted in the service. * [**Benchmarking API:**](https://oss-vizier.readthedocs.io/en/latest/guides/index.html#for-benchmarking) A wide collection of objective functions and methods to benchmark and compare algorithms. Additionally, it contains [advanced API](https://oss-vizier.readthedocs.io/en/latest/advanced_topics/index.html) for: * [**Tensorflow Probability:**](https://oss-vizier.readthedocs.io/en/latest/advanced_topics/index.html#tensorflow-probability) For writing Bayesian Optimization algorithms using Tensorflow Probability and Flax. * [**PyGlove:**](https://oss-vizier.readthedocs.io/en/latest/advanced_topics/index.html#pyglove) For large-scale evolutionary experimentation and program search using OSS Vizier as a distributed backend. Please see OSS Vizier's [ReadTheDocs documentation](https://oss-vizier.readthedocs.io/) for detailed information. ## Installation <a name="installation"></a> **Quick start:** For tuning objectives using our state-of-the-art JAX-based Bayesian Optimizer, run: ```bash pip install google-vizier[jax] ``` ### Advanced Installation **Minimal installation:** To install only the core service and client APIs from `requirements.txt`, run: ```bash pip install google-vizier ``` **Full installation:** To support all algorithms and benchmarks, run: ```bash pip install google-vizier[all] ``` **Specific installation:** If you only need a specific part "X" of OSS Vizier, run: ```bash pip install google-vizier[X] ``` which installs add-ons from `requirements-X.txt`. Possible options: * `requirements-jax.txt`: Jax libraries shared by both algorithms and benchmarks. * `requirements-tf.txt`: Tensorflow libraries used by benchmarks. * `requirements-algorithms.txt`: Additional repositories (e.g. EvoJAX) for algorithms. * `requirements-benchmarks.txt`: Additional repositories (e.g. NASBENCH-201) for benchmarks. * `requirements-test.txt`: Libraries needed for testing code. **Developer installation:** To install up to the latest commit, run: ```bash pip install google-vizier-dev[X] ``` Check if all unit tests work by running `run_tests.sh` after a full installation. OSS Vizier requires Python 3.10+, while client-only packages require Python 3.8+. ## Citing and Highlights <a name="citing_vizier"></a> <ins>**Citing Vizier:**</ins> Please consider citing the appropriate paper(s): [Algorithm](https://arxiv.org/abs/2408.11527), [OSS Package](https://arxiv.org/abs/2207.13676), and [Google System](https://dl.acm.org/doi/10.1145/3097983.3098043) if you found any of them useful. <ins>**Highlights:**</ins> We track [notable users](https://oss-vizier.readthedocs.io/en/latest/highlights/applications.html) and [media attention](https://oss-vizier.readthedocs.io/en/latest/highlights/media.html) - let us know if OSS Vizier was helpful for your work. Thanks! ```bibtex @article{gaussian_process_bandit, author = {Xingyou Song and Qiuyi Zhang and Chansoo Lee and Emily Fertig and Tzu-Kuo Huang and Lior Belenki and Greg Kochanski and Setareh Ariafar and Srinivas Vasudevan and Sagi Perel and Daniel Golovin}, title = {The Vizier Gaussian Process Bandit Algorithm}, journal = {Google DeepMind Technical Report}, year = {2024}, eprinttype = {arXiv}, eprint = {2408.11527}, } @inproceedings{oss_vizier, author = {Xingyou Song and Sagi Perel and Chansoo Lee and Greg Kochanski and Daniel Golovin}, title = {Open Source Vizier: Distributed Infrastructure and API for Reliable and Flexible Black-box Optimization}, booktitle = {Automated Machine Learning Conference, Systems Track (AutoML-Conf Systems)}, year = {2022}, } @inproceedings{google_vizier, author = {Daniel Golovin and Benjamin Solnik and Subhodeep Moitra and Greg Kochanski and John Karro and D. Sculley}, title = {Google Vizier: {A} Service for Black-Box Optimization}, booktitle = {Proceedings of the 23rd {ACM} {SIGKDD} International Conference on Knowledge Discovery and Data Mining, Halifax, NS, Canada, August 13 - 17, 2017}, pages = {1487--1495}, publisher = {{ACM}}, year = {2017}, url = {https://doi.org/10.1145/3097983.3098043}, doi = {10.1145/3097983.3098043}, } ``` ", Assign "at most 3 tags" to the expected json: {"id":"11581","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"