base on A Pythonic framework to simplify AI service building <img src="https://raw.githubusercontent.com/leptonai/leptonai/main/assets/logo.svg" height=100> # Lepton AI **A Pythonic framework to simplify AI service building** <a href="https://lepton.ai/">Homepage</a> • <a href="https://dashboard.lepton.ai/playground">API Playground</a> • <a href="https://github.com/leptonai/examples">Examples</a> • <a href="https://lepton.ai/docs/">Documentation</a> • <a href="https://lepton.ai/references">CLI References</a> • <a href="https://twitter.com/leptonai">Twitter</a> • <a href="https://leptonai.medium.com/">Blog</a> The LeptonAI Python library allows you to build an AI service from Python code with ease. Key features include: - A Pythonic abstraction `Photon`, allowing you to convert research and modeling code into a service with a few lines of code. - Simple abstractions to launch models like those on [HuggingFace](https://huggingface.co) in few lines of code. - Prebuilt examples for common models such as Llama, SDXL, Whisper, and others. - AI tailored batteries included such as autobatching, background jobs, etc. - A client to automatically call your service like native Python functions. - Pythonic configuration specs to be readily shipped in a cloud environment. ## Getting started with one-liner Install the library with: ```shell pip install -U leptonai ``` This installs the `leptonai` Python library, as well as the commandline interface `lep`. You can then launch a HuggingFace model, say `gpt2`, in one line of code: ```python lep photon runlocal --name gpt2 --model hf:gpt2 ``` If you have access to the Llama2 model ([apply for access here](https://huggingface.co/meta-llama/Llama-2-7b)) and you have a reasonably sized GPU, you can launch it with: ```python # hint: you can also write `-n` and `-m` for short lep photon runlocal -n llama2 -m hf:meta-llama/Llama-2-7b-chat-hf ``` (Be sure to use the `-hf` version for Llama2, which is compatible with huggingface pipelines.) You can then access the service with: ```python from leptonai.client import Client, local c = Client(local(port=8080)) # Use the following to print the doc print(c.run.__doc__) print(c.run(inputs="I enjoy walking with my cute dog")) ``` Fully managed Llama2 models and CodeLlama models can be found in the [playground](https://dashboard.lepton.ai/playground). Many standard HuggingFace pipelines are supported - find out more details in the [documentation](https://www.lepton.ai/docs/advanced/prebuilt_photons#hugging-face-photons). Not all HuggingFace models are supported though, as many of them contain custom code and are not standard pipelines. If you find a popular model you would like to support, please [open an issue or a PR](https://github.com/leptonai/leptonai/issues/new). ## Checking out more examples You can find out more examples from the [examples repository](https://github.com/leptonai/examples). For example, launch the Stable Diffusion XL model with: ```shell git clone [email protected]:leptonai/examples.git cd examples ``` ```python lep photon runlocal -n sdxl -m advanced/sdxl/sdxl.py ``` Once the service is running, you can access it with: ```python from leptonai.client import Client, local c = Client(local(port=8080)) img_content = c.run(prompt="a cat launching rocket", seed=1234) with open("cat.png", "wb") as fid: fid.write(img_content) ``` or access the mounted Gradio UI at [http://localhost:8080/ui](http://localhost:8080/ui). Check the [README file](https://github.com/leptonai/examples/blob/main/advanced/sdxl/README.md) for more details. A fully managed SDXL is hosted at [https://dashboard.lepton.ai/playground/sdxl](https://dashboard.lepton.ai/playground/sdxl) with API access. ## Writing your own photons Writing your own photon is simple: write a Python Photon class and decorate functions with `@Photon.handler`. As long as your input and output are JSON serializable, you are good to go. For example, the following code launches a simple echo service: ```python # my_photon.py from leptonai.photon import Photon class Echo(Photon): @Photon.handler def echo(self, inputs: str) -> str: """ A simple example to return the original input. """ return inputs ``` You can then launch the service with: ```shell lep photon runlocal -n echo -m my_photon.py ``` Then, you can use your service as follows: ```python from leptonai.client import Client, local c = Client(local(port=8080)) # will print available paths print(c.paths()) # will print the doc for c.echo. You can also use `c.echo?` in Jupyter. print(c.echo.__doc__) # will actually call echo. c.echo(inputs="hello world") ``` For more details, checkout the [documentation](https://lepton.ai/docs/) and the [examples](https://github.com/leptonai/examples). ## Contributing Contributions and collaborations are welcome and highly appreciated. Please check out the [contributor guide](https://github.com/leptonai/leptonai/blob/main/CONTRIBUTING.md) for how to get involved. ## License The Lepton AI Python library is released under the Apache 2.0 license. Developer Note: early development of LeptonAI was in a separate mono-repo, which is why you may see commits from the `leptonai/lepton` repo. We intend to use this open source repo as the source of truth going forward. ", Assign "at most 3 tags" to the expected json: {"id":"3102","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"