base on A rewritten YouTube source manager for Lavaplayer. # youtube-source A rewritten YouTube source manager for Lavaplayer. This source aims to provide robustness by leveraging multiple InnerTube clients for requests. Where one client fails, another will try to load the request. Which clients are used is entirely configurable. ## Table of Contents - [Common](#common) - Information about the `common` module and usage of. - [V2](#v2) - Information about the `v2` module and usage of. - [Plugin](#plugin) - Information about the `plugin` module and usage of. - [Available Clients](#available-clients) - Information about the clients provided by `youtube-source`, as well as their advantages/disadvantages. - [Using OAuth tokens](#using-oauth-tokens) - Information on using OAuth tokens with `youtube-source`. - [Using a poToken](#using-a-potoken) - Information on using a `poToken` with `youtube-source`. - [REST Routes (`plugin` only)](#rest-routes-plugin-only) - Information on the REST routes provided by the `youtube-source` plugin module. - [Migration Information](#migration-from-lavaplayers-built-in-youtube-source) - Information on migrating from Lavaplayer's built-in Youtube source manager. - [Additional Support](#additional-support) - For everything else. ## common This module provides the base source manager, which can be used with any `com.sedmelluq.discord.lavaplayer` packages still on major version `1`. <details> <summary>Using in Gradle:</summary> ```kotlin repositories { // replace with https://maven.lavalink.dev/snapshots if you want to use a snapshot version. maven(url = "https://maven.lavalink.dev/releases") } dependencies { // Replace VERSION with the current version as shown by the Releases tab or a long commit hash `-SNAPSHOT` for snapshots. implementation("dev.lavalink.youtube:common:VERSION") } ``` </details> Example usage: ```java YoutubeAudioSourceManager youtube = new YoutubeAudioSourceManager(); // Optionally, you may instantiate the source with a custom options, such as toggling use of searching, and clients. YoutubeAudioSourceManager youtube = new YoutubeAudioSourceManager(/*allowSearch:*/ true, new Client[] { new Music(), new Web(), new AndroidTestsuite() }); ``` You may also extend the `Client` interface to support additional InnerTube clients. There are a few abstract classes to make this easier, notably, `MusicClient` (for `music.youtube.com` InnerTube clients), `NonMusicClient` (for youtube.com innertube clients) and `StreamingNonMusicClient` (for clients that can be used to stream videos). Support for IP rotation has been included, and can be achieved using the following: ```java AbstractRoutePlanner routePlanner = new ... YoutubeIpRotatorSetup rotator = new YoutubeIpRotatorSetup(routePlanner); // 'youtube' is the variable holding your YoutubeAudioSourceManager instance. rotator.forConfiguration(youtube.getHttpInterfaceManager(), false) .withMainDelegateFilter(youtube.getContextFilter()) // IMPORTANT .setup(); ``` ## v2 This modules expands on `common` by providing additional support for Lavaplayer `2.x` clients, such as [Lavalink-Devs/Lavaplayer](https://github.com/lavalink-devs/lavaplayer). Such features currently include thumbnail support within `AudioTrackInfo`. Additional clients are included that provide access to this additional information. These clients are suffixed with `Thumbnail`, such as `WebWithThumbnail`, `AndroidWithThumbnail` etc. <details> <summary>Using in Gradle:</summary> ```kotlin repositories { // replace with https://maven.lavalink.dev/snapshots if you want to use a snapshot version. maven(url = "https://maven.lavalink.dev/releases") } dependencies { // Replace VERSION with the current version as shown by the Releases tab or a long commit hash `-SNAPSHOT` for snapshots. implementation("dev.lavalink.youtube:v2:VERSION") } ``` </details> Example usage: ```java // same as the 'common' module but there are additional clients that provide video thumbnails in the returned metadata. YoutubeAudioSourceManager youtube = new YoutubeAudioSourceManager(/*allowSearch:*/ true, new Client[] { new MusicWithThumbnail(), new WebWithThumbnail(), new AndroidTestsuiteWithThumbnail() }); ``` ## plugin This module serves as the plugin for use with [Lavalink](https://github.com/lavalink-devs/Lavalink). To use this plugin with Lavalink, you must declare the dependency. <details> <summary>Using with Lavalink v3:</summary> ```yaml lavalink: plugins: # Replace VERSION with the current version as shown by the Releases tab or a long commit hash for snapshots. - dependency: "dev.lavalink.youtube:youtube-plugin:VERSION" repository: "https://maven.lavalink.dev/releases" # use https://maven.lavalink.dev/snapshots if you want to use a snapshot version. ``` </details> <details> <summary>Using with Lavalink v4:</summary> ```yaml lavalink: plugins: # Replace VERSION with the current version as shown by the Releases tab or a long commit hash for snapshots. - dependency: "dev.lavalink.youtube:youtube-plugin:VERSION" snapshot: false # Set to true if you want to use a snapshot version. ``` </details> Configuring the plugin: > [!IMPORTANT] > You must make sure to disable the built-in YouTube source like so: ```yaml lavalink: server: sources: youtube: false ``` > [!NOTE] > Existing options, such as `ratelimit` and `youtubePlaylistLoadLimit` will be picked up automatically by the plugin, > so these don't need changing. > ```yaml plugins: youtube: enabled: true # Whether this source can be used. allowSearch: true # Whether "ytsearch:" and "ytmsearch:" can be used. allowDirectVideoIds: true # Whether just video IDs can match. If false, only complete URLs will be loaded. allowDirectPlaylistIds: true # Whether just playlist IDs can match. If false, only complete URLs will be loaded. # The clients to use for track loading. See below for a list of valid clients. # Clients are queried in the order they are given (so the first client is queried first and so on...) clients: - MUSIC - ANDROID_VR - WEB - WEBEMBEDDED ``` ### Advanced Options ```yaml # The below section of the config allows setting specific options for each client, such as the requests they will handle. # If an option, or client, is unspecified, then the default option value/client values will be used instead. # If a client is configured, but is not registered above, the options for that client will be ignored. # WARNING!: THE BELOW CONFIG IS FOR ILLUSTRATION PURPOSES. DO NOT COPY OR USE THIS WITHOUT # WARNING!: UNDERSTANDING WHAT IT DOES. MISCONFIGURATION WILL HINDER YOUTUBE-SOURCE'S ABILITY TO WORK PROPERLY. # Write the names of clients as they are specified under the heading "Available Clients". clientOptions: WEB: # Example: Disabling a client's playback capabilities. playback: false videoLoading: false # Disables loading of videos for this client. A client may still be used for playback even if this is set to 'false'. WEBEMBEDDED: # Example: Configuring a client to exclusively be used for video loading and playback. playlistLoading: false # Disables loading of playlists and mixes. searching: false # Disables the ability to search for videos. ``` ## Available Clients Currently, the following clients are available for use: | Identifier | Opus Formats | OAuth | Age-restriction Support | Playback Support | Metadata Support | Additional Notes | |-------------------|--------------|-------|-------------------------|------------------|------------------------------|------------------------------------------------------| | `MUSIC` | No | No | No | No | Search | YouTube music search support via `ytmsearch:` prefix | | `WEB` | Yes | No | No | Yes + Livestream | Video, Search, Playlist, Mix | | | `MWEB` | Yes | No | No | Yes + Livestream | Video, Search, Playlist, Mix | | | `WEBEMBEDDED` | Yes | No | Limited | Yes + Livestream | Video | | | `ANDROID` | Yes | No | No | Yes + Livestream | Video, Search, Playlist, Mix | Heavily restricted, frequently dysfunctional | | `ANDROID_MUSIC` | Yes | No | No | Yes | Video, Search, Mix | | | `ANDROID_VR` | Yes | No | No | Yes + Livestream | Video, Search, Playlist, Mix | | | `IOS` | No | No | No | Yes + Livestream | Video, Search, Playlist, Mix | | | `TV` | Yes | Yes | With OAuth | Yes + Livestream | None | Playback requires sign-in | | `TVHTML5EMBEDDED` | Yes | Yes | With OAuth | Yes + Livestream | Video, Search, Mix | Playback requires sign-in | > [!NOTE] > Clients that do not return Opus formats will require transcoding. > Livestreams do not yield Opus formats so will always require transcoding. ## Using OAuth Tokens You may notice that some requests are flagged by YouTube, causing an error message asking you to sign in to confirm you're not a bot. With OAuth integration, you can request that `youtube-source` use your account credentials to appear as a normal user, with varying degrees of efficacy. **You do _not_ need to use `poToken` with OAuth.** > [!WARNING] > Similar to the `poToken` method, this is NOT a silver bullet solution, and worst case could get your account terminated! > For this reason, it is advised that **you use burner accounts and NOT your primary!**. > This method may also trigger ratelimit errors if used in a high traffic environment. > USE WITH CAUTION! > [!NOTE] > You may need to set your log level for `dev.lavalink.youtube.http.YoutubeOauth2Handler` to `INFO`, to see additional information > within your terminal regarding completing the OAuth flow. > [!NOTE] > If you do not have a refresh token, then do not supply one. The source will output your refresh token into your terminal upon > successfully completing the OAuth flow at least **once**. If you do not see your token, you may need to configure your > logging (see above note). You can instruct `youtube-source` to use OAuth with the following: ### Lavaplayer ```java YoutubeAudioSourceManager source = new YoutubeAudioSourceManager(); // This will trigger an OAuth flow, where you will be instructed to head to YouTube's OAuth page and input a code. // This is safe, as it only uses YouTube's official OAuth flow. No tokens are seen or stored by us. source.useOauth2(null, false); // If you already have a refresh token, you can instruct the source to use it, skipping the OAuth flow entirely. // You can also set the `skipInitialization` parameter, which skips the OAuth flow. This should only be used // if you intend to supply a refresh token later on. You **must** either complete the OAuth flow or supply // a refresh token for OAuth integration to work. source.useOauth2("your refresh token", true); ``` ### Lavalink ```yaml plugins: youtube: enabled: true oauth: # setting "enabled: true" is the bare minimum to get OAuth working. enabled: true # if you have a refresh token, you may set it below (make sure to uncomment the line to apply it). # setting a valid refresh token will skip the OAuth flow entirely. See above note on how to retrieve # your refreshToken. # refreshToken: "paste your refresh token here if applicable" # Set this if you don't want the OAuth flow to be triggered, if you intend to supply a refresh token later. # Initialization is skipped automatically if a valid refresh token is supplied. Leave this commented if you're # completing the OAuth flow for the first time/do not have a refresh token. # skipInitialization: true ``` ### Passing an oauth token from your client Another option to use oauth is by using oauth access tokens that are managed from your client. In this case your bot/client provides LavaLink with the token to use when playing a track. To do this simply add the oauth access token to a track's [userData](https://lavalink.dev/api/rest#track) field in a json format when updating the player to play a track like: ```json { "oauth-token": "access token to use" } ``` ## Using a `poToken` A `poToken`, also known as a "Proof of Origin Token" is a way to identify what requests originate from. In YouTube's case, this is sent as a JavaScript challenge that browsers must evaluate, and send back the resolved string. Typically, this challenge would remain unsolved for bots as more often than not, they don't simulate an entire browser environment, instead only evaluating the minimum amount of JS required to do its job. Therefore, it's a reasonable assumption that if the challenge is not fulfilled, the request origin is a bot. To obtain a `poToken`, you can use https://github.com/iv-org/youtube-trusted-session-generator, by running the Python script or the docker image. Both methods will print a `poToken` after a successful run, which you can supply to `youtube-source` to try and work around having automated requests blocked. > [!NOTE] > A `poToken` is not a silver bullet, and currently it only applies to requests made via the `WEB` & `WEBEMBEDDED` client. > You do not need to specify a `poToken` if using OAuth, and vice versa. Specifying the token is as simple as doing: ### Lavaplayer ```java // Web is dev.lavalink.youtube.clients.Web Web.setPoTokenAndVisitorData("your po_token", "your visitor_data"); ``` ### Lavalink ```yaml plugins: youtube: pot: token: "paste your po_token here" visitorData: "paste your visitor_data here" ``` ## REST routes (`plugin` only) ### `POST` `/youtube` Body: > [!NOTE] > You do not need to provide everything as it is shown. > For example, you can specify just `refreshToken` and `skipInitialization`, or just `poToken` and `visitorData`. > You do **not** need to use `poToken` with OAuth and vice versa. ```json { "refreshToken": "your new refresh token", "skipInitialization": true, "poToken": "your po_token", "visitorData": "your visitor_data" } ``` Response: If the YouTube source is not enabled, or the `refreshToken` is invalid: `500 - Internal Server Error` Otherwise: `204 - No Content` ### `GET` `/youtube` Response: If the YouTube source is not enabled: `500 - Internal Server Error` Otherwise: ```json { "refreshToken": "your current refresh token, or null" } ``` ### `GET` `/youtube/stream/{videoId}` Query parameters: | Key | Value Type | Required | Notes | |--------------|------------|----------|-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------| | itag | integer | No | The [itag](https://gist.github.com/AgentOak/34d47c65b1d28829bb17c24c04a0096f) of the desired format. If unspecified, youtube-source's default format selector will be used. | | withClient | string | No | The identifier of the client to use for streaming. Uses all clients if unspecified. | Response: If `videoId` could not be found or loaded, or the `itag` does not exist, or if no client supports format loading: `400 - Bad Request` Otherwise: `200 - OK` accompanied by the selected format stream (audio or video). `Content-Type` header will be set appropriately. ### `GET` `/youtube/oauth/{refreshToken}` Response: If the `refreshToken` is invalid, expired, or cannot be processed: `500 - Internal Server Error` If the refresh process succeeds and a new access token is generated: `200 - OK` accompanied by the new access token in JSON format. Example response: ```json { "access_token": "AccessToken", "expires_in": 69420, "scope": "used scope", "token_type": "type" } ``` ## Migration from Lavaplayer's built-in YouTube source This client is intended as a direct replacement for Lavaplayer's built-in `YoutubeAudioSourceManager`, which has been deprecated in a recent release of [Lavalink-Devs/Lavaplayer](https://github.com/lavalink-devs/lavaplayer). When using `AudioSourceManagers.registerRemoteSources(AudioPlayerManager)`, Lavaplayer will register its own deprecated `YoutubeAudioSourceManager`, which must be disabled. Some versions of Lavaplayer may include an optional `excludeSources` parameter, allowing you to toggle the adding of the source. If the version you are using does not support this, you will need to manually register each `AudioSourceManager` yourself. First, create and register an instance of the supported `YoutubeAudioSourceManager` from the `youtube-source` package. ```java AudioPlayerManager playerManager = new DefaultAudioPlayerManager(); YoutubeAudioSourceManager ytSourceManager = new dev.lavalink.youtube.YoutubeAudioSourceManager(); playerManager.registerSourceManager(ytSourceManager); ``` If your version of Lavaplayer supports an `excludeSources` parameter or equivalent, you may exclude the built-in `YoutubeAudioSourceManager` using the following: ```java AudioSourceManagers.registerRemoteSources(playerManager, com.sedmelluq.discord.lavaplayer.source.youtube.YoutubeAudioSourceManager.class); ``` Otherwise, you will need to register each source manager individually. In addition, there are a few significant changes to note: - This source's class structure differs so if you had custom classes that you were initialising the source manager with (e.g. an overridden `YoutubeTrackDetailsLoader`), this **is not** compatible with this source manager. ## Versioning Policy This project follows [Semantic Versioning](https://semver.org/), except in the case of [client](#available-clients) removal. Typically, clients are not removed unless there is good reason, such as being deprecated, irreparably broken or removed from YouTube's client lifecycle. In such scenarios, we anticipate that you have ceased usage of such clients prior to their removal, so do not expect any code breakage, however we advise that you periodically check and keep your client list up to date due to this. ## Additional Support If you need additional help with using this source, that's not covered here or in any of the issues, [join our Discord server](https://discord.gg/ZW4s47Ppw4). ", Assign "at most 3 tags" to the expected json: {"id":"13614","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"