base on A MySQL-compatible relational database with a storage agnostic query engine. Implemented in pure Go. <img height="240" src="./mascot.png"/> # A MySQL compatible database engine written in pure Go **go-mysql-server** is a data-source agnostic SQL engine and server which runs queries on data sources you provide, using the MySQL dialect and wire protocol. A simple in-memory database implementation is included, and you can query any data source you want by implementing your own backend. [Dolt](https://www.doltdb.com), a SQL database with Git-style versioning, is the main production database implementation of this package. [Check out](https://docs.dolthub.com/introduction/what-is-dolt) that project for a reference implementation. Or, hop into the Dolt Discord server [here](https://discord.com/invite/RFwfYpu) if you want to talk to the [core developers](https://www.dolthub.com/team) behind **go-mysql-server** and Dolt. ## Compatibility With the exception of specific limitations (see below), **go-mysql-server** is a drop-in replacement for MySQL. Any client library, tool, query, SQL syntax, SQL function, etc. that works with MySQL should also work with **go-mysql-server**. If you find a gap in functionality, please file an issue. For full MySQL compatibility documentation, see the [Dolt docs](https://docs.dolthub.com/sql-reference/sql-support) on this topic. ## Scope of this project - SQL server and engine to query your data sources. - In-memory database backend implementation suitable for use in tests. - Interfaces you can use to implement new backends to query your own data sources. - With a few caveats and using a full database implementation, a drop-in MySQL database replacement. **go-mysql-server** has two primary uses case: 1. Stand-in for MySQL in a golang test environment, using the built-in `memory` database implementation. 2. Providing access to arbitrary data sources with SQL queries by implementing a handful of interfaces. The most complete real-world implementation is [Dolt](https://github.com/dolthub/dolt). ## Installation Add **go-mysql-server** as a dependency to your project. In the directory with the `go.mod` file, run: ``` go get github.com/dolthub/go-mysql-server@latest ``` ## Using the in-memory test server The in-memory test server can replace a real MySQL server in tests. Start the server using the code in the [_example directory](_example/main.go), also reproduced below. ```go package main import ( "context" "fmt" "time" "github.com/dolthub/vitess/go/vt/proto/query" sqle "github.com/dolthub/go-mysql-server" "github.com/dolthub/go-mysql-server/memory" "github.com/dolthub/go-mysql-server/server" "github.com/dolthub/go-mysql-server/sql" "github.com/dolthub/go-mysql-server/sql/types" ) // This is an example of how to implement a MySQL server. // After running the example, you may connect to it using the following: // // > mysql --host=localhost --port=3306 --user=root mydb --execute="SELECT * FROM mytable;" // +----------+-------------------+-------------------------------+----------------------------+ // | name | email | phone_numbers | created_at | // +----------+-------------------+-------------------------------+----------------------------+ // | Jane Deo | janedeo@gmail.com | ["556-565-566","777-777-777"] | 2022-11-01 12:00:00.000001 | // | Jane Doe | jane@doe.com | [] | 2022-11-01 12:00:00.000001 | // | John Doe | john@doe.com | ["555-555-555"] | 2022-11-01 12:00:00.000001 | // | John Doe | johnalt@doe.com | [] | 2022-11-01 12:00:00.000001 | // +----------+-------------------+-------------------------------+----------------------------+ // // The included MySQL client is used in this example, however any MySQL-compatible client will work. var ( dbName = "mydb" tableName = "mytable" address = "localhost" port = 3306 ) func main() { pro := createTestDatabase() engine := sqle.NewDefault(pro) session := memory.NewSession(sql.NewBaseSession(), pro) ctx := sql.NewContext(context.Background(), sql.WithSession(session)) ctx.SetCurrentDatabase(dbName) // This variable may be found in the "users_example.go" file. Please refer to that file for a walkthrough on how to // set up the "mysql" database to allow user creation and user checking when establishing connections. This is set // to false for this example, but feel free to play around with it and see how it works. if enableUsers { if err := enableUserAccounts(ctx, engine); err != nil { panic(err) } } config := server.Config{ Protocol: "tcp", Address: fmt.Sprintf("%s:%d", address, port), } s, err := server.NewServer(config, engine, sql.NewContext, memory.NewSessionBuilder(pro), nil) if err != nil { panic(err) } if err = s.Start(); err != nil { panic(err) } } func createTestDatabase() *memory.DbProvider { db := memory.NewDatabase(dbName) db.BaseDatabase.EnablePrimaryKeyIndexes() pro := memory.NewDBProvider(db) session := memory.NewSession(sql.NewBaseSession(), pro) ctx := sql.NewContext(context.Background(), sql.WithSession(session)) table := memory.NewTable(db, tableName, sql.NewPrimaryKeySchema(sql.Schema{ {Name: "name", Type: types.Text, Nullable: false, Source: tableName, PrimaryKey: true}, {Name: "email", Type: types.Text, Nullable: false, Source: tableName, PrimaryKey: true}, {Name: "phone_numbers", Type: types.JSON, Nullable: false, Source: tableName}, {Name: "created_at", Type: types.MustCreateDatetimeType(query.Type_DATETIME, 6), Nullable: false, Source: tableName}, }), db.GetForeignKeyCollection()) db.AddTable(tableName, table) creationTime := time.Unix(0, 1667304000000001000).UTC() _ = table.Insert(ctx, sql.NewRow("Jane Deo", "janedeo@gmail.com", types.MustJSON(`["556-565-566", "777-777-777"]`), creationTime)) _ = table.Insert(ctx, sql.NewRow("Jane Doe", "jane@doe.com", types.MustJSON(`[]`), creationTime)) _ = table.Insert(ctx, sql.NewRow("John Doe", "john@doe.com", types.MustJSON(`["555-555-555"]`), creationTime)) _ = table.Insert(ctx, sql.NewRow("John Doe", "johnalt@doe.com", types.MustJSON(`[]`), creationTime)) return pro } ``` This example populates the database by creating `memory.Database` and `memory.Table` objects via golang code, but you can also populate it by issuing `CREATE DATABASE`, `CREATE TABLE`, etc. statements to the server once it's running. Once the server is running, connect with any MySQL client, including the golang MySQL connector and the `mysql` shell. ```bash > mysql --host=localhost --port=3306 --user=root mydb --execute="SELECT * FROM mytable;" +----------+-------------------+-------------------------------+----------------------------+ | name | email | phone_numbers | created_at | +----------+-------------------+-------------------------------+----------------------------+ | Jane Deo | janedeo@gmail.com | ["556-565-566","777-777-777"] | 2022-11-01 12:00:00.000001 | | Jane Doe | jane@doe.com | [] | 2022-11-01 12:00:00.000001 | | John Doe | john@doe.com | ["555-555-555"] | 2022-11-01 12:00:00.000001 | | John Doe | johnalt@doe.com | [] | 2022-11-01 12:00:00.000001 | +----------+-------------------+-------------------------------+----------------------------+ ``` ## Limitations of the in-memory database implementation The in-memory database implementation included with this package is intended for use in tests. It has specific limitations that we know of: - [Not threadsafe](https://github.com/dolthub/go-mysql-server/issues/1306). To avoid concurrency issues, limit DDL and DML statements (`CREATE TABLE`, `INSERT`, etc.) to a single goroutine. - [No transaction support](https://github.com/dolthub/go-mysql-server/issues/1506). Statements like `START TRANSACTION`, `ROLLBACK`, and `COMMIT` are no-ops. - [Non-performant index implementation](https://github.com/dolthub/go-mysql-server/issues/1347). Indexed lookups and joins perform full table scans on the underlying tables. ## Custom backend implementations You can create your own backend to query your own data sources by implementing some interfaces. For detailed instructions, see the [backend guide](./BACKEND.md). ## Technical documentation for contributors and backend developers - [Architecture](./ARCHITECTURE.md) is an overview of the various packages of the project and how they fit together. - [Contribution guide](./CONTRIBUTING.md) for new contributors, including instructions for how to get your PR merged. ## Powered by go-mysql-server * [dolt](https://github.com/dolthub/dolt) * [gitbase](https://github.com/src-d/gitbase) (defunct) Are you building a database backend using **go-mysql-server**? We would like to hear from you and include you in this list. ## Security Policy [go-mysql-server's security policy](https://github.com/dolthub/go-mysql-server/blob/main/SECURITY.md) is maintained in this repository. Please follow the disclosure instructions there. Please do not initially report security issues in this repository's public GitHub issues. ## Acknowledgements **go-mysql-server** was originally developed by the `{source-d}` organzation, and this repository was originally forked from [src-d](https://github.com/src-d/go-mysql-server). We want to thank the entire `{source-d}` development team for their work on this project, especially Miguel Molina (@erizocosmico) and Juanjo Álvarez Martinez (@juanjux). ## License Apache License 2.0, see [LICENSE](/LICENSE) ", Assign "at most 3 tags" to the expected json: {"id":"9240","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"