base on Database migrations. CLI and Golang library. [](https://github.com/golang-migrate/migrate/actions/workflows/ci.yaml?query=branch%3Amaster) [](https://pkg.go.dev/github.com/golang-migrate/migrate/v4) [](https://coveralls.io/github/golang-migrate/migrate?branch=master) [](https://packagecloud.io/golang-migrate/migrate?filter=debs) [](https://hub.docker.com/r/migrate/migrate/)  [](https://github.com/golang-migrate/migrate/releases) [](https://goreportcard.com/report/github.com/golang-migrate/migrate/v4) # migrate __Database migrations written in Go. Use as [CLI](#cli-usage) or import as [library](#use-in-your-go-project).__ * Migrate reads migrations from [sources](#migration-sources) and applies them in correct order to a [database](#databases). * Drivers are "dumb", migrate glues everything together and makes sure the logic is bulletproof. (Keeps the drivers lightweight, too.) * Database drivers don't assume things or try to correct user input. When in doubt, fail. Forked from [mattes/migrate](https://github.com/mattes/migrate) ## Databases Database drivers run migrations. [Add a new database?](database/driver.go) * [PostgreSQL](database/postgres) * [PGX v4](database/pgx) * [PGX v5](database/pgx/v5) * [Redshift](database/redshift) * [Ql](database/ql) * [Cassandra / ScyllaDB](database/cassandra) * [SQLite](database/sqlite) * [SQLite3](database/sqlite3) ([todo #165](https://github.com/mattes/migrate/issues/165)) * [SQLCipher](database/sqlcipher) * [MySQL / MariaDB](database/mysql) * [Neo4j](database/neo4j) * [MongoDB](database/mongodb) * [CrateDB](database/crate) ([todo #170](https://github.com/mattes/migrate/issues/170)) * [Shell](database/shell) ([todo #171](https://github.com/mattes/migrate/issues/171)) * [Google Cloud Spanner](database/spanner) * [CockroachDB](database/cockroachdb) * [YugabyteDB](database/yugabytedb) * [ClickHouse](database/clickhouse) * [Firebird](database/firebird) * [MS SQL Server](database/sqlserver) * [rqlite](database/rqlite) ### Database URLs Database connection strings are specified via URLs. The URL format is driver dependent but generally has the form: `dbdriver://username:password@host:port/dbname?param1=true&param2=false` Any [reserved URL characters](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_reserved_characters) need to be escaped. Note, the `%` character also [needs to be escaped](https://en.wikipedia.org/wiki/Percent-encoding#Percent-encoding_the_percent_character) Explicitly, the following characters need to be escaped: `!`, `#`, `$`, `%`, `&`, `'`, `(`, `)`, `*`, `+`, `,`, `/`, `:`, `;`, `=`, `?`, `@`, `[`, `]` It's easiest to always run the URL parts of your DB connection URL (e.g. username, password, etc) through an URL encoder. See the example Python snippets below: ```bash $ python3 -c 'import urllib.parse; print(urllib.parse.quote(input("String to encode: "), ""))' String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D $ python2 -c 'import urllib; print urllib.quote(raw_input("String to encode: "), "")' String to encode: FAKEpassword!#$%&'()*+,/:;=?@[] FAKEpassword%21%23%24%25%26%27%28%29%2A%2B%2C%2F%3A%3B%3D%3F%40%5B%5D $ ``` ## Migration Sources Source drivers read migrations from local or remote sources. [Add a new source?](source/driver.go) * [Filesystem](source/file) - read from filesystem * [io/fs](source/iofs) - read from a Go [io/fs](https://pkg.go.dev/io/fs#FS) * [Go-Bindata](source/go_bindata) - read from embedded binary data ([jteeuwen/go-bindata](https://github.com/jteeuwen/go-bindata)) * [pkger](source/pkger) - read from embedded binary data ([markbates/pkger](https://github.com/markbates/pkger)) * [GitHub](source/github) - read from remote GitHub repositories * [GitHub Enterprise](source/github_ee) - read from remote GitHub Enterprise repositories * [Bitbucket](source/bitbucket) - read from remote Bitbucket repositories * [Gitlab](source/gitlab) - read from remote Gitlab repositories * [AWS S3](source/aws_s3) - read from Amazon Web Services S3 * [Google Cloud Storage](source/google_cloud_storage) - read from Google Cloud Platform Storage ## CLI usage * Simple wrapper around this library. * Handles ctrl+c (SIGINT) gracefully. * No config search paths, no config files, no magic ENV var injections. [CLI Documentation](cmd/migrate) (includes CLI install instructions) ### Basic usage ```bash $ migrate -source file://path/to/migrations -database postgres://localhost:5432/database up 2 ``` ### Docker usage ```bash $ docker run -v {{ migration dir }}:/migrations --network host migrate/migrate -path=/migrations/ -database postgres://localhost:5432/database up 2 ``` ## Use in your Go project * API is stable and frozen for this release (v3 & v4). * Uses [Go modules](https://golang.org/cmd/go/#hdr-Modules__module_versions__and_more) to manage dependencies. * To help prevent database corruptions, it supports graceful stops via `GracefulStop chan bool`. * Bring your own logger. * Uses `io.Reader` streams internally for low memory overhead. * Thread-safe and no goroutine leaks. __[Go Documentation](https://pkg.go.dev/github.com/golang-migrate/migrate/v4)__ ```go import ( "github.com/golang-migrate/migrate/v4" _ "github.com/golang-migrate/migrate/v4/database/postgres" _ "github.com/golang-migrate/migrate/v4/source/github" ) func main() { m, err := migrate.New( "github://mattes:personal-access-token@mattes/migrate_test", "postgres://localhost:5432/database?sslmode=enable") m.Steps(2) } ``` Want to use an existing database client? ```go import ( "database/sql" _ "github.com/lib/pq" "github.com/golang-migrate/migrate/v4" "github.com/golang-migrate/migrate/v4/database/postgres" _ "github.com/golang-migrate/migrate/v4/source/file" ) func main() { db, err := sql.Open("postgres", "postgres://localhost:5432/database?sslmode=enable") driver, err := postgres.WithInstance(db, &postgres.Config{}) m, err := migrate.NewWithDatabaseInstance( "file:///migrations", "postgres", driver) m.Up() // or m.Steps(2) if you want to explicitly set the number of migrations to run } ``` ## Getting started Go to [getting started](GETTING_STARTED.md) ## Tutorials * [CockroachDB](database/cockroachdb/TUTORIAL.md) * [PostgreSQL](database/postgres/TUTORIAL.md) (more tutorials to come) ## Migration files Each migration has an up and down migration. [Why?](FAQ.md#why-two-separate-files-up-and-down-for-a-migration) ```bash 1481574547_create_users_table.up.sql 1481574547_create_users_table.down.sql ``` [Best practices: How to write migrations.](MIGRATIONS.md) ## Coming from another db migration tool? Check out [migradaptor](https://github.com/musinit/migradaptor/). *Note: migradaptor is not affiliated or supported by this project* ## Versions Version | Supported? | Import | Notes --------|------------|--------|------ **master** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | New features and bug fixes arrive here first | **v4** | :white_check_mark: | `import "github.com/golang-migrate/migrate/v4"` | Used for stable releases | **v3** | :x: | `import "github.com/golang-migrate/migrate"` (with package manager) or `import "gopkg.in/golang-migrate/migrate.v3"` (not recommended) | **DO NOT USE** - No longer supported | ## Development and Contributing Yes, please! [`Makefile`](Makefile) is your friend, read the [development guide](CONTRIBUTING.md). Also have a look at the [FAQ](FAQ.md). --- Looking for alternatives? [https://awesome-go.com/#database](https://awesome-go.com/#database). ", Assign "at most 3 tags" to the expected json: {"id":"2450","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"