base on BOSH release for a Cloud Foundry Redis service broker that supports shared-vm plans # Cloud Foundry Shared Redis Release This repository contains a BOSH release for a Cloud Foundry Shared Redis service broker. It supports a shared-vm plan. **Note: Breaking change > version 434.3.6 - the `redis.limits.maxclients` property is now `redis.maxclients`** ```shell git clone https://github.com/pivotal-cf/shared-redis-release ~/workspace/shared-redis-release cd ~/workspace/shared-redis-release git submodule update --init --recursive ``` ## Deployment dependencies 1. [BOSH CLI v2+](https://github.com/cloudfoundry/bosh-cli) (you may use the old [BOSH CLI](https://github.com/cloudfoundry/bosh) but instructions will use the new one) 2. [direnv](https://github.com/direnv/direnv) (or set environment variables yourself) 3. a bosh director 4. a cloud foundry deployment ## Deployment Run the following steps: 1. fill out the following environment variables of the `.envrc.template` file and save as .envrc or export them. All or almost all the variables are required for tests but these are the minimum required for deploy: - BOSH_ENVIRONMENT - BOSH_CA_CERT - BOSH_CLIENT - BOSH_CLIENT_SECRET - BOSH_DEPLOYMENT 1. if you're using the `.envrc` file ```shell direnv allow ``` 1. upload dependent releases ```shell bosh upload-release http://bosh.io/d/github.com/cloudfoundry-incubator/cf-routing-release bosh upload-release http://bosh.io/d/github.com/cloudfoundry/syslog-release bosh upload-release http://bosh.io/d/github.com/cloudfoundry-incubator/bpm-release # required for routing 180+ ``` Populate a vars file (using `manifest/vars-lite.yml` as a template), save it to `secrets/vars.yml`. You will need values from both your cloud-config and secrets from your cf-deployment. To deploy: ```shell bosh upload-stemcell https://s3.amazonaws.com/bosh-core-stemcells/warden/bosh-stemcell-97.3-warden-boshlite-ubuntu-xenial-go_agent.tgz bosh create-release bosh upload-release bosh deploy --vars-file secrets/vars.yml manifest/deployment.yml ``` ## Network Configuration The following ports and ranges are used in this service: - broker vm, port 12350: access to the broker from the cloud controllers - broker vm, ports 32768-61000: on the service broker from the Diego Cell and Diego Brain network(s). This is only required for the shared service plan - dedicated node, port 6379: access to all dedicated nodes from the Diego Cell and Diego Brain network(s) ## Testing 1. install gem requirements ```shell bundle install ``` 2. run the tests ``` bundle exec rspec spec ``` ## Related Documentation * [BOSH](https://bosh.io/docs) * [Service Broker API](http://docs.cloudfoundry.org/services/api.html) * [Redis](http://redis.io/documentation) ## Known Issues ### Possible Data Leak when disabling Static IPs In the past cf-redis-release expected to be deployed [with static IPs](https://github.com/pivotal-cf/cf-redis-release/blob/23a218a06188ba9dd5694698bfa9fd4b5131b707/manifest/deployment.yml#L54) for dedicated nodes specified in the manifest. It is often more desired to deploy without static IPs leaving BOSH to manage IP allocation. There is a risk of data leak when transitioning from static to dynamic IPs. Consider the following scenario: The operator: 1. has deployed cf-redis-release with static IPs 1. now decides to use dynamic IP allocation, and removes the static IPs from the manifest 1. then DOES NOT remove the static IPs from [cloud config](https://bosh.io/docs/networks/) 1. re-deploys cf-redis-release with the new manifest In the previous scenario BOSH will dynamically allocate IPs to the dedicated instances. BOSH will not try to re-use the previous IPs since those are still restricted in the cloud config. Since the IPs have changed, application bindings will break and the state in the broker will be out of sync with the new deployment. It is possible that previously allocated instances containing application data are erroneously re-allocated to another unrelated application, causing data to be leaked. In order to avoid this scenario, the operator must remove the reserved static IPs in the cloud config at the same time as they are remove from the manifest. This will allow BOSH to keep the same IP addresses for the existing nodes. To safely transition from static to dynamic IPs: 1. look up the static IPs that were specified in the manifest when deploying your dedicated nodes 1. ensure these IPs are no longer included in the static range of the network in your cloud config 1. remove the static IPs from the manifest 1. deploy using the manifest without static IPs ## Errands The Cf-Redis Service Broker offers the following [BOSH errands](https://bosh.io/docs/errands/): ### broker-registrar Communicates to the CloudFoundry [Cloud Controller](https://docs.cloudfoundry.org/concepts/architecture/cloud-controller.html) which maintains the database for the [CF Marketplace](https://cli.cloudfoundry.org/en-US/cf/marketplace.html). This enables the service plans on the CF Marketplace so that App Developers can create and delete service instances of those plans. ### broker-deregistrar Communicates with the CF Cloud Controller to remove the cf-redis broker and service plans from the CF Marketplace as well as [delete the broker deployment](https://cli.cloudfoundry.org/en-US/cf/delete-service-broker.html) from BOSH. **Note:** If any service instances exist, the errand will **fail**. This is to prevent orphan deployments which the CF Cloud Controller will lose record of but will continue to live as a BOSH deployment which would continue to incur costs. If you wish to remove the service broker, perform a `cf delete-service SERVICE_INSTANCE` and remove all service instances associated with the broker and run the errand after all service instances have been removed. ### smoke-tests Runs smoke tests which tests the lifecycle and functionality of the both the dedicated-vm and shared-vm services. See the [Redis documentation](https://docs.pivotal.io/redis/smoke-tests.html) for more information. The CI pipeline for this Repo lives here: https://runway-ci.eng.vmware.com/teams/tanzu-redis/pipelines/shared-redis", Assign "at most 3 tags" to the expected json: {"id":"10409","tags":[]} "only from the tags list I provide: [{"id":77,"name":"3d"},{"id":89,"name":"agent"},{"id":17,"name":"ai"},{"id":54,"name":"algorithm"},{"id":24,"name":"api"},{"id":44,"name":"authentication"},{"id":3,"name":"aws"},{"id":27,"name":"backend"},{"id":60,"name":"benchmark"},{"id":72,"name":"best-practices"},{"id":39,"name":"bitcoin"},{"id":37,"name":"blockchain"},{"id":1,"name":"blog"},{"id":45,"name":"bundler"},{"id":58,"name":"cache"},{"id":21,"name":"chat"},{"id":49,"name":"cicd"},{"id":4,"name":"cli"},{"id":64,"name":"cloud-native"},{"id":48,"name":"cms"},{"id":61,"name":"compiler"},{"id":68,"name":"containerization"},{"id":92,"name":"crm"},{"id":34,"name":"data"},{"id":47,"name":"database"},{"id":8,"name":"declarative-gui "},{"id":9,"name":"deploy-tool"},{"id":53,"name":"desktop-app"},{"id":6,"name":"dev-exp-lib"},{"id":59,"name":"dev-tool"},{"id":13,"name":"ecommerce"},{"id":26,"name":"editor"},{"id":66,"name":"emulator"},{"id":62,"name":"filesystem"},{"id":80,"name":"finance"},{"id":15,"name":"firmware"},{"id":73,"name":"for-fun"},{"id":2,"name":"framework"},{"id":11,"name":"frontend"},{"id":22,"name":"game"},{"id":81,"name":"game-engine "},{"id":23,"name":"graphql"},{"id":84,"name":"gui"},{"id":91,"name":"http"},{"id":5,"name":"http-client"},{"id":51,"name":"iac"},{"id":30,"name":"ide"},{"id":78,"name":"iot"},{"id":40,"name":"json"},{"id":83,"name":"julian"},{"id":38,"name":"k8s"},{"id":31,"name":"language"},{"id":10,"name":"learning-resource"},{"id":33,"name":"lib"},{"id":41,"name":"linter"},{"id":28,"name":"lms"},{"id":16,"name":"logging"},{"id":76,"name":"low-code"},{"id":90,"name":"message-queue"},{"id":42,"name":"mobile-app"},{"id":18,"name":"monitoring"},{"id":36,"name":"networking"},{"id":7,"name":"node-version"},{"id":55,"name":"nosql"},{"id":57,"name":"observability"},{"id":46,"name":"orm"},{"id":52,"name":"os"},{"id":14,"name":"parser"},{"id":74,"name":"react"},{"id":82,"name":"real-time"},{"id":56,"name":"robot"},{"id":65,"name":"runtime"},{"id":32,"name":"sdk"},{"id":71,"name":"search"},{"id":63,"name":"secrets"},{"id":25,"name":"security"},{"id":85,"name":"server"},{"id":86,"name":"serverless"},{"id":70,"name":"storage"},{"id":75,"name":"system-design"},{"id":79,"name":"terminal"},{"id":29,"name":"testing"},{"id":12,"name":"ui"},{"id":50,"name":"ux"},{"id":88,"name":"video"},{"id":20,"name":"web-app"},{"id":35,"name":"web-server"},{"id":43,"name":"webassembly"},{"id":69,"name":"workflow"},{"id":87,"name":"yaml"}]" returns me the "expected json"