Deploy multiple branches on your staging environment

In software development, the testing phase can be a significant bottleneck in the process. Long validation times and waiting for customer's validation can cause delays in delivery and impact the overall project's timeline. However, the multi-build staging environment has emerged as a valuable solution to these problems.

By providing the ability to deploy multiple isolated builds at the same time we are able to offer several key benefits:

• give the customer all the time he needs to review and validate a feature
• be able to repeat multiple time the review process introducing all the changes needed by the customer before merging the feature branch to the main branch
• avoids to lock an entire environment just to let the customer validate a feature
• fasten the identification of bugs as the scope of the review is limited to smaller changes

All those combined gives developers a better developer experience and fasten the customer's validation process as there might be multiple features being validated at the same time by multiple teams.

How we did it

In order to implement that we needed to find a solution for the following points:

• route requests to a specific build stage
• isolated build stages
• have a single line command to deal with the whole deployment (experience tells us that when processes are complex users tends to use them less)

Route requests to a specific build stage

To be able to route requests to a specific build stage we used Traefik.

Traefik is a reverse proxy, which means it is the door to your platform.
It intercepts and routes every incoming request: it knows all the logic and every rule that determines which services handle which requests
(based on the path, the host, headers, and so on ...).  

The cool thing with Traefik is that you don't have much configuration to write.
With only few docker-compose labels on services, you have a powerful reverse-proxy that handles the traffic of your stack.

services:
  traefik:
    image: traefik
    networks:
      proxy: {}
    command:
      - "--logLevel=ERROR"
      - "--traefiklog.format=json"
      - "--defaultEntryPoints=http,https"
      - "--entryPoints=Name:http Address::80 Redirect.Regex:^http://((.+\\.)?staging\\.contoso\\.com/?(.*)) Redirect.Replacement:https://$$1"
      - "--entryPoints=Name:https Address::443 TLS"
      - "--docker"
      - "--docker.watch=true"
      - "--docker.swarmmode=true"
      - "--docker.exposedByDefault=false"
      - "--acme=true"
      - "--acme.entrypoint=https"
      - "--acme.httpchallenge.entrypoint=http"
      - "--acme.domains=*.staging.contoso.com"
      - "--acme.email=hello@knplabs.com"
      - "--acme.storage=/etc/traefik/acme/acme.json"
      - "--acme.acmelogging"
      - "--acme.onHostRule=false"
      - "--acme.dnsChallenge.provider=cloudflare"
      - "--acme.dnsChallenge.delayBeforeCheck=0"
      - ...
    ...

networks:
  proxy:
    external: true

Isolated build stages

In order to be able to deploy isolated build stages we defined a tagging system where, at deploy time, we assign a tag that is then used to create isolated networks and stacks.

services:
  frontend:
    image: nginx
    networks:
      default: {}
      proxy: {}
    deploy:
      labels:
        - "traefik.frontend.rule=Host:${STAGE_TAG}.staging.contoso.com"
        - "traefik.enable=true"
        - "traefik.frontend.entryPoints=http,https"
        - "traefik.backend=${STAGE_TAG}-frontend"
        - "traefik.port=80"
        - "traefik.docker.network=proxy"
	  ...
    ...
  ...

networks:
  default:
    attachable: true
  proxy:
    external: true

Single line deployment command

In order to improve the developer experience at the maximum we created a single line command that takes care of configuring and deploying a new isolated build stage.

/bin/deploy-staging-build feature-1

This script is taking care of doing the following things:

• build Docker images
• push Docker images on the repository
• initialize all the necessary folders on the remote server and copy all necessary files (docker compose, scripts, etc...)
• configure the default environment variables (you can add a step to prompt for them in case you need it)
• create the isolated networks
• deploy the services
• execute post deployment procedures (database initialization, etc...)

After the script completed the new build stage will be accessible via "https://feature-1.staging.contoso.com".

Conclusions

The multi-build staging environment has become an essential tool in our software development workflow, providing a way to ease the reviewing process and ensure that developed products are of the highest quality. By allowing customers to review and validate work at their own pace, eliminating the need to lock environments, and improving issue identification and collaboration, multi-build staging environments offer significant benefits that can help to improve the overall development process.

🧐 Did you like this article ? This one might be interesting https://knplabs.com/fr/blog/comment-dockeriser-un-projet-symfony/

Furthermore, we propose plenty of trainings in order to share our experience and help other developers to create great web or mobile applications! Have a look and contact us here https://knplabs.com/en/services/trainings