home » locations » poland » stabilizing environments for rapidly growing teams

Stabilizing environments for rapidly growing teams


This story describes how adding new people to a team need not destabilize workflows but, instead can enhance efficiencies

Due to the growing market opportunities our team increased expanded exponentially. The rapid increase of developers and QAs committed to the same repository quickly showed that our current workflow was insufficient to maintain a stable integration environment. Let’s see how our workflow evolved together with the growing team.

Let’s look at what our initial few-coders workflow looked like.

Requirements to merge branch to master:

  • review is done,
  • branch build is green (no build errors, all tests passed).

Responsibilities:

  • merging team member should verify if all requirements are met.

Process of merging:

  • done manually by anyone.

For the small team, this process was working without key issues. Code review and test results from the branch were good enough to detect major problems and keep our master branch together with our integration environment stable. Keeping functional mock tests as a part of our build was key at this point. Situations, where two or more branches were built in parallel and merged to master without testing both changes on a single feature branch, were rare. Problems caused by this were extremely rare. Unfortunately, it quickly changed when we expanded our team count. The process required an improvement.

20 coders workflow:
Requirements to merge branch to master:

  • review is done,
  • branch build is green (no build problems, all tests passed),
  • latest master must be merged into the future branch

Responsibilities:

  • BitBucket verifies if all requirements are met.

Process of merging:

  • dedicated persons can merge via BitBucket.

Adding one more requirement fully stopped the most problematic scenario from occurring. Limiting the group with rights to change master branch made communication about the problems and taking responsibilities much easier. Using an automated tool for the verification and merge process greatly reduced the number of human errors. We were happy with the changes and saw a significant improvement in the stability of our integration environment. Soon after introducing recent changes, we started encountering some unpleasant situations. From time to time, during the final build of the feature branch, someone merged latest changes to the master and was forcing other branches to repeat the master to feature merge and build process. It was manageable with 20 coders. It became a nightmare when the team kept growing. We decided that managing the merges process by a designated person is unacceptable, so we introduced the automated tool with the codename “Merge Queue”.

50+ coders workflow:
Requirements to merge branch to master:

  • review is done,
  • branch build is green (no build problems, all tests passed),
  • branch is added to the Queue.

Process of merging:

  • Automated by the Queue.

Responsibilities:

  • QA or Dev working on the branch have to put it on queue when finished,
  • queue verifies if all requirements are met, do all the merging, and building process.

The queue picks ready branches from the list and takes care of merging the latest master to it and builds branches one by one. If all requirements are met, the branch is merged to master. In case of any problems (like code conflicts or problems emerging after adding the latest master to the feature branch), the branch goes back to the development and testing process.

After going through all the improvements, do we see our initial approach as bad? Not at all. Different team sizes and assorted products require different solutions. The key factor is to keep in mind that our entry criteria might change during the development cycle. The final described solution didn’t solve all our problems. It even created a new one, but this is a story for another time.

Sabre Red 360 (SR360) is the travel agencies’ access point to the content of the Sabre services. With the global travel evolving continually, products and teams creating them have move with the times. SR360 team never stops improving and is ready to catch the opportunity when it comes.

Jakub Irzyk is making travel happen with Sabre’s Travel Solution for 9 years. A Software Architect in Testing, he is engaged in the SR360 project from the earliest stages. Jakub takes care of expanding testing frameworks and automated tools to ensure the highest standards for the delivered product.


Share this post