Sautez le pas : migration de SVN vers Git

Matt Shelton
Matt Shelton
Retour à la liste

Nous migrons vers Git et nous avons défini comment utiliser Gitflow et Maven combinés dans un workflow de développement efficace. Avant de décrire notre workflow actuel, il est important de savoir de quoi nous sommes partis.

Auparavant…

In the our previous world, before migrating from SVN to Git, all of our version management was manual. Development took place on trunk simultaneously across all active features for the team. When a developer committed their changes to SVN, Bamboo would kick off a snapshot build (1.0.0-SNAPSHOT). If their work passed integration testing, they'd run the release plan after manually verifying nobody else had run a subsequent snapshot[1]. That release build (1.0.0-1)[2] would be quickly smoke tested and then handed to QA for functional validation.

We were "releasing" all the time; every build that went to QA was from a Maven release goal without incrementing the minor or patch number. We had Bamboo tack on that -buildnumber to each release so that we could track specific releases to QA,.

Then, once QA blessed the "last one" for that release, we'd increment the minor version using mvn version:set. This meant that if yours truly wasn't completely on top of his game, we ran the risk of forgetting to increase that version number and building a "release" of 1.0.0-x that was after what we had released to production. Big mess. Big pain. But it meant that every build out of development had a clear, trackable, permanent number.

C'était bien, mais le suivi était un cauchemar.

Nous ne voulions plus procéder ainsi. Nous voulions simplement effectuer une livraison quand quelque chose était prêt à être transmis au client. Nous ne voulions plus de numéros de build en production (majeur.mineur.patch uniquement).

However, we also wanted to ensure QA had a way to track a specific delivery to a specific release. QA couldn't be testing SNAPSHOT releases all the time. Thankfully, Jira Software makes some of this easy by showing links between Jira issues, Bitbucket repositories and build plans containing builds for that issue.

À la fin, nous avons décidé d'appliquer certaines règles :

  1. Developers perform integration testing from their feature branch, which they are keeping up to date with the develop branch.
  2. Once integration testing is complete and passing, they issue a pull-request and have their code pulled into the develop branch.
  3. They promote a QA candidate cut of our common project, and then trigger a QA candidate cut of develop for their product project. Since we need to use a specific common project release at build time[3], we prompt a developer for the aforementioned common QA candidate release version.
  4. Builds candidats ayant réussi les tests d'AQ uniquement. Jamais d'instantanés.

Nous répétons les étapes 1 à 5 pour toutes les fonctionnalités, les bugs trouvés lors de l'assurance qualité, etc. Ensuite, une fois que nous avons terminé toutes nos tâches pour une version, nous suivons le Gitflow et créons une branche sur une branche de livraison en faisant la promotion d'un build de candidat AQ dans Bamboo. Nous créons ainsi notre premier ensemble de candidats de livraison et l'AQ effectue les tests finaux, les tests de régression, etc.

De bons builds

Getting your builds just right can be tricky. Before we moved everybody off of SVN and our original Bamboo build plans, we set up a POC project and then worked out all of our Bamboo plans using those. In the end, we found the following plans met our needs:

  1. develop: Triggered from Bitbucket Server, it creates SNAPSHOT builds with every change pulled in from pull requests. The build plan has three stages:
    1. Instantané : crée des instantanés sans toucher aux versions.
    2. QA Build - Builds a "QA Candidate" release numbered X.Y.Z-qa{buildnunber} from the same git commit used to create that snapshot.
    3. Promouvoir en livraison : crée la branche release et incrémente le numéro de version mineure de develop à partir de la commande commit git utilisée pour créer le build d'AQ.
  2. feature: Set up with a default build plan, but mostly is watching for new branches created with feature/* in the name, using Bamboo's plan branches feature, and creates a branch plan for that branch whenever it does. It has only one stage, which builds SNAPSHOTs.[4] Also, because we don't want feature to build by itself, we point its repository to a branch called "fake-branch" so that it never triggers.
  3. release and hotfix: These have the same set of build and release steps so they get to share a plan. The plan also points to a "fake-branch" and only cuts branch plan-based releases with the following two stages:
    1. Build Candidate Release - This creates a release numbered X.Y.Z-rc{buildnumber}.
    2. Finaliser la livraison : définit le numéro de livraison final, fait un merge sur la branche master et crée un tag. Cette étape supprime ensuite la branche release, ce qui désactive le plan.
  4. support: This is nearly identical to release and hotfix, but it never merges to master. Instead, we increament the minor version at the end of the Finalize Release stage.

Because feature, releasehotfix, and support plans are all running branch plans, when you view them in Bamboo, you see “Never built” for each plan. The first time this happened everyone had a blank look on their face... _where are my builds?? but then we realized this made sense.

We have so many build plans[5] that if bamboo displayed all of the plans inline including branch plans, you'd be scrolling forever and a day. The information overload would actually be less helpful. So we click once more to see our branch plan status. It's not a big deal, but it would be nice to find an easy way to see them all.[6]

Processus de migration

Lorsque nous avons constaté que le projet POC avait fonctionné comme nous le souhaitions et que le workflow des développeurs était cohérent et fiable, nous avons importé notre base de code SVN dans Git et nous avons soumis chaque workflow à plusieurs autres tests pour éliminer les tickets du build, les anomalies du workflow, etc.

We found some things like my note above about cleaning the Maven workspace every time you run a feature branch build in Bamboo, and certain times we needed various flags for a specific maven lifecycle. Generally, these were easy to figure out, but once in awhile there was much shaking of fists and gnashing of teeth.[8]

Nous avons alors décidé que le moment était idéal et nous avons annoncé la date de notre migration. Dans le cadre de la migration, nous avons décidé de faire le grand nettoyage afin de poser des bases saines pour notre dépôt Git. Nous avons par exemple procédé aux opérations suivantes :

  • Reformatage du code en vrac afin d'autoriser certaines vérifications de style plus strictes dans le cadre du process de build.
  • Conversion de toutes les fins de ligne au format UNIX
  • Compactage de trois projets de dépendance de haut niveau en un et refactoring de tout le code dépendant en conséquence.

Cela fait, nous avons exclu tout le monde hors de SVN, l'avons passé en lecture seule, puis avons réalisé un dernier pull dans Git. Nous avons organisé une session de formation avec tous nos développeurs pour parcourir une fois de plus notre workflow et le présenter en action avec la base de code réelle.

Je n'ai jamais effectué de migration plus simple. Nous avons arrêté SVN lundi vers 17 h et à 22 h, nous en avions terminé tous nos tickets de build initiaux. Nous n'avons rencontré aucun problème majeur : il nous restait juste à nous armer de patience pour certains points. La formation a eu lieu le lendemain matin, et au déjeuner, nous recommencions déjà le développement de nouvelles fonctionnalités.

Nouveau flux de développeur

Une fois dans la migration, nous avons pu voir comment ce workflow fonctionnerait en situation réelle. Lorsqu'un développeur se met à travailler sur une fonctionnalité (ABC-4321), il a besoin de certaines informations pour commencer :

  1. From Jira Software, in the Development area of the issue, click on Create Branch.
  2. This opens a screen within Bitbucket Server that lets them select the branch type (usually feature), repository, and branch name. Because of our Maven extension I mentioned in the previous post, the branch name is always the Jira issue key, no description.
  3. Répétez les étapes 1 et 2 pour tous les projets associés à cette fonctionnalité, en utilisant toujours la même clé de ticket unique.
  4. git pull && git checkout origin/feature/ABC-4321 feature/ABC-4321[7]

This workflow is straightforward, repeatable and reliable. Developers can work in isolation and pull in contributed changes from develop as they move forward. The branching action can feel a bit repetitive if, say, a user story has work in all four of our product verticals and the common project. We've been thinking about automating this with some sort of Jira workflow post-function to call the Bitbucket Server REST API, but that might be overkill for something that isn't costing us too much developer time.

Enseignements

La migration de SVN vers Git avec un workflow tout nouveau a pris près de sept mois. Nous avons passé une grande partie de ce temps à nous battre avec Maven.

Je dois aussi admettre que nous avons rencontré quelques problèmes de personnel en cours de route ; parallèlement à ce projet, le groupe d'ingénieurs qui était chargé de la migration devait fournir son assistance à un service de 800 personnes pour l'ensemble d'outils Atlassian, un support de production pour nos applications de plateforme et travailler sur nos projets opérationnels de R&D. Finalement, nous avons dédié trois personnes quasiment à temps plein à la migration, que nous avons achevée en un mois à peu près.

Malgré tout, nous avons appris beaucoup :

  1. Quel que soit le niveau de préparation, vous n'êtes jamais vraiment préparé aux conditions réelles. Par exemple, chaque type de build de livraison que nous avons essayé d'exécuter échouait systématiquement au premier essai, pour une raison ou une autre. Après avoir réparé un build de projet pour ce type, nous avons copié la configuration sur les autres et nous n'avons plus eu une seule répétition.
  2. This workflow generates a lot of builds. So. Many. Builds. We needed to double our Bamboo agents to keep up.
  3. This workflow generates a lot of build artifacts. Within the first week we ran out of disk space on our Artifactory instance and had to spend a solid day manually purging old release candidates and QA builds no longer needed. Then we needed to think of a way to ensure that when feature branches are removed, we also remove all of their branch-specific snapshot artifacts.
  4. The team doesn't really like needing to pick between a hotfix or having a support branch. It makes sense to be able to cut a hotfix, but most of the time they want a support branch. We might decide to only use hotfixes on special occaisions when the merge would truly be straightforward.
  5. The combination of Jira Software, Bitbucket Server, and Bamboo are seriously killer. Watching someone start work in Jira Software, create a branch and immediately have a branch plan built and ready to validate their work is beautiful.
  6. Pull Requests in Bitbucket Server are the greatest thing since sliced bread. Between keeping a push-happy engineer at bay or making sure we're ready for an offshore team's contributions, we couldn't be happier with the pull-request process. Given we perform code inspection in Crucible rather than at pull-request, we're able to use it for quick sanity checks as well.
  7. Our previous SVN-based tags had been tagged by a service account that was performing the build. Since that user wasn't real, when we tried to create branches from tags, our git-hook to validate the user was valid for a given commit failed. I wrote an article on my personal blog on how to change the author of a single commit in Git, which came in handy the first time we needed to create a support branch from an old SVN-based tag... which was the day after the migration!

Dans l'ensemble, notre migration a été une réussite. Elle n'a pas résolu tous les problèmes de mon équipe, mais elle en a éliminé beaucoup et nous a apporté un socle solide pour aller de l'avant.

[1]: An uncomfortable amount of Hipchat conversations in our Developers room went to asking if anyone needed to commit any changes before a release build was made. Prior to Hipchat: Lync, email, or yelling over a cube wall.

[2]: The -1 is an incrementing build number for the plan that never resets to 1. It didn't take too long for release builds to have numbers in the high 100s.

[3]: The POM still refers to a -SNAPSHOT release of common here and can't risk the build pulling the wrong SNAPSHOT.

[4]: It also forcibly cleans up the workspace every time it builds. YMMV, but we found this to be necessary.

[5]: Across the org we have something like 120 plans in this particular Bamboo instance, growing all the time. Filtered just to my team's plans, we're roughly half of that list. With all of our branches we're pushing 200.

[6]: One of my engineers wrote a greasemonkey script that lets him see all branch plans. It only works if you aren't a Bamboo admin due to the number of visible plans. I'm working on whipping up a dedicated AtlasBoard for myself.

[7]: The exact steps here vary depending on if the developer is using the command line or using Eclipse to switch branches.

[8]: We only burned maven in effigy once or twice.

Prêt à découvrir Git ?

Essayez ce tutoriel interactif.

Démarrez maintenant