Developers are familiar with the problems of integration: new code has been written and most of the work is now done. It is now a question of integrating the source code into the overall project and that’s when the problems arise. In order to avoid a disaster at the end of the long development phase, many teams opt for continuous integration: changes are integrated directly into the project on a day-to-day basis and, in the best case, several times a day.
Just like continuous delivery, continuous integration is first and foremost in an agile software development environment. The goal of this modern approach is to progress in stages in order to design the development process more efficiently and to be able to react flexibly to changes. Continuous integration was first mentioned in Kent Beck’s description of the agile method of Extreme Programming. But a priori, the idea of continuous integration would precede it. For example, it is already involved in the Booch method.
Related: Best continuous integration tools.
What is continuous integration?
Continuous integration provides us with a good solution when the company is working on a large project or a client wants to have software that is both complete and complex. Different teams work on the design of parts of the application and the developers are responsible for programming the different features. After a work of several months or even years, the entire work must be grouped together and that’s when the problems arise. In such a case, error detection and correction, grouping all code snippets can take several months to finally get closer to the final testing phase and deployment.
In the context of continuous integration, the integration of the new code is done much earlier and not only when all stakeholders have completed their subdomain. Instead, developers embed their finished code once or more times a day into the mainline, the source code that is accessible by all programmers. Since these are always relatively short sections of code, the integration is also rather fast. It only takes a few minutes for a developer to make the result of his work available to the rest of the team. If an error is then discovered, it can easily be located and, in the best case, corrected quickly.
How to proceed
In practice, continuous integration typically takes place as follows: a developer is tasked with creating a feature. Before starting the programming work, he downloads the code of the current version of the application, called mainline. He can work in this version which is now on his personal computer (also called working copy). When he has completed his task, he tests the program, corrects any errors, and can now publish the new version to the mainline.
However, the developer does not work on the program alone. While he was making his changes, his colleagues most likely performed other tasks, so each developer on the team has a different version on their computer. In addition, changes may have been made to the mainline version since it was downloaded, and the programmer must first integrate them into his working copy. If an error occurs, it is up to him to correct it. Once this work is done, the programmer can then insert his changes into the mainline. When no errors occur during the test (an error that can occur, for example, if he has not refreshed his working copy correctly), the process is effectively completed, and the developer can focus on his next task.
Behaviors
Several rules must be observed when working in this way: in doing so, programmers generally rely on the principles developed at the time by Martin Fowler (himself a software developer) to achieve continuous integration. These principles aim first and foremost to ensure that all stakeholders are at the same level and that no one causes a disaster by taking a different approach.
One source
Even if it seems obvious, this is one of the main factors: all team members must use the same source (the same repository or repository) when working on the code. This doesn’t just apply to the source code itself. Indeed, other elements, such as databases, are needed to ensure a viable application. These databases must also be consolidated in one place. That’s why Martin Fowler recommends setting up a repository, so that even a developer with a completely new work computer can work and find all the necessary files centrally in one place.
Automated builds
In order to design a viable program from source code, developers need to compile it, refresh databases, and move files to the right place. This process can be automated. Ideally, it should be possible to run a build process using a single command.
System with automatic test
By integrating testing mechanisms into the build process, the team benefits from greater automation and thus faster continuous integration. Just like the build process itself, tests can be run with minimal effort (to do it right, you can implement a whole bunch of different test mechanisms).
Daily onboarding
Continuous integration can only work when all team members contribute to the system. When one colleague publishes a non-definitive code in the mainline, the others leave on the wrong basis. All agree to work on a stable system, but if someone retains the code for too long (more than a day), the ensuing error finding can become a problem. As is often the case, communication is a critical factor in continuous integration, but when developers keep themselves regularly informed, small difficulties can be quickly overcome.
Operational mainline
The program code found in the mainline should always be tested and operational. That’s why each developer can embed their code only in this primary branch and not in a personal secondary branch. In doing so, each developer must also ensure that their personal contribution is operational. After integration, it must also test the code and build. If an error occurs, it must correct it. This ensures that the code is always error-free.
Immediate repair
In the case of continuous integration, it is essential that the mainline does not have an erroneous version. For developers, this means that fixes cannot be postponed. According to Martin Fowler, the fact that builds do not work and that the code needs to be reworked is not a problem in itself, but the continuous integration system requires that the correction be done immediately. Indeed, all developers must be able to rely on functional code in the Mainline. If this is not the case, the entire team works in unstable systems, which triggers an avalanche of errors.
A short period of integration
The actual integration of the code (including the testing phase) should be done as quickly as possible. Extreme programming (XP) allows only 10 minutes for integration. Because in some circumstances a developer has to integrate code several times a day, build delays result in significant time wastes. It is therefore necessary to put in place mechanisms to speed up the process. In order to achieve such speed, one must also accept not to directly perform every possible and imaginable test. Instead, we choose to implement a two-step system: in the first phase, which occurs during the daily work, the tests are carried out in such a way that short build times can be achieved. The second test phase, on the other hand, lasts several hours and also includes more in-depth checks.
Copied test environment
It is in principle relevant to perform tests in a remote and secure environment. However, care must be taken to ensure that the tests are configured in the same way as in the production environment. In some cases, this can be very expensive, as the machine must be adjusted precisely in the same way. However, with the virtualization of complete computers, this cost factor is always less important.
Easy access
All stakeholders must be able to easily access the latest version and run the program. Achieving this is relatively simple: since continuous integration requires in all cases the grouping of all files in a single repository, this location must be known to all. There are several good reasons for this: testers can start additional testing without waiting during the ongoing programming process, stakeholders can use executable files for demonstration purposes, and quality managers can verify the numbers.
Clear and understandable communication
It is essential that all stakeholders have access to the text and the executable file and are clear about who needs to make what changes. Communication also includes the fact that developers get along with each other when they are in a build process. To do this, some teams use separate views or visual representations that indicate that an integration is in progress.
Automated dispatch
It is also necessary to automate software deployment. Files must be transposed from one environment to another, which can be time-consuming. Here, it is therefore relevant to use scripts. These scripts automate and speed up the process.
Advantages and disadvantages of continuous integration
In the day-to-day work, it is often found that continuous integration does not only have advantages despite its qualities. While it effectively saves a long and tedious integration phase at the end of the project and resolves problems early, for the teams involved, the transition to continuous integration can be very complicated. In such a case, this process may even take longer than it saves.
| Benefits | Disadvantages |
|---|---|
| Possibility of early error detection | Conversion of usual processes |
| Ongoing feedback | Requires additional servers and environments |
| No overhead unlike one big final integration | Need to develop appropriate testing processes |
| Accurate recording of changes | If multiple developers want to integrate their code at approximately the same time, timeouts may occur |
| Continuous availability of a current operational version | |
| Need for progressive work |
Overview of useful continuous integration tools
In principle, continuous integration can be managed without a specially created tool, as all work steps can be performed manually. However, this requires a great deal of discipline and an even greater effort. It is possible to make it easier with useful tools. These usually provide a server and help with construction and version control.
- Jenkins: The hugely popular Jenkins is programmed under Java and is a fork of Hudson whose development has since been stopped. This open source software uses different build tools and version control systems.
- Travis CI: This continuous integration tool is particularly popular because it works seamlessly with GitHub. Online filing notifies Travis of changes in the code. The software is available in a free version for open source projects and in a paid version for all other projects.
- Bamboo: With the Bamboo server, developers can perform integration, deployment, and versioning. The manufacturer Atlassian offers this software as a web interface based on Java. Bamboo supports developers with automations and works with different build tools. For open source projects, this normally paid offer is also available for free.
- GitLab CI: GitLab offers its own program for continuous integration that works with this popular version control system. The pipeline can be customized and tailored to each project. In addition, GitLab also supports CI Docker.
- CircleCI: This continuous integration program is available in two versions. It is possible to opt for a version directly on the provider’s cloud or to create a dedicated local server for use.
- CruiseControl: Originally developed by ThoughtWorks (a company built around Martin Fowler), CruiseControl has since become an independent project. This free software is based on Java and can be used on different platforms. In particular, CruiseControl offers developers a dashboard, a clean website, on which they can identify the status of the build.
- Codeship: Codeship aims to provide developers with the ability to seamlessly integrate with ease. Automations can be easily created on the basis of container technology. For this task, the company offers two different versions: the basic version and the pro version.
- TeamCity: TeamCity commercial software places great importance on interoperability with other tools. Many programs are already supported by the standard version, but the range can also be extended with plugins. There is also another peculiarity in the “Pre-Tested Commit” approach. TeamCity automatically checks the rewritten code before integration into the mainline and notifies the developer of any errors.