With the prevalence of distributed computing, software integration becomes a vital point of any software project these days. The challenge of any large software project is finding problems early and often. A continuous integration (CI) environment allows us to detect problems from build to unit test to basic deployment. If a change breaks the process, CI allows us to find out immediately and fix the problem before it simmers for days or even longer.
Many organizations have realized that the continuous integration paradigm is required to be agile and able to meet business demands. To that end, their development teams have created their own processes and tools/scripts to automate the build and test processes as much as possible. Such tools/scripts are at times difficult to build, costly and time consuming. Today, however, a number of free and open-source CI tools are available to help with the build and test aspects of a developer's job.
In this article, I present the pain points that CI helps alleviate and then survey a pair of the best known open source CI tools that can help you implement CI in your organization.
Before Continuous Integration: The Integration Problem
Integration costs time, money and other resources. As software components become more complicated, the cost to integrate various pieces increases. Another important factor is that as the number of components increases, the number of integration points increases exponentially. Figure 1 further illustrates this challenge. In a large enterprise, this model is ever more complex as the number of software components tends to be in the hundreds, if not thousands.
Figure 1. The Difficulty of Software Integration
In the 90s, messaging middleware and eventually Service Oriented Architecture (SOA) helped with this integration problem by inserting a common bus or interface for all the components to speak through. The Enterprise Service Bus (ESB) is still a major part of the infrastructure for many organizations, large and small. SOA further commoditized the concept of using a known interface and dynamic discovery of a "service" as a measure to ease integration (see Figure 2).
Figure 2. The "Integration Problem" Conceptually Solved
The "integration problem" solved, right? Not exactly; the SOA paradigm just moved the bottleneck to other parts of the software engineering process lifecycle. Let's look at the software engineering lifecycle (SEL) at a very high-level (see Figure 3).
Figure 3. The Software Engineering "Cycle"
Figure 3 is not a depiction of what a real SEL is or should be. It merely presents a conceptual representation of the process:
What ESB, SOA and other such technologies did was to dramatically decrease the design and development cycles, but the side effect was an increase in the test and QA cycles! Many books and papers have been written on this topic (e.g. Object-Oriented Defect Management of Software by Houman Younessi, 2002), but decoupling essentially introduces bugs into the design that are harder to catch until the code is ready to be tested. This increases testing time, and some of those bugs go undetected until deployment.
A number of other factors have contributed to the increase in QA/testing cycles, some of which are business driven (demand for faster delivery) and some of which are simply human error (dispersed team members). The fact of the matter is that a change of approach was required to increase efficiency in the process.