If you're new to DevOps, take a look at this tutorial on creating your first five-stage pipeline.


DevOps has become the standard solution for fixing slow, fragmented, or inoperative software development processes. The problem is that if you are new to DevOps and don’t know where to start, then you may lack understanding of these methods. This article will discuss the definition of the DevOps pipeline, and will also offer instructions for creating it in five steps. Although this tutorial is not exhaustive, it should give you the foundation to begin your journey and expand your knowledge in the future. But let's start with the story.

My journey through DevOps

I used to work in the Citi Group cloud team, developing the Infrastructure-as-a-Service (IaaS) web application for managing Citi cloud infrastructure, but I was always interested in how to make the development process more efficient and bring positive cultural changes to the development team. I found the answer in a book recommended by Greg Lavender, Citi's CTO for cloud architecture and infrastructure. The book was called The Phoenix Project ( The Phoenix Project ), and it explains the principles of DevOps, and it reads like a novel.

The table on the back of the book shows how often various companies deploy their systems in a release environment:

Amazon: 23,000 A Day
Google: 5,500 a day
Netflix: 500 a day
Facebook: Once a Day
Twitter: 3 times a week
Typical company: Once every 9 months

How are Amazon, Google and Netflix frequencies possible? That's because these companies figured out how to make an almost perfect DevOps pipeline.

We were far from that until we implemented DevOps in Citi. Then my team had different environments, but deployment to the development server was completely manual. All developers had access to only one development server based on IBM WebSphere Application Server Community Edition. The problem was that the server shut down whenever several users tried to deploy at the same time, so the developers had to tell each other about their intentions, which was pretty painful. In addition, there were problems with low-level test coverage of the code, cumbersome manual deployment processes, and the inability to track the deployment of code associated with a specific task or user story.

I realized that I needed to do something, and found a like-minded colleague. We decided to collaborate in creating the original DevOps pipeline - it installed the Tomcat virtual machine and application server while I was working on Jenkins, integrated Atlassian Jira and BitBucket, and also worked on test code coverage. This side project was very successful: we almost completely automated many processes, achieved almost 100% operability of our development server, provided tracking and improved test coverage of the code, and also added the ability to associate branches in Git with tasks in Jira or deployment. Most of the tools we used to build our DevOps pipeline were open source.

Now I understand how simple our DevOps pipeline was: we did not use extensions like Jenkins files or Ansible. However, this simple conveyor worked well, perhaps thanks to the Pareto principle (also known as the 80/20 rule).

A Brief Introduction to DevOps and CI/CD Pipeline

If you ask a few people, “What is DevOps?”, You will probably get several different answers.DevOps, like Agile, has evolved to span many different disciplines, but most people will agree on a few things: DevOps is a software development practice or software development life cycle (SDLC) that focuses on changing the culture in which developers and non-developers exist in an environment in which:

Automated operations that were previously performed manually;
Everyone does what he knows best;
The number of deployments for a certain period of time increases; Bandwidth increases;
Increased development flexibility.

Although having the right software tools is not the only thing you need to create a DevOps environment, some tools are needed. The key tool is continuous integration and continuous deployment (CI/CD). This pipeline has various stages (for example, DEV, INT, TST, QA, UAT, STG, PROD), many operations are automated, and developers can write high-quality code, achieve development flexibility and high deployment frequency.

This article describes a five-step approach to creating a DevOps pipeline similar to the one in the following diagram using open source tools.

Step 1: CI/CD Methods

The first thing you need is a CI/CD tool. Jenkins, an open source Java-based tool distributed under the MIT license, is the tool that has popularized DevOps and become the de facto standard.

So what is Jenkins? Consider this a kind of magical universal remote control that can talk with and organize various services and tools. A CI/CD tool itself, such as Jenkins, is useless, but it becomes more powerful as it connects to various tools and services.

Jenkins is just one of many open source CI/CD tools you can use to build your DevOps pipeline.

Jenkins: Creative Commons and MIT
Travis CI: MIT
CruiseControl: BSD
Buildbot: GPL
Apache Gump: Apache 2.0
Cabie: GNU

This is what DevOps processes look like with the CI/CD tool:


You have a CI/CD tool running on your localhost, but you can do little at the moment. Let's move on to the next stage of the journey into the world of DevOps.

Step 2: Manage source control systems

The best (and possibly easiest) way to verify that your CI/CD tool can do the magic is to integrate with the source code control (SCM) tool. Why do you need source control? Suppose you are developing an application. Whenever you create an application, you program, and it doesn’t matter if you use Java, Python, C++, Go, Ruby, JavaScript, or any of the gazillion programming languages. The code you write is called source code. In the beginning, especially when you work alone, you can probably put everything in a local directory. But when the project becomes larger, and you invite other people to collaborate, you need a way to prevent conflicts with the effective exchange of modifications. You also need a way to restore previous versions, because creating backups and copying/pasting into them is already outdated. You (and your teammates) need something better.

This is where source code control becomes almost a necessity. This tool saves your code in repositories, keeps track of versions and coordinates the work of project participants.

Although there are many tools for controlling source code, Git is the standard, and rightly so. I highly recommend using Git, although there are other open source options if you like.

Git: GPLv2 and LGPL v2.1
Subversion: Apache 2.0
Concurrent Versions System (CVS): GNU
Vesta: LGPL
Mercurial: GNU GPL v2 +

This is what the DevOps pipeline looks like with the addition of source code controls.


The CI/CD tool can automate verification, source code, and collaboration between members. Not bad? But how to make a working application out of it so that billions of people can use and evaluate it?

Step 3: Create an assembly automation tool

Fine! You can check the code and make changes to the source code control system, as well as invite your friends to collaborate in development. But you have not created an application yet. To make a web application, it must be compiled and packaged in a deployable batch format or run as an executable file. (Please note that an interpreted programming language, such as JavaScript or PHP, does not need to be compiled).

Use the build automation tool. Regardless of which build automation tool you decide to use, they all have the same goal: to collect the source code in some desired format and automate the task of cleaning, compiling, testing and deploying in a specific environment. The build tools will vary depending on your programming language, but here are some common open source options.

Title License Programming Language
Maven Apache 2.0 Java
Ant Apache 2.0 Java
Gradle Apache 2.0 Java
Bazel Apache 2.0 Java
Make GNU N/A
Grunt MIT JavaScript
Gulp MIT JavaScript
Buildr Apache Ruby
Rake MIT Ruby
A-A-P GNU Python
SCons MIT Python
BitBake GPLv2 Python
Cake MIT C #
Cabal BSD Haskell

Wow! You can put the build automation tool configuration files into the source code management system and let your CI/CD tool put everything together.


All is well, right? But where to deploy your application?

Step 4: Server for web applications

So far, you have a packaged file that can be either executable or installable. For any application to be truly useful, it must provide some kind of service or interface, but you need a container to host your application.

A server for web applications is just such a container. The server provides an environment in which the logic of the package being deployed can be determined. The server also provides an interface and offers web services, opening sockets for the outside world. You need an HTTP server, as well as some environment (such as a virtual machine) to install it. In the meantime, let's assume that you learn more about this later (although I will talk about containers below).

There are several servers for open source web applications.

Title License Programming Language
Tomcat Apache 2.0 Java
Jetty Apache 2.0 Java
WildFly GNU Lesser Public Java
CDDL & amp; GNU Less Public Java
Django 3-Clause BSD Python
Tornado Apache 2.0 Python
Gunicorn MIT Python
Python MIT Python
Rails MIT Ruby
Node.js MIT Javascript

Ваш DevOps-пайплайн почти готов к использованию. Хорошая работа!


Хотя на этом можно остановиться и заниматься интеграцией самостоятельно, качество кода - важная вещь для разработчика приложений, и об этом нужно беспокоиться.

Шаг 5: Покрытие тестирования кода

Реализация тестов может быть еще одним громоздким требованием, но разработчики должны ловить любые ошибки в приложении на ранней стадии и улучшать качество кода, чтобы гарантировать, что конечные пользователи будут удовлетворены. К счастью, существует множество инструментов с открытым исходным кодом для терианиеинниниени ва Еще лучше то, что большинство инструментов CI/CD могут подключаться к этим инструментам и автоматизирсее.

Тестирование кода состоит из двух частей: фреймворки для тестирования кода, которые помогают писать и запускать тесты, а также инструменты для формирования предложений, которые помогают улучшить качество кода.

Системы тестирования кода

Название Лицензия Язык программирования
JUnit Eclipse Public License Java
EasyMock Apache Java
Mockito MIT Java
PowerMock Apache 2.0 Java
Pytest MIT Python
Hypothesis Mozilla Python
Tox MIT Python

Системы рекомендаций по улучшению кода

Название Лицензия Язык программирования
Cobertura GNU Java
CodeCover Eclipse Public (EPL) Java
Coverage.py Apache 2.0 Python
Emma Common Public License Java
JaCoCo Eclipse Public License Java
Hypothesis Mozilla Python
Tox MIT Python
Jasmine MIT JavaScript
Karma MIT JavaScript
Mocha MIT JavaScript
Jest MIT JavaScript

Обратите внимание, что большинство инструментов и фреймворков, упомянутых выше, написаны для Java, JavaScript и Python, C++ поскольку и C # являются проприетарными языками программирования (HAI хотя и имеет открытый исходный код).

Теперь, когда вы реализовали инструменты покрытия кода тестами, ваш DevO-пайплайн дольена по,ана

Дополнительные шаги


Как я уже говорил, вы можете размещать свой сервер на виртуальной машине или сервере, но контейненпа

Что такое контейнеры? Краткое объяснение заключается в том, что виртуальная машина нуждается в огромном объеме памяти операционной системы, превышающем размер приложения, в то время как контейнеру нужно всего лишь несколько библиотек и конфигураций для запуска приложения.Obviously, the virtual machine still has important applications, but the container is an easy solution for hosting the application, including the application server.

Although there are other container options, the most popular are Docker and Kubernetes.

Docker: Apache 2.0
Kubernetes: Apache 2.0

Intermediate Automation Tools

Our DevOps pipeline is primarily focused on co-creating and deploying applications, but there are many other things you can do with DevOps tools. One of them is the use of Infrastructure as Code (IaC) tools, which are also known as intermediate automation tools. These tools help automate installation, management, and other tasks for middleware. For example, an automation tool can retrieve applications like a web application server, database, and monitoring tool, with the correct configurations, and deploy them to the application server.

Here are some open source middleware tools:

Ansible: GNU Public
SaltStack: Apache 2.0
Chef: Apache 2.0
Puppet: Apache or GPL


Learn the details of how to get a sought-after profession from scratch or Level Up in skills and salary, taking SkillFactory paid online courses: