The translation of the article was prepared ahead of the start of the Automated JavaScript Testing course .

ITKarma picture

I was asked several times about the difference between a QA (Quality Assurance) engineer and a tester (QC - Quality Control), and I realized that even if people who develop software confuse such simple concepts, what’s already talk about the rest? But by and large, you understand that they do not care, and that is their right!

People just need quality products! Managers want to have software without bugs [1] of the best quality, users need high-quality applications, and this can be achieved through tireless testing [2]. But what does this mean? What is quality? Satisfaction is all so trivial! A measure of quality is the happiness of end users! Achieving this abstraction becomes a real challenge for the team [3].

When developing any product, especially software without testing, we face risks [4], and they, with a high degree of probability, involve a loss of funds [5]. So, if you are not a big fan of testing, but quality is important for you, then I suggest you think again and let the quality assurance processes do their job. This applies to all team members and especially to engineers who are involved in test design [6] - quality assurance engineers.

Thus, the message that a person should be satisfied with a quality application is reinforced by approaches that have already been formed in best practices [7]. Once you start testing, you will learn about the existence of various types of testing [8] and testing documentation (for example, a test plan). Then you can process test cases [9] based on different test design techniques [10]. In the end, you will discover automation and by then you will understand the difference between QA and QC [11].

Glossary: ​​

  1. mismatch between actual and expected behavior
  2. complete product verification by verification, validation based on specifications to detect problems
  3. people who develop applications, such as designers, programmers, testers
  4. (a) the possibility of a negative outcome (in accordance with the Cambridge Dictionary);
    (b) actions in the hope of a positive outcome (in accordance with the Google dictionary)
  5. (a) 125 million Mars soil probes lost due to simple mathematical error
    (b) many human errors that led to the crash of the Boeing 737 Max
  6. test pyramid , a framework for developing a test suite that optimizes test execution speed with the confidence that your application is working properly way
  7. seven testing principles on the International Software Testing Qualifications Board
  8. various types of software testing from Atlassian
  9. stages of reproduction with expected results in accordance with the requirements of the application
  10. general testing techniques that you should know
  11. look for the answer here


  • Testing Dot Kom, or a Handbook on Cruelty to Bugs in Internet Startups, Savin R. 2007
  • Continuous delivery. The practice of continuous updates, Eberhard W. 2017
  • Software Testing Automation Tips, Alpayev G.2017

[Puppeteer] [Mocha] Upgrade your code with test coverage.

Test Pyramid

Since the release of Puppeteer end-to-end testing has become a fast and reliable way to test functionality. Most of the things you can do manually in the browser can be done with Puppeteer. Moreover, Chrome without graphics reduces the load on performance, and native access to the DevTools protocol makes Puppeteer just a terrific tool. Imagine that every time we develop an interface, we just check the final look in the browser - without TDD, we are faced with an ice cream-antipater test pyramid.

ITKarma picture
Top to bottom: Manual tests/Automated GUI tests/Integration tests/Unit tests

But we like ice cream, why do we need alternative solutions? I will show you how to upgrade your source code with tests, no matter what engine you use, with the confidence that your application works as expected, because Puppeteer will check the functionality for you.


I have a complete step-by-step instruction based on a simple project , which I forked and provided it with a multifunctional test upgrade for demonstration purposes (or to show off). So, if you have one more, please:

1) Install the dependencies in the root

npm i puppeteer mocha puppeteer-to-istanbul nyc -D 

2) Provide your instance on the endpoint (my simple solution for *.html http server )

3) Create the test directory and fill out {yourFeature} _test.js with the following appropriate template ( fill in the hooks before and after ), try expanding it with project-specific selectors and behaviors:

const puppeteer=require('puppeteer'); const pti=require('puppeteer-to-istanbul'); const assert=require('assert');/** *./test/script_test.js * @name Feature testing * @desc Create Chrome instance and interact with page. */let browser; let page; describe('Feature one...', () => { before(async () => {//Создание экземпляра браузера browser=await puppeteer.launch() page=await browser.newPage() await page.setViewport({ width: 1280, height: 800 });//Подключите покрытие JavaScript и CSS await Promise.all([ page.coverage.startJSCoverage(), page.coverage.startCSSCoverage() ]);//Конечная точка для эмуляции изолированного окружения await page.goto('http://localhost:8080', { waitUntil: 'networkidle2' }); });//Первый тестовый набор describe('Visual regress', () => { it('title contain `Some Title`', async () => {//Настройка let expected='Some Title';//Выполнение let title=await page.title();//Проверка assert.equal(title, expected); }).timeout(50000); });//Второй тестовый набор describe('E2E testing', () => { it('Some button clickable', async () => {//Настройка let expected=true; let expectedCssLocator='#someIdSelector'; let actual;//Выполнение let actualPromise=await page.waitForSelector(expectedCssLocator); if (actualPromise != null) { await; actual=true; } else actual=false;//Проверка assert.equal(actual, expected); }).timeout(50000);//Сохранить покрытие и закрыть контекст браузера after(async () => {//Отключить покрытие JavaScript и CSS const jsCoverage=await page.coverage.stopJSCoverage(); await page.coverage.stopCSSCoverage(); let totalBytes=0; let usedBytes=0; const coverage=[...jsCoverage]; for (const entry of coverage) { totalBytes += entry.text.length; console.log(`js fileName covered: ${entry.url}`); for (const range of entry.ranges) usedBytes += range.end - range.start - 1; }//вывести в лог исходное байтовое покрытие console.log(`Bytes used: ${usedBytes/totalBytes * 100}%`); pti.write(jsCoverage);//Закрыть экземпляр браузера await browser.close(); }); }); 


  1. Run the test described above for endpoint scripting using the mocha
  2. command
  3. Get coverage collected by running the test with nyc report .
  4. I suggest you use your package.json with the following scripts, which makes it very easy to perform tasks like npm test or npm run cover

"scripts": { "pretest": "rm -rf coverage && rm -rf.nyc_output", "test": "mocha --timeout 5000 test/**/*_test.js", "server": "http-server./public", "coverage": "nyc report --reporter=html" }, 


My project is approximately 62% covered

ITKarma picture

We can get the report in html and see a closer look.

ITKarma picture

You can see that Branches and Functions are 100% covered. While I tested the Puppeteer coverage feature (like Coverage devTool ) I created this bugreport.

ITKarma picture

[Bug] Invalid branches fall into coverage statistics # 22
Posted by storenth on December 27, 2018
When the nyc report --reporter=htm is ready, I tried to look at ./coverage/index.html and found a big defect in the Branch coverage number, it is always 100%. To study this problem, I suggest cloning this simple repository for local playback.

ITKarma picture

Holivar Unit vs E2E

As someone who has enough passion for research so as not to get bored with this process, here's what I tell you: we need to pay more attention to unit testing environments such as Mocha , for writing from Unit to Acceptance tests, rather than writing them yourself. I think it doesn’t matter which test you write if your code base is covered. Times have changed. Now, with available coverage functionality, other tools, such as a traceability matrix as a quality measure, look awful because stakeholders still need to rely on the tester's words.


I highly recommend taking some time and getting to know my github-working-draft project before you break firewood. I would appreciate any cooperation and feedback. Feel free to contact me with any questions.

Learn more about the Automated JavaScript Testing course