TARTARE

Gherkin-like Mocha extension and reporter,
and testing tools library for Node.js

https://github.com/telefonicaid/tartare/

WHAT


Tartare is a BDD framework which
extends Mocha to use Gherkin syntax.

It also provides useful tools for testing.

WHY


By the time Tartare was born, BDD frameworks
for Node.js didn't fit our needs:

  • Immature frameworks lacking key features (example tables)
  • Not using Mocha behind the scenes (unit testing is being done with Mocha)
  • Mocha itself use BDD but not Gherkin syntax
  • Asynchronous code may mess up steps implementation (callback hell)

GHERKIN SYNTAX

New keywords

  • feature
  • scenario
  • given
  • when
  • then
  • and

New before/after hooks

  • beforeAll
  • afterAll
  • beforeFeature
  • afterFeature
  • beforeEachScenario
  • afterEachScenario
  • beforeScenario
  • afterScenario
  • beforeEachVariant
  • afterEachVariant

New reporters

Gherkin reporter for console


Coloured output adapted to Gherkin syntax

Real stats and metrics counting
features, scenarios, variants and steps

New reporters

Markdown Gherkin reporter

Using Tartare


On your code


var tartare = require('tartare');


Running tests


$ tartare --reporter gherkin --recursive ./acceptance
					

$ tartare --reporter gherkin-md --recursive ./acceptance

Features



feature('Addition',
		'In order to avoid silly mistakes',
		'As a math idiot',
		'I want to be told the sum of two numbers',
		function() {

});

Scenarios



feature('Addition', function() {
  scenario('Add two numbers', function() {

  });
});

Steps



feature('Addition', function() {
  scenario('Add two numbers', function() {
    given('I have entered 50 into the calculator', function() {

    });
    and('I have entered 70 into the calculator', function() {

    });
    when('I press add', function() {

    });
    then('the result should be 120 on the screen', function() {

    });
  });
});

Report output


Manual

Features/Scenarios/Steps



feature('Multiplication', function() {
  scenario('Multiply two numbers');  // This is a manual scenario
});
feature.manual('Addition', function() {  // This is a manual feature
  scenario('Add two numbers', function () { [...] });
});

Skipped

Features/Scenarios/Steps



// This feature is completely ignored
feature.skip('Multiplication', function() {
  scenario('Multiply two numbers', function() {
    [...]
  });
});

Variants



var dataset = [
  { desc: '50 + 70 = 120', num1: 50, num2: 70, result: 120 },
  { desc: '100 + 90 = 190', num1: 100, num2: 90, result: 190 }
];

feature('Addition', function() {
  scenario('Add two numbers', dataset, function(variant) {
    given('I have entered ' + variant.num1 + ' into the calculator', function() { });
    and('I have entered ' + variant.num2 + ' into the calculator', function() { });
    when('I press add', function() { });
    then('the result should be ' + variant.result + ' on the screen', function() { });
  });
});

Variants


Variants


Manual & Skipped


var dataset = [
  { manual: true, desc: '50 + 70 = 120', num1: 50, num2: 70, result: 120 },
  { skip: true, desc: '100 + 90 = 190', num1: 100, num2: 90, result: 190 }
];

Bug monitoring

Minor bugs


They are not executed and count as passed


feature('Addition', function() {
  scenario('Add two numbers', function() {
    given('I have entered 50 into the calculator', function() { });
    and('I have entered 70 into the calculator', function() { });
    when('I press add', function() { });
    then('the result should be 120 on the screen', function() { });
  }).minorBug('bug12345');
});

Bug monitoring

Major bugs


They are executed and count as failed


feature('Addition', function() {
  scenario('Add two numbers', function() {
    given('I have entered 50 into the calculator', function() { });
    and('I have entered 70 into the calculator', function() { });
    when('I press add', function() { });
    then('the result should be 120 on the screen', function() { });
  }).majorBug('bug12345');
});

Bug monitoring

Variants



var dataset = [
  { minorBug: 'bug12345', desc: '50 + 70 = 120', num1: 50, num2: 70, result: 120 },
  { majorBug: 'bug54321', desc: '100 + 90 = 190', num1: 100, num2: 90, result: 190 }
];

Bug monitoring

Links


On markdown reports, bug ids will be links
if the --bugid-link parameter is provided. 


$ tartare --reporter gherkin-md --bugid-link "http://bugtrackingsystem/%s" --recursive ./acceptance

Tags

Tagging features/scenarios/variants



feature('Addition', function() {
  var dataset = [
    { tag: 'tag4', desc: '50 + 70 = 120', num1: 50, num2: 70, result: 120 },
    { tag: 'tag5', desc: '100 + 90 = 190', num1: 100, num2: 90, result: 190 }
  ];
  scenario('Add two numbers', dataset, function(variant) { }).tag('tag2');
  scenario('Add two negative numbers', function() { }).tag('tag3', 'tagA');
}).tag('tag1');

Tags

Filtering test execution



# Run tests having tag4 and tag3
$ tartare --reporter gherkin --filter "+tag4,+tag3" --recursive ./acceptance
# Run tests having tag2 and not having tag5
$ tartare --reporter gherkin --filter "+tag2,-tag5" --recursive ./acceptance
# Run tests having tag2 or tagA
$ tartare --reporter gherkin --filter "+tag2|-tagA" --recursive ./acceptance
# Run tests having tag4 or having tag3 and tagA
$ tartare --reporter gherkin --filter "+tag4|(+tag3&+tagA)" --recursive ./acceptance

'manual' and 'bug' are reserved tags that are set by Tartare when using the .manual modifier or minorBug/majorBug methods.

Before / After hooks



var dataset = [
  { desc: '50 + 70 = 120', num1: 50, num2: 70, result: 120 },
  { desc: '100 + 90 = 190', num1: 100, num2: 90, result: 190 },
];

feature('Addition', function() {
  scenario('Add two numbers', dataset, function(variant) {
    beforeEachVariant(function() {
      clearScreen();
    });
    given('I have entered ' + variant.num1 + ' into the calculator', function() { });
    and('I have entered ' + variant.num2 + ' into the calculator', function() { });
    when('I press add', function() { });
    then('the result should be ' + variant.result + ' on the screen', function() { });
  });
});

Before / After hooks



feature('Addition', function() {
  beforeFeature(function() {
    switchOnCalculator();
  });
  scenario('Add two numbers', function() {
    given('I have entered 50 into the calculator', function() { });
    and('I have entered 70 into the calculator', function() { });
    when('I press add', function() { });
    then('the result should be 120 on the screen', function() { });
  });
});

Sync vs Async


Node.js code is essentially asynchronous

Though many tests look better using synchronous code

Sync code


Writing sync code is straightforward



  scenario('Add two numbers', function() {
    given('I have entered 50 into the calculator', function() {
      num1 = 50;
    });
    and('I have entered 70 into the calculator', function() { });
    when('I press add', function() { });
    then('the result should be 120 on the screen', function() { });
  });

Converting Async into Sync


This module has steps implementation (using async code)


var fs = require('fs');

module.exports = {
  readConf: function readConf(cb) {
    fs.readFile('config.json', function(err, data) {
      if (err) {
        return cb(err);
      }
      cb(null, JSON.parse(data));
    });
  }
};

Converting Async into Sync


Without Tartare


var steps = require('./steps');

describe('Configuration', function() {
  describe('Read configuration', function() {
    var config = null;
    [...]
    it('I read the config file', function(done) {
      steps.readConf(function(err, cfg) {
        config = cfg;
        done();
      });
    });
    [...]
  });
});

Converting Async into Sync


Now we can use async functions as if it were sync


var tartare = require('tartare');
var steps = require('./steps');

tartare.synchronize(steps);

feature('Configuration', function() {
  scenario('Read configuration', function() {
    var config = null;
    [...]
    when('I read the config file', function() {
      config = steps.readConf();
    });
    [...]
  });
});

What if I need to use async code?


  1. Add a callback (usually named done) to the step keyword
  2. Invoke that callback when you test is complete

var tartare = require('tartare');
var steps = require('./steps');

feature('Configuration', function() {
  scenario('Read configuration', function() {
    var config = null;
    [...]
    when('I read the config file', function(done) {
      steps.readConf(function(err, cfg) {
        config = cfg;
        done();
      });
    });
    [...]
  });
});

It also works with before and after hooks

Only


You can use the .only version
of feature and scenario keywords


feature.only('Addition', function() {
  scenario.only('Add two numbers', function() {
    [...]
  });
});

It also works with variants including
the field only set to a truthy value


var dataset = [
  { only: true, desc: '50 + 70 = 120', num1: 50, num2: 70, result: 120 },
  { desc: '100 + 90 = 190', num1: 100, num2: 90, result: 190 },
];

TARTARE AS A QA LIBRARY

Chai Plugins

Chai is a BDD assertion library that can be
paired with Mocha (and Tartare) and can be
extended with our own assertions 



expect('test').to.be.a('string');
expect([1,2,3]).to.include(2);
expect(var).to.be.null;
expect([]).to.be.empty;
expect('foo').to.have.length.above(2);
expect(7).to.be.within(5,10);
expect({ foo: 1, bar: 2 }).to.have.keys(['foo', 'bar']);

Tartare Chai Plugins

HTTP Chai plugins


expect(req).to.have.httpMethod('POST');
expect(res).to.have.httpStatusCode(200);
expect(res).to.have.httpHeaders(['content-type', 'x-forwarded-for']);
expect(req).to.have.httpHeaders({
  'content-type': 'application/json',
  'x-forwarded-for': '127.0.0.1'
}]);
expect(req).to.have.httpBody(expectedBody);
expect(res).to.be.httpChunked;
expect(res).to.have.httpCharset('utf-8');

Tartare Chai Plugins

UNICA Chai plugins


expect(res).to.be.a.wellFormedJsonApiResponse;
expect(res).to.be.a.wellFormedXMLApiResponse;
expect(res).to.be.a.wellFormedSoap11ApiResponse;
expect(res).to.be.a.jsonApiError(404, 'SVC1001');
expect(res).to.be.an.xmlApiError(404, 'SVC1001', 'v1');

Collections

An HTTP REST client to make easier testing REST APIs


Let's have an API in http://someserver.com/provision/v1 with the following CRUD resources:

  • apps
  • developers
  • products

Collections


Creating an API client based on Tartare collections 


var collections = require('tartare').collections;
var provisionApi = collections.createCollectionsGroup({
  baseUrl: 'http://someserver.com/provision/v1'
});
var applications = provisionApi.createCollection('apps/');

applications.get({ id: '1d1272b3-2bde-470a-8925-0bbd74a8516f' }, function(err, res) {
  if (err) {
    console.log(err);
  } else if (res.statusCode !== 200) {
    console.log('ERROR trying to read an application:', res.statusCode, res.body)));
  } else {
    console.log(JSON.parse(res.body));
  }
});

API mock

A mock server ready to behave as an API server


Start the mock


$ <tartare_root>/bin/apimockserver --admin-port 8080 -port 8000
  • admin-port: port used to configure mock behaviour
  • port: port where the mock expects service requests


It also supports HTTPS and 2waySSL modes

API mock

Configuring the mock



POST /admin/v1 HTTP/1.1
Host: server.com
Content-Type: application/json
Content-Length: 126

{
  "method": "GET",
  "path": "/provision/v1/apps/1d1272b3-2bde-470a-8925-0bbd74a8516f",
  "response": {
    "statusCode": 200,
    "headers": {
      "Content-Type": "application/json",
      "Unica-Correlator": "{{{headers.unica-correlator}}}"
    },
    "body": "[Application details]",
    "delay": 200,
    "chunked": true
  }
}

API mock

Asking for the last request for a pair method-path



GET /admin/v1/lastrequests?method=GET HTTP/1.1
Host: server.com
						

HTTP/1.1 200 OK
Content-Type: application/json
Content-Length: 126

[
  {
    "method": "GET",
    "requestUri": "/provision/v1/apps/1d1272b3-2bde-470a-8925-0bbd74a8516f",
    "path": "/provision/v1/apps/1d1272b3-2bde-470a-8925-0bbd74a8516f",
    "query": null,
    "headers": {
      "Host": "server.com"
      "Unica-Correlator": "13cf5c4f-3670-46e1-9b7f-931c8ef17236"
    },
    "charset": null,
    "chunked": false,
    "connection": { "remoteAddress": "192.168.0.101, "remotePort": 11487 },
    "body": ""
  }
]

API mock

ApiMockAdminClient: making easier mock configuration



var mockAdminClient = tartare.apiMockAdminClient.createClient('localhost', 8080);
mockAdminClient.configs.create({
  method: 'GET',
  path: '/provision/v1/apps/1d1272b3-2bde-470a-8925-0bbd74a8516f',
  response: {
    statusCode: 200,
    headers: {
      'Content-Type': 'application/json',
      'Unica-Correlator': '{{{headers.unica-correlator}}}'
    },
    body: JSON.stringify(myApp),
    delay: 200,
    chunked: true
  }
}, function(err, res) {
});

Server utils


A set of functions to manage servers
configuration, start and stop


  • renderConfigFile: create config files from a mustache template and a config object.
  • [start|stop]Server: start/stop any server. Starting a server is synchronous and it can wait for the to start by timeout, or looking for a string in the stdout/stderr.
  • killServersByTpcPorts: Kill all processes listening to the given ports (supports RHEL, Ubuntu and OSX).
  • [start|stop]ApiMockServer: start/stop the API Mock.

Server utils

renderConfigFile


Config Template


"server": {
  "address": "{{{serverCfg.address}}}",
  "port": {{{serverCfg.port}}}
}

Config Object


var serverCfg = {
  address: 'myserver.com',
  port: 8080
}

Resulting Config


"server": {
  "address": "myserver.com",
  "port": 8080
}

Native Types Extensions



'string'.startsWith('str'); // => true
'string'.endsWith('ing');   // => true
'copyme'.repeat(5);         // => 'copymecopymecopymecopymecopyme'
RegExp.escape('^\(a+)*/$'); // => '\\^\\(a\\+\\)\\*\\/\\$'

THE END