One file example to run data driven tests using python and nose with XML and HTML reports

I was discussing data driven tests with a colleague and how easy it to create in python just with nose test framework.

The file looks like:

def test_dirs():
    for data in test_data():
        yield check_dir, data[0], data[1]

def check_dir(dir_ref, dir_new_gen):
    assert len(dir_ref) > 1
    assert len(dir_new_gen) > 1

def test_data():
    return [["dir1_ref", "dir1_gen"], ["dir2_ref", "dir2_gen"]]

I created a repository  and shared with him following is current content of the Readme:

https://github.com/mubbashir/py_nosetest_data_driven

One file example to run data driven tests using python and nose tests with XML and HTML reports

Documents

• nose test
• Test-generators
• nose-html-output

Optional VENV (Python virtual environment)

• Create virtual environment python -m venv venv
• Activate virtual env
  • Linux/Mac source venv/bin/activate

$ source venv/bin/activate
(venv) [akhan@seq-akh-fc pipeline_workflow_tests] which python
~/workspace/pipeline_workflow_tests/venv/bin/python

Install and execute

• Install dependencies: pip install -r requirements.txt
• Run tests

$nosetests -v
nose_test_generator.test_dirs('dir1_ref', 'dir1_gen') ... ok
nose_test_generator.test_dirs('dir2_ref', 'dir2_gen') ... ok
nose_test_generator.test_data ... ok

----------------------------------------------------------------------
Ran 3 tests in 0.003s

OK

Run with reports

$nosetests -v --with-xunit --xunit-file=nose_test_results.xml --with-html-output --html-out-file=nose_test_results.html
nose_test_generator.test_dirs('dir1_ref', 'dir1_gen') ... ok
nose_test_generator.test_dirs('dir2_ref', 'dir2_gen') ... ok
nose_test_generator.test_data ... ok

----------------------------------------------------------------------
XML: /home/akhan/workspace/py_nosetest_data_driven/nose_test_results.xml
----------------------------------------------------------------------
Ran 3 tests in 0.002s

OK

• XML Report
• HTML Report

Repository: https://github.com/mubbashir/py_nosetest_data_driven

Gradle task to check files/commands in PATH environment variable before running tests

Recently I was debugging a failing tests and it turn out the when Jenkins runs the tests on a remote node shell environment is not initialized hence one of binary dependency of the application under tests was missing.

Solution:
Add the additional location in the PATH from jenkins
e.g. export PATH=$PATH:/opt/bin/

Force checking in gradle:
In the light of principle like ""treat test code with the same level of care as production code" and To help my fellow tester (and my future self) it's better to add this as check in gradle.

// rest of your build
// Verfied with gradle 4.10
task("checkEnv"){
    doFirst {
        def listOfFileToCheckInPath = ['find', 'grep']
        listOfFileToCheckInPath.each { file ->
            if(!isFoundInPath(file))
                throw new GradleException("${file} was not found in any of the folder in PATH: ${System.getenv('PATH').split(File.pathSeparator)}")
        }
    }
}
/**
 * Static function to verify if a file/command exist in PATH environment 
 * @param file
 * @return true if found, else false
 */
def static isFoundInPath( file){
    def PATH_ENV = System.getenv('PATH')
    def fileFound = PATH_ENV.split(File.pathSeparator).find{ folder ->
        println("Looking for ${file} in ${folder}")
        if (Paths.get( "${folder}${File.separator}${file}").toFile().exists()){
            println("Found ${file} in ${folder}")
            return true
        }
    }
    return fileFound
}
// Making test task to depend on checkEnv
test.dependsOn checkEnv

The output would look something like:

./gradlew checkEnv
> Task :checkEnv
Looking for find in /usr/local/bin
Looking for find in /usr/bin
Found find in /usr/bin
Looking for grep in /usr/local/bin
Looking for grep in /usr/bin
Found grep in /usr/bin

BUILD SUCCESSFUL in 0s

Gist on github

Jenkins Script Console and some Groovy Scripts

I am one of those jenkins users who use Jenkins Script Console  [1] extensively. I have been maintaining some of those scripts in a private repository, lately I came across this gist [2] by Damien Nozay which uses a much simpler way to share these scripts i.e. using gists :D
After forking it I have started adding my scripts in gists as well   https://gist.github.com/mubbashir/484903fda934aeea9f30 [3].

I found it quite useful e.g. lately I need to know the results of all the downstream jobs triggered by a certain job:
https://gist.github.com/mubbashir/#file-jobstatustriggeredbyupstreamcause

/ author : Ahmed Mubbashir Khan
// Licensed under MIT
// ---------------------------------------------------------
// This script prints out information of last dwonstream job of Upstream job
// e.g. printInformationOfDownstreamJobs("ChangeListner", 11, "All Tests")
// will print all the downstream jobs invodked by ChangeListner build 11 in the view "All Tests"
// ---------------------------------------------------------

import hudson.model.*
//printInformationOfDownstreamJobs("ChangeListner", 11, "All Tests")

def printInformationOfDownstreamJobs(jobName, buildnumber, viewName){
  def upStreamBuild = Jenkins.getInstance().getItemByFullName(jobName).getBuildByNumber(buildnumber)
  println "${upStreamBuild.fullDisplayName}" +
   "${upStreamBuild.getCause(hudson.model.Cause.UpstreamCause).upstreamRun}"
  def cause_pattern = /.*${jobName}.*${buildnumber}.*/
  println "Cause pattern: ${cause_pattern}"
  // Upated for any builds of the upstream  job in a view from only the last build
  def view = Hudson.instance.getView(viewName)
  def buildsByCause = []
   // For each item in the view 
  view.getItems().each{ 
    def jobBuilds=it.getBuilds() // get all the builds 
    jobsBuildsByCause = jobBuilds.findAll { build ->    
    build != null &&
    build.getCause(hudson.model.Cause.UpstreamCause)!= null &&
    build.getCause(hudson.model.Cause.UpstreamCause).upstreamRun==~cause_pattern
    }
    buildsByCause.addAll(jobsBuildsByCause)
  }
   // printing information 
  buildsByCause.each{ d_build->
   // def d_build = job.lastBuild
    println("Build: ${d_build.fullDisplayName}->"+
     "result:${d_build.result}->${d_build.buildStatusSummary.message}, " +
     "(was triggered by:${d_build.getCause(hudson.model.Cause.UpstreamCause).upstreamRun})" )
  }
}




Or information about the builds in a new e.g. node on  which they were executed along with the time they took:
https://gist.github.com/mubbashir/#file-jobs-in-view-with-duration-label-groovy 

// author : Ahmed Mubbashir Khan
// ---------------------------------------------------------
// This script goes through all the jobs in a view, filters succesful and failed jobs seprately
// Then prints outs them along with the time they took
// ---------------------------------------------------------
import hudson.model.*
def str_view = "Pipeline Tests"
def view = Hudson.instance.getView(str_view)
def successfulJobs = view.getItems().findAll{job -> job.lastBuild != null && job.lastBuild.result == hudson.model.Result.SUCCESS}
def faildJobs = view.getItems().findAll{job -> job.lastBuild != null && job.lastBuild.result == hudson.model.Result.FAILURE}
def disabledJob = view.getItems().findAll{job -> job.disabled == true}
def enabledJob = view.getItems().findAll{job -> job.disabled != true}
println "Total jobs: " + view.getItems().size +" Successful: " +successfulJobs.size+
  " Failed: " + faildJobs.size + " Enabled jobs: " +enabledJob.size + " Disabled jobs: " +disabledJob.size 
println "Current Successful job:"
successfulJobs.each{job -> printInfo(job)}
println "Current Fail job:"
faildJobs.each{job -> printInfo(job)}
println "Current disabled job:"
disabledJob.each{job -> printInfo(job)}
println "Current enabled job:"
enabledJob.each{job -> printInfo(job)}

def printInfo(job){
  println "Job: ${job.name} build on ${job.getAssignedLabelString()}, "+
    "took ${job.lastBuild.getDurationString()} to build, is disabled : ${job.disabled}"
}

[1] https://wiki.jenkins.io/display/JENKINS/Jenkins+Script+Console
[2] https://gist.github.com/dnozay/e7afcf7a7dd8f73a4e05
[3] https://gist.github.com/mubbashir/484903fda934aeea9f30 

Update:
Updated https://gist.github.com/mubbashir/#file-jobstatustriggeredbyupstreamcause to list any downstream job which matches and upstream cause

Interacitve shell for php-webdriver to debug selenium

I do most of my selenium (actually locator’s) debugging in chrome console, $() for css selectors and and $x() for xpath selectors but when it actually comes to debugging selenium/webdriver api calls I prefer interactive shell of programming language (python, ruby and php and pretty decent built in shells)

Bellow is how you can configure php shell to play around with selenium

Installation

• get composer Composer (If you don’t already use Composer)

curl -sS https://getcomposer.org/installer | php
mv composer.phar /usr/local/bin/composer

• get psysh A runtime developer console, interactive debugger and REPL for PHP

wget psysh.org/psysh
chmod +x psysh
./psysh

Then install the library:

php composer.phar require facebook/webdriver

All you need as the server for this client is the selenium-server-standalone-#.jar file provided here: http://selenium-release.storage.googleapis.com/index.html

Download and run that file, replacing # with the current server version.

java -jar selenium-server-standalone-#.jar

// author : Ahmed Mubbashir Khan
require_once 'vendor/autoload.php'; //composer generated autoload file
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
use Facebook\WebDriver\WebDriverBy;

$host = 'http://192.168.99.100:4444/wd/hub';
$driver = RemoteWebDriver::create($host, DesiredCapabilities::firefox());
/* 
Or  if you wan't to modify $capabilities
use Facebook\WebDriver\Remote\WebDriverCapabilityType;
$capabilities = array(\WebDriverCapabilityType::BROWSER_NAME => 'firefox');
$driver = RemoteWebDriver::create($host, $capabilities);
*/
//e.g. Mouse over on an element
$element = $driver->findElement(WebDriverBy::id('some_id'));
$driver->getMouse()->mouseMove( $element->getCoordinates() );

Reference:
 - [1]: http://selenium-release.storage.googleapis.com/index.html
 - [2]: https://github.com/facebook/php-webdriver
 - [3]: https://getcomposer.org/
-  [4]: http://psysh.org/

Jenkins: List all the jobs for which SCM is not currently configured

Put the following in  Jenkins Script Console  to  list all the jobs for which SCM is not currently configured:

// Licensed under MIT
// author : Ahmed Mubbashir Khan
// ---------------------------------------------------------
// This script goes through all the jobs and checks if they configured SCM is hudson.scm.NullSCM
// if they are, then prints it's info
// ---------------------------------------------------------
 

counter = 0
jobs = Jenkins.instance.getAllItems()
for (job in jobs) {
  if (job.scm instanceof hudson.scm.NullSCM){
    println "Job= '${counter++}' '${job.name}' scm '${job.scm}'"
  }
}

SeConf 2015 - Portland

Selenium, Jbehave , gradle and Serenity - Perfect combination for automating BDD styled checks in Java

I have just created this demo project using Selenium, gradle, Jbehave and Serenity

https://github.com/mubbashir/jbehave-selenium-bdd

Why add Serenity?

Serenity adds plugins to Jbehave and makes it a lot easier to write acceptance tests as mentioned on it site Serenity BDD helps you write cleaner and more maintainable automated acceptance and regression tests faster.
The other things I loved about Serenity is the it has wonderful reports  for BDD "Serenity also uses the test results to produce illustrated, narrative reports that document and describe what your application does and how it works."
See:

Structure

├── ReadMe.md  ## ReadME file of cource ;)
├── build.gradle ## gradle build file 
├── gradle
│   └── wrapper
│       ├── gradle-wrapper.jar
│       └── gradle-wrapper.properties
├── gradlew ## grdadle warapper
├── src
│   └── test
│       ├── java
│       │   └── test
│       │       ├── WebTest.java ## This is where the tests resides which add all the steps using net.serenitybdd.jbehave.SerenityStepFactory
│       │       └── selenium
│       │           ├── page ## This where all the pages extending net.serenitybdd.core.pages.PageObject
│       │           │   ├── google
│       │           │   │   └── GoogleLanding.java
│       │           │   └── interfaces
│       │           │       └── Landing.java
│       │           └── steps ## Pages are used in steps 
│       │               └── GoogleSteps.java
│       └── resources
│           └── stories
│               └── google
│                   ├── Search2.story
│                   └── search.story

Refrences

• [jbehave]: http://jbehave.org/
• [gradle]: http://gradle.org/
• [serenity]: http://thucydides.info/docs/serenity/#first-steps