Get IP of Grid node when using Selenium2Library & Robot Framework

Sometimes it is useful to know which machine is executing a Selenium test when using Selenium Grid.

The following code was adapted from a post written in Java. I converted it into a keyword for Robot Framework and added code to grab the session ID from Selenium2Library.

 import urllib2, json  
 from robot.libraries.BuiltIn import BuiltIn  
 from robot.api import logger  
 from urlparse import urlparse  

 def get_node_hostname(hub_url):  
    '''Returns the hostname/IP of the node executing the session belonging to the active browser.'''  
     
    s2l = BuiltIn().get_library_instance('Selenium2Library')  
    session_id = s2l._current_browser().session_id  
    parse_result = urlparse(hub_url)  
    host = parse_result.hostname  
    port = parse_result.port  
    query_url = 'http://%s:%s/grid/api/testsession?session=%s' % (host, port, session_id)  
    req = urllib2.Request(url=query_url)  
    resp = urllib2.urlopen(req).read()  
    json_blob = json.loads(resp)  
    if 'proxyId' in json_blob:  
        proxy_id = json_blob['proxyId']  
        print '*INFO* proxyId is %s' % proxy_id  
        parse_result = urlparse(proxy_id)  
        return parse_result.hostname  
    else:  
        raise RuntimeError('Failed to get hostname. Is Selenium running locally? hub response: %s' % resp)

Using the Copy Artifact Plugin in a system Groovy script

When Groovy Plugin for Jenkins added build context variables (build, launcher, listener) to the script environment in version 1.13, it gave users the power to do great things in a Groovy script. We have a build that has numerous builds that are triggered by a parent job. Some jobs may not be triggered on every build. The nicest solution was to iterate over the projects that were built and copy their artifacts so that the test results could be aggregated.

One of the difficulties in writing this was dealing with Groovy's dispatch. Jenkins has a compatibility layer to keep older plugins working with a newer version of Jenkins. When written as you would write the code in Java, Groovy selected the deprecated overload of perform regardless of the argument types used in the call. All the deprecated overload did was throw an exception. Groovy looks at the run-time type of arguments and does not care about the declared or reference type of the argument you are using. In this case, Groovy's logic made the deprecated method appear to be the best choice when there were multiple overloads with compatible parameters. Casting did not help here.

This was overcome by using reflection to manually pick the newer method. It is ugly but it works.

Below you can see an example of a system Groovy Script that repeatedly invokes the Copy Artifact Plugin to aggregate the artifacts of jobs triggered with the Parameterized Trigger Plugin. The list of jobs that were built is stored as a comma-delimited list in the build variable PROJS prior to executing this.

 // copy the output from all the triggered jobs by invoking the Copy Artifact plugin for the  
 // builds that were triggered  
 // Assumptions: test output is in test-output folder, output and report files are  
 // named output.xml and report.html  
 import hudson.plugins.copyartifact.SpecificBuildSelector  
 import hudson.plugins.copyartifact.CopyArtifact  
 import hudson.model.AbstractBuild  
 import hudson.Launcher  
 import hudson.model.BuildListener  
 import hudson.FilePath  
   
 def envVars= build.getEnvironment()  
 def projs = envVars.get("PROJS")  
   
 projs.split(",").each {  
   println "Fetching output from " + it + "..."  
   copyTriggeredResults(it)  
 }  
   
 def copyTriggeredResults(projName) {  
   def buildNbr = build.getEnvironment().get("TRIGGERED_BUILD_NUMBER_" + projName)  
   def selector = new SpecificBuildSelector(buildNbr)  
   
   // CopyArtifact(String projectName, String parameters, BuildSelector selector,  
   // String filter, String target, boolean flatten, boolean optional)  
   def copyArtifact = new CopyArtifact(projName, selector, "**/*.*", null, false, false)  
   
   // use reflection because direct call invokes deprecated method  
   // perform(Build<?, ?> build, Launcher launcher, BuildListener listener)  
   def perform = copyArtifact.class.getMethod("perform", AbstractBuild, Launcher, BuildListener)  
   perform.invoke(copyArtifact, build, launcher, listener)  
   
   // rename test output to be output_${project name}.xml  
   def target = new FilePath(build.workspace, "test-output/output_" + projName + ".xml")  
   def source = new FilePath(build.workspace, "test-output/output.xml")  
   source.renameTo(target)  
 }  

Getting Robot Framework Results in the Email from Jenkins

The report and log pages generated by Robot Framework are great, but it is also nice to have an executive summary of the test results. If you use the Robot Framework Plugin and Email-ext plugin for Jenkins, you can get results in the email body that look like this:

Robot Framework Results Detailed Report Pass Percentage: 80.0% Test Name Status Execution Datetime Customer Customer.Ordering Place Credit Card Order PASS Mon Jul 30 16:17:25 CDT 2012 Save to Wishlist PASS Mon Jul 30 16:19:42 CDT 2012 Place Paypal Order PASS Mon Jul 30 16:29:01 CDT 2012 Customer.History View Last Order PASS Mon Jul 30 16:32:00 CDT 2012 Shipping Status FAIL Mon Jul 30 16:38:02 CDT 2012

Here is a snippet from the Groovy template that generated the above:

 <%  
 import java.text.DateFormat  
 import java.text.SimpleDateFormat  
 %>  
 <!-- Robot Framework Results -->  
 <%  
 def robotResults = false  
 def actions = build.actions // List<hudson.model.Action>  
 actions.each() { action ->  
  if( action.class.simpleName.equals("RobotBuildAction") ) { // hudson.plugins.robot.RobotBuildAction  
   robotResults = true %>  
 <p><h4>Robot Framework Results</h4></p>  
 <p><a href="${rooturl}${build.url}robot/report/report.html">Detailed Report</a></p>  
 <p>Pass Percentage: <%= action.overallPassPercentage %>%</p>  
 <table cellspacing="0" cellpadding="4" border="1" align="center">  
 <thead>  
 <tr bgcolor="#F3F3F3">  
  <td><b>Test Name</b></td>  
  <td><b>Status</b></td>  
  <td><b>Execution Datetime</b></td>  
 </tr>  
 </thead>  
 <tbody>  
 <% def suites = action.result.allSuites  
   suites.each() { suite ->   
    def currSuite = suite  
    def suiteName = currSuite.displayName  
    // ignore top 2 elements in the structure as they are placeholders  
    while (currSuite.parent != null && currSuite.parent.parent != null) {  
     currSuite = currSuite.parent  
     suiteName = currSuite.displayName + "." + suiteName  
    } %>  
 <tr><td colspan="3"><b><%= suiteName %></b></td></tr>  
 <%  DateFormat format = new SimpleDateFormat("yyyyMMdd HH:mm:ss.SS")
    def execDateTcPairs = []
    suite.caseResults.each() { tc ->  
      Date execDate = format.parse(tc.starttime)
      execDateTcPairs << [execDate, tc]
    }
    // primary sort execDate, secondary displayName
    execDateTcPairs = execDateTcPairs.sort{ a,b -> a[1].displayName <=> b[1].displayName }
    execDateTcPairs = execDateTcPairs.sort{ a,b -> a[0] <=> b[0] }
    execDateTcPairs.each() {
      def execDate = it[0]
      def tc = it[1]  %>
 <tr>  
  <td><%= tc.displayName %></td>  
  <td style="color: <%= tc.isPassed() ? "#66CC00" : "#FF3333" %>"><%= tc.isPassed() ? "PASS" : "FAIL" %></td>  
  <td><%= execDate %></td>  
 </tr>  
 <%  } // tests  
   } // suites %>  
 </tbody></table><%  
  } // robot results  
 }  
 if (!robotResults) { %>  
 <p>No Robot Framework test results found.</p>  
 <%  
 } %>  

The date conversion is there to convert the date from the format Robot uses to the Java default format, which is more familiar to us.

This is the first thing I have ever done in Groovy, and I must say it is a pleasant language to write in. I especially like how getters and setters are mapped to the Groovy concept of properties.

Happy roboting.

Multiple Instances of a Remote Test Library in Robot Framework

One big shortcoming with the remote library API is that is has no mechanism for distinguishing between different instances of Robot Framework running either locally or remotely (i.e. it is a session-less protocol). If the library has any kind of state information, then two callers cannot reliably share the same instance of the library. This is clearly a problem when two tests are executing concurrently that both use the same library.

I have devised a simple solution that applies to execution by Jenkins if there is only one instance of Robot Framework running per executor. Jenkins defines many environmental variables that can be useful, including EXECUTOR_NUMBER, which is number of the executor running the current job. We can use this as an offset to a base port to be assured that each job will have its own instance of the remote library. For example, using a base port of 4000, the Robot Framework instance in executor #3 would use port 4003. The library must be parameterized so that the port can be chosen on startup. Here is some code showing how to start a library on the correct port prior to running Robot Framework in a Windows environment.

 set MY_REMOTE_LIBRARY_PORT=4000  
 if defined EXECUTOR_NUMBER set /a MY_REMOTE_LIBRARY_PORT=%MY_REMOTE_LIBRARY_PORT% + %EXECUTOR_NUMBER%  
 start java -jar my-remote-library.jar %MY_REMOTE_LIBRARY_PORT%  

When the library is imported, the appropriate port number must be used. You could set the variable as a command line argument to or use a variables file as shown below.

 import os  
   
 def get_variables():  
   port_offset = 0 if not 'EXECUTOR_NUMBER' in os.environ else int(os.environ['EXECUTOR_NUMBER'])  
   return {'MY REMOTE LIBRARY PORT': str(4000 + port_offset)}  

Note that if running outside of Jenkins, the base port is used.
The resource file would look something like this:

*** Settings ***
Variables         remote_library_port.py
Library           Remote    http://127.0.0.1:${MY REMOTE LIBRARY PORT}/

If you run Robot Framework instances in parallel in the same executor, then some other mechanism for assigning port numbers would be required. I cannot think of a simple, robust way to do this and would love to hear if you do.

Let Jenkins visitors know their access is restricted

We use Jenkins in the QA department and expose it to other teams and departments. For new users it is easy to be confused as to why they cannot do certain things, such as start a build, because Jenkins simply hides icons that a visitor does not have access to. In this article I explain how to get a header on every page Jenkins serves that lets them know their access is restricted:



Follow these simple steps:

1. Install the Page Markup Plugin
2. Go to Manage Jenkins > Configure System
3. For Header HTML in the Additional Page HTML Markup section, put:
   

    <div id="my-banner" align="center" style="display:none;">  
    <p /><img src="/userContent/error.png" /><b>You are not logged in. To start a job or make changes you must log in. If you do not have an account, you can <a href="/signup">sign up</a> for one.</b>  
    </div>  
   

   Note that the image referenced here does not come with Jenkins and needs to be put in the userContent folder.
4. For Footer HTML in the Additional Page HTML Markup section, put:
   

    <script type="text/javascript">  
    function loggedIn() {  
        var x;  
        x = document.getElementById('login-field')  
        if(x.innerText) {  
            x = x.innerText;  
        }else{  
            x = x.textContent;  
        }  
        return (x.indexOf('log out') > 0);  
    }  
    if (!loggedIn()) {  
        document.getElementById('my-banner').style.display = 'block';   
    }  
    </script>  
   

5. Hit Save

The header will only display if a visitor is not signed in and Javascript is enabled.