Automating Web Performance Data Collection with Behat and BrowserMob Proxy
30 OctBrowserMob Proxy is a utility which is used for capuring HTTP traffic and performance data from the browser. BrowserMob-Proxy adds in essential missing capabilities such as checking HTTP status codes and injecting headers for HTTP Basic Auth. Web Perfomance data can be manually captured by other tools like Firebug or Developers Tools. Using BrowserMob Proxy we can capture perfonace data in HAR format while running automated tests. There is lots of food to learn about BrowserMob on thier official website.
In this article, we will see how to integrate BrowserMob Proxy with Behat. In order to get started we need to install PHPBrowserMob package.
PHPBrowserMob
$ sudo pear channel-discover element-34.github.com/pear $ sudo pear install -f element-34/PHPBrowserMobProxy $ sudo pear install -f element-34/Requests
Download latest version of the BrowserMob Proxy from the site. You can launch BrowserMob proxy like this
$ cd browsermob-proxy-2.0-beta-6/ $ cd bin $ sh browsermob-proxy -port 9090
You will see something like this:
$ sh browsermob-proxy -port 9090 INFO 10/30 21:29:50 o.b.p.Main - Starting BrowserMob Proxy version 2.0-beta-6 INFO 10/30 21:29:51 o.e.j.u.log - jetty-7.3.0.v20110203 INFO 10/30 21:29:51 o.e.j.u.log - started o.e.j.s.ServletContextHandler{/,null} INFO 10/30 21:29:52 o.e.j.u.log - Started SelectChannelConnector@0.0.0.0:9090
This means, BrowserMob Proxy is running correcrtly.
Download and start selenium version 2.25.0. Note that, proxy won’t work for some version of selenium server. Better Download latest.
$ java -jar selenium-server-standalone-2.25.0.jar
Now, We need to install Behat by following official doc Or just follow my previous post on Behat installtion with composer.
$ sudo mkdir Behat-BrowserMob $ sudo chmod -R 777 Behat-BrowserMob $ cd Behat-BrowserMob/ $ behat --init
Now create a feature file for exporting the performance data in the HAR format.
$ sudo vim BehatBrowserMob.feature Feature: BrowserMob Proxy with Behat In order to check website performance As a automated tester I need to see network traffic captured in the HAR format @javascript Scenario: Behat Bowsermob Given I setup BrowserMobProxy And I Navigate to "http://www.facebook.com/" When I export HAR Then I should see network traffic in the HAR file
You need to include ‘PHPWebDriver‘ and ‘PHPBrowserMobProxy‘
We can implement the feature using following code :
<?php use Behat\Behat\Context\ClosuredContextInterface, Behat\Behat\Context\TranslatedContextInterface, Behat\Behat\Context\BehatContext, Behat\Behat\Exception\PendingException; use Behat\Gherkin\Node\PyStringNode, Behat\Gherkin\Node\TableNode; use Behat\MinkExtension\Context\MinkContext; require_once __DIR__ . '/PHPBrowserMobProxy/Client.php'; require_once __DIR__ . '/php-webdriver/PHPWebDriver/WebDriver.php'; require_once __DIR__ . '/php-webdriver/PHPWebDriver/WebDriverProxy.php'; require_once 'PHPUnit/Framework/Assert/Functions.php'; require_once 'PHPUnit/Autoload.php'; /** * Features context. */ class FeatureContext extends BehatContext { protected static $driver; protected static $BrowserMob; protected static $BrowserMobSession; public $data; /** * Initializes context. * Every scenario gets it's own context object. * * @param array $parameters context parameters (set them up through behat.yml) */ public function __construct(array $parameters) { } /** * @BeforeScenario */ public function cleanup() { $HARFile = "/tmp/BROWSERMOBHAR.php"; if (file_exists($HARFile)) { echo "The file $HARFile exists, I am going to delete it \n"; unlink($HARFile); } } /** * @Given /^I setup BrowserMobProxy$/ */ public function iSetupBrowsermobproxy() { self::$driver = new PHPWebDriver_WebDriver(); self::$BrowserMob = new PHPBrowserMobProxy_Client("localhost:9090"); $additional_capabilities = array(); $webDriverProxy = new PHPWebDriver_WebDriverProxy(); self::$BrowserMob->new_har("google"); $webDriverProxy->httpProxy = self::$BrowserMob->url; $webDriverProxy->add_to_capabilities($additional_capabilities); $this->session = self::$driver->session('firefox', $additional_capabilities); } /** * @Given /^I Navigate to "([^"]*)"$/ */ public function iNavigateTo($url) { $this->session->open($url); } /** * @When /^I export HAR$/ */ public function iExportHar() { file_put_contents("/tmp/BROWSERMOBHAR.php", var_export(self::$BrowserMob->har, true)); } /** * @Then /^I should see network traffic in the HAR file$/ */ public function iShouldSeeNetworkTrafficInTheHarFile() { assertFileExists('/tmp/BROWSERMOBHAR.php'); } }
Now, run ‘behat’ and watch the test running in the Browser. You will find data stored in the ‘/tmp/BROWSERMOBHAR’ file. You can use HAR viewer to see performance data in fancy way !
There are number of possibilities you can use this data. Here is best article which describe use of HAR file.
- Calculate Load time.
- Set/Get HTTP Headers during your tests
- Capture performance data with HAR files.
- Simulate network traffic and latency
- Rewrite HTTP requests and respones
SourcCode on GitHub:
Source Code is available on GitHub repo ‘Behat-BrowserMobProxy‘ . Just give it a try !
Final Thoughts:
Combination of Behat, WebDriver and BrowserMob Proxy can be used for capturing the network trafic while running automated tests. We can use HAR files as per our need eg. Calculate Page load times, Page performance analysis etc etc..
Behat2.4 : Painless installation with Composer
31 AugBehat is a BDD framework for PHP application. Behat has becoming so popular in recent days. Behat has drastic changes in recent days in order to add some cool features into it.
One of the amazing feature is, behat installation with use of Composer. Composer is dependency manager for PHP application which helps us to manage all the third party dependencies at one place and use them properly. Another great feature behat added is ‘Behat Extensions’. You can extend behat by adding extensions of your choice.
In this tutorial, we will see how to upgrade Behat vesion with use of composer . Most of us have been installed Behat with ‘pear’ packages. behat version installed with pear has no longer supported. Read Behat installation guide for further information.
Now, get ready to uninstall old version of behat and install new one. Open your terminal and type in following comands.
Check your existing installation,
Shashi-MacBook-Pro:~ user$ which behat
You will see something like this, if you have installed Behat with pear
Shashi-MacBook-Pro:~ user$ which behat /usr/local/pear/bin/behat
Remove this Behat vesion by typing
Shashi-MacBook-Pro:~ user$ sudo pear uninstall behat/behat Password: uninstall ok: channel://pear.behat.org/behat-2.3.5
Remove ‘Gherkin’ package
Shashi-MacBook-Pro:~ user$ sudo pear uninstall behat/gherkin uninstall ok: channel://pear.behat.org/gherkin-2.1.1
Remove ‘Mink‘ as we are going to use ‘MinkExtension’
Shashi-MacBook-Pro:~ user$ sudo pear uninstall behat/mink uninstall ok: channel://pear.behat.org/mink-1.3.3
MinkExtension creates Mink instance in each sub-context or it could be even used as subcontext on its own.
You can read more about it on MinkExtension documentation.
Behat 2.4 installation
As we have just removed old version of behat, so we won’t find anything in binary when you execute ‘behat’ command
Shashi-MacBook-Pro:~ user$ behat -bash: /usr/local/pear/bin/behat: No such file or directory
Now create new Behat installation Direcory, I have created in /opt/behat like this
Shashi-MacBook-Pro:~ user$ sudo mkdir /opt/behat Shashi-MacBook-Pro:~ user$ cd /opt/behat/ Shashi-MacBook-Pro:behat user$ sudo chmod 777 /opt/behat/
Create ‘composer.json’ file with all the required dependencies like this
Shashi-MacBook-Pro:behat user$ sudo vi composer.json
Insert following dependencies in the json file, you can add more
{ "require": { "behat/behat": "2.4.*@stable", "behat/mink": "1.4@stable", "behat/mink-goutte-driver": "*", "behat/mink-selenium-driver": "*", "behat/mink-selenium2-driver": "*", "behat/mink-extension": "*", "behat/mink-sahi-driver": "*" }, "minimum-stability": "dev", "config": { "bin-dir": "bin/" } }
Note that we have included ‘Mink Extension’ as well in the dependencies. Now download the composer and intstall dependencies.
Shashi-MacBook-Pro:behat user$ ls composer.json
Shashi-MacBook-Pro:behat user$ curl http://getcomposer.org/installer | php % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 11038 0 11038 0 0 88692 0 --:--:-- --:--:-- --:--:-- 307k #!/usr/bin/env php All settings correct for using Composer Downloading... Composer successfully installed to: /opt/behat/composer.phar Use it: php composer.phar
This will download ‘composer.phar’ file. Check it
Shashi-MacBook-Pro:behat user$ ls composer.json composer.phar
Now install it using
php composer.phar install
This will download all the dependencies. You can see it in the ‘Behat’ directory.
Shashi-MacBook-Pro:behat user$ ls bin composer.json composer.lock composer.phar vendor
Now, you are ready to run Behat using ‘bin/behat’ command.
Shashi-MacBook-Pro:behat user$ ./bin/behat
You will see something like this in the output. This means you have installed behat successfully !!
Shashi-MacBook-Pro:behat user$ ./bin/behat [RuntimeException] Context class not found. Maybe you have provided wrong or no `bootstrap` path in your behat.yml: http://docs.behat.org/guides/7.config.html#paths behat [--init] [-f|--format="..."] [--out="..."] [--lang="..."] [--[no-]ansi] [--[no-]time] [--[no-]paths] [--[no-]snippets] [--[no-]snippets-paths] [--[no-]multiline] [--[no-]expand] [--story-syntax] [-d|--definitions="..."] [--name="..."] [--tags="..."] [--cache="..."] [--strict] [--dry-run] [--rerun="..."] [--append-snippets] [features]
Check Behat version
Shashi-MacBook-Pro:behat user$ ./bin/behat --version Behat version DEV
Make Behat Global
Now Create a global symlink to access behat from anywhere
Shashi-MacBook-Pro:behat user$ sudo ln -s /opt/behat/bin/behat /usr/local/bin/behat
You can run behat from anywhere. It completes Behat2.4 installation process.
Note: You may need to restart your terminal session but not nessessarily.
Shashi-MacBook-Pro:bin user$ which behat /usr/local/bin/behat
Enjoy New Behat !!
Enjoy ‘MinkExtension’ for Behat
7 May‘
Note : This post may not work with recent Behat. Please use and my Github repo for to get it working without any issue https://github.com/Shashikant86/BehatDemo
MinkExtension‘ has been just released which has additional services for Behat. This extension has ‘Mink’ instance for your ‘FeatureContext’ and ‘SubContext’. UI testers would be happy now, as they don’t need to create Mink instance every time in order to use Mink API’s.
Well, in order to use ‘MinkExtension’ you need latest Behat and Mink versions (Behat 2.4+ and Mink 1.4+). Don’t worry so much about the versions, creator of Behat has taken care of all this with composer.
Ok. Let’s clone the MinkExtenion-Example repository from GitHub. See, everything is mentioned in README file.
git clone https://github.com/Behat/MinkExtension-example.git Cloning into MinkExtension-example... remote: Counting objects: 31, done. remote: Compressing objects: 100% (16/16), done. remote: Total 31 (delta 13), reused 31 (delta 13) Receiving objects: 100% (31/31), 4.43 KiB, done. Resolving deltas: 100% (13/13), done.
Navigate to directory & Get the composer and Install dependencies
cd MinkExtension-example curl http://getcomposer.org/installer | php
moonstar:MinkExtension-example sjagtap$ curl http://getcomposer.org/installer | php % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 7438 100 7438 0 0 52386 0 --:--:-- --:--:-- --:--:-- 105k #!/usr/bin/env php All settings correct for using Composer Downloading... Composer successfully installed to: /Users/sjagtap/MinkExtension-example/composer.phar Use it: php composer.phar
Install Dependencies
php composer.phar install
oonstar:MinkExtensionTest sjagtap$ php composer.phar install Installing dependencies - Package symfony/yaml (dev-master) Cloning 48fcffece8de2ade6f6dbc909f22444356b7659f - Package symfony/translation (dev-master) Cloning 34b64346b24024d54f42877b7c34ccc05ff1eaa8 - Package symfony/event-dispatcher (dev-master) Cloning 0c1ae4898196f5e96b79028d8d2f35de4b584659 - Package symfony/dependency-injection (dev-master) Cloning b01877ddbde08c1eded2502a6fdce8dbde89bd35 - Package symfony/config (dev-master) Cloning 2f2fb45da8100bc03d2d9cbcd75e3ff36e325b89 - Package symfony/finder (2.0.x-dev) Cloning v2.0.13 - Package symfony/css-selector (2.0.x-dev) Cloning v2.0.12 - Package symfony/dom-crawler (2.0.x-dev) Cloning v2.0.13 - Package symfony/browser-kit (2.0.x-dev) Cloning v2.0.12 - Package symfony/process (dev-master) Cloning 1bc1398d51180fd2aeded9c44f4d7b73bea817cd - Package symfony/console (dev-master) Cloning 8d58af16c425977018293eb0a1570c9629b37994 - Package behat/gherkin (2.2.x-dev) Cloning ada4bda5823ffe1df8228e9dddefa38d6420f983 - Package zendframework/zend-registry (2.0.0beta3) Downloading: 100% - Package zendframework/zend-loader (2.0.0beta3) Downloading: 100% - Package zendframework/zend-stdlib (2.0.0beta3) Downloading: 100% - Package zendframework/zend-validator (2.0.0beta3) Downloading: 100% - Package zendframework/zend-http (2.0.0beta3) Downloading: 100% - Package zendframework/zend-uri (2.0.0beta3) Downloading: 100% - Package facebook/php-webdriver (master-dev) Cloning master - Package fabpot/goutte (master-dev) Cloning master - Package behat/mink-extension (dev-master) Cloning cd8eb2c85859f7d5e88e5787d4e171eaa52002eb - Package behat/mink (v1.4.0beta2) Downloading: 100% - Package behat/behat (v2.4.0beta3) Downloading: 100% symfony/event-dispatcher suggests installing symfony/http-kernel (dev-master) behat/mink suggests installing behat/sahi-client (Sahi-Client is needed to use the SahiDriver) behat/mink suggests installing alexandresalome/php-selenium (Php-Selenium is needed to use the SeleniumDriver) behat/behat suggests installing behat/yii-extension (for integration of with Yii framework) Writing lock file Generating autoload files
Now Run Behat command,
./bin/behat
Look, if you still using PHP5.3.x Then you will get
moonstar:MinkExtension-example sjagtap$ ./bin/behat PHP Parse error: syntax error, unexpected T_USE, expecting T_FUNCTION in /Users/sjagtap/MinkExtension-example/features/bootstrap/TraitedFeatureContext.php on line 5 PHP Stack trace: PHP 1. {main}() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/bin/behat:0 PHP 2. Symfony\Component\Console\Application-&gt;run() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/bin/behat:27 PHP 3. Behat\Behat\Console\BehatApplication-&gt;doRun() /Users/sjagtap/MinkExtension-example/vendor/symfony/console/Symfony/Component/Console/Application.php:106 PHP 4. Symfony\Component\Console\Application-&gt;doRun() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/src/Behat/Behat/Console/BehatApplication.php:93 PHP 5. Symfony\Component\Console\Command\Command-&gt;run() /Users/sjagtap/MinkExtension-example/vendor/symfony/console/Symfony/Component/Console/Application.php:193 PHP 6. Behat\Behat\Console\Command\BaseCommand-&gt;initialize() /Users/sjagtap/MinkExtension-example/vendor/symfony/console/Symfony/Component/Console/Command/Command.php:225 PHP 7. Behat\Behat\Console\Processor\AggregateProcessor-&gt;process() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/src/Behat/Behat/Console/Command/BaseCommand.php:55 PHP 8. Behat\Behat\Console\Processor\LocatorProcessor-&gt;process() /Users/sjagtap/MinkExtension-example/vendor/behat/behat/src/Behat/Behat/Console/Processor/AggregateProcessor.php:57
This is time to update your PHP version. If you still want run features with same PHP version, then comment out/ remove everything from ‘TraitedFeatureContext.php’ which is for PHP5.4
Now, Start Selenium server & Run behat with ‘inheritance’ or ‘subcontext’ profile.
./bin/behat -p=inheritance
./bin/behat -p=subcontexts
You will see test running with headless as well as with browser. You will see output like this
moonstar:MinkExtension-example sjagtap$ ./bin/behat -p=subcontexts Feature: Search In order to see a word definition As a website user I need to be able to search for a word Scenario: Searching for a page that does exist # features/search.feature:6 Given I am on &quot;/wiki/Main_Page&quot; # Behat\MinkExtension\Context\MinkContext::visit() When I fill in &quot;search&quot; with &quot;Behavior Driven Development&quot; # Behat\MinkExtension\Context\MinkContext::fillField() And I press &quot;searchButton&quot; # Behat\MinkExtension\Context\MinkContext::pressButton() Then I should see &quot;agile software development&quot; # Behat\MinkExtension\Context\MinkContext::assertPageContainsText() Scenario: Searching for a page that does NOT exist # features/search.feature:12 Given I am on &quot;/wiki/Main_Page&quot; # Behat\MinkExtension\Context\MinkContext::visit() When I fill in &quot;search&quot; with &quot;Glory Driven Development&quot; # Behat\MinkExtension\Context\MinkContext::fillField() And I press &quot;searchButton&quot; # Behat\MinkExtension\Context\MinkContext::pressButton() Then I should see &quot;Search results&quot; # Behat\MinkExtension\Context\MinkContext::assertPageContainsText() @javascript Scenario: Searching for a page with autocompletion # features/search.feature:19 Given I am on &quot;/wiki/Main_Page&quot; # Behat\MinkExtension\Context\MinkContext::visit() When I fill in &quot;search&quot; with &quot;Behavior Driv&quot; # Behat\MinkExtension\Context\MinkContext::fillField() And I wait for the suggestion box to appear # SubcontextedFeatureContext::iWaitForTheSuggestionBoxToAppear() Then I should see &quot;Behavior Driven Development&quot; # Behat\MinkExtension\Context\MinkContext::assertPageContainsText() 3 scenarios (3 passed) 12 steps (12 passed) 0m14.59s
What’s new for Testers in MinkExtension?
- You will get Mink instance in your ‘FeatureContext’ as well as ‘SubContexts’ classes.
- You can extend Base Contextfile to MinkContext as shown in below. For other context you need to extend from RawMinkContext
class MyContext extends Behat\MinkExtension\Context\MinkContext
- You can use MinkExtension as a SubContext in your files like this:
class SubcontextedFeatureContext extends Behat\Behat\Context\BehatContext { public function __construct() { $this->useContext('mink', new Behat\MinkExtension\Context\MinkContext); }
This extension is a base for future extensions. We can extend Behat/Mink with any other services testing services like SauceLabs and many more. This feature will definitely useful for Web Testers using Behat and Mink for browser automation. SauceLabs extension is on it’s way.. stay tuned 🙂
Happy BDD !!!
Adding More Sauce To Behat
24 Apr
Note : This post may not work with recent Behat. Please use and my Github repo for to get it working without any issue https://github.com/Shashikant86/BehatDemo
Further to my previous post regarding ‘Behat-Sauce’ integration, ‘Adding Sauce To Behat‘, I am glad to know that there is another Behat and SauceLabs integration from Laura Beth. Thanks a lot Laura for that excellent effort.
On the occasion of Selenium Conference, I met with Noah Sussman talking about “Selenium In the Enterprise: What Went Right, What Went Wrong (So Far)” and one of his slides indicated that, team using Behat and Mink for the functional testing at Etsy. Noah then introduced me to Laura Beth who setup Behat/Mink and SauceLabs integration. Behat-Sauce configuration by Laura Beth is very easy to use.
Now, we will see this in action.
Behat Installation
Create a ‘Demo’ directory and Clone the repository from Laura Beth’s GitHub account
moonstar:~ sjagtap$ mkdir Demo moonstar:~ sjagtap$ cd Demo moonstar:Demo sjagtap$ git clone https://github.com/elblinkin/Behat-Sauce.git Cloning into Behat-Sauce... remote: Counting objects: 178, done. remote: Compressing objects: 100% (105/105), done. remote: Total 178 (delta 50), reused 166 (delta 38) Receiving objects: 100% (178/178), 58.74 MiB | 503 KiB/s, done. Resolving deltas: 100% (50/50), done.
Ok, you got all the code in ‘Behat-Sauce’ directory. Now you need to download dependencies for Behat. Navigate to Behat-Sauce and download composer.
moonstar:Behat-Sauce sjagtap$ wget http://getcomposer.org/composer.phar % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 442k 100 442k 0 0 988k 0 --:--:-- --:--:-- --:--:-- 1436k
Now, update ‘composer.phar’ file. It will download all dependencies.
moonstar:Behat-Sauce sjagtap$ php composer.phar update Updating dependencies - Package symfony/yaml (dev-master) Cloning 52807f723e8a9743852c2105f11bd01c5af7f977 - Package symfony/translation (dev-master) Cloning cac219a6dee410df41ac5dae15553313f48f1fd1 - Package symfony/event-dispatcher (dev-master) Cloning b98d68d3b8513c62d35504570f09e9d3dc33d083 - Package symfony/dependency-injection (dev-master) Cloning 61e664ab2eead27d83ebb45d2bf83dd55ef509a8 - Package symfony/config (dev-master) Cloning 175adaf7dbed4ad5350e75a4fd3de39480df6cf3 - Package symfony/console (dev-master) Cloning f354e1e5d59261ae03c0fc768bfa4043c0e6ab9c - Package symfony/process (dev-master) Cloning 3f95a0126588f00d70e745c1ef96631fb10c3a8e - Package symfony/css-selector (dev-master) Cloning dd695aadd80d8e6e726a7a050535c04325cefa02 - Package symfony/dom-crawler (dev-master) Cloning 69b857977d96e4c726be5524fa00370aef5e9b20 - Package symfony/browser-kit (dev-master) Cloning f03173e232da5d84eb799cad5d57354e6541b0f8 - Package symfony/finder (dev-master) Cloning 5b8658e52eaf424fe943a5857e2404ce95c3b90e - Package behat/mink (dev-master) Cloning v1.3.3 - Package behat/behat (dev-master) Cloning d8ff3fe0f8c44c664c5eebbb6b972bbdff70c166 - Package alexandresalome/php-selenium (master-dev) Cloning master - Package behat/gherkin (v2.1.1) Downloading: 100% symfony/translation suggests installing symfony/config (dev-master) symfony/translation suggests installing symfony/yaml (dev-master) symfony/event-dispatcher suggests installing symfony/dependency-injection (dev-master) symfony/event-dispatcher suggests installing symfony/http-kernel (dev-master) symfony/dependency-injection suggests installing symfony/yaml (dev-master) symfony/dependency-injection suggests installing symfony/config (dev-master) symfony/dom-crawler suggests installing symfony/css-selector (dev-master) symfony/browser-kit suggests installing symfony/process (dev-master) Writing lock file Generating autoload files
You have just installed Behat and all dependencies. Now initialize project by running ‘bin/behat-sauce –init’ command.
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --init +d features - place your *.feature files here +d features/bootstrap - place bootstrap scripts and static files here +f features/bootstrap/FeatureContext.php - place your feature related code here +d config - edit you config settings here +f /Users/sjagtap/Demo/Behat-Sauce/configbehat.yml - place your feature related code here
Once you initialized ‘Behat’. It will ‘features’ and ‘features/bootstrap’ directories and ‘features/bootstrap/FeatureContext.php’ file for you.
Feature file
You can also see ‘config’ directory with ‘behat.yml’ file for you. You need to add SauceLabs ‘Username’ and API key to the ‘behat.yml’.
Now write a simple feature file and save it to ‘features’ directory.
Feature: wikiSearch In order to search information on wiki As a Wiki user I want to get sensible results from site @javascript Scenario Outline: Search Keywords on wiki Given I am on "/" And I fill in searchBox with "<input type="text" />" When I press search button Then I should see "<output>" Examples: | input | output | | London | lʌndən/ |
Step Definition
You need implement step definitions using Mink API’s. You feature context file will look like this now.
<!--?php use Behat\Behat\Context\ClosuredContextInterface, Behat\Behat\Context\TranslatedContextInterface, Behat\Behat\Exception\PendingException; use Behat\Gherkin\Node\PyStringNode, Behat\Gherkin\Node\TableNode; use Behat\Sauce\Context\SauceContext; /** * Features context. */ class FeatureContext extends SauceContext { /** * Initializes context. * Every scenario gets it's own context object. * * @param array $parameters context parameters (set them up through behat.yml) */ public function __construct(array $parameters) { parent::__construct($parameters); } /** * @Given /^I fill in searchBox with "([^"]*)"$/ */ public function iFillInSearchboxWith($input) { $this--->fillField("searchInput",$input); } /** * @When /^I press search button$/ */ public function iPressSearchButton() { $this->getMink()->getSession()->getDriver()->click("//*[@id='searchButton']"); $this->getMink()->getSession()->wait("3000"); } }
Well, now you are ready to run your tests on SauceLabs just by running
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce
You will see test running on the SauceLabs. Once tests finished running, you will see output like this:
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce Feature: wikiSearch In order to search information on wiki As a Wiki user I want to get sensible results from site @javascript Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7 Given I am on "/" # FeatureContext::visit() And I fill in searchBox with "<input type="text" />" # FeatureContext::iFillInSearchboxWith() When I press search button # FeatureContext::iPressSearchButton() Then I should see "<output>" # FeatureContext::assertPageContainsText() Examples: | input | output | | London | lʌndən/ | 1 scenario (1 passed) 4 steps (4 passed) 1m8.949s
You can watch video of the test here
As per Laura Beth, default arguments are:
--browser SauceLabs browser name. Default is: firefox --browser-version SauceLabs browser version. Default is: 7 --os SauceLabs operating system. Default is: Windows 2003
Now let’s run our tests on different browser and different OS:
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --browser="firefox" --browser-version="3.6" --os="Linux"
You will see same output:
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --browser="firefox" --browser-version="3.6" --os="Linux" Feature: wikiSearch In order to search information on wiki As a Wiki user I want to get sensible results from site @javascript Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7 Given I am on "/" # FeatureContext::visit() And I fill in searchBox with "<input type="text" />" # FeatureContext::iFillInSearchboxWith() When I press search button # FeatureContext::iPressSearchButton() Then I should see "<output>" # FeatureContext::assertPageContainsText() Examples: | input | output | | London | lʌndən/ | 1 scenario (1 passed) 4 steps (4 passed) 1m38.828s
Video of the tests can be found here
Sauce-Connect:
You can also start Sauce-Connect tunnel with the same credentials that you provided in config/behat.yml file
bin/behat-sauce --tunnel
You will see something like this:
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --tunnel [-u, Shashikant86, -k, API Key, -d, sauce-connect.proxy, -s, 127.0.0.1, -p, 80, --ssh-port, 443, -b, --rest-url, https://saucelabs.com/rest/v1, --se-port, 4445, --squid-opts, ] * Debug messages will be sent to sauce_connect.log 2012-04-24 17:27:32.354:INFO::jetty-7.x.y-SNAPSHOT 2012-04-24 17:27:32.439:INFO::Started SelectChannelConnector@0.0.0.0:50044 .---------------------------------------------------. | Have questions or need help with Sauce Connect? | | Contact us: http://support.saucelabs.com/forums | | Terms of Service: http://saucelabs.com/tos | ----------------------------------------------------- 2012-04-24 17:27:32,509 - / Starting \ 2012-04-24 17:27:32,513 - Please wait for "You may start your tests" to start your tests. 2012-04-24 17:27:32,526 - Forwarding: ['sauce-connect.proxy']:['80'] -> 127.0.0.1:['50044'] 2012-04-24 17:27:32,538 - Succesfully connected to local server 127.0.0.1:50044 in 6ms 2012-04-24 17:27:33,201 - {"squid_config":[""],"use_caching_proxy":true,"fast_fail_regexps":[""],"ssh_port":443,"metadata":{"PythonVersion":"2.5.1","OwnerHost":"127.0.0.1","Release":"3.0-r18","OwnerPorts":["50044"],"Ports":["80"],"Platform":"Java-1.6.0_26-Java_HotSpot-TM-_64-Bit_Server_VM,_20.1-b02-383,_Apple_Inc.-on-Mac_OS_X-10.7.1-x86_64","Build":"26","ScriptRelease":26,"ScriptName":"sauce_connect"},"use_kgp":true,"domain_names":["sauce-connect.proxy"]} 2012-04-24 17:27:33,486 - Tunnel remote VM is provisioned (a86bd10139b8464081e185b016b6579d) 2012-04-24 17:27:33,756 - Tunnel remote VM is new .. 2012-04-24 17:27:37,068 - Tunnel remote VM is deploying .. 2012-04-24 17:27:40,440 - Tunnel remote VM is booting .. 2012-04-24 17:28:06,611 - Tunnel remote VM is running at maki10148.miso.saucelabs.com 2012-04-24 17:28:06,723 - Succesfully connected to local server 127.0.0.1:50044 in 0ms 2012-04-24 17:28:06,729 - Starting connection to tunnel host... 2012-04-24 17:28:06,732 - Connecting to tunnel host maki10148.miso.saucelabs.com as Shashikant86 2012-04-24 17:28:06,994 - Forwarding Selenium with ephemeral port 50478 2012-04-24 17:28:06.996:INFO::jetty-7.x.y-SNAPSHOT 2012-04-24 17:28:06.998:INFO::Started SelectChannelConnector@0.0.0.0:4445 2012-04-24 17:28:07,000 - Selenium HTTP proxy listening on port 4445 2012-04-24 17:28:07,855 - Successful handshake with Sauce Connect server 2012-04-24 17:28:08,109 - Tunnel host version: 0.1.0, remote endpoint ID: 2851094ae70e4e949a0e96c8ca453748 2012-04-24 17:28:08,112 - Connected! You may start your tests.
Sauce connect is used for testing internal site.
Test Locally:
Well, if you don’t want to waste sauceLabs minutes. You can run your test locally:
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --local
You will see tests running on local machine with Firefox and output is same:
moonstar:Behat-Sauce sjagtap$ ./bin/behat-sauce --local Feature: wikiSearch In order to search information on wiki As a Wiki user I want to get sensible results from site @javascript Scenario Outline: Search Keywords on wiki # features/wikiSearch.feature:7 Given I am on "/" # FeatureContext::visit() And I fill in searchBox with "<input type="text" />" # FeatureContext::iFillInSearchboxWith() When I press search button # FeatureContext::iPressSearchButton() Then I should see "<output>" # FeatureContext::assertPageContainsText() Examples: | input | output | | London | lʌndən/ | 1 scenario (1 passed) 4 steps (4 passed) 0m16.109s
Contribution:
Well, I have forked repository and started implementing full scale SauceLabs grid which support all browsers & OS combination. I am also trying to plug the features on continuous integration server like Jenkins. If you also wish to contribute then fork “Behat-Sauce”
Source code:
Code for the above example can be found “Behat-Sauce-Demo” repository.
Conclusion:
Believe me, this is simply awesome. Thanks again Laura Beth.
Happy BDD !!
SeleniumConf came to London!
23 AprAs mentioned on the Selenium Conference 2012 website, this year conference would be across the pond and it was London. Conference venue ‘IET London‘ was amazing place situated on the bank of Thames river and great view of historical London Eye. I live and work in London so it’s very convenient for me to get down to the location of the conference. I met with with selenium users all across the world and shared thoughts and ideas about open-source testing. Selenium conference started on Monday April,16 and wrapped up on Wednesday, April,18. I spend three memorable days at Selenium conference and learned a lot about new tools, methodologies and approaches of testing applications with Selenium. I would like to take this opportunity to thank SauceLabs and all other sponsors of the Selenium Conference.
Let me see, how can I explain what happened in the conference.
WORKSHOP DAY
It was drizzling, when I came out of the Embankment tube station, and need to walk for 2-3 minutes to get to the venue. It wasn’t difficult to find venue and reception of the Selenium conference. I met with Ashley Wilson, (Customer Development Manager at SauceLabs) at registration desk. Ashley played key role in organizing and managing Selenium Conference. then I didn’t forget to pick a T-Shirt labeled with ‘Selenium Conference 2012’. The breakfast and tea was ready.
It was difficult to choose a workshop as both were interesting. I decided to go for workshop ” Contributing to Selenium” by ‘WebDiver Guy‘ (Simon Stewart) in the morning session.
“Contributing to Selenium” Workshop
The workshop aimed at contribution to selenium open-source project. I was ready with all the code downloaded from googleCode and brand new IntelliJ IDE. I have heard about cool features of the IntelliJ but never tried it before. It is really awesome. Simon already mentioned in technical requirements that ‘ Setting up project in Eclipse is not supported’.
Within the workshop, I had a look at directory structure of the selenium codebase, setup the project in the IntelliJ and also done some code modifications as suggested by Simon. At the middle of the workshop, I was lagging behind while running the JavaScript tests. I ran build with ‘build.desc’ file also ran some test with Firefox driver. Lately, I realised that I was surounded by ‘software engineer in test’ from Google. Lucky me !!
After the break, we started with running tests within IDE. Being new to the ItelliJ, I faced some issues while running tests but managed to get it worked with the help of Simon. At the end of the workshop, someone asked question about ‘How to measure coverage of the Selenium/Functional tests’ and there were interesting discussion around this topic. Being from PHP background, I couldn’t resist myself to ask about ‘PHP Webdriver bindings and current development on PHP bindings’. Simon said ” In terms of PHP, FaceBook-Webdriver bindings are recommended”. Simon then introduced me to Damien Sereni who works in Facebook. At the end of the session. Me and Demien spend some time discussing about future of the PHP webdriver bindings also shared contact details. The lunch was waiting for us then.
After lunch, again I had to make tough decision about selecting workshop. I have decided to go to “If Ikea made instructions for Selenium (or: How to fix bugs in Selenium for yourself)” by Daniel Wagner-Hall, and the Selenium Committers. I really missed the another one “Handmade Etsy Tests” by Stephen Hardisty & Michelle D’Netto. Lately realised that Etsy uses Behat and Mink for functional testing which is interesting topic for me which I missed a lot.
How to fix bugs in Selenium for yourself
In this workshop, Daniel talked about selenium codebase and different drivers. I have already checked out selenium trunk in my system and need to look into Selenium bugs to pick up. Being a tester and having limited development knowledge, I realised that I can’t fix any bug in selenium so just started doing some interesting stuff. I managed to get the tests running on my Andriod phone with the help of Andriod driver. That was enough for me it this session and it was end of first day of the selenium conference.
TUESDAY, APRIL 17
April,17 is my birthday. I reached early on that day in order to register my unconference talk on “Behat and Sauce Labs Integration”. I registered my talk at 10PM on that day. I was very excited to talk on the occasion of my birthday.
Anyway, I saw huge crowd on second day of the conference and met with people from different countries while having tea and breakfast. At 9 PM, we get into the lecture hall, which was huge and amazing. There was one tweet from Frisco Del Rosario (@frdelrosario) saying ” Auditorium at #SeConf had balconies, microphones at each chair, flawless wi-fi. http://twitpic.com/9c30la“
As per the schedule, Jason and Simon introduced sponsors and explained schedule of Track A and Track B talks.
Testing Rapidly Changing Applications With Self-Testing Object-Oriented Selenium Infrastructure
Most people turned on for Track B talk which was on ” Testing Rapidly Changing Applications With Self-Testing Object-Oriented Selenium Infrastructure” by Dan Cuellar. I also headed over to that talk. Slides of his talk are available on here Dan explained how to test rapidly changing applications with just small code change in the test infrastructure. Dan implemented object oriented approach that can handle such a rapid change. As per the Dan if you abstracted the code well then you can build robust automated test scripts. Dan is looking for someone who can help him to extend his work to support iOS. If you are interested to help him then email him at danc@zoosk.com
Web acceptance Testing with Behat
After Dan’s talk, it was my turn to give talk on ‘Web Acceptance Testing with Behat’. There was another Track A talk where Anand Ramdeo was talking about “One Step At a Time” most people turned out there. I was bit disappointed seeing only 10 people attending my talk but still I carried it forward. My talk covered BDD and ATDD with the Behat, Sauce Labs , Jenkins and Relish. Slides of my talk are available here.
So you’re automated… now let’s make it fast!
I have decided to stay at Track B where Sauce Ninja Santiago was about to talk on making test automation fast. The title of his talk was ” So you’re automated… now let’s make it fast!”. Santiago shared some tips to make selenium tests faster like avoiding static waits in the tests or running tests in parallel. At the end of the Santiago’s talk, there are couple of raised which sounds interesting to Santiago. There was a question from Mitch about database integration with selenium tests which was interesting. I asked question about flaky tests that fails in the cloud complaining “element not found” but works well in local machine. As per Santiago, it’s bit tricky to judge random failures in browsers in the cloud.
Using Selenium to Improve a Team’s Development Cycle
After the break, I headed over to lecture hall to hear Mike Devis talking about “Using Selenium to Improve a Team’s Development Cycle“. Mike shared some cool tips about improving quality of the functional tests. Mike also talked about involving developers to write tests. I really like the idea of profiling the tests. Ultimately, it was very pleasant talk with lots of good tips
Geb: Very Groovy Browser Automation
This is one of the best talk from my point of view. Thanks to Luke Daley. This talk was about Geb. Geb is a browser automation framework which combines cool features of WebDriver, jQuery and the Page Objects. Most interesting thing, I found in this talk is $() method. The dollar method can be used to interact with the web pages and accessing web elements. Another interesting thing is integration of Geb with different frameworks like JUnit, TestNG, Spock, Cucumber. At the end of the talk, I asked question on integration of Geb with new Cucumber JVM. Once Luke finished his talk, we went to reception to have a lunch.
Robots: If You Can’t Beat ‘Em, Build ‘Em
Well, This was most interesting demo to watch that how Jason “Teach Robot to play Angry Bird“. Click the link and watch the video inside the presentation, you will be amazed by seeing that how it can be achieved with Selenium. I think, only God and Jason knows how to write code for this. I just enjoyed the demo and applauded at the end of talk.
Building Quality with Foundations of Mud
After Jason’s talk, Kristian Rosenvold was talking about ‘Building Quality’. Kristian explained how to build test environment, test data and Continuous Integration practices in order to build quality products. He again explained some cool tips to handle flaky tests and errors. His slides are explanatory. Kristian made one statement who liked by selenium contributors. It was “Make your dev team live and Breathe the build” At the end of the presentation, I asked question on Tolerance rate of their functional tests. Kristian answered that they can tolerate failing integration tests but not the functional tests. Well, then it was time for quick break then.
Massively Continuous Integration
Continuous Integration is area of my interest so I didn’t want to miss talk on my favorite topic. I headed over to the Track B where Jesse Dowdle & David Tolley were talking on their huge continuous integration system. As per their slides, they have reduced cycle time of running tests from 3 days to 30 minutes. The amount of selenium tests and cycle time to run them over Grid literally amused me. They used Jenkins and amazon web for running the tests. I would rather recommend looking at the slides of the presentation. It was amazing talk.I would like to see demo how they run massive test suite within 30 mins. Thanks to speakers & Jenkins.
How to Test the Inside of Your Head
Speaker Liz Keogh is vary famous agile/BDD coach and international speaker. I was impressed by her talks on BDD at Skills Matter. No wonder her talk at Selenium Conference was one the best. Liz talked about collaboration between developers, testers, BA, project managers and customers. As per Liz “Testers are Evil so let them think before writing code“. Most of people on twitter channel were saying it was brilliant talk.
Well, how can I forget that there was amazing party sponsored by FaceBook at the GlassHouse Store, London. I met with people all around world and had great chat with them. Food and drinks were amazing. Great opportunity to celebrate birthday with unlimited drinks and food. Thanks Facebook.
WEDNESDAY, APRIL 18
Last day of the selenium conference was full of interesting talks, lighting talks and end up with closing keynote session. At the start of the day ‘ Simon Stewart’ kicked off with his talk on status of Selenium projects since last selenium conference
State of the Union
Simon didn’t forget to mention that he is on twitter (@shs96c) at the start of his talk. Simon highlighted recent enhancements of the Selenium project. He also point out an year in a browser and new selenium committers. The most interesting part of his presentation was his analysis of job trends in year, the graph amused everyone.
Simon addressed future development will be on mobile drivers as they are being valuable in recent days. He also expecting new stuff with respect to W3C standard. Let’s see how it will go on in near future. Slide of his talks are available here.
After Simon’s talk I headed over to Track B where “TFT Test Automation Framework” talk was going on. It was all about maintaining and writing automated tests just by using spreadsheets. I really missed “Introducing Selenium Builder – the Future of Test Development” talk by Adam Christian and David Stark. Hope to see videos of the talk soon.
Selenium in the Enterprise: What Went Right, What Went Wrong (So Far)
Noah Sussman was talking about how they used selenium at Etsy. This talk was more about test maintenance and test infrastructure at Etsy. Being a big fan of Behat, I came across one the slide saying “Etsy used Behat & Mink for functional testing”. I was keen to know more about that and asked question on how they used behat in their projects also had a chat after talk with Noah.
Automated Security Testing
Alan Parkinson was talking about ‘Zed Attack Proxy (ZAP) Web-Driver’ for automated security testing. Alan introduced ZAP during his talk and then integrated with build system like ANT. Alan’s demo of ZAP and Jenkins was amazing. I couldn’t resist myself to tweet about that. It’s great approach to perform automated security check on build. I will definitely give a try.
Pluggable Test Infrastructure
As far I remember, I have seen video of last selenium conference Dima talking about Selenium and Cucumber. This year Dima and Jeff comes up with Pluggable Test Infrastructure. They are using Cucumber, Capybara and webdriver at Groupon. Jeff said that using Cucumber is debatable. Jeff & Dima combined API tests, iPhone tests, Sauce tests and Selenium tests with Cucumber. It would be interesting to see demonstration of how they achieved this.
Speed up Your Selenium Tests with PhantomJS, the Headless Browser
Ivan De Marino was then talking about PhantomJS in order to make selenium tests fast. In his talk he showed how you could run selenium tests with PhantomJS and Ghostdriver. Ghostdriver is wire protocol on PhantomJS. One of the tweet was saying Ghostdriver has awesome potential but needs your contribution. If you wish to contribute then fork Github repo now.
Robot Framework
Once Ivan finished his talk on PhantomJS, I have decided to go for Track B where Philip Noetzel was talking about “Robot Framework: Semicolon free testing”. I heard a lot about Robot Framework but didn’t got chance to look into it. This talk was very beneficial for me as Philip shows some cool stuff about robot framework and also shown sample code. Philip also shared his thoughts about pros and cons of using Robot framework. At the end of the presentation we asked Philip if we can use Robot framework for PHP applications.
More Than Automation – How Good Acceptance Tests Can Make Your Team Happier
I was keen to hear Matt Wynne, author of Cucumber book talking about ‘More Than Automation‘. Matt introduced Mortgage Driven Development (MDD) where developers try to make feature completely readable and unmaintainable in order to save your job!. Matt also added concept of “Refuctoring” where developers try to add brittle steps and scenarios in order to make it messy. Well, Matt also said ” Cucumber is a threat” but then audience understood whats intention behind all this idea. As per Matt, Cucumber is cool take which allow everyone to collaborate and write tests. Matt also suggest to read Cucumber book in order to get yourself familiar with Behavior Driven Development (BDD). Matt shared cool tool for living documentation called ‘Relish’. At the end of the talk, there were couple of question around Relish. one of them was asked by me.
Lightning Talks
There were about 10 lightning talks (including mine) scheduled on the board and each one of 5 minutes. Lightning talks was fun. I liked Anand and Komal talked about how to do things wrong. It was nice talk. I am keenly waiting to see videos of Lightning talks.
Closing Keynotes
After the lighting talks there were another session called ‘Closing Keynotes’ where all the selenium committers discuss about the future enhancements and answered questions. There was one interesting question comes up in this session about Selenium 3. Conference was then wrapped up by thanking everyone who attended the conference.
SELENIUM CONFERENCE LIVE WITH #SECONF
I bet, everyone who attended conference had loads of fun and learned new things. Those were was not able to attend the conference, they got live conference updates from twitter hashtag #SeConf. I got a tweet from Michelle Flynn (@MichelleFlynn) saying “Great twitter chat about #SeConf, love how you can follow a conference even if you cannot attend”
Many people all over the world was getting updates on selenium conference every minute. Thanks to organizers for creating such a unique hashtag also thanks to all twitter users who twitted in #SeConf
Interesting things I found.
- Mortgage Driven Development
- Mobile Drivers
- Use of BDD tools with Selenium like Cucumber, SpecFlow, Behat
- Geb: Very groovy browser automation framework
- ‘Atkins’ plug-in for Jenkins
- Robot Framework for BDD and DDD
- Large scale Continuous Integration with Jenkins and Amazon cloud services
CONCLUSION
Selenium conference taught me lot of new things around Selenium also given an opportunity to meet with Selenium users all across the globe which was amazing experience. Great speakers, great learning experience and amazing location. Thanks SauceLabs, Google and all other sponsors. Looking forward next year !
PHPUnit + Behat/Mink + Page Object: The Rockstar Combination of Testing
7 AprLast month, we had discussion about implementing page object pattern in Behat/Mink framework at London Behat Users meetup . Page object pattern is a cool way to make tests maintainable, reusable and readable. Everyone was interested to know more about Page Object Pattern.
In this short tutorial, we will implement Mink and PHPUnit combination for functional testing. Mink and PHPUnit combined with Pageness (Page Object) can be used for maintainable and readable tests. Mink is a web acceptance testing framework for PHP application. Mink has clean API’s which allows browsers emulation and browser controls. Mink has headless browser emulators as well as browser controls which covers all browsers interactions. PHPUnit is testing framework for PHP applications.
Installations
Lets start with some installations assuming PHP, Pear package is already installed.
Install PHPUnit
Now, upgrade pear package & install PHPUnit
$ pear update-channels $ sudo pear channel-discover pear.phpunit.de $ sudo pear channel-discover pear.symfony-project.com $ pear channel-discover components.ez.no $ sudo pear install –alldeps phpunit/PHPUnit $ phpunit --version
Install Behat/Mink
$ pear channel-discover pear.symfony.com $ pear channel-discover pear.behat.org $ pear install behat/mink
Mink is ready to use just including in PHP classes.
require_once 'mink/autoload.php';
PHPUnit-Mink Framework on GitHub
PHPUnit-Mink framework designed to use combination of PHPUnit, Mink and Page Object Pattern to write functional tests with various browser emulators like Goutte, Selenium, Sahi and WebDriver. PHPUnit-Mink Framework has used Mink and PHPUnit to write tests. Driver support for Selenium, Sahi, WebDriver for browser emulation. Test Report Generation which can plugged in Continuous Integration server. Page Objects which can be used directly in tests. Abstracted common elements up in the framework in order to make it maintainable.
How to use:
- Clone GitHub Repository
$git clone git@github.com:Shashi-ibuildings/PHPUnit-Mink.git $cd PHPUnit-Mink
- Start your Driver
Sahi Driver :
Download sahi zip file from SourceForge
Now launch Sahi Server using command below:
$ cd /path/to/sahi $ cd userdata/bin $./start_sahi.sh
Selenium Driver:
You need to download selenium server jar file and execute following command:
$ cd /path/to/selenium-server $java -jar selenium-server-standalone-2.20.0.jar
In this tutorial, we are using sahi driver.
- Now run tests using ANT
$cd /path/to/PHPUnit-Mink $ant Mink
Directory structure
- conf : YAML files can be used with Behat
- core : Abstracted common elements/ methods
- Page : Page objects (reusable Methods for page)
- report : Generate Junit, Agile doc reports
- tests : PHPUnit tests using Mink Api’s
Don’t forget to include Mink and PHPUnit in Test like this:
<?php require_once 'mink/autoload.php'; use Behat\Mink\Mink, Behat\Mink\PHPUnit\TestCase;
Page Objects Pattern
There are some areas within application UI where your test interacts with. These areas can be identified by their functionality, they offer e.g login, registration, search etc. Page object pattern models as a object within you test code. While writing tests, we just need to create an object of the page and access methods and variables withing that page objects. This approach is useful when UI changes, we need to make changes in page objects not in the tests. Tests become readable and maintainable using page objects.
Page objects classes generally:
- Set of public methods that page offer which can be reused later.
- don’t make any assertion. Assertions can be added later into tests not in the page objects
- checks if user is on the same page that page object is created.
- don’t represent entire page but some important functions of that particular page.
GitHub & Other sources of Page object pattern
Read more about page object on google wiki. There are couple of GitHub repositories that explains page object pattern, Saunter and pelenium
Writing Page Objects:
Common methods can be abstracted in the ‘page’ directory. You can specify driver of your choice. Now we have test with Sahi session. Example page looks like this
<?php require_once 'mink/autoload.php'; use Behat\Mink\Mink, Behat\Mink\PHPUnit\TestCase; require_once 'core/core_PHPUnitMink_CommonElementFunctions.php'; class page_search extends TestCase { function search($input) { $this->getSession('sahi')->getPage()->fillField("searchInput",$input); $this->getMink()->getSession('sahi')->getDriver()->click("//*[@id='searchButton']"); $this->getMink()->getSession('sahi')->wait("3000"); } } ?
Using Page Objects in Tests
You can use pages just by creating new objects and accessing variables,methods in the test. Example of test looks like this
<?php class wikiSearchTest extends core_PHPUnitMink_CommonElementFunctions { protected static $mink;</pre> public function testwikiSearchsingle() { $page = new page_search($this): $page->visit(); $page->search(“mumbai”); <pre>assertEquals(“Mumbai”, $page->findFirstHeading())); }
Writing Data-Driven Tests
You can write data driven tests by using PHPUnit and Mink combination. Data can be placed in CSV or in an array. Example data-driven test looks like this
<?php class wikiSearchTest extends core_PHPUnitMink_CommonElementFunctions { protected static $mink; function searchData() { return array( array('london','London'), array('newyork','New York') ); } /** * * @param <type> $searchfor * @param <type> $expectedResult * @dataProvider searchData */ public function testwikiSearchMultiple($input,$output) { $this->getSession('sahi') ->visit('http://en.wikipedia.org/wiki/Main_Page'); $this->getSession('sahi')->getPage()->fillField("searchInput",$input); $this->getMink()->getSession('sahi')->getDriver()->click("//*[@id='searchButton']"); $this->getMink()->getSession('sahi')->wait("3000"); assertEquals($output, $this->getMink()->getSession('sahi')->getDriver()->getText(".//*[@id='firstHeading']/span")); } }
Starting Engines
Before running tests, you need to start Mink drivers, here we are using Sahi. You can update tests to run Selenium driver. Launch driver from command line like this:
- Sahi : Navigate to Sahi directory and run:
moonstar:~ sjagtap$ cd sahi/userdata/bin/ moonstar:bin sjagtap$ ./start_sahi.sh -------- SAHI_HOME: ../.. SAHI_USERDATA_DIR: ../../userdata SAHI_EXT_CLASS_PATH: -------- Sahi properties file = /Users/sjagtap/sahi/config/sahi.properties Sahi user properties file = /Users/sjagtap/sahi/userdata/config/userdata.properties Added shutdown hook. >>>> Sahi started. Listening on port: 9999 >>>> Configure your browser to use this server and port as its proxy >>>> Browse any page and CTRL-ALT-DblClick on the page to bring up the Sahi Controller ----- Reading browser types from: /Users/sjagtap/sahi/userdata/config/browser_types.xml -----
- Selenium
moonstar:downloads sjagtap$ java -jar selenium-server-standalone-2.20.0.jar Apr 7, 2012 3:34:19 PM org.openqa.grid.selenium.GridLauncher main INFO: Launching a standalone server 1 [main] INFO org.openqa.selenium.server.SeleniumServer - Java: Apple Inc. 20.1-b02-383 1 [main] INFO org.openqa.selenium.server.SeleniumServer - OS: Mac OS X 10.7.1 x86_64 8 [main] INFO org.openqa.selenium.server.SeleniumServer - v2.20.0, with Core v2.20.0. Built from revision 16008 131 [main] INFO org.openqa.selenium.server.SeleniumServer - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub 132 [main] INFO org.openqa.jetty.http.HttpServer - Version Jetty/5.1.x 132 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/selenium-server/driver,/selenium-server/driver] 133 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/selenium-server,/selenium-server] 133 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/,/] 273 [main] INFO org.openqa.jetty.util.Container - Started org.openqa.jetty.jetty.servlet.ServletHandler@62f47396 273 [main] INFO org.openqa.jetty.util.Container - Started HttpContext[/wd,/wd] 279 [main] INFO org.openqa.jetty.http.SocketListener - Started SocketListener on 0.0.0.0:4444 279 [main] INFO org.openqa.jetty.util.Container - Started org.openqa.jetty.jetty.Server@4a79717e
Running Tests with PHPUnit
Now you can run wikisearch test with PHPUnit like this :
moonstar:PHPUnit-Mink sjagtap$ phpunit tests/wikiSearchTest.php Temporary SauceLabs fork 3.5.24 of PHPUnit by Sebastian Bergmann. Supports parallel testing. Becomes obsolete when PHPUnit 3.7.0 is released. ... Time: 33 seconds, Memory: 11.25Mb OK (3 tests, 3 assertions)
Running Tests with Ant
You can run tests using apache ANT. Navigate to project root directory (PHPUnit-Mink) and run an
moonstar:PHPUnit-Mink sjagtap$ ant Mink Buildfile: /Users/sjagtap/PHPUnit-Mink/build.xml [delete] Deleting directory /Users/sjagtap/PHPUnit-Mink/report [mkdir] Created dir: /Users/sjagtap/PHPUnit-Mink/report Mink: [exec] [exec] [exec] [exec] [exec] Temporary SauceLabs fork 3.5.24 of PHPUnit by Sebastian Bergmann. Supports parallel testing. Becomes obsolete when PHPUnit 3.7.0 is released. [exec] [exec] .... [exec] [exec] Time: 45 seconds, Memory: 11.75Mb [exec] OK (4 tests, 4 assertions) BUILD SUCCESSFUL Total time: 46 seconds moonstar:PHPUnit-Mink sjagtap$
Once all tests finished running, you will see build successful message. This build can be easily plugged into Continuous Integration server like Jenkins.
Conclusion:
Functional testing become much easier with use of Mink with PHPUnit. Page Object pattern can be used with PHPUnit-Mink to make tests reusable and readable.
You can find source code on GitHub PHPUnit-Mink repository. Happy PHP Testing !
Behat-SauceLabs Integration on GitHub
17 MarBehat is a new generation Behavior Driven Development tool for PHP applications. Behat is inspired by Ruby’s Cucumber. Behat can be used for web acceptance testing by using Mink. Mink uses browser emulators like Selenium, Sahi, WebDriver to launch browser.
SauceLabs is cloud testing service which allow selenium based tests into cloud machine with different browser combination. Create an account with Saucelabs and you will definitely love it
To install and setup Behat project, read SauceLabs blog ‘Adding Sauce To Behat‘.
Let’s directly move on to code. I assume you have installed Behat and Mink as per the blog ‘Adding Sauce To Behat’.
You can use code available on GitHub project ‘Behat-Sauce‘ directly into your project.
Requirement:
*Behat/Mink Installed
*Ant
*SauceLabs Account (Username and API Key)
*Jenkins (optional)
How to use?
* Clone the repository from my GitHub
$ git clone git@github.com:Shashi-ibuildings/Behat-Sauce.git $ cd /path to/Behat-Sauce
* Edit ‘Sauce.yml’ file in order to specify your username and API key.
* Now, Run ANT command to execute feature on sauce Labs
$ ant runSauce
* Once All tests executed see the reports generated in ‘report/report.html’ file.
* Use it in your project by updating ‘behat.yml’ and ‘sauce.yml’. Change Base url to your application url.
* Write and implement features in ‘features’ directory.
You can setup your Jenkins Continuous Integration server by following the blog post ‘Adding Sauce To Behat’
Don’t invest your time/money and effort in setting up different machine/browser combinations for cross browser testing. Better to use SauceLabs.
Any questions? Let me know.
Behat Meetup in a nutshell
17 FebNote : This post may not work with recent Behat. Please use and my Github repo for to get it working without any issue https://github.com/Shashikant86/BehatDemo
On Feb 15, we had a great evening at Ibuildings, London office. We organized the first ever Behat meetup in the United Kingdom. A group called “London Behat Users” organized an event to demonstrate Behavior Driven Development for the PHP developers and testers who wish to get started with Behat.
Behat is an amazing BDD framework for PHP developers which works exactly the same as Cucumber. Assuming that PHP developers and testers are new to BDD, we made the agenda ‘Behat from Scratch‘. There were about 15 behat users who attended this event.
As per the schedule, we had great Pizza before kick-off. While having drinks and Pizza, An Event organizer, Mitch asked everyone to introduced themselves and asked about their experience with Behat. All members were keen to know about other behat users and how long they have been using behat and BDD. After introductions, I explained a little bit about the ‘London Behat users’ group.
Live Coding (Developers View)
Jakub Zalas kicked it off with a talk about the developer’s view in BDD projects. He started the demonstration with a short introduction to BDD, how BDD is different from TDD and so on. Jakub covered following topics while coding:
1. Behat/Mink Installation
Jakub created composer.json file in the project root:
Then he downloaded composer.phar and ran install command which installed all the dependencies of Behat as well as Mink.
2. Creating first project
Once he was done with all the installations, he created a project called “Behat From Scratch” and initialized behat using:
$ behat --init +d features - place your *.feature files here +d features/bootstrap - place bootstrap scripts and static files here +f features/bootstrap/FeatureContext.php - place your feature related code here
The theme of the application was very simple. There was a form with two fields called ‘Title” and ‘Body” and a button called “Preview.” When a user fills in data in the ‘Title’ & ‘Body” fields and clicked on the “Preview” button, then the user should see the text that they entered.
3. Writing first Gherkin feature
Next he wrote one Gherkin feature file with couple of scenarios:
Feature: Previewing article As a Content Manager I want to preview an article before saving it In order to make sure it is displayed correctly Scenario: Article is displayed in a preview area Given I visit "article form" And I fill in "Hello Behat!" for "Title" And I fill in "BDD is fun" for "Body" When I press "Preview" Then I should see "Hello Behat!" in the preview area And I should see "BDD is fun" in the preview area Scenario: Preview is not visible initially Given I visit "article form" Then the preview area should not be visible
After that, he implemented Step Definitions for the integrated classes based on the Silex micro-framework, uses Twig templates and Symfony‘s form component. He implemented all the step definitions until all the steps passed.
The Source code of this is available on GitHub: Behat From Scratch
Browser Automation with Mink (Testers View)
Once Jakub finished with implementation, he handed it over to me to explain how to do browser automation using Mink. I modified the ‘config/behat.yml’ file and tagged the scenarios with ‘@javascript’ tags.
I also explained how to run feature files using different Mink drivers. Behat has provided lot of driver options for testers. It is not limited to Selenium. It has following mink drivers.
Selenium Server
You need to download the selenium server jar file and execute the following command:
$ cd /path/to/selenium-server $java -jar selenium-server-standalone-2.15.0.jar
Sahi Server
Download the sahi zip file from SourceForge
Now launch Sahi Server using the command below:
$ cd /path/to/sahi $ cd userdata/bin $bash start_sahi.sh
There was a quick demonstration of the tests running in browsers using the Selenium 1, 2 (WebDriver), and Sahi servers. This was followed by a short question and answer session.
Try this on your own
You may be interested in trying all this stuff on your own. Jakub mentioned all of the instructions in his GitHub project. I will briefly explain the steps to follow:
- Clone the repository “Bahat From Scratch“
$ moonstar:~ sjagtap$ git clone git://github.com/jakzal/behat-from-scratch.git Cloning into behat-from-scratch... remote: Counting objects: 123, done. remote: Compressing objects: 100% (59/59), done. remote: Total 123 (delta 47), reused 123 (delta 47) Receiving objects: 100% (123/123), 63.34 KiB, done. Resolving deltas: 100% (47/47), done. moonstar:~ sjagtap$ cd behat-from-scratch/
- Install Behat with Composer
Download the Composer and install Behat:
$ moonstar:behat-from-scratch sjagtap$ wget http://getcomposer.org/composer.phar % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 308k 100 308k 0 0 366k 0 --:--:-- --:--:-- --:--:-- 582k $moonstar:behat-from-scratch sjagtap$ php composer.phar install
Here, you may face some problems while installing, depending on your network and permission issues. (There may be a bug in composer.)
I managed to install it properly and it’s up and running in my localhost. You need to cross your fingers at this point.
By default the custom Silex micro-framework session is used so there’s no need for additional configuration. Behat will create a Silex application and simulate requests.
All the project’s scenarios can be run with:
$ moonstar:behat-from-scratch sjagtap./bin/behat As a Content Manager I want to preview an article before saving it In order to make sure it is displayed correctly Scenario: Article form is accessible from the homepage # features/previewing-article.feature:7 Given I visit "homepage" # FeatureContext::iVisit() When I follow "Add article" # FeatureContext::clickLink() Then I should see "Title" input on the article form # FeatureContext::iShouldSeeInputOnTheArticleForm() And I should see "Body" textarea on the article form # FeatureContext::iShouldSeeTextareaOnTheArticleForm() Scenario: Article is displayed in a preview area # features/previewing-article.feature:14 Given I visit "article form" # FeatureContext::iVisit() And I fill in "Hello Behat!" for "Title" # FeatureContext::fillField() And I fill in "BDD is fun" for "Body" # FeatureContext::fillField() When I press "Preview" # FeatureContext::pressButton() Then I should see "Hello Behat!" in the preview area # FeatureContext::iShouldSeeInPreviewArea() And I should see "BDD is fun" in the preview area # FeatureContext::iShouldSeeInPreviewArea() Scenario: Preview is not visible initially # features/previewing-article.feature:23 Given I visit "article form" # FeatureContext::iVisit() Then the preview area should not be visible # FeatureContext::thePreviewAreaShouldNotBeVisible() Scenario: Title and body are required # features/previewing-article.feature:28 Given I visit "article form" # FeatureContext::iVisit() When I press "Preview" # FeatureContext::pressButton() Then I should see "Article title is required" # FeatureContext::assertPageContainsText() And I should see "Article body is required" # FeatureContext::assertPageContainsText() And the preview area should not be visible # FeatureContext::thePreviewAreaShouldNotBeVisible() 4 scenarios (4 passed) 17 steps (17 passed) 0m12.957s
You don’t need to configure the web server to run scenarios using Silex driver. But you do need to configure the web server while running tests in browser.
Configure Web Server.
Follow these steps to configure the web server.
- Update ‘hosts’ file and add ‘behat.dev’ and IP as follows
$moonstar:~ sjagtap$ cd /etc $moonstar:etc sjagtap$ sudo vi hosts # Host Database # localhost is used to configure the loopback interface # when the system is booting. Do not change this entry. ## 127.0.0.1 localhost 127.0.0.1 behat.dev
- Make sure you have updated ‘/etc/apache2/httpd.conf’ to enable virtual host. Remove the comment from Virtual hosts.
$moonstar:~ sjagtap$ cd /etc/apache2/ $moonstar:apache2 sjagtap$ cat httpd.conf # Virtual hosts Include /private/etc/apache2/extra/httpd-vhosts.conf
- Now you need to update the ‘/etc/apache2/extra/httpd-vhosts.conf ‘ file and add the following configuration to it:
$mmoonstar:~ sjagtap$ cd /etc/apache2/extra/ $moonstar:extra sjagtap$ sudo vi httpd-vhosts.conf VirtualHost *:80; ServerName behat.dev DocumentRoot /var/www/behat.dev/web DirectoryIndex index.php Directory /var/www/behat.dev/web> Options FollowSymLinks AllowOverride All Order allow,deny allow from all /Directory> /VirtualHost>
You are done with web server configuration, and now you can run your tests through a browser. The Paths to source code and a domain name will have to be updated. Also the base_url in ‘config/behat.yml’ needs to be changed accordingly.
Run Tests in Browser
- Tag all your scenarios with ‘@javascript’ tags
- Run Selenium server as mentioned above
$ cd /path/to/selenium-server $java -jar selenium-server-standalone-2.15.0.jar
- Update ‘config/behat.yml’
default: context: parameters: default_session: goutte javascript_session: webdriver browser: firefox base_url: http://behat.dev/
- Run ‘./bin/behat’ from a terminal and watch results in browser.
$ moonstar:behat-from-scratch sjagtap./bin/behat Feature: Previewing article As a Content Manager I want to preview an article before saving it In order to make sure it is displayed correctly @javascript Scenario: Article form is accessible from the homepage # features/previewing-article.feature:7 Given I visit "homepage" # FeatureContext::iVisit() When I follow "Add article" # FeatureContext::clickLink() Then I should see "Title" input on the article form # FeatureContext::iShouldSeeInputOnTheArticleForm() And I should see "Body" textarea on the article form # FeatureContext::iShouldSeeTextareaOnTheArticleForm() @javascript Scenario: Article is displayed in a preview area # features/previewing-article.feature:14 Given I visit "article form" # FeatureContext::iVisit() And I fill in "Hello Behat!" for "Title" # FeatureContext::fillField() And I fill in "BDD is fun" for "Body" # FeatureContext::fillField() When I press "Preview" # FeatureContext::pressButton() Then I should see "Hello Behat!" in the preview area # FeatureContext::iShouldSeeInPreviewArea() And I should see "BDD is fun" in the preview area # FeatureContext::iShouldSeeInPreviewArea() @javascript Scenario: Preview is not visible initially # features/previewing-article.feature:23 Given I visit "article form" # FeatureContext::iVisit() Then the preview area should not be visible # FeatureContext::thePreviewAreaShouldNotBeVisible() @javascript Scenario: Title and body are required # features/previewing-article.feature:28 Given I visit "article form" # FeatureContext::iVisit() When I press "Preview" # FeatureContext::pressButton() Then I should see "Article title is required" # FeatureContext::assertPageContainsText() And I should see "Article body is required" # FeatureContext::assertPageContainsText() And the preview area should not be visible # FeatureContext::thePreviewAreaShouldNotBeVisible() 4 scenarios (4 passed) 17 steps (17 passed) 0m12.957s
It’s done !!
Discussion about Behat and BDD
After we finished with the demo, we started a discussion about BDD and behat. To me, this was the most interesting part of the meetup. Lot of things about BDD, tester-developer collaboration, acceptance criteria, Jenkins and other CI systems were discussed.
Some of the highlights of the discussion are as follows:
- Ben and Jakub Zalas shared thoughts about test reporting with Jenkins. (using HTML reports).
- Mitch raised points about making scenarios and running scenarios with different data sets and everybody contributed
- Marcello talked about the role of testers and developers in BDD
- Miles and Alex raised some good questions about web testing using Gherkin DSL.
- John played a key role within the whole session talking about collaborative tools, challenges in BDD and some tricks about BDD.
Conclusion:
The first Behat meetup was an amazing experience for all the attendees. Everybody learned by sharing experiences and ideas. In short, all of us enjoyed our evening of knowledge sharing. We are looking forward to arrange monthly meetups to further discuss Behat and BDD. The next meetup is tentatively scheduled for Wed Mar 21 at 7:00 PM (GMT). Click here for details.
Many Thanks to the creator of Behat: ‘Konstantin Kudryashov‘
Happy BDD !!