Navigation
Powered by Squarespace
« Dan Ariely's Predictably Irrational... | Main | Vite,Vite! Squid Reverse Proxy ... »
Saturday
Nov222008

Web Automation fun with Selenium RC

So, many of you are probably familiar with Selenium for deploying automated web-application testing. I've recently begun a small side project that involves automated calls to web-sites that are AJAX and Javascript heavy in order to scrape content from the pages. Typically, I would turn to my trusty friend cURL for most screen-scraping, but in cases where the content on a page is dynamically generated by AJAX, its a lot easier to emulate a browser as closely as possible. Enter Selenium-RC, and in my case, the PHP Selenium driver. More on this topic later, but for now, if you are facing a strange intermittent problem with the Selenium server complaining about SocketException: Broken Pipe , here is your solution: The command handler in the Testing/Selenium.php uses (apparently) an incomplete socket connection routine. I have found that the same handler provided by the PHPUnit's Selenium Driver is much more robust.

  protected function doCommand($verb, $args = array())
  {
      $url = sprintf('http://%s:%s/selenium-server/driver/?cmd=%s', $this->host, $this->port, urlencode($verb));
      for ($i = 0; $i < argnum =" strval($i" s="%s',">sessionId)) {
          $url .= sprintf('&amp;%s=%s', 'sessionId', $this->sessionId);
      }

      if (!$handle = fopen($url, 'r')) {
          throw new Testing_Selenium_Exception('Cannot connected to Selenium RC Server');
      }

      /*
      stream_set_blocking($handle, false);
      stream_set_timeout($handle, 0, 60000);
      $response = stream_get_contents($handle);
      fclose($handle);
      */
  
      stream_set_blocking($handle, 1);
      stream_set_timeout($handle, 0, 60000);

      $info     = stream_get_meta_data($handle);
      $response = '';

      while (!$info['eof'] &amp;&amp; !$info['timed_out']) {
          $response .= fgets($handle, 4096);
          $info = stream_get_meta_data($handle);
      }

      fclose($handle);

      return $response;
  }

 

 

Replace the doCommand function with the above code, and you should find better results. More on this to come...

PrintView Printer Friendly Version

EmailEmail Article to Friend

Reader Comments

There are no comments for this journal entry. To create a new comment, use the form below.

PostPost a New Comment

Enter your information below to add a new comment.

My response is on my own website »
Author Email (optional):
Author URL (optional):
Post:
 
Some HTML allowed: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <code> <em> <i> <strike> <strong>