RIATest review

: Function split() is deprecated in /home/nbilyk/nbilyk.com/wwwroot/themes/amadou/node-os_flash.tpl.php on line 60.
Tagged:  

I started using RIATest a few days ago, and so far, it's pretty awesome. RIATest is a utility that lets you record and play back test scripts.

Test-driven development can be quite troublesome. Few companies do any sort of automated testing on their code. Deadlines are short, and nobody wants to take the time to write unit tests. I can't blame them. That's why I was very pleased to find this tool. It's rather pricey at 400$, but if you compare that to the cost you'll save compared to manual testing, or the time it takes to write unit test scripts by hand, it's well worth it.
Typically, the way unit testing works, is you write a script that executes functions of your main program, and makes sure it returns what it is supposed to given your arguments. This works fine for algorithms, I use that kind of testing for my physics engine, or to make sure utility functions work. But I'd say about 99% of the time, you want to test a specific part of your program, like, "Does registration work?". So the way you do it with RIATest, is you run the application, hit record, register your user like you normally would, and you're done. After any changes you make that might affect registration, just run the script and you'll know. Then of course you can edit the script and make it try random usernames or what have you.

If you're looking for a quick laundry list of pros and cons, here you go:

Pros

  • Once you know how it works, it's remarkably easy to use. The learning curve is about 2 hours to figure things out, but hopefully this article will cut that down.
  • Support. They've been quick and thorough in answering my noob questions.
  • Light-weight. I hate bloatware. I want all bloated and sluggish programs to die. (Sorry Flash CS3, MS Office, Vista, McAffee)
  • You can run multiple test scripts at once. It goes through them so fast though that I haven't used this much.
  • Works even with complex programs with animations and transitions without having to code anything.

Cons

  • Requires Flex Pro: $600
  • Price-tag: $400
  • You have projects with a list of scripts. But as far as I know, you can't just run an individual script, you have to run the whole project. Please drop a comment if I'm wrong about that.
  • There are a couple ways to use RIATest, by compiling in the framework into your application, or using their loader. It's obvious why you wouldn't want to do the former, and using the latter I don't believe you can pass in FlashVars to your application. Again, I might be wrong about this.
  • Along with FlashVars, it would be nice if there's a scripted way to be able to change the url so you can test deep linking.

There are a few pitfalls to watch out for when you're getting started. The first one is that you need Flex pro. Flex standard only has a few features behind what pro does, and unfortunately, this is one of them:
Support for automated functional testing
It feels like it's almost going to work on standard, but things just quickly go splat.
The next pitfall is automationName. RIATest and all other testing engines that use Flex's automation support do their selectors based on the UIComponent property automationName.
Here is an example selector:

FlexButton("Register")=>click();

The selector finds the Register button based on its label: "Register". Of course this is preposterous, who on earth would want to rely on content of any sort to never change. Select it based on the ID, damn-it!
Well, to do that, you need to set the automationName property on that button component. So just:
<mx:Button id="registerButton" automationName="registerButton"/>

Here's what Adobe help says about automation name:

"The default value of this property varies depending on the type of component. For example, a Button control's automationName is the label of the Button control. Sometimes, the automationName is the same as the control's id property, but this is not always the case."

That sounds technical, eh?

Well, anyway, here's a way to kinda help automatically make the automationName thing a little better.

addEventListener(Event.ADDED, addedEventHandler, false, 0, true);
 
private function addedEventHandler(event:Event):void {
	if (event.target is UIComponent) {
		var component:UIComponent = UIComponent(event.target);
		if (component.automationName) return;
		if (component.id) {
			component.automationName = component.id;
		}
	}
}

If you put the addEventListener into somewhere like your preinitialize function of your Application, any time a child or grandchild gets added, the handler will check if an id was assigned to it, and if so, make the automationName the id.
Of course, now you have to give everything you might want to test an id. And it still doesn't work in all cases.

Here's the video demonstrating the use of RIATest. Also, I just made this video player, so please let me know if there are any bugs with it.

This site requires the Flash player to display correctly. Go to http://www.adobe.com/go/getflash/ to download this plugin.



In Using RIATest Loader Remotely I get error RIATest Loader cannot load application
Error #2124
Does Ria test supports HTTPS?

Nice video

Does Ria test supports HTTPS

I know that FunFX has an uncommitted patch that makes it work on OS X too. Does anyone know how far RIATest is from supporting OS X?

RIATest for Mac OS X has been announced. See http://www.riatest.com/forums.html?func=view&catid=9&id=1127

FlexMonkey is similar to RIATest and was recently open sourced by Gorilla Logic. It records Flex UI interactions and generates FlexUnit test cases.

Check it out a http://flexmonkey.googlecode.com

Stu Stern
Gorilla Logic, Inc.

stu,is there any good specific doc on the flexmonkey and AIR desktop applications. we have developed very complicated metadata UI framework that works with sqlite database, it also has has server upsync and downsync capabilites, anyway i tried flex monkey and my application just hangs? thanks

and good video :-)

We are glad you found RIATest valuable.

A couple comments:
- The ability to run one test script (or a group of scripts) at a time will be implemented in RIATest 2.0, which will be released in November.
- You are right that you cannot pass your own FlashVars to the application when using the Loader. We will see what can be done to make this a possibility in future versions.
- Could you please elaborate more on how you think could url change work to support deep linking?

Cory, when "verifying" during playback the script ensures that the values of select properties match those that existed at the recording time. Each Flex class has a list of default properties that will be "verified" when you click Verify, however it is up to you to select other properties as well. So the test script is generally a sequence of actions interleaved with verifications which make sure that the state of the application has changed as it was expected as a result of preceding actions. Of course more complex cases involving conditional checks, loops, etc are possible with manual coding the test scripts.

Check back real soon for RIATest 2.0 Beta release :-)

Regards,
Tigran Najaryan
RIATest.com

I imagine just a basic way to communicate with the browser. Just being able to give javascript commands would be perfect.

javascript:history.back(1);
javascript:document.location.href = "www.example.com#action=activation";

That would allow you to do whatever you want with javascript, including calling functions that talk to flex via ExternalInterface.

I don't know how difficult it would be to allow the page to be reloaded and then resume the script without timing out, but that's pretty much what I'm picturing.

I don't really understand what you're "verifying." You click on a component, click verify, and then... verify what?

Maybe walk me through a more practical example. I have a Flex app that loads a complex engine rendering thingy into it and lets you interact and paint graphics in. I added a menu bar with an option to load a map, which pops up a window and lets you paste a password in to deserialize. I want to test that clicking the menu item actually pops up the window, and that clicking load in the window actually triggers the load process.

(The load process itself can be tested with a more typical, non-UI testing suite).

How would this tool help me verify my GUI worked as described above?

When you have a selector, say: FlexTextArea("firstName")
RIATest waits until it can actually find a text area with automationName "firstName". If it can't find it within X seconds, then the script fails, so just having these selectors verifies that they exist and your program at least has the components you expect. Then using the inspect and verify adds lines like:
var comp = FlexButton("viewProfileButton");
verifyEqual(comp=>enabled, true);
You can verify any property of the viewProfileButton. If it's a custom component, and you want to assert a property that isn't on an ancestor flex component, you can provide RIATest with an xml descriptor of that custom class.

So far, few of my tests have many assertions, I know that just the fact that the test progressed to that point is enough to know it's working. So with your menu item that pops up a window, when you're recording, if you have any selectors or actions that belong in that window, you know the window is there because RIATest will fail if the window is never seen.

Thank you for sharing!