Could manual testing work?

Let me tell you about my experience with software testing. In many projects (but not all) there has been a group of testers. Their job has been to test the system after the developers are done, file bug reports, wait for fixes, and then re-test. It all looks sound on the surface, right? However in practice this classical setup has a number of  problems :

1. The tester isn't part of the team. He needs written documentation to know what to test. Somebody has to write this very detailed documentation.
2. Testers I've worked with have had absolutely no clue on how to automate even the simplest test cases. Instead they have spent lots of time doing monkey work.
3. The fact that there is a specific phase for testing means the developer, consciously or unconsciously, gets lazy with his own testing. That results in many bugs.
4. If there are many bugs most of the tests will probably be impossible to run. The tester will have to wait for the developers to fix the bugs.
5. Developers will quickly hack the bug fixes since we're already out of time.
6. Nobody will know when or if it will be possible to deliver anything to the actual users.

 The  solutions  are pretty obvious. In the same order as above:

1. Put the tester into the developer team. The tester will now have a chance to write tests even before code gets written.
2. The tester is a programmer capable of automating tests.
3. There is no testing phase. Stories are done only when they have been tested.
4. No waiting for fixes will occur.
5. No quick hacking will occur.
6. Delivery is on demo day.

Problem solved. Relief.

 Right? 

Actually... Yes, I think so!

In my current project (which more or less follows Scrum) however we have yet to set up automated testing at a story level. The test cases so far have been completely manual. This fact gave rise to a discussion today among team members that I summarize like this:

"The thing I'm working on now affects more than just one story. I'll probably need to re-test quite a few stories and forms. What if the next story is similar? I'd have to re-re-test those same things again! That's going too take to much time. What we need to do is to have a code-freeze a number of days before sprint ending. We'll use those days to test the whole system and fix whatever bugs turn up."

Yes, that would do it.

 No! Panic Button! Incoming Godzilla! Iceberg on starboard! The drachma has arisen from the dead!

If we do this, we re-introduce a testing phase at the end of the sprint. Having a testing phase leads to sloppy developers, which leads to many bugs, which leads to end-of-sprint chaos!

The only really good solution is to automate the testing. But since we haven't done that we need a compromise solution. Suggestion for manual testing:

• Multiple story implementations that affect the same existing functionality get tested together and just once. Before code-freeze. Fix for: Time
• There is a code-freeze and the extra days will help us confirm the done-ness of stories. The testing will cover functionality we didn't even think about when doing story specific testing. Fix for: Regression testing
• Only stories that are completed according to the DoD before the code-freeze occurs, may be included in the sprint delivery. No stories are allowed to enter into the state of Done after code-freeze. Fix for: Bad habits

That might work. Let's see if I get a chance to try it out.

Browser versions and testing Swedish web sites

Judging from the Top 12 Browser Versions and the Top 5 Operating Systems in Sweden in April 2012, the following is my list of OS/Browser/Versions necessary to test on. I've exluded iOS since Sprend doesn't work on it. I might add Linux even if it doesn't reach very high on the OS top list.

And btw, who said the browser was done? Hopefully the list will be even shorter as all browsers move towards continuous upgrading.

The Gate Challenge

Cinemas get more competition every year. Films are being watched in home cinemas and on smartphones.  If the cinema business is going to keep up it needs to do two things:

1. Enhance the experience 
2. Reduce the friction

Since Peter Jackson is busy with the first, let's concentrate on the latter: reducing the friction when buying a ticket. Picture this: You're walking down the street with a friend and you see the cinema building 50 meters ahead. Your friend asks: Fancy catching a flick? And by the time you reach the doors you are holding two tickets in your hand. Magic?

Let's back up. Fifty meters take roughly 30 seconds to walk. In that time you need to grab your phone, start the ticket app, select a movie, pick seats, and pay for the tickets which are delivered within the app itself.

Would it even be possible to create this app? I think so. As it turns out it takes 30 seconds to walk from the gate of Råsunda Filmstad to the doors of the three screen "multiplex" Filmstaden Råsunda. This is..

The Gate Challenge: To create and app that lets you pick up your phone at the gate of Råsunda Filmstad and have the tickets in your hand before you reach the doors of the cinema.

Let's say that in pictures.

1. At the gate of Filmstaden: 30 seconds left

Details

• The app has just been started and is showing tonights movies.
• The cinema is automatically selected based on either proximity or last purchase. I guess last purchase makes more sense. The user may choose another cinema if needed.
• The movies are showing in chronological order, and the list contains only those movies for which tickets can be bought.

2. Passing Portvaktsstugan: 23 seconds left

Details

• Hunger Games at 17:30 has been selected
• The movie has an IMDb score of 7.7 of 10, or 4 stars. The user may select the rating system of his/her choice, or a combination of different rating systems.
• The user selects the number of tickets with the buttons at the bottom.

3. Outside SDI Media: 15 seconds left

Details

• It's possible to zoom, pan and select seats just using your thumb.
• No double tapping or multifinger gestures are needed.
• There is no count down before it's too late to select.
• The available seats are updated in realtime.

4. At the Little Studio: 7 seconds left

Details

• The last step is to confirm the purchase.
• There needs to be a countdown timer here after which the seats are released. But we can wait to display the timer until 30 seconds or so has passed.
• But hey, what about the 30% cut that Apple takes for in-app purchases? Not a problem since the app doesn't utilize the in-app purchase function of AppStore. Apple won't mind since the product sold will not be consumed within the phone. We are free to offer credit card payments or any other payment solution.

5. Just outside the cinema entrance doors: Time is up!

Details

• Yes, we made it in 30 seconds! The two tickets are now showing in the app itself.
• There is no back button and you can't buy more tickets before these ones have been used.
• The tickets inform you that the scheduled starting time is Now!

6. Cinema lobby

Details

• The ticket taker swipes his finger in an S shape to "tear" the ticket. Or some other shape.
• One might want to switch shape regularly to make counterfeiting difficult.

7. Just relax

Details

• Relax and don't forget to switch on Movie Mode (= Airplane Mode).

Finally...

Råsunda Filmstad also houses the headquarters of SF Bio. The question is, is SF up for the challenge? I'm right here should you need a helping hand.

Titanium Breaks

I should probably re-deploy and it'll work...

I've just finished implementing the ROT-avdraget app for iOS as well as for Android. The iOS app was made with Xcode, and the Android app was made with its native tools plus the excellent IntelliJ IDEA.

For the last few days I have tried to re-implement the app using Appcelerator Titanium. The question was whether I could save any time using a cross-platform tool instead of developing in the native environments. The result has been very disappointing. It's difficult even to get the provided sample apps to run, particularly on the Android emulator. Also you can't debug on hardware, so you have to rely on the buggy emulator support.

If only Titanium worked it could be a really nice environment. Most of the development time with the ROT-avdraget app was spent creating the user interface. I found both the iOS interface builder and the Android xml layouts to have a lot of room for improvement. With Titanium the gui is created in code and I think that is a good thing. I found it easier to put views in their right place with Titanium JavaScript code than with the native approaches.

If Titanium is ever going to be good it has to abstract away the differences between the native platforms (when that makes sense). Let me show you a specific example how that fails.

At the top of the screen there is a webview that shows a banner. Before the banner loads, an imageview displays a default image on top of the webview. As soon as the page load has completed, the default image fades away. If the page load fails we wait for 60 seconds and try again.

For iOS the code is very straightforward.

• In webView:didFailLoadWithError we wait 60 seconds and then try again.
• In webViewDidFinishLoad: the default imageview is faded out.

This is the code:

- (void)webView:(UIWebView *)webView
        didFailLoadWithError:(NSError *)error
{
    [self performSelector:@selector(openUrlInTopBanner)
          withObject:nil
          afterDelay:60];
}

- (void)webViewDidFinishLoad:(UIWebView *)webView {
    [self fadeOutImageToShowBeforeBannerLoads];
}

For Android the logic is slightly different.

• onReceivedError() sets a failure flag, and then creates a timer for reload.
• onPageFinished() will get called on both success and failure which is why we need to check the flag before fading out the default imageview.

This is the code:

public void onReceivedError(WebView view, int errorCode, String description, String failingUrl)
{
    isFailedTopBannerPageLoad = true;
    if (topBannerLoadTimer == null) topBannerLoadTimer = new Timer("topBannerLoadTimer");
    topBannerLoadTimer.schedule(new TimerTask()
    {
        public void run()
        {
            isFailedTopBannerPageLoad = false;
            openUrlInTopBanner();
        }
    }, 60 * 1000);
}

public void onPageFinished(WebView view, String url)
{
    if (!isFailedTopBannerPageLoad)
    {
        fadeOutImageToShowBeforeBannerLoads();
    }
}

And here's the Titanium version of the same bit of functionality. The code is a reflection of the iOS code above:

 
topBanner.addEventListener("error", function(e) {
    setTimeout(function() {
        openUrlInTopBanner()
    }, 60 * 1000);
});

topBanner.addEventListener("load", function(e) {
    fadeOutImageToShowBeforeBannerLoads();
});

It looks simple enough. When run on an iPhone it works exactly like the Objective-C code. But bad things happen as you run it on Android:

• The load event listener gets called when the page succeeds. Good!
• The load event listener gets called also when the page fails to load. What? Hmm, so it works like the native Android event listener?
• The error event listener never gets called.
• Titanium Fail!

So I would assume there are workarounds. The problem is that there had better exist a truck load of workarounds. I kept a little log during the development. Here is part of that log:

• The New Project Wizard fails to find the Android sdk! Not until I create an empty folder for an old version of the sdk does it succeed to locate the sdk.
• The Hello World sample project fails to deploy to the iPhone simulator. On the second attempt... it mysteriously works.
• The Hello World sample projects fails to deploy to the Android simulator. The second attempt also fails. After restarting Eclipse... it mysteriously works.
• Deploy to an iOS device uses iTunes synchronization. Since I don't normally sync my iPhone with iTunes on my developer machine it means all the installed apps are deleted when I deploy the Titanium-built app.
• The call to Ti.UI.createImageView() gives a runtime error. Workaround: Rebuild and redeploy and it works.
• createImageView(): Android requires a slash at the beginning of a path. iPhone does not.

The Android simulator (or is it emulator?) is a story of itself.

• Starting the simulator creates five console logs. Where does one look for error messages?
• I need to disconnect the physical Android device to run the app on the simulator, or else I get "More than one device and emulator".
• Android simulator hangs when deploying an app. And again. And again.
• Rebooting computer. Trying again. Android simulator hangs on "get app.js"...
• Finally the app installs and starts. I press the back button. Next deploy fails. I uninstall the app from within Android. Now it seems to work again, for a while.

I'm giving up on Titanium. It is just broken.

It seems I'm not alone:

• A few months with Titanium Appcelerator
• Appcelerator's Titanium: A truncated review

Android GUI building

This post is an update to my previous post about xml vs programmatic GUI development.


After a bit more Android experience it seems to be true that the xml syntax is indeed simpler than the Java equivalent. The Java code turns out to be even more verbose than the xml is. Here is a simple example:

ScrollView rootView = new ScrollView(this);

setContentView(rootView,

  ScrollView.LayoutParams.FILL_PARENT,

  ScrollView.LayoutParams.FILL_PARENT));




LinearLayout linear = new LinearLayout(this);

rootView.addView(linear,

  new LinearLayout.LayoutParams(

    LinearLayout.LayoutParams.FILL_PARENT,

    LinearLayout.LayoutParams.FILL_PARENT);

Also, it is more difficult to find documentation for the programmatic method. You are expected to create the GUI with xml.


It is still true that debugging xml is impossible. However, it hasn't been necessary to debug this xml. The declarative programming style seems to work in this case.


All the above speaks for the xml approach. However, the xml is simpler not because of some inherent quality, but rather because the Java API is really bad.

For my next Android project I might try to wrap the view classes into user-friendlier versions. After all I can make my own API if I'm not happy with the one provided by Google. Perhaps like this:

MyScrollView rootView = new MyScrollView(this, LP.FILL_PARENT, LP.FILL_PARENT);

setContentView(rootView);

MyLinearLayout linear = rootView.addLinearLayout(LP.FILL_PARENT, LP.FILL_PARENT);

A third approach would be a visual GUI builder. The one in Xcode 4 is a good start, but far from really good.

Debugging XML?

Let me quote the Hello World tutorial at developer.android.com:

"The general structure of an Android XML layout file is simple: it's a tree of XML elements. ... This structure makes it easy to quickly build up UIs, using a more simple structure and syntax than you would use in a programmatic layout."

Hmm, let's try that. First the xml version:

<TextView
  android:id="@+id/textview"
  android:layout_width="fill_parent"
  android:layout_height="fill_parent"
  android:text="Hello, Android"/>

And now the Java version:

TextView tv = new TextView(this);
tv.setText("Hello, Android");

Is the xml version simpler? Not sure. It's a little bit verbose but still readable.

What about editor support? In a good editor, like that of IntelliJ, I can Ctrl+Space Java code as well as Android layout xml. So, no difference.

But there is one difference:

You can step Java code, but you cannot step XML!

Case closed.

Safari Books Online

I signed up for Safari Books Online today because I wanted to refresh my iOS skills with the latest edition of Beginning iOS 5 Development: Exploring the iOS SDK. The experience is however far from what I expected. Safari Books Online has been around for years and years so I thought they had figured it out and ironed out the rough edges with online reading.

The big screen reader is Flash based and it has a number of problems:

• The text is blurry.
• The pages are formatted for print, not for screen reading.
• The scrolling speed is so fast it's almost not usable (at least on a MacBook trackpad).

The "mobile optimized site" is perhaps not so optimized:

• The pages are shown as blurry images.
• The text is so small it is barely legible.
• There is no pinch-resizing.
• The zoom function makes the pages just too large to fit in landscape mode on an iPhone 4.
• The system doesn't remember where you left off, when you switch between the big screen reader and the mobile site.

And there's more:

• Why aren't errors in the text fixed?
• Why are links in the text not clickable?

Who said things are moving fast in our business? The IT sector is moving sloooowly.

But guess what, I'll probably become a paying customer just because of the convenience of immediate and ubiquitous access.