Tuesday, October 14, 2008

Usability testing in entrepreneurial software

Jeremy Liew of Lightspeed Venture Partners has an excellent blog post on Usability Testing and how it applies to building entrepreneurial software. In his post he mentions that it's not just about rapid prototyping and turnaround (which he approves of) but also says that at some point you have to step back and do surveys, and psychological profiles of users and see exactly HOW they used the site. This is a critical step in building software, and obviously (since Jeremy commented on it) important to getting venture funding as well.

At OS-Cubed, we believe that usability testing should be scoped into the project itself, and the amount invested in it should be scaled according to the entrepreneur's budget, but it should be conducted throughout the development process as you're building your demo or proof of concept and rolling out your RAD prototype. In the early stages usability testing does not need to be either expensive or costly. You can do much of the testing yourself with the demo as it's rolled out. Noah Kagan of Mint lays out the steps for you in his excellent post. They involve basically: determining what to test, finding a representative user base, and then testing by observing. Noah recommends using Webex products, so far we've been lucky enough to have user bases that are representative right here in Rochester.

As Noah points out - doing this can be frustrating for the observer. The observer needs to sit back and let the user do it - without intervening or showing them how.

In our experience most of the user testing we've done with KaJour our product for Knowledge Athletes has shown up issues in the scalability section - which you'd expect in demo software. We'd actually anticipated these issues but didn't have the budget to build production level. We consciously chose with the entrepreneur to build more features, rather than make the features we had more robust. The users have also come up with unique and interesting ways of using the product that we didn't anticipate and neither did the Knowledge Athletes staff. Through our user testing we've found new markets, new ways of thinking about what we're doing and developed new teaching paradigms for the community we're serving (at the moment mostly high school and college students with this product).

Although Jeremy points out the obvious benefits of using usability testing to be sure your software is stable and simple to use, he doesn't mention that really listening to your users can allow you to create new and better uses for your software, and take your product in an entirely new direction. To do that you need to truly listen to your users and find out what it is they need - what pain you are solving - and how you could do it better.


DaveM said...

Lee, like what you have to say about the importance of usability testing, and yes, we're fortunate that we have had some great pilot tests with substantial user feedback. One thing I would add to the conversation beyond the rapid prototyping, surveys, and psychological profiles of users that you mention is the formal usability testing that can be provided by "expert" groups that approach usability testing holistically. These groups can blend all the field data collected by the developer with a project approach and a group of users who have experience in use of applications that exist in the targeted market space in which the product is going to launch or extend the beta testing. One example we're considering is a local college that has developed the for-credit classrooms and/or for-fee testing labs that employ students to dig in and hammer on the product in very productively critical ways. The outcomes are suggestions for making the product more useful for the targeted audience of marketplace users.

And I have to agree that identifying usability testing as part of the user test plan, at the beginning of the project, can pay dividends and save development costs and attenuate surprises during an extended development cycle, not uncommon for software projects.

Lee Drake said...

Just a quick note to point out that Dave is the CEO of Knowledge Athletes, one of our entrepreneurial clients. His excellent product is poised to take the education market by storm. You can check out details at http://www.knowledgeatheletes.com.

Post a Comment

Note: Only a member of this blog may post a comment.