Information
Resources
Categories
Authors
Choosing a programming language: Is type safety worth it?
Posted by fred.bertsch in Software at 08:45 | Tuesday, July 31. 2007
I'm a big fan of the strongly typed language vs weakly typed language debate. It's old, but it's also important.
I'm revisiting this topic because I'm trying to decide exactly that problem on a project that I'm working on. In my case, I'm torn between client side JavaScript on the web browser and Google Web Toolkit (GWT) compiled to JavaScript. The only reason I have to mention this is because I want to eliminate the quality of the libraries from the debate. GWT has fairly rudimentary libraries, but they're growing, and you can add JavaScript libraries if you really have to.
I'm going to try to have a debate with myself about it. I can't claim that I'm unbiased, and I'm going to ignore arguments that I don't consider significant. I'd love to hear further arguments from you in the comments.
Weak typing allows you to write code faster. Well, that's what the proponents of it claim, anyway. I haven't seen any good measurements of this. There's certainly less typing to do, though. You don't have to declare the type of your variables. (In JavaScript, you do have to declare your local variables, however.) (It is worth pointing out that I have never written software and been limited by my typing speed. I'm skeptical of this argument.)
If you're designing a scripting language, then weak typing is easier to explain to inexperienced programmers. There's a whole step that doesn't have to be done. In some circumstances, this is a clear win. (In my case, this is irrelevant.)
Strong typing allows the compiler to do more work for you. Essentially, you're constraining how variables can be used, and the compiler can use those constraints to detect errors or perform optimizations.
The optimization part is important for a few people. The error detection is great for everyone.
Unit testing is the weakly typed solution to error checking. (It's also the strongly typed solution, but for a different class of error.) It's either hard or impossible to write a unit test that checks that function arguments have the correct properties before going into a function. However, you can write a unit test that ensures that functions behave correctly in a variety of different circumstances. That's what programmers really care about, anyway.
Also, suppose we want to have a function that supports multiple types? In a strongly typed language like Java, we can only support different types if they share an interface. In a weakly typed language, we can support multiple types if they support the same operations. If I want a function that adds all the elements of a collection together, I can write one function that supports both strings and complex numbers correctly. (This is possible with generic programming as well in a strongly typed language, but that's not really what we're arguing about here.)
Unit testing is a good guarantee, and it's one that's required by strongly typed languages as well. However, it's a fundamentally different guarantee than you get by having the compiler check the types of variables as they're used by the code. First, a compiler checks every line of code even if it's hard to get to or you forgot to write one of the unit tests. Second, the combination of tests and strong typing is a better guarantee than tests alone.
Here's a Ruby example of some dangerous code:
This code has two "overloads". One takes a string and a negative number. The other takes a non-negative number and might take a string. The problem is, there is no good way to tell this based on the function signature. You can only tell by looking at the implementation. This means that you can't safely change the implementation of your function.
Unit testing partially guarantees that functions have working implementations. Strong typing partially guarantees that they are used correctly.
I'm going to go with Google Web Toolkit, which imposes strong typing on top of JavaScript. The mediocre library support may bite me later, but I'll be happier knowing that my compiler knows a bit about what my code will do.
I'm hoping I get flamed by someone, so please post your comments!
I'm revisiting this topic because I'm trying to decide exactly that problem on a project that I'm working on. In my case, I'm torn between client side JavaScript on the web browser and Google Web Toolkit (GWT) compiled to JavaScript. The only reason I have to mention this is because I want to eliminate the quality of the libraries from the debate. GWT has fairly rudimentary libraries, but they're growing, and you can add JavaScript libraries if you really have to.
I'm going to try to have a debate with myself about it. I can't claim that I'm unbiased, and I'm going to ignore arguments that I don't consider significant. I'd love to hear further arguments from you in the comments.
Argument for weak typing
Weak typing allows you to write code faster. Well, that's what the proponents of it claim, anyway. I haven't seen any good measurements of this. There's certainly less typing to do, though. You don't have to declare the type of your variables. (In JavaScript, you do have to declare your local variables, however.) (It is worth pointing out that I have never written software and been limited by my typing speed. I'm skeptical of this argument.)
If you're designing a scripting language, then weak typing is easier to explain to inexperienced programmers. There's a whole step that doesn't have to be done. In some circumstances, this is a clear win. (In my case, this is irrelevant.)
Argument for strong typing
Strong typing allows the compiler to do more work for you. Essentially, you're constraining how variables can be used, and the compiler can use those constraints to detect errors or perform optimizations.
The optimization part is important for a few people. The error detection is great for everyone.
Counterargument from weak typing.
Unit testing is the weakly typed solution to error checking. (It's also the strongly typed solution, but for a different class of error.) It's either hard or impossible to write a unit test that checks that function arguments have the correct properties before going into a function. However, you can write a unit test that ensures that functions behave correctly in a variety of different circumstances. That's what programmers really care about, anyway.
Also, suppose we want to have a function that supports multiple types? In a strongly typed language like Java, we can only support different types if they share an interface. In a weakly typed language, we can support multiple types if they support the same operations. If I want a function that adds all the elements of a collection together, I can write one function that supports both strings and complex numbers correctly. (This is possible with generic programming as well in a strongly typed language, but that's not really what we're arguing about here.)
Rebuttal from strong typing
Unit testing is a good guarantee, and it's one that's required by strongly typed languages as well. However, it's a fundamentally different guarantee than you get by having the compiler check the types of variables as they're used by the code. First, a compiler checks every line of code even if it's hard to get to or you forgot to write one of the unit tests. Second, the combination of tests and strong typing is a better guarantee than tests alone.
Here's a Ruby example of some dangerous code:
function foo( a, b )
if a < 0.0
print( b )
end
end
foo( -12, 'This string is printed' )
foo( 5 )This code has two "overloads". One takes a string and a negative number. The other takes a non-negative number and might take a string. The problem is, there is no good way to tell this based on the function signature. You can only tell by looking at the implementation. This means that you can't safely change the implementation of your function.
Unit testing partially guarantees that functions have working implementations. Strong typing partially guarantees that they are used correctly.
Conclusion
I'm going to go with Google Web Toolkit, which imposes strong typing on top of JavaScript. The mediocre library support may bite me later, but I'll be happier knowing that my compiler knows a bit about what my code will do.
I'm hoping I get flamed by someone, so please post your comments!
Ridding ourselves of the C++ template bloat
Posted by Bender in Software at 08:39 | Tuesday, July 31. 2007
C++ is all about object oriented programming. Combined with templating, objects become generic and �reusable.� Or at least thats what your typical beginning computer science instructors will tell you in high school or college. The truth is that while objects appear reusable to you, they are rarely actually reused by your compiler. This article, the first in a series on C++ optimizations, discusses a simple way to reduce your application's code size while improving execution speed.Lets start out the article with a typical C++ class declaration for an Array template.
Lets say your program needs an array of integers, you would declare an instance as:
Then later in code, you find out that you need an array of unsigned integers:
When you compile your program in C++, the compiler actually creates two full working copies of the Array class. One instance is for the integer data type, the other for the unsigned integer data type. This naturally doubles the size of your compiled code. As your C++ application gets more and more complex, the amount of binary code to implement all instances of the Array class increases significantly.
As code size goes up, you can expect performance to go down for a number of reasons. The two big ones that I often talk about is that it takes longer to initially load code from the hard drive and your CPU cannot keep very much code in its on-die cache so it has to swap it out to the main system memory.
Luckily for us C++ has a little known and rarely used set of features that can not only help us reduce our application's code size, but also improve cache-friendliness.
The first feature that we'll discuss is the concept of inline-casting with template specialization. When you have two data types, which consume the same amount of memory, you can do this method. Here is an example of how it works:
If you have an
The basic array with an example method to retrieve a reference to element
of the array.
When your compiler sees the above code, it declares a generic template for Array of type
Most compilers will honor your
The above example only demonstrates one relatively simple method of the Array class, but some methods of your container data type
template < class T >
class Array {
public:
T & operator[](int n) { return m_elements[n]; }
... // Other methods
private:
T m_elements;
};Lets say your program needs an array of integers, you would declare an instance as:
Array< int > myIntArr;Then later in code, you find out that you need an array of unsigned integers:
Array< unsigned int > myUnsignedIntAr;When you compile your program in C++, the compiler actually creates two full working copies of the Array class. One instance is for the integer data type, the other for the unsigned integer data type. This naturally doubles the size of your compiled code. As your C++ application gets more and more complex, the amount of binary code to implement all instances of the Array class increases significantly.
As code size goes up, you can expect performance to go down for a number of reasons. The two big ones that I often talk about is that it takes longer to initially load code from the hard drive and your CPU cannot keep very much code in its on-die cache so it has to swap it out to the main system memory.
Luckily for us C++ has a little known and rarely used set of features that can not only help us reduce our application's code size, but also improve cache-friendliness.
The first feature that we'll discuss is the concept of inline-casting with template specialization. When you have two data types, which consume the same amount of memory, you can do this method. Here is an example of how it works:
If you have an
Array< int > and Array< unsigned int > both used by your program, we'll �wrap� one implementation within the other. First you declare your generic Array template which will handle Array<int> and then you declare a special cast version which will handle Array<unsigned int>.The basic array with an example method to retrieve a reference to element
nof the array.
template < class T >
class Array {
public:
T & operator[](int n) { return m_elements[n]; }
... // Other methodsprivate:
T m_elements;
};
#define INTERNAL_TYPE int
#define EXTERNAL_TYPE unsigned int
template <>
class Array< EXTERNAL_TYPE > {
public:
inline EXTERNAL_TYPE & operator[](int n) { return (EXTERNAL_TYPE) m_array[n]; }
... // Other methods
private:
Array< INTERNAL_TYPE > m_array;
};
When your compiler sees the above code, it declares a generic template for Array of type
T, whereT is any given type. The type unsigned int is an exception though, because it sees that the template has been specialized to have a different implementation than the generic Array<T>. We actually utilize Array<T> from within Array<unsigned int> for the member data called m_array.Most compilers will honor your
inline request and therefore automatically map down any usage of Array<unsigned int> to actually be the exact same binary code asArray<int> when you do this.The above example only demonstrates one relatively simple method of the Array class, but some methods of your container data type
Array<T> might be amazingly complex or produce large amounts of CPU instructions during compiling. Regardless of how complex and large the implementation of Array<int> becomes, you can count on your inline cast specialization of each of Array's methods to add little or no code to your final binary application.
Keeping them safe or making them vulnerable?
Posted by Chris Hill in Grab Bag at 11:01 | Monday, July 30. 2007
With the advent of modern, more affordable technology, parents tracking their children and teens is becoming ever prevalent. Their every move and in some cases, every action, are being monitored in real time with precision. Take for instance a new system provided by some insurance agencies. AIG and Safeco Insurance agencies are launching and/or piloting new programs to track teens driving habits. One such system labeled "Teensurance" uses GPS and has notifiers to inform the parents via email or text message when their teen driver breaks a predetermined speed limit, reaches a distance from home that's further than approved, or doesn't return home for that 9 o'clock curfew. The car's location can even be listed on the Teensurance website.
"Peace of mind at an anxious time" and "Knowing they're safe means less checking in" are a few marketing taglines found at the Teensurance website. These statements may not be entirely true. Officials have stated that there are now 29,000 sex offenders on MySpace. Criminals are becoming more and more technically savvy. Not only do we now have to monitor our children's internet use, but imagine a sex offender finding a back door allowing him to track your child's every move. As the father of three, that gets the stomach turning -- I think I can feel my hair turning gray as I type.
Another interesting service that begs the question, "am I keeping them safe or making them vulnerable" is Disney Mobile. Overall this sounds like a great service, with functionality to limit minutes and text messages, sending prioritized messages to family members, and controlling calls by time of day, etc. The more questionable feature is Family Locator.
Again, how secure can this really be? Imagine someone hacking into Disney's GPS tracking system and locating every child that holds a Disney Mobile phone. Not only is my hair turning gray now but I can feel an ulcer forming. Hacking is a serious security concern these days. Banks, businesses, government databases, they're hacked regularly. I can live with identity theft. Can I live with one of my children being attacked because I wanted to make myself feel more secure about where my children are and what they're doing? Absolutely not! Besides, isn't that what we call, yup, you guessed it...BEING A RESPONSIBLE PARENT! If you have to resort to measures such as these, I say you need to look in the mirror and question your ability to be a good parent.
Privacy issues aside, is all this "protection" worth the risk? That's your call to make and not one that should be taken lightly.
"Peace of mind at an anxious time" and "Knowing they're safe means less checking in" are a few marketing taglines found at the Teensurance website. These statements may not be entirely true. Officials have stated that there are now 29,000 sex offenders on MySpace. Criminals are becoming more and more technically savvy. Not only do we now have to monitor our children's internet use, but imagine a sex offender finding a back door allowing him to track your child's every move. As the father of three, that gets the stomach turning -- I think I can feel my hair turning gray as I type.
Another interesting service that begs the question, "am I keeping them safe or making them vulnerable" is Disney Mobile. Overall this sounds like a great service, with functionality to limit minutes and text messages, sending prioritized messages to family members, and controlling calls by time of day, etc. The more questionable feature is Family Locator.
What is Family Locator?
"The Family Locator feature enables you to locate your child's handset using advanced GPS technology. From your phone or online, you can locate your child's Disney Mobile phone and receive a location and a map of the location without calling. "
Again, how secure can this really be? Imagine someone hacking into Disney's GPS tracking system and locating every child that holds a Disney Mobile phone. Not only is my hair turning gray now but I can feel an ulcer forming. Hacking is a serious security concern these days. Banks, businesses, government databases, they're hacked regularly. I can live with identity theft. Can I live with one of my children being attacked because I wanted to make myself feel more secure about where my children are and what they're doing? Absolutely not! Besides, isn't that what we call, yup, you guessed it...BEING A RESPONSIBLE PARENT! If you have to resort to measures such as these, I say you need to look in the mirror and question your ability to be a good parent.
Privacy issues aside, is all this "protection" worth the risk? That's your call to make and not one that should be taken lightly.
Top 6 reasons why you should learn a scripting language, even if you are not a programmer
Posted by hector in Software at 15:41 | Friday, July 27. 2007
Computers are supposed to make our lives easier, but everyone once in a while finds themselves doing repetitive tasks on a monitor screen. Well, aren't computers designed to take care of that?
By learning a scripting language you can easily make a computer do the tedious work for you and more! Here is a list of 6 more reasons to learn a scripting language, even if you are not a programmer.
So now you have all the reasons to start learning a scripting language. Which one should you pick? I'll look at different ones next time to help you decide which one is right for you.
By learning a scripting language you can easily make a computer do the tedious work for you and more! Here is a list of 6 more reasons to learn a scripting language, even if you are not a programmer.
- It will let you automate repetitive tasks. This is definitely the biggest selling point. As I said before, computers are supposed to do the tedious work for us and by using a scripting language we can easily give commands to our emotionless servants in a language they understand.
- They are simple to learn. If you are already a programmer, many scripting languages closely resemble the syntax of your favorite compiled language (small investment, big returns). If you are not a programmer, scripting languages are usually and very good introduction into the world of programming (medium investment, big returns).
- Write it and run it. Who doesn't like instant gratification? When you write a script you don't necessarily have to transform it into machine code to run it. No compile, no wait, just run, right away. This of course carries a small performance penalty, but if you are after ultimate performance by using a script, then you are probably looking in the wrong place.
- Most of them are free. Very inexpensive price of admission. No try before you buy, just use it and still have the peace of mind that it will always be yours.
- Community and commercially supported. Some people are very passionate about their scripting language. If you have one close to you, he'll be more than happy to answer your questions. If you don't have one around, you can always find them at Internet forum flame wars defending their scripting language of choice. If you still can't find one, you can always find a company that will give you commercial support. Just make sure to read the manual before you even start looking.
- It will make you stronger. Having another weapon on your arsenal can only benefit you. Or at least give you bragging rights.
So now you have all the reasons to start learning a scripting language. Which one should you pick? I'll look at different ones next time to help you decide which one is right for you.
ActiveX is still around!
Posted by fred.bertsch in Software at 07:33 | Friday, July 27. 2007
ActiveX has been around a while. When Microsoft was battling Netscape, they needed a way to put custom, active content on web pages. Java was being used by Netscape, and people thought it was great. Microsoft needed something they could develop quickly that would let programmers put new types of content on the web browser. ActiveX was born.
The basic idea behind ActiveX is really simple. A programmer creates a DLL that can be accessed by anyone. Some introspection is added, and now a web browser can call native code! Once you're in native code, you can do whatever the heck you want, so Microsoft's work was done. Of course, Microsoft added a bunch of ways to make it complicated, but the basic architecture is extremely simple.
That was back when Bill Gates thought that no one would pay money for security. The security model Microsoft used was also extremely simple: All ActiveX DLLs are signed. If someone hijacks thousands of computers using your DLL, then Microsoft will know who's responsible!
Of course, Microsoft signed some DLLs that had some big holes in them. In fact, lots of legitimate companies did. For the next decade and a half, a whole team of Microsoft employees dealt with the consequences of these design decisions.
ActiveX is still here, though.
Even in Vista, ActiveX is still available. It's still possible to run whatever code you want in Internet Explorer under Windows Vista. If you don't believe me, go over to your Vista machine and head over to this URL: Microsoft Windows Update. This page can replace your drivers and reboot your computer!
So, what has Microsoft changed? Is it still business as usual in the land of ActiveX? No, it's not. A lot has changed.
First of all, enough warnings pop up around an ActiveX control that both programmers and users avoid them like the plague. Back in the early days, programmers were supposed to put UI widgets on the browser window because Microsoft said it was easier to do it that way than by using HTML. (This conveniently prevented the page from loading under Netscape, so no one actually took this advice.) Now, almost no one makes ActiveX controls. Once you've got some video players, Flash, and a few others, you're done. No one else has to write them anymore! Certainly no one has to write one that requires administrative access to the computer. Once Windows Update was finished, the designers probably concluded that that was all you needed.
There's very close to no documentation on the subject, but it's still possible to have your ActiveX control run as an administrator. The strange part is that now, instead of being a mainstream programmer, you have to put on your dark sunglasses and visit some very murky areas. Microsoft won't tell you exactly what to do, but they do put clues in a variety of blog postings and tech notes. Bugs will haunt you as you make your way toward what you need, and you'll never really know if you're exploiting the OS or doing it correctly.
It's amazing what's changed. It's even more amazing how little has changed.
The basic idea behind ActiveX is really simple. A programmer creates a DLL that can be accessed by anyone. Some introspection is added, and now a web browser can call native code! Once you're in native code, you can do whatever the heck you want, so Microsoft's work was done. Of course, Microsoft added a bunch of ways to make it complicated, but the basic architecture is extremely simple.
That was back when Bill Gates thought that no one would pay money for security. The security model Microsoft used was also extremely simple: All ActiveX DLLs are signed. If someone hijacks thousands of computers using your DLL, then Microsoft will know who's responsible!
Of course, Microsoft signed some DLLs that had some big holes in them. In fact, lots of legitimate companies did. For the next decade and a half, a whole team of Microsoft employees dealt with the consequences of these design decisions.
ActiveX is still here, though.
Even in Vista, ActiveX is still available. It's still possible to run whatever code you want in Internet Explorer under Windows Vista. If you don't believe me, go over to your Vista machine and head over to this URL: Microsoft Windows Update. This page can replace your drivers and reboot your computer!
So, what has Microsoft changed? Is it still business as usual in the land of ActiveX? No, it's not. A lot has changed.
First of all, enough warnings pop up around an ActiveX control that both programmers and users avoid them like the plague. Back in the early days, programmers were supposed to put UI widgets on the browser window because Microsoft said it was easier to do it that way than by using HTML. (This conveniently prevented the page from loading under Netscape, so no one actually took this advice.) Now, almost no one makes ActiveX controls. Once you've got some video players, Flash, and a few others, you're done. No one else has to write them anymore! Certainly no one has to write one that requires administrative access to the computer. Once Windows Update was finished, the designers probably concluded that that was all you needed.
There's very close to no documentation on the subject, but it's still possible to have your ActiveX control run as an administrator. The strange part is that now, instead of being a mainstream programmer, you have to put on your dark sunglasses and visit some very murky areas. Microsoft won't tell you exactly what to do, but they do put clues in a variety of blog postings and tech notes. Bugs will haunt you as you make your way toward what you need, and you'll never really know if you're exploiting the OS or doing it correctly.
It's amazing what's changed. It's even more amazing how little has changed.
Wobbly polarity is key to preventing magnetic avalanches on disk drives
Posted by ken in Hardware at 07:40 | Wednesday, July 25. 2007
This article Magnetic Wobbles Cause Disk Failure may be a good way to improve our hard drive diagnostics, especially as drives get larger. It did get me to thinking that the conventional wisdom that hard drive defects will tend to cluster radially under the head as seen in a head crash may not be the whole picture. Perhaps spiraling outward from a defect is another good approach.
UCSC Press Release on Wobbly Polarity
Typical Crash:
UCSC Press Release on Wobbly Polarity
Typical Crash:
Bandwidth Envy, the State of our Nation
Posted by Chris Hill in Grab Bag at 09:32 | Monday, July 23. 2007
[display_podcast]
As reported by Foxnews.com, a Swedish woman, age 75, has the world's fastest broadband internet connection. In less than 2 seconds Sigbritt Lothberg can download a full-length movie. How does she do it? She's tapped into a 40 gigabit-per-second connection in Karlstad, Sweden. Seriously! Me and my puny 5 Mbps connection sit in awe and envy. Sure, I can get upwards of 10 Mbps if I want to pay for it. What's even more disturbing is that she's not utilizing the connection, only using it to view online newspapers. Sad, very sad.
"The speed is reached using a new modulation technique that allows the sending of data between two routers placed up to 1,240 miles apart, without any transponders in between." This does raise a question. What's the deal with American ISPs? Broadband coverage throughout America still isn't what it should be and speeds for the most part are no where near what's available throughout Europe and parts of Asia. According to a CWA survey, and as reported by Ars Technica, the average broadband speed is a mere 1.9 Mbps. Consider this, it's 61 Mbps in Japan, 45 Mbps in South Korea, 18 Mbps in Sweden, and 17 Mpbs in France.
It's no secret, I'm not an advocate of big government and government regulation, but when it comes to R&D and advances in technology, I'm typically all for it. As an example, there was legislation introduced this year titled the Broadband Census of America Act. Here's what the bill would do:
- Define "high-speed" Internet access as 2 megabits per second for downloading and 1 megabit per second for uploading. Currently, the Federal Communications Commission (FCC) defines high-speed as 200 kilobits per second.
- Discontinue the FCC practice of claiming an entire US zip code has broadband access if just one resident in that zip code has access.
- Require the FCC to compare broadband deployment with that of other nations.
- Requite the National Telecommunications and Information Administration to provide a comprehensive nationwide inventory of existing broadband service and infrastructure.
- Allow the NTIA to make grants to states and local governments to assist the NTIA in collecting the data needed for the broadband availability map.
- Direct the FCC to conduct surveys on the prices, advertised and actual speeds, and availability of broadband services.
- Authorize twelve million dollars per year in fiscal years 2008, 2009 and 2010 to carry out the provisions of this legislation. No less than eight million dollars per year would be reserved for the grants to state and local governments.
This sounds like a great start, but I'd still like to see more. What about the price gouging that American ISPs are inflicting on Americans every year for what they like to tag as "broadband"? We pay more for slower speeds than many other countries. How can America compete in the global economy and in R&D if our networks and infrastructure simply aren't up to snuff?
Larry Cohen, CWA President, has it right:
"Speed Matters on the Internet. It determines what is possible; whether we will have the 21st century networks we need to grow jobs and our economy, and whether we will be able to support innovations in telemedicine, education, public safety, and public services to improve our lives and communities. High speed Internet could even help address the global warming crisis by allowing people to get things done without getting into their car."
With all this in mind we're approaching an election year with many items on the agenda to focus our attention on; a war on terror/Iraq war (arguably the same thing), Iran's nuclear program, North Korea's nuclear program, immigration, health care, social security reform, foreign oil dependency, budget deficits, trade gaps, and so on. All of those things listed are extremely important issues that we must face as a nation. However, we must not lose site of what was once a nation capable of great innovation such as the Industrial Revolution. Congress and our next presidential candidate must be one that can help re-invigorate old school American Research & Development. We need to put ourselves back on the map of innovation. This new innovation will help foster solutions to many of the other problems that I previously mentioned.
What is usability testing all about?
Posted by fred.bertsch in Software at 15:55 | Wednesday, July 18. 2007
The phrase usability testing gets thrown around a lot. It sounds great when you're planning a project. If you say you'll do some usability testing, then people get a warm feeling about your project plan.
After discussing it with a few people, I've concluded that there are a lot of myths out there about usability testing. I'll outline all of them that I've either heard from someone or thought to myself.
First let me explain what it is.
No. I lied. That's a huge topic, and I'm going to bypass it here. Instead, I'll refer you to the best summary I've read on the subject: Dumas and Redish, A Practical Guide to Usability Testing. It's a bit old, and there are likely better summaries out there, but linking it nicely lets me avoid explaining what usability testing is.
I will, however, explain what the goals of usability testing are:
That description is designed to shoot down several of the bigger myths that I've run into.
Myth: Usability testing gathers statistical evidence that you can use to make decisions about solving a usability problem.
Running a good usability test on a single user is expensive and time consuming. There's a lot of data that gets analyzed, and I've found it to be a heck of a lot more useful to get more data about a single user than it is to test multiple users.
You end up running tests on very small numbers of people. I generally spend a fair amount of time before and after a single test preparing the test and analyzing the results. The test will find some problems, but you correct those problems before you analyze how bad the problems were. If a problem is bad, then you're likely to run into it. If a problem is small, you're not likely to be bothered by it again. It's much better to just assume that all problems you run into are large enough that they should be solved.
All forms of usability testing that I've done follow this formula: Watch a single user use your product. For every problem that user runs into, figure out what caused the problem, and decide if and how you need to fix it. All problems a user runs into are considered real problems until proven otherwise.
Myth: Usability testing requires a lot of fancy equipment.
I hear this sometimes after people read about what large software companies use when they do usability testing. I've never seen it, but I've heard rumors about rooms filled with hidden cameras, one-way mirrors, and eye tracking devices. It sounds like fun, but it's really not needed.
When I do some usability testing, I try to understand the user as well as possible. I want to know what they're thinking when they click the wrong menu item. I tend to be in their face a lot more than someone standing behind one-way mirrors would be, but, for a lot of problems, a user being in an artificial environment with a couple of engineers breathing down their neck isn't as bad of a problem as it sounds. Certainly, a lot can be done this way.
Myth: Any old user will work.
The background of your user has a huge impact on how they view your product.
In the early stages of testing, I like to use coworkers as much as possible. They're easy to sucker into these tests; the first few times you do it, they even think it's fun! However, the data you generate from these tests requires so much interpretation that you can quickly get to the point where it's not worth your time. (They're great for the early tests, though. If nothing else, they can help test your testing procedure!)
Myth: Usability testing has to take a long time.
Myth: Usability testing can be done very quickly.
These myths are different, but the answer is the same. Usability testing can take as long as you need it to. It's fairly difficult to anticipate how long you'll need, however. Unexpected usability problems frequently come up. They need to be fixed in code, and if you're not lucky, that might take time.
If the dialog box or web page that you're testing isn't all that important, then it's possible to run tests that will fail to catch smaller problems. This can greatly speed things up, and it is possible to test something quickly and get away with it.
One of my favorite easy test to run is one that tries to decide between a small number of completely different approaches. This can be quick, but it's also possible that none of the approaches tried works all that well. I haven't been great at predicting the time required for usability tests that I've done, and I claim that it's a problem with usability testing rather than my own shortcoming.
One of the more dangerous forms of this myth is the belief that a large, complex product can be tested comprehensively in an amount of time that management will be happy with. Usability testing of a significant amount of UI code is a major project, and it really needs to be done continuously over the entire lifecycle of the product.
Myth: There aren't any other myths.
As I find some more time, I'll come back and address some more issues that I've run into. I think I've gotten the biggest ones that I've seen, though.
After discussing it with a few people, I've concluded that there are a lot of myths out there about usability testing. I'll outline all of them that I've either heard from someone or thought to myself.
First let me explain what it is.
No. I lied. That's a huge topic, and I'm going to bypass it here. Instead, I'll refer you to the best summary I've read on the subject: Dumas and Redish, A Practical Guide to Usability Testing. It's a bit old, and there are likely better summaries out there, but linking it nicely lets me avoid explaining what usability testing is.
I will, however, explain what the goals of usability testing are:
The idea is to watch your users interact with your product (or something similar to your product) in a way that allows you to see how well the product works for the users. In addition to finding problems, usability testing also tries to gather data that the testers can use to figure out how the problems should be corrected.
That description is designed to shoot down several of the bigger myths that I've run into.
Myth: Usability testing gathers statistical evidence that you can use to make decisions about solving a usability problem.
Running a good usability test on a single user is expensive and time consuming. There's a lot of data that gets analyzed, and I've found it to be a heck of a lot more useful to get more data about a single user than it is to test multiple users.
You end up running tests on very small numbers of people. I generally spend a fair amount of time before and after a single test preparing the test and analyzing the results. The test will find some problems, but you correct those problems before you analyze how bad the problems were. If a problem is bad, then you're likely to run into it. If a problem is small, you're not likely to be bothered by it again. It's much better to just assume that all problems you run into are large enough that they should be solved.
All forms of usability testing that I've done follow this formula: Watch a single user use your product. For every problem that user runs into, figure out what caused the problem, and decide if and how you need to fix it. All problems a user runs into are considered real problems until proven otherwise.
Myth: Usability testing requires a lot of fancy equipment.
I hear this sometimes after people read about what large software companies use when they do usability testing. I've never seen it, but I've heard rumors about rooms filled with hidden cameras, one-way mirrors, and eye tracking devices. It sounds like fun, but it's really not needed.
When I do some usability testing, I try to understand the user as well as possible. I want to know what they're thinking when they click the wrong menu item. I tend to be in their face a lot more than someone standing behind one-way mirrors would be, but, for a lot of problems, a user being in an artificial environment with a couple of engineers breathing down their neck isn't as bad of a problem as it sounds. Certainly, a lot can be done this way.
Myth: Any old user will work.
The background of your user has a huge impact on how they view your product.
In the early stages of testing, I like to use coworkers as much as possible. They're easy to sucker into these tests; the first few times you do it, they even think it's fun! However, the data you generate from these tests requires so much interpretation that you can quickly get to the point where it's not worth your time. (They're great for the early tests, though. If nothing else, they can help test your testing procedure!)
Myth: Usability testing has to take a long time.
Myth: Usability testing can be done very quickly.
These myths are different, but the answer is the same. Usability testing can take as long as you need it to. It's fairly difficult to anticipate how long you'll need, however. Unexpected usability problems frequently come up. They need to be fixed in code, and if you're not lucky, that might take time.
If the dialog box or web page that you're testing isn't all that important, then it's possible to run tests that will fail to catch smaller problems. This can greatly speed things up, and it is possible to test something quickly and get away with it.
One of my favorite easy test to run is one that tries to decide between a small number of completely different approaches. This can be quick, but it's also possible that none of the approaches tried works all that well. I haven't been great at predicting the time required for usability tests that I've done, and I claim that it's a problem with usability testing rather than my own shortcoming.
One of the more dangerous forms of this myth is the belief that a large, complex product can be tested comprehensively in an amount of time that management will be happy with. Usability testing of a significant amount of UI code is a major project, and it really needs to be done continuously over the entire lifecycle of the product.
Myth: There aren't any other myths.
As I find some more time, I'll come back and address some more issues that I've run into. I think I've gotten the biggest ones that I've seen, though.