top | item 5298663

Why are there not more freelance testers out there?

56 points| rosiesherry | 13 years ago |testninjas.com | reply

58 comments

order
[+] scott_w|13 years ago|reply
From my personal experience, I wouldn't use a freelance tester because they won't understand the system well enough to provide value for money.

The most valuable testing is that which exposes your lack of understanding of what the system should do. These also usually require the most extensive rewrites, so you definitely want them as early as possible. Unfortunately, they also require the most extensive understanding of the system as well as an ability to detail their thinking at a level that can be understood by the development team(s).

That's why I don't see the value of freelance testers, and I'd much rather have somebody employed on a more permanent basis, either part-time or full-time.

[+] SODaniel|13 years ago|reply
As true as your comment is I would like to point out that there is a value in itself to have 'fresh eyes' on QA. An experienced and talented tester unfamiliar with a system/product/service can actually work as both QA/UI and UX resource if he or she has the right background.

TLDR; Sometimes you can gain valuable input from a fresh pair of eyes.

[+] nicolasedgwick|13 years ago|reply
The question then is how do you assess value for money before you use the service. I work as a freelance tester and I often see that customers see the value after working with me and then return for repeat business (as someone else has mentioned - you build a reputation).

As an example, an error in an iPad app was dismissed as a known connection issue by the dev team because there were no server side logs so the request wasn't hitting the server. This was the dev team's first iOS app. My specialist knowledge of mobile testing helped identify quickly that the app was connecting to the servers but that this one request wasn't and provide console logs showing where the communication error was occurring.

This customer were to launch in public the following week and this would have killed their reputation due to the critical nature of the function that was failing. This was my first day testing with them after they had put in a year of development effort and dev testing (they brought me in specifically because they felt they were getting "tunnel vision"). This was one bug masking behind another which I spotted due to 3 years experience in testing mobile apps. I think that provides value for money without being employed permanently, however I appreciate that companies/teams don't see that value until it's their reputation that has been protected by the investment.

[+] lifeisstillgood|13 years ago|reply
That's the same argument for not hiring freelance developers or project managers. And IMHO it leads to lazy coupling of software systems and services, a system that has to be built and tested by different or changing groups has to have much much better scope and decoupling.

In short, if you have small (micro) services, they are easy to build and easier to test.

[+] Too|13 years ago|reply
There's a big difference between blackbox testing and whitebox testing. For blackbox testing, having someone who doesn't now the inner workings of the system to find workarounds can many times be an advantage.

Of course you should do both kinds for security critical projects.

[+] bpatrianakos|13 years ago|reply
I think there's much more to it than what the post says. It seems the author is looking at the freelance testing industry (is 'industry' the right word here?) through rose colored glasses.

Freelance web designers and SEOs are everywhere because you don't need any experience and often times don't even require any actual skill to do the job. All you need to know is how to repeat the whole "Social media marketing your business reach customers user experience blah blah nonsense" spiel and people say "Yeah! I want to miraculously reach the top of Google for everything and make a billion online like Zuck" and hire some dude for $300 who read "HTML and CSS for Dummies" last week at a Barnes and Noble. I know this because I used to be that asshole (not the business owner, the freelancer).

Other kinds of web/tech freelancing have very simple-to-understand values. In the minds of a lot of people who hire freelancers, Web design = Branding or Marketing = Profits. Simple. Not exactly correct, but that's the line being fed to everyone these days. Same with SEO. SEO = Rankings = $1B profit.

Testing is different though. I have to say however that if a company even needs to test software to begin with then they're probably playing at a higher level than the companies hiring freelance designer/devs and SEOs. Still, the value of a tester is still nebulous. You don't hear freelance testers out there drilling "Testing will make your customers love to use your product, increase efficiency on yours and the customer's end, and save you a ton of money" or something equally easy to market. To most businesses testing means "we played with the app on the staging server and it worked fine for us. time to push it live". It just comes off as an extra expense that isn't worth the investment.

And why is that? Because freelance testers, unlike other freelancers, have failed to create a consistent, marketable appeal to business owners (i.e. tell them why it's going to make them tons of cash) the way other freelancers have. And it's probably because those who do testing are a little more thoughtful and less about pizazz and flash than designer/developers and the nature of testing itself is kind of boring unless you're really into it yourself. Design is fun because everyone wants to be an artist. SEO is fun because people are given the idea they're in control of which keywords they'll pop up for and such. Testing? Totally boring. Some guy's gonna come in here and find bugs in our software? We can do that ourselves or just make sure to mark all the complaint emails as TODOs in Gmail.

[+] ssong|13 years ago|reply
I think you make a great point about perceived value of testing vs SEO/design. However, with the increased proliferation of downloading software through app stores, which favors apps with better reviews/ratings, the perceived value of testing should increase. For example, take a look at apps with mixed or low ratings on the AppStore and most of them will contain complaints about bugs in the app.

(random point + small shameless plug: I founded http://www.testelf.com as flat priced testing service for iOS apps)

[+] mgkimsal|13 years ago|reply
you're very rarely going to 'make' someone money - you're going to save them money. A well-tested app doesn't necessarily out-sell the competition if the competition gets a better distribution deal through 'good old boys' networking. Demonstrably bad software often becomes dominant in specific niches, and people put up with the bugs, because "it's what everyone uses".

Your well-tested software still has a huge marketing battle and possibly integration issues to replace incumbents, regardless of how many bugs they have. You end up needing to create a negative campaign about the headaches and hassles of using XYZ, and that you don't have those issues (because it's well-tested, but you generally can't promote that to most people, as it's 'too technical').

[+] kenjackson|13 years ago|reply
There is one niche area of testing where it is more common... security testing.

Testing in this space is often more a function of the technology stack than the industry of the client. This allows security test companies to build up great expertise without knowing anything about the client's specific app or industry.

[+] 3am|13 years ago|reply
I wouldn't even consider penetration/security testing to be in the same field. My experience in the traditional QA areas is that in order of likelihood to freelance are performance testers, automation/development testers, and finally manual testers.
[+] 3am|13 years ago|reply
I don't know if uTest is still around, but they attempted to fill a similar niche. If you are in a testing lead position, then you know that an enormous amount of contracting is arranged via recruiters to fill this need also.

I think the reasons it's really hard for freelance testing communities to get traction are

1. real or perceived lack of return - you have a 2-3 month engagement and realistically the first month is training and acclimatization (hence wasted). better off hiring a FT resource.

2. ample market supply - there are plenty of people looking for FT roles. I'm a testing manager in a secondary technology city in the US, and it's no trouble to get resumes.

3. limited scope where they're useful - I would consider freelancing or hiring a freelancer that specialized in a particular tool (Selenium, Robot, Cucumber, QTP, etc) if building out a testing framework or conducting major refactoring but otherwise consider testing an iterative process.

4. This is a bit of a bootstrapping problem, but it would need a (much) better version of Angie's List. Because of the reasons I listed before, this would be a high risk decision and most people would be very reluctant to go with an unknown quantity. There would have to be a substantial number of verified ratings and reviews. [EDIT: this is also why so much contract testing is arranged via recruiters - the recruiter wants repeat business. A bad referral can poison a relationship, so you have a strong element of trust]

[+] wh-uws|13 years ago|reply
Also I feel like smaller shops that do test would rather write their own automated tests (we are all developers after all) until they get enough traction to afford a fulltime tester
[+] cantlin|13 years ago|reply
We've needed a crowd-sourced human testing platform for software products for some time.

It would need granularity: as a developer I should be able to purchase N minutes of time from N people, potentially in such and such a demographic.

It would need an API: I should be able to build human testing into my deployment infrastructure.

It would need accountability: at minimum I want webcam footage of the tester using my product.

There are complications, but I see no reason why almost all of these couldn't be overcome by sufficiently smart automation.

Someone go build this!

[+] anandkulkarni|13 years ago|reply
We actually do this at MobileWorks (mobileworks.com). There's an API that lets you get N minutes from N people filtered by demographic. Some folks use our Premier product for on-demand testing set up by email -- they send test requests to [email protected].

If you're looking for this more as a formal end-to-end service, you can take a look at our buddies at Rainforest (https://www.rainforestqa.com/). You can also look at services like uTest, though they're real pricey.

[+] darkmethod|13 years ago|reply
If https://www.usertesting.com had an API, I think it would handle the majority of the points you listed.

UserTesting may already have one, however my brief two minutes of poking around didn't reveal that they do.

[+] bengl3rt|13 years ago|reply
Some friends of mine have actually made a business out of this over at http://testloft.com

From what I understand the phone won't stop ringing so clearly there is strong demand for something like this out there.

[+] jot|13 years ago|reply
There's less demand for freelance testers because having a one is a nice to have rather than a must have.

Clients rarely understand the value of testers, they appear to simply be an optional additional cost. Freelance developers have little incentive to recommend them since it inevitably leads a smaller slice of client budget for them.

When an organisation gets to the stage at which it understands the value of, and develops a need for, a tester they probably have enough work and cash to occupy one full-time.

[+] manaskarekar|13 years ago|reply
How about 'Disaster Management/Save Face' freelancers?

To cater to the companies that thought testing was an additional expense and didn't go for it.

[+] nicolasedgwick|13 years ago|reply
Wouldn't be great if freelance developers saw the benefit of a freelance tester that they worked well with and the two of them marketed themselves as a joint enterprise for even better customer value.
[+] obviouslygreen|13 years ago|reply
Yep. As a web development shop, finding a client that understands or can be convinced of the value of any testing is rare.
[+] PaulHoule|13 years ago|reply
I'd say the psychology and politics of testing are different from other things, really quite strange.

It most white collar work, there is strong pressure to tell people what they want to hear. Well, it makes me very happy to get a clean bill of health from a tester, but if I was expecting that to happen, I wouldn't need a tester.

Testing takes a kind of special discipline that the average office worker doesn't have. Whenever I've been an organization that has tried to press ordinary people into a tester role, it's always been highly ineffective.

That said, there isn't one formula to make a good tester. Some testers I know are very systematic, they belong to quality control associations, others are expert in project management, requirements, etc. On the other hand, one of the best testers I worked with had bombed out of every other job we gave him because he was always able to screw up anything we gave him. As a tester, he was great, because if he couldn't break it, nobody else could.

All of these things, I think, point to a need to a longer-term relationship with a tester than you're going to get in a freelance situation.

[+] nwh|13 years ago|reply
This is advertising for a service that CHARGES people to sign up and test websites. The blog post serves no purpose other than to funnel traffic to their domain.
[+] patja|13 years ago|reply
Testers are mostly a labor commodity. Seems like the body shop contracting agencies and big offshore firms (Wipro, TCS, etc.) have this market cornered. Any potential client who really values having a dedicated testing team will build up the domain knowledge in house rather than re-educate freelancers over and over, or require the body shop to build that knowledge base and do that training for them when bringing on a new tester.

And how often do you really need just one tester? What are you going to do, bring in 10 different freelancers?

Freelancers need to wear more hats than just "tester".

[+] testerfromleic|13 years ago|reply
TL;DR - we (testers) need to tackle these problems head on ourselves rather than wait for people to "get it".

It seems to me, a lot of folks here wouldn't be adverse to using a freelance tester in general, but are extremely adverse to using bad freelance testing (and have the scars to prove it!).

I'm a contract tester and recently decided to make the transition to freelance. From my experience (and confirmed by the comments here) the main barrier is one of perception - how can testers / add value to my organisation? And how do I know this tester is as good as they claim?

There are numerous ways someone like me can help an organisation:

Help organisations find the right permanent employees; Mentor said permanent employees by sharing knowledge; provide testing experience to a known or unknown business domain; supplement the clients test lab with your own equipment; evangelise testing throughout the company, demonstrate what good testing can do, what it looks like

I had to stop myself there! And pretty much all of those examples are from recent (within 12 months) history. Luckily for me I've been able build a decent enough reputation that there is potential for repeat business.

Two more key problems: first, the kind of testers that people here need may live in a bubble where everyone knows the value of testing and as such may not realise the rest of us exist - my colleagues and I need to start dispensing some red pills :). Second, the testers that are trying to be better appear to be outnumbered by the ones that aren't (snake oil merchants, testing zombies, call them what you will) so again we need to tackle that head on too.

Nice discussion. Thanks for the illuminating comments folks.

[+] lutusp|13 years ago|reply
> I think freelance software testers should be easy to find. But they are not. I think there should be loads of good freelance testers out there. But there are not. I think it should be easier to find freelance testers. But it is not.

There's a reason -- software developers have figured out that they're better off just releasing the product and hearing from their customers. It's called "beta testing" -- no special panel of testers, just users eager to try a new product, willing to put up with imperfect operation for the privilege.

And not all beta tests are identified as beta tests -- many of them appear to be a product release, but the real purpose is to wring out the bugs. The classic example is Fedora Linux, which is one very long beta test (and one that most of its users understand).

As to Web sites and online games, developers use A/B testing to accomplish the same ends, without the need to locate people whose special role is to test.

[+] libovness|13 years ago|reply
I've gotten good results through uTest (for bugs) and extremely good feedback from usertesting.com. Neither is cheap though.
[+] pasbesoin|13 years ago|reply
Testing is, in the large, not respected.

When you have a really good, competent, creative tester, the contribution can be invaluable.

But... many organizations tend to rank testers somewhere around the janitorial staff.

(By the way, get incompetent janitorial service, and watch what happens to your work environment... And, personally, I've found many of them to be plenty competent, caring, and helpful -- better people than many of the suits.)

Then, there is the other side of testing... It does exist. People who couldn't hack (aka "handle") any more, and who really are more or less button pushers. They are the ones who provide "QA signoff" as opposed to real troubleshooting.

It can be difficult to argue your value, when much of your field is a morass on two axes: Lack of respect, and lack of competence.

[+] TeeDub|13 years ago|reply
What do you mean when you say tester? If you mean "someone to run manual test cases," it is probably because those sorts of jobs are pure tedium... They can be outsourced to websites too.

If you mean a software engineer that designs software to validate that other complex software systems function (often dealing w nontrivial engineering efforts and undecidable problems) , they probably don't freelance because we denigrate them by calling them "testers" and they'll get hired fulltime by companies that understand the value of their work.

[+] geogra4|13 years ago|reply
I would love to be a freelance tester. I have no idea how I would start though.

I have a lot of experience testing desktop apps for my day job, but I'd love to start something on the side.

Anyone here have ideas on where to start?

[+] rosiesherry|13 years ago|reply
I'm bias, but helping people get their freelance career off the ground is what we are trying/doing at Test Ninjas.
[+] beezee|13 years ago|reply
The time it takes to walk someone through all the use cases that need to be tested is not trivial. The reason for having a dedicated tester is so that knowledge of the system accumulates. I can't imagine how having to re-introduce your system to a new tester for every upcoming deployment could possibly be a realistic approach. In order for this to make sense I'd have to build a relationship with a freelance tester so that the same accumulation of knowledge could take place.
[+] cantlin|13 years ago|reply
In some circumstances "accumulation of knowledge" in a tester is exactly what you're trying to avoid. Fresh eyes can be very valuable. Amongst other reasons, this is why we don't give developers sole responsibility for testing their own work.
[+] bluetidepro|13 years ago|reply
This is a great idea! I was actually just recently looking for something like this, to help with testing on an app.

I think my only concern would be security with those who are testing your app, because you would most likely need them to test use cases by giving them special accounts and such. Is there some sort of checks and balances for people who are testers on the site? Can you make the testers sign a NDA before they test (if the project happen to still be in BETA or something)?

[+] rosiesherry|13 years ago|reply
Thanks for the positive feedback :) The testers work independently and often sign NDAs at the moment I am working closely with the testers and clients as a match making kind of service.
[+] MadQA|13 years ago|reply
I actually run a small QA-as-a-service company (for 3+ years) and I should say that we strive to distance ourselves from "freelance" word as much as possible.

As most other cheap labors in a remote freelance mode — it is discredited. I think it would be just hard to build a personal brand strong enough to compete with tons of people, who can look like a diamond on a pre-sale phase and then just disappear.

[+] jonstjohn|13 years ago|reply
How has that worked out for you? Do you find it difficult to find clients? Much client turnover? Are you doing mostly manual or automated testing?
[+] Jmshabangu|13 years ago|reply
One of the reason is that people are scared to freelance for a month full time, and the next month not getting any job the next month, In SA we dont have lots of companies that are looking for freelancer.I believe its always good for a company to get new tester that can produce quality assurance into their systems.
[+] bdcravens|13 years ago|reply
Testing tends to be an intermittent need. Most apps that are to be tested tend to be off of the public Internet. I think that most orgs are unwilling to take the time to provide testers access they need (VPN access, etc) given the ephemeral nature of the relationship.