OP here. I quit my full-time job almost a year ago to travel and work on startup ideas, and this is the fruit of my labor in the past six months that I hope you'll find useful. I wanted to make it easy for startups and indie devs to test their iOS applications. How does testelf work? You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective, logging what we did or saw. The goal is to charge a flat price per test, provide good coverage and actionable feedback, and have a 36-hour turnaround time.
Over the past few months, I recruited and trained a small team of testers, developed internal testing tools, and worked with a handful of startups who are piloting our service. The service is now open for beta -- I would love to hear feedback from hackers and startups on HN who can give it a shot.
"You upload a build of your app without providing any additional instructions, and we'll test your app from a user's perspective..."
It seems like a user should have at least some idea of what the app is. For example, I'll read the first couple sentences of an app's description and look at some screenshots/video before I decide to download it. This might be something you want to consider instead of having absolutely no idea what the app is before running it, since I assume most users have at least some context when opening an app.
This is a pretty cool idea, and something I would have considered trying to get my boss to pay for when I was a developer. I didn't see on your website - do you test on a wide variety of hardware platforms, or just iPhone 5 and iPad? Different versions of iOS?
Also, do you have any terms, in particular, regarding confidentiality?
Congrats, looks really nice! (already forwarded to our testing team)
If you add a 'proxy layer' to your solution, that would be awesome. To explain the scenario, it is really very valuable for us app startups to see how it behaves in China, in India, in Europe, in US etc. You may keep your current system intact, and 'if' the customer desires so, you can route your connection over those proxies(from real ISS'es) from all over the world and test it 'as if...'
This is really important and not so hard to implement... But sometimes, you can catch bugs and usability horrors with slow or dropped connection or some weird proxy mambo jambo played by ISS providers.
Also, would be very interested to know if you have plans as to how (or if) you will overcome the issues you would have if you wanted to scale such a bespoke service?
Thanks for the comment! Android is probably the next platform for us if all goes well. I started off with iOS because I'm more familiar with it and it has less device variations. Extending the offering won't be trivial though. Before we start support for a new platform, we'd need to build the testing tools, develop new training materials, and acquire new hardware.
Looks great, i run a mobile app agency and we are forever having issues due to developers not testing adequately, having some external company test would be really useful!
This is a great idea sir. I signed up and hope to hear from you guys. I can use more eyeballs looking over our work. I really like not having to worry about provisions.
You can sign ipa files with your developer (private) key. There's an app called InstaSign that I use for "testing" apps before purchasing them. You download a cracked version, sign it with your own key (so iPad thinks YOU have created this app, like when you do in Xcode) and see if it's worthwhile.
It's saved me a lot of money that otherwise would've wasted on scammy, buggy, "not-working-as-advertised" apps. If you're a developer (i.e., can sign apps and load them unto your device), give InstaSign a try.
They're probably using a similar method (as they seemingly test on iPad/iPhone, retina/non-retina, iOS5/6 devices and it might be a bit more work for you to add at least 6 new UDIDs).
[+] [-] ssong|13 years ago|reply
Over the past few months, I recruited and trained a small team of testers, developed internal testing tools, and worked with a handful of startups who are piloting our service. The service is now open for beta -- I would love to hear feedback from hackers and startups on HN who can give it a shot.
[+] [-] matthuggins|13 years ago|reply
It seems like a user should have at least some idea of what the app is. For example, I'll read the first couple sentences of an app's description and look at some screenshots/video before I decide to download it. This might be something you want to consider instead of having absolutely no idea what the app is before running it, since I assume most users have at least some context when opening an app.
[+] [-] jmatthew3|13 years ago|reply
Also, do you have any terms, in particular, regarding confidentiality?
[+] [-] kevining|13 years ago|reply
[+] [-] umut|13 years ago|reply
If you add a 'proxy layer' to your solution, that would be awesome. To explain the scenario, it is really very valuable for us app startups to see how it behaves in China, in India, in Europe, in US etc. You may keep your current system intact, and 'if' the customer desires so, you can route your connection over those proxies(from real ISS'es) from all over the world and test it 'as if...' This is really important and not so hard to implement... But sometimes, you can catch bugs and usability horrors with slow or dropped connection or some weird proxy mambo jambo played by ISS providers.
[+] [-] cstrat|13 years ago|reply
[+] [-] thomseddon|13 years ago|reply
Also, would be very interested to know if you have plans as to how (or if) you will overcome the issues you would have if you wanted to scale such a bespoke service?
[+] [-] timdorr|13 years ago|reply
[+] [-] ssong|13 years ago|reply
[+] [-] gmig|13 years ago|reply
[+] [-] akaru|13 years ago|reply
[+] [-] speeder|13 years ago|reply
Well, I hope we can send our more non-gameish apps :)
[+] [-] king_magic|13 years ago|reply
Definitely going to consider this for future testing :)
[+] [-] swanify|13 years ago|reply
[+] [-] ianstallings|13 years ago|reply
[+] [-] PanMan|13 years ago|reply
[+] [-] pooriaazimi|13 years ago|reply
It's saved me a lot of money that otherwise would've wasted on scammy, buggy, "not-working-as-advertised" apps. If you're a developer (i.e., can sign apps and load them unto your device), give InstaSign a try.
They're probably using a similar method (as they seemingly test on iPad/iPhone, retina/non-retina, iOS5/6 devices and it might be a bit more work for you to add at least 6 new UDIDs).
[+] [-] tachim|13 years ago|reply
[+] [-] thelarry|13 years ago|reply
[+] [-] jplehmann|13 years ago|reply
[+] [-] eclipxe|13 years ago|reply
[+] [-] hariseshadri|13 years ago|reply
[+] [-] shawiz|13 years ago|reply