A customer recently asked me to look into a program that used to run in 5 minutes but now took 1 to 4 hours. It's used by thousands of people all over the world all day long.
It iterated through an array doing 3 SQL SELECTs against non-indexed files for each element. There used to be about 50 elements in the array; now there were more than 5000. I rewrote the whole thing in one day to do a total of 4 SELECTs and run in 12 seconds.
But it took 6 days to get through QA (while the users continued to suffer). QA's biggest complaint? I indented 4 spaces instead of the (unpublished) standard of 5 spaces.
Of all the things I have to deal with, nothing pisses me off more. Software QA is becoming more and more like TSA security at the airport: illogical, and obviously so. Last year, flagrantly unacceptable code was promoted without question while its replacement was held up on a meaningless detail.
I got the feeling from the programmer's quotes in this essay that the same thing is happening at the app store.
We programmers are a funny lot. Make us struggle for business or technical reasons and we adapt beautifully. Make us struggle for something stupid and we just get pissed off and do something else. What a pity.
I find when this stuff happens with QA teams, it's almost always a case of protecting turf.
When no one is looking (the first, buggy, submission is made), they will do the minimum necessary and approve it, often quickly.
Then, when everything goes to hell, all eyes are on them. The developers submit a fix with the note "this needs to be approved ASAP to put out fires"... the QA person resents being told what to do, and suddenly feels compelled to justify his job by finding anything and everything that could possibly be considered 'wrong'. There is also the issue that things like indentation are more visibly 'wrong' than obscure bugs that only show up in production environments due to the way database replication is set up.
Now, I'm not saying all QA people are like this. I've worked with some downright awesome QA people before. But awesome QA people don't reject things for spite, and usually don't stay in QA long before moving up.
Not that the indentation complaints are valid, but the fix should definitely go through QA while the users continue to suffer. While I'm sure it works on the developer machine, it is much better to have a program that takes a long time and works right than for it to give incorrect results to users, or make a different part of the application stop working.
If the feature was really critical and important, QA should find a way to expedite the verification, but it should never be skipped in order to ship the fix faster.
We programmers are a funny lot. Make us struggle for business or technical reasons and we adapt beautifully. Make us struggle for something stupid and we just get pissed off and do something else.
This is the best YC quote of all time. Thx, edwin!
Value - stuff should work after it's been through them
Costs that I've seen
- more stuff doesn't get done/fixed because of the delay time associated with QA
- things don't get scheduled because of lack of QA bandwidth
- additional specifications required in order to go through QA
- programmers / PMs worry less about catching bugs because they feel it'll come back from QA
When is it better to QA something rather than just measure when it's broken? Everything we do goes through QA but I'm sure most of it is a net loss.
Anyone have a good way of knowing what to QA and what to simply monitor?
You indented with spaces instead of tabs!? Just kidding of course, but the QA team might have been on the other side of that holy war. I'm going to have to start using your TSA parallel though... it's so painfully true.
Agreed. The last company I worked at we had some similar idiocy.
We had a backend process that connected out to a network and did some things. However, the network it connected to changed their protocol, so our backend process was broken - didn't do anything - completely non functional.
We created a fix, but had to wait for it to go through QA for ages. What's the worst that could happen? It doesn't do anything like the current version? It wasn't even a massive update. Maddening.
Part of Paul's reasoning depends on Apps being central to the iPhone experience the same way software is central to the desktop experience. I'm not totally sure this is the case. While apps are certainly a major component to the iPhone, are they really the major factor in end-user adoption? With the exception of games, how many killer iPhone apps are there that don't already ship with the phone? The phone shipped for an entire year without an App Store at all. I even read something not that long ago that said the majority of apps sit unused on people's phones after they get them.
Now, this doesn't mean that developers won't want a developer friendly phone, or that Apple is hurting their reputation with developers. I'm certain it has been frustrating to deal with the whole process. However, if 3rd party software isn't essential for the iPhone as a platform, then developer satisfaction moves down a bit in terms of importance in Apple's eyes.
Now, it's clear that games are major for the app store. Games are also a special class of software application that benefits disproportionately from having access to the hardware at a low level. They also have properties more similar to music and movies (incidentally, things the iTunes store is good at selling). Except for the simplest games, they require a lot of up-front design work and investment. It's rare that a released game goes through a ton of rapid iterations to "get it right". Games may still have bugs after release, but in general the functionality they are going to have is there on day one. They also ship with the final sound effects, music, artwork, etc... Games are also probably very unlikely to "duplicate existing functinoality" or any of the other cases where "normal" apps hit barriers. Perhaps Apple has (intentionally or unintentionally) created an environment optimized for game approval?
I'm certainly not defending the app store process, but I just wonder if the software development community has a disproportionate view of its importance to the iPhone as a whole.
There was a story on here (or somewhere) a while back about how 70% of iPhone apps go unused after the first month of purchasing. Couldn't agree more. AFAICT there is no killer 3rd party app for the iPhone. I'd posit that 95% of the iPhone's awesomeness is wrapped up in the following four (OEM) components: 1) Browser, 2) Maps, 3) E-mail / SMS, 4) Visual voicemail. Everything else, including games, is just fluff for most users. Relative to those four, I think the app store is a relatively small driver of adoption. I don't think a lot of people are that wedded to it, and I don't think it makes Apple a particularly large amount of money either. Which would explain why they continue to neglect it in spite of the vociferous scorn of a few programmers and pundits.
I too noticed the similarity to the console game development. Usually console game developers have to go through lengthy approval process of the platform provider. Still developers accept that (although grudgingly), since once approved and the released, that will be the end of the process; no more bug fixes; the team dissolves and everybody starts working on the next project. (PC games are different, and certainly network games are continuous development, but console games have this feeling of "done is done").
Part of it may be a byproduct of the way the console games distributed on ROMs or disks. That may change, for consoles nowadays have network connection and can get updates anytime. There's another part, however, that some type of games fits this "done is done" philosophy. Like novels or movies, which you don't expect them to be improved over time.
Well, I don't defend the app store process either, and I believe majority of software should be developed in the iterative process. It's just interesting that games may be a marginal area that has some peculiar properties.
You're sort of right, but that's true for all other platforms as well, so it doesn't really tell us anything about the state of app use on the iPhone platform.
The OS bundled ones are the ones people use the most. Across the board.
If I had to guess, I'd say third party apps are just as often or more used on iPhone than on, say, Windows and Mac OS X. Simply because, on iPhone, a lot of web apps out there are used via a special-purpose iPhone app rather than the web interface.
I think you may be right, but I have a couple misgivings.
The year that iPhone existed without the App Store, no smartphone competitors came close to replicating it's core functionality and user experience. I don't think that's quite true anymore, and so I think that the App Store is one of the iPhone's key advantages. Specifically, the iPhone didn't need the App Store to differentiate itself from RIM and WinMo in 2007, but I think it absolutely needs the App Store to differentiate itself from the Pre and Android in 2009.
I don't think there is one killer 3rd-party app on the phone that everyone needs to have, but there could very well be many third party apps that smaller niches need to have. For me personally, MLB At-Bat (live streaming of baseball games) and the Kindle reader are actually the two biggest factors keeping me from switching to Droid. Those applications could easily be ported to Android, but haven't been yet, and in both cases a third party holds the distribution rights for the content so it's not like me or another hacker could replicate those apps easily.
> While apps are certainly a major component to the iPhone, are they really the major factor in end-user adoption?
All the ads I've seen lately for the iPhone are of the "there's an app for that" variety. So Apple themselves seem to think that the apps are a critical factor.
That being said, the iPhone sold very well before there were any apps at all.
I know lots of people who bought in iPhone or or iPod touch because it could run some unique app. It was a different app in each case and in each case it was an app I personally saw no use for, but for them it was a killer app.
Any app that makes someone get an iPhone instead of something else is a "killer app" as far as Apple is concerned, even if it only sold a total of 27 copies and lost the developer a lot of money. So even if the iPhone doesn't have any one killer app, the sheer diversity of apps means there is probably a killer app out there for most people, and that is vital for Apple.
It may not have started out as central to the experience, but I think that it's starting to quickly become just that and Apple knows this or at least they want it to be. So far all the recent iPhone commercials I've seen only focus on the thousands of apps you can get for the iphone - not the iphone's "awesome" feature set. I was looking at a friend's iphone the other day and I saw page after page of 3rd party applications. My wife wanted an iPod Touch because she saw some applications she wanted. Another person I know bought an iPod Touch just so they can run a bird identification program. I don't think these are unusual cases. What's funny is that in the beginning, Apple didn't even want to provide an API for native applications. They wanted all developers to use the Web API. When developers balked at that idea Apple reluctantly laid out plans for native applications.
Developer relations are central to Apple as a whole. Apples needs iPhone developers, desktop developers, web developers and developers for whatever they do after the iPhone. If Apple creates hostile feelings in the developer community with its App Store behavior, Apple is going to have ongoing problems regardless of whether it needs third party developers at the moment.
Talk about it. I just got a rejection today for a major bugfix on my last update. The only change in the update is to use utf-8 encoding---one tiny change on one line of code. After 2 weeks, they tell me they can't approve the update because I'm using undocumented APIs in a couple places. Sure, OK, looks like I was using some methods that went from deprecated to undocumented... but that's exactly what's in the store right now... ugh. Meanwhile, users are, rightfully, pissed-off.
I'd say about 50% of the time my submissions get rejected, and always for a stupid reason like this; this is one of the better ones, actually. Will I stop developing? Probably not, because it's good money. But am I seriously considering moving to Android? You bet.
I have a simple "free" app in the iPhone App Store. It lets users browse "available" Web domains. One feature is a BUY button that forwards them to Godaddy.com, if they want to buy the domain.
I don't make any money from Godaddy.
The app has been approved for a while, but I wanted to patch one little bug, so I submitted a patch.
... And waited two weeks. Apple's testers found a bug that was in the original approved app, and asked me to re-submit.
... And I waited three weeks. This is classic: Apple rejects my app again because they want to test the domain "buying" feature on Godaddy.
It's a website! Not even my app! Not even a new feature.
Same thing here. We just got an update rejected for a line of code that calls an undocumented API within a function that was no where called in the program, dead code and they rejected the update that had an important bug fix that we emailed them about for an expedited review.
The expedited review request was accepted but in the end took 14 days to come back and got rejected for a piece of code that was never ran and has been there since day 1.
Talk about frustration, hell yeah.
But as everyone does, after getting over the anger and back to reality and business sense, what else CAN I do but only to resubmit since there lies a never seen before first of it kinds, an easy software distribution and payment system that many other platforms did not provide to small independent developers a year ago.
It's not about just supporting one SINGLE platform, that has never been the case. Just like making web apps run nicely in IE, Safari, Firefox, etc or even Windows, MacOS, Linux. As a developer, what matter is that I support the platform that has massive adoption/users. I will not drop iPhone and go Android, or any other platform. The only decision I will make is if there's enough users on the platform that I do care about to invest and develop for that platform. One day Android might be there and I see the possibility of developing for both Android and the iPhone.
I think it's worth pointing out that even Apple's own developers don't get the luxury of being able to push out bug fixes whenever they want. Point releases of the iPhone OS are often months apart, so good testing is not optional for them.
My vote for a truly free hand-held platform is the n900[1].
It's beautiful, clean and comes with with Debian based distro, Maemo.
My opinions about apple were never positive, probably because I'm only 25 and did not program in the 80's.
I've only saw apple as a manufacturer of shiny, well designed rich boy toys, but that is probably because I live in Brazil, where only rich people have it and they think they are 'computer people' because of it.
I was always more influenced by the open source community, and apple's record is far from shiny on that aspect.
I've got this baby on preorder, can't wait to see if Maemo lives up to the hype.
Personally I've never been an iPhone fan either purely because you can get phones with far, far better technology for the same price. iPhone wins hands down in UI right now (I still hate touchscreen-only phones though), but that really won't be an issue for long with Nokia/Samsung/SE/HTC working double time to improve their OSes.
I don't think the claim saying that "Apple doesn't understand software" is correct.
Apple do understand software.
You cannot release Mac OS X, iLife and other jewels without understanding software.
Also Mac developers jumping off the iPhone bandwagon are not jumping off the Mac plateform.
For the iPhone, Apple tried a different model that nobody really tried before.
They indeed have a broken store but the previous models on other platforms never worked. The most successful one before was for the PalmOS and it wasn't over the air.
However Apple always been an arrogant company. With their customers, their providers and their developers.
The climax of this arrogance has been reached with the iPhone eco system (certainly helped by the arrogance of the mobile phone operators).
And for many it is not sustainable.
But anyways, who really cares?
Apple can open the doors of the store overnight.
It wouldn’t have been the first time they do one thing making you beleive it was the only way and then suddenly change in the opposite direction.
That’s the strength of Apple: they can change.
Not only they can change but they usually know when, with what, for who and at which price.
I never realised the arrogance was a trade mark of this process though.
Note: I don't mean being evil by being arrogant.
My biggest problem is that it's a half-assed review process. Neither is it an open platform nor is it a relatively clear and thorough review process like you get on the gaming device platforms (Microsoft, Nintendo, and Sony).
I made three applications (only one that was popular - WiFinder) for the AppStore. Every time I interacted with reviewers, either by e-mail or phone, I got the feeling they had no idea what they had signed up for, and had not thought out beforehand how to deliver on the promise of only having quality applications in the AppStore. Which is a shame because it's an old problem.
Though I can't marshall any evidence to support this claim, it was specifically not my impression that Apple was a company pro developers loved working with before the iPhone.
Fully agree. I recall in particular reading John Carmack a few years back talking about how he was glad Apple wasn't as dominant as Microsoft, because they were far harder to work with.
I wish I could remember the quote well enough to search accurately for it.
Apple has always treated third-party developers with suspicion. The developer website is only accessible by creating an account and agreeing to their license agreement. They churn through and deprecate APIs comparatively fast. They will quite happily steam-roller over third-party developers by adding competing features to the OS:
http://en.wikipedia.org/wiki/Dashboard_%28software%29#Compar...
None of this is new, the only difference is that there's now a release bottleneck that requires Apple's approval, and there's a lot of developers new to the platform.
The thing is, this is part of Apple's culture for a reason. Steve is an obsessive control-freak when it comes to a great user experience, and third party developers are seen as a threat to that. Microsoft has always been a great company to deal with as a third-party developer - they value their ecosystem, put effort into free documentation and solid IDEs, and rely on outside developers for key applications. The trouble with this comparative openness is that you end up with 800 pieces of software written by different companies fighting it out on your desktop, leading to crashes and inconsistent UIs all over the place.
I think Apple is in danger of screwing up a great opportunity with the app store, but this way of operating is in their DNA and it won't be easy to change. Despite what the geek consensus might be, the record shows they've made a lot of money in the past whilst ignoring third-party developers.
Paul captured my whole feeling about the App Store matter, especially about buying Apple products. Now I feel like I'm doing something wrong. I still like the products but I no longer like the company.
Today my small team of developers submitted the first "toy" we released just to test how the approval process works. Let's see if we'll have some bad experience as well.
Oldtimers have told me that they liked PCs better than Apples in the early 80s because they were more hackable: you could mix and match and tinker and make them do your own thing, while Apple's stuff was closed. Users loved Apple, but hackers didn't like their we-know-best mentality. If that's true, it may not be the first time Apple has made this mistake. Perhaps losing hackers was what caused them to lose to PCs the first time around.
I agree with the primary point that Apple is making a mistake in not treating developers well.
However, I think that the essay misses a couple of points.
1. The essay rejects the notion of an "intermediary" or a "software publisher". However, it fails to appreciate the fact Apple has opened up the mobile app market to developers and eliminated far worse intermediaries (the walled gardens of carriers). This is a huge order-of-magnitude improvement.
2. A lot of developers do release badly-buggy apps out there and then end up "blaming" the Apple review process for their customers dissatusfaction. While I agree with the launch-fast-and-iterate approach, I don't think developers/companies should use that as a license to release poor-quality crashing apps. Cutting/postponing features to launch the app quickly can be a good thing, not testing the app and releasing an app with crashing bugs is not a good thing.
---
I don't expect Apple to throw out the review process, but I do hope that they improve the review process. A simple/easy improvement would be to show the latest queue ranking of apps that are waiting for review. That will go a long way towards reducing the "ongoing karma leak" that pg mentions.
Replying promptly to developer emails about app-store rejection will also help. Setting a limit of n apps for a $99 license may also help Apple cope with the incoming deluge of apps
1. While I agree with most developers that Apple has room for improvement, I think we should take a step back and realize how much of an improvement the iPhone has made mobile developers from a couple of years ago when the carriers were in charge of mobile applications.
2. I think the solution to the iPhone fiasco should be for Apple to hire more reviewers and allow developers to pay more money so that they can buy a faster turnaround time. It doesn't make sense for me to wait 2-3 weeks for another shot at the review process because I placed a bad keyword in the description.
I strongly disagree with your conclusion. Both smack of 'religious' thinking. Your point 1 is essentially 'be thankful to the merciful god who has ended the drought'. Your point 2 doesn't solve the problem, it just throws more priests at it.
The solution is to open the platform to all developers. Imagine if Microsoft had vetted every DOS and Windows app, or if we had to submit Linux apps to Linus for review.
Your first point is a bit like saying we should take a step back and be glad we're only being imprisoned instead of being imprisoned and beaten. Perhaps we should be happy about any improvements, but the situation still isn't very good. Android and Maemo both treat developers much better, and while I suspect Maemo will remain a niche platform, Android most certainly will not.
1. I'eve mixed feelings about that. The idea of a store was great because it was a great marketplace without tricks to get inside, as it's just a simple as paying 100$ (and lose two weeks understanding how the whole process works...).
But the review process is not the part of the App Store that made it successful, nor the fact the applications are so limited and can't use APIs outside a strict list.
2. This is not something you can fix with more reviewers... it's not just a matter of time, but there no way to create an objective review system, at least if you don't limit the review process to macroscopic stuff like "the app does not work" or "there is a huge disney logo", but in big companies you can't do things like this. If you have got the review process then even a 3 pixel logo of something remotely copyrighted will stop the process.
Also, the more reviewers you need, the less qualified you get, and it requires to be very sensible to handle well the review process. I don't think this can work. Jeff FooBar will simply read all the rules and apply them without the ability to figure if it's really applicable in this particular case. So like it happened recently if there is a program to connect to your mac, and it shows an icon accordingly to the model of mac detected, and there is a 10x20 pixel mac os x default aurora background, he will stop the application.
1. A couple of years ago, I had a Windows Mobile phone and tonnes of applications. While it wasn't a great phone, it was a still a pretty good computing platform. A couple of years before that, I had an SE smartphone running Symbian which had fewer apps but was still open (I even developed a few). For smartphones, carriers were no more in charge of mobile applications then as they are now. In fact, in Apple's case, the carrier (AT&T) has more control than they do on blackberries or WinMo phones.
2. The review process is almost exponential, every week you have more apps and updates to all the existing apps. Maybe Apple is hiring more reviewers; they might have to just to keep up. Having a paid review tier just means that big developers get a 1 day turn around and indy developers get the left overs.
You know on BlackBerry AppWorld each submission of an update costs you $20 AND goes through a lengthy approval process. (you need to purchase 10 submissions for $200 at a time). Oh there's a bug in my app? I could fix it in 5 minutes, but too bad..
Since when was Apple's reputation with developers great? They've always been capricious, flighty and controlling - not what you need in a solid platform.
I wish Amazon gets into the handheld device business.
They clearly understand consumers, software, infrastructure & user-experience. They missed the boat with the ipod, here's a fantastic opportunity to do something about it.
I cross-post this from the other thread I started since I noticed the Apple heat is in this one, if you want to learn the background about my app, check this thread: http://news.ycombinator.com/item?id=951360
Below is what I've learned after 2 months of AppStore dev.
Here are a few dirty tricks & ideas that I hope will make life in AppStore purgatory easier:
Use the 'Easter Egg' field to plead directly to the reviewer. It's a free-text field that every reviewer will read and a little social engineering will work wonders.
If the app name is important to you, reserve it by submitting a dummy app without binary. You'll need dummy 512x512 and 480x320 artwork in order to be able to fill out the form.
Phone contacts at Apple are pure gold. I consistently got on the line with the same reviewer. Depending on their mood reviewers can expedite things. But it's still pretty random.
Set your app price low ($0.99) while it is in the review queue & jack it up immediately on release date. I got an app rejected because the reviewer did not like the price I set.
It pays not to be a bottom-feeder. Maximize revenue, not units of sales. Often a slightly higher price will give you more revenue. There's more than the $0.99 price-point. At a higher price point you can also afford to pay higher CPM/CPC for initial customer acquisition.
Count on doing your own marketing. The time when you could solely rely on rankings to boost your sales is over. The AppStore UI is so broken right now that you have to assume that your customers will not find your app without some effort of your own.
Expect long approval times. I've started thinking in development/update cycles of at least one month. Agile is not the mindset at Apple. Work on new apps while your other apps are in the queue. I wish Apple would crowdsource the approval process or look into setting up community-managed 'repositories' like Linux.
App Complexity is a good predictor of approval time. This is a big problem because it does not reward taking on the risks of building better apps. It took me 2 months to build up the confidence to start a more complex project.
Private API calls are an absolute no-no: even if you were approved before, your next update will be rejected because Apple started using automated tools to inspect your code.
If you give unlimited access to the internet through a UIWebView, your app will be pulled unless it has a 17+ rating.
There's no (easy) way to measure where your app sales come from. This makes traditional SEM/SEO difficult. You have no conversion numbers so your marketing funnel is broken.
Don't mention Apple products in your description. Don't use images of Apple products. Don't mention real-life persons in your description, I got an app rejected for that last one.
Don't count on being able to schedule a release date. Once you do get the approval mail, go into iTunes Connect & set your release date to $NOW. Remember, this can work both ways: I got surprised by an early release before my marketing materials were ready.
Yes, you will be pushed back to the queue upon rejection, even if you have the reviewer on the line and beg him to make an exception.
In all fairness, not all is bad. Apple seems to have started picking up speed lately. My experience with App approvals has been that it got faster in the last 2 weeks, at the same time when the AppStore itself went in disarray (wonky rankings, wrong release dates, ...). However, take care and remember "Correlation, Causation et Al.". Also, once I was in contact with Apple over the phone they were very courteous and professional in resolving issues. Slow & polite.
This is a very developer-centric view, though. For users, the app store is great - tons of applications, and they're easy to find, buy, and to install. The approval process, for the user, makes a better overall experience, increasing the overall quality of available apps. For each bugfix waiting for approval there I'm certain there's lots of buggy apps being rejected, and for each anedocte of rejection due to using the wrong icon or such nonsense, there's lots of appropriate rejections due to unstable apps.
Obviously without good and satisfied developers those benefits are moot. I'm certain the approval process needs to be [improved/changed/redesigned/removed], and maybe some kind of sanctioned jailbreak should exist where more advanced users can manually install third party apps, but I do believe the App Store itself is a good thing when you you look at it from a user perpective.
[+] [-] edw519|16 years ago|reply
It iterated through an array doing 3 SQL SELECTs against non-indexed files for each element. There used to be about 50 elements in the array; now there were more than 5000. I rewrote the whole thing in one day to do a total of 4 SELECTs and run in 12 seconds.
But it took 6 days to get through QA (while the users continued to suffer). QA's biggest complaint? I indented 4 spaces instead of the (unpublished) standard of 5 spaces.
Of all the things I have to deal with, nothing pisses me off more. Software QA is becoming more and more like TSA security at the airport: illogical, and obviously so. Last year, flagrantly unacceptable code was promoted without question while its replacement was held up on a meaningless detail.
I got the feeling from the programmer's quotes in this essay that the same thing is happening at the app store.
We programmers are a funny lot. Make us struggle for business or technical reasons and we adapt beautifully. Make us struggle for something stupid and we just get pissed off and do something else. What a pity.
[+] [-] Xichekolas|16 years ago|reply
When no one is looking (the first, buggy, submission is made), they will do the minimum necessary and approve it, often quickly.
Then, when everything goes to hell, all eyes are on them. The developers submit a fix with the note "this needs to be approved ASAP to put out fires"... the QA person resents being told what to do, and suddenly feels compelled to justify his job by finding anything and everything that could possibly be considered 'wrong'. There is also the issue that things like indentation are more visibly 'wrong' than obscure bugs that only show up in production environments due to the way database replication is set up.
Now, I'm not saying all QA people are like this. I've worked with some downright awesome QA people before. But awesome QA people don't reject things for spite, and usually don't stay in QA long before moving up.
[+] [-] lt|16 years ago|reply
If the feature was really critical and important, QA should find a way to expedite the verification, but it should never be skipped in order to ship the fix faster.
[+] [-] brown9-2|16 years ago|reply
Or do you really mean "held up by code review" when you say "get through QA"?
[+] [-] indiejade|16 years ago|reply
This is the best YC quote of all time. Thx, edwin!
[+] [-] cschep|16 years ago|reply
[+] [-] petenixey|16 years ago|reply
Value - stuff should work after it's been through them
Costs that I've seen - more stuff doesn't get done/fixed because of the delay time associated with QA - things don't get scheduled because of lack of QA bandwidth - additional specifications required in order to go through QA - programmers / PMs worry less about catching bugs because they feel it'll come back from QA
When is it better to QA something rather than just measure when it's broken? Everything we do goes through QA but I'm sure most of it is a net loss.
Anyone have a good way of knowing what to QA and what to simply monitor?
[+] [-] dylanz|16 years ago|reply
[+] [-] bumblebird|16 years ago|reply
We had a backend process that connected out to a network and did some things. However, the network it connected to changed their protocol, so our backend process was broken - didn't do anything - completely non functional.
We created a fix, but had to wait for it to go through QA for ages. What's the worst that could happen? It doesn't do anything like the current version? It wasn't even a massive update. Maddening.
[+] [-] rbanffy|16 years ago|reply
[+] [-] JunkDNA|16 years ago|reply
Now, this doesn't mean that developers won't want a developer friendly phone, or that Apple is hurting their reputation with developers. I'm certain it has been frustrating to deal with the whole process. However, if 3rd party software isn't essential for the iPhone as a platform, then developer satisfaction moves down a bit in terms of importance in Apple's eyes.
Now, it's clear that games are major for the app store. Games are also a special class of software application that benefits disproportionately from having access to the hardware at a low level. They also have properties more similar to music and movies (incidentally, things the iTunes store is good at selling). Except for the simplest games, they require a lot of up-front design work and investment. It's rare that a released game goes through a ton of rapid iterations to "get it right". Games may still have bugs after release, but in general the functionality they are going to have is there on day one. They also ship with the final sound effects, music, artwork, etc... Games are also probably very unlikely to "duplicate existing functinoality" or any of the other cases where "normal" apps hit barriers. Perhaps Apple has (intentionally or unintentionally) created an environment optimized for game approval?
I'm certainly not defending the app store process, but I just wonder if the software development community has a disproportionate view of its importance to the iPhone as a whole.
[+] [-] hyperbovine|16 years ago|reply
[+] [-] shiro|16 years ago|reply
Part of it may be a byproduct of the way the console games distributed on ROMs or disks. That may change, for consoles nowadays have network connection and can get updates anytime. There's another part, however, that some type of games fits this "done is done" philosophy. Like novels or movies, which you don't expect them to be improved over time.
Well, I don't defend the app store process either, and I believe majority of software should be developed in the iterative process. It's just interesting that games may be a marginal area that has some peculiar properties.
[+] [-] Retric|16 years ago|reply
[+] [-] tjogin|16 years ago|reply
The OS bundled ones are the ones people use the most. Across the board.
If I had to guess, I'd say third party apps are just as often or more used on iPhone than on, say, Windows and Mac OS X. Simply because, on iPhone, a lot of web apps out there are used via a special-purpose iPhone app rather than the web interface.
[+] [-] mlinsey|16 years ago|reply
The year that iPhone existed without the App Store, no smartphone competitors came close to replicating it's core functionality and user experience. I don't think that's quite true anymore, and so I think that the App Store is one of the iPhone's key advantages. Specifically, the iPhone didn't need the App Store to differentiate itself from RIM and WinMo in 2007, but I think it absolutely needs the App Store to differentiate itself from the Pre and Android in 2009.
I don't think there is one killer 3rd-party app on the phone that everyone needs to have, but there could very well be many third party apps that smaller niches need to have. For me personally, MLB At-Bat (live streaming of baseball games) and the Kindle reader are actually the two biggest factors keeping me from switching to Droid. Those applications could easily be ported to Android, but haven't been yet, and in both cases a third party holds the distribution rights for the content so it's not like me or another hacker could replicate those apps easily.
[+] [-] wvenable|16 years ago|reply
All the ads I've seen lately for the iPhone are of the "there's an app for that" variety. So Apple themselves seem to think that the apps are a critical factor.
That being said, the iPhone sold very well before there were any apps at all.
[+] [-] dagw|16 years ago|reply
Any app that makes someone get an iPhone instead of something else is a "killer app" as far as Apple is concerned, even if it only sold a total of 27 copies and lost the developer a lot of money. So even if the iPhone doesn't have any one killer app, the sheer diversity of apps means there is probably a killer app out there for most people, and that is vital for Apple.
[+] [-] gcheong|16 years ago|reply
[+] [-] joe_the_user|16 years ago|reply
Developer relations are central to Apple as a whole. Apples needs iPhone developers, desktop developers, web developers and developers for whatever they do after the iPhone. If Apple creates hostile feelings in the developer community with its App Store behavior, Apple is going to have ongoing problems regardless of whether it needs third party developers at the moment.
[+] [-] joecode|16 years ago|reply
I'd say about 50% of the time my submissions get rejected, and always for a stupid reason like this; this is one of the better ones, actually. Will I stop developing? Probably not, because it's good money. But am I seriously considering moving to Android? You bet.
[+] [-] jorgem|16 years ago|reply
I have a simple "free" app in the iPhone App Store. It lets users browse "available" Web domains. One feature is a BUY button that forwards them to Godaddy.com, if they want to buy the domain.
I don't make any money from Godaddy.
The app has been approved for a while, but I wanted to patch one little bug, so I submitted a patch.
... And waited two weeks. Apple's testers found a bug that was in the original approved app, and asked me to re-submit.
... And I waited three weeks. This is classic: Apple rejects my app again because they want to test the domain "buying" feature on Godaddy.
It's a website! Not even my app! Not even a new feature.
Please kill me.
[+] [-] khangtoh|16 years ago|reply
The expedited review request was accepted but in the end took 14 days to come back and got rejected for a piece of code that was never ran and has been there since day 1.
Talk about frustration, hell yeah.
But as everyone does, after getting over the anger and back to reality and business sense, what else CAN I do but only to resubmit since there lies a never seen before first of it kinds, an easy software distribution and payment system that many other platforms did not provide to small independent developers a year ago.
It's not about just supporting one SINGLE platform, that has never been the case. Just like making web apps run nicely in IE, Safari, Firefox, etc or even Windows, MacOS, Linux. As a developer, what matter is that I support the platform that has massive adoption/users. I will not drop iPhone and go Android, or any other platform. The only decision I will make is if there's enough users on the platform that I do care about to invest and develop for that platform. One day Android might be there and I see the possibility of developing for both Android and the iPhone.
[+] [-] nearestneighbor|16 years ago|reply
[+] [-] akamaka|16 years ago|reply
[+] [-] b-man|16 years ago|reply
It's beautiful, clean and comes with with Debian based distro, Maemo.
My opinions about apple were never positive, probably because I'm only 25 and did not program in the 80's. I've only saw apple as a manufacturer of shiny, well designed rich boy toys, but that is probably because I live in Brazil, where only rich people have it and they think they are 'computer people' because of it.
I was always more influenced by the open source community, and apple's record is far from shiny on that aspect.
My 2 cents.
[1]http://en.wikipedia.org/wiki/Nokia_N900
[+] [-] unknown|16 years ago|reply
[deleted]
[+] [-] kirse|16 years ago|reply
Personally I've never been an iPhone fan either purely because you can get phones with far, far better technology for the same price. iPhone wins hands down in UI right now (I still hate touchscreen-only phones though), but that really won't be an issue for long with Nokia/Samsung/SE/HTC working double time to improve their OSes.
[+] [-] poub|16 years ago|reply
Apple do understand software. You cannot release Mac OS X, iLife and other jewels without understanding software. Also Mac developers jumping off the iPhone bandwagon are not jumping off the Mac plateform.
For the iPhone, Apple tried a different model that nobody really tried before.
They indeed have a broken store but the previous models on other platforms never worked. The most successful one before was for the PalmOS and it wasn't over the air.
However Apple always been an arrogant company. With their customers, their providers and their developers.
The climax of this arrogance has been reached with the iPhone eco system (certainly helped by the arrogance of the mobile phone operators).
And for many it is not sustainable.
But anyways, who really cares?
Apple can open the doors of the store overnight. It wouldn’t have been the first time they do one thing making you beleive it was the only way and then suddenly change in the opposite direction.
That’s the strength of Apple: they can change. Not only they can change but they usually know when, with what, for who and at which price.
I never realised the arrogance was a trade mark of this process though. Note: I don't mean being evil by being arrogant.
[+] [-] e40|16 years ago|reply
I think he just left out a word: marketing. They know how to build it, for sure. Paul's point is that they're marketing it like it is music.
[+] [-] larsberg|16 years ago|reply
I made three applications (only one that was popular - WiFinder) for the AppStore. Every time I interacted with reviewers, either by e-mail or phone, I got the feeling they had no idea what they had signed up for, and had not thought out beforehand how to deliver on the promise of only having quality applications in the AppStore. Which is a shame because it's an old problem.
[+] [-] tptacek|16 years ago|reply
[+] [-] barrkel|16 years ago|reply
I wish I could remember the quote well enough to search accurately for it.
[+] [-] petewarden|16 years ago|reply
None of this is new, the only difference is that there's now a release bottleneck that requires Apple's approval, and there's a lot of developers new to the platform.
The thing is, this is part of Apple's culture for a reason. Steve is an obsessive control-freak when it comes to a great user experience, and third party developers are seen as a threat to that. Microsoft has always been a great company to deal with as a third-party developer - they value their ecosystem, put effort into free documentation and solid IDEs, and rely on outside developers for key applications. The trouble with this comparative openness is that you end up with 800 pieces of software written by different companies fighting it out on your desktop, leading to crashes and inconsistent UIs all over the place.
I think Apple is in danger of screwing up a great opportunity with the app store, but this way of operating is in their DNA and it won't be easy to change. Despite what the geek consensus might be, the record shows they've made a lot of money in the past whilst ignoring third-party developers.
[+] [-] antirez|16 years ago|reply
Today my small team of developers submitted the first "toy" we released just to test how the approval process works. Let's see if we'll have some bad experience as well.
[+] [-] gruseom|16 years ago|reply
[+] [-] credo|16 years ago|reply
However, I think that the essay misses a couple of points.
1. The essay rejects the notion of an "intermediary" or a "software publisher". However, it fails to appreciate the fact Apple has opened up the mobile app market to developers and eliminated far worse intermediaries (the walled gardens of carriers). This is a huge order-of-magnitude improvement.
2. A lot of developers do release badly-buggy apps out there and then end up "blaming" the Apple review process for their customers dissatusfaction. While I agree with the launch-fast-and-iterate approach, I don't think developers/companies should use that as a license to release poor-quality crashing apps. Cutting/postponing features to launch the app quickly can be a good thing, not testing the app and releasing an app with crashing bugs is not a good thing. ---
I don't expect Apple to throw out the review process, but I do hope that they improve the review process. A simple/easy improvement would be to show the latest queue ranking of apps that are waiting for review. That will go a long way towards reducing the "ongoing karma leak" that pg mentions.
Replying promptly to developer emails about app-store rejection will also help. Setting a limit of n apps for a $99 license may also help Apple cope with the incoming deluge of apps
[+] [-] scscsc|16 years ago|reply
[+] [-] charlesju|16 years ago|reply
2. I think the solution to the iPhone fiasco should be for Apple to hire more reviewers and allow developers to pay more money so that they can buy a faster turnaround time. It doesn't make sense for me to wait 2-3 weeks for another shot at the review process because I placed a bad keyword in the description.
[+] [-] jgrahamc|16 years ago|reply
The solution is to open the platform to all developers. Imagine if Microsoft had vetted every DOS and Windows app, or if we had to submit Linux apps to Linus for review.
[+] [-] Zak|16 years ago|reply
[+] [-] antirez|16 years ago|reply
But the review process is not the part of the App Store that made it successful, nor the fact the applications are so limited and can't use APIs outside a strict list.
2. This is not something you can fix with more reviewers... it's not just a matter of time, but there no way to create an objective review system, at least if you don't limit the review process to macroscopic stuff like "the app does not work" or "there is a huge disney logo", but in big companies you can't do things like this. If you have got the review process then even a 3 pixel logo of something remotely copyrighted will stop the process.
Also, the more reviewers you need, the less qualified you get, and it requires to be very sensible to handle well the review process. I don't think this can work. Jeff FooBar will simply read all the rules and apply them without the ability to figure if it's really applicable in this particular case. So like it happened recently if there is a program to connect to your mac, and it shows an icon accordingly to the model of mac detected, and there is a 10x20 pixel mac os x default aurora background, he will stop the application.
[+] [-] wvenable|16 years ago|reply
2. The review process is almost exponential, every week you have more apps and updates to all the existing apps. Maybe Apple is hiring more reviewers; they might have to just to keep up. Having a paid review tier just means that big developers get a 1 day turn around and indy developers get the left overs.
[+] [-] zyb09|16 years ago|reply
[+] [-] cpr|16 years ago|reply
He just realizes he can't build a successful product, long-term, without them.
But his main internal smart software guys know better, and build a great software ecosystem.
Thus Apple is now projecting a seriously schizoid personality.
[+] [-] mattmaroon|16 years ago|reply
I always think of them more in terms of Animal Farm. Microsoft was Mr Jones, Google and Apple are Snowball and Napoleon.
[+] [-] barrkel|16 years ago|reply
[+] [-] prakash|16 years ago|reply
They clearly understand consumers, software, infrastructure & user-experience. They missed the boat with the ipod, here's a fantastic opportunity to do something about it.
[+] [-] c1sc0|16 years ago|reply
Below is what I've learned after 2 months of AppStore dev.
Here are a few dirty tricks & ideas that I hope will make life in AppStore purgatory easier:
Use the 'Easter Egg' field to plead directly to the reviewer. It's a free-text field that every reviewer will read and a little social engineering will work wonders.
If the app name is important to you, reserve it by submitting a dummy app without binary. You'll need dummy 512x512 and 480x320 artwork in order to be able to fill out the form.
Phone contacts at Apple are pure gold. I consistently got on the line with the same reviewer. Depending on their mood reviewers can expedite things. But it's still pretty random.
Set your app price low ($0.99) while it is in the review queue & jack it up immediately on release date. I got an app rejected because the reviewer did not like the price I set.
It pays not to be a bottom-feeder. Maximize revenue, not units of sales. Often a slightly higher price will give you more revenue. There's more than the $0.99 price-point. At a higher price point you can also afford to pay higher CPM/CPC for initial customer acquisition.
Count on doing your own marketing. The time when you could solely rely on rankings to boost your sales is over. The AppStore UI is so broken right now that you have to assume that your customers will not find your app without some effort of your own.
Expect long approval times. I've started thinking in development/update cycles of at least one month. Agile is not the mindset at Apple. Work on new apps while your other apps are in the queue. I wish Apple would crowdsource the approval process or look into setting up community-managed 'repositories' like Linux.
App Complexity is a good predictor of approval time. This is a big problem because it does not reward taking on the risks of building better apps. It took me 2 months to build up the confidence to start a more complex project.
Private API calls are an absolute no-no: even if you were approved before, your next update will be rejected because Apple started using automated tools to inspect your code.
If you give unlimited access to the internet through a UIWebView, your app will be pulled unless it has a 17+ rating.
There's no (easy) way to measure where your app sales come from. This makes traditional SEM/SEO difficult. You have no conversion numbers so your marketing funnel is broken.
Don't mention Apple products in your description. Don't use images of Apple products. Don't mention real-life persons in your description, I got an app rejected for that last one.
Don't count on being able to schedule a release date. Once you do get the approval mail, go into iTunes Connect & set your release date to $NOW. Remember, this can work both ways: I got surprised by an early release before my marketing materials were ready.
Yes, you will be pushed back to the queue upon rejection, even if you have the reviewer on the line and beg him to make an exception.
In all fairness, not all is bad. Apple seems to have started picking up speed lately. My experience with App approvals has been that it got faster in the last 2 weeks, at the same time when the AppStore itself went in disarray (wonky rankings, wrong release dates, ...). However, take care and remember "Correlation, Causation et Al.". Also, once I was in contact with Apple over the phone they were very courteous and professional in resolving issues. Slow & polite.
[+] [-] lt|16 years ago|reply
Obviously without good and satisfied developers those benefits are moot. I'm certain the approval process needs to be [improved/changed/redesigned/removed], and maybe some kind of sanctioned jailbreak should exist where more advanced users can manually install third party apps, but I do believe the App Store itself is a good thing when you you look at it from a user perpective.