Testing In-App Purchases Ruins Your Phone

June 2018 Update

With the introduction of a permanent login for Sandbox accounts in iOS 12 this problem has basically disappeared. Glad we’re making a difference.

Years of unpaid technical debt has made testing in-app purchases a miserable experience

Getting in-app purchases right is a goal that every app developer should have. Your customer’s experience when handing over their money should be seamless. To guarantee a good experience you must rigorously test. However, there are many pain points involved in doing this. Each individual issue is only irritating, but together they create enough friction that properly testing in-app purchases is a miserable experience.

Sandbox accounts

To test in-app purchases outside of the App Store you need a sandbox account. Theses are created via iTunes Connect, a bloated and slow web-app powered by a 21-year-old framework. They require a unique email address for each test account you create. You are faced with two options: use a fake random email address that you will forget or use your real email address, appending “+(something unique)” to the end. Both annoying options after a while.

After creating a sandbox account, you need to login to use it. Apple requires you to log out of your real App Store account first to use your sandbox credentials, all too often leaving your device in an unusable state. Of course, until you log back in at least, which for me doesn’t happen until I’m trying to download someone’s app in front of them, they are staring at me, I have to open 1Password, it’s taking forever, and it’s awkward.)

There is a severe gotcha associated with logging into sandbox that Apple so wonderfully buries in a programming guide:

If you log in to the App Store app with your sandbox credentials, it will ruin your sandbox account.

You will not just fail to login, but you will burn the account, sending yourself back to step one. The account can’t be used for testing ever again. This bug guarantees, especially if you are on a team that shares sandbox credentials, that you will be making many trips through the iTunes Connect spanking machine and creating many accounts. Worst of all the failure is latent and only presents itself in the form of in-app purchase testing randomly failing much later, likely sending your team into a pre-ship fire drill. You will be scrambling to find the regression that in reality was just a side effect of an iTunes accounts bug.

RevenueCat is the best way to implement subscriptions in your mobile app. We handle all the complicated parts so you can get back to building.

The looping login modal from hell

My personal least favorite when working with recurring subscriptions: If you make a test purchase of a recurring subscription, you will be forever presented with a looping App Store login modal, usually when unlocking your phone. This renders any personal device you are testing on ruined.

The worst part is that in some cases, it will present itself 5 or 6 times (my record is 20). The only fix I’ve found for this is to uninstall the app you’re testing. This is a major quality of life tradeoff for the developer. While working on Elevate, I was unable to keep our test builds on my phone, limiting my ability to test, and creating awkward situations with my co-workers.

Uh, erm, I don’t have the build installed ‘cause I tested purchases 2 years ago and now my phone is broken forever. I still like the app. I swear.— Jacob

A failure without a cause

There are many of missed details that can cause in-app purchases to fail while testing. Most of these failures don’t have specific error messages, leaving the developer with no indication of the cause. You can see this in the mountain of Stack Overflow questions referencing the generic error message language. The most common of these causes is the failure to accept your iTunes connect contracts, a step that, in a company environment, is often out of the developer’s control.

You will see a similar, non-descriptive error message if you try to use StoreKit in the simulator. This leads to a lot of developers stumbling around trying to get in-app purchases working in the simulator. Apple should add a simple check for simulator in StoreKit and log an error or raise an exception. It would save a lot of developer time.

Different environments, different user interface

The purchasing UI is different in sandbox vs. production, misleading developers about the final user experience.

In iOS 11, Apple upgraded the purchase flow, making it a much cleaner, one tap experience to make an in-app purchase. For some reason, the old, UIAlertView flow is still being used on sandbox.

Dealing with both environments server side is error prone

Correctly supporting sandbox and production environments in the developer’s backend leads to unnecessary complexity, bugs, and, possibly, rejections. The Apple receipt file is a signed cryptographic container that contains all purchases made by a user in your app. Most developers, rather than writing code to verify and parse the receipt file, send it to Apple’s hosted verification service to extract the contents.

There are two versions of this /verifyReceipt endpoint, one for sandbox and one for production. If you send a receipt to the incorrect environment, you will receive an error response that tells you to send it to the opposite endpoint. This can cause a few problems.

A common practice is to have two versions of your backend: a development, and a production backend. Usually, you configure your development backend to use the sandbox /verifyReceipt endpoint and production to the production App Store. This works fine in testing since you will be testing against your development backend using a sandbox App Store account. However, when you submit for review, your app will be configured for production, but the app review team uses the sandbox environment for testing in-app purchases. This often leads to unexpected rejections.

I’m sure there are reasons why Apple forces us to figure out the correct backend through trial and error, but it would be much better if they just verified both environments from one endpoint. This is how RevenueCat does it; we dispatch any receipt to the correct environment for you.

Time to end sandbox

Apple should eliminate the sandbox environment. They should just allow us to whitelist a limited number of regular App Store accounts that we can use as test accounts. These accounts would, for our apps, receive free purchases and be able to see products before they are cleared for sale. We could then use just one account for production and testing.

I’m sure there is a mountain of technical debt built up around this slipshod set of systems that makes doing this a politically and technically tricky task. But I also know Apple has the money to fix the problem, they just need the will.

Death by a thousand tiny bugs

None of these issues on their own is a complete show-stopper. However, taken together they create an environment where no developer looks forward to working with in-app purchases. This leads to in-app purchase features being pushed off, under architected, and under-tested. Neglect of the developer experience runs downhill and consumers are the ones who ultimately suffer.

‍