Recent Posts | Archive

I have lots of ideas, but I have the memory of a doorknob.

Since middleschool, I’ve carried around really small notebooks to help me remember things: phone numbers, addresses, TODOs, random ideas.

I wrote the Field Notes Brand website, and even started a company with Laurel Hechanova that sold subscriptions to paper notebooks.

So, as a software engineer that is on a computer or phone a lot, I’ve spent a lot of time trying to find a note taking app that feels like paper and pen.

The Apple Notes app is too complicated. Messages I send to myself on Slack or iMessage get mixed up with important communications. The reminders app gets cluttered and I stop using it, missing important reminders. Markdown files with a good editor works, but I feel like such an idiot creating a file with only a fragment of a sentence, and get overwhelmed trying to figure out how to organize the files so I can find them in the future.

Then, earlier this year, when I was working on a different app with Laurel, frustrated that all the ideas I was having throughout the day had no place to go, I built an insanely simple app that really only allows appending text to one super long, running document. I never published it to the app store, but then a friend said at random that he wanted a really simple notes app, and described, almost exactly, the app that I had already built.

And so, I’m thrilled to announce Appended, a stupid simple note app.

You can add text, and, if you want to, scratch it out. (If you really want to, you can delete them too.) There’s robust search, integration with Siri, and it syncs across your devices (iCloud required, Mac app coming soon). No complicated tagging, just add “#tag” to your text and search for it later. No complicated UI for grouping, say, notes with links, just search for “http”. No complicated organization, just one long, ongoing, fast-to-load stream of thoughts, sorted by the time you wrote it. No pressure to fix your old typos, or correct your past self, because there’s no ability to edit.

Appended is awaiting App Store review, but I’ll be sure to update this when it’s approved!

As an excuse to play around with SwiftUI, I decided to work on an app idea I had years ago. A third party API I’d chosen for the app to depend on, I found, is free to use for n requests per day, and then a fraction of a cent per request after that, and the API key is sent with each request.

While pretending to sleep on my kid’s floor in order to get her to nap, I thought about how I could avoid exposing my API key such that a malicious actor couldn’t use the service at my expense, possibly racking up a significant bill, despite my own usage coming in far under their free request limit. (More on my solution in a later post.) After I felt confident about my solution, I decided to see what other apps did.

Web Security

Long time developers of web based services should know the cardinal rule of application security: all client traffic is untrusted and should be considered malicious. User input saved in the database should be escaped to avoid sql injection; user input that gets rendered on an HTML page should be HTML encoded to avoid running client side scripts; we encypt data in transit using HTTPS to avoid man-in-the-middle attacks, which could either reveal private data from the client, or alter it en route to the server.

I learned this not through academia, or through some awareness of common exploits, but by accidentally not escaping user input, and seeing the side effects first hand. (The first time I experienced this, for example, was when I didn’t escape users’ names on an early website I built, such that an apostrophe caused the raw SQL to get dumped for the user to see.)

Sadly, as I’ve worked with an increasing number of engineers raised using well established ORMs like ActiveRecord, or templating engines like ERB, I’ve seen more and more people forget this rule, and introduce vulnerabilities when they’re not protected by such a framework.

Mobile App Security

Ideally developers of mobile APIs treat traffic coming from apps as untrusted, since a mobile app is, afterall, a client. But in this case, there’s a new attack vector: the consumer of your app may act as a man-in-the-middle between app and client, so secrets shared between the two are at risk.

It is this vector that got me thinking about how to protect my API key while in transit. Specifically, if my app included its API key in requests to the third party API, even if encrypted in transit using HTTPS, the user could enable SSL proxying, using an intermediate certificate, which would allow him/her to see the unencrypted request.

Findings

It took me about three minutes to switch on Charles, launch a couple apps, and log the unencrypted traffic. (I’ve done this before.)

The third party requires apps using their API to credit them on the app, so finding apps that use their API was easy. I checked two, which I won’t name, and the API provider’s own app.

Sure enough, all three were leaving the API key exposed:

So?

Like any vulnerability, there’s a balance between risk and cost to prevent. In reality, it’s extremely unlikely someone would steal my key to gain access to an already-mostly-free API. Preventing this requires running a server that authenticates the app somehow, and then proxies the request to the third party. This costs money, and, more importantly for my case, is another point of failure, and a set of things to maintain (domain registration, SSL certificates, scaling, etc).

However, if I were less honest, I’d also be able to use this API for free, since I have the service’s very own API key. ;)

UPDATE 2019-01-02: privacy.com responded with the following: “I’ve shared this with the team and will ensure we investigate this thoroughly and remove it in the next iOS deploy.”

UPDATE 2019-01-06: Privacy International has written a very similar post about Facebook Tracking on Android, available here. Thanks to Andy for the tip.

As I noted in my recent post about Mint sending data to Facebook, I discovered several POST requests going to graph.facebook.com containing data about my device, among other things, despite having no Facebook account, using no Facebook services personally, and blocking all traffic to *.facebook.com from my home network. I learned shortly after that post that many apps on my device are making such requests, and decided to investigate a bit further.

Disclaimer: I was not authenticated in all apps when running these tests, so it’s very possible there’d be more requests once I authenticate. This is the result of very manual, non-lab-like research. I’m sure I missed some things.

Facebook Software Development Kit

The iOS Facebook SDK, with user-agent FBiOSSDK, is made available to any developer who wishes to use Facebook’s services in their app. Such services include push notifications, the horrifying “login using facebook” oauth function, App Events, Facebook Analytics, and Graph API.

Dependencies

Possibly unbeknownst to app developers using this SDK, since developers rarely read terms when using open source libraries, and because we live in an era where “don’t re-invent the wheel” is synonymous with “I don’t want to do that myself,” Facebook makes clear in their README that:

By enabling Facebook integrations, including through this SDK, you can share information with Facebook, including information about people’s use of your app. … These integrations also enable us and our partners to serve ads on and off Facebook.

My initial theory was that Intuit (via Mint’s iOS app) was providing data to Facebook so they could better serve ads. That may still be the case, but after seeing a great number of other applications making similar requests, I now wonder if most are doing this out of sheer neglect. After all, how else would developers of possibly millions of apps know that their dependencies included a module just for left padding a string?

So if an app you use, for example, allows logging in via Facebook, or maybe has a button to share the app with your Facebook friends, odds are good the app includes this library, and is sending Facebook your data just by being bootstrapped when the application launches.

Data Sent

The data sent to Facebook seems to vary slightly by application and SDK version, which makes sense given the SDK allows sending custom attributes with each request.

Much of this data is mostly harmless: iOS version, screen size, the bundle ID of the application. But some of this could easily be used to identify patterns, or infer details, about you personally.

All Requests

All requests that I identified sent the following (among other fields):

advertising_id

Back in the day, iOS provided a globally unique identifier to all apps to identify which device the app was running on. Apple removed this in favor of a new identifier they call the advertising_id. This value differs between apps, so it is of limited use for identifying you as a human. However, if this isn’t reset often, it is enough for the application – and now Facebook – to track your behavior across launches.

Apple recently found this was being exploited and not used for its intended purpose, so they started pulling apps from the App Store that used the advertising ID but never showed ads. I’m not sure how some of the apps currently sending this to Facebook are getting around this, because the ones I’ve noted do not serve ads.

I strongly recommend you disable your device’s advertising ID by going to Settings > Privacy > Advertising, and enable “Limit Ad Tracking”. If you’d prefer to leave ad tracking as is, I suggest you periodically reset your advertising identifier.

POST /activities

This was the most common request, as it was made multiple times when applications were in memory.

anon_id

Each POST /activities request contains a field called anon_id, or old_anon_id, which is a UUID, prefixed with XZ, that appears to be unique to each application.

I did a little digging through the source and found that this value is persisted locally in a file called com-facebook-sdk-PersistedAnonymousID.json. Because of iOS sandboxing, this value cannot be shared outside of the app’s sandbox.

session_id

I wanted to see if this file contained anything else, so I took a look at a recent iPhone backup on my laptop (after decrypting). That file didn’t include anything else, but I did come across another file, com-facebook-sdk-AppEventsTimeSpent.json, which included:

• lastSuspendTime
• numInterruptions
• sessionID
• secondsSpentInCurrentSession

The sessionId is included in these requests as well. This is likely used for behavioral tracking.

Coarse Grain Location Data

I could not find an instance where Facebook was gathering my GPS coordinates, even for apps I know to have access to my location (such as Transit App), but they all sent my timezone (both CST and America/Chicago), and of course also sent my IP as part of the request. In other words, by sending any request at all, Facebook knows roughly where my phone is any time I launch or keep in memory one of these apps.

Carrier

Along with each request is my carrier. The fact that they’re getting my IP seems much worse to me, but the recent discovery that most or all of the major US carriers are attaching a unique identifier to all request headers makes me incredibly wary of what else a company as powerful as Facebook could do knowing my carrier.

Event

This end point is all about tracking events, afterall. So each includes an event type, eg MOBILE_APP_INSTALL.

Misc

A key called fb_mobile_launch_source was provided with launch events. My guess is this refers to whether an app was launched directly, via push notification, or with a URL.

I was able to find out which of these apps use ReactNative too. Which is fun.

POST /user_properties

user_unique_id

This is stored in NSUserDefaults, and appears to be much like the anon_id. Given my limited data set, I was unable to determine if this matched any other ID fields.

Custom Data

This contains a bunch of custom data based on the app developer’s implementation.

For example, in a block called custom_data, one app sent:

• is_old_user
• has_subsribed
• has_paid

Another app sent only { "vpn_provider": "primary" }, while a third sent g_numberOfTrackedMetroAreas.

POST /

This is a method for POSTing batches, and the request body contains an array of the above request types. In my limited data set, I found very few apps to be batching requests.

Offending Applications

This is by no means an exhaustive list, but is a list of apps I personally use regularly that I found to be making requests to Facebook. (Let me know if you find others!)

I found this list using Charles for iOS, entering low power mode to minimize noise from background-running apps, launched a whole lot of apps, then looked through the logs. (Note: you must enable SSL proxying for this to work, because Facebook’s SDK uses HTTPS.)

Bundle IDs are below for requests where available.

• At least one major credit card
• AirVisual (com.airvisual.airvisual)
• AmpliFi Client (com.ubnt.amplifi)
• Belingual (com.beelinguapp.beelinguapp)
• Belly
• ChowNow
• Curb (com.ridecharge.TaxiMagic)
• FXNOW
• Garmin Connect (com.garmin.connect.mobile)
• Happy Cow (com.smoothlandon.happycow)
• IMDb (com.imdb.imdb)
• KAYAK (com.kayak.travel)
• Kindle (com.amazon.Lassen)
• Launcher
• Mint (com.mint.internal)
• Outlook (com.microsoft.office.outlook)
• Pocket (com.ideashower.ReadItLaterPro)
• Privacy (com.privacy.paybeta – see 2018-01-02 update)
• REI
• Seafood Watch
• Songkick
• Spotify
• Strava (com.strava.stravaride)
• Stryd (com.athletearchitect.stryd)
• Tor Browser (com.pentaloop.torbrowser) – note: this made the most requests to Facebook
• Transit App (com.samvermette.Transit and com.transitapp.transitx)
• UPS My Choice (com.ups.m.iphone)
• Vivino Wine Scanner
• Walk Score
• White Noise (com.tmsoft.WhiteNoiseLite)
• Yelp (com.yelp.yelpiphone) – this made a reasonable number of requests to Facebook, but makes an insane number of request to Yelp services when in the background
• Zoom

I’ll point out that a company called Privacy is a strange one to be giving data willingly to Facebook at their users’ expense. This one was particularly heartbreaking.

Conclusion

As a consumer, this is terrifying. I’ve already reached out to several of the services above, or deleted others from my phone after closing my account.

As an app developer, this is extremely disappointing. Is it worth betraying your users to add a feature you may not need? If the product is free, you’re the product.

As a guy who likes being paid for his work, this should come as a warning. This willingness among the tech community to share (or sell?) data is abhorrent, and people need to start holding companies accountable. I’ve already uninstalled many of these apps, and will discontinue using some services entirely. (Note: I ripped MapBox out of an app when I discovered they were tracking my users’ location even when my app asked the framework to stop.)

And if you think Facebook is the only company lately hoarding data, you are absolutely wrong. In fact, while looking through my request logs, I found far more requests to other services: Crashlytics, app-measurement.com, play.googleapis.com, among others.

However, I’m now even more in love with AdBlocker, and found a new love with Charles Proxy. Also, 1Password and Brave both don’t send any such traffic anywhere; I’m very likely to uninstall all apps I find tracking me and just stick with cookie-less web browsing through 1Password’s 1Browser or Brave.

Intuit’s Mint application on iOS is sending data to graph.facebook.com without explicit permission. Their privacy policy includes a link to a Facebook page to opt-out of targeted ads on Facebook, which requires a Facebook account, and notes that data may be shared with Facebook for “Social Media Features”.

DNS Level Ad Blocking

For the last year or so, I’ve been using an app on my iPhone called AdBlock for iOS. The app creates a local VPN, and when enabled, returns 0.0.0.0 when doing DNS lookups for blacklisted domains.

Despite its name suggesting this is only useful for blocking ads, I use this primarily for blocking tracking or apps sharing data with third parties. The app can optionally log all requests, making it easier to maintain your own blacklist, and to see how absolutely disgusting some applications can be.

For example, I have for years not had a Facebook account. As such, none of my apps are “linked” to a Facebook account, I never “login using Facebook”, and I certainly receive no benefit from any requests sent to Facebook. Yet several apps make many requests to Facebook – mostly POST requests to graph.facebook.com. These requests will always fail on my device because the DNS lookup will always return 0.0.0.0, thanks for AdBlock.

Indentifying Mint as the Culprit

Due to the nature of iOS network requests, it is impossible for AdBlock to know which apps are making these requests. The best you can do is enable low-power mode, force quit an app you’re suspicious about, then launch it and watch the logs.

I eventually became suspicious that Intuit’s Mint app was sending data to Facebook, as I noticed DNS requests in AdBlock every time I launched Mint.

To be sure, I fired up Charles for iOS. Charles is a proxy that runs locally, similar to AdBlock, and logs all network requests. Since almost all requests on iOS use SSL (thanks to iOS’ app transport security), viewing request or response bodies require installing and trusting an intermediate certificate. The app has very easy to understand instructions for doing so.

Data Shared

I should note that the user agent provided is FBiOSSDK.3.21.0, suggesting Mint is simply using a Facebook SDK for other purposes, and that Facebook may be sending this data to themselves without Mint involved.

POST /.../activities?...

This request is made every few minutes, as long as the application is in memory.

Here’s a JSON format of the query parameters included in the request URL:

{
	"bundle_id": "com.mint.internal",
	"event": "CUSTOM_APP_EVENTS",
	"application_tracking_enabled": "1",
	"advertiser_id": "00000000-0000-0000-0000-000000000000",
	"old_anon_id": "XZFFA40E06-2B72-46EE-8D25-REDACTED",
	"bundle_short_version": "6.6.0",
	"sdk": "ios",
	"format": "json",
	"url_schemes": [
		"intu4468714322444XXXXXX",
		"mint",
		"fb501820783XXXXXX"
	],
	"advertiser_tracking_enabled": "0",
	"bundle_version": "97.20234.4012"
}

Note: the advertising ID is all zeros, as I’ve opted for “Limited Ad Tracking” at the iOS level.

I’m not sure what the old_anon_id is, but I imagine it is generated by the app to identify my instance.

GET /.../fields?...

This is a GET request listing several fields, eg supports_attribution, supports_implicit_sdk_logging, and responds with hash indiciating which fields are enabled.

This request appears to be made once per launch.

This response incudes, in addition to the fields enabled, an id, which matches the string begining with fb in the url schemes above. I’m unable to determine if this string is unique to my installation, or is more like a version number for the Facebook SDK. The value did not change between launches.

Conclusion

As you can see, there’s not a lot of data going to Facebook, but frankly I’m not sure why there’s any, especially because there is no way to opt out of this without running third party blocking services.

Mint users trust them with far more important data – namely, banking credentials and purchasing behavior – and do so knowing Mint makes money by serving targeted ads. It’s thus not a tremendous surprise that they send data to one of the biggest ad companies in the world. Nonetheless, they’ve lost my business as a result.

After trying out simple KVO, I wanted to try assigning a single property to each observableclass that would emit events when something changed. I liked this idea because it would, in theory, require very few changes to the actual observable class, and could thus live separately from it, such as in an extension or in a subclass even. To keep it simple, I opted to not worry about sending the current (or previous) value, as KVO does, and instead just let observers know that something changed, and make them investigate.

The event emitter, which I’ve called Notifier, might look like this:

class Notifier<T> {
    typealias NotificationCallback = (T) -> Void
    var notificationCallbacks = [NotificationCallback]()

    func observe(_ callback: @escaping NotificationCallback) {
        notificationCallbacks.append(callback)
    }

    func send(_ value: T) {
        for callback in notificationCallbacks {
            callback(value)
        }
    }
}

Consider our previous example of having a Person class with observable first and last names, and a ViewModel that puts the pieces together for its view.

To our really simple Person class, we’d add a single, public Notifier instance, where the generic T is a Swift enum containing the field name that changed.

enum PersonProperty {
    case firstName, lastName
}

class Person {
    var notifier = Notifier<PersonProperty>()
    var firstName: String { didSet { notifier.send(.firstName) } }
    var lastName: String { didSet { notifier.send(.lastName) } }
}

Then, our View Model would just register a handler with likely a switch statement to decide what to do:

class PersonViewModel {
    var personNameText: String

    //...

    func bindPerson() {
        person.notifier.observe({[unowned self](property) in
            switch property {
            case .firstName:
                self.firstNameChanged()
            case .lastName:
                self.lastNameChanged()
            }
        })
    }
}

Benefits

As noted above, this method hardly changes the observed class at all. The only changes necessary are adding a Notifier instance, adding a swift didSet hook to each observable property, and defining which properties can be observed in an enum. There is zero dependence on Foundation, NSObjects, or Objective-C.

Costs

Unfortunately, there are some very real costs.

Reference Counters

To be able to stop observing, you’d need to probably create a class containing a callback and some way to identify it. Then, like with KVO, you’d need to keep a reference to it (or its identifier), and when you no longer want to observe, you’d have to tell the Notifier to remove it. I wound up doing this for subsequent patterns, meaning this problem isn’t unique to this pattern.

Initial Values

Remember in my KVO post how I noticed that getting an initial value was pretty nice? Without some modification, this pattern won’t do that. I wound up getting around this fairly easily, but it wasn’t pretty.

class Notifier<T> {
    func observe(_ callback: @escaping NotificationCallback, withInitialValue initialValue: T? = nil) {
        //...
        if let initialValue = initialValue {
            callback(initialValue)
        }
    }
}

enum PersonProperty {
    case .initial, ...
}

class PersonViewModel {
    func bindPerson() {
        person.notifier.observe({[unowned self](property) in
            //...
        }, withInitialValue: .initial)
    }
}

Rating: C+

The simplicity of this approach goes a really long way. In many cases, where there’s really only one observer, you could make the notifier instance optional, and set it to nil to effectively stop observing, the initialize a new one when you need to. But it just requires too much funny business to make it practical, like passing in that initial value when observing.

Still, I think we may be onto something with this pattern, and I’m excited to explore some more options before we can take the best of all of them!