Reverse Engineering Xiaomi’s Analytics app

I own a Xiaomi Mi4 and I discovered it comes with a pre-installed app called AnalyticsCore, package name com.miui.analytics, that’s running in the background. I’m not a big fan of apps gaining information without my permissions, so I started investigating its activities. For those who don’t know, Xiaomi is the largest smartphone manufacturer in China and actively growing worldwide.

For this I downloaded dex2jar and Java Decompiler and started AnalyticsCore.apk in it. The APK is downloadable here if you want to take a look yourself.
I first googled what its purpose is, and I found a single thread on the Xiaomi forums, but there is no response or explanation on what it does. See this thread.

Inside Java Decompiler there are mainly three interesting classes in how AnalyticsCore gets his updates, named c.class, e.class and f.class by Java Decompiler. Here is the code of a function inside f.class. (all decompiled code)

private boolean I()
  {
    boolean bool = false;
    if (b.t()) {}
    for (;;)
    {
      return bool;
      long l2 = J();
      m.d("Analytics-UpdateManager", "last update check time is " + new Date(l2).toString());
      long l1 = new Random(System.currentTimeMillis()).nextLong();
      if (System.currentTimeMillis() - l2 >= (l1 % (2L * 43200000L) + 2L * 43200000L) % (2L * 43200000L) - 43200000L + 86400000L) {
        bool = true;
      }
    }
  }

The above function checks some time within every 24 hours for a new Analytics update. It makes the following request every day within 24 hours, which is very often if you ask me:

  public void run()
  {
    int i = 0;
    long l1 = System.currentTimeMillis();
    for (;;)
    {
      int j = i + 1;
      if (i < 2) {}
      try
      {
        Object localObject2 = new java/lang/StringBuilder;
        ((StringBuilder)localObject2).();
        Object localObject1 = f.a(this.A);
        ((StringBuilder)localObject2).append("currentApiVersion0.0.0");
        Object localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("currentCoreVersion" + k.t(f.b(this.A)));
        localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("imei" + com.miui.analytics.internal.a.k.j(f.b(this.A)));
        localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("mac" + com.miui.analytics.internal.a.k.k(f.b(this.A)));
        localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("model" + com.miui.analytics.internal.a.k.getModel());
        localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("nonce" + (String)localObject1);
        localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("package" + f.b(this.A).getPackageName());
        localObject3 = new java/lang/StringBuilder;
        ((StringBuilder)localObject3).();
        ((StringBuilder)localObject2).append("ts" + l1);
        ((StringBuilder)localObject2).append("[email protected]#)(*[email protected]!#");
        localObject3 = n.getMd5Digest(((StringBuilder)localObject2).toString()).toLowerCase(Locale.getDefault());
        localObject2 = new java/lang/StringBuilder;
        ((StringBuilder)localObject2).("http://sdkconfig.ad.xiaomi.com/api/checkupdate/lastusefulversion?");
        ((StringBuilder)localObject2).append("currentApiVersion=0.0.0");
        Object localObject4 = new java/lang/StringBuilder;
        ((StringBuilder)localObject4).();
        ((StringBuilder)localObject2).append("¤tCoreVersion=" + k.t(f.b(this.A)));
        localObject4 = new java/lang/StringBuilder;
        ((StringBuilder)localObject4).();
        ((StringBuilder)localObject2).append("&imei=" + com.miui.analytics.internal.a.k.j(f.b(this.A)));
        localObject4 = new java/lang/StringBuilder;
        ((StringBuilder)localObject4).();
        ((StringBuilder)localObject2).append("&mac=" + com.miui.analytics.internal.a.k.k(f.b(this.A)));
        localObject4 = new java/lang/StringBuilder;
        ((StringBuilder)localObject4).();
        ((StringBuilder)localObject2).append("&model=" + URLEncoder.encode(com.miui.analytics.internal.a.k.getModel(), "utf-8"));
        localObject4 = new java/lang/StringBuilder;
        ((StringBuilder)localObject4).();
        ((StringBuilder)localObject2).append("&nonce=" + (String)localObject1);
        localObject1 = new java/lang/StringBuilder;
        ((StringBuilder)localObject1).();
        ((StringBuilder)localObject2).append("&package=" + f.b(this.A).getPackageName());
        localObject1 = new java/lang/StringBuilder;
        ((StringBuilder)localObject1).();
        ((StringBuilder)localObject2).append("&ts=" + l1);
        localObject1 = new java/lang/StringBuilder;
        ((StringBuilder)localObject1).();
        ((StringBuilder)localObject2).append("&sign=" + (String)localObject3);
        localObject1 = new java/net/URL;
        ((URL)localObject1).(((StringBuilder)localObject2).toString());
        localObject1 = (HttpURLConnection)((URL)localObject1).openConnection();
        ((HttpURLConnection)localObject1).setRequestMethod("GET");
        ((HttpURLConnection)localObject1).setConnectTimeout(5000);
        ((HttpURLConnection)localObject1).connect();
        localObject2 = new java/lang/String;
        ((String)localObject2).(d.a(((HttpURLConnection)localObject1).getInputStream()));
        localObject1 = new java/lang/StringBuilder;
        ((StringBuilder)localObject1).();
        m.d("Analytics-UpdateManager", "result " + (String)localObject2);
        localObject1 = new org/json/JSONObject;
        ((JSONObject)localObject1).((String)localObject2);
        localObject3 = ((JSONObject)localObject1).optString("url");
        i = ((JSONObject)localObject1).optInt("code", 0);
        localObject2 = ((JSONObject)localObject1).optString("v");
        f.a(this.A, ((JSONObject)localObject1).optInt("force", 0));
        f.a(this.A, ((JSONObject)localObject1).optBoolean("wifi", true));
        if ((!TextUtils.isEmpty((CharSequence)localObject3)) && (!TextUtils.isEmpty((CharSequence)localObject2)))
        {
          localObject4 = new com/miui/analytics/internal/a;
          ((a)localObject4).((String)localObject2);
          if ((b.q()) || (((a)localObject4).a == 0))
          {
            f.a(this.A, ((JSONObject)localObject1).optString("md5"));
            f.b(this.A, (String)localObject3);
            f.c(this.A).execute(this.A.aP);
          }
        }
        while (i != -8) {
          return;
        }
        long l2 = f.c(this.A, ((JSONObject)localObject1).optString("failMsg"));
        l1 = l2;
        i = j;
      }
      catch (Exception localException)
      {
        f.a(this.A, 0L);
        m.e("Analytics-UpdateManager", "exception ", localException);
        i = j;
      }
    }

As you can see, it makes a request to http://sdkconfig.ad.xiaomi.com/api/checkupdate/lastusefulversion? which is of course an official Xiaomi domain. It sends some parameters with it: including IMEI, MAC address, Model, Nonce, Package name and signature.

After the above code has been executed, it might get an (updated) apk file back. Inside e.class this APK file gets downloaded:

public void run()
  {
    try
    {
      if ((!k.m(f.b(this.A))) && (f.d(this.A))) {}
      for (;;)
      {
        return;
        Object localObject1 = new java/net/URL;
        ((URL)localObject1).(f.e(this.A));
        localObject1 = (HttpURLConnection)((URL)localObject1).openConnection();
        ((HttpURLConnection)localObject1).setRequestMethod("GET");
        ((HttpURLConnection)localObject1).setConnectTimeout(5000);
        ((HttpURLConnection)localObject1).connect();
        if (((HttpURLConnection)localObject1).getResponseCode() == 200)
        {
          Object localObject2 = d.a(((HttpURLConnection)localObject1).getInputStream());
          localObject1 = localObject2;
          Object localObject3;
          if (!TextUtils.isEmpty(f.f(this.A)))
          {
            localObject3 = a.a((byte[])localObject2);
            localObject1 = localObject2;
            if (!f.f(this.A).equalsIgnoreCase((String)localObject3)) {
              localObject1 = null;
            }
          }
          if (localObject1 != null)
          {
            Log.d("Analytics-UpdateManager", "download apk success.");
            localObject2 = new java/io/File;
            ((File)localObject2).(f.g(this.A));
            localObject3 = new java/io/FileOutputStream;
            ((FileOutputStream)localObject3).((File)localObject2);
            ((FileOutputStream)localObject3).write((byte[])localObject1);
            ((FileOutputStream)localObject3).close();
            f.h(this.A);
          }
        }
      }
...


The download location for the APK is set in f.class, where also the 24h time check was placed:

private String G()
  {
    try
    {
      Object localObject = new java/lang/StringBuilder;
      ((StringBuilder)localObject).();
      localObject = this.mContext.getExternalCacheDir().getAbsolutePath() + "/Analytics.apk";
      return (String)localObject;
    }
    catch (Exception localException)
    {
      for (;;)
      {
        String str = "";
      }
    }
  }


Now the question is, where does this APK gets installed? I couldn’t find any proof inside the Analytics app itself, so I’m guessing that a higher privileged Xiaomi app runs the installation in the background. The question is then: does it verify the correctness of the APK, and does it make sure that it is in fact an Analytics app? If it does not, that means Xiaomi can install any app on your device it wants, as long as it’s named Analytics.apk.

Update 12:31: Someone told me the package gets installed from l.class, with following code:

try
    {
      paramContext.getPackageManager().getClass().getMethod("installPackage", new Class[] { Uri.class, Class.forName("android.content.pm.IPackageInstallObserver"), Integer.TYPE, String.class }).invoke(paramContext.getPackageManager(), new Object[] { Uri.parse(paramString), null, paramContext.getPackageManager().getClass().getField("INSTALL_REPLACE_EXISTING").get(null), null });
      m.d("AppInstaller", "install apk success.");
      return;
    }
...


It seems like there indeed is no validation on what APK is getting installed. So it looks like Xiaomi can replace any (signed?) package they want silently on your device within 24 hours. And I’m not sure when this AppInstaller gets called, but I wonder if it’s possible to place your own Analytics.apk inside the correct dir, and wait for it to get installed (edit: getExternalCacheDir() is inside the app’s sandbox, so probably not). But this sounds like a vulnerability to me anyhow, since they have your IMEI and Device Model, they can install any apk for your device specifically.

If you own a Xiaomi device yourself, you might want to block all access to Xiaomi related domains, because by far this isn’t the only request to a Xiaomi site. I use AdAway for this. It does require root access, but that should be no problem if you run the International ROM. I don’t know if the official rom supports root access out of the box.
My AdAway:

Here is a link to a post with other bloatware apps you can safely remove from your device, next to Analytics: https://forum.xda-developers.com/xiaomi-mi-3/general/tip-safe-to-remove-bloatware-list-miui-t2999283

If anyone has tips or a comment, please email or contact me.

Read articles on social media for free. Good idea?

Yesterday I opened a news article someone shared on Facebook which brought me to the website below. Notice the message box on top saying: “A subscriber shared this article so you can read it for free. Want to read more? Sign up and read 5 articles per month for free.” Because someone shared the article on Facebook I can read it for free. That’s interesting, because there are only a few ways for this site to know it’s a shared article, and most of those ways are easy to spoof / manipulate. Are we able to read more articles for free that are not shared…?

 

How does it work?

I can think of two ways this site knows I’m viewing a shared article:

  1. The link to the article is unique and specifically generated to be shared. By checking the unique part of the URL the website recognizes it as a free, shared article.
  2. The website checks which site I’m coming from. If it’s from Facebook, that must mean someone shared the article.

It is not option one, which can be checked by just looking at the URL. The URL contains only the article’s number and title, so no uniquely generated part. It must be option two: it checks where I’m coming from. If it’s from facebook.com (or twitter.com or any other social media site) it shows the article for free. There is a second requirement. If I open a different article (also shared), I’m still required to sign up. It remembers whether I’ve already seen a free article and only shows the first one for free. Again two scenarios:

  1. After I open the first article the website stores my IP inside a database. Every other link after that queries the database first. It only shows the full article if my IP is not in there yet.
  2. After opening the first article it stores a cookie in my browser and the next time it checks if that cookie already exists. If it does, it’s not free and I must sign up first.

The first option, saving IP addresses inside a database, is regularly the best way to go because IPs are harder to manipulate than cookies. I could change my IP address temporary by using a proxy, VPN or similar, but that’s a lot of hassle for reading an article. A cookie on the other hand is fairly easy to edit or remove as it is stored locally. The IP based way, option one, has a major downside: only one device in a network can see one article. That’s not very ideal in schools, workplaces etc that have shared IPs, and it’s probably not what this news site wants. I checked it and they indeed placed a cookie called ‘socialread’ with the article’s number as value (in my case 1118069; I confirmed it in Firefox by rightclick, View Page Info, Security, View Cookies).

Is this a reliable method to let people read only one (shared) article?

In short: no. As said, cookies are easy to manipulate. I used Cookie Manager+ for this in Firefox. This addon makes it possible to change every cookie that’s set in your browser. I was able to change the cookie ‘socialread’ to a different article number. Or even easier: just disable cookies at all, that way the site can’t even save the cookie and surprisingly it’s still showing articles for free. The news site isn’t confirming whether the cookie is actually set or not. (I won’t get into detail on how to disable or change cookies, but I can assure you, it’s really easy). The other check, if we’re coming from Facebook, was a little bit harder to spoof (but still not hard). Of course, with the cookie check bypassed, it is already possible to see more than one shared article. But if I want to see a certain article on this news website, I have to look it up on Facebook first and find where it’s shared. Not the most efficient way. Easier would be to just always act like we’re coming directly from Facebook, even if we’re not, to unlock all articles. The news site checks our history by the Referer header inside each request. If you’re for example visiting youtube.com from reddit.com, a lot of info is sent to youtube.com, including the exact URL you’re coming from. That last information is stored inside a header called ‘Referer’ and that’s the header you want to change to facebook.com. A way to do this automatically is by (again) using a Firefox addon. I used Modify Headers for this, but there are many addons available that can spoof headers.

Other method

Social media isn’t the only trigger that makes the full text available, it’s also when coming from Google. They probably did this to get a higher ranking in Google (so-called SEO). Instead of setting the default Referer to facebook.com, set it to google.com. There is no need to disable or edit cookies now, because not only the first article is free: as long as you’re from Google the full article will be visible. With these two methods (Referer spoofed and cookies disabled), it is possible to bypass all checks and read everything on the website for free. If they want to make it more difficult to bypass, they should go for an IP address based solution instead of cookies. If that’s too strict (only one device in a network seeing one article for free), they should reconsider if viewing one shared article for free is a great idea at all, because as far as I know there is no other way to make this work without being able to easily bypass it. There is no Facebook API available that confirms if a visitor is coming from a shared post. A different approach could be to put a share button on every page that generates a unique link on click. That results in only unique links being free, and just copy-pasting the article URL in a Facebook post does not. And that’s not very ideal either. Do you have another solution? I’m happy to hear it. Put it in the comments below or contact me by social media on top of this page! Also, if you liked it, please share it on social media, and you’ll be able to read everything for free on my site afterwards ;D

Bruteforcing coupon codes for discount

I sometimes buy stuff from Chinese webstores because of their low prices. Now the yuan value is dropping it is now cheaper than ever to ship products from China.

Banggood.com is one of the more popular Chinese webshops. I was looking around on their website and comparing prices when I found the coupon code b185f7 by googling for 5% off.

I noticed there were more coupons of form b185f7: six characters long, only letters and numbers. I tried them with caps and without caps; it didn’t matter. That means there are only 36 possibilities for every character (0-9a-z) which gives a total possibility of 36^6. That’s not very much: enough to try a bruteforce (a full bruteforce will still take some time; I’m trying it randomly in this post).
Edit: possibly it’s in Hex, which limits the possibilities to 16^6 (0-9a-f), which is even lower and much faster to bruteforce. I didn’t test it, however.

A further thing I noticed that it is unfortunately not possible to enter more than one coupon on a single order. You can only use one at a time. That is a bummer because now two coupons with 5% off won’t give you more discount.
One thing that is good in my case, is that you can endlessly try to enter coupons. It doesn’t matter if they are valid or not; it won’t disable the field after a few wrong tries. And it doesn’t give a captcha to solve.

I wrote the little bash script below to try many coupon possibilities by randomly generating them (not really bruteforcing, just hoping we’re lucky):

#!/bin/bash

while [ 1 ]
do

  couponcode=$(cat /dev/urandom | tr -dc 'a-z0-9' | fold -w 6 | head -n 1)
  curling=$(curl -sS --data "com=shopcart&t=useCoupon&coupon_code=$couponcode" -H 'Cookie: banggood_SID=0f18fd4cf40bfb1dec646807c7fa5522' "https://www.banggood.com/index.php")

  if [[ $curling == *"Coupon is only allowed"* ]] || [[ $curling == *"Invalid"* ]] || [[ $curling == *"expired"* ]] || [[ $curling == "" ]]
then
    echo "$couponcode invalid";
  else
    echo "$couponcode => $curling" >> win.txt;
    echo "$couponcode VALID";
  fi

  sleep 5
done

As you see, it sends a curl request to the banggood website with my session id connected to my cart. I’m trying a infinite amount of time if a random coupon code I get is valid or not. If it gives the message “Coupon is not allowed” or “Invalid Coupon Code” if the code is invalid.
I’m using /dev/urandom as a randomness source and with tr and fold I make sure it is 6 characters long and only contains numbers and letters. As mentioned, caps or not does not matter.

I ran the script for a few hours and it didn’t take long to find valid ones. Unfortunately, they are either only 5% off or only for a specific user account. My hopes were I found more than 5% discount but that wasn’t the case.

I contacted banggood customer service (with only mail address on their site) on the 17th of August but they did not respond. I explained to them what was wrong with their coupon code system and how to fix it. I explained that 6 characters case insensitive isn’t cryptographically secure enough to prevent people from cheating. As a solution I told them to set a maximum on the amount of wrong coupon codes that can be entered. That would solve the problem without breaking functionality.

Unfortunately they didn’t respond. Here are some coupon codes I found while running the script for a short period of time. They are valid coupon codes for 5% discount each:

5bf7bb is 5%
edb44d is 5%
533876 is 5%
e7e744 is 5%
cc9ec5 is 5%
25d342 is 5%

If you are a website owner or developer and you run a webshop, make sure your coupon codes are not easily crackable. Many webshops use original names for their coupon codes instead of generating them, like ‘christmas5off’ or something. These are more difficult to bruteforce (but dictionary may have small chance of success) and therefore more safe to use.

 

How to not implement Premium features in Android

I love to regularly watch new episodes of American series and I prefer to watch them with English or Dutch subtitles. I use a script called subliminal to download them right after downloading an episode or movie from usenet (my Cubox-i4Pro handles this automatically, I can much recommend it as upgrade for your Raspberry Pi!).

Unfortunately subliminal can’t always find the right subtitles, and especially not for downloading an episode that has just been broadcasted in America. This made me look for a way to directly download subtitles from my couch, and I came across this Android app called ‘MightySubs‘ which let you download subtitles from all popular subtitle sites to your samba or ftp share.

MightySubs has a free version and a premium version that costs 99 cents. The free version has some limitations including:

  • There are only two languages available: English and your device’s language setting (premium version has 22 languages).
  • Not more than 10 daily downloads allowed (premium version increases this limit by providing custom account details for addic7ed).

 
When trying to edit a premium setting it gives a toast message like in the picture above, stating it is not available in the free version and the premium version must be bought to enable this option. That got me thinking whether it only blocks the settings panel or actually blocks the functionality inside the app itself. I figured it’s only blocking the option and that it must be possible to change this directly in the saved settings.

Most of the time, an app saves user’s settings by using the Android SharedPreferences class. It writes saved data to an xml file stored inside the ‘shared_prefs’ folder located at /data/data/com.yourpackage where com.yourpackage is the package name of the app.

This is most likely also how MightySubs works. The only thing we need is the package name. That is easy to find by looking at the Google Play store URL: https://play.google.com/store/apps/details?id=info.toyonos.mightysubs. So the app’s prefs data is stored at /data/data/info.toyonos.mightysubs/shared_prefs.
The device needs to be rooted to get inside this dir. Mine is, so I browsed to /data/data/info.toyonos.mightysubs/shared_prefs and there was indeed a file named info.toyones.mightysubs_preferences.xml with the following contents:

<?xml version='1.0' encoding='utf-8' standalone='yes' ?>
<map>
    <string name="PrefCurrentProfileIp">192.168.1.10</string>
    <string name="PrefSubtitlesFetchersList">{&quot;activeFetchers&quot;:[2,1,6]}</string>
    <string name="PrefDailyDownloadCounter">20150814:0</string>
    <boolean name="PrefOnlyWithoutSubtitles" value="false" />
    <boolean name="PrefCurrentProfileDownloadPathEnable" value="false" />
    <string name="PrefSubtitlesManualSelection">1</string>
    <string name="PrefAdditionalExtensions"></string>
    <string name="PrefAddic7edPassword"></string>
    <string name="PrefAddic7edUsername"></string>
    <string name="PrefCurrentProfilePath">/path/to/series/</string>
    <boolean name="PrefFullLanguageExtension" value="false" />
    <string name="PREF_VERSION_KEY">1.5.0</string>
    <string name="PrefCurrentProfileName">Local profile</string>
    <string name="PrefHearingImpaired">2</string>
    <string name="PrefCurrentProfileType">1</string>
    <boolean name="PrefKeepSubtitleFilename" value="false" />
    <string name="PrefActiveProfile">1</string>
    <string name="PrefDefaultLanguage">EN</string>
    <string name="PrefLanguageExtension">0</string>
    <string name="PrefCurrentProfileMediaType">0</string>
    <null name="PrefCurrentProfileUsername" />
    <null name="PrefCurrentProfilePassword" />
</map>

This is the file where all the settings are stored. And it is possible to edit them thanks to our root access. Looking at the names of the strings, it seems like PrefDefaultLanguage is the one that contains the enabled languages. It is currently set to “EN”.

Editing this string should give us more languages. My first try was to change it to “EN NL” to enable Dutch subtitles, but that made the app crash. I tried “EN,NL”, but same result. My last try was “EN;NL” and that worked. It unlocked the premium feature.

 
I also entered my addic7ed account details, which also bypasses the daily download limit. There is no need to buy the premium version anymore.

Contacting the developer

I sent an email about my discovery to the MightySubs developer on 31th of July, to the email provided in the Google Play Store. Unfortunately I have not received a response and the app has not been updated. (article updated, see below)

I would still suggest buying the premium version of the app if you are going to use premium features. It’s a great working app and well worth the 99 cents.

If you make premium features and you want to make sure people aren’t directly editing your settings, do not just disable the settings dialog, but also confirm in the app itself whether the right options are chosen. Even better would be to completely remove unused functionality: this would also decrease the app’s download size (only removing is probably a bit more work).

By all means, make sure that people are not able to use a setting that you don’t want them to use. This counts for a lot of things in security though, and sometimes people tend to forget this, also often on website dropdown forms.
 
Update 16/08/2015: I received a reply from the developer. He wrote (click here for all):

“I read your blog entry 🙂 Nice job. Just one thing. The trick you explained, about the language, is not accurate. You transformed the language setting from EN to EN;NL (both English and Dutch). But it is permitted in the free version. Both languages are already here and can be selected together. A real hack would have been turning EN to DE or FR (not allowed in the free version for you).
About Addic7ed credentials, filling them doesn’t overcome the limitation (10 per day). It just allows premium users to have a better quota on this site. It could be a real pain to be unlimited on MightySubs but limited on Addic7ed.

Anyway your demonstration is accurate, the hack is real and I should fix that.”

He says that both English and Dutch are permitted in the free version, but Dutch was not available for me and it showed a message “Purchase the premium version to get more languages”. Any how, it doesn’t matter, as it is possible to change it to whatever language you want including DE;FR (I tested it).
I wasn’t aware that the account details didn’t bypass the limitation, however in the file above there is also a string called PrefDailyDownloadCounter which probably counts the number of subtitles downloaded so far. I did not test it, but resetting it to zero after ten downloads would probably work.

The developer further wrote that he will bring out a new version in September with new features. Hopefully this hack is also fixed by then.