Converge updated to use new aggregator service

Converge

TLDR: Get latest app update from store to continue using Converge starting next month.

Converge was built using parse.com, which was great until Facebook bought and decided to shut it down. šŸ˜± Since the announcement last year, I have looked at alternative “back-end as a service” offerings to replace parse.com but couldn’t find anything as good and free. After all, Converge is a free app without any ads so didn’t make any sense to spend $$$ every month.

I finally decided to build my own back-end service to also change the news aggregation infrastructure which can scale to many more news publishers without ongoing additional work, and be almost real time. The initial version was finally complete last week including the app changes required to use the new service. All these changes are now live. Yay!

To continue using Converge once parse.com shuts down at the end of this month, you will need to download the latest version from store.

Now that the app and aggregation service areĀ live, I am hoping to add more publishers and bring Converge to other platforms very soon, starting with Android. Eventually, Converge will need to make money to pay for server costs but that can be figured out later.Ā šŸ˜Š

Cheers!

image credit: wpcentral

Using gzip compression when making HTTP requests in Windows Phone apps

Bandwidth and data consumption on mobile devices remain one of the most critical resources an app consumes on any platform. A majority of users have limited data plans and actually pay more if they use more data on their mobile devices. This makes it critical for us, as app developers, to ensure our apps consume as less data as possible. And when apps download less data, they perform better and become more responsive too.

I overlooked this myself earlier but fixed it as soon as I noticed. In fact,Ā a lot of apps is store are not using compression including some heavily used apps like famous YouTube clients. The response size for 10 entries in most popular list from YouTube is ~50KB by default and ~7.4KB when compression is used, that’s 85% less data downlaoded. It varies for each request but the savings are somewhere from 30% to 85%.

Overview

We all use compressed archives on PCs in one format or the other like zip, rar, 7z and others. For the web, what it really means is that the data being transferred over the network is compressed and hence uses less bandwidth. This is defined in http spec as a standard. When a client, like your app, sends a request, it can tell the server that it’s capable of handling compressed response in Accept-Encoding request header. The server then sends the response in compressed format (if it’s capable of doing so) and tells the client using Content-Encoding response header. Obviously, the server and client both not only need to support this but also send consent that this capability is used. This is how http works for everything so no surprises there.

Good news is that you don’t have to implement compression to take advantage of this as the HttpClient library you should be using added supports for this over a year ago. Unfortunately, this wasn’t the case in early days of Windows Phone but workarounds existed. Ideally, the API should use compression by default but it doesn’t.

Clarification

The HttpClient I mentioned above is for Windows Phone 7.x and 8.x apps using Silverlight not WinRT. The .NET stack has had multiple APIs for making web request overtime but HttpClient is the one everyone should be using now which is wrapper/replacement for all other HTTP APIs in .NET framework.

Starting with WinRT which is used for developing “universal” apps, there is a separate HttpClient API. This seems to be using compression by default but please verify that it is. Here is a good starting point for HttpClient in WinRT and samples. If this all seems very confusing, you are not alone. The HTTP API in Microsoft world is a mystery wonderland of confusion so ask if you have questions.

Implementation

Using compression in your app is extremely simple and quick as explained by the framework team. When you create an instance of HttpClient class, send an instance of HttpClientHandler in constructor with appropriate setup and you are done. All http requests sent using this instance will now send correct request header and automatically un-compress the response.

var handler = new HttpClientHandler();
if (handler.SupportsAutomaticDecompression)
{
    handler.AutomaticDecompression = DecompressionMethods.GZip |
                                     DecompressionMethods.Deflate;
}

var client = new HttpClient(handler);

 

Performance Tip

While you are at it, here is another performance tip. When you are downloading data from internet and deserializing it, instead of first storing response as a string and then parsing it, parse data directly from the response stream. This works seamlessly for JSON as well as XML content.

For JSON, JSON.NET supports deserializing data from stream. It’s not as obvious as deserializing from a string though. Using the HttpClient instance we created above, here is a code snippet to deserialize response stream.

try
{
    using (var stream = await httpClient.GetStreamAsync(url))
    {
        using (var streamReader = new StreamReader(stream))
        {
            using (var textReader = new JsonTextReader(streamReader))
            {
                var serializer = new JsonSerializer();
                var data = serializer.Deserialize<List<Entry>>(textReader);
            }
        }
    }
}
catch (ObjectDisposedException ex)
{
    // ignore this exception as stream gets closed somehow and "using" block tries to dispose it again
    Debug.WriteLine(ex);
}

For XML response, XDocument supports parsing streams directly.

using (var stream = await httpClient.GetStreamAsync(url))
{
    XDocument.Load(stream);
}

For using stream with JSON.NET, I came across an issue though. As I am using “using” constructs to ensure the resources get cleaned up, the stream gets closed twice which causes an exception. To workaround that, ignore ObjectDisposedException exception as I have doneĀ above.

One drawback of using compression is that the phone will have to spend some CPU cycles to uncompress data. With today’s phones though, this is negligible. There is no reason to not use compression in your apps.

Hope this helps.

image credit: wpcentral

Notifying users when app update is available in Windows Phone store

Ensuring that users of your app are on latest version is critically important. I have seen users reporting issues and requesting features which were implemented a few updates ago. With Windows Phone 8.1, users can enable automatic app updates installationĀ but for users whoĀ are still on Windows 8 or who do not enable automatic updates, the responsibility is on you, as a developer.

Unfortunately, there is no support in Windows Phone SDK to find out if there is a new version of the current app in store or what version is available in store. For theĀ converge app, I was looking to solve this issue so I traced store app’s network traffic using Fiddler to understand the available APIs. Looking at the network traffic, the API used by store app is fairly obvious. Here is the API request Windows Phone 8 store app sends for app details.

GET http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/b658425e-ba4c-4478-9af3-791fd0f1abfe?os=8.0.10521.0&cc=US&lang=en-US&hw=520208901&dm=RM-940_nam_att_200&oemId=NOKIA&moId=att-us&cf=99-1
Accept: */*
Accept-Encoding: gzip
User-Agent: ZDM/4.0; Windows Mobile 8.0
X-WP-Client-Config-Version: 107
X-WP-MO-Config-Version: 1224
X-WP-Device-ID: ****
X-WP-ImpressionId: ****:API.BDIGeneric
X-WP-ImpressionK: 5

You don’t need to send all headers or parameters in requests. The response to this call has complete details for your application in the target country/language in xml format. From here on, you can use LINQ to XML to get the data you need to know what version is available in store. Here is slightly modified version of the code I use in the converge app. Feel free to reuse.

//This code has dependency on Microsoft.Net.Http NuGet package.
public async Task CheckUpdate()
{
    const string storeAppDetailsUri = "http://marketplaceedgeservice.windowsphone.com/v8/catalog/apps/b658425e-ba4c-4478-9af3-791fd0f1abfe?os={0}&cc={1}&lang={2}";

    var updatedAvailable = false;

    try
    {
        var osVersion = Environment.OSVersion.Version.ToString(4);
        var lang = CultureInfo.CurrentCulture.Name;
        var countryCode = lang.Length == 5 ? lang.Substring(3) : "US";

        using (var message = new HttpRequestMessage(HttpMethod.Get, string.Format(storeAppDetailsUri, osVersion, countryCode, lang)))
        {
            message.Headers.Add("User-Agent", "Windows Mobile 8.0");

            using (var client = new HttpClient())
            {
                var response = await client.SendAsync(message);
                if (response.StatusCode != HttpStatusCode.OK) return;

                using (var stream = await response.Content.ReadAsStreamAsync())
                {
                    XNamespace atom = "http://www.w3.org/2005/Atom";
                    XNamespace apps = "http://schemas.zune.net/catalog/apps/2008/02";

                    var doc = XDocument.Load(stream);
                    if (doc.Document == null) return;

                    var entry = doc.Document.Descendants(atom + "feed")
                        .Descendants(atom + "entry")
                        .FirstOrDefault();

                    if (entry == null) return;

                    var versionElement = entry.Elements(apps + "version").FirstOrDefault();
                    if (versionElement == null) return;

                    Version storeVersion;

                    if (Version.TryParse(versionElement.Value, out storeVersion))
                    {
                        var currentVersion = new AssemblyName(Assembly.GetExecutingAssembly().FullName).Version;

                        updatedAvailable = storeVersion > currentVersion;
                    }
                }
            }
        }
    }
    catch (Exception ex)
    {
        // HANDLE ERROR HERE. THERE IS NO POINT IN SHOWING USER A MESSAGE. GOOD PLACE TO SEND SILENT ERROR REPORT OR JUST SWALLOW THE EXCEPTION.
        Debug.WriteLine(ex);
    }

    if (updatedAvailable)
    {
        // APP UPDATE IS AVAILABLE. SHOW FIREWORKS IN APP OR JUST OPEN STORE APP TO UPDATE THE APP AFTER USER'S CONSENT.
    }
}

This is definitely not a documented and supported APIĀ  but considering there are millions of devices using this API, I doubt this will break in foreseeable future. I just don’t understand why Windows Phone teamĀ doesn’t make this API public and have folks build stuff on top of it. They have struggled to produce a good store experience since Windows Phone launched anyways. For Windows 8, the APIs are as useless as the store app is so don’t bother trying to do something similar there.

A side note: regardless of how good you think the updates are, don’t update the app too frequently (multiple times in a week) Ā if you have this notification implemented in the app. This becomes annoying and users start reporting it in reviews. This happened when I had to release multiple updates in a week to resolve content infringement complaints submitted by The Verge.

Hope this help.

The Verge app for Windows Phone

If you are part of the technology world, there is a good chance you read The Verge for getting your fix of news, reviews and their fantastic long reads not only related to technology but also science, art and culture. If want to do that on a Windows Phone though, there are no good options. Until now.

The Verge

Over the last few weekends, I have built a new app for Windows Phone for consuming everything The Verge has to offer. Here are the features it currently supports.

  • Super smooth experience for browsing and reading all the latest news, reviews, features, and exclusives.
  • Read comments posted by The Verge readers.
  • Watch “On The Verge,” “The Vergecast,” “Small Empires,” and more without ever leaving the app.
  • Share news and stories with your friends via Facebook, Twitter, and email.

More features like forums and live tiles are not implemented in the app just yet but I plan to add these over the coming weeks. Feel free to suggest or vote on feature requests on the feedback site. Here is some feedback from the community so far.

Great. The best by far. Great design. Better and more good looking than the android app.

As much as most of The Verge crew don’t like WP, this is a great app to prove how good it can be.

I suddenly feel like we don’t need an official app anymore. Awesome.

Obviously, theĀ app is neither sponsored nor endorsed by The Verge or Vox Media.

Khan Academy on Windows 8: Now Open Source

Khan Academy app launched with Windows 8 last October. A few of us worked together to get the first version developed just in time for Windows 8 launch. Since then, there have been a few minor updates and a big overhaul which was published in store last month. This week is another great milestone: the app is now open source and ready for anyone to contribute.

KhanAcademyInStore

How We Got Here

I developed the initial version with a lot of help from folks over at Khan Academy and an internal Microsoft team. It was really a port of Learning Circle phone app to Windows 8 so naturally, this was developed using C# and XAML.

Later last year, a few more folks from Microsfot and BGC3 stepped up to help and we got some funding. That helped to get Pixel Lab team involved with development and after working on it for a few months, Ā the next version of the app was published in store last month. This new version was developed in HTML5/JavaScript and this is what we open sourced this week.

Get Involved

If you have experience in developing Windows 8 apps using HTML5 and JavaScript, come join us in making a great Khan Academy app for Windows 8. In addition to HTML5 and JavaScript, having good understanding of WinJS, WinRT, TypeScript and web in general would be great. If you don’t have development experience, you can still help with testing.

Let’s Go!