Sharing Media in Progressive Web Apps

With the widespread adoption of HTML5 standards, the lines between native and web applications are becoming more blurry. Now, increasing native features have become available for the browsers, making it difficult to decide whether to build a native or web application.

Here at NearForm we love pushing the boundaries of disruptive technology, and in a sense, this is what Progressive Web Apps (PWAs) are.

For more info see our other articles on Progressive Web Apps :

Let’s remind ourselves what Progressive Web Apps are:

Given that the APIs available for web applications are very close to what the native version has to offer we decided to build a Progressive Web App that can take pictures and then share them via the native device methods. Demo - Source In order to take the actual picture, we made use of the HTML5 <video width="300" height="150"> tag element. This allows us to use one of the devices `videoinput` sensors as a streaming source essentially piping all the input that is coming through via that stream to the <video></video> element in the UI. In order to access the video sensors on your device, you need to use the `navigator.mediaDevices` web API which has good support across browsers.

What we need to do is select the desired `videoinput` sensor from the available mediaDevices and pipe that into the <video width="300" height="150"></video> HTML5 element.

For example, to list all available devices on an Android device you would print out:

And you would get a list similar to this:

The list contains all the devices available for streaming input to your website, could be audio or video, but for taking pictures the `videoinput` kind is the one we want.

We can see that the two available `videoinput` sensors are for the front and back camera, as they are labelled accordingly.

In order to grab a picture from the stream what we want is essentially a freeze frame of the stream.

To do that we want to draw to the canvas a frame from the video stream, then stop that stream.

This gives us the image we want, but it is not in a usable format yet.

If we want to upload it to a static asset hosting service then we need to convert it to a binary object, and with JS that would be a `Blob` type.

After conversion we upload the file and retrieve the link, so now our next goal is to share it using the native share context menu.

The Web Share API is not available on most browsers, so as we want to be Progressive about it we will use it where available.

The problem now is that part of the devices that want to use the App will come from iOS so we need a sensible fallback for the lack of support.

A good workaround for this missing feature for iOS is using deep-links for sending data directly to the native apps and bypassing the context menu.

On both iOS and Android, you can deeply link to an app by using a specific url, and you can send the app some url encoded data:

You don’t have access to the native sharing context on iOS but you can build your own web components to share the link to your picture to the various native applications.

This makes it more customizable but is different from the native share.

The Media Streaming API gives us access to the video stream so we can do processing on the stream frames.

We can use libraries to extract data from the frames, and there are various third-party libraries available for decoding different interesting information from images.

For demo purposes, we used QR codes as they provide a standard for encoding text in images.

We used a JS port of a popular library reading and generating QR codes. ZXing has ports into many languages and is a de facto standard for working with QR codes.

The port used in our project is a smaller version that allows for detecting and decoding the QR from an image. The library used is jsQR .

What the library does is, it scans a given image and detects the position in the frame of a QR code if any, and if one is present it decodes it, returning the text data.

The approach we use for taking the pictures and also scanning the QR code allows us to go beyond only decoding, but we could also allow highlighting the QR code if one is detected, eg:

We can use the Streaming Media API in real time and extract a lot of information from the video/pictures. We can decode standardized images and extract information.

There are quite a few libraries that have been ported to JS that we can use for image processing and the HTML5 canvas is a very powerful tool that allows us to enhance them. QR code scanning is only a small example of the potential usage in Progressive Web Apps.

The Web Share API is not yet mature enough to support the native sharing methods, but the fallback for the browser that doesn’t yet support it is acceptable for most use cases. It is only when the share context menu is highly custom that this is a realistic limitation.

Sharing only works with text content so the sharing of binary content(images) is not supported via either the Web Share API or the deep-linking alternative. That being said a deep-linking mechanism is a valid option and it works for both iOS and Android, so it can be used in older devices too.

Most big companies have built Progressive Web Apps to complement their existing native applications; Instagram, Facebook, Pinterest and Twitter just to mention a few. Even though the support for some of the features is not yet fully there we should use the feature if it’s there and provide a sensible fallback if it is not.

The code for the above app can be found here. For any further queries about PWA's or whether you need to design and build new apps or modernise existing applications, please contact NearForm at info@nearform.com .