A huge problem facing modern enterprises is managing the large software systems and applications they deal with on a daily basis.
Be it the CRM system purchased by a predecessor, a bundled HRM product thrown in to sweeten a deal, or the CMS that marketing could not live without, silos of information exist in the modern enterprise, and it can often be difficult to utilize the data that these systems contain. When an enterprise decides to buy a proprietary system, often little consideration is given to future interoperability of that product. Many of these decisions were made before the mobile era of computing began, and these products share many common characteristics which are ill-suited to the changing face of computing.
Designed for the Desktop Era
These applications are typically delivered through a desktop browser. The limited screen real estate of modern smartphones and tablets makes interacting cumbersome. Mobile experiences are often retro-fitted to these products as an afterthought. Sometimes vendors release a mobile application to the app store, but these can be limited in functionality.
Designed to Silo Data
I’m not going to present as fact the concept that many software vendors design products which are a closed black box of data to keep you coming back to buy more - but I can certainly suggest this.
It’s often not in the interest of the vendor to make a product interoperable with other systems, because that same interoperability can make it very easy to jump ship. This means any API for connecting to the system is at best an afterthought, and often doesn’t exist at all.
This leaves us with a silo of information which has grown over time, and a system of record which severely limits our future expansion.
Designed for the Desk, not the Pocket
Interaction with software was vastly different pre-mobile. Users were behind a desk, or perhaps sat with a laptop computer. The tasks they sought to accomplish were longer-running, e.g. sending an email or filing an expense report. People spent time interacting with software, over a more traditional request-response API interaction.
In the mobile era, users perform tasks standing in line at the coffee shop, or walking through the airport terminal. These tasks are geared towards shorter interactions e.g. adding a comment to a case or approving a workflow.
Designed for Some-time, not Real-time
We’re interacting with APIs in short, bursty sessions - so we need to design our API for this new world of mobile. We’ve established in a previous blog post[1] the importance of making cohesive, mobile-specific APIs which trim payloads, and combine multiple API responses, but sometimes this isn’t enough.
We may need to consider building an API which can respond in real-time to changes in data over an always-open connection like websockets, or be able to use push APIs to update a mobile user with some new information, even when the device is asleep in their pocket.
This is a far cry from the functionality which these monolithic silo applications provide.
Designed for Dumb Sensors, not Smart Ones
Modern smart-devices have a plethora of sensors which, when used collectively, make much smarter systems of engagement. Traditional software systems were not built with such extensive inputs in mind, often only storing simple text and numbers, with no awareness of concepts such as location.
Designed to be Expensive
Chances are this same vendor of a piece of software which costs a large sum of money in the late 2000’s wants to charge you more to add mobile functionality. There’s an alternative! Instead of paying for the vendor-supplied solution for mobile, which will likely be outdated in two years, why not take ownership and create your own?
Setting Information Free
We’ve established that most enterprise systems haven’t been built with mobile in mind, and using vendor-provided solutions can often introduce greater cost. However, using an open source software stack, combined with a microservices architecture, it’s very possible to free data from these proprietary systems, moving from a closed, desktop-centric system of record to a highly expansible, mobile-enabled system of engagement.
The Traveling Salesperson
Remember our travelling umbrella salesperson of Microservices for Mobile fame? In this previous post, we found using a microservices architecture made for much faster response times on lossy networks.
They’ve since discovered umbrellas aren’t really a hot market right now, and have decided to move into selling birdhouses. They are as of yet unaware of the migratory avian patterns which make October a bad time of year for business, and, yes, I’m well aware these examples keep getting more and more contrived - but that’s part of the fun!
Today, we’re going to build a Mobile-Specific Microservice, which will accept information from the field about birdhouse orders, and interface with Salesforce and Sharepoint connectors in a flexible manner.
We’re also going to make use of the sensory capabilities of the device which we wouldn’t get with paper to build a mobile app which collects the location where the birdhouses are to be stored.
Lastly, we’re going to use some open source Node.js modules to pull customer data from Salesforce and product information from a Sharepoint List, and use these to populate the mobile app with data.
Standing Up Connectors
I’m going to be using a trial account of both Salesforce and Sharepoint 365 for today’s integration. Salesforce will contain the Accounts with which we’re going to associate our orders.
SharePoint will have a list, each item thereof representing a product’s information.
Before standing up these microservices, I need to gather some information about my instances - usernames, passwords and the service URLs associated. I’ve used the Red Hat Mobile Sharepoint and Salesforce connector. These expose discoverable interfaces to these services within our platform, but can be used standalone too.
You could also write your own connectors - we’ve used the Sharepointer and Node Salesforce modules in our connectors. Whatever approach you take, know that these connector microservices should be reusable by any future projects which may need to pull data from Sharepoint or Salesforce.
Taking Ownership of our Data
Instead of storing our installation information in one of the aforementioned systems, we’re going to take ownership of our IT infrastructure the microservices way.
We’re going to build a small mobile microservice which stores information in an orders database. If we ever need to move this data elsewhere, we’ve already got ownership over the schema, and control over the location of the system of record. We can swap this microservice for a more resilient storage solution in future, should the need arise.
We’re just going to expose a Create (i.e. POST /orders) and List (i.e. GET /orders) interface to this service. We've already set up a db connection.
Code Snippet:
const COLLECTION = 'orders';
orders.get('/', function(req, res) {
var collection = db.collection(COLLECTION);
collection.find().toArray(function(err, results){
if (err) {
return res.status(500).json(err);
}
return res.json(results);
});
});
orders.post('/', function(req, res) {
var collection = db.collection(COLLECTION);
collection.insert([req.body], function(err) {
if (err) {
return res.status(500).json(err);
}
return res.json(req.body);
});
});
You can also see the full Orders Database Microservice on GitHub.
Enter the Mobile Microservice
Now that we’ve created a series of microservices to help us with this project, there’s one step left. We’re going to create a microservice responsible for interfacing with the mobile app, a gateway of sorts.
Within this microservice, we’ll massage data into a format more suitable for mobile, and even mash up data from multiple sources to reduce the number of requests from mobile - since we learned in a previous blog post this can have a drastic impact on performance.
Overcoming Sharepoint API Oddities
Sharepoint has a number of unique intricacies which make it a challenging platform to integrate with - I’ve written about these at length.
It came as some surprise that there could be another surprise the SharePoint API could throw at me.
Turns out, when I create a custom list, like I did to store my products, the field names I set up in my headings are output in the API response keyed with 4 random letters and numbers.
I then have to look these four character IDs up, iterating over the entire fields array, to find out what it relates to.
This would be an incredibly fragile binding to have baked into our mobile app, so we’ll do this in our mobile microservice.
Adding Caching
When integrating with Salesforce, we encounter a different problem. The time it took to login then retrieve accounts varied wildly, from instant responses to timing out after 60 seconds.
It’s entirely possible I was doing something incorrect with the API code, but since we’re reaching out to third parties where the data doesn’t change too often, I figured a cache wouldn’t hurt.
Since we’re writing ExpressJS middleware in Node.js, we can add in some middleware after we mount our orders routes - that way, they don’t get cached.
As with fh.db before, fh.cache is a convenience wrapper on the Redis cache - of course, we could just use Redis directly.
Code Snippet:
api.use(function(req, res, next){
var cacheKey = _.last(req.path.split('/'));
return fh.cache({ act : 'load', key : cacheKey }, function(err, cacheRes){
if (!err && cacheRes){
cacheRes = JSON.parse(cacheRes);
return res.json(cacheRes)
}
return next();
});
});
You can also see this middleware in action on GitHub - note that we included it after the ‘orders’ routes.
Now, we’re caching both Salesforce and Sharepoint responses - much better!
The Mobile Specific Mashup
Now that we have two connectors, can return orders and products, and can create orders in our database microservice, there’s one piece of cleverness left to do. When we retrieve our list of orders from the database, we store the Id of the product in SharePoint, and the Id of the Account in Salesforce. These are foreign keys of sorts, into external systems, and we’d like to know more about these fields in our mobile apps. The solution is simple - when we return the field to our mobile app, replace its contents with the full Account and Product objects. We have this information in our cache, so there’s no need to do a full API call - so with some async Node.js magic, we can retrieve the information we need.
var async = require(‘async’);
// Load both our Accounts list and our Products list from our cache
async.parallel({
accounts : async.apply(fh.cache, { act : 'load', key : 'accounts' }),
products : async.apply(fh.cache, { act : 'load', key : 'products' })
}, function(err, cacheResults){
// We store stringified values in the cache.
// JSON.parse them & we have a full list of accounts and products
var accounts = JSON.parse(cacheResults.accounts),
products = JSON.parse(cacheResults.products);
//TODO: Now we just need to find the relevant account and product,
// and include this in our response
});
You can see the full code on GitHub.
Writing the Mobile Client
We have our server-side code finished, now I’m going to create a hybrid mobile app to utilize these new services. We’re going to inject the contents of our products listing from Sharepoint into a select element, and our accounts listing into another select element. We’re going to list all completed orders, and draw a table - making use of our mashup data by retrieving information which wouldn’t otherwise be available to us such as the account name and phone number.
The Result
Through the use of a microservices architecture, we’ve enabled field staff to place orders, and retrieve historical order information. We’re no longer at ransom to big vendors for our data - instead of storing this information in a proprietary data silo, we’ve taken steps to ensure we take control and ownership. You can see all the components of this example on GitHub.
Through our use of connecting microservices to integrate third party systems, and business logic in our mobile specific microservice, we’ve ensured our relationship with third party software is loosely coupled.
We can swap out integrations for other systems in future should we ever choose to migrate, or take even more of this data in house.
Adopting a microservices approach for mobile helps us reduce our costs, iterate faster, and helps with future expansibility of our mobility efforts. It should be considered an essential part of any enterprise app developer’s toolkit.
[1] Evolving a Mobile-centric Architecture: The Microservices Way
https://developers.redhat.com/blog/2015/04/15/evolving-a-mobile-centric-architecture-the-microservices-way/
Last updated: September 19, 2023
