Skip to main content
Redhat Developers  Logo
  • Products

    Featured

    • Red Hat Enterprise Linux
      Red Hat Enterprise Linux Icon
    • Red Hat OpenShift AI
      Red Hat OpenShift AI
    • Red Hat Enterprise Linux AI
      Linux icon inside of a brain
    • Image mode for Red Hat Enterprise Linux
      RHEL image mode
    • Red Hat OpenShift
      Openshift icon
    • Red Hat Ansible Automation Platform
      Ansible icon
    • Red Hat Developer Hub
      Developer Hub
    • View All Red Hat Products
    • Linux

      • Red Hat Enterprise Linux
      • Image mode for Red Hat Enterprise Linux
      • Red Hat Universal Base Images (UBI)
    • Java runtimes & frameworks

      • JBoss Enterprise Application Platform
      • Red Hat build of OpenJDK
    • Kubernetes

      • Red Hat OpenShift
      • Microsoft Azure Red Hat OpenShift
      • Red Hat OpenShift Virtualization
      • Red Hat OpenShift Lightspeed
    • Integration & App Connectivity

      • Red Hat Build of Apache Camel
      • Red Hat Service Interconnect
      • Red Hat Connectivity Link
    • AI/ML

      • Red Hat OpenShift AI
      • Red Hat Enterprise Linux AI
    • Automation

      • Red Hat Ansible Automation Platform
      • Red Hat Ansible Lightspeed
    • Developer tools

      • Red Hat Trusted Software Supply Chain
      • Podman Desktop
      • Red Hat OpenShift Dev Spaces
    • Developer Sandbox

      Developer Sandbox
      Try Red Hat products and technologies without setup or configuration fees for 30 days with this shared Openshift and Kubernetes cluster.
    • Try at no cost
  • Technologies

    Featured

    • AI/ML
      AI/ML Icon
    • Linux
      Linux Icon
    • Kubernetes
      Cloud icon
    • Automation
      Automation Icon showing arrows moving in a circle around a gear
    • View All Technologies
    • Programming Languages & Frameworks

      • Java
      • Python
      • JavaScript
    • System Design & Architecture

      • Red Hat architecture and design patterns
      • Microservices
      • Event-Driven Architecture
      • Databases
    • Developer Productivity

      • Developer productivity
      • Developer Tools
      • GitOps
    • Secure Development & Architectures

      • Security
      • Secure coding
    • Platform Engineering

      • DevOps
      • DevSecOps
      • Ansible automation for applications and services
    • Automated Data Processing

      • AI/ML
      • Data Science
      • Apache Kafka on Kubernetes
      • View All Technologies
    • Start exploring in the Developer Sandbox for free

      sandbox graphic
      Try Red Hat's products and technologies without setup or configuration.
    • Try at no cost
  • Learn

    Featured

    • Kubernetes & Cloud Native
      Openshift icon
    • Linux
      Rhel icon
    • Automation
      Ansible cloud icon
    • Java
      Java icon
    • AI/ML
      AI/ML Icon
    • View All Learning Resources

    E-Books

    • GitOps Cookbook
    • Podman in Action
    • Kubernetes Operators
    • The Path to GitOps
    • View All E-books

    Cheat Sheets

    • Linux Commands
    • Bash Commands
    • Git
    • systemd Commands
    • View All Cheat Sheets

    Documentation

    • API Catalog
    • Product Documentation
    • Legacy Documentation
    • Red Hat Learning

      Learning image
      Boost your technical skills to expert-level with the help of interactive lessons offered by various Red Hat Learning programs.
    • Explore Red Hat Learning
  • Developer Sandbox

    Developer Sandbox

    • Access Red Hat’s products and technologies without setup or configuration, and start developing quicker than ever before with our new, no-cost sandbox environments.
    • Explore Developer Sandbox

    Featured Developer Sandbox activities

    • Get started with your Developer Sandbox
    • OpenShift virtualization and application modernization using the Developer Sandbox
    • Explore all Developer Sandbox activities

    Ready to start developing apps?

    • Try at no cost
  • Blog
  • Events
  • Videos

Tear Down Data Silos with Mobile Microservices

November 3, 2015
Cian Clarke
Related topics:
Microservices
Related products:
Red Hat OpenShift

Share:

    A huge problem facing modern enterprises is managing the large software systems and applications they deal with on a daily basis.

    Be it the CRM system purchased by a predecessor, a bundled HRM product thrown in to sweeten a deal, or the CMS that marketing could not live without, silos of information exist in the modern enterprise, and it can often be difficult to utilize the data that these systems contain. When an enterprise decides to buy a proprietary system, often little consideration is given to future interoperability of that product. Many of these decisions were made before the mobile era of computing began, and these products share many common characteristics which are ill-suited to the changing face of computing.

    Designed for the Desktop Era

    These applications are typically delivered through a desktop browser. The limited screen real estate of modern smartphones and tablets makes interacting cumbersome. Mobile experiences are often retro-fitted to these products as an afterthought. Sometimes vendors release a mobile application to the app store, but these can be limited in functionality.

    Designed to Silo Data

    I’m not going to present as fact the concept that many software vendors design products which are a closed black box of data to keep you coming back to buy more - but I can certainly suggest this.
    It’s often not in the interest of the vendor to make a product interoperable with other systems, because that same interoperability can make it very easy to jump ship. This means any API for connecting to the system is at best an afterthought, and often doesn’t exist at all.
    This leaves us with a silo of information which has grown over time, and a system of record which severely limits our future expansion.

    Designed for the Desk, not the Pocket

    Interaction with software was vastly different pre-mobile. Users were behind a desk, or perhaps sat with a laptop computer. The tasks they sought to accomplish were longer-running,  e.g. sending an email or filing an expense report. People spent time interacting with software, over a more traditional request-response API interaction.

    In the mobile era, users perform tasks standing in line at the coffee shop, or walking through the airport terminal. These tasks are geared towards shorter interactions e.g. adding a comment to a case or approving a workflow.

    Designed for Some-time, not Real-time

    We’re interacting with APIs in short, bursty sessions - so we need to design our API for this new world of mobile. We’ve established in a previous blog post[1] the importance of making cohesive, mobile-specific APIs which trim payloads, and combine multiple API responses, but sometimes this isn’t enough.
    We may need to consider building an API which can respond in real-time to changes in data over an always-open connection like websockets, or be able to use push APIs to update a mobile user with some new information, even when the device is asleep in their pocket.

    This is a far cry from the functionality which these monolithic silo applications provide.

    Designed for Dumb Sensors, not Smart Ones

    Modern smart-devices have a plethora of sensors which, when used collectively, make much smarter systems of engagement. Traditional software systems were not built with such extensive inputs in mind, often only storing simple text and numbers, with no awareness of concepts such as location.

    Designed to be Expensive

    Chances are this same vendor of a piece of software which costs a large sum of money in the late 2000’s wants to charge you more to add mobile functionality. There’s an alternative! Instead of paying for the vendor-supplied solution for mobile, which will likely be outdated in two years, why not take ownership and create your own?

    Setting Information Free

    We’ve established that most enterprise systems haven’t been built with mobile in mind, and using vendor-provided solutions can often introduce greater cost. However, using an open source software stack, combined with a microservices architecture, it’s very possible to free data from these proprietary systems, moving from a closed, desktop-centric system of record to a highly expansible, mobile-enabled system of engagement.

    The Traveling Salesperson

    Remember our travelling umbrella salesperson of Microservices for Mobile fame? In this previous post, we found using a microservices architecture made for much faster response times on lossy networks.
    They’ve since discovered umbrellas aren’t really a hot market right now, and have decided to move into selling birdhouses. They are as of yet unaware of the migratory avian patterns which make October a bad time of year for business, and, yes, I’m well aware these examples keep getting more and more contrived - but that’s part of the fun!

    Today, we’re going to build a Mobile-Specific Microservice, which will accept information from the field about birdhouse orders, and interface with Salesforce and Sharepoint connectors in a flexible manner.
    We’re also going to make use of the sensory capabilities of the device which we wouldn’t get with paper to build a mobile app which collects the location where the birdhouses are to be stored.
    Lastly, we’re going to use some open source Node.js modules to pull customer data from Salesforce and product information from a Sharepoint List, and use these to populate the mobile app with data.

    Standing Up Connectors

    I’m going to be using a trial account of both Salesforce and Sharepoint 365 for today’s integration. Salesforce will contain the Accounts with which we’re going to associate our orders.
    SharePoint will have a list, each item thereof representing a product’s information.
    Before standing up these microservices, I need to gather some information about my instances - usernames, passwords and the service URLs associated. I’ve used the Red Hat Mobile Sharepoint and Salesforce connector. These expose discoverable interfaces to these services within our platform, but can be used standalone too.

    You could also write your own connectors - we’ve used the Sharepointer and Node Salesforce modules in our connectors. Whatever approach you take, know that these connector microservices should be reusable by any future projects which may need to pull data from Sharepoint or Salesforce.

    Taking Ownership of our Data

    Instead of storing our installation information in one of the aforementioned systems, we’re going to take ownership of our IT infrastructure the microservices way.

    We’re going to build a small mobile microservice which stores information in an orders database. If we ever need to move this data elsewhere, we’ve already got ownership over the schema, and control over the location of the system of record. We can swap this microservice for a more resilient storage solution in future, should the need arise.

    We’re just going to expose a Create (i.e. POST /orders) and List (i.e. GET /orders) interface to this service. We've already set up a db connection.

    Code Snippet:

    const COLLECTION = 'orders';
    orders.get('/', function(req, res) {
      var collection = db.collection(COLLECTION);
      collection.find().toArray(function(err, results){
        if (err) {
          return res.status(500).json(err);
        }
        return res.json(results);
      });
    });
    
    orders.post('/', function(req, res) {
      var collection = db.collection(COLLECTION);
      collection.insert([req.body], function(err) {
        if (err) {
          return res.status(500).json(err);
        }
        return res.json(req.body);
      });
    });
    

    You can also see the full Orders Database Microservice on GitHub.

    Enter the Mobile Microservice

    Now that we’ve created a series of microservices to help us with this project, there’s one step left. We’re going to create a microservice responsible for interfacing with the mobile app, a gateway of sorts.
    Within this microservice, we’ll massage data into a format more suitable for mobile, and even mash up data from multiple sources to reduce the number of requests from mobile - since we learned in a previous blog post this can have a drastic impact on performance.

    Overcoming Sharepoint API Oddities

    Sharepoint has a number of unique intricacies which make it a challenging platform to integrate with - I’ve written about these at length.
    It came as some surprise that there could be another surprise the SharePoint API could throw at me.

    Turns out, when I create a custom list, like I did to store my products, the field names I set up in my headings are output in the API response keyed with 4 random letters and numbers.
    I then have to look these four character IDs up, iterating over the entire fields array, to find out what it relates to.
    This would be an incredibly fragile binding to have baked into our mobile app, so we’ll do this in our mobile microservice.

    Adding Caching

    When integrating with Salesforce, we encounter a different problem. The time it took to login then retrieve accounts varied wildly, from instant responses to timing out after 60 seconds.
    It’s entirely possible I was doing something incorrect with the API code, but since we’re reaching out to third parties where the data doesn’t change too often, I figured a cache wouldn’t hurt.
    Since we’re writing ExpressJS middleware in Node.js, we can add in some middleware after we mount our orders routes - that way, they don’t get cached.
    As with fh.db before, fh.cache is a convenience wrapper on the Redis cache - of course, we could just use Redis directly.

    Code Snippet:

    api.use(function(req, res, next){
      var cacheKey = _.last(req.path.split('/'));
      return fh.cache({ act : 'load', key : cacheKey }, function(err, cacheRes){
        if (!err && cacheRes){
          cacheRes = JSON.parse(cacheRes);
          return res.json(cacheRes)
        }
        return next();
      });
    });
    

    You can also see this middleware in action on GitHub - note that we included it after the ‘orders’ routes.

    Now, we’re caching both Salesforce and Sharepoint responses - much better!

    The Mobile Specific Mashup

    Now that we have two connectors, can return orders and products, and can create orders in our database microservice, there’s one piece of cleverness left to do. When we retrieve our list of orders from the database, we store the Id of the product in SharePoint, and the Id of the Account in Salesforce. These are foreign keys of sorts, into external systems, and we’d like to know more about these fields in our mobile apps. The solution is simple - when we return the field to our mobile app, replace its contents with the full Account and Product objects. We have this information in our cache, so there’s no need to do a full API call - so with some async Node.js magic, we can retrieve the information we need.

    var async = require(‘async’);
    // Load both our Accounts list and our Products list from our cache
    async.parallel({
      accounts : async.apply(fh.cache, { act : 'load', key : 'accounts' }),
      products : async.apply(fh.cache, { act : 'load', key : 'products' })
    }, function(err, cacheResults){
      // We store stringified values in the cache.
      //  JSON.parse them & we have a full list of accounts and products
      var accounts = JSON.parse(cacheResults.accounts),
      products = JSON.parse(cacheResults.products);
    
      //TODO: Now we just need to find the relevant account and product,
      // and include this in our response
    });
    

     

    You can see the full code on GitHub.

    Writing the Mobile Client

    We have our server-side code finished, now I’m going to create a hybrid mobile app to utilize these new services. We’re going to inject the contents of our products listing from Sharepoint into a select element, and our accounts listing into another select element. We’re going to list all completed orders, and draw a table - making use of our mashup data by retrieving information which wouldn’t otherwise be available to us such as the account name and phone number.

    The Result

    Through the use of a microservices architecture, we’ve enabled field staff to place orders, and retrieve historical order information. We’re no longer at ransom to big vendors for our data - instead of storing this information in a proprietary data silo, we’ve taken steps to ensure we take control and ownership. You can see all the components of this example on GitHub.

    Through our use of connecting microservices to integrate third party systems, and business logic in our mobile specific microservice, we’ve ensured our relationship with third party software is loosely coupled.
    We can swap out integrations for other systems in future should we ever choose to migrate, or take even more of this data in house.

    Adopting a microservices approach for mobile helps us reduce our costs, iterate faster, and helps with future expansibility of our mobility efforts. It should be considered an essential part of any enterprise app developer’s toolkit.

    [1] Evolving a Mobile-centric Architecture: The Microservices Way
    https://developers.redhat.com/blog/2015/04/15/evolving-a-mobile-centric-architecture-the-microservices-way/

     

    Last updated: September 19, 2023

    Recent Posts

    • How to run a fraud detection AI model on RHEL CVMs

    • How we use software provenance at Red Hat

    • Alternatives to creating bootc images from scratch

    • How to update OpenStack Services on OpenShift

    • How to integrate vLLM inference into your macOS and iOS apps

    Red Hat Developers logo LinkedIn YouTube Twitter Facebook

    Products

    • Red Hat Enterprise Linux
    • Red Hat OpenShift
    • Red Hat Ansible Automation Platform

    Build

    • Developer Sandbox
    • Developer Tools
    • Interactive Tutorials
    • API Catalog

    Quicklinks

    • Learning Resources
    • E-books
    • Cheat Sheets
    • Blog
    • Events
    • Newsletter

    Communicate

    • About us
    • Contact sales
    • Find a partner
    • Report a website issue
    • Site Status Dashboard
    • Report a security problem

    RED HAT DEVELOPER

    Build here. Go anywhere.

    We serve the builders. The problem solvers who create careers with code.

    Join us if you’re a developer, software engineer, web designer, front-end designer, UX designer, computer scientist, architect, tester, product manager, project manager or team lead.

    Sign me up

    Red Hat legal and privacy links

    • About Red Hat
    • Jobs
    • Events
    • Locations
    • Contact Red Hat
    • Red Hat Blog
    • Inclusion at Red Hat
    • Cool Stuff Store
    • Red Hat Summit

    Red Hat legal and privacy links

    • Privacy statement
    • Terms of use
    • All policies and guidelines
    • Digital accessibility

    Report a website issue