In part one of this post, we discussed building a hybrid client application which scans a barcode, and sends this code to the server side of our application. Now, we're going to build the server code to take this barcode number, and look it up on an external service.
We'll be writing this using Node.js - a technology which uses the Javascript programming language, just like we wrote in our hello.js
file in part one, only this time on the server side. We'll also be integrating with a web service, or API. Our applications often need to retrieve data from some system over the internet. To do this, it talks to an API - short for Application Programming Interface. The APIs can talk a number of different languages.
The example we'll see today uses the SOAP protocol, with an interesting twist. We'll show how easy Node.js & some community plugins make integrating with this SOAP service.
The language of APIs: , SOAP, XML, CSV, JSON?
We've mentioned some acronyms already, let's talk a little bit about what all this means. If these terms are already familiar to you, feel free to skip this section.
Most business systems and services talk something called XML - a bit like the HTML we saw earlier in our index.html file
, only for laying out data rather than web pages. SOAP is a protocol built on top of XML which was popular in the late 90's, early 2000's. Although no longer in-vogue, many services - especially in the business world - talk SOAP.
CSV is not a particularly common format for APIs on the web. It's a simple format for representing data often exported from a spreadsheet, and can be useful in certain scenarios where non-technical users need to supply data to developers.
Lastly, there's JSON. Us developers like JSON. It's is a much more modern language which APIs talk. It's much more concise, and better still - it's a native type to JavaScript. What do we mean when we say "native type"? Well, put simply - we can copy a snippet of JSON, paste it into a JavaScript program, and it'll work! Pretty nice.
Let's take a look at some examples of each of these interchange formats. Here, we'll represent a really simple memo, with a "subject" and "body" field.
SOAP
<?xml version="1.0"?> <SOAP-ENV:Envelope xmlns:SOAP-ENV="http://www.w3.org/2001/12/soap-envelope" SOAP-ENV:encodingStyle="http://www.w3.org/2001/12/soap-encoding" > <SOAP-ENV:Body xmlns:m="http://www.example.org/message" > <m:Message> <m:Subject>Hi Cian</m:Subject> <m:Body>Hello World</m:Body> </m:Message> </SOAP-ENV:Body> </SOAP-ENV:Envelope>
XML
<?xml version="1.0"?> <Message> <Subject>Hi Cian</Subject> <Body>Hello World</Body> </Message> </xml>
JSON
{ "message" : { "subject" : "Hi Cian", "body" : "Hello World" } }
CSV
subject,body Hi Cian,Hello World
You'll notice as we move from SOAP to XML, and finally to JSON, our message gets more and more concise - and arguably, more human readable too.
CSV is the exception here - you'll notice we've lost our ability to represent beyond 1 level of depth of data, making it suitable for very little. In this case, we can't have a parent "message" element represented, so we've just dropped it.
WSDL Preamble
One of the pieces of this integration puzzle we'll encounter today is something called a WSDL. This definition describes the web service to a system and the responses we can expect.
Developers often use a WSDL in conjunction with a UI tool like SoapUI, which illustrates how the service works.
In fact, the RedHat Mobile platform uses a much more modern technology called API Blueprint to describe APIs and make them interactive in a similar fashion!
We're going to load this WSDL into a Node.js SOAP client, but it helps to have a rough idea of how the service works first - what params it expects, and what the output will look like. SoapUI can help with this.
Let's get Coding!
Now that we've introduced the various technologies we'll be dealing with, let's look at the anatomy of a Node.js app.
(If you haven't completed part one, you can still this tutorial - just follow steps 1-3 in "Create the Project")
-
- Navigate to our Barcode Project and open the "Cloud App", the tile in the middle of the "Apps, Cloud Apps and Services" screen.
- Visit the editor by clicking on it's menu entry on the left. First, let's open the file `application.js`.
- We're not going to make any changes to this file, but since it's the first piece of code executed in all our templates it's worth taking a look. In here, we set up a web server listening for connections from the outside world. There's one particularly relevant line in here, where we mount the `hello` route, and assign it to the file `hello.js`:
app.use('/hello', require('./lib/hello.js')());
We could change this to something more relevant such as `/barcode`, but for simplicity sake we're just going to leave this be! You'll recall in our previous post that the app sends its barcode to a route with path `hello` in the client javascript we wrote
We'll look at this `hello` route in more detail in a moment - first, we're going to collect the plugins, or Node modules which we need for our project. - Our project needs two new libraries not included by default. One is Node Soap, and the other is CSV. To include these, first open the file `package.json`. We'll add entries into the end of the dependencies section of package.json which look like this:
"csv": "^0.4.2", "soap": "^0.8.0"
The finished file looks like this.
- Now, let's expand the `lib` directory, and open the `hello.js` file. This is where we handle all calls to the `hello` route.
In here, you'll see we bind two different listeners - one for requests which come as GET messages (..our web browser sends these when we type a URL in an address bar), and one for POST messages.
We're not going to worry about this for today, and we're going to just listen to all HTTP verbs. This isn't very good practice - but nor is calling our barcode route "hello", we're just keeping things simple! - Let's get rid of the POST route, we're not going to use it - so delete lines 23 thru 19.
- Let's modify the GET route, and instead make it handle all HTTP verbs. Change the text `get` to `all`. Let's also delete all the comments and new lines, so we're left with something like this:
hello.all('/', function(req, res) { var world = req.query && req.query.hello ? req.query.hello : 'World'; res.json({msg: 'Hello ' + world}); });
- Let's examine the line starting with `var world`. Here, we take a paramater from the web address (or URL) which our service is called from, called a query param. This query param is called hello, and it's accessed by calling `req.query.hello`. If our request uses the HTTP verb `POST` (our client application does), we can also look for params in the request body. It'd be nice if we could look for the param in both locations, to keep our options open. Also remember our param is now called `barcode`, and not `hello`. Let's change this line to look like this:
var barcode = req.query.barcode || req.body.barcode;
Now, we have our barcode number stored in a variable with the same name. Let's also store the URL of our SOAP service description in another variable.
var wsdlUrl = 'http://www.searchupc.com/service/UPCSearch.asmx?wsdl';
Lastly, let's delete our call to `res.json` for now. This is used to send a response back to the client which sent the request (in our case the app). We'll come back to this later.
This leaves us with a route handler which looks like this:hello.all('/', function(req, res) { var barcode = req.query.barcode || req.body.barcode; var wsdlUrl = 'http://www.searchupc.com/service/UPCSearch.asmx?wsdl'; });
- It's now time to import Node SOAP, and instantiate a SOAP client, and tell it to call a function from our WSDL called `GetProduct`. First, at the top of the hello.js file we prepend the import statement for Node SOAP:
var soap = require('soap');
Now, let's call Node SOAP after defining our wsdlUrl variable, instantiating our SOAP client. That looks something like this:
soap.createClient(wsdlUrl, function(err, soapClient){ // we now have a soapClient - we also need to make sure there's no `err` here. });
Once we have our soapClient, we can use it to call the GetProduct service. We know from looking at the WSDL earlier that GetProduct requires us to specify `upc` (..the barcode) and `accesstoken` params (used to make sure we don't call the service too often - register your own here). Here's how we call the SOAP service with our newly created SOAP client:
soapClient.GetProduct({ upc : barcode, accesstoken : '924646BB-A268-4007-9D87-2CE3084B47BC' }, function(err, result){ });
- Note that we're using the `barcode` variable which we retrieved from the request above.
Lastly, we need to return our result variable back to the client, in this case our mobile app. We can do that using the `res.json` call, which we deleted earlier.
Putting all of our code snippets together, correctly nesting all the callbacks gives us a route handler which looks like this:hello.all('/', function(req, res) { var barcode = req.query.barcode || req.body.barcode; var wsdlUrl = 'http://www.searchupc.com/service/UPCSearch.asmx?wsdl'; soap.createClient(wsdlUrl, function(err, soapClient){ // we now have a soapClient - we also need to make sure there's no `err` here. if (err){ return res.status(500).json(err); } soapClient.GetProduct({ upc : barcode, accesstoken : '924646BB-A268-4007-9D87-2CE3084B47BC' }, function(err, result){ if (err){ return res.status(500).json(err); } return res.json(result); }); }); });
We've also added some really simple error handling. The full `hello.js` file as it currently stands is available here.
- Now, let's change gears a little. We've updated our Hello service to accept a barcode, and call our SOAP service. Let's now update the API documentation for our service to reflect this change. Open the file `README.md`.
This file is written in an API documenting format called API Blueprint. It has it's own unique syntax, so we can't just put what we like in here!
Even though we didn't change the path our barcode service is mounted on (`/hello`), we can still update the documentation to tell users about its new purpose! Let's change the title (the first line) to read:# Barcode Lookup
We can also optionally update the description on line 3.
The last change we need to make is to set the property name which our barcode service accepts. On line 18, change the "hello" to "barcode". Let's also give it a default value of "". The body section should now look like this (note the indentation is important here):{ "barcode": "9780201896831" }
The completed file should look like this.
We can now try our service. Visit the `Docs` page, by clicking on the link on the left. This gives us an interactive console where we can try our new API, and see what the response looks like! You'll see we get a response which looks like this:{"GetProductResult":"\"productname\",\"imageurl\",\"producturl\",\"price\",\"currency\",\"saleprice\",\"storename\"\n\"The Art of Computer Programming, Vol. 1: Fundamental Algorithms, 3rd Edition\",\"http://ecx.images-amazon.com/images/I/41Jon2rS8nL._SL160_.jpg\",\"\",\"45.55\",\"USD\",\"\",\"N/A\""}
But wait, what's with all these funny characters in the result? Well, turns out our API isn't just SOAP. Remember we said there was a "twist"? Well, there's a reason we introduced the rarely used CSV above.
Not only is this API returning SOAP, it embeds some CSV inside the SOAP body. Not only is this incredibly bad practice, it's also a pain for us to deal with! Fear not, Node to the rescue - a few lines of code should sort this out. - Back to the editor, expanding the `lib` directory open the `hello.js` file we were working with earlier. First, let's include the CSV module. At the top of our file, add this line:
var csv = require('csv');
- Now, let's use the CSV module to take this "GetProductResult" property of our result (a string of CSV data), and parse it into a JSON object. First, we set up the CSV parser with the relevant options.
var responseAsCsv = result.GetProductResult; csv.parse(responseAsCsv, {columns : true}, function(err, parsedResponse){ if (err) return res.status(500).json(err); // finally, we're ready to return this back to the client. return res.json(parsedResponse); })
That's it - no more coding! We can try out our API by visiting the "docs" section of the UI, and using the "Try It!" button.
If you completed part one of this tutorial and loaded the application onto your phone, you can also try out the barcode scanner on your own device. Because all of our development during this tutorial was in Cloud Code, there's no need to update the application on the phone.
Cheat Sheet
If you want to skip the step-by-step tutorial above, and just see the finished files, here's all the files you'll need to change in the Cloud App:
Video Walkthrough
Summary
In this tutorial, we've built upon the work completed in part one by developing a cloud microservice for our mobile app to talk to. We successfully integrated with a SOAP-based API, which also returns a CSV response in its body. We've completed a moderately complicated integration in a handful of lines of code, and hopefully also illustrated the powerful combination of Node.js as a server-side technology and the extensive community module ecosystem.
Last updated: February 11, 2024