DevNation 2014: Scott McClellan – Hadoop and Beyond

Abstract: Historically, the term “Hadoop” has been considered synonymous with its core technologies: MapReduce and the Hadoop Distributed File System (HDFS). But today the definition of Hadoop is rapidly evolving.

The Hadoop community is generalizing the application runtime model beyond MapReduce. On the storage front, we’re seeing the emergence of many alternative Hadoop-compatible file systems. Red Hat has built an interface layer for its Red Hat Storage Server product. This complete implementation of the Hadoop file system interface lets Hadoop-related projects run transparently, directly on a Red Hat Storage Server cluster.

This talk will concentrate on contrasting the architectures of the HDFS against alternate Hadoop file systems—focused primarily on GlusterFS (the underlying technology for Red Hat Storage Server), but also examining implementations done on top of NoSQL and object storage. We’ll talk about:

* The current implementation.
* What we’ve learned about compatibility, performance, and scalability.
* The roadmap for next-generation implementations.

DevNation 2014 – Scott McClellan – Hadoop and Beyond.


Join Red Hat Developers, a developer program for you to learn, share, and code faster – and get access to Red Hat software for your development.  The developer program and software are both free!

 

Leave a Reply