Business Case for NoSQL and Big Data
I’m often approached by clients, colleagues, and friends asking: “What’s this Big Data/NoSQL thing I keep hearing about?” Oftentimes they’ve been told that these are the buzzwords that will help them get answers from the plethora of data out there. However, like any tool, it won’t help solve a problem unless used effectively. As an SEI consultant, I’ve often worked with my clients to help evaluate if Big Data and the other trendy buzzwords in tech could actually help solve their problems. At a recent Data Management forum, I presented information on the different uses for newer Data Management technologies, like Big Data and NoSQL, when compared to traditional relational solutions.
While my intention was neither to sell the audience on new technology nor to warn away from different approaches, I wanted to leave an important message behind: instead of choosing the latest cutting edge technology and then searching for a business use, consider the business need first. Satisfying the business need with the correct solution can lead to better rewards and more intelligent business decisions.
NoSQL technology overview
For newer Data Management technologies, there are two aspects critical to maintaining high levels of availability: processing capabilities that scale horizontally and the ability to adjust data consistency controls.
When managing large data, processing capabilities that can grow on demand without significant technical infrastructure management are crucial to get meaningful results in a reasonable amount of time.
Manipulating data consistency controls allows expected quality levels to be maintained as data sets continuously grow and definitions are refined through analysis. As mentioned above, the business need will help determine the processing requirements, quality expectations, and the feature sets required to utilize the data.
One such technology, NoSQL, can be classified into four groups based upon feature sets.
NoSQL technologies were created to address evolving data management use cases at the limits of traditional relational solutions. These might include applications such as:
- Online catalogs with dynamically changing schemas
- Real time stock quotes
- Ingestion and storage of machine generated or internet of things (IOT)
- Social interactions and human intelligence
- Product recommendations
However, NoSQL should not be seen as a replacement for traditional relational solutions, but as a complement that addresses use cases that are difficult or overly costly to implement with relational technologies.
Big Data Technology Overview
Hadoop is the hallmark big data technology. While Hadoop solutions are moving toward more near-real time applications, Hadoop is primarily known as the batch analytical big data solution. Developed by Yahoo! to provide a platform for search and index processing, many traditional organizations are evaluating the technology as a lower cost storage and processing platform for variable structure and voluminous data sets. As I’ve often told colleagues, it’s the “go-to” technology for when you feel like you have so much unstructured data you aren’t sure how you should tackle it.
Some business drivers for the adoption of Hadoop include:
- Cost of storage per capacity decreasing over time
- Processing power needs increasing but processor technology not keeping pace
- Rise of distributed computing
- Trending from scale up to scale out systems
- Increasing need for faster cycles turning data to actionable information
Risks and Considerations
Considerations for using NoSQL or Hadoop encompass more than just technical architecture. The decision warrants evaluation of business processes, people, and cultural considerations. These new classes of technologies are at varying states of adoption and maturity. This creates not only the well-documented technology skills gap, but also a thin body of knowledge around best practices in both solution development and operational management disciplines. Organizations with stringent security or industry compliance concerns will want to assess product capabilities to support compliance and governance. These new technologies are powerful and can be disruptive to existing cultural norms. The introduction of a new set of technologies and business capabilities necessitates a Change Management strategy for the organization.
Lead with the Business Question
With technology decisions, the best approach is to start with the business problem or opportunity and choose the right technology to address it. IT solutions should always be aligned with the organization’s business strategy. This guarantees that the technology provider, whether internal or external, is providing the services that the business wants and supports the organizational goals and objectives. Once the business need is understood, start investigating technologies that are accepted within the organization and existing skill sets. Evaluate for cultural sensitivity to change and adaptability.
Remember, satisfying a business need is a greater and more rewarding feat than implementing the latest cutting edge technology.
