For years, IBM’s Netezza proved to be an excellent option for companies to run advanced analytics on a single data warehouse appliance without needing to set up and configure a traditional data warehouse. When IBM decided to replace Netezza with Db2 Warehouse, some companies didn’t want to make the switch because of its costs, refresh issues, and other reasons.
Eventually, IBM opened the opportunity for companies to swap their Netezza workloads for the cloud-based database offerings of other vendors. These solutions promised the scale and cost benefits of a subscription cloud service instead of the larger upfront investment of the Netezza appliance.
As a result, some companies moved some of their Netezza workloads to these vendors. However, over time, companies often realized that the reality of the costs and management needs for these offerings proved to be greater than anticipated.
For Netezza fans, IBM has now brought an answer to the market. With Netezza Performance Server, companies can finally run Netezza wherever they want it—on-cloud, on-premises, or hybrid deployments—providing all the benefits and simplicity Netezza users grew to enjoy. In the coming years, companies have an important choice to make.
The end-of-life date for Netezza appliances is coming up in 2023. Companies have a few years to make a thoughtful decision about their next move. To help them make an informed decision, this blog series will explore the history of Netezza, its current state in the marketplace, the options for the future, and the benefits and drawbacks of pursuing the different options available on the market.
For a more complete picture of the current state, this first blog will dive deeper into how we got where we are today with Netezza.
The then—and the now—of Netezza
For years, many of our clients invested in IBM’s Netezza appliance to take advantage of its simplicity, performance, and scale-out capabilities. A few of our clients also leveraged onboard and push-down data science, which Netezza also provided.
Along the way, all of these clients benefited from Netezza’s features for call-home and self-diagnostics, and IBM’s soup-to-nuts support for hardware replacements, OS managed updates, and database updates.
Some of our clients started with early versions of the appliance and gathered a series of new benefits, such as cache, replication, flash, and encrypted at rest drives over two, three and even four hardware refreshes.
After many years and many easy and painless refreshes, along came a new branding and approach to IBM’s data warehouse appliances called Sailfish: Db2 Warehouse and Db2 Warehouse in the cloud. This was a very promising opportunity for our Netezza clients to again get the basis of Netezza and the benefits of in-columnar store, graph, and a growing list of benefits that we will discuss in future blogs.
Unfortunately, the costs, efforts, and pains of refreshing a Netezza box with a Db2 Warehouse was too much for some clients. So, IBM created an opportunity for all of the “me-too” database warehouse products or the “born-in-the-cloud” database warehouse products to take hold.
We followed the lead of our clients, and we developed the skills and selling capacity for each of the newborn, in-the-cloud data warehouses. We also tried our best to convert the Netezza fan base into Db2 fans.
However, this didn’t work. The three technologies were simply not built for the same purpose. The Netezza appliance is made for analytics. Db2 Warehouse is made for data science. The cloud solutions are made for scale and temporal uses. Each are great in their own way, and each have their uses. But at this time, none of them can completely replace the other.
The promise and reality of the cloud-based “me-too” Netezza replacements
This time in the marketplace created an opportunity for all of the growth-on-demand, scale-out, JSON-based, and cloud-only solutions for Netezza fans. Our Netezza clients loved the POCs, the enablement, and the simplicity that these “me-too” vendors provided.
They looked for the cost take-outs that these vendors offered—considering that Netezza’s high-performance on-premises compute can be expensive. They looked at the common tasks, such as replicate, that could be simplified with cloud providers. And they increasingly sought the ability to scale up and down as their businesses needed.
Though, like a lot of things in life, if you are great at one thing, you are typically limited somewhere else. And if you are good at everything, you typically are not great at one thing (think SQL Server). Although, some of our clients developed a strategy for data platform take-out with a cloud-only solution. They looked for all of those things, and we helped them see these values.
They reviewed the “new” “transformed” “modernized” approach, and it made sense: simplify, use what you need when you need it, scale it out, scale it up, drop it, and stop it. They were interested and looked to the “me too” vendors for a solution.
In making the switch, they looked at what they knew about their workload management on Netezza, and what cloud-based vendors asked about their workload on Netezza. From there, the vendors provided a size and cost estimate for replacing these Netezza workloads with the vendor’s offering.
If all of the modernized and digitized approaches of the vendor were followed and the transformation was followed through, these customers had a good cloud-only solution for an aggregate database with some inventive features for replicate and point-in-time/time-series analysis.
If the client data teams that had years with relational databases, and Netezza did not, and the transformation could not be completed or was years in the making, the client had to either find secondary options for cold storage, data-ride-along (like S3), or try the other “me-too” data warehouse in the cloud.
And then there were the costs…
The promise of getting the performance of an expensive appliance with an use what you need cloud based solution for penny’s on the dollar is compelling. But like a lot of things, it’s important to read the fine print.
The baseline workloads were not analyzed in depth to truly understand “what would this workload take in the cloud?” Those clients were interviewed by “me-too” vendors for what they knew and could provide. To the vendor’s questions, our clients response would often sound something like this: “We have 100 managed reports.”
However, their best-guesses missed a large list of unmanaged queries, ad-hoc queries, “because-I-need-it” reports queries, data science, as well as messy and poorly written queries. The reality was that Netezza did a great job of abstracting some of the complexities in today’s enterprise has in data hid a lot of sins.
As one can imagine, a metered cloud-based solution sized for a minimum workload began to show the signs of this reality. It quickly started taking up the cloud-credits that clients had paid for, and it consumed a growing amount of computer store and pipeline from on-premises sources to load through ETL and reports to be fed by the data in the cloud.
An abbreviated history of “me too” Netezza replacements
As we’ve seen, the promise of cloud-based Netezza workload replacements didn’t always meet our clients expectations going in. Let’s boil things down into an abbreviated timeline of how things usually went down. If the engagement were a week, this is what it would look like:
Day 1: Everything was great. The 100 managed and curated reports were fast enough and having been “transformed” were optimized for new data pipelines.
Day 2: The old “must-haves” and “I-just-need-reports” exposed the reality that there were reports and queues consuming a lot of resources and credits.
Day 3: The greater team began to build new reports and queries, and not all of them were great, which took up even more resources and credits.
Day 4: The client’s team realized that humans fall back into habits, and perhaps Netezza had spoiled them with its “set and forget” capabilities. As it turns out, the scale-up always happened, but the scale-down, stop, and pause rarely did.
Day 5: The client began looking around for tools, policies, operations, and best practices. A lot of these were not readily available—since innovation, speed, dynamics, and growth in sales had been the approach.
Day 6: IBM released the Cloud Pak for a data containerized version of Netezza called NPS or Netezza Performance Server.
Day 7: We stopped and looked around and began to realize that we were solving yesterday’s problems (aggregate data) with today’s answer (cloud), and what we should be doing is building today in 2020 for a platform we will be supporting through 2030.
This series will dive deeper into each of these topics in detail. Please join me on this journey to learn, discuss, and review where we were, where we are, where we are headed, and where we should aim our efforts going forward.
