Migrating from Netezza Mako to Hammerhead (CP4D 1.0.7.8)

I've been a database professional for the past 35 years; my early days in Silicon valley were spent with the likes of Oracle, Ingres, Informix, and Sybase. All database vendors in the 80s and early 90s used a circular transaction log eventually abandoning the design in the mid-90s. In 2004, I was introduced to Netezza, a brand-new database company with a revolutionary approach to data retrieval. The Netezza concept used minimal I/O for data retrieval which led to the development of specialized technology that 'brought the query to the data'. Instead of caching data for retrieval, Netezza developed specialized FPGAs (database accelerator hardware) along with massively parallel processing architecture to minimize I/O and accelerate data retrieval. Prior to this innovation, people were faced with adding more hardware and increasing cache sizes to improve database performance. Netezza really was the holy grail of database performance we were all looking for. 

Keeping track of the various IBM legacy Netezza replacement solutions can be a challenge. What is important to know: for legacy Netezza Mako customers facing EOS in April, IBM’s latest iteration CP4D - “IBM Cloud Pak for Data System 1.0.7.8” a.k.a. “Hammerhead” is their best attempt at recreating the Netezza magic yet.

What I like about CP4D 1.0.7.8 Hammerhead:

1. Compared to their previous attempt “Sailfish” (eg.: The DB2 variant of Netezza) IBM has pivoted back to a ‘simplicity’ model while improving performance 10x in most cases over prior versions of Netezza. While the hardware is very similar to SailFish, the containers (i.e. dockers) are simpler, and the lines between O/S and Docker are merged. We will leave this for a later discussion.
2. Similar to previous legacy Netezza migrations (e.g.: Twinfin to Striper), data migrations are a piece of cake. Netezza isn’t binary compatible, but you could restore data from one Netezza appliance to another appliance using the nzrestore command. NZRestore uses a proprietary backup format so in theory and irrespective of the versions you should be able to restore to a newer version.
3. Also, as CP4D 1.0.7.8 runs Netezza Performance Server (a.k.a.: NPS), migrating workloads is largely a straightforward process. Many function names remain the same, and the vast majority of your queries will run out of the box.

What IBM isn’t telling you about CP4D 1.0.7.8 Hammerhead:

Migrating between legacy Netezza appliances was a dream. Move your data, move your queries; done! As much as IBM tried to recreate this magic with Hammerhead, they didn’t quite make it.

1. The hardware platform that Hammerhead runs on is completely different than that of the legacy Netezza appliances. The Hammerhead platform is effectively running on Netezza sailfish hardware with a different chipset. On my most recent projects, we have seen up to 50% project time expansion due to waiting on IBM to repair broken hardware. Furthermore, trying to get to the root cause of hardware failures is much more difficult with a Hammerhead when compared to legacy Netezza appliances.
2. Migration - while a heck of a lot better than migrating to Sailfish, migrating to Hammerhead is not as straightforward as IBM will lead you to believe. Migrating data - yes, the standard commands work! You can copy data easily from the source legacy system to the new Hammerhead. There are some caveats including different column data sizes, data limitations, and SQL performance can be inconsistent. Migrating workload - while north of 80% of your queries should migrate easily, it's the 20%, typically the largest and most important queries that can take many months to migrate to Hammerhead. Stored procedures, UDF’s. will need to be refactored to work with Hammerhead. Application migration isn’t always straightforward, as invariably you will run into compatibility issues with data size limits, strange application behaviors, and performance. Query Tuning and workload management (WLM) have been considerably improved. However, you will need to adjust your mindset because of the differences from legacy systems.
3. Patching - is a much more difficult and time-consuming process with Hammerhead when compared to legacy Netezza. The IBM CP4D and Sailfish run in containers i.e. VMs on appliances with multiple O/S. The DB OS and hardware isn’t as tightly coupled as it was with legacy Netezza. As such, patching, i.e.upgrading NPS (Netezza Performance Server) could involve separate hardware and software changes which can many hours to complete.

In theory, migrating between the various Netezza versions should be simple and straightforward, but in practice, it can be quite the opposite. 

At Natrinsic we have successfully migrated clients from legacy Netezza to CP4D with challenges. Be mindful, 
if you've not done these before it's best to seek experienced consultants that can assist you. Contact us today to begin your Hammerhead migration journey by starting with a Hammerhead readiness assessment.