Cloud Data Lakes Are Having a Renaissance—Are You Ready?

February 06, 2020
by: Matthew Halliday, Co-Founder and VP of Products
Blog

We live in a world where incremental change is no longer enough to succeed in business. Take a look at today’s most valuable companies—they all approach innovation the same way: fearlessly swinging for the fences and regularly disrupting not only their industry but also themselves. This is the new model for innovation—and data executives would be wise to take note.  

The great irony here is that while data is at the heart of much of this disruptive innovation, the underlying data analytics systems haven’t seen any real disruption in decades. Sure, we’ve seen improvements here and there over the years, but real innovation that changes the game entirely? Not even close—at least until now.

Cloud data lakes are having a renaissance as Microsoft ADLS Gen2 and others push the boundaries on what’s possible in cloud data lake storage. At the same time, we are witnessing the dawn of a new era in data analytics with the rise of Unified Data Analytics Platforms (#riseofUDAP)—a new product category that reduces fragmentation and helps companies get orders of magnitude more value out of their data. 

It’s a brave new world, and the ecosystem that is emerging around it—companies with a shared vision for the future of data analytics and how we get there as an industry—is creating a flywheel of innovation unlike anything this industry has seen. Today, we take another giant leap forward with the launch of Incorta 4.6, the latest version of our Unified Data Analytics Platform. We’re dubbing Incorta 4.6 the “cloud data lake release” and it’s one of our most anticipated platform updates since we began this journey six years ago. 

All Signs Point to Cloud

Incorta 4.6 doubles down on our commitment to bringing data engineers, data scientists, and data analysts together on a single platform by introducing powerful new tools for making cloud data lakes analytics-ready. 

Why now? We’ve been tracking the shift to cloud since Incorta’s inception, building the platform around microservices and designing it to work in any cloud environment. But large enterprises—household brand names—were hesitant to move their data off premises until recently because of security and reliability concerns. The truth, of course, is that the cloud has proven more reliable and secure than their on-premises environments. The elastic nature of the cloud also means it can deliver much better performance at a much lower cost. 

Talking with customers lately, it’s clear the tides are turning in a big way. Large enterprises that were once dead set against the cloud (and flush with enough resources to do it all themselves) are now racing to push data into cloud data lakes. That’s welcome news to Incorta—we’ve been building for this moment for years, working with all major cloud vendors to make sure Incorta is on the cutting edge of their storage technologies.  


Incorta 4.6 Makes Cloud Data Lakes Analytics-Ready

Data lakes are called that for a reason—a lake is an expanse of water that grows and adapts to its surroundings. The same goes for cloud data lakes. Unlike highly structured data warehouses, a cloud data lake can grow and evolve alongside the data flowing into it without needing to force-fit that data into a predefined model that ultimately forces you to sacrifice data fidelity in order to make any of it work. 

It’s no wonder, then, why data lakes are so alluring—especially to companies with massive amounts of data. Historically, though, the process of extracting insights from data lakes has been rather painful. But not anymore. 

Incorta 4.6 makes cloud data lakes continuously available for analytics. Specifically, it enables seamless integration and support for ADLS Gen2 and AWS S3 cloud storage technologies, giving companies a powerful new way to (1) provide data constituents with ready access to data that is currently locked up in cloud data lakes, and (2) bring new data into cloud data lake storage as open standard file format (Parquet). What’s more, the latest version of our Unified Data Analytics Platform powered by Direct Data Mapping (DDM) also introduces user experience and performance enhancements to all stages of the data workflow, further breaking down silos that traditionally prevent data engineers, data scientists, and data analysts from seamlessly working together. 


A New Standard for CIOs

Data scientists and engineers spend so much of their time wrestling with fragmented data workflows and technology that data enrichment often falls to the wayside. Even the companies that are widely known as “innovators” and “disruptors” are stuck on decades-old systems that can’t keep up with the accelerating rate, volume, or complexity of modern enterprise data. Instead of looking for ways to add value to the data, it takes everything their data teams have just to populate some version of the it for business users and try to keep up with a growing backlog of new requests. It’s like trying to supply New York City’s water with a garden hose—you just fall further and further behind with each passing day, while most people in your organization go thirsty (and smelly). 

Against this backdrop, today’s CIOs need to be obsessed with secure, accessible, trustworthy data. They are looking to provide value to their users and at the same time they must stay on top of data governance and security—which is no small feat—and do it all without burning out their engineers. Stumble on any of these fronts and tough questions from above will shortly follow.  

To thrive in this new reality, CIOs need a platform that can support all three fronts simultaneously. Incorta is that platform. Built with open standards and designed to integrate with cloud-friendly tools and platforms, Incorta’s Unified Data Analytics Platform (#UDAP) makes it easy to consolidate data in the cloud and extract meaningful insights with unmatched speed and efficiency. By making any data source continuously available for analytics, we help data engineers, data scientists, data analysts, and business leaders make more accurate, timely, and transparent decisions with faster access to richer data sets.

Here’s a challenge for you: What’s one question that you haven’t been able to answer with data? Bring it to Incorta and we’ll give you the answer in a matter of days. It doesn’t matter how large or small, or how many years you’ve been trying to crack it. Too good to be true? Try me—I’m @layereddelay on Twitter. 

Read the full release here