There’s no denying the importance of speed and agility in data analytics – and yet the debate about how to achieve it continues. Traditional approaches can no longer keep up with the accelerating rate, volume, and complexity of modern business data. A plethora of new point solutions have emerged that claim to solve the issues of traditional architecture. But are they powerful enough to justify the integration risks and operational overhead? Or are unified data analytics platforms the solution? It depends.
In a recent webinar, Dr. Claudia Imhoff, an internationally recognized expert on analytics, BI and data architecture, explored the merits of each approach – what works, what doesn’t, the criteria companies should consider when making the call, and where concessions frequently come into play when building a modern data architecture.
But before that, we first need to understand what we mean by “modern data architecture” and how that is enabled by a data fabric. Called out by Gartner as a top data and analytics trend, data fabric is a new term for a long-held vision of the ideal enterprise analytics environment. In a nutshell, the data fabric provides unfettered access to data in any shape or form. Anyone, from business users to data scientists, can easily find and navigate data from a centralized location. This is the key to generating enterprise-wide value from data.
Any organization that is challenged with meeting the demands of digital transformation and increasing regulatory requirements with its current EDW approach will want to consider creating a data fabric. So will companies that plan to shift their architecture to a distributed, agile application architecture, and any company seeking to better serve the analytics needs of business users.
Two Ways to Weave Data Fabric
To create such a data fabric, all the components need to be tightly woven together. There are two basic ways to achieve this:
- Assemble a collection of point solutions for each component needed.
- Use an integrated data platform that has most — if not all — of the components for the fabric.
This has led to an emerging debate about the merits of point solutions vs. unified analytics platforms.
Nine out of ten respondents in the February 2021 Forrester Consulting survey of analytics and business intelligence professionals reported that their organizations’ current analytics solutions cannot meet all of their business objectives. Prime reasons cited include the inability to architect for real time analytics, data security challenges, and solution integration challenges. In response to these challenges, a majority said that they are either moving to a full stack analytics platform or considering it.
As organizations evaluate their options, it’s important to understand the differences between approaches. During the webinar, Dr. Imhoff detailed the different architectures for supporting a data fabric and weighed in on the pros and cons of each. Here’s how she sums them up:
Many organizations have also been taking a third approach — assembling a hybrid architecture using an integrated data platform as the base, adding point solutions as needed.
According to Imhoff this is quite common because it’s rare that a single technology could have everything a company possibly could need. It’s important to understand where there may be some gaps or holes vis a vis your needs, and how the vendor can help you address them. Partnerships are critical. Connections to and from all kinds of desired point solutions are critical, as is extensibility.
Why Incorta
In the second segment of the webinar, Ethan Post, head of Incorta’s center of pre-sales enablement and a former BI and analytics consultant, detailed the reasons why organizations looking to create a modern data architecture may want to consider Incorta. The core components of the unified data analytics platform are built from the ground up to work together and for extensibility. The platform allows you to acquire, enrich, analyze and act on data with unmatched speed — no data warehouse required:
- Data acquisition: Using over 200 data connectors, Incorta’s data acquisition layer can connect to any database; enterprise applications such as Salesforce and NetSuite; data streams such as those from IoT devices; and data in any file format. This allows you to ingest and aggregate data in real-time.
- Data enrichment: Modern analytics is a combination of analytical functions and data science. Unlike a data warehouse, Incorta maintains all of your data in its original form, providing you with a single source-of-truth for strategic and operational decision-making, data science and machine learning, and other computational workloads. That means you can drill into the details, ask any question, and even train machine learning models.
- Advanced analytics: Analyze complex, full-fidelity business data using your preferred analytics tool — from Incorta’s native visualization capabilities to integrated PySpark, Scala, and R notebooks, and third-party BI tools such as Power BI and Tableau.
- Self-service semantic layer: Allowing business decision makers to work with data is one of the primary goals of modern data architecture. Incorta’s secure semantic layer lets you deliver a real end-to-end self-service user experience that requires no training so that truly everyone in the organization can curate, refine, visualize and share data insights.
Living the Dream
In a way, the data fabric concept is a new take on the dream of a single, fully integrated enterprise data hub, data warehouse, data lake, or lakehouse. To support it, the world of analytics is moving from some (or a lot) of assembly required to a more integrated environment that addresses the whole analytic process.
Incorta is taking a new approach. We are not just unifying the architecture; we are simplifying the entire process with our Direct Data Mapping™ engine. It delivers crazy-fast query performance without the need to pre-aggregate, reshape, or transform the data in any way. Summaries and rollups are computed on-the-fly against full-fidelity business data.
It allows users to ask any question and explore any direction immediately, even against billions of records, without having to wait for new models or ask for new data sets. It’s the dream of the data fabric, realized.
How do you determine what architecture is right for your company? To hear Dr. Imhoff’s eight questions for consideration, get the full webinar here.