WEKA’s Zvibel on Streamlining the Data Pipeline that Fuels AI/ML

This exclusive Q&A interview was conducted by Nages Sieslack of the ISC 2022 conference organization, with Liran Zvibel, co-founder and CEO of WEKA, which offers a data platform built for I/O-intensive workloads and latency-sensitive applications at exabyte scale. WEKA will be an exhibitor at the ISC conference, May 30-June1.

Nages Sieslack: What challenges are you helping customers to solve?

Liran Zvibel: Today, organizations are increasingly looking to harness the power of artificial intelligence (AI) and machine learning (ML) to support their research and discovery initiatives, drive business innovation, and increase operational efficiencies. But running AI/ML projects at enterprise scale requires massive amounts of data and performance to fuel them, and many organizations struggle to move their mission critical business initiatives and research projects forward as a result. A key reason is their legacy data infrastructure is not equipped to support the insatiable performance and scalability demands of these next generation workloads – and it’s holding them back.

WEKA is on a mission is to address this challenge with a software-based data platform that is purpose-built to streamline and accelerate the data pipelines that fuel AI/ML and other modern performance-intensive workloads.

The WEKA Data Platform delivers radical simplicity, epic performance, infinite scale, and seamless data portability to support enterprise AI workloads in virtually any location. Whether on-premises, in the cloud, at the edge or bursting between platforms, WEKA accelerates every step of the enterprise AI data pipeline, from data ingestion, cleansing and modeling, to training validation or inference.

Sieslack: Can you provide an example of how you helped a customer with a significant challenge?

Zvibel: In the U.S., the Oklahoma Medical Research Foundation (OMRF) computing team was looking to architect a system that could deliver more compute power, faster storage, and bigger volumes of data with increased velocity to support its growing informatics needs for scientific research.

A common workflow is next-generation sequencing (NGS) analysis using the GATK pipeline for sequence alignment and variant calling. However, the cluster supports numerous research jobs running simultaneously with unique toolsets that need to be carefully orchestrated so as not to negatively impact other jobs or workloads.

By implementing the WEKA Data Platform, the OMRF team was able to achieve better throughput and run more research jobs concurrently. As a result, their research outcomes are no longer limited by how much data can be stored locally and their research workflows have been greatly simplified. Further, the complexity of staging in and out data in a compute node’s local SSD was eliminated; turnaround times were better because jobs finished faster, getting results to their scientists quicker and accelerating the next stage of their research.

Ultimately, OMRF’s research jobs were reduced by 10x – one job was reduced from 70 days to seven, and another common analysis workflow was reduced from 12 hours to two. OMRF’s researchers no longer need to think about their data infrastructure environment —instead, they’re free to focus on saving lives.

Sieslack: What, if any, impact has the pandemic had on your customers’ businesses, and how are you helping them to overcome these issues?

WEKA’s Liran Zvibel

Zvibel: The COVID-19 pandemic was a wake-up call on many fronts around the world. This virus brought forth key researchers who are looking to solve the problem with vaccines and other pharmaceutical solutions. Genomics researchers needed to look at the RNA sequences of the novel coronavirus and rapidly compare them with those of other viruses to better understand the behavior of the viruses and find potential treatments.

To contribute to the effort to address the COVID-19 crisis, WEKA offered Life Sciences researchers free use of the WEKA Data Platform on AWS for 120 days. In partnership with AWS, WEKA also supplied credit for complimentary AWS data storage and compute services.We also tracked how COVID affected our customers’ work and found some interesting results:

Over one third of our customers told us that their cloud adoption increased due to COVID. The verticals that claimed to have higher adoption of cloud due to the pandemic are Life Science (no surprise here), Cloud (due to the overall increased demand for cloud instances), and Banking, which was already showing strong cloud adoption and signs of growth. Additionally, COVID-19 increased the demand for compute and usage of data in verticals such as Semiconductors, Automotive and Media.

Sieslack: What is your partnership strategy for server vendors?

Zvibel: WEKA generally does not sell direct. We partner with a broad range of server vendors and cloud providers to let customers choose the underlying storage that best matches their preferences and needs. WEKA’s business model makes partnering attractive for established server and data infrastructure companies.

Our server partners together with their reseller partners offer complete appliances and service offerings based on WEKA’s data platform software and their hardware. This strategy gives our customers the broadest server choice while empowering our server partners to best tailor offerings that combine their hardware and go-to-market expertise.

Sieslack: To what do you attribute your rapid growth over the past couple of years? What lies ahead for the company?

Zvibel: Over the past few years, use of AI has moved from boutique science experiments to full-fledged enterprise deployments that are delivering new ways to support customer innovation and drive business forward. Companies of all sizes are now using AI workloads to support a variety of use cases, like improving manufacturing processes, predicting customer churn, optimizing supply chains, honing technical systems design, accelerating genomics research, and much more. And they are deploying these projects in the cloud, at the edge and on-premises – or a combination thereof – based on their unique needs.

As legacy data management and infrastructure technologies are not well suited to supporting the unique performance demands of AI across these diverse environments, customers are increasingly turning to modern solutions like WEKA’s Data Platform. As a result, WEKA is now in a period of hypergrowth that’s being fueled by customer adoption and demand. We doubled our customer base over the last year and expect it to continue to grow arithmetically – but we can only achieve this if we continue to delight our customers. Today, we have exceptionally high scores from customers on Gartner Peer Insights – a 4.9 out of 5-star rating – and all those surveyed said they recommend WEKA.

As we look ahead, we intend to continue expanding the capabilities of our data platform technology to ensure we are helping our customers to fully capitalize on AI to achieve their research and business goals today and prepare them to harness the next generation applications and workloads of the future.

Filed Under: HPC Software, Machine Learning, News Tagged With: Data Management, data platform, I/O, ISC 2022, latency, Weekly Newsletter Articles, Weka