Wallaroo: The Enterprise Platform for Production AI

Founded in 2017, Wallaroo is a group of veteran software and data engineers. Together, they have got more than a century of distributed computing, open-source development and low-latency systems engineering experience. The company has a fully distributed team with a base of operations in NYC and engineers ranging from California to the Netherlands. Its processes are always distributed first, with an emphasis on effective asynchronous communication.

Wallaroo's enterprise ML/AI platform turns data into business results faster, easier, and with a far lower investment. It is streamlining the ML lifecycle and giving data scientists the freedom to use the tools they love. The Wallaroo platform makes it simple, fast, and very low cost to get AI algorithms live against production data. The platform is built on four core components: MLOps, Distributed Processing Engine, Data Connectors, and Audit and Performance Metrics.

Wallaroo eliminates weeks/months of data engineering once you have a model. It lets you run multiple models on shared infrastructure and adds almost no latency overhead to your model’s compute time and reduces the compute costs typically by 80%. Wallaroo’s core engine is 50MB and runs as a compiled binary with C-speeds in any cloud, on-premise, and at the edge.

"Wallaroo eliminates weeks/months of data engineering once you have a model. It lets you run multiple models on shared infrastructure and adds almost no latency overhead to your model’s compute time and reduces the compute costs typically by 80%."

The company's Data Connectors enable integrations with popular data sources and sinks (e.g. Kafka, Postgres, and S3), as well as custom integrations with your own in-house solutions. This allows Wallaroo to get data in to and out of a wide variety of systems.

Wallaroo's underlying distributed data processing engine provides a highly scalable, high performance environment for production model scoring, as well as pre- and post-processing. Wallaroo has built an engine from the ground up to add minimal overhead. It compiles to native code and can easily be deployed in a wide range of environments, whether cloud, on-premise, or at the edge. Scaling is as easy as adding or removing workers from the cluster.

It also provides observability to data scientists, operations, compliance/risk teams, and business heads and finance leads. We support event-by-event audit logs, compute and model performance metrics, and A/B testing comparisons.

In a smaller startup like Wallaroo everyone needs to be a generalist to some degree. That’s part of the reason why Wallaroo values sharing knowledge, skills, and perspectives so much. Its engineers are eager to keep learning, but they’re also eager to teach their colleagues what they know, and to experiment with more effective ways to make this kind of information readily available.

"The company has a fully distributed team with a base of operations in NYC and engineers ranging from California to the Netherlands."

Wallaroo lets data science teams deploy easily and adapt quickly, gives operations simplified infrastructure, and provides visibility for compliance and business leaders to understand risk and return.

When an international investment and real estate company wanted to deploy 36 pricing models with real-time customer segmentation to dynamically price thousands of units, Wallaroo came to the rescue.  The obstacle was that the company had 9 months to build and deploy a prototype, 144 servers were needed to run the pricing models and several technologies to ingest, compute, and run models.

Wallaroo took just three months to deploy build and deploy ML prototypes. It used 18 servers to run 36 pricing models. This brought down the computing costs by a remarkable 87%.

Vid Jain, Founder and CEO

Vid holds a Ph.D. from UC Berkeley, is the author of 3 internet patents, and has spent the last 20 years pushing the technology envelope in various industries. He was a co-founder of an adtech startup, worked in the Electronic Trading group at Merrill Lynch and most recently is the CEO and founder of Wallaroo Labs.

A Cybersecurity Use Case

A Fortune 100 enterprise needed to deploy over 100 ML models to detect security breaches. Plus, the models had to be retrained and updated monthly. This meant that two weeks were needed to retrain and deploy a new model, 600 servers to run them and 5 data scientist required for deployment.

With Wallaroo's help the enterprise was able to redeploy updated models in seconds. It took 96 servers to run over 100 ML security models, which means that data scientists were now free to innovate. Overall, with Wallaroo's help the company was able to reduce its computing costs by an amazing 84%.

"Wallaroo has built an engine from the ground up to add minimal overhead. It compiles to native code and can easily be deployed in a wide range of environments, whether cloud, on-premise, or at the edge. Scaling is as easy as adding or removing workers from the cluster."