A modular, fully-integrated workflow designed to train, quantize, adapt and deploy edge AI neural networks: Latent AI

Artificial intelligence is one of the most incredible techs that will transform lives in almost all facets of modern society. This technology provides significant development opportunities that many businesses and companies have already been fast to grab. AI technology can learn and handle vast amounts of information that will enhance and transform operations in different fields effectively. Over a certain period of learning and comprehending, AI technology can anticipate needs and make informed and relevant decisions. Unlike business intelligence technology, which analyzes data and leaves the decision-making parts for human beings, AI will make decisions after analyzing data. Artificial intelligence machines can also analyze large amounts of data within a few seconds. They usually take less time to make decisions as long as they have relevant information.

Latent AI, Inc. is an early-stage venture spinout of SRI International, dedicated to building solutions that enable the adaptive edge to transform AI processing. Latent AI is well-funded by industry-leading investors with support from Fortune 500 clients. The Latent AI Efficient Inference Platform (LEIP™) brings AI to the edge by optimizing for compute, energy, and memory without requiring changes to existing AI/ML infrastructure and frameworks. The company is founded by Jags Kandasamy and Sek Chai. They bring decades of combined industry experience and R&D expertise in machine learning AI (lifelong learning), computer vision, embedded systems, IoT applications, and efficient computing. Latent AI recently closed its $3.5 million seed round of financing, led by Steve Jurvetson at Future Ventures and including industry leaders such as SRI International Ventures, Perot Jain and Gravity Ranch. LEIP is a unified SDK built for machine learning engineers and AI software developers, working anywhere on the edge continuum, to optimize and accelerate the deployment of AI models for production inference services.  LEIP is modular in design enabling developers with the optimization services they need now while providing a flexible foundation to extend the platform with new services for the future. Today, the LEIP SDK consists of two core modules:  LEIP Compress and LEIP Compile.  LEIP Compress is powered by state-of-the-art quantization algorithms to automate optimizing trained AI models for efficiency by compressing the millions of parameters that make up today’s advanced AI models to significantly reduce the memory footprint requirements.

Bringing AI Processing to the Edge

LEIP Compile takes the optimized AI model produced by LEIP Compress and automates the generation of compiled code with additional inference performance optimizations that can be targeted for a broad set of hardware devices including servers, PCs, smartphones, surveillance cameras, home automation devices, sensors, and more. Latent AI understands that for many companies, data security is of primary importance.  This is why Latent AI has designed LEIP to run on-premise with your security policies as a priority.  LEIP supports integration into existing AI/ML pipelines to provide an automated and unified process for optimizing Al models across a heterogeneous computing environment.  LEIP also generates a compression report for each run to provide quantization effect insights for each model.  This enables ML engineers to make more informed trade-off decisions between model size and accuracy performance. Latent AI focus will continue to be concentrated on providing improved tooling to help ML engineers and AI software developers optimize AI models for production inference services.  This includes additional diagnostic capabilities and management services to easily track LEIP optimization runs.  Additionally, Latent AI is developing new modules to the LEIP platform to extend the optimization capabilities from the build phase to the run phase.  Through its efforts on LEIP Adapt Latent AI plans on introducing inference optimization that is intelligent and context-aware to further minimize the compute requirements for performing inference in production.

We have an insatiable demand for technology in all aspects of our lives. We need technology to make our work more efficient, our lives safer and healthier, and our lifestyles more comfortable. Central to the technology are the semiconductor chips that are responsible for the computations that make our electronics smarter.  Experts have been warning us for some time of the chip shortage due to the demand growth that is outpacing supply capacity. Such a major shortfall can have more far-reaching consequences than you might think. Companies are now anticipating reducing their manufacturing capacity at various times because the chips that run their products are not available. Repair and maintenance of existing systems are being postponed. The shortage crisis could lead to factory shutdowns, jobs lost, longer wait times, and higher prices. It can leave critical infrastructure unamended, even impacting our safety and wellbeing.

Q. How LatentAI Can Help Solve the Chip Shortage Crisis

First, there are only a limited number of companies around the world that have the ability to manufacture high-end chips. They include TSMC, Intel, and Global Foundries, just to name a few. Production line upgrades cost billions of dollars and it is difficult to keep up with the advancements of the new manufacturing techniques for nanoscale-sized transistors. Second, electronic products are getting smarter, and that means more processing. For example, consider the new electric cars, which essentially are computers on wheels. Smartphones are really just small connected computers with an ever-growing list of sensors. Smart IoT devices like refrigerators and toasters are digitizing all manners of raw audio and video signals to enable touchless interfaces and to produce useful consumer analytics. Third, the pandemic is also putting huge stress on the global semiconductor supply chains resulting in labor shortages for fabrication, testing, and assembly. Our reliance on tech devices to support remote working environments adds to the already stressed global demand. Gartner has warned that the shortage will persist until at least the second quarter of 2022, and given the uncertainty in the pandemic and global economics, the chip shortage crisis won’t be over anytime soon. Businesses are re-evaluating their supply chain, including diversifying the supplier base. For example, Intel just announced they will be producing chips for Qualcomm. This is a good example of risk mitigation with local/regional sourcing. Others are resorting to stockpiling chips to address both sourcing issues and possibly other geopolitical sanctions. More innovative companies are exploring alternative solutions, including using substitute chips. There is more hard work involved, including rewriting software and firmware to support the alternative chips. Tesla has recently reported using this approach just to keep up with the production level. Latent AI can help companies directly mitigate risk and revenue loss during the global chip shortage. With the Latent AI Efficient Inference Platform (LEIP) Software Development Kit (SDK), businesses have an alternative software-based solution to weather out the crisis. The LEIP SDK consists of end-to-end developer tools that help AI practitioners extract the best performance of an AI model. LEIP is designed to take the hard work out of AI system development and deployment, by making it easy for AI scientists, software developers, and IT/OT professionals to solve real-world problems. Latent AI’s mission is to enable next-generation AI factories to build efficient AI models. Developers can use Latent AI’s software tools to design, explore and target a variety of model architectures and hardware. With support for different chips, LEIP opens up design alternatives using efficient software to unlock hardware capabilities. It is an integral tool to help the next-generation AI factory scale production goals amidst a chip shortage crisis.

Jags Kandasamy, Co-Founder & Chief Executive Officer

“Our culture is built on trust, respect, integrity, inclusion, excellence, transparency, and accountability.  We are highly motivated and deeply passionate about our work and how we make our customers and ourselves successful.”