Company Logo

Home technology artificial-intelligence nVIDIA announces new tech for training Giant Artificial Intelligence Models

NVIDIA announces new tech for training Giant Artificial Intelligence Models

Artificial Intelligence

NVIDIA announces new tech for training Giant Artificial Intelligence Models

At the GTC 2021, NVIDIA made several announcements including, the development and deployment of a newly designed AI platform to accommodate the training of future AI models that almost rival the number of synapses in a human brain. Ian Buck, NVIDIA's GM and VP of Accelerated Computing, said there has been a dramatic growth in AI models' size, particularly for language (NLP) models.

For comparison, in 2018, Google had published a transformer-based machine learning technique for natural language processing (NLP) pre-training. This early model was called BERT and had 340 million connections. BERT was open-source and developed for a wide range of search engine-related tasks such as answering questions and language inference.

The model from NVIDIA is way ahead of BERT, surpassing 175 billion connections. For comparison, a human brain has 150 trillion connections. As the AI model sizes continue to grow, NVIDIA believes that, by 2023, the models will have 100 trillion or more connections. Models of that size will exceed the technical capabilities of existing platforms.

NVIDIA has not yet released detailed architecture information on the newly announced Grace Module. However, it could possibly be made available in 2023. The vital breakthrough for NVIDIA is that the CPU is capable of giant scale AI and HPC applications. The CPU is comparable to existing GPUs with a high-speed interconnect between the two processors. The interconnecting link has 900 gigabytes/second bandwidth, which is 14X faster than today's standards. It will allow upto trillion connection models to be trained and perform inferencein real-time.

Business News

Recommended News

Most Featured Companies

ciobulletin-aatrix software.jpg ciobulletin-abbey research.jpg ciobulletin-anchin.jpg ciobulletin-croow.jpg ciobulletin-keystone employment group.jpg ciobulletin-opticwise.jpg ciobulletin-outstaffer.jpg ciobulletin-spotzer digital.jpg ciobulletin-virgin incentives.jpg ciobulletin-wool & water.jpg ciobulletin-archergrey.jpg ciobulletin-canon business process services.jpg ciobulletin-cellwine.jpg ciobulletin-digital commerce bank.jpg ciobulletin-epic golf club.jpg ciobulletin-frannexus.jpg ciobulletin-growth institute.jpg ciobulletin-implantica.jpg ciobulletin-kraftpal technologies.jpg ciobulletin-national retail solutions.jpg ciobulletin-pura.jpg ciobulletin-segra.jpg ciobulletin-the keith corporation.jpg ciobulletin-vivolor therapeutics inc.jpg ciobulletin-cox.jpg ciobulletin-lanner.jpg ciobulletin-neuro42.jpg ciobulletin-Susan Semmelmann Interiors.jpg ciobulletin-alpine distilling.jpg ciobulletin-association of black tax professionals.jpg ciobulletin-c2ro.jpg ciobulletin-envirotech vehicles inc.jpg ciobulletin-leafhouse financial.jpg ciobulletin-stormforge.jpg ciobulletin-tedco.jpg ciobulletin-transigma.jpg ciobulletin-retrain ai.jpg
ciobulletin-abacus semiconductor corporation.jpg ciobulletin-agape treatment center.jpg ciobulletin-cloud4wi.jpg ciobulletin-exponential ai.jpg ciobulletin-lexrock ai.jpg ciobulletin-otava.jpg ciobulletin-resecurity.jpg ciobulletin-suisse bank.jpg ciobulletin-wise digital partners.jpg ciobulletin-appranix.jpg ciobulletin-autoreimbursement.jpg ciobulletin-castle connolly.jpg ciobulletin-cgs.jpg ciobulletin-dth expeditors.jpg ciobulletin-form.jpg ciobulletin-geniova.jpg ciobulletin-hot spring it.jpg ciobulletin-kirkman.jpg ciobulletin-matrix applications.jpg ciobulletin-power hero.jpg ciobulletin-rittenhouse.jpg ciobulletin-stt logistics group.jpg ciobulletin-upstream works.jpg ciobulletin-x2engine.jpg ciobulletin-kastle.jpg ciobulletin-logix.jpg ciobulletin-preclinical safety (PCS) consultants ltd.jpg ciobulletin-xcastlabs.jpg ciobulletin-american battery solutions inc.jpg ciobulletin-book4time.jpg ciobulletin-d&l education solutions.jpg ciobulletin-good good natural sweeteners llc.jpg ciobulletin-sigmetrix.jpg ciobulletin-syncari.jpg ciobulletin-tier44 technologies.jpg ciobulletin-xaana.jpg

Latest Magazines

© 2024 CIO Bulletin Inc. All rights reserved.