Nvidia advances medical AI and digital twin capabilities



Nvidia has announced several new medical tools, partnerships and workflows at the Fall GTC Conference in San Francisco. Interactive labeling can reduce by 75% the time required to label raw imaging data for training AI models. Additionally, the Monai Flare platform supports federated learning to enhance the privacy of medical data. The summit will take place on October 4 in San Francisco.


Nvidiaは、サンフランシスコで開催された秋GTC会議で、いくつかの新しい医療ツール、パートナーシップ、ワークフローを発表しました。インタラクティブなラベル付けは、AIモデルをトレーニングするために生のイメージングデータにラベルを付けるのに必要な時間を75%削減できます。さらに、Monai Flareプラットフォームは、医療データのプライバシーを強化するためのフェデレートラーニングをサポートしています。サミットは10月4日にサンフランシスコで開催されます。


Nvidia has been a leader in providing AI and digital twin infrastructure for the medical community.


Its various offerings improve diagnostics, the development of new medical devices, medical research and drug development.


At the Fall GTC Conference, Nvidia announced various new medical tools, partnerships and workflows.  “GTC is a really unique healthcare conference, where we learn how AI and accelerated computing are advancing the field, from things like surgery all the way through to pharmaceutical research,” Nvidia’s VP of healthcare, Kimberly Powell, said in a press conference.


Highlights include: These various announcements build on and extend each other.


Let’s walk through them one at a time.  MetaBeat 2022 MetaBeat will bring together thought leaders to give guidance on how metaverse technology will transform the way all industries communicate and do business on October 4 in San Francisco, CA.

一度に1つずつ歩きましょう。Metabeat 2022 Metabeatは、10月4日にカリフォルニア州サンフランシスコで、すべての産業がコミュニケーションとビジネスを行う方法をメタバーステクノロジーがどのように変化させるかについてのガイダンスを提供するために、Sound Leadersを集めます。

[Follow along with VB’s ongoing Nvidia GTC 2022 coverage »] Nvidia and King’s College London introduced MONAI in April 2020 to simplify AI medical imaging workflows.

[VBの進行中のNVIDIA GTC 2022カバレッジに従ってください»] NvidiaとKing’s College Londonは、2020年4月にAI医療イメージングワークフローを簡素化するためにMonaiを紹介しました。

This helps transform raw imaging data into interactive digital twins to improve analysis or diagnostics, or guide surgical instruments.


The development and adoption of the platform have picked up steam with over 600,000 downloads, half of these in the last six months.  They are now officially rolling out Monai 1.0.


It comes with several critical capabilities baked in.


Interactive labeling can reduce by 75% the time required to label data for training AI models.


Auto3D adapts AutoML techniques for automatically choosing machine learning models for 3D segmentation and interpretation.


Monai Flare supports federated learning to enhance the privacy of medical data.

Monai Flareは、医療データのプライバシーを強化するためのフェデレーションラーニングをサポートしています。

Model Zoo comes with over 15 pretrained models.


Native support for streaming imaging applications like endoscopy, ultrasound and surgical video helps streamline medical imaging workflows.  Nvidia introduced Clara Holoscan MGX earlier this year as a reference design for a medical device platform.

内視鏡検査、超音波、外科ビデオなどのストリーミングイメージングアプリケーションのネイティブサポートは、医療イメージングワークフローを合理化するのに役立ちます。Nvidiaは、医療機器プラットフォームの参照設計として、今年初めにClara Holoscan MGXを導入しました。

Clara Holoscan on IGX builds on Nvidia’s experience to further streamline and industrialize medical device development on top of Nvidia’s new IGX platform for robotics.

IGXのClara Holoscanは、Nvidiaの新しいIGXプラットフォームに加えて、医療機器の開発をさらに合理化および工業化するためのNvidiaの経験に基づいています。

This reduces the effort it takes to integrate Holoscan into new products with integrated security and management capabilities.


Over 70 leading companies have been developing equipment on top of Clara Holoscan MGX, including Siemens Healthineers for MRI, Olympus for endoscopy, and Intuitive Ion for better lung biopsies.

MRIのSiemens Healthineers、内視鏡検査用のオリンパス、より良い肺生検のための直感的なイオンなど、70を超える大手企業がClara Holoscan MGXの上に機器を開発しています。

New products based on Clara Holoscan and IGX include Activ Surgical’s hyperspectral blood flow imager, Moon Surgical’s robotic-assisted surgeon, and Proximie’s telepresence surgery system.  “We learned that what we’re building for these medical device use cases is actually applicable to a much broader market,” said Powell.

Clara HoloscanとIGXに基づく新製品には、Actic SurgicalのHyperspectral Blood Flow Imager、Moon Surgicalのロボット支援外科医、および近接テレプレゼンス手術システムが含まれます。「これらの医療機器のユースケースのために構築しているものは、実際にはるかに広範な市場に適用できることを学びました」とパウエルは言いました。

“Industrial automation and smart factories all have a similar robotics pipeline that needs to be executed on the far edges of the network and incorporate things like functional safety so that humans and robots can be in the same place.” The platform also helps minimize new applications’ latency to ensure patient safety.


Powell said they set the goal of keeping latency down to 50 milliseconds.


The latest version of Holoscan can do straight-up video processing in less than 10 milliseconds and supports more than 30 simultaneously running AI algorithms at less than 50 milliseconds.  Powell said they are aligning Clara with Nvidia’s Isaac platform for robotics and Omniverse platform for industrial digital twins.


“We’re leveraging everything the company makes, and we’re connecting these platforms together because robotics isn’t unique in healthcare as it is in other domains,” Powell said.


“And we take all the lessons learned and the necessary interconnections between these platforms to provide it back to the medical device industry.” Nvidia’s new BioNeMo Framework helps medical researchers train and develop large biomolecular language models at supercomputing scales.


It extends efforts like the Nvidia NeMo Megatron framework and research projects like AlphaFold that use large language models to analyze proteins to support DNA, protein and chemical research.   Each domain has its own unique way of encoding data into strings.

Nvidia nemo megatronフレームワークのような努力を拡大し、Alphafoldのような研究プロジェクトを拡張し、大規模な言語モデルを使用してDNA、タンパク質、化学研究をサポートするタンパク質を分析します。各ドメインには、文字列にデータをエンコードする独自の方法があります。

DNA uses nucleic acid sequences, proteins use amino acid sequences, and chemicals use simplified molecular-input line-entry system (SMILES) strings.


We have over 10,000 diseases and only 500 cures,” said Powell.


“We need to boost numerical and experimental methods with AI to explore the nearly infinite chemistry and protein space.


Nvidia BioNeMo LLM framework and cloud services will accelerate the development of AI that understands chemistry and biology.”  The new framework comes with four pretrained models.

Nvidia Bionemo LLMフレームワークとクラウドサービスは、化学と生物学を理解しているAIの開発を加速します。」新しいフレームワークには、4つの前提条件モデルが付属しています。

ESM-1, introduced by Meta AI Labs, processes amino acid sequences to predict properties and functions.

Meta AI Labsによって導入されたESM-1は、特性と機能を予測するためにアミノ酸配列を処理します。

OpenFold helps visualize proteins.


MegaMolBART can help predict chemical reactions, optimize mixtures or generate new ones.


ProtT5 helps extend the capabilities of protein large language models to sequence generation.  Powell said Nvidia is providing BioNeMo as both a framework and a service.


The framework will help researchers develop new pre-trained language models at any scale for chemistry, protein, DNA and RNA.


It also supports data transformations necessary for biomolecules.


Nvidia plans to provide early access to the BioNeMo service in October.  Nvidia has also announced an extensive partnership with the Broad Institute of MIT and Harvard, a top genetics research group and tools provider.

Nvidiaは、10月にBionemoサービスへの早期アクセスを提供する予定です。Nvidiaはまた、Top Genetics Research Group and Tools ProviderであるMITとハーバードのブロード研究所との広範なパートナーシップを発表しました。

Nvidia is porting Clara Parabricks computational genomics framework to the Broad Institute’s Terra cloud platform used by 25,000 leading medical researchers.

Nvidiaは、Clara Parabricks Computational Genomics Frameworkを、25,000人の主要な医学研究者が使用するBroad InstituteのTerra Cloud Platformに移植しています。

Initially, they plan to support six new workflows.


For example, a new whole genome sequencing workflow running on GPUs shortens the process from a day to an hour and cuts the cost in half compared to a CPU approach.  The two will also partner on building large language models for analyzing DNA and RNA.


Nvidia is also contributing a new deep learning model to the Broad Institute’s genome analysis toolkit that more than 100,000 researchers use.  Powell said combining the Broad Institute’s deep domain expertise with Nvidia technology expertise could accelerate the deployment of new AI medical innovations from years to months.

Nvidiaはまた、100,000人以上の研究者が使用するBroad Instituteのゲノム分析ツールキットに新しい深い学習モデルを提供しています。パウエルは、Broad Instituteの深いドメインの専門知識とNVIDIA技術の専門知識を組み合わせることで、長年から数か月から数か月にかけて、新しいAI医療革新の展開を加速できると述べました。

VentureBeat’s mission is to be a digital town square for technical decision-makers to gain knowledge about transformative enterprise technology and transact.