{"id":461267,"date":"2024-09-21T13:38:00","date_gmt":"2024-09-21T13:38:00","guid":{"rendered":"https:\/\/Blockchain.News\/news\/nvidia-nccl-2-22-memory-efficiency-faster-initialization"},"modified":"2024-09-21T13:38:00","modified_gmt":"2024-09-21T13:38:00","slug":"nvidia-unveils-nccl-2-22-with-enhanced-memory-efficiency-and-faster-initialization","status":"publish","type":"post","link":"https:\/\/e-bitco.in\/index.php\/2024\/09\/21\/nvidia-unveils-nccl-2-22-with-enhanced-memory-efficiency-and-faster-initialization\/","title":{"rendered":"NVIDIA Unveils NCCL 2.22 with Enhanced Memory Efficiency and Faster Initialization"},"content":{"rendered":"<figure class=\"figure mt-2\">\n<p> <a href=\"https:\/\/blockchain.news\/Profile\/Caroline-Bishop\">Caroline Bishop<\/a> <span class=\"publication-date ml-2\"> Sep 21, 2024 13:38<\/span> <\/p>\n<p class=\"lead\">NVIDIA introduces NCCL 2.22, focusing on memory efficiency, faster initialization, and cost estimation for improved HPC and AI applications.<\/p>\n<p> <a href=\"https:\/\/image.blockchain.news:443\/features\/D8E08E86F8EDBDDCD68414CF49BDD8B1401B11A69515DFF98E6B2B03EE9CF9D7.jpg\"> <img decoding=\"async\" class=\"rounded\" src=\"https:\/\/image.blockchain.news:443\/features\/D8E08E86F8EDBDDCD68414CF49BDD8B1401B11A69515DFF98E6B2B03EE9CF9D7.jpg\" alt=\"NVIDIA Unveils NCCL 2.22 with Enhanced Memory Efficiency and Faster Initialization\"> <\/a> <\/figure>\n<p>The NVIDIA Collective Communications Library (NCCL) has released its latest version, NCCL 2.22, bringing significant enhancements aimed at optimizing memory usage, accelerating initialization times, and introducing a cost estimation API. These updates are crucial for high-performance computing (HPC) and <a rel=\"nofollow noopener\" href=\"https:\/\/blockchain.news\/tag\/artificial-intelligence\" target=\"_blank\"><u><b><i><dfn data-info=\"Artificial Intelligence (AI), a branch of computer science, is concerned with developing computers capable of doing tasks that would typical...\">artificial intelligence<\/dfn><\/i><\/b><\/u><\/a> (AI) applications, according to the <a rel=\"nofollow\" href=\"https:\/\/developer.nvidia.com\/blog\/memory-efficiency-faster-initialization-and-cost-estimation-with-nvidia-collective-communications-library-2-22\/\">NVIDIA Technical Blog<\/a>.<\/p>\n<h2>Release Highlights<\/h2>\n<p>NVIDIA Magnum IO NCCL is designed to optimize inter-GPU and multi-node communication, which is essential for efficient parallel computing. Key features of the NCCL 2.22 release include:<\/p>\n<ul>\n<li><strong>Lazy Connection Establishment:<\/strong> This feature delays the creation of connections until they are needed, significantly reducing GPU memory overhead.<\/li>\n<li><strong>New API for Cost Estimation:<\/strong> A new API helps optimize compute and communication overlap or research the NCCL cost model.<\/li>\n<li><strong>Optimizations for <code>ncclCommInitRank<\/code>:<\/strong> Redundant topology queries are eliminated, speeding up initialization by up to 90% for applications creating multiple communicators.<\/li>\n<li><strong>Support for Multiple Subnets with IB Router:<\/strong> Adds support for communication in jobs spanning multiple InfiniBand subnets, enabling larger DL training jobs.<\/li>\n<\/ul>\n<h2>Features in Detail<\/h2>\n<h3>Lazy Connection Establishment<\/h3>\n<p>NCCL 2.22 introduces lazy connection establishment, which significantly reduces GPU memory usage by delaying the creation of connections until they are actually needed. This feature is particularly beneficial for applications that use a narrow scope, such as running the same algorithm repeatedly. The feature is enabled by default but can be disabled by setting <code>NCCL_RUNTIME_CONNECT=0<\/code>.<\/p>\n<h3>New Cost Model API<\/h3>\n<p>The new API, <code>ncclGroupSimulateEnd<\/code>, allows developers to estimate the time required for operations, aiding in the optimization of compute and communication overlap. While the estimates may not perfectly align with reality, they provide a useful guideline for performance tuning.<\/p>\n<h3>Initialization Optimizations<\/h3>\n<p>To minimize initialization overhead, the NCCL team has introduced several optimizations, including lazy connection establishment and intra-node topology fusion. These improvements can reduce <code>ncclCommInitRank<\/code> execution time by up to 90%, making it significantly faster for applications that create multiple communicators.<\/p>\n<h3>New Tuner Plugin Interface<\/h3>\n<p>The new tuner plugin interface (v3) provides a per-collective 2D cost table, reporting the estimated time needed for operations. This allows external tuners to optimize algorithm and protocol combinations for better performance.<\/p>\n<h3>Static Plugin Linking<\/h3>\n<p>For convenience and to avoid loading issues, NCCL 2.22 supports static linking of network or tuner plugins. Applications can specify this by setting <code>NCCL_NET_PLUGIN<\/code> or <code>NCCL_TUNER_PLUGIN<\/code> to <code>STATIC_PLUGIN<\/code>.<\/p>\n<h3>Group Semantics for Abort or Destroy<\/h3>\n<p>NCCL 2.22 introduces group semantics for <code>ncclCommDestroy<\/code> and <code>ncclCommAbort<\/code>, allowing multiple communicators to be destroyed simultaneously. This feature aims to prevent deadlocks and improve user experience.<\/p>\n<h3>IB Router Support<\/h3>\n<p>With this release, NCCL can operate across different InfiniBand subnets, enhancing communication for larger networks. The library automatically detects and establishes connections between endpoints on different subnets, using FLID for higher performance and adaptive routing.<\/p>\n<h2>Bug Fixes and Minor Updates<\/h2>\n<p>The NCCL 2.22 release also includes several bug fixes and minor updates:<\/p>\n<ul>\n<li>Support for the <code>allreduce<\/code> tree algorithm on DGX Google Cloud.<\/li>\n<li>Logging of NIC names in IB async errors.<\/li>\n<li>Improved performance of registered send and receive operations.<\/li>\n<li>Added infrastructure code for NVIDIA Trusted Computing Solutions.<\/li>\n<li>Separate traffic class for IB and RoCE control messages to enable advanced QoS.<\/li>\n<li>Support for PCI peer-to-peer communications across partitioned Broadcom PCI switches.<\/li>\n<\/ul>\n<h2>Summary<\/h2>\n<p>The NCCL 2.22 release introduces several significant features and optimizations aimed at improving performance and efficiency for HPC and AI applications. The improvements include a new tuner plugin interface, support for static linking of plugins, and enhanced group semantics to prevent deadlocks.<\/p>\n<p><span><i>Image source: Shutterstock<\/i><\/span> <!-- Divider --> <!-- Author info END --> <!-- Divider --> <a href=\"https:\/\/blockchain.news\/\">Source<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p>Caroline Bishop Sep 21, 2024 13:38 NVIDIA introduces NCCL 2.22, focusing on memory efficiency, faster initialization, and cost estimation for improved HPC and AI applications. The NVIDIA Collective Communications Library (NCCL) has released its latest version, NCCL 2.22, bringing significant enhancements aimed at optimizing memory usage, accelerating initialization times, and introducing a cost estimation API. [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":461268,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[12],"tags":[1129,15003,21526,25,2148],"class_list":{"0":"post-461267","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-blockchain","8":"tag-ai","9":"tag-high-performance-computing","10":"tag-nccl","11":"tag-news","12":"tag-nvidia"},"_links":{"self":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/posts\/461267","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/comments?post=461267"}],"version-history":[{"count":0,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/posts\/461267\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/media\/461268"}],"wp:attachment":[{"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/media?parent=461267"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/categories?post=461267"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/e-bitco.in\/index.php\/wp-json\/wp\/v2\/tags?post=461267"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}