[doc-only] cuda_core: add "10 minutes to cuda.core" tutorial to docs by sri-koundinyan · Pull Request #2289 · NVIDIA/cuda-python

sri-koundinyan · 2026-07-01T17:59:30Z

This adds a beginner-friendly "10 minutes to cuda.core" guide to the cuda.core docs. It walks through the core workflow with small, runnable snippets: selecting a device, compiling a CUDA C++ kernel, allocating memory, copying, launching, timing with events, using multiple streams, capturing a CUDA graph, and interoperating with CuPy and PyTorch. The new page lives at cuda_core/docs/source/10_minutes_to_cuda_core.rst and is added to the docs table of contents between the Installation and Examples sections. All code snippets were verified to run end-to-end on CUDA 13.

Add a beginner-friendly "10 minutes to cuda.core" guide that walks through the core workflow (select a device, compile a kernel, allocate memory, copy, launch, time with events, use multiple streams, capture a CUDA graph, and interoperate with CuPy/PyTorch), then wire it into the cuda.core docs table of contents between the installation and examples sections. Signed-off-by: Sri Koundinyan <skoundinyan@nvidia.com>

Andy-Jost

This looks like a great addition to the docs. Blocking merge for a few minor correctness issues but mostly it looks fine to me.

Andy-Jost · 2026-07-06T17:18:01Z

+:doc:`examples <examples>` show this scaled up across multiple GPUs.
+
+
+Capturing work in a CUDA graph


It's probably worth adding a sentence to note that cuda.core also provides an explicit graph interface. It can be used to build, inspect, and modify graphs (including graphs built via stream capture). There are lots of examples under cuda_core/tests/graph.

Good call, added a short paragraph pointing to the explicit cuda.core.graph interface with a link to the graph tests.

Andy-Jost · 2026-07-06T17:22:52Z

+
+   .. code-block:: console
+
+      $ pip install cuda-core[cu13]


Square brackets require being quoted in most shells.

Andy-Jost · 2026-07-06T17:27:30Z

+
+.. code-block:: python
+
+   dev = Device()        # device 0 by default; Device(1) selects another GPU


The comment is misleading. Device() returns the current device, while falling back to device 0 if none is current.

Fixed: now "current device (device 0 if none is current)".

Andy-Jost · 2026-07-06T17:37:34Z

+:class:`Device` objects are thread-local singletons: ``Device(0)`` always hands
+you back the same object for device 0 on that thread, so libraries sharing your
+process see and use the same GPU. Rich device attributes live under


This is a false cause-and-effect. Thread-locality is not the reason why libraries can share the same device.

What users may observe is that on a given thread Device(0) always hands back the same object for device 0, so its context, default memory resource, and cached attributes are reused rather than rebuilt on each call.

On different threads, Device(0) returns different objects bound to distinct contexts. This makes it possible to interact with CUDA from multiple threads in cuda.core.

Got it. Updated to include a short note (same-thread Device(0) returns the same object, so repeat calls are cheap) and left the primary-context sharing content to the interop section.

Andy-Jost · 2026-07-06T17:40:21Z

+   try:
+       Program(bad, code_type="c++", options=opts).compile("cubin")
+   except Exception as e:
+       print(e)        # includes: error: identifier "not_a_real_symbol" is undefined


Could say assert 'identifier "not_a_real_symbol" is undefined' in str(e)

Andy-Jost · 2026-07-06T18:03:59Z

+Capturing work in a CUDA graph
+------------------------------
+
+When you launch the same sequence of operations repeatedly, per-launch CPU


We can't use "launch" to refer to an arbitrary CUDA operation. CUDA graphs capture many types of operations, and kernel launches are only one type.

Reworded to "operations" rather than "launches". "launch" now only refers to launching the graph itself.

Andy-Jost · 2026-07-06T18:05:42Z

+------------------------------
+
+When you launch the same sequence of operations repeatedly, per-launch CPU
+overhead adds up. A CUDA graph lets you record that sequence once and replay it


"Sequence" here is potentially confusing. In general, a graph is not a sequence.

Got it. Removed "sequence" from the prose and the code comment.

Andy-Jost · 2026-07-06T18:06:41Z

+
+When you launch the same sequence of operations repeatedly, per-launch CPU
+overhead adds up. A CUDA graph lets you record that sequence once and replay it
+with a single launch. Use the stream's graph builder: begin building, issue the


The graph lets you replay a computation many times without incurring the cost of CUDA API calls each time.

A slight reframing: "record that work once and then launch the whole graph repeatedly, instead of re-issuing each operation every time."

Andy-Jost · 2026-07-06T18:11:14Z

+   ts = dev.create_stream(PyTorchStreamWrapper(torch.cuda.current_stream()))
+   launch(ts, config, scale, t.data_ptr(), np.float32(2.0), np.uint64(t.numel()))
+   ts.sync()
+   assert torch.allclose(t, torch.full_like(t, 2.0))


Exact comparison is appropriate here. It could match line 342 exactly or at least use torch.equal instead of allclose.

Andy-Jost · 2026-07-06T18:21:46Z

+   # or scope a buffer with a with-statement:
+   with dev.allocate(nbytes, stream=stream) as tmp:
+       ...  # tmp is freed at the end of the block


@leofang I'm surprised to see context manager support in Buffer. I'm for it, but I thought we were avoiding context managers in cuda.core. FYI, it was added in PR #1701 ("Make GraphicsResource inherit from Buffer").

Users reading this might expect other cuda.core objects with close members to support this as well. Is that something we can add?

I thought there was a special reason why graphic resource works differently. Are we teaching it here? If not, we should not teach about context managers.

Removed the context-manager example

juenglin · 2026-07-06T22:43:08Z

+   for _ in range(100):
+       launch(stream, config, scale, dbuf, np.float32(1.0), np.uint64(n))
+   stream.record(end)
+   end.sync()


Should this be stream.sync()?

Thanks. I kept end.sync(). It matches the pattern in the Event docstring. But stream.sync() would also work since end is the last op.

juenglin · 2026-07-06T23:53:00Z

+   # PyTorch: wrap torch's stream via the __cuda_stream__ protocol
+   import torch
+
+   class PyTorchStreamWrapper:


This adapter is no longer needed. You can use

ts = dev.create_stream(torch.cuda.current_stream())

below.

But note my comment here: #2305 (comment)

Dropped the wrapper. Now dev.create_stream(torch.cuda.current_stream()) directly.

Mainly: fix the Device()/thread-local wording for accuracy, simplify the PyTorch stream example (torch supports __cuda_stream__ natively), stop teaching Buffer as a context manager, and tidy up the CUDA graph section. Signed-off-by: Sri Koundinyan <skoundinyan@nvidia.com>

Andy-Jost

Looks great, thanks for the updates.

github-actions · 2026-07-08T21:12:32Z

Doc Preview CI
Preview removed because the pull request was closed or merged.

github-actions Bot added the cuda.core Everything related to the cuda.core module label Jul 1, 2026

This comment has been minimized.

Sign in to view

sri-koundinyan force-pushed the docs/cuda-core-10min branch from 976cc4d to 046eac2 Compare July 1, 2026 18:40

leofang added the PR review get-together Mark PRs you'd like the team to review at the weekly PR review get-together. label Jul 6, 2026

Andy-Jost requested changes Jul 6, 2026

View reviewed changes

juenglin approved these changes Jul 7, 2026

View reviewed changes

leofang added documentation Improvements or additions to documentation P1 Medium priority - Should do labels Jul 7, 2026

sri-koundinyan requested review from Andy-Jost and leofang July 8, 2026 20:06

Andy-Jost approved these changes Jul 8, 2026

View reviewed changes

Andy-Jost changed the title ~~cuda_core: add "10 minutes to cuda.core" tutorial to docs~~ [doc-only] cuda_core: add "10 minutes to cuda.core" tutorial to docs Jul 8, 2026

Andy-Jost added this to the cuda.core v1.1.0 milestone Jul 8, 2026

Andy-Jost self-assigned this Jul 8, 2026

Andy-Jost mentioned this pull request Jul 8, 2026

cuda-core v1.1.0 release checklist #2291

Closed

14 tasks

Andy-Jost merged commit b1c474c into NVIDIA:main Jul 8, 2026
104 of 106 checks passed

		:doc:`examples <examples>` show this scaled up across multiple GPUs.


		Capturing work in a CUDA graph


		.. code-block:: python

		dev = Device() # device 0 by default; Device(1) selects another GPU

Uh oh!

Conversation

sri-koundinyan commented Jul 1, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

Andy-Jost left a comment

Choose a reason for hiding this comment

Uh oh!

Andy-Jost Jul 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sri-koundinyan Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sri-koundinyan Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sri-koundinyan Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sri-koundinyan Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sri-koundinyan Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Andy-Jost left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

github-actions Bot commented Jul 8, 2026

Uh oh!

Reviewers

sri-koundinyan commented Jul 1, 2026 •

edited

Loading

Andy-Jost Jul 6, 2026 •

edited

Loading

sri-koundinyan Jul 8, 2026 •

edited

Loading

sri-koundinyan Jul 8, 2026 •

edited

Loading

sri-koundinyan Jul 8, 2026 •

edited

Loading

sri-koundinyan Jul 8, 2026 •

edited

Loading

sri-koundinyan Jul 8, 2026 •

edited

Loading