2

我有这个 cuda 代码，当我用 cuda-memcheck 执行时它没有返回错误，它正常退出，我得到的结果实际上是预期的......同时，有一个文件“cuda-memcheck-(在这里放置各种数字）.out”创建，即为空。

当我在 cuda-gdb 下运行相同的程序时，它也正常退出，没有错误报告。

但是当我“设置 cuda memcheck on”（在 cuda-gdb 下）然后运行程序时，会创建一个文件“cuda-memcheck.out”，上面写着：

启动 cuda-memcheck...

cuda-memcheck 遇到错误 (3,2,2)

一旦我从 cuda-gdb 中执行“运行”，就会发生这种情况。然后，在执行开始后不久（实际上非​​常接近 CUFFT 内核执行），我得到以下信息：

Program received signal CUDA_EXCEPTION_1, Lane Illegal Address.
[Switching focus to CUDA kernel 81, grid 82, block (0,0,0), thread (1,17,0), device 0, sm 0, warp 2, lane 5]
*** glibc detected *** cuda-gdb: double free or corruption (!prev): 0x000000001070e5d0 ***
======= Backtrace: =========
/lib/libc.so.6(+0x774b6)[0x7f597c4814b6]
/lib/libc.so.6(cfree+0x73)[0x7f597c487c83]
/lib/libc.so.6(obstack_free+0x48)[0x7f597c48b128]
cuda-gdb[0x5c5a25]
cuda-gdb[0x4352d0]
cuda-gdb[0x5649c4]
cuda-gdb[0x5e42d7]
cuda-gdb[0x5e45fd]
cuda-gdb[0x56436c]
cuda-gdb[0x50d772]
cuda-gdb[0x50e3c3]
cuda-gdb[0x51373b]
cuda-gdb[0x50faff]
cuda-gdb[0x510194]
cuda-gdb[0x51373b]
cuda-gdb[0x50eaac]
cuda-gdb[0x504311]
cuda-gdb[0x50a09d]
cuda-gdb[0x4ffc4f]
cuda-gdb[0x410b9d]
cuda-gdb[0x519c48]
cuda-gdb[0x51a82c]
cuda-gdb[0x5f9b57]
cuda-gdb[0x519cb9]
cuda-gdb[0x5186c8]
cuda-gdb[0x51995a]
cuda-gdb[0x51373b]
cuda-gdb[0x4a1a40]
cuda-gdb[0x407529]
cuda-gdb[0x51373b]
cuda-gdb[0x408056]
cuda-gdb[0x51373b]
cuda-gdb[0x407464]
cuda-gdb[0x40742e]
/lib/libc.so.6(__libc_start_main+0xfe)[0x7f597c428d8e]
cuda-gdb[0x407339]
======= Memory map: ========
00400000-008b6000 r-xp 00000000 08:01 190634                             /usr/local/cuda/bin/cuda-gdb
00ab5000-00ab6000 r--p 004b5000 08:01 190634                             /usr/local/cuda/bin/cuda-gdb
00ab6000-00ac9000 rw-p 004b6000 08:01 190634                             /usr/local/cuda/bin/cuda-gdb
00ac9000-0f0f8000 rw-p 00000000 00:00 0 
0fef3000-11b2c000 rw-p 00000000 00:00 0                                  [heap]
7f5974000000-7f5974021000 rw-p 00000000 00:00 0 
7f5974021000-7f5978000000 ---p 00000000 00:00 0 
7f597a4df000-7f597aacc000 rw-p 00000000 00:00 0 
7f597aacc000-7f597aacd000 rw-p 00000000 00:00 0 
7f597b538000-7f597b54d000 r-xp 00000000 08:01 954799                     /lib/libgcc_s.so.1
7f597b54d000-7f597b74c000 ---p 00015000 08:01 954799                     /lib/libgcc_s.so.1
7f597b74c000-7f597b74d000 r--p 00014000 08:01 954799                     /lib/libgcc_s.so.1
7f597b74d000-7f597b74e000 rw-p 00015000 08:01 954799                     /lib/libgcc_s.so.1
7f597b75d000-7f597b764000 r-xp 00000000 08:01 954961                     /lib/libthread_db-1.0.so
7f597b764000-7f597b963000 ---p 00007000 08:01 954961                     /lib/libthread_db-1.0.so
7f597b963000-7f597b964000 r--p 00006000 08:01 954961                     /lib/libthread_db-1.0.so
7f597b964000-7f597b965000 rw-p 00007000 08:01 954961                     /lib/libthread_db-1.0.so
7f597b965000-7f597b966000 ---p 00000000 00:00 0 
7f597b966000-7f597c166000 rw-p 00000000 00:00 0 
7f597c166000-7f597c40a000 r--p 00000000 08:01 49399                      /usr/lib/locale/locale-archive
7f597c40a000-7f597c584000 r-xp 00000000 08:01 954957                     /lib/libc-2.12.1.so
7f597c584000-7f597c783000 ---p 0017a000 08:01 954957                     /lib/libc-2.12.1.so
7f597c783000-7f597c787000 r--p 00179000 08:01 954957                     /lib/libc-2.12.1.so
7f597c787000-7f597c788000 rw-p 0017d000 08:01 954957                     /lib/libc-2.12.1.so
7f597c788000-7f597c78d000 rw-p 00000000 00:00 0 
7f597c78d000-7f597c78f000 r-xp 00000000 08:01 954973                     /lib/libdl-2.12.1.so
7f597c78f000-7f597c98f000 ---p 00002000 08:01 954973                     /lib/libdl-2.12.1.so
7f597c98f000-7f597c990000 r--p 00002000 08:01 954973                     /lib/libdl-2.12.1.so
7f597c990000-7f597c991000 rw-p 00003000 08:01 954973                     /lib/libdl-2.12.1.so
7f597c991000-7f597c9b7000 r-xp 00000000 08:01 954792                     /lib/libexpat.so.1.5.2
7f597c9b7000-7f597cbb7000 ---p 00026000 08:01 954792                     /lib/libexpat.so.1.5.2
7f597cbb7000-7f597cbb9000 r--p 00026000 08:01 954792                     /lib/libexpat.so.1.5.2
7f597cbb9000-7f597cbba000 rw-p 00028000 08:01 954792                     /lib/libexpat.so.1.5.2
7f597cbba000-7f597cc3c000 r-xp 00000000 08:01 954964                     /lib/libm-2.12.1.so
7f597cc3c000-7f597ce3b000 ---p 00082000 08:01 954964                     /lib/libm-2.12.1.so
7f597ce3b000-7f597ce3c000 r--p 00081000 08:01 954964                     /lib/libm-2.12.1.so
7f597ce3c000-7f597ce3d000 rw-p 00082000 08:01 954964                     /lib/libm-2.12.1.so
7f597ce3d000-7f597ce53000 r-xp 00000000 08:01 954914                     /lib/libz.so.1.2.3.4
7f597ce53000-7f597d053000 ---p 00016000 08:01 954914                     /lib/libz.so.1.2.3.4
7f597d053000-7f597d054000 r--p 00016000 08:01 954914                     /lib/libz.so.1.2.3.4
7f597d054000-7f597d055000 rw-p 00017000 08:01 954914                     /lib/libz.so.1.2.3.4
7f597d055000-7f597d095000 r-xp 00000000 08:01 954818                     /lib/libncurses.so.5.7
7f597d095000-7f597d294000 ---p 00040000 08:01 954818                     /lib/libncurses.so.5.7
7f597d294000-7f597d298000 r--p 0003f000 08:01 954818                     /lib/libncurses.so.5.7
7f597d298000-7f597d299000 rw-p 00043000 08:01 954818                     /lib/libncurses.so.5.7
7f597d299000-7f597d2b1000 r-xp 00000000 08:01 954959                     /lib/libpthread-2.12.1.so
7f597d2b1000-7f597d4b0000 ---p 00018000 08:01 954959                     /lib/libpthread-2.12.1.so
7f597d4b0000-7f597d4b1000 r--p 00017000 08:01 954959                     /lib/libpthread-2.12.1.so
7f597d4b1000-7f597d4b2000 rw-p 00018000 08:01 954959                     /lib/libpthread-2.12.1.so
7f597d4b2000-7f597d4b6000 rw-p 00000000 00:00 0 
7f597d4b6000-7f597d4d6000 r-xp 00000000 08:01 954965                     /lib/ld-2.12.1.so
7f597d4df000-7f597d672000 rw-p 00000000 00:00 0 
7f597d672000-7f597d678000 r--p 00a01000 08:06 26722558                   /home/user/workspace/cuda/fullcu/current_debug/Default/shl_3D_cu
7f597d678000-7f597d67e000 r--p 00a15000 08:06 26722558                   /home/user/workspace/cuda/fullcu/current_debug/Default/shl_3D_cu
7f597d67e000-7f597d687000 r--p 00a06000 08:06 26722558                   /home/user/workspace/cuda/fullcu/current_debug/Default/shl_3D_cu
7f597d687000-7f597d6c0000 r--p 009c5000 08:06 26722558                   /home/user/workspace/cuda/fullcu/current_debug/Default/shl_3D_cu
7f597d6c0000-7f597d6c5000 rw-p 00000000 00:00 0 
7f597d6c5000-7f597d6ca000 r--p 009fd000 08:06 26722558                   /home/user/workspace/cuda/fullcu/current_debug/Default/shl_3D_cu
7f597d6cb000-7f597d6cd000 rw-p 00000000 00:00 0 
7f597d6cd000-7f597d6d4000 r--s 00000000 08:01 49265                      /usr/lib/gconv/gconv-modules.cache0x000000000366a8d0 in fdividef<<<(16,1,1),(4,64,1)>>> (Aborted

和 cuda-gdb 崩溃。

我是否应该认为我的代码中实际上存在错误的内存访问？还是在初始化 cuda-memcheck 时遇到的错误？？？

有人见过这种行为吗？

谢谢你的任何想法。

4

3 回答 3

1

您的应用程序中似乎存在可检测到 memcheck 的错误。由于某种原因，当您的应用程序在断点处暂停时，cuda-gdb 会崩溃。

  1. 当您在设备代码中的常规断点处停止时，您的 cuda-gdb 会崩溃吗？
  2. 您使用的是什么 CUDA Toolkit 和 NVIDIA 显示驱动程序版本？我们建议您尝试最新的 CUDA Toolkit 5.0RC 版本，因为它具有许多稳定性和功能改进。
  3. 如果您可以直接通过 cudatools@nvidia.com 与团队联系以提供更多信息，那将是非常宝贵的（这样我们可能能够在下一个 CUDA Toolkit 版本中解决问题）。能否也提供触发崩溃的应用程序？

先感谢您。

于 2012-09-12T17:36:37.833 回答
0

为了很好地追踪这个问题，最好向 Nvidia 提交一个错误。提交 bug 的步骤如下： 1. 打开页面http://developer.nvidia.com/cuda/join-cuda-registered-developer-program；2、如未注册，请点击“立即加入”，否则点击“立即登录”；3、输入邮箱和密码登录；4. 左侧面板首页有“Bug Report”项，点击即可提交Bug；5. 填写必填项，其他项可选，但详细信息将有助于我们定位和修复问题；6. 如有必要，应上传附件；7、Linux系统最好附上nvidia-bug-report；8. 如果问题与特定代码模式有关，则需要示例代码和编译说明以进行复制。

于 2012-09-14T10:16:22.687 回答
0

Based on Eugene's question on what happens on regular breakpoints, I added some in key points in the code ('key' with respect to the debugging output I have gotten until now) and I have the following 'odd' gdb output to report:

The following is from a CUFFT kernel call:

[Launch of CUDA Kernel 41 (spVector0016B_kernelTex<(fftDirection_t)-1><<<(16,1,1),(4,64,1)>>>) on Device 0]

0x00007ffff569c8b3 in select () from /lib/libc.so.6

(cuda-gdb) n

Single stepping until exit from function select, which has no line number information.

0x00007ffff3fe8de7 in ?? () from /usr/lib/libcuda.so

Is '??' normal output in that case because it's a CUFFT kernel?

The following is from another kernel launch, mine this time and I don't understand why it says 'no such file':

Breakpoint 5, BCSG<<<(1,1,1),(512,1,1)>>> (glerror=0x200620df8, sL=0x2009ffff8, ijL=0x200aa7ffc, X=0x2006035f8, D=0x2006075f8, Pre=0x200941ff8, error=1.00000001e-10, L=700, N=1024, flag=0x200c00000, r=0x2006095f8, r0=0x20060b5f8, p1=0x20060d5f8, p2=0x20060f5f8, vv=0x2006115f8, s1=0x2006135f8, s2=0x2006155f8, t=0x2006175f8, T=0x2006195f8, r1=0x20061b600, a=0x20061cbf8, w=0x20061e1f8, beta=0x20061f7f8, ns1=0x200affffc) at BCGC_solver.cu:96

96 BCGC_solver.cu: No such file or directory.

in BCGC_solver.cu

Also i would like to ask if this:

warning: Warp(s) other than the current warp had to be single-stepped.

...happens because of the execution within cuda-gdb, or if it is something that would happen in normal program execution as well. I do have the program hanging at parts of the code, soon after it freezes for good. But I think that if it was only a matter of serial execution, it would continue soon after, as it does in conventional code.

于 2012-09-13T12:20:09.693 回答