BRP5-cuda32-nv270 runs faster on old card

Forums › Problems and Bug Reports

felixonmars
felixonmars
Joined: 16 Nov 12
Posts: 7
Credit: 3409581
RAC: 0
16 Oct 2013 7:08:22 UTC
Topic 197225
(moderation:
    )

    Hi all, I have two machines with different display card, one is GTX 650 Ti, the other is 9600 GT.

    The problem is, the WUs run 8x faster on the older one (9600 GT)

    I did try to run it on 550 Ti, and it's still a lot faster than 650 Ti.

    All tasks were running under same OS (Arch Linux x86_64, with latest 3.11.x linux kernel, and latest nvidia driver 325.15)

    I'm not familiar with GPU, so I could not really tell why it's like this - but could anyone help me?

    Two WUs for comparison:
    http://einsteinathome.org/task/405344492
    http://einsteinathome.org/task/404679225

    And one more strange thing is, I claim fairly low credits from the WUs, like for the two above WUs:
    Claimed credit / Granted credit
    48.38 / 3,333.00
    20.51 / 3,333.00

    They were both shown as "Completed and validated" though.

    Thanks for any help!

    Twitter: @felixonmars

    felixonmars
    felixonmars
    Joined: 16 Nov 12
    Posts: 7
    Credit: 3409581
    RAC: 0

    BRP5-cuda32-nv270 runs faster on old card

    16 Oct 2013 7:12:20 UTC
    Message 118957
    (moderation:
      )

      Oops I think I found some clue:

      Using CUDA device #0 "GeForce 9600 GT" (64 CUDA cores / 312.00 GFLOPS)

      vs

      Using CUDA device #0 "GeForce GTX 650 Ti" (0 CUDA cores / 0.00 GFLOPS)

      Why's the latter shown as 0?

      Twitter: @felixonmars

      Neil Newell
      Neil Newell
      Joined: 20 Nov 12
      Posts: 176
      Credit: 169699457
      RAC: 0

      It seems very likely the 8x

      16 Oct 2013 18:53:40 UTC
      Message 118958 in response to message 118957
      (moderation:
        )

        It seems very likely the 8x WU's were using the GPU (9600GT) but the slower ones are just using the CPU instead. For some reason the GTX650Ti isn't being used.

        Can you post the BOINC start-up messages?

        On the strange thing about claiming low credits for WU's, I think this is what happens your GPU does work a lot faster than a CPU can. If the CPU did the work, the claimed credit would be higher - but because GPU's are so much faster, the claimed credit is lower. The 'granted' credit for a particular task is always the same, no matter how you did the work - so really 'claimed credit' isn't something to worry about.

        mountkidd
        mountkidd
        Joined: 14 Jun 12
        Posts: 175
        Credit: 11111978849
        RAC: 4786361

        RE: Oops I think I found

        16 Oct 2013 20:13:45 UTC
        Message 118959 in response to message 118957
        (moderation:
          )

          Quote:

          Oops I think I found some clue:

          Using CUDA device #0 "GeForce 9600 GT" (64 CUDA cores / 312.00 GFLOPS)

          vs

          Using CUDA device #0 "GeForce GTX 650 Ti" (0 CUDA cores / 0.00 GFLOPS)

          Why's the latter shown as 0?


          The application code reporting these values works for GTX 5xx and earlier. GTX 6XX/7XX are reported incorrectly as zeroes, but do in fact produce results in spite of what the message says.

          Gord

          felixonmars
          felixonmars
          Joined: 16 Nov 12
          Posts: 7
          Credit: 3409581
          RAC: 0

          RE: RE: Oops I think I

          17 Oct 2013 3:59:08 UTC
          Message 118960 in response to message 118959
          (moderation:
            )

            Quote:
            Quote:

            Oops I think I found some clue:

            Using CUDA device #0 "GeForce 9600 GT" (64 CUDA cores / 312.00 GFLOPS)

            vs

            Using CUDA device #0 "GeForce GTX 650 Ti" (0 CUDA cores / 0.00 GFLOPS)

            Why's the latter shown as 0?


            The application code reporting these values works for GTX 5xx and earlier. GTX 6XX/7XX are reported incorrectly as zeroes, but do in fact produce results in spite of what the message says.

            Gord

            Thanks for the info, and is this also the cause that GTX 6XX cards run so slowly?

            Twitter: @felixonmars

            Neil Newell
            Neil Newell
            Joined: 20 Nov 12
            Posts: 176
            Credit: 169699457
            RAC: 0

            RE: The application code

            17 Oct 2013 7:47:15 UTC
            Message 118961 in response to message 118960
            (moderation:
              )

              Quote:
              The application code reporting these values works for GTX 5xx and earlier. GTX 6XX/7XX are reported incorrectly as zeroes, but do in fact produce results in spite of what the message says.
              Gord


              Aha, thanks; I didn't know that.

              Quote:

              Thanks for the info, and is this also the cause that GTX 6XX cards run so slowly?

              I'm not familiar with the GTX650, but 185,000 seconds for the GTX650 task seems far too long (especially given your 9600GT did a similar task in about ~27,000 seconds).

              Here's an extract from the 9600GT log:

              [21:53:24][12146][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 293 MB (219 MB free / 512 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[21:53:24][12146][INFO ] Using CUDA device #0 "GeForce 9600 GT" (64 CUDA cores / 312.00 GFLOPS)
[21:53:24][12146][INFO ] Version of installed CUDA driver: 5050
[21:53:24][12146][INFO ] Version of CUDA driver API used: 3020

              ...and here's the GX650Ti.

              ------> Used in total: 145 MB (879 MB free / 1024 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[20:17:13][7147][INFO ] Using CUDA device #0 "GeForce GTX 650 Ti" (0 CUDA cores / 0.00 GFLOPS)
[20:17:13][7147][INFO ] Version of installed CUDA driver: 5050
[20:17:13][7147][INFO ] Version of CUDA driver API used: 3020

              That all looks reasonable, though I see that 'Signal 15' was thrown once in the log for the GTX650Ti.

              Any ideas, Gord? They are both BRP5 tasks, both credited 3,333.

              mountkidd
              mountkidd
              Joined: 14 Jun 12
              Posts: 175
              Credit: 11111978849
              RAC: 4786361

              RE: Thanks for the info,

              17 Oct 2013 14:55:17 UTC
              Message 118962 in response to message 118960
              (moderation:
                )

                Quote:
                Thanks for the info, and is this also the cause that GTX 6XX cards run so slowly?


                I don't think so. That message is [INFO] only and I don't believe the values are used anywhere.

                What I do see in your scheduler log is that you have not freed up a cpu core to feed the gpu. A second issue might be the driver. This thread talks about a similar problem to yours and has a link to some info about recent Linux drivers.

                Gord

                felixonmars
                felixonmars
                Joined: 16 Nov 12
                Posts: 7
                Credit: 3409581
                RAC: 0

                RE: What I do see in your

                18 Oct 2013 4:57:28 UTC
                Message 118963 in response to message 118962
                (moderation:
                  )

                  Quote:

                  What I do see in your scheduler log is that you have not freed up a cpu core to feed the gpu. A second issue might be the driver. This thread talks about a similar problem to yours and has a link to some info about recent Linux drivers.

                  Gord

                  Thanks a lot. I've tried freed up a cpu core but that doesn't make a difference - now I'm going to downgrade my driver to 304.xx and test.

                  Twitter: @felixonmars

                  felixonmars
                  felixonmars
                  Joined: 16 Nov 12
                  Posts: 7
                  Credit: 3409581
                  RAC: 0

                  Oops, after downgraded driver

                  18 Oct 2013 5:17:45 UTC
                  Message 118964 in response to message 118963
                  (moderation:
                    )

                    Oops, after downgraded driver to 304.108 (from 325.15), it's much faster and should be normal now.

                    Thanks a lot for all your help!

                    Twitter: @felixonmars

                    Bikeman (Heinz-Bernd Eggenstein)
                    Bikeman (Heinz-...
                    Moderator
                    Joined: 28 Aug 06
                    Posts: 3522
                    Credit: 692128497
                    RAC: 94635

                    Hi! You might have ssen

                    27 Oct 2013 19:26:31 UTC
                    Message 118965
                    (moderation:
                      )

                      Hi!

                      You might have ssen the thread (in the Crunchers' Corner" section) that discusses results from the latest 331.xx series Beta drivers. It seems the problem is fixed in the latest beta drivers from NVIDIA, so there is hope for the next official driver.

                      Cheers
                      HB

                      felixonmars
                      felixonmars
                      Joined: 16 Nov 12
                      Posts: 7
                      Credit: 3409581
                      RAC: 0

                      Thanks for the info! I'll try

                      10 Nov 2013 6:41:20 UTC
                      Message 118966 in response to message 118965
                      (moderation:
                        )

                        Thanks for the info! I'll try the 331 series once a stable version is tagged.

                        Twitter: @felixonmars

                        Comment viewing options

                        Select your preferred way to display the comments and click "Save settings" to activate your changes.

                        Forums › Problems and Bug Reports