Not sending work because client is outdated

Horacio
Horacio
Joined: 3 Oct 11
Posts: 205
Credit: 80557243
RAC: 0
Topic 196331

Ive seen that message in the rpc logs...

Quote:
2012-05-17 13:49:21.9722 [PID=11301] [send] CPU: req 442.88 sec, 0.00 instances; est delay 0.00
...
2012-05-17 13:49:22.0310 [PID=11301] outdated client version 61060 < min core version 70027
...
2012-05-17 13:49:22.0351 [PID=11301] Not sending work because client is outdated
...
2012-05-17 13:49:22.0370 [PID=11301] Sending reply to [HOST#4232972]: 0 results, delay req 86400.00


I really dont want to upgrade, because the newer versions of Boinc have too long backoffs and as the ISPs on my city are rather bad it makes almost impossible to keep the hosts feeded... (It is worst for other projects but it happens even with Einstein that has really good download rates)...

I dont have any ATI GPU, so there is no need for the OpenCL support... Is there any other good reason to upgrade Boinc, besides the long delays between requests?

(By the way, AFAIK, 7.0.27 is still beta, requiring it as minimun version is a bit annoying... It is supossed that you have to upgrade it as soon as there is a new release... forcing every donor to do that is a bit too much...)

Richard Haselgrove
Richard Haselgrove
Joined: 10 Dec 05
Posts: 2142
Credit: 2801929892
RAC: 862436

Not sending work because client is outdated

On the other hand, I got

Quote:
2012-05-17 15:32:28.7011 [PID=17251] Request: [USER#xxxxx] [HOST#3674581] [IP xxx.xxx.xxx.133] client 5.10.13
...
2012-05-17 15:32:28.7072 [PID=17251] [send] work_req_seconds: 278.62 secs
...
2012-05-17 15:32:28.7227 [PID=17251] [send] [HOST#3674581] Sending app_version hsgamma_FGRP1 2 23 ; 2.44 GFLOPS
2012-05-17 15:32:29.9553 [PID=17251] [send] est. duration for WU 123395878: unscaled 24595.34 scaled 27516.67
2012-05-17 15:32:29.9553 [PID=17251] [HOST#3674581] Sending [RESULT#288860872 LATeah2478A_96.0_27300_-1.9e-11_1] (est. dur. 27516.67 seconds)


which would imply that the v7.0.27 requirement doesn't apply in all cases. One would expect it to apply to ATI requests only, but yours says CPU. We might need to see a longer log extract to work out what's going on.

Gavin
Gavin
Joined: 21 Sep 10
Posts: 191
Credit: 40644321073
RAC: 4882

I have also been seeing this

I have also been seeing this behaviour over the 2 to 3 few days or so.
Here is the full message log from the last time one of my hosts contacted the server:

2012-05-17 16:45:12.8508 [PID=22609] Request: [USER#xxxxx] [HOST#4876288] [IP xxx.xxx.xxx.133] client 6.10.58
2012-05-17 16:45:12.8546 [PID=22609] [send] effective_ncpus 2 max_jobs_on_host_cpu 999999 max_jobs_on_host 999999
2012-05-17 16:45:12.8546 [PID=22609] [send] effective_ngpus 1 max_jobs_on_host_gpu 999999
2012-05-17 16:45:12.8546 [PID=22609] [send] Not using matchmaker scheduling; Not using EDF sim
2012-05-17 16:45:12.8546 [PID=22609] [send] CPU: req 66.87 sec, 0.00 instances; est delay 0.00
2012-05-17 16:45:12.8546 [PID=22609] [send] CUDA: req 0.00 sec, 0.00 instances; est delay 0.00
2012-05-17 16:45:12.8546 [PID=22609] [send] work_req_seconds: 66.87 secs
2012-05-17 16:45:12.8546 [PID=22609] [send] available disk 74.08 GB, work_buf_min 0
2012-05-17 16:45:12.8546 [PID=22609] [send] active_frac 0.999881 on_frac 0.959359 DCF 1.503522
2012-05-17 16:45:12.8595 [PID=22609] [send] [HOST#4876288] is reliable
2012-05-17 16:45:12.8596 [PID=22609] [send] set_trust: random choice for error rate 0.000010: yes
2012-05-17 16:45:12.8741 [PID=22609] [version] Checking plan class 'BRP4SSE'
2012-05-17 16:45:12.8746 [PID=22609] [version] reading plan classes from file '../plan_class_spec.xml'
2012-05-17 16:45:12.8746 [PID=22609] [version] parsed project prefs setting 'also_run_cpu' : true : 1.000000
2012-05-17 16:45:12.8747 [PID=22609] [version] project prefs setting 'also_run_cpu' (1.000000) prevents using plan class.
2012-05-17 16:45:12.8747 [PID=22609] outdated client version 61058 < min core version 70027
2012-05-17 16:45:12.8747 [PID=22609] [version] Checking plan class 'BRP4cuda32'
2012-05-17 16:45:12.8747 [PID=22609] [version] parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2012-05-17 16:45:12.8747 [PID=22609] [version] Don't need CUDA jobs, skipping version 125 for einsteinbinary_BRP4 (BRP4cuda32)
2012-05-17 16:45:12.8747 [PID=22609] [version] Checking plan class 'BRP4cuda32nv301'
2012-05-17 16:45:12.8747 [PID=22609] [version] parsed project prefs setting 'gpu_util_brp' : true : 1.000000
2012-05-17 16:45:12.8747 [PID=22609] [version] driver version required min: -30100, supplied: 28558
2012-05-17 16:45:12.8748 [PID=22609] [version] no app version available: APP#19 (einsteinbinary_BRP4) PLATFORM#2 (windows_intelx86) min_version 0
2012-05-17 16:45:12.8797 [PID=22609] [send] stopping work search - no locality app selected
2012-05-17 16:45:12.8798 [PID=22609] [send] stopping work search - no locality app selected
2012-05-17 16:45:12.8798 [PID=22609] [send] stopping work search - no locality app selected
2012-05-17 16:45:12.8798 [PID=22609] Not sending work because client is outdated
2012-05-17 16:45:12.8816 [PID=22609] [debug] [HOST#4876288] MSG(high) No work sent
2012-05-17 16:45:12.8816 [PID=22609] [debug] [HOST#4876288] MSG(high) No work is available for Gravitational Wave S6 GC search
2012-05-17 16:45:12.8816 [PID=22609] [debug] [HOST#4876288] MSG(high) No work is available for Gravitational Wave S6 LineVeto search
2012-05-17 16:45:12.8816 [PID=22609] [debug] [HOST#4876288] MSG(high) see scheduler log messages on http://einstein.phys.uwm.edu//host_sched_logs/4876/4876288
2012-05-17 16:45:12.8816 [PID=22609] [debug] [HOST#4876288] MSG(high) No work available for the applications you have selected. Please check your preferences on the web site.
2012-05-17 16:45:12.8816 [PID=22609] [debug] [HOST#4876288] MSG(high) (your BOINC client is old - please install current version)
2012-05-17 16:45:12.8817 [PID=22609] Sending reply to [HOST#4876288]: 0 results, delay req 86400.00
2012-05-17 16:45:12.8819 [PID=22609] Scheduler ran 0.037 seconds

I may be wrong, but from my observation these messages started to appear with the release of the AMD/ATI app. which shouldn't affect me as all my rigs have Nvidia cards.

All I can say, for sure, is that letting your work queue run down a bit then manually reporting the completed tasks gets you more work as would normally happen.

Also, from my observations, the 24 hour project backoff does not appear to happen until the host later (either automatically or manualy) contacts the server with 0 results to report as is shown in the above log.

Gavin.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4284
Credit: 245672554
RAC: 10327

This looks like a serious bug

This looks like a serious bug in the server code. Apparently after reading the plan class atiOpenCL (which isn't even logged) the minimum client version isn't reset. There is no Client version limit on CPU Apps.

This is a long weekend in Germany, and I'm not sure I'll come to fix it before Monday.

BM

BM

Horacio
Horacio
Joined: 3 Oct 11
Posts: 205
Credit: 80557243
RAC: 0

RE: On the other hand, I

Quote:

On the other hand, I got

Quote:
2012-05-17 15:32:28.7011 [PID=17251] Request: [USER#xxxxx] [HOST#3674581] [IP xxx.xxx.xxx.133] client 5.10.13
...
2012-05-17 15:32:28.7072 [PID=17251] [send] work_req_seconds: 278.62 secs
...
2012-05-17 15:32:28.7227 [PID=17251] [send] [HOST#3674581] Sending app_version hsgamma_FGRP1 2 23 ; 2.44 GFLOPS
2012-05-17 15:32:29.9553 [PID=17251] [send] est. duration for WU 123395878: unscaled 24595.34 scaled 27516.67
2012-05-17 15:32:29.9553 [PID=17251] [HOST#3674581] Sending [RESULT#288860872 LATeah2478A_96.0_27300_-1.9e-11_1] (est. dur. 27516.67 seconds)

which would imply that the v7.0.27 requirement doesn't apply in all cases. One would expect it to apply to ATI requests only, but yours says CPU. We might need to see a longer log extract to work out what's going on.

I dont have (and never had) an ATI GPU... The log was as the one Gavin posted, it asked just for CPU and not for GPU.

What Ive seen is that BOINC makes a couple of requests as ussual and then it just gets this long delay with that message. It happened on all my hosts and Ive not changed anything on them.
May be is as Gavin said that it happens when there is no tasks reported... I dont know...
And, yes, if I do a manual update it works even if Im still on delayed time...

EDIT: Thanks Bernd... If it is not intended, then there is not issue waiting until it can be fixed.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4284
Credit: 245672554
RAC: 10327

This is another oddity of the

This is another oddity of the "mixed (locality and non-locality) scheduling". If you happen to catch the locality scheduler first (which should happen in 60% of all cases) you are more likely to get CPU work.

BM

BM

inomyabcs
inomyabcs
Joined: 11 Feb 05
Posts: 6
Credit: 16488057
RAC: 0

I had this same problem and I

I had this same problem and I am not running any ATI GPUs at the moment. I did have the E@H preference for use an ATI GPU if available selected. I turned it off and was able to connect and download CPU/cuda GPU workunits again. Before, I had to manually update to receive workunits. This only happened when the ATI openCL update went live.

inomyabcs
inomyabcs
Joined: 11 Feb 05
Posts: 6
Credit: 16488057
RAC: 0

An update from my last

An update from my last message.

This only temporarily solved my problem. I am going to try to turn off the cuda GPU preference and see if that helps. I should know after my CPU run is done (5 hours from now) if it lets me download more CPU units. Sorry about any confusion. There is a server log returned in my event log for troubleshooting.

5/19/2012 10:19:19 AM | Einstein@Home | Requesting new tasks for NVIDIA
5/19/2012 10:19:19 AM | Einstein@Home | [http] HTTP_OP::init_post(): http://einstein.phys.uwm.edu/EinsteinAtHome_cgi/cgi
5/19/2012 10:19:19 AM | Einstein@Home | [http] HTTP_OP::libcurl_exec(): ca-bundle set
5/19/2012 10:19:19 AM | Einstein@Home | [http] [ID#1] Info: Connection #0 seems to be dead!
5/19/2012 10:19:19 AM | Einstein@Home | [http] [ID#1] Info: Closing connection #0
5/19/2012 10:19:19 AM | Einstein@Home | [http] [ID#1] Info: About to connect() to einstein.phys.uwm.edu port 80 (#0)
5/19/2012 10:19:19 AM | Einstein@Home | [http] [ID#1] Info: Trying 129.89.61.70...
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Info: Connected to einstein.phys.uwm.edu (129.89.61.70) port 80 (#0)
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: POST /EinsteinAtHome_cgi/cgi HTTP/1.1
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: User-Agent: BOINC client (windows_x86_64 7.0.25)
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: Host: einstein.phys.uwm.edu
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: Accept: */*
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: Accept-Encoding: deflate, gzip
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: Content-Type: application/x-www-form-urlencoded
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: Content-Length: 68924
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server: Expect: 100-continue
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Sent header to server:
5/19/2012 10:19:20 AM | Einstein@Home | [http] [ID#1] Received header from server: HTTP/1.1 100 Continue
5/19/2012 10:19:23 AM | Einstein@Home | [http] [ID#1] Received header from server: HTTP/1.1 200 OK
5/19/2012 10:19:23 AM | Einstein@Home | [http] [ID#1] Received header from server: Date: Sat, 19 May 2012 01:20:59 GMT
5/19/2012 10:19:23 AM | Einstein@Home | [http] [ID#1] Received header from server: Server: Apache/2.2.3 (CentOS)
5/19/2012 10:19:23 AM | Einstein@Home | [http] [ID#1] Received header from server: Transfer-Encoding: chunked
5/19/2012 10:19:23 AM | Einstein@Home | [http] [ID#1] Received header from server: Content-Type: text/xml
5/19/2012 10:19:23 AM | Einstein@Home | [http] [ID#1] Received header from server:
5/19/2012 10:19:24 AM | Einstein@Home | [http] [ID#1] Info: Connection #0 to host einstein.phys.uwm.edu left intact
5/19/2012 10:19:25 AM | Einstein@Home | Scheduler request completed: got 0 new tasks
5/19/2012 10:19:25 AM | Einstein@Home | No work sent
5/19/2012 10:19:25 AM | Einstein@Home | see scheduler log messages on http://einstein.phys.uwm.edu//host_sched_logs/4206/4206337
5/19/2012 10:19:25 AM | Einstein@Home | (your BOINC client is old - please install current version)

inomyabcs
inomyabcs
Joined: 11 Feb 05
Posts: 6
Credit: 16488057
RAC: 0

This is confirmed. The

This is confirmed. The workaround is to disable requesting GPU workunits until the scheduling issue is resolved.

Horacio
Horacio
Joined: 3 Oct 11
Posts: 205
Credit: 80557243
RAC: 0

RE: This is confirmed. The

Quote:
This is confirmed. The workaround is to disable requesting GPU workunits until the scheduling issue is resolved.

Or, just do a manual update if you see that you're running low on WUs cached.
Or also, disable GPU just for a while, and when the cache is filled for CPU enable it again.
Or, just set the chache a bit higher and let it run as is... In the worst case your CPUs will be idle until the delay is over...

Completely disabling GPU work will affect Einstein RAC, way more, than having idle CPU cores...

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4284
Credit: 245672554
RAC: 10327

Unlike I suspected there is

Unlike I suspected there is no (obvious bug in the server code. The only thing is that the log / notification message is somewhat misleading. If there is work for any application version which has a limit on the Core Client version that is not satisfied by your client, you will get this message. This is slightly stupid, because this check is made before checking whether your computer can actually run that type of work or even asked for it. But the 'outdated' Client is just the reason why you don't get that particular type of work, and certainly not the reason why you don't get any work at all.

BM

BM

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.