How do I get YAFU to stop sending me tasks that use up 100% of my cores?

Message boards : Number crunching : How do I get YAFU to stop sending me tasks that use up 100% of my cores?
Message board moderation

To post messages, you must log in.

AuthorMessage
NullNVoid

Send message
Joined: 26 May 17
Posts: 5
Credit: 8,540,565
RAC: 0
United States
Message 1022 - Posted: 14 Jul 2017, 16:03:23 UTC
Last modified: 14 Jul 2017, 16:03:58 UTC

YAFU occasionally sends tasks from the 16-core app that wants to use all 28 cores I have available in my Computing Preferences.

Is there a setting I can use to stop this from happening? This results in my computer running a single 4-core task for an entire day (while there's a 28-core in queue) when it could be running two tasks (a 16t and a 4t, or a 16t and an 8t, etc).

Just want to make sure I'm utilizing my cores efficiently. The 28-core tasks it's sending me would be okay if that were all it sent me. I suppose I could just remove 4 and 8 thread tasks on my YAFU settings but I feel like that would hurt my RAC.

Any advice?
ID: 1022 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 647
Credit: 6,554,178
RAC: 8,845
Germany
Message 1023 - Posted: 16 Jul 2017, 10:58:21 UTC - in response to Message 1022.  

Hello,

4t and 8t tasks uses 4 and 8 cores. But all otheres use as much cores as available. You can select only the 16t tasks in your settings and all cores are used. I would appreciate if more users run the 16t.

yoyo
ID: 1023 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
NullNVoid

Send message
Joined: 26 May 17
Posts: 5
Credit: 8,540,565
RAC: 0
United States
Message 1025 - Posted: 17 Jul 2017, 14:11:41 UTC - in response to Message 1023.  

Thanks - that's what I've landed on. Running solely 16t tasks now and everything is running much more efficiently with all cores utilized.
ID: 1025 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
NullNVoid

Send message
Joined: 26 May 17
Posts: 5
Credit: 8,540,565
RAC: 0
United States
Message 1027 - Posted: 18 Jul 2017, 17:35:07 UTC
Last modified: 18 Jul 2017, 17:43:50 UTC

@yoyo_rkn

I'm having something peculiar going on. Wondering if you have any insight. I changed my YAFU settings to only allow 16t tasks (which is working out great, btw).

I have two computers that have the same Ryzen 1800x processor and here's what's happening:

Computer A: Receives 16t tasks that specify that they will use all available cores. So for example, it might be a 16t task but it shows that it will use 24 cores.

Computer B: Receives 16t tasks, but they do not request to use all available cores. Instead of specifying that it'll use 24 cores, it presumably is just using 16 cores.

The difference is that Computer A seems to get more than 6x the work done (judging by Cobblestones) than Computer B. Is there a certain setting I need to change on Computer B so that it receives tasks the same way as Computer A?

EDIT: In case you need to look at my computers:

Computer A is #18327
Computer B is #18065

The only major differences between the two computers is that Computer A has far more disk space available (~400GB available as opposed to just 40GB on Computer B). And Computer A is also running Windows 10 Professional as opposed to Windows 10 Home.
ID: 1027 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
NullNVoid

Send message
Joined: 26 May 17
Posts: 5
Credit: 8,540,565
RAC: 0
United States
Message 1028 - Posted: 18 Jul 2017, 23:44:27 UTC

@yoyo --- you can disregard my previous message. I had forgotten that I had an app_config file on the computer that was specifying how many cores I wanted the 16t tasks to use.

Deleted it and all is fixed. 16t tasks requesting all cores available.
ID: 1028 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile bcavnaugh
Avatar

Send message
Joined: 6 Jan 17
Posts: 4
Credit: 1,223,498
RAC: 0
United States
Message 1101 - Posted: 28 Jan 2018, 22:55:19 UTC

While the below works I still can only get 2 Tasks to run even through I have 56 Threads run between to CPU's
<app_config>
<app>
<name>yafu-8t</name>
<max_concurrent>4</max_concurrent>
<fraction_done_exact/>
</app>
<app_version>
<app_name>yafu-8t</app_name>
<plan_class>8t</plan_class>
<cmdline>-t 12</cmdline>
<avg_ncpus>12</avg_ncpus>
</app_version>
</app_config>

Crunching@EVGA The Number One Team in the BOINC Community. Folding@EVGA The Number One Team in the Folding@Home Community.
ID: 1101 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 647
Credit: 6,554,178
RAC: 8,845
Germany
Message 1103 - Posted: 3 Feb 2018, 21:37:25 UTC - in response to Message 1101.  

The server limits number of running tasks per host to 2.
ID: 1103 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond

Send message
Joined: 4 Oct 14
Posts: 35
Credit: 48,525,087
RAC: 107,336
United States
Message 1114 - Posted: 13 Feb 2018, 23:29:37 UTC - in response to Message 1023.  

Hello,

4t and 8t tasks uses 4 and 8 cores. But all otheres use as much cores as available. You can select only the 16t tasks in your settings and all cores are used. I would appreciate if more users run the 16t.

yoyo

Hi yoyo! Is there a post somewhere that explains how this all works in excruciating, painstaking detail for those of us who are mentally challenged?
Best Regards/Beyond
ID: 1114 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 647
Credit: 6,554,178
RAC: 8,845
Germany
Message 1116 - Posted: 14 Feb 2018, 17:07:11 UTC - in response to Message 1114.  

4t and 8t app uses exactly 4 or 8 cores. All other uses all available cores.
ID: 1116 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond

Send message
Joined: 4 Oct 14
Posts: 35
Credit: 48,525,087
RAC: 107,336
United States
Message 1117 - Posted: 15 Feb 2018, 1:00:40 UTC - in response to Message 1116.  

Thanks. Question, if I have a machine that allows 15 cores to be used, can it get 16t and 32t tasks? So far haven't seen any.
ID: 1117 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 647
Credit: 6,554,178
RAC: 8,845
Germany
Message 1118 - Posted: 15 Feb 2018, 6:18:47 UTC - in response to Message 1117.  

No, with 15 cores you will not get 16t or 32t tasks by default.
Some user play with app_config.xml and configured a app which runs with 15 or 3 or whatever amount of cores.
But I'm not familiar how to do this.
ID: 1118 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile Beyond

Send message
Joined: 4 Oct 14
Posts: 35
Credit: 48,525,087
RAC: 107,336
United States
Message 1119 - Posted: 15 Feb 2018, 8:53:07 UTC - in response to Message 1118.  

Thanks again. Trying some BOINC configs to try to allow 16t...
ID: 1119 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
[H]Skillz

Send message
Joined: 28 Jul 17
Posts: 1
Credit: 1,086,424
RAC: 0
Message 1164 - Posted: 4 Mar 2018, 2:37:17 UTC - in response to Message 1023.  

Hello,

4t and 8t tasks uses 4 and 8 cores. But all otheres use as much cores as available. You can select only the 16t tasks in your settings and all cores are used. I would appreciate if more users run the 16t.

yoyo


The problem with running the 16t and 32t applications is they're a waste of resources. They always get hung up and end up just canceling out. Plus the fact that a 16t (16 thread?) process will utilize all 64 cores of a single rig and take 24+ hours to complete sometimes is a complete waste.

For example. One of my 64 core rigs is running a 32t process right now. It's been running for around 22 hours, but it's only utilizing 7 cores of my system right now. This means this process is letting 57 cores on my rig sit idle, but since BOINC thinks the process is using all 64 of them it wont assign those cores to anything else. Complete waste.

This is why I reverted back to the 4t and 8t processes. At least they run no more than what they claim allowing BOINC to assign those idle cores to different projects.
ID: 1164 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
marmot

Send message
Joined: 5 Nov 15
Posts: 28
Credit: 53,517,297
RAC: 0
United States
Message 1202 - Posted: 20 Apr 2018, 3:45:11 UTC - in response to Message 1164.  


This is why I reverted back to the 4t and 8t processes. At least they run no more than what they claim allowing BOINC to assign those idle cores to different projects.


If you watch those tasks, you will see they do the same thing. The GNFS or ECM sub tasks exit one by one as they finish their data set and eventually the 4t or 8t will also only use 1 core for some minutes. The 32 and 16t are just more noticeable as the data sets are larger and they count down from larger thread counts of 16 or 32 threads to finally reach 1 thread then a restart with a new subset.

You can set the thread count on a 16t to 16 threads and max_concurrent to 1 in App_config.xml and assure that 48 cores are left for other projects.
ID: 1202 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum

Send message
Joined: 26 Dec 18
Posts: 7
Credit: 2,269,234
RAC: 0
United States
Message 1343 - Posted: 26 Dec 2018, 23:12:55 UTC - in response to Message 1101.  
Last modified: 26 Dec 2018, 23:43:38 UTC

I think your app_config.xml should be more like:
<app_config>
<app>
<name>yafu-8t</name>
<max_concurrent>4</max_concurrent>
<fraction_done_exact/>
</app>
<app_version>
<app_name>yafu-8t</app_name>
<plan_class>mt</plan_class>
<cmdline>--nthreads 12</cmdline>
<avg_ncpus>12</avg_ncpus>
</app_version>
</app_config>

I suspect <avg_ncpus>12</avg_ncpus> is not needed if <cmdline>--nthreads 12</cmdline> is issued.
Edit: Tried it sans <avg_ncpus> and Yafu wants every thread I've got. Need to include it.

See: https://boinc.berkeley.edu/trac/wiki/AppPlanSpec and https://boinc.berkeley.edu/trac/wiki/AppPlan#Predefinedplanclasses
ID: 1343 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Aurum

Send message
Joined: 26 Dec 18
Posts: 7
Credit: 2,269,234
RAC: 0
United States
Message 1344 - Posted: 27 Dec 2018, 2:12:22 UTC - in response to Message 1116.  

Hi YoYo, When you define your own class plans (<plan_class>16t</plan_class> & <plan_class>32t</plan_class>) don't you have to pass the client your plan_class_spec.xml file???
https://boinc.berkeley.edu/trac/wiki/AppPlanSpec

This app_config.xml is not working to control number of threads used:
   <app>
      <name>yafu-16t</name>
   </app>
   <app_version>
      <app_name>yafu-16t</app_name>
      <plan_class>16t</plan_class>
      <cmdline>--nthreads 16</cmdline>
      <nthreads_cmdline>1</nthreads_cmdline>
   </app_version>
   <app>
      <name>yafu-32t</name>
   </app>
   <app_version>
      <app_name>yafu-32t</app_name>
      <plan_class>32t</plan_class>
      <cmdline>--nthreads 32</cmdline>
      <nthreads_cmdline>1</nthreads_cmdline>
   </app_version>
My client_state.xml file says:
<app_version>
    <app_name>yafu-32t</app_name>
    <version_num>13401</version_num>
    <platform>x86_64-pc-linux-gnu</platform>
    <avg_ncpus>40.000000</avg_ncpus>
    <flops>122619847136.720169</flops>
    <plan_class>32t</plan_class>
    <api_version>7.1.0</api_version>
    <cmdline>--nthreads 32</cmdline>
And 32t still takes all 40 CPU threads.
https://boinc.berkeley.edu/trac/wiki/AppPlan#Predefinedplanclasses

BTW, can't see the check marks in my Yafu Preferences Run only the selected applications.
ID: 1344 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 647
Credit: 6,554,178
RAC: 8,845
Germany
Message 1345 - Posted: 27 Dec 2018, 9:32:24 UTC - in response to Message 1344.  
Last modified: 27 Dec 2018, 9:32:48 UTC

plan_class_spec is a server thing. This is the decission of the server which app with wich parameter to send. The 16t and 32t app use at least 16 / 32 threads but up to as many as you have cores. At least if you do not fiddle with app_config.xml.
ID: 1345 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
mikey

Send message
Joined: 13 Apr 17
Posts: 10
Credit: 5,855,584
RAC: 36,129
United States
Message 1405 - Posted: 17 Jul 2019, 22:16:46 UTC - in response to Message 1344.  

Hi YoYo, When you define your own class plans (<plan_class>16t</plan_class> & <plan_class>32t</plan_class>) don't you have to pass the client your plan_class_spec.xml file???
https://boinc.berkeley.edu/trac/wiki/AppPlanSpec

This app_config.xml is not working to control number of threads used:
[code] <app>
<name>yafu-16t</name>
</app>
<app_version>
<app_name>yafu-16t</app_name>
<plan_class>16t</plan_class>
<cmdline>--nthreads 16</cmdline>
<nthreads_cmdline>1</nthreads_cmdline>
</app_version>
<app>
<name>yafu-32t</name>
</app>


BTW, can't see the check marks in my Yafu Preferences Run only the selected applications.


This is the app_config.xml file I use at another project to run 1 workunit at a time and use 4 cpu cores per workunit.
It is a 6 core machine so only 1 unit can run at a time.

Obviously you will need to change the "llrsr5" line to the app used here and put it in the project folder for Yafu.

<app_config>
<app>
<name>llrSR5</name>
<fraction_done_exact/>
<max_concurrent>1</max_concurrent>
</app>
<app_version>
<app_name>llrSR5</app_name>
<cmdline>-t 6</cmdline>
<avg_ncpus>4</avg_ncpus>
</app_version>
</app_config>
ID: 1405 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : How do I get YAFU to stop sending me tasks that use up 100% of my cores?




Datenschutz / Privacy Copyright © 2011-2021 Rechenkraft.net e.V. & yoyo