Message boards :
Number crunching :
How do I get YAFU to stop sending me tasks that use up 100% of my cores?
Message board moderation
Author | Message |
---|---|
NullNVoid Send message Joined: 26 May 17 Posts: 5 Credit: 8,540,565 RAC: 0 |
YAFU occasionally sends tasks from the 16-core app that wants to use all 28 cores I have available in my Computing Preferences. Is there a setting I can use to stop this from happening? This results in my computer running a single 4-core task for an entire day (while there's a 28-core in queue) when it could be running two tasks (a 16t and a 4t, or a 16t and an 8t, etc). Just want to make sure I'm utilizing my cores efficiently. The 28-core tasks it's sending me would be okay if that were all it sent me. I suppose I could just remove 4 and 8 thread tasks on my YAFU settings but I feel like that would hurt my RAC. Any advice? |
yoyo_rkn Volunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 22 Aug 11 Posts: 736 Credit: 17,612,101 RAC: 51 |
Hello, 4t and 8t tasks uses 4 and 8 cores. But all otheres use as much cores as available. You can select only the 16t tasks in your settings and all cores are used. I would appreciate if more users run the 16t. yoyo |
NullNVoid Send message Joined: 26 May 17 Posts: 5 Credit: 8,540,565 RAC: 0 |
Thanks - that's what I've landed on. Running solely 16t tasks now and everything is running much more efficiently with all cores utilized. |
NullNVoid Send message Joined: 26 May 17 Posts: 5 Credit: 8,540,565 RAC: 0 |
@yoyo_rkn I'm having something peculiar going on. Wondering if you have any insight. I changed my YAFU settings to only allow 16t tasks (which is working out great, btw). I have two computers that have the same Ryzen 1800x processor and here's what's happening: Computer A: Receives 16t tasks that specify that they will use all available cores. So for example, it might be a 16t task but it shows that it will use 24 cores. Computer B: Receives 16t tasks, but they do not request to use all available cores. Instead of specifying that it'll use 24 cores, it presumably is just using 16 cores. The difference is that Computer A seems to get more than 6x the work done (judging by Cobblestones) than Computer B. Is there a certain setting I need to change on Computer B so that it receives tasks the same way as Computer A? EDIT: In case you need to look at my computers: Computer A is #18327 Computer B is #18065 The only major differences between the two computers is that Computer A has far more disk space available (~400GB available as opposed to just 40GB on Computer B). And Computer A is also running Windows 10 Professional as opposed to Windows 10 Home. |
NullNVoid Send message Joined: 26 May 17 Posts: 5 Credit: 8,540,565 RAC: 0 |
@yoyo --- you can disregard my previous message. I had forgotten that I had an app_config file on the computer that was specifying how many cores I wanted the 16t tasks to use. Deleted it and all is fixed. 16t tasks requesting all cores available. |
bcavnaugh Send message Joined: 6 Jan 17 Posts: 4 Credit: 2,473,394 RAC: 0 |
While the below works I still can only get 2 Tasks to run even through I have 56 Threads run between to CPU's <app_config> <app> <name>yafu-8t</name> <max_concurrent>4</max_concurrent> <fraction_done_exact/> </app> <app_version> <app_name>yafu-8t</app_name> <plan_class>8t</plan_class> <cmdline>-t 12</cmdline> <avg_ncpus>12</avg_ncpus> </app_version> </app_config> Crunching@EVGA The Number One Team in the BOINC Community. Folding@EVGA The Number One Team in the Folding@Home Community. |
yoyo_rkn Volunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 22 Aug 11 Posts: 736 Credit: 17,612,101 RAC: 51 |
The server limits number of running tasks per host to 2. |
Beyond Send message Joined: 4 Oct 14 Posts: 36 Credit: 148,742,365 RAC: 91,832 |
Hello, Hi yoyo! Is there a post somewhere that explains how this all works in excruciating, painstaking detail for those of us who are mentally challenged? Best Regards/Beyond |
yoyo_rkn Volunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 22 Aug 11 Posts: 736 Credit: 17,612,101 RAC: 51 |
4t and 8t app uses exactly 4 or 8 cores. All other uses all available cores. |
Beyond Send message Joined: 4 Oct 14 Posts: 36 Credit: 148,742,365 RAC: 91,832 |
Thanks. Question, if I have a machine that allows 15 cores to be used, can it get 16t and 32t tasks? So far haven't seen any. |
yoyo_rkn Volunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 22 Aug 11 Posts: 736 Credit: 17,612,101 RAC: 51 |
No, with 15 cores you will not get 16t or 32t tasks by default. Some user play with app_config.xml and configured a app which runs with 15 or 3 or whatever amount of cores. But I'm not familiar how to do this. |
Beyond Send message Joined: 4 Oct 14 Posts: 36 Credit: 148,742,365 RAC: 91,832 |
Thanks again. Trying some BOINC configs to try to allow 16t... |
Skillz Send message Joined: 28 Jul 17 Posts: 1 Credit: 28,039,228 RAC: 304 |
Hello, The problem with running the 16t and 32t applications is they're a waste of resources. They always get hung up and end up just canceling out. Plus the fact that a 16t (16 thread?) process will utilize all 64 cores of a single rig and take 24+ hours to complete sometimes is a complete waste. For example. One of my 64 core rigs is running a 32t process right now. It's been running for around 22 hours, but it's only utilizing 7 cores of my system right now. This means this process is letting 57 cores on my rig sit idle, but since BOINC thinks the process is using all 64 of them it wont assign those cores to anything else. Complete waste. This is why I reverted back to the 4t and 8t processes. At least they run no more than what they claim allowing BOINC to assign those idle cores to different projects. |
marmot Send message Joined: 5 Nov 15 Posts: 33 Credit: 53,531,496 RAC: 0 |
If you watch those tasks, you will see they do the same thing. The GNFS or ECM sub tasks exit one by one as they finish their data set and eventually the 4t or 8t will also only use 1 core for some minutes. The 32 and 16t are just more noticeable as the data sets are larger and they count down from larger thread counts of 16 or 32 threads to finally reach 1 thread then a restart with a new subset. You can set the thread count on a 16t to 16 threads and max_concurrent to 1 in App_config.xml and assure that 48 cores are left for other projects. |
Aurum Send message Joined: 26 Dec 18 Posts: 9 Credit: 36,482,872 RAC: 0 |
I think your app_config.xml should be more like: <app_config> <app> <name>yafu-8t</name> <max_concurrent>4</max_concurrent> <fraction_done_exact/> </app> <app_version> <app_name>yafu-8t</app_name> <plan_class>mt</plan_class> <cmdline>--nthreads 12</cmdline> <avg_ncpus>12</avg_ncpus> </app_version> </app_config> I suspect <avg_ncpus>12</avg_ncpus> is not needed if <cmdline>--nthreads 12</cmdline> is issued. Edit: Tried it sans <avg_ncpus> and Yafu wants every thread I've got. Need to include it. See: https://boinc.berkeley.edu/trac/wiki/AppPlanSpec and https://boinc.berkeley.edu/trac/wiki/AppPlan#Predefinedplanclasses |
Aurum Send message Joined: 26 Dec 18 Posts: 9 Credit: 36,482,872 RAC: 0 |
Hi YoYo, When you define your own class plans (<plan_class>16t</plan_class> & <plan_class>32t</plan_class>) don't you have to pass the client your plan_class_spec.xml file??? https://boinc.berkeley.edu/trac/wiki/AppPlanSpec This app_config.xml is not working to control number of threads used: <app> <name>yafu-16t</name> </app> <app_version> <app_name>yafu-16t</app_name> <plan_class>16t</plan_class> <cmdline>--nthreads 16</cmdline> <nthreads_cmdline>1</nthreads_cmdline> </app_version> <app> <name>yafu-32t</name> </app> <app_version> <app_name>yafu-32t</app_name> <plan_class>32t</plan_class> <cmdline>--nthreads 32</cmdline> <nthreads_cmdline>1</nthreads_cmdline> </app_version>My client_state.xml file says: <app_version> <app_name>yafu-32t</app_name> <version_num>13401</version_num> <platform>x86_64-pc-linux-gnu</platform> <avg_ncpus>40.000000</avg_ncpus> <flops>122619847136.720169</flops> <plan_class>32t</plan_class> <api_version>7.1.0</api_version> <cmdline>--nthreads 32</cmdline>And 32t still takes all 40 CPU threads. https://boinc.berkeley.edu/trac/wiki/AppPlan#Predefinedplanclasses BTW, can't see the check marks in my Yafu Preferences Run only the selected applications. |
yoyo_rkn Volunteer moderator Project administrator Project developer Project tester Volunteer developer Volunteer tester Project scientist Send message Joined: 22 Aug 11 Posts: 736 Credit: 17,612,101 RAC: 51 |
plan_class_spec is a server thing. This is the decission of the server which app with wich parameter to send. The 16t and 32t app use at least 16 / 32 threads but up to as many as you have cores. At least if you do not fiddle with app_config.xml. |
mikey Send message Joined: 13 Apr 17 Posts: 16 Credit: 14,952,152 RAC: 740 |
Hi YoYo, When you define your own class plans (<plan_class>16t</plan_class> & <plan_class>32t</plan_class>) don't you have to pass the client your plan_class_spec.xml file??? This is the app_config.xml file I use at another project to run 1 workunit at a time and use 4 cpu cores per workunit. It is a 6 core machine so only 1 unit can run at a time. Obviously you will need to change the "llrsr5" line to the app used here and put it in the project folder for Yafu. <app_config> <app> <name>llrSR5</name> <fraction_done_exact/> <max_concurrent>1</max_concurrent> </app> <app_version> <app_name>llrSR5</app_name> <cmdline>-t 6</cmdline> <avg_ncpus>4</avg_ncpus> </app_version> </app_config> |