CPU usage jumps // 16t WUs use 64cores but only 2% CPU load

Message boards : Number crunching : CPU usage jumps // 16t WUs use 64cores but only 2% CPU load
Message board moderation

To post messages, you must log in.

AuthorMessage
Tobias Becke [Gridcoin]

Send message
Joined: 17 Sep 15
Posts: 2
Credit: 1,808,298
RAC: 0
Message 1034 - Posted: 20 Jul 2017, 19:08:04 UTC

Hello Yafu community,

I have an 80 thread Xeon server. I wanted to use all threads on yafu so I set CPU usage and CPU time usage to 100% and yafu as only project. The result: a maximum of 64 threads is used on one single WU. The CPU usage jumps between ~10 and 90%.

To compensate this, I have set <ncpus>128</ncpus> so 2 tasks with each 64 threads started running, using more or less 100% CPU, still with small jumps. After some time, the CPU load went to about 60%. Now, one task still uses 50%+, the other is "stuck" at 2% CPU load. The time to finish is decreasing in both asks, but in one it is decreasing very slowly.

Can I somehow improve the situation? I read yoyo is prefering to run 16t tasks so I want to stick to that, but if the CPU is not maxed out I guess I crunch sub-optimal.

Kind Regards
ID: 1034 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 732
Credit: 17,121,463
RAC: 15,251
Germany
Message 1035 - Posted: 21 Jul 2017, 6:20:12 UTC

Hello,

this is actually a good idea to set to a higher value. I do the same to run the 16t on a 8 core system.
Yafu runs in different stages to factor the composite.

One of the first stage is to try to find a factor with ecm. There one ecm process is forked per core, you will see many ecm processes.

If this doesn't factorize the composite it has to go the long way and do a complete factorization with gnfs, you will see many gnfs proecesses.
Afterwards it checks if it has enough relations found. If not yafu proceeds with gnfs. You will also see a big growing nfs.dat file in the slot folder.

At the end yafu consolidates the gnfs results and does a linear algebra. Unfortunately this stage is mostly single threaded.

yoyo
ID: 1035 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Tobias Becke [Gridcoin]

Send message
Joined: 17 Sep 15
Posts: 2
Credit: 1,808,298
RAC: 0
Message 1036 - Posted: 21 Jul 2017, 6:49:15 UTC

Thanks for the input.

I experienced if a second 64 thread task starts there are problems. It opens ecm processes but the processes seem to be shut down a few seconds after being opened. I suppose it can not find enough free cores to run it? If I got you right, the process will finish eventually, but will take more time with gnfs instead of ecm?

Kind Regards
ID: 1036 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote
Profile yoyo_rkn
Volunteer moderator
Project administrator
Project developer
Project tester
Volunteer developer
Volunteer tester
Project scientist

Send message
Joined: 22 Aug 11
Posts: 732
Credit: 17,121,463
RAC: 15,251
Germany
Message 1038 - Posted: 21 Jul 2017, 10:40:09 UTC

Check the factor.log file in the slot folder.
There is logged what is done.
ID: 1038 · Rating: 0 · rate: Rate + / Rate - Report as offensive     Reply Quote

Message boards : Number crunching : CPU usage jumps // 16t WUs use 64cores but only 2% CPU load




Datenschutz / Privacy Copyright © 2011-2024 Rechenkraft.net e.V. & yoyo