Optimized Apps for Projects

General News and Banter all users and guests can see this forum.
Forum rules
Welcome to The Scottish Boinc Team boards. See forum rules in pinned post. If you can't be bothered then try not to be too naughty as I have a delete button to press and a ban hammer to swing.
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#1 Re: Optimized Apps for Projects

Post by Bryan »

New optimized apps as of 1/13/19 for Acoustics - both Windows and Linux. 2X improvement over the apps the project gives.

The 1st post HERE gives the GitHub link and installation instructions.
Image
User avatar
Hal Bregg
Boinc Sergeant
Boinc Sergeant
Posts: 175
Joined: Thu Nov 08, 2018 1:22 pm
Location: Cumbria

#2 Re: Optimized Apps for Projects

Post by Hal Bregg »

Optimisiation for Acoustics@home works well.

Will we see more projects getting similair updates?
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#3 Re: Optimized Apps for Projects

Post by Bryan »

It happens sometimes. Usually it is one of the users that produces them and then after some period of time the project will pick those up as the standard apps.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#4 Re: Optimized Apps for Projects

Post by davidbam »

Thought I might try this on the 2990WX as the plan is to have that purring along for all sprints. Do I need all 5 of these or only the first 4?
  • acoustic_linux64_avx2_v10.tgz
    acoustic_linux64_avx_v10.tgz
    acoustic_linux64_sse2_v10.tgz
    acoustic_linux64_sse41_v10.tgz
  • acoustic_linux64_avx512_v10.tgz
I tried to work this out based on my flags (below) but still figured it was safer to ask :lol: :lol:

flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht sy
scall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nopl nonstop_tsc cpuid extd_apicid amd_dcm aperfmperf pn
i pclmulqdq monitor ssse3 fma cx16 sse4_1 sse4_2 movbe popcnt aes xsave avx f16c rdrand lahf_lm cmp_legacy svm extapic cr
8_legacy abm sse4a misalignsse 3dnowprefetch osvw skinit wdt tce topoext perfctr_core perfctr_nb bpext perfctr_llc mwaitx
cpb hw_pstate sme ssbd sev ibpb vmmcall fsgsbase bmi1 avx2 smep bmi2 rdseed adx smap clflushopt sha_ni xsaveopt xsavec x
getbv1 xsaves clzero irperf xsaveerptr arat npt lbrv svm_lock nrip_save tsc_scale vmcb_clean flushbyasid decodeassists pa
usefilter pfthreshold avic v_vmsave_vmload vgif overflow_recov succor smca
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#5 Re: Optimized Apps for Projects

Post by Bryan »

I played around with the apps tonight. Windows is still 4.5% faster than Linux but a huge improvement. On stock apps Win is 35% faster.

On the 2990WX, running Linux, the AVX2 app is 3X faster than the stock app. It is around 30% faster than SSE4.1 opti app on the 2990WX.
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#6 Re: Optimized Apps for Projects

Post by Bryan »

You only need the 1st one ... AVX2 for the 2990WX or your Intel machines that support AVX2.
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#7 Re: Optimized Apps for Projects

Post by Bryan »

Unpack the file and copy the 4 files (that were in the tarball) to your Acoustics project folder. STOP BOINC and then RESTART BOINC and you are good to go. Even if you are already running WU they will finish using the opti app.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#8 Re: Optimized Apps for Projects

Post by davidbam »

Glad I asked - many thanks
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#9 Re: Optimized Apps for Projects

Post by davidbam »

Run time is 25min / 1500secs (give or take a few secs) on 2990WX 3GHz for SSPEMDD v0.18 x86_64-pc-linux-gnu. Temperature stable (yes, I still watch this carefully)

56 pending but 8 computational errors at end of run. Might this be caused by too many jobs finishing at the same time?
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#10 Re: Optimized Apps for Projects

Post by Bryan »

davidBAM wrote: Sat Jan 26, 2019 5:20 am
56 pending but 8 computational errors at end of run. Might this be caused by too many jobs finishing at the same time?

Yep! I'm surprised there weren't more than that ... I had 36 my 1st load and then I spread them out and haven't seen anymore errors.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#11 Re: Optimized Apps for Projects

Post by davidbam »

They are a bit naughty with the deadlines they issue on new WU. I make it that some of them have passed the deadline before they get downloaded
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#12 Re: Optimized Apps for Projects

Post by Bryan »

Are you seeing some WU that are going into panic mode? I have that on a couple of machines but it will correct itself before long. When a new WU starts up BOINC thinks it will take 2 hours to complete and on those machines they take about 30 minutes.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#13 Re: Optimized Apps for Projects

Post by davidbam »

Not sure TBH, I wondered if it was just a way to get run priority from the boinc_client. Mainly I mentioned it to see if it affected bunkering for sprints
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#14 Re: Optimized Apps for Projects

Post by davidbam »

So I make that about 150K per day for a 2990WX / 3GHz / Linux on Acoustics
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#15 Re: Optimized Apps for Projects

Post by Bryan »

If you are using BoincTasks it shows the deadline. There are some projects that will throw out some short deadline WU ... WCG for example. If you are bunkering WCG you need to make sure that all WU you are going to cache have a long enough deadline otherwise they time out while you have the lid on it and the machine becomes "untrusted". When you are ready go to "no new work" and run those shorties and turn them in. They you can put the lid on it.

I didn't see any short Acoustics WU on my machines. What is happening is BOINC doesn't really know how long it will take to finish a WU so it makes a guess. As you complete more and more WU it will figure it out and adjust itself and you will stop going into high-priority mode.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#16 Re: Optimized Apps for Projects

Post by davidbam »

I'll let it finish the WCG backlog and then try a 4 day bunker for Acoustics
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#17 Re: Optimized Apps for Projects

Post by Bryan »

Go ahead and load up the machine with Acoustics WU. It will give you 1000 and they have a 7 day deadline. That way they will be aged and start validating almost immediately when you get around to them. Right now I'm only at a 41% validation rate. During the last challenge I found that it takes 24-36 hour before pendings stabilize.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#18 Re: Optimized Apps for Projects

Post by davidbam »

davidBAM wrote: Sat Jan 26, 2019 5:23 am They are a bit naughty with the deadlines they issue on new WU. I make it that some of them have passed the deadline before they get downloaded
Total nonsense. I got the date wrong by a week. In mitigation, it was 5am when I formed that opinion :oops:
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Hal Bregg
Boinc Sergeant
Boinc Sergeant
Posts: 175
Joined: Thu Nov 08, 2018 1:22 pm
Location: Cumbria

#19 Re: Optimized Apps for Projects

Post by Hal Bregg »

What about tweaking amount of WU stored? I read somewhere that if you set those values too hight you might end up in WU not finishing on time.

Speaking of optimalization I went for fastets one based results.
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#20 Re: Optimized Apps for Projects

Post by Bryan »

Unless you are bunkered (network communication disabled) then it isn't a problem. If the project sees a WU is past the deadline but it is running on the computer it is okay. Those that go past the deadline and haven't started will get cancelled and removed from your computer by the project.

If you are bunkered and the WU goes beyond the deadline it may or may not get counted and give you credit. When a WU goes past the deadline the project will reissue that to someone else. Most (but not all) projects will give you credit for a late WU if it is turned in before the resend is sent back. There are some projects that will not give you any credit if you are over the deadline.

On projects like Acoustics that have a huge amount of "pendings" it is best to load up as many WU as you can. That way when you start crunching a WU it has "aged" and will and validate sooner.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#21 Re: Optimized Apps for Projects

Post by davidbam »

Bryan wrote: Sat Jan 26, 2019 4:36 am You only need the 1st one ... AVX2 for the 2990WX or your Intel machines that support AVX2.
Do you happen to know if these apps will work on a FreeBSD machine? I have a 44 thread Xeon e5 v4 which definitely has AVX2
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#22 Re: Optimized Apps for Projects

Post by davidbam »

Looks like it does - there is an ABI in place for Linux compatibility for other projects. I will report once I get some validated WU
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#23 Re: Optimized Apps for Projects

Post by davidbam »

E5-2690 v2 Xeons don't have AVX2, is AVX the next-best optimisation to use please? Flags at end of lscpu output

Code: Select all

$ lscpu
Architecture:        x86_64
CPU op-mode(s):      32-bit, 64-bit
Byte Order:          Little Endian
CPU(s):              40
On-line CPU(s) list: 0-39
Thread(s) per core:  2
Core(s) per socket:  10
Socket(s):           2
NUMA node(s):        2
Vendor ID:           GenuineIntel
CPU family:          6
Model:               62
Model name:          Intel(R) Xeon(R) CPU E5-2690 v2 @ 3.00GHz
Stepping:            4
CPU MHz:             3292.020
CPU max MHz:         3600.0000
CPU min MHz:         1200.0000
BogoMIPS:            5985.49
Virtualisation:      VT-x
L1d cache:           32K
L1i cache:           32K
L2 cache:            256K
L3 cache:            25600K
NUMA node0 CPU(s):   0-9,20-29
NUMA node1 CPU(s):   10-19,30-39
Flags:               fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid dca sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#24 Re: Optimized Apps for Projects

Post by Bryan »

Theoretically it should be. However, if the 2690 downclocks when running AVX then the SSE3 might actually be faster.
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#25 Re: Optimized Apps for Projects

Post by davidbam »

2264 secs on average for first 20 WU with AVX. None have yet validated
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
davidbam
General Bitchin'
General Bitchin'
Posts: 6371
Joined: Wed Aug 15, 2018 1:15 pm
Location: Huntly, Scotland
Contact:

#26 Re: Optimized Apps for Projects

Post by davidbam »

Fairly hot for a Xeon but can't say I've ever checked it when running WCG. The temperature difference between the two CPUs is quite normal in a Dell as the exhaust from first CPU fan is intake for the second

Code: Select all

Package id 0:  +73.0°C  (high = +81.0°C, crit = +95.0°C)
Core 0:        +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 1:        +69.0°C  (high = +81.0°C, crit = +95.0°C)
Core 2:        +71.0°C  (high = +81.0°C, crit = +95.0°C)
Core 3:        +71.0°C  (high = +81.0°C, crit = +95.0°C)
Core 4:        +73.0°C  (high = +81.0°C, crit = +95.0°C)
Core 8:        +69.0°C  (high = +81.0°C, crit = +95.0°C)
Core 9:        +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 10:       +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 11:       +70.0°C  (high = +81.0°C, crit = +95.0°C)
Core 12:       +69.0°C  (high = +81.0°C, crit = +95.0°C)

Package id 1:  +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 0:        +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 1:        +78.0°C  (high = +81.0°C, crit = +95.0°C)
Core 2:        +78.0°C  (high = +81.0°C, crit = +95.0°C)
Core 3:        +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 4:        +76.0°C  (high = +81.0°C, crit = +95.0°C)
Core 8:        +75.0°C  (high = +81.0°C, crit = +95.0°C)
Core 9:        +80.0°C  (high = +81.0°C, crit = +95.0°C)
Core 10:       +79.0°C  (high = +81.0°C, crit = +95.0°C)
Core 11:       +81.0°C  (high = +81.0°C, crit = +95.0°C)
Core 12:       +78.0°C  (high = +81.0°C, crit = +95.0°C)
I think this is fool-proof but could you just try it for me please? • There are 10 types of people in the world; those who understand binary, and those who don’t
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#27 Re: Optimized Apps for Projects

Post by Bryan »

The V3 and V4s have AVX2 and they down clock when running AVX2 due to the temperature rise. The V3s have a bug that causes all cores to down clock if any of them are on AVX2. The V4s will only down clock the cores that are running AVX2.
Image
Post Reply Previous topicNext topic

Return to “General”