Competition FAQ and how to Bunker

For those wishin to entertain us with their competitive streaks
User avatar
Alez
[ TSBT's Pirate ]
[ TSBT's Pirate ]
Posts: 10363
Joined: Thu Oct 04, 2012 1:22 pm
Location: roaming the planet

#1 Competition FAQ and how to Bunker

Post by Alez »

A rough guide to competitions.

When a competition is set, an announcement will be made in this section. The team automatically signs up to the competition.
To take part all a team member has to do is crunch that project and all credits accumulated during the duration of the challenge will be added to both your individual and the team score.

At the top of each competition announcement will be a link to the Boincstat topic for the competition in question. There you will find the updated team stats and a shoutbox for banter between teams. There you will also a link to your individual stats within the competition.

At the end of the competition, the final team position and individual positions within the competition will be posted up here, along with the chat between as things progress.

It's that simple so why not join in and give it a go ?

You will also hear of the term bunkering. Bunkering is hoarding, caching, storing etc of units. Basically fill your cache up with units some time before the challenge ( how long depends on your cache size you set and what else you are running ) When your cache is full you then suspend network ( or block the ip of the project in question ) and crunch away. Hopefully by the start of the competition you have a bunch of units ready to upload , which you do after the comp starts. This gives you and the team a head start on the opposition. You then crunch away as normal on the competition. This is regarded as normal practice ( not cheating ) and is actively encouraged. Some people will bunker for a week before the challenge :shock:

A guide to bunkering is in the next post.
Image
The best form of help from above is a sniper on the rooftop....
User avatar
Alez
[ TSBT's Pirate ]
[ TSBT's Pirate ]
Posts: 10363
Joined: Thu Oct 04, 2012 1:22 pm
Location: roaming the planet

#2 Re: Competition FAQ and how to Bunker

Post by Alez »

How to Bunker for a competition.

This guide is hopelessly cribbed from the Seti.usa site with minimal changes as quite frankly I could not have written it better. The original can be found here http://www.setiusa.us/faq.php?faq=adv#faq_bunk101

There seems to be some confusion on how to bunker a project with some trying to limit % completion before suspending the unit or limiting amount of time before switching units. While this may work to some extent it is NOT the correct way to get it done unless you really like sitting in front of your computer :lol:

HOW TO MAXIMIZE YOUR CACHE

First set all projects to no new tasks in Boinc manager except the project you want to fill the cache with. After that under Tools - computing preferences - network usage. Set the min and additional work buffer values to get work for as long as you think you will require prior to the start of the competition. Update the project if it does not automatically start downloading units to fill the cache

Some projects will issue a maximum number of WUs to your machine regardless of how fast it is, how many cores it has, and how many GPUs it has. Other projects rely totally on the BOINC scheduler to determine how many wu you need for the MAX days of cache you have requested. The object here is to REALLY fill your cache so that you have enough wu to keep crunching until you are ready to dump.

If you see a message from the project to the effect "You have reached your limit" then there is nothing more you can do so go ahead and block the IP address and start crunching. Collatz, for example, used to issue something like 300 wu and there was nothing you could do to get more.

PROBLEM WITH THE BOINC SCHEDULER
On projects that rely on the BOINC scheduler for determing the correct cache size several things can limit what BOINC thinks you need. BOINC uses a variable called "Duration Correction Factor" to compute how many wu you need. If you crunch enough WU then this value stabilizes and is correct for your machine. There are several things that can screw up this value.

1. If you haven't crunched the project before -or- for a long time the project will download a starting value that is some pie-in-the-sky value of how long it will take your machine to crunch a wu. As you crunch and complete MANY wu this number will get modified until it eventual is right. BUT, at the initial download it probably isn't anywhere near right.

2. You have been crunching CPU tasks on a project and then you start crunching GPU tasks. BOINC will continue to use the "DCF" value from the CPU time until you crunch enough GPU tasks to lower the value.

GETTING ADDITIONAL WU There are 2 files that contain the "Duration Correction Factor" infomation and they are located in the BOINC directory: client_state and client_state_prev(ious). If you open one of these files, do a search on the project name (ie Rosetta), located in that section you will find Duration Correction Factor. It will usually be a number like 0.9 or 1.15 or whatever.

After you have set your cache to 10 days and the project has filled your cache you can try this:

1. Stop BOINC 2. Open both the client_state.xml and client_state_prev.xml files.
3. Search for the project name (use CTRL F to bring up text search)
4. Find Duration Correction Factor and change the number to something like 0.04
5. Save the files
6. Wait 10 minutes ... this is to make sure you don't request work to soon from the project which pisses it off :)
7. Restart BOINC
8. Check and see if BOINC asked the project for more work ... if not UPDATE the project

This will quite often make the project issue more work. Sometimes it works, sometimes it doesn't.

Now it is time to block the IP address and start crunching. When the time comes to dump your finished WU if you have more WU than you can finish before the deadline then ABORT those tasks immediately. If you wait for the BOINC scheduler to do it it will wait until the deadline passes and then abort them. If you abort immediately then you will load up NEW wu which will be "aged" by the time your machine gets around to crunching them. BTW, your DCF will climb and by the time you have done the dump the BOINC scheduler will know how many wu are correct for your machine.


BLOCK THE IP ADDRESS OF THE PROJECT
For example Block the Rosetta IP address on your FIREWALL after you've downloaded wu and filled your caches.

Go into your firewall and make a RULE that blocks outbound communications to Rosetta. Then when you are ready to dump just remove the rule, go to BOINC TRANSFERS and hit RETRY ALL and after they have uploaded update Rosetta.

The IP address for Rosetta's homepage is 128.95.160.140. Assuming a different IP is used for program communications you need to block the range 128.95.160.0 to 128.95.160.255.

After you have setup your rule you can check if it works by going to BOINC and try to UPDATE Rosetta. If the IP address and RULE are correct then in BOINC Messages you will see it try to connect but this message will appear:

17374 DistrRTgen 4/30/2012 12:07:51 AM Sending scheduler request: To fetch work.
17375 DistrRTgen 4/30/2012 12:07:51 AM Requesting new tasks for NVIDIA GPU
17376 DistrRTgen 4/30/2012 12:07:52 AM Scheduler request failed: Couldn't connect to server
17377 4/30/2012 12:07:56 AM Project communication failed: attempting access to reference site
17378 4/30/2012 12:07:58 AM Internet access OK - project servers may be temporarily down.
I obviously blocked DirT since I'm not running Rosetta

You can crunch to 100% because there will be NO leaks. This method will also allow you to download other projects ie GPU tasks on one project whilst bunkering cpu tasks on another or vica versa. It will also allow you to run another project ie WUProp whilst still bunkering.

This same technique can be used on your router and it only has to be done at one location. I haven't experiemented with that so I can't tell you exactly how to do it however.

HOW TO FIND A PROJECTS IP ADDRESS
In the BOINC directory open client_state and do a search on the projects name and it will usually take you to the projects master URL. A search on Rosetta takes you to:

http://boinc.bakerlab.org/rosetta/

Now open a Windows command prompt either from Accessories or just type CMD in the START search line and hit enter.

TYPE
nslookup boinc.bakerlab.org
Hit ENTER and the following IP address is returned

128.95.160.140

If you paste this IP into your browser and hit go it will take you to the Home Page of Rosetta.

This technique can be used to find any projects IP and if you block the range from 0 to 255 you can stop any communications from your BOINC Client to the project.



SETTING UP A RULE WITH NORTON FIREWALL

TRAFFIC RULES TAB

ADD
Block - NEXT
Connections to other computers - NEXT
Only the computers and sites listed below
ADD
Using a range
128.95.160.0
128.95.160.255
OK
NEXT
All types of communications - NEXT
NEXT
Name your rule ie: BLOCK ROSETTA - NEXT
FINISH

If done correctly your Browser will no long connect to the Rosetta homepage. If you update Rosetta in BOINC you will get the error message it can't connect.

TO REMOVE THE RULE

TRAFFIC RULES TAB
Highlight your rule - REMOVE
YES

Now go to BOINC, TRANSFERS, and select RETRY ALL and your completed wu will dump.

PS rather than REMOVING the RULE, you can just uncheck it to turn it off. That way if you want to use it in the future you can just "check" it to turn it back on.



SETTING UP A RULE WITH WINDOWS FIREWALL

CONTROL PANEL > WINDOWS FIREWALL > Advanced Settings
On the left side of the panel
OUTBOUND RULES
On right side of panel
NEW RULE
CUSTOM - NEXT
All Programs - NEXT
NEXT
Under "Which remote IP addresses does this rule apply to?"
THESE IP ADDRESSES - ADD
THIS IP RANGE
From 128.95.160.0
To 128.95.160.255
OK - NEXT
Block the connection - NEXT
check Domain, Private, Public - NEXT
Name your rule ie BLOCK ROSETTA - FINISH

You're done

To REMOVE the rule go to OUTBOUND RULES, highlight the rule, then on the right side click on DELETE and then YES.
Image
The best form of help from above is a sniper on the rooftop....
User avatar
Silver
Boinc Brigadier
Boinc Brigadier
Posts: 2210
Joined: Sun Apr 14, 2013 7:09 pm
Location: Aberdeen/Anguilla

#3 Re: Competition FAQ and how to Bunker

Post by Silver »

Thanks Alez!
I'm going to try the ip address block next time instead of just disabling Boincs network access and having to stockpile all the collatz units too.
Just goes to show you do indeed learn something everyday :)
Image
User avatar
scole of TSBT
Boinc Major General
Boinc Major General
Posts: 5980
Joined: Mon Feb 03, 2014 2:38 pm
Location: Goldsboro, (Eastern) North Carolina, USA

#4 Re: Competition FAQ and how to Bunker

Post by scole of TSBT »

After the Test4Theory challenge, I started thinking about using multiple clients to bunker. I usually can't get as many WUs as I'd like, so I'm going to bunker for the NSF challenge by first suspending network activity on the primary client and when it finishes all it's NSF WUs, start a second client running NSF. You can do this as many times as you want as long as the completed WUs don't expire. Anyone tried this before?
Image
User avatar
Alez
[ TSBT's Pirate ]
[ TSBT's Pirate ]
Posts: 10363
Joined: Thu Oct 04, 2012 1:22 pm
Location: roaming the planet

#5 Re: Competition FAQ and how to Bunker

Post by Alez »

Actually no, never considered that before. Let me know if it works, but try it before to see if you run into problems with computer id's on the project. Before T4T I had never considered running multiple clients on the same machine. If it works then should be an easier solution to bunker for days if you wanted :D
Image
The best form of help from above is a sniper on the rooftop....
User avatar
scole of TSBT
Boinc Major General
Boinc Major General
Posts: 5980
Joined: Mon Feb 03, 2014 2:38 pm
Location: Goldsboro, (Eastern) North Carolina, USA

#6 Re: Competition FAQ and how to Bunker

Post by scole of TSBT »

Something I found useful to bunker before a challenge is a spreadsheet to calculate how many WUs you need to download to a particular system. All you have to do enter hours until challenge starts, number of cores/gpus, tasks per cpu/GPU, seconds per task and it will calculate the number WUs you need to download to that system. Not rocket science, but it's here for everyone for quick access rather than having to do the math every time...

Bunker Calculator...
https://drive.google.com/file/d/0B7Q1OY ... sp=sharing
Image
User avatar
Alez
[ TSBT's Pirate ]
[ TSBT's Pirate ]
Posts: 10363
Joined: Thu Oct 04, 2012 1:22 pm
Location: roaming the planet

#7 Re: Competition FAQ and how to Bunker

Post by Alez »

You can use IPTables in Linux to block the IP. For example Rosetta on IP 128.95.160.140

Type this in cmd terminal to block the IP:
iptables -A INPUT -s 128.95.160.140 -j DROP

Then type this to remove it
iptables -D INPUT -s 128.95.160.140 -j DROP
Image
The best form of help from above is a sniper on the rooftop....
User avatar
scole of TSBT
Boinc Major General
Boinc Major General
Posts: 5980
Joined: Mon Feb 03, 2014 2:38 pm
Location: Goldsboro, (Eastern) North Carolina, USA

#8 Re: Competition FAQ and how to Bunker

Post by scole of TSBT »

If your system restarts, that iptables directive will go away. If you want to implement a persistent firewall, see this post...https://tsbt.co.uk/forum/viewtopic.php?f=139&t=9305
Image
User avatar
Bryan
Boinc Brigadier
Boinc Brigadier
Posts: 2621
Joined: Thu May 21, 2015 6:18 pm

#9 Re: Competition FAQ and how to Bunker

Post by Bryan »

Or if your router supports "Parental controls" just tell your router to block anything with the keyword rosetta.
Image
User avatar
scole of TSBT
Boinc Major General
Boinc Major General
Posts: 5980
Joined: Mon Feb 03, 2014 2:38 pm
Location: Goldsboro, (Eastern) North Carolina, USA

#10 Re: Competition FAQ and how to Bunker

Post by scole of TSBT »

Another narrative...

My current go to method to bunker is...
1. Determine best sub-project app/WUs to run, the highest paying per run sec. Set project default to only allow that app.
2. Determine the best OS to run that sub-project app under, Win64 or linux. Boot to that OS (if you have dual system)
3. Set <ncpus> to the correct number of threads you can run and set WUs cache to .01 days of works with 0 additional days.
4. Run a test set of WUs under full load to determine the avg run time for a WU. Not running a full load will give overstated performance results.
5. Use the challenge/bunker spreadsheet to calculate the number of WUs you need. Target number of WUs should cover the number of hours until the start of the challenge.
6. Anywhere from 1-12 hours after it's within the deadline of the WUs, set the <ncpus> parm in the cc_config.xml to high number, like 1000.
7. Set WUs cache to 10 days of works with 10 additional days.
8. Make sure you have an app_config.xml limiting the number of active WUs to the number of threads you wish, like so...
<app_config>
<project_max_concurrent>64</project_max_concurrent>
</app_config>
9. Reload/read the config files on that boinc client and watch the WUs download. You can set the client to never run if you want until the cache is loaded.
10. When you have the desired number of WUs loaded or the max the project will give, put a lid on it (set allow network communication to never), set allow to always and let 'er run.
11. After the challenge starts, set allow network communication to always and it's bombs away.
12. Allow your cache to reload with enough work until the end of the challenge. Either let it run open or put a lid on that bunker too.
13. If the project did not give you enough WUs, pause that client, start a another client and load it up. Repeat as needed. Just remember to only run the correct number of total threads.
Image
Post Reply Previous topicNext topic

Return to “Throw down the Gauntlet”