"Setting ig manually" was on the original menu I proposed. That seems
reasonable to me. The only "problem" is that our tutorials somewhat
claim, implicitly, that "ig=-1" is an option you can specify safely, and
not have to think about it again.
Guids are very effective in my commercial coding experience - but 128
bit strings - not 31 (which I continue to hope is the correct bit-count
of these seeds, if not 32) And, they require linking to libraries that
may not be quite "standard enough" for the culture of AMBER.
For now, I have "solved" my problem on our platform by sleeping
"RunSubscript" (0-99) seconds before start of each of the 100 runs. I
carefully check for duplicate runs on completion of each stage now. So
far, so good.
Adding another 10 bits of entropy from /dev/urandom to the 20 bits from
the microsceonds, still strikes me as a nice "to do" for the next major
AMBER release.
As final geek-out, I'll leave you with a silly mathematical "proof" of
the problem :) (Apologies if this went out already - but I am not
seeing it in any replies).
Like birthdays, the microseconds are uniformly distributed :) At least,
I found that if you repeatedly call gettimeofday() (and nothing else),
and stuff the results in an array, you get back all the contiguous
microseconds on fast hardware :) (which contradicts the lower time
resolution that the google hits reported from a few years ago).
The "proof" :)
Following the math at this site - which seems right:
https://betterexplained.com/articles/understanding-the-birthday-paradox/
<
https://betterexplained.com/articles/understanding-the-birthday-paradox/>
It is helpful to restate the familiar birthdays phenomenon first:
With 23 people in a room there are 253 possible pairs of people in the
room: (23 x 22 / 2) = 253
Chance of 2 people having different birthdays: (1 - 1/365) = (364/365)
Probably of making 253 comparisons and seeing all paris have different
birthdayis (364 / 365) ^ 253 -> . probability of 0.4995
Jobs on the cluster next:
Extending the same math to 100 jobs with 1000000 randomly distributed
microsecond start times :)
With 100 jobs there are (100 * 99 / 2) = 4950 possible pairs
Chance of 2 jobs having same microsecond assignment = (1-1/1000000) =
(999999/1000000)
Making 4950 comparisons and have them all be different is
(999999/1000000) ^ 4950 = .9951
OK - oops - it is 1 in 200 batch submissions of (100 jobs) that should
see a duplicate microsecond assignment :) But still.......
And, if the scheduler (quite plausibly) launches them all in 1/100 of a
second (which ours does not... yet) the picture gets 100x worse for
duplicate random seeds:
(99999/100000)^4950 = .952 (1 in 20 of the 100-job batch submissions
should have a duplicate random seed)
With the current algorithm, this tiny/silly/nuisance problem should only
get "worse" as hardware improves, more parallel runs are requested, and
their start times cluster closer and closer together on clusters.
OK - back to real work :)
On 6/21/2017 7:42 AM, Daniel Roe wrote:
> One last option that I don't think has been mentioned yet: if you
> absolutely want to make sure you use a different random seed for
> multiple jobs you can always set 'ig' manually for each one according
> to whatever scheme you like.
>
> -Dan
>
> On Tue, Jun 20, 2017 at 7:41 PM, Bill Ross <ross.cgl.ucsf.edu> wrote:
>>> Another way to guarantee uniqueness is serialize access to a generator at startup, which could be on one node.
>> Or you could have a voting scheme that would harvest the entropy between
>> the nodes (but still produce seeds serially).
>>
>> Bill
>>
>> On 6/20/17 1:55 PM, Bill Ross wrote:
>>> > https://www.guidgenerator.com/
>>>
>>> Oops, I spoke too soon, I assumed they were guaranteed to be unique.
>>>
>>> Another way to guarantee uniqueness is serialize access to a generator
>>> at startup, which could be on one node.
>>>
>>> Bill
>>>
>>>
>>> On 6/20/17 1:50 PM, Bill Ross wrote:
>>>>> we set up a web server that keeps a list of all seeds that have ever been used for any Amber simulation
>>>> Good idea, done in a general way here:
>>>>
>>>> https://www.guidgenerator.com/
>>>>
>>>> Bill
>>>>
>>>> On 6/20/17 1:20 PM, David Case wrote:
>>>>> On Tue, Jun 20, 2017, Adrian Roitberg wrote:
>>>>>
>>>>>> One can, but again, that is system dependent, so we do not want to
>>>>>> depend on that.
>>>>> I'm lost here: we expect the results for ig=-1 to be system dependent, since
>>>>> surely gettimeofday() is implemented in different ways on different hardware.
>>>>>
>>>>> The key design goal is not to end up with duplicate seeds; once you have a
>>>>> seed, it gets printed in the output file, and you can use that to re-run the
>>>>> job if needed.
>>>>>
>>>>> But whether or not the particular seed chosen is system dependent or not seems
>>>>> irrelevant(?)
>>>>>
>>>>> Here's my idea: we set up a web server that keeps a list of all seeds that
>>>>> have ever been used for any Amber simulation. Then pmemd can interrogate that
>>>>> database at startup, and make sure that it is using a seed that has never been
>>>>> used before.
>>>>>
>>>>> ....dac
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> AMBER mailing list
>>>>> AMBER.ambermd.org
>>>>> http://lists.ambermd.org/mailman/listinfo/amber
>>>> _______________________________________________
>>>> AMBER mailing list
>>>> AMBER.ambermd.org
>>>> http://lists.ambermd.org/mailman/listinfo/amber
>>> _______________________________________________
>>> AMBER mailing list
>>> AMBER.ambermd.org
>>> http://lists.ambermd.org/mailman/listinfo/amber
>>
>> _______________________________________________
>> AMBER mailing list
>> AMBER.ambermd.org
>> http://lists.ambermd.org/mailman/listinfo/amber
>
>
_______________________________________________
AMBER mailing list
AMBER.ambermd.org
http://lists.ambermd.org/mailman/listinfo/amber
Received on Wed Jun 21 2017 - 15:00:02 PDT
