-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Feature/experiment scaling #375
base: develop
Are you sure you want to change the base?
Conversation
variant( | ||
"strong-scaling-factor", | ||
default="2", | ||
description="Strong-scaling factor (factor by which to increase resources)", | ||
) | ||
|
||
variant( | ||
"strong-scaling-num-exprs", | ||
default="4", | ||
description="Number of strong-scaling experiments", | ||
) | ||
|
||
variant( | ||
"weak-scaling-factor", | ||
default="2", | ||
description="Weak-scaling factor (factor by which to increase resources and problem sizes)", | ||
) | ||
|
||
variant( | ||
"weak-scaling-num-exprs", | ||
default="4", | ||
description="Number of weak-scaling experiments", | ||
) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We need to implement conditional variants so we can make this clearer. This could be
variant("scaling", default="none", values=("none", "strong", "weak"), description="...")
variant("scaling-factor", default=2, values=lambda x: x >= 2, when="scaling=weak", description="...")
variant("scaling-factor", default=2, values=lambda x: x >= 2, when="scaling=strong", description="...")
variant("scaling-iterations", default=4, values=lambda x: x > 1, when="scaling=weak", description="...")
variant("scaling-iterations", default=4, values=lambda x: x > 1, when="scaling=strong", description="...")
And with a little additional syntax beyond that we could get rid of the duplicate lines.
from benchpark.directives import variant | ||
|
||
|
||
class Scaling(object): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I would call this ScalingExperiment
instead of just Scaling
, I think it's a bit clearer that way
def generate_strong_scaling_parameters(self, initial_resource_list: list): | ||
scaling_factor = int(self.spec.variants["strong-scaling-factor"][0]) | ||
num_exprs = int(self.spec.variants["strong-scaling-num-exprs"][0]) - 1 | ||
round_robin_order = self.compute_round_robin_order(initial_resource_list) | ||
resource_list = [[x] for x in initial_resource_list] | ||
|
||
while num_exprs > 0: | ||
for idx in round_robin_order: | ||
for i, r in enumerate(resource_list): | ||
r.append(r[-1]*scaling_factor if i == idx else r[-1]) | ||
num_exprs=num_exprs-1 | ||
if not num_exprs: | ||
break | ||
return resource_list |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think this method can be simplified slightly
def generate_strong_scaling_parameters(self, initial_resource_list: list): | |
scaling_factor = int(self.spec.variants["strong-scaling-factor"][0]) | |
num_exprs = int(self.spec.variants["strong-scaling-num-exprs"][0]) - 1 | |
round_robin_order = self.compute_round_robin_order(initial_resource_list) | |
resource_list = [[x] for x in initial_resource_list] | |
while num_exprs > 0: | |
for idx in round_robin_order: | |
for i, r in enumerate(resource_list): | |
r.append(r[-1]*scaling_factor if i == idx else r[-1]) | |
num_exprs=num_exprs-1 | |
if not num_exprs: | |
break | |
return resource_list | |
def generate_strong_scaling_parameters(self, initial_resource_list: list): | |
scaling_factor = int(self.spec.variants["strong-scaling-factor"][0]) | |
num_exprs = int(self.spec.variants["strong-scaling-num-exprs"][0]) | |
round_robin_order = self.compute_round_robin_order(initial_resource_list) | |
resource_list = [[x] for x in initial_resource_list] | |
for i in range(num_exprs - 1): | |
idx = (i + round_robin_order[0]) * len(initial_resource_list) | |
for r_idx, resource in enumerate(resource_list): | |
next_value = resource[-1] * scaling_value if r_idx == idx else resource[-1] | |
resource.append(next_value) | |
return resource_list |
break | ||
return resource_list | ||
|
||
def generate_weak_scaling_parameters(self, initial_resource_list: list, initial_problem_size_list: list): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't think it's necessarily safe for us to assume that the resource requirements and the problem size are expressed in such neatly compatible ways. I think we should probably assume these could be different dimensions (e.g. resource list is a scalar, problem size is 3D) and round-robin them each separately.
The variant logic can be simplified once we merge #379 |
Draft implementation of the scaling functionality for benchmarks. Currently only strong scaling is implemented. Works for amg2023/openmp
bin/benchpark experiment init --dest ./experiments kripke programming_model=openmp scaling=strong strong-scaling-factor=2 strong-scaling-num-exprs=3
and kripke/openmp
bin/benchpark experiment init --dest ./experiments amg2023 programming_model=openmp experiment=strong strong-scaling-factor=2 strong-scaling-num-exprs=4