Skip to content

Commit

Permalink
notebook updates
Browse files Browse the repository at this point in the history 
Signed-off-by: Niels Bantilan <[email protected]>
  • Loading branch information
@cosmicBboy
cosmicBboy committed Jul 11, 2023
1 parent 2407e5c commit 4eb8155
Showing 1 changed file with 39 additions and 31 deletions.
70 changes: 39 additions & 31 deletions scipy-2023/workshop.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -113,9 +113,8 @@
"`pyflyte run` also supports this use case through the `--remote` flag.\n",
"\n",
"```bash\n",
"pyflyte --config ~/.flyte/config-sandbox.yaml \\\n",
" run --remote \\\n",
" --image ghcr.io/flyteorg/flyte-conference-talks:scipy-2023-latest \\\n",
"pyflyte run --remote \\\n",
" --image $IMAGE \\\n",
" workflows/example_intro.py training_workflow \\\n",
" --hyperparameters '{\"C\": 0.01}'\n",
"```\n",
Expand Down Expand Up @@ -183,16 +182,16 @@
},
{
"cell_type": "code",
"execution_count": 17,
"execution_count": 1,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'http://localhost:30080/console/projects/flytesnacks/domains/development/executions/f00b4149410964e44876'"
"'http://localhost:30080/console/projects/flytesnacks/domains/development/executions/f1d6ad307ef9642cead7'"
]
},
"execution_count": 17,
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
Expand All @@ -215,7 +214,7 @@
},
{
"cell_type": "code",
"execution_count": 18,
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -224,7 +223,7 @@
},
{
"cell_type": "code",
"execution_count": 19,
"execution_count": 3,
"metadata": {},
"outputs": [
{
Expand All @@ -236,7 +235,7 @@
"LogisticRegression(C=0.1, max_iter=5000)"
]
},
"execution_count": 19,
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -316,7 +315,7 @@
},
{
"cell_type": "code",
"execution_count": 2,
"execution_count": 4,
"metadata": {},
"outputs": [
{
Expand All @@ -342,14 +341,14 @@
},
{
"cell_type": "code",
"execution_count": 23,
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"<FlyteLiteral id { project: \"flytesnacks\" domain: \"development\" name: \"fcbe6f63596a67d10000\" } spec { launch_plan { resource_type: LAUNCH_PLAN project: \"flytesnacks\" domain: \"development\" name: \"scheduled_training_workflow\" version: \"M2MrgLxOIHJfFcZ-1C2AOQ==\" } metadata { mode: SCHEDULED scheduled_at { seconds: 1689044400 } system_metadata { } } labels { } annotations { } auth_role { } } closure { outputs { uri: \"s3://my-s3-bucket/metadata/propeller/flytesnacks-development-fcbe6f63596a67d10000/end-node/data/0/outputs.pb\" } phase: SUCCEEDED started_at { seconds: 1689044400 nanos: 87098000 } duration { seconds: 77 nanos: 644313000 } created_at { seconds: 1689044400 nanos: 79244000 } updated_at { seconds: 1689044477 nanos: 731411000 } }>\n"
"<FlyteLiteral id { project: \"flytesnacks\" domain: \"development\" name: \"f11cda85b61621f34000\" } spec { launch_plan { resource_type: LAUNCH_PLAN project: \"flytesnacks\" domain: \"development\" name: \"scheduled_training_workflow\" version: \"4wzbYyFc3La_GyxHoyZR_w==\" } metadata { mode: SCHEDULED scheduled_at { seconds: 1689099360 } system_metadata { } } labels { } annotations { } auth_role { } } closure { outputs { uri: \"s3://my-s3-bucket/metadata/propeller/flytesnacks-development-f11cda85b61621f34000/end-node/data/0/outputs.pb\" } phase: SUCCEEDED started_at { seconds: 1689099360 nanos: 42934000 } duration { seconds: 53 nanos: 602100000 } created_at { seconds: 1689099360 nanos: 35797000 } updated_at { seconds: 1689099413 nanos: 645034000 } }>\n"
]
}
],
Expand All @@ -369,7 +368,7 @@
},
{
"cell_type": "code",
"execution_count": 24,
"execution_count": 7,
"metadata": {},
"outputs": [
{
Expand All @@ -381,7 +380,7 @@
"LogisticRegression(C=0.1, max_iter=1000)"
]
},
"execution_count": 24,
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
Expand Down Expand Up @@ -413,7 +412,7 @@
},
{
"cell_type": "code",
"execution_count": 5,
"execution_count": 8,
"metadata": {},
"outputs": [
{
Expand Down Expand Up @@ -571,14 +570,19 @@
"- **Portability:** each task is portable, which means that it can be run on\n",
" any Flyte cluster, or any infrastructure that can run containers.\n",
"- **Heterogeneity:** each task in a workflow can be written in any language and\n",
" can be run with varying compute requirements.\n",
" can be run with varying compute requirements."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"Now let's run the `get_data` task from `example_intro.py` with pyflyte run:\n",
"\n",
"```bash\n",
"pyflyte --config ~/.flyte/config-sandbox.yaml \\\n",
" run --remote \\\n",
" --image ghcr.io/flyteorg/flyte-conference-talks:scipy-2023-latest \\\n",
"pyflyte run --remote \\\n",
" --image $IMAGE \\\n",
" workflows/example_intro.py get_data\n",
"```"
]
Expand Down Expand Up @@ -905,17 +909,22 @@
"Pandera is a data validation tool for dataframe-like objects. In\n",
"[example_05_pandera_types.py](./workflows/example_05_pandera_types.py), we define\n",
"a pandera schema that validates the output of `get_data` as well as the DataFrame\n",
"input of `split_data` at runtime.\n",
"input of `split_data` at runtime."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"**Exercise:**\n",
"\n",
"Uncomment line 49 in the `example_05_pandera_types.py` and run the workflow with\n",
"`pyflyte run`:\n",
"\n",
"```bash\n",
"pyflyte --config ~/.flyte/config-sandbox.yaml \\\n",
" run --remote \\\n",
" --image ghcr.io/flyteorg/flyte-conference-talks:scipy-2023-latest \\\n",
"pyflyte run --remote \\\n",
" --image $IMAGE \\\n",
" workflows/example_05_pandera_types.py \\\n",
" get_splits --test_size 0.2 --random_state 42\n",
"```\n",
Expand Down Expand Up @@ -1163,7 +1172,7 @@
"\n",
"```bash\n",
"pyflyte register \\\n",
" --image ghcr.io/flyteorg/flyte-conference-talks:scipy-2023-latest \\\n",
" --image $IMAGE \\\n",
" --project flytesnacks \\\n",
" --domain development \\\n",
" workflows\n",
Expand All @@ -1179,9 +1188,8 @@
"Now let's run the newly registered `example_intro.py` workflow:\n",
"\n",
"```bash\n",
"pyflyte --config ~/.flyte/config-sandbox.yaml \\\n",
" run --remote \\\n",
" --image ghcr.io/flyteorg/flyte-conference-talks:scipy-2023-latest \\\n",
"pyflyte run --remote \\\n",
" --image $IMAGE \\\n",
" workflows/example_intro.py training_workflow \\\n",
" --hyperparameters '{\"C\": 0.01}'\n",
"```\n",
Expand Down Expand Up @@ -1212,7 +1220,7 @@
"\n",
"```bash\n",
"pyflyte register \\\n",
" --image ghcr.io/flyteorg/flyte-conference-talks:scipy-2023-latest \\\n",
" --image $IMAGE \\\n",
" --project flytesnacks \\\n",
" --domain production \\\n",
" --non-fast \\\n",
Expand Down Expand Up @@ -1261,16 +1269,16 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'http://localhost:30080/console/projects/flytesnacks/domains/development/executions/f7483f3766a8044cc86f'"
"'http://localhost:30080/console/projects/flytesnacks/domains/development/executions/ff4559ebefeeb480c96f'"
]
},
"execution_count": 9,
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
Expand Down

0 comments on commit 4eb8155

Please sign in to comment.