Skip to content

Kristenliu/bench 8074 #19

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
kvo3 wants to merge 5 commits into
base: main
Choose a base branch
from
+60 −134
Open

Kristenliu/bench 8074 #19

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
a9a9345
Fixed working_with_groups.ipynb
kvo3 May 11, 2026
c3d8a5d
Fixed working_with_bq_resources.ipynb
kvo3 May 11, 2026
6f0e450
Fixed working_with_resources.ipynb
kvo3 May 11, 2026
fae6f02
Fixed creating_a_data_collection.ipynb
kvo3 May 11, 2026
9615753
Fixed typo
kvo3 May 12, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 31 additions & 36 deletions first_hour_on_vwb/creating_a_data_collection.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,6 +182,7 @@
" self.input_name = wu.TextInputWidget(\"<WORKSPACE_NAME>\",\"Workspace Name:\").get()\n",
" self.input_description = wu.TextInputWidget(\"<DESCRIPTION>\",\"Description:\").get()\n",
" self.input_workspace_id = wu.TextInputWidget(\"<WORKSPACE_ID>\",\"Workspace ID:\").get()\n",
" self.input_pod = wu.TextInputWidget(\"<POD_ID>\",\"Pod ID:\").get()\n",
" self.output_workspace_id = widgets.Text()\n",
" self.output_workspace_id.value = self.input_workspace_id.value\n",
" self.button = wu.StyledButton('Create workspace','Click to create a new workspace','plus').get()\n",
Expand All @@ -190,7 +191,7 @@
" self.vb = widgets.VBox(\n",
" children = [self.label, self.warning,\n",
" self.input_name, self.input_description,\n",
" self.input_workspace_id,\n",
" self.input_workspace_id, self.input_pod,\n",
" self.button, self.output],\n",
" layout = wu.vbox_layout)\n",
" \n",
Expand All @@ -204,6 +205,7 @@
" f\"--id={self.input_workspace_id.value.strip()}\",\n",
" f\"--description={self.input_description.value.strip()}\",\n",
" f\"--name={self.input_name.value.strip()}\",\n",
" f\"--pod={self.input_pod.value.strip()}\",\n",
" ]\n",
" print('Running command to create workspace...')\n",
" print('\\n'.join(createWorkspaceCommandList))\n",
Expand All @@ -222,19 +224,20 @@
"tags": []
},
"source": [
"### Convert new workspace to data collection\n",
"<a id=\"convert-to-dc\"></a>\n",
"### Create a new data collection\n",
"<a id=\"create-dc\"></a>\n",
"\n",
"Now you'll convert your newly created workspace, to which you have added resources, into a data collection which can be shared with others and added to other workspaces. \n",
"Run the cell below to create a widget, then populate the widget's input fields and click the button to convert the workspace to a data collection. Please note that until you <a href=\"#publish-version\">publish a version</a> in the next section, your data collection will not appear in the data catalog.\n",
"Now you'll create a new data collection which can be shared with others and added to other workspaces. \n",
"Run the cell below to create a widget, then populate the widget's input fields and click the button to create a data collection. Please note that until you <a href=\"#publish-version\">publish a version</a> in the next section, your data collection will not appear in the data catalog.\n",
"\n",
"Widget input parameters include:\n",
"- `Workspace ID`: Automatically populated with the workspace ID of the workspace created in the previous step.\n",
"- `Workspace Name`: Must be a string. This value is displayed in the Data Collection modal once the workspace is converted to a data collection, so the value should communicate the intended purpose (e.g. `<STUDY_NAME> Data Collection`).<br> While the Workbench UI and this widget require a workspace name to be provided, the CLI does not; if no workspace name is provided to the CLI, a UUID is generated instead.\n",
"- `Workspace ID`: Must be unique and consist only of lowercase letters, numbers and underscores. Provide a workspace ID that suggests something about the contents of the data collection you'd like to create and include the date of its creation, such as `<STUDY_NAME>_<YYMMDD>_dc_ws`. *You cannot change the workspace ID after workspace creation.* \n",
"- `Short Description`: Must be a string. This description will be visible in the Add a Data Collection modal and should summarize the purpose and/or contents of your data collection.\n",
"\n",
"The output should resemble:\n",
"```\n",
"Workspace properties successfully updated.\n",
"Workspace successfully created.\n",
"ID: <STUDY_NAME>_<YYMMDD>_dc_ws\n",
"Name: <STUDY_NAME>-Data-<YYMMDD>\n",
"Description: <DESCRIPTION>\n",
Expand All @@ -247,7 +250,6 @@
"Created: YYYY-MM-DD\n",
"Last updated: YYYY-MM-DD\n",
"# Resources: <NUMBER_OF_RESOURCES>\n",
"Workspace properties successfully updated.\n",
"```"
]
},
Expand All @@ -259,53 +261,46 @@
},
"outputs": [],
"source": [
"class ConvertToDataCollectionWidget(object):\n",
"class CreateDataCollectionWidget(object):\n",
" def __init__(self,prev_widget):\n",
" self.label = widgets.Label(value = 'Please provide appropriate values in the input boxes.')\n",
" self.workspace_ids = self.get_workspace_ids()\n",
" self.new_ws_id = prev_widget.get_workspace_id();\n",
" self.input_workspace_id = wu.DropdownInputWidget([self.new_ws_id],self.new_ws_id,\"Workspace ID:\").get()\n",
" self.input_name = wu.TextInputWidget(\"<DATA_COLLECTION_NAME>\",\"Data Collection Name:\").get()\n",
" self.input_data_collection_id = wu.TextInputWidget(\"<DATA_COLLECTION_ID>\",\"Data Collection ID:\").get()\n",
" self.input_pod = wu.TextInputWidget(\"<POD_ID>\",\"Pod ID:\").get()\n",
" self.input_short_description = wu.TextInputWidget(\"<SHORT_DESCRIPTION>\",\"Short Description:\").get()\n",
" self.button = wu.StyledButton('Convert to data collection','Click to convert to data collection','check').get()\n",
" self.button.on_click(self.convert_to_data_collection)\n",
" self.button = wu.StyledButton('Create data collection','Click to create data collection','check').get()\n",
" self.button.on_click(self.create_data_collection)\n",
" self.output = widgets.Output()\n",
" self.vb = widgets.VBox([\n",
" self.label,\n",
" self.input_workspace_id,\n",
" self.input_data_collection_id,\n",
" self.input_short_description,\n",
" self.input_pod,\n",
" self.input_name,\n",
" self.button,\n",
" self.output\n",
" ], layout=wu.vbox_layout)\n",
" \n",
"\n",
"\n",
" def get_workspace_id(self):\n",
" return self.input_workspace_id.value.strip()\n",
"\n",
" def get_workspace_ids(self):\n",
" result = subprocess.run([\"wb\",\"workspace\",\"list\",\"--format=JSON\"],capture_output=True,text=True)\n",
" ids_list = wu.list_workspace_ids(result.stdout)\n",
" # Insert empty string to display as value of dropdown until changed by user.\n",
" ids_list.insert(0, \" \")\n",
" return ids_list\n",
" \n",
" def convert_to_data_collection(self,b):\n",
" workspace_id = self.input_workspace_id.value\n",
" \n",
" def create_data_collection(self,b):\n",
" short_desc = self.input_short_description.value\n",
" with self.output:\n",
" prettyConvertToDataCollectionCommand = f\"\"\"wb workspace set-property \\\\\n",
" --workspace={workspace_id} \\\\\n",
" --properties=\\\"terra-type=data-collection,terra-workspace-short-description={short_desc}\\\"\n",
" \"\"\"\n",
" print(\"Running command to convert workspace to data collection...\")\n",
" print(prettyConvertToDataCollectionCommand)\n",
" print(\"Running command to create data collection...\")\n",
" print(\"Your data collection will be ready in less than one minute...\")\n",
" result = subprocess.run([\"wb\",\"workspace\",\"set-property\",\n",
" f\"--workspace={workspace_id}\",\n",
" result = subprocess.run([\"wb\",\"workspace\",\"create\",\n",
" f\"--id={self.input_data_collection_id.value.strip()}\",\n",
" f\"--name={self.input_name.value.strip()}\",\n",
" f\"--pod={self.input_pod.value.strip()}\",\n",
" f\"--properties=terra-type=data-collection,terra-workspace-short-description={short_desc}\"],\n",
" capture_output=True,text=True)\n",
" print(result.stderr) if not result.stdout else print(result.stdout)\n",
"\n",
"convert_to_dc_widget = ConvertToDataCollectionWidget(create_ws_widget)\n",
"display(convert_to_dc_widget.vb)"
"create_dc_widget = CreateDataCollectionWidget(create_ws_widget)\n",
"display(create_dc_widget.vb)"
]
},
{
Expand Down Expand Up @@ -429,7 +424,7 @@
" publishVersionResult = subprocess.run(publishVersionCommand, shell = True, capture_output = True, text = True, check = True)\n",
" print(publishVersionResult.stderr) if not publishVersionResult.stdout else print(publishVersionResult.stdout)\n",
"\n",
"publish_version_widget = PublishVersionWidget(convert_to_dc_widget)\n",
"publish_version_widget = PublishVersionWidget(create_dc_widget)\n",
"display(publish_version_widget.vb)"
]
},
Expand Down
18 changes: 16 additions & 2 deletions first_hour_on_vwb/working_with_bq_resources.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -86,7 +86,7 @@
"'''\n",
"Resolves bucket URL from bucket reference in workspace.\n",
"'''\n",
"def get_bucket_url_from_reference(resource_id):\n",
"def get_bucket_url_from_reference(bucket_reference):\n",
" BUCKET_CMD_OUTPUT = !wb resolve --name={bucket_reference}\n",
" BUCKET = BUCKET_CMD_OUTPUT[0]\n",
" return BUCKET\n",
Expand Down Expand Up @@ -219,7 +219,12 @@
},
"outputs": [],
"source": [
"%bigquery_stats bigquery-public-data.human_genome_variants.1000_genomes_pedigree"
"from google.cloud import bigquery\n",
"client = bigquery.Client()\n",
"table = client.get_table(\"bigquery-public-data.human_genome_variants.1000_genomes_pedigree\")\n",
"print(f\"Rows: {table.num_rows}, Size: {table.num_bytes} bytes\")\n",
"for field in table.schema:\n",
" print(f\" {field.name}: {field.field_type}\")"
]
},
{
Expand All @@ -233,6 +238,15 @@
"Run the cell below to total the number of distinct families represented in the 1000 Genomes dataset."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"%load_ext google.cloud.bigquery"
]
},
{
"cell_type": "code",
"execution_count": null,
Expand Down
25 changes: 13 additions & 12 deletions first_hour_on_vwb/working_with_groups.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@
},
{
"cell_type": "code",
"execution_count": 3,
"execution_count": null,
"metadata": {
"scrolled": true,
"tags": []
Expand Down Expand Up @@ -168,7 +168,7 @@
" def list_members(self,b):\n",
" self.output.clear_output()\n",
" with self.output:\n",
" wb_command = [\"wb\",\"group\",\"list-users\",f\"--name={self.input_group_name.value}\",\"--format=JSON\"]\n",
" wb_command = [\"wb\",\"group\",\"role\",\"list\",f\"--name={self.input_group_name.value}\",\"--format=JSON\"]\n",
" result = subprocess.run(wb_command,capture_output=True,text=True)\n",
" if not result.stdout:\n",
" self.output.append_display_data(result.stderr)\n",
Expand Down Expand Up @@ -497,7 +497,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": null,
"metadata": {
"tags": []
},
Expand Down Expand Up @@ -544,7 +544,7 @@
" with self.output:\n",
" wb_command = [\"wb\", \"group\", \"role\", \"grant\", \"user\", f\"--name={self.input_group_name.value}\",f\"--org={self.input_org_id.value}\",f\"--email={self.input_user_email.value}\",f\"--role={self.role_drop_down.value}\"]\n",
" if self.require_grant_reason.value:\n",
" commandList.append(f\"--reason={self.input_reason.value}\")\n",
" wb_command.append(f\"--reason={self.input_reason.value}\")\n",
" result = subprocess.run(wb_command,capture_output=True,text=True)\n",
" print(result.stderr) if not result.stdout else print(result.stdout)\n",
"\n",
Expand Down Expand Up @@ -573,7 +573,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": null,
"metadata": {
"tags": []
},
Expand Down Expand Up @@ -602,6 +602,7 @@
" self.input_org_id = wu.TextInputWidget('<ORG_ID>', \"Organization ID:\").get()\n",
" self.role_drop_down = wu.DropdownInputWidget(['MEMBER','ADMIN'],'MEMBER',\"Role:\").get()\n",
" self.input_reason = wu.TextInputWidget('<REASON>', \"Reason:\").get()\n",
" self.require_grant_reason = wu.LongLabelCheckbox('Require admins to provide a reason when granting a role on this group.').get()\n",
" self.output = widgets.Output()\n",
" self.button = wu.StyledButton('Remove user','Click to remove a user from the group.','user-minus').get()\n",
" self.vb = widgets.VBox(\n",
Expand All @@ -620,7 +621,7 @@
" with self.output:\n",
" wb_command = [\"wb\", \"group\", \"role\", \"revoke\", \"user\", f\"--name={self.input_group_name.value}\",f\"--org={self.input_org_id.value}\",f\"--email={self.input_user_email.value}\",f\"--role={self.role_drop_down.value}\"]\n",
" if self.require_grant_reason.value:\n",
" commandList.append(f\"--reason={self.input_reason.value}\")\n",
" wb_command.append(f\"--reason={self.input_reason.value}\")\n",
" result = subprocess.run(wb_command,capture_output=True,text=True)\n",
" print(result.stderr) if not result.stdout else print(result.stdout)\n",
"\n",
Expand Down Expand Up @@ -653,7 +654,7 @@
},
{
"cell_type": "code",
"execution_count": 10,
"execution_count": null,
"metadata": {
"tags": []
},
Expand Down Expand Up @@ -692,9 +693,9 @@
" with open(self.input_file.value) as csv_file:\n",
" csv_reader = csv.DictReader(csv_file,delimiter=',')\n",
" for user in csv_reader:\n",
" wb_command = [\"wb\", \"group\", \"role\", \"grant\",\"user\",f\"--name={self.input_group_name.value}\",f\"--org={self.input_org_id.value}\",f\"--email={user['WORKBENCH_USER_EMAIL']}\",f\"--policy={user['ROLE']}\"]\n",
" wb_command = [\"wb\", \"group\", \"role\", \"grant\",\"user\",f\"--name={self.input_group_name.value}\",f\"--org={self.input_org_id.value}\",f\"--email={user['WORKBENCH_USER_EMAIL']}\",f\"--role={user['ROLE']}\"]\n",
" if self.input_reason.value:\n",
" commandList.append(f\"--reason={self.input_reason.value}\")\n",
" wb_command.append(f\"--reason={self.input_reason.value}\")\n",
" result = subprocess.run(wb_command,capture_output=True,text=True)\n",
" print(result.stderr) if not result.stdout else print(result.stdout)\n",
"\n",
Expand Down Expand Up @@ -728,7 +729,7 @@
},
{
"cell_type": "code",
"execution_count": 11,
"execution_count": null,
"metadata": {
"tags": []
},
Expand Down Expand Up @@ -767,9 +768,9 @@
" with open(self.input_file.value) as csv_file:\n",
" csv_reader = csv.DictReader(csv_file,delimiter=',')\n",
" for user in csv_reader:\n",
" wb_command = [\"wb\", \"group\", \"role\",\"revoke\",\"user\",f\"--name={self.input_group_name.value}\",f\"--org={self.input_org_id.value}\",f\"--email={user['WORKBENCH_USER_EMAIL']}\",f\"--policy={user['POLICY']}\"]\n",
" wb_command = [\"wb\", \"group\", \"role\",\"revoke\",\"user\",f\"--name={self.input_group_name.value}\",f\"--org={self.input_org_id.value}\",f\"--email={user['WORKBENCH_USER_EMAIL']}\",f\"--role={user['POLICY']}\"]\n",
" if self.input_reason.value:\n",
" commandList.append(f\"--reason={self.input_reason.value}\")\n",
" wb_command.append(f\"--reason={self.input_reason.value}\")\n",
" result = subprocess.run(wb_command,capture_output=True,text=True)\n",
" print(result.stderr) if not result.stdout else print(result.stdout)\n",
"\n",
Expand Down
Loading