Skip to content
Merged
Show file tree
Hide file tree
Changes from 26 commits
Commits
Show all changes
70 commits
Select commit Hold shift + click to select a range
17aca93
Add process column to Session table to record if processing is reques…
d-j-hatton May 9, 2025
dc652ba
Allow analysis to be turned off via a query parameter
d-j-hatton May 9, 2025
8f38607
The server can work out from the session whether there should be proc…
d-j-hatton May 12, 2025
8ee1de8
Add some prefixes and tags to existing routers
d-j-hatton May 12, 2025
97b16c8
More moving endpoints around
d-j-hatton May 12, 2025
76b43b6
Move workflow setup specific endpoints to their own router
d-j-hatton May 12, 2025
a9c50db
Still moving endpoints under their own routers
d-j-hatton May 12, 2025
4c2971e
Move SPA and tomo workflow endpoints
d-j-hatton May 13, 2025
0538013
More endpoint moving
d-j-hatton May 13, 2025
90db284
Move most context API calls to use a lookup based on function name ra…
d-j-hatton May 14, 2025
ed99d47
Make two endpoints in the cases where functionality needs to be share…
d-j-hatton May 14, 2025
c3e03d8
More moving shared endpoints
d-j-hatton May 14, 2025
23119cb
Move grid square and foil hole endpoints
d-j-hatton May 14, 2025
e4d2dd7
Move correlative workflow related endpoints
d-j-hatton May 14, 2025
0b9c459
Move some remaining generic endpoints
d-j-hatton May 14, 2025
ffb6d8d
Move prometheus update endpoints
d-j-hatton May 14, 2025
533d47b
Translate some URLs from TUI app
d-j-hatton May 14, 2025
14bc081
Clear URLs from app.py
d-j-hatton May 15, 2025
6443c34
The last big push to switch to using routers to determine URLs
d-j-hatton May 20, 2025
986905a
Function changed position in code
d-j-hatton May 21, 2025
9a427a2
Fix typo
d-j-hatton May 21, 2025
3f6d415
Some test fixing
d-j-hatton May 21, 2025
9975d95
URL fix
d-j-hatton May 21, 2025
c1f06ab
Mocking the wrong thing now
d-j-hatton May 21, 2025
477f58c
visit now needed in environment as it is used by url_path_for functio…
d-j-hatton May 21, 2025
726c5b1
Merge main
d-j-hatton May 21, 2025
8f3bfe1
Small fix
d-j-hatton May 21, 2025
9ba6410
Moved 'sanitise()' and 'sanitise_path()' from 'murfey.server.__init__…
tieneupin May 21, 2025
87db4a3
Emptied out 'murfey.server.__init__' as part of eliminating unnecessa…
tieneupin May 21, 2025
dfacae2
'murfey.server' entry point needed to be updated
tieneupin May 21, 2025
807c8cf
Emptied out 'murfey.server.api.__init__'; migrated 'inspect_prometheu…
tieneupin May 21, 2025
8e3bd26
Fixed broken imports due to 'murfey.server' refactor; moved FastAPI t…
tieneupin May 22, 2025
ef9e7e3
Removed 'tests/client/test.py' as it's very outdated
tieneupin May 22, 2025
4e7bca2
Moved 'murfey.server.websocket' to 'murfey.server.api.websocket'; fix…
tieneupin May 22, 2025
22603a1
Fixed broken import in 'spa_ispyb_messages' CLI due to 'murfey.server…
tieneupin May 22, 2025
da5a7ab
Adjusted imports of 'murfey.server.ispyb.DB'; imported the object as …
tieneupin May 22, 2025
340fe8f
Reverted unexpected change to how analysers are disabled in 'start_mu…
tieneupin May 22, 2025
eaa16af
Titlecased router names; rearranged order of routers as loaded in the…
tieneupin May 22, 2025
b758b60
Created route manifest of all the routers, paths, and endpoint functi…
tieneupin May 22, 2025
ad5d0a2
Added path parameter information to route manifest
tieneupin May 22, 2025
6629a12
Added utility function and supporting functions to generate URL path …
tieneupin May 22, 2025
55e71d8
Updated function documentation
tieneupin May 23, 2025
589fce7
Replaced 'router.url_path_for()' instances in client with use of our …
tieneupin May 23, 2025
3caeb0c
Re-added 'register_client_to_visit()' to 'murfey.server.api.session_c…
tieneupin May 23, 2025
cb499fb
Added 'register_client_to_visit' to route manifest
tieneupin May 23, 2025
12a6000
Added logging to 'murfey.util.api'
tieneupin May 23, 2025
12b4116
Replaced most other instances of URL requests to the instrument or ba…
tieneupin May 23, 2025
9e17e5a
Added CLI to generate FastAPI route manifest
tieneupin May 23, 2025
c37d5d9
Added logic to toggle partial or exact matches between str pattern an…
tieneupin May 23, 2025
863a61c
Fixed broken API call in tests
tieneupin May 23, 2025
015c7e8
Typo in function name
tieneupin May 23, 2025
0b4fc6b
Changed incoming path parameters validation logic
tieneupin May 23, 2025
214baeb
Updated Docker images used to newer versions
tieneupin May 23, 2025
bdf7256
Added route manifest as a file to add to Python package upon install
tieneupin May 23, 2025
24b4461
Convert UUID4 into string upon generation
tieneupin May 23, 2025
86316a9
Some types are complicated 'typing' strings; skip validating those fo…
tieneupin May 23, 2025
219f14e
Used wrong function name
tieneupin May 23, 2025
4e2e6c9
More detail in error logs
tieneupin May 23, 2025
c628091
Fixed typo in when calling 'suggest_path'
tieneupin May 23, 2025
841dac7
Use 'murfey.server._transport_object' directly as in 'murfey.server.f…
tieneupin May 27, 2025
3eaa517
Logs to keep track of 'TransportManager' setup upon server startup
tieneupin May 27, 2025
49a39f0
Standardised how the Analyser, DirWatcher, and Rsyncer objects are re…
tieneupin May 27, 2025
43dca41
Forgot to close the bracket
tieneupin May 27, 2025
a917ba1
Fixed broken imports in SPA workflows module
tieneupin May 27, 2025
5d122b6
Convert grid square name ('gs_name') into int upon extraction instead
tieneupin May 27, 2025
067e70f
Searched for 'start_dc' under wrong router
tieneupin May 27, 2025
9ecc94c
Looked for 'register_spa_proc_params' under wrong router
tieneupin May 27, 2025
69afc1d
Merged CLEM workflow fix changes into current branch
tieneupin May 27, 2025
7d34ed5
Updated logs and parameter names in 'murfey.util.api'
tieneupin May 28, 2025
ba85127
Made sure that 'gsid' in 'register_grid_square' receives an int
tieneupin May 28, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 14 additions & 1 deletion src/murfey/client/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -26,13 +26,26 @@
from murfey.client.instance_environment import MurfeyInstanceEnvironment
from murfey.client.tui.app import MurfeyTUI
from murfey.client.tui.status_bar import StatusBar
from murfey.util.client import _get_visit_list, authorised_requests, read_config
from murfey.server.api.session_control import router as session_router
from murfey.util.client import authorised_requests, read_config
from murfey.util.models import Visit

log = logging.getLogger("murfey.client")

requests.get, requests.post, requests.put, requests.delete = authorised_requests()


def _get_visit_list(api_base: ParseResult, instrument_name: str):
proxy_path = api_base.path.rstrip("/")
get_visits_url = api_base._replace(
path=f"{proxy_path}{session_router.url_path_for('get_current_visits', instrument_name=instrument_name)}"
)
server_reply = requests.get(get_visits_url.geturl())
if server_reply.status_code != 200:
raise ValueError(f"Server unreachable ({server_reply.status_code})")
return [Visit.parse_obj(v) for v in server_reply.json()]


def write_config(config: configparser.ConfigParser):
mcch = os.environ.get("MURFEY_CLIENT_CONFIG_HOME")
murfey_client_config_home = Path(mcch) if mcch else Path.home()
Expand Down
14 changes: 9 additions & 5 deletions src/murfey/client/contexts/spa.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@
MurfeyID,
MurfeyInstanceEnvironment,
)
from murfey.server.api.file_manip import router as file_manip_router
from murfey.server.api.session_control import router as session_router
from murfey.server.api.session_control import spa_router as session_spa_router
from murfey.server.api.workflow import spa_router as workflow_spa_router
from murfey.util.client import (
authorised_requests,
capture_get,
Expand Down Expand Up @@ -262,7 +266,7 @@ def gather_metadata(
binning_factor = 1
if environment:
server_config_response = capture_get(
f"{str(environment.url.geturl())}/instruments/{environment.instrument_name}/machine"
f"{str(environment.url.geturl())}{session_router.url_path_for('machine_info_by_instrument', instrument_name=environment.instrument_name)}"
)
if server_config_response is None:
return None
Expand Down Expand Up @@ -426,7 +430,7 @@ def _position_analysis(
local_atlas_path,
grid_square=str(grid_square),
)[str(grid_square)]
gs_url = f"{str(environment.url.geturl())}/sessions/{environment.murfey_session}/grid_square/{grid_square}"
gs_url = f"{str(environment.url.geturl())}{session_spa_router.url_path_for('register_grid_square', session_id=environment.murfey_session, gsid=grid_square)}"
gs = grid_square_data(
grid_square_metadata_file,
grid_square,
Expand Down Expand Up @@ -467,7 +471,7 @@ def _position_analysis(
)
foil_hole = foil_hole_from_file(transferred_file)
if foil_hole not in self._foil_holes[grid_square]:
fh_url = f"{str(environment.url.geturl())}/sessions/{environment.murfey_session}/grid_square/{grid_square}/foil_hole"
fh_url = f"{str(environment.url.geturl())}{session_spa_router.url_path_for('register_foil_hole', session_id=environment.murfey_session, gs_name=grid_square)}"
if environment.murfey_session is not None:
fh = foil_hole_data(
grid_square_metadata_file,
Expand Down Expand Up @@ -580,7 +584,7 @@ def post_transfer(
eer_fractionation_file = None
if file_transferred_to.suffix == ".eer":
response = capture_post(
f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/eer_fractionation_file",
f"{str(environment.url.geturl())}{file_manip_router.url_path_for('write_eer_fractionation_file', visit_name=environment.visit, session_id=environment.murfey_session)}",
json={
"eer_path": str(file_transferred_to),
"fractionation": environment.data_collection_parameters[
Expand Down Expand Up @@ -609,7 +613,7 @@ def post_transfer(
)
foil_hole = None

preproc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/spa_preprocess"
preproc_url = f"{str(environment.url.geturl())}{workflow_spa_router.url_path_for('request_spa_preprocessing', visit_name=environment.visit, session_id=environment.murfey_session)}"
preproc_data = {
"path": str(file_transferred_to),
"description": "",
Expand Down
12 changes: 7 additions & 5 deletions src/murfey/client/contexts/spa_metadata.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@
from murfey.client.context import Context
from murfey.client.contexts.spa import _file_transferred_to, _get_source
from murfey.client.instance_environment import MurfeyInstanceEnvironment, SampleInfo
from murfey.server.api.session_control import spa_router as session_spa_router
from murfey.server.api.workflow import router as workflow_router
from murfey.util.client import (
authorised_requests,
capture_post,
Expand Down Expand Up @@ -166,7 +168,7 @@ def post_transfer(
environment.samples[source] = SampleInfo(
atlas=Path(partial_path), sample=sample
)
url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/register_data_collection_group"
url = f"{str(environment.url.geturl())}{workflow_router.url_path_for('register_dc_group', visit_name=environment.visit, session_id=environment.murfey_session)}"
dcg_search_dir = "/".join(
p for p in transferred_file.parent.parts if p != environment.visit
)
Expand Down Expand Up @@ -202,7 +204,7 @@ def post_transfer(
for gs, pos_data in gs_pix_positions.items():
if pos_data:
capture_post(
f"{str(environment.url.geturl())}/sessions/{environment.murfey_session}/grid_square/{gs}",
f"{str(environment.url.geturl())}{session_spa_router.url_path_for('register_grid_square', session_id=environment.murfey_session, gsid=gs)}",
json={
"tag": dcg_tag,
"x_location": pos_data[0],
Expand All @@ -221,7 +223,7 @@ def post_transfer(
and environment
):
# Make sure we have a data collection group before trying to register grid square
url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/register_data_collection_group"
url = f"{str(environment.url.geturl())}{workflow_router.url_path_for('register_dc_group', visit_name=environment.visit, session_id=environment.murfey_session)}"
dcg_search_dir = "/".join(
p
for p in transferred_file.parent.parent.parts
Expand Down Expand Up @@ -270,7 +272,7 @@ def post_transfer(
visitless_source = str(visitless_source_images_dirs[-1])

if fh_positions:
gs_url = f"{str(environment.url.geturl())}/sessions/{environment.murfey_session}/grid_square/{gs_name}"
gs_url = f"{str(environment.url.geturl())}{session_spa_router.url_path_for('register_grid_square', session_id=environment.murfey_session, gsid=gs_name)}"
gs_info = grid_square_data(
transferred_file,
int(gs_name),
Expand All @@ -295,7 +297,7 @@ def post_transfer(

for fh, fh_data in fh_positions.items():
capture_post(
f"{str(environment.url.geturl())}/sessions/{environment.murfey_session}/grid_square/{gs_name}/foil_hole",
f"{str(environment.url.geturl())}{session_spa_router.url_path_for('register_foil_hole', session_id=environment.murfey_session, gs_name=gs_name)}",
json={
"name": fh,
"x_location": fh_data.x_location,
Expand Down
26 changes: 15 additions & 11 deletions src/murfey/client/contexts/tomo.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,10 @@
MurfeyID,
MurfeyInstanceEnvironment,
)
from murfey.server.api.file_manip import router as file_manip_router
from murfey.server.api.session_control import router as session_router
from murfey.server.api.workflow import router as workflow_router
from murfey.server.api.workflow import tomo_router as workflow_tomo_router
from murfey.util.client import (
authorised_requests,
capture_post,
Expand Down Expand Up @@ -109,7 +113,7 @@ def register_tomography_data_collections(
)
return
try:
dcg_url = f"{str(environment.url.geturl())}/visits/{str(environment.visit)}/{environment.murfey_session}/register_data_collection_group"
dcg_url = f"{str(environment.url.geturl())}{workflow_router.url_path_for('register_dc_group', visit_name=environment.visit, session_id=environment.murfey_session)}"
dcg_data = {
"experiment_type": "tomo",
"experiment_type_id": 36,
Expand All @@ -121,7 +125,7 @@ def register_tomography_data_collections(

for tilt_series in self._tilt_series.keys():
if tilt_series not in self._tilt_series_with_pjids:
dc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/start_data_collection"
dc_url = f"{str(environment.url.geturl())}{workflow_router.url_path_for('start_dc', visit_name=environment.visit, session_id=environment.murfey_session)}"
dc_data = {
"experiment_type": "tomography",
"file_extension": file_extension,
Expand Down Expand Up @@ -157,7 +161,7 @@ def register_tomography_data_collections(
)
capture_post(dc_url, json=dc_data)

proc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/register_processing_job"
proc_url = f"{str(environment.url.geturl())}{workflow_router.url_path_for('register_proc', visit_name=environment.visit, session_id=environment.murfey_session)}"
for recipe in ("em-tomo-preprocess", "em-tomo-align"):
capture_post(
proc_url,
Expand Down Expand Up @@ -262,7 +266,7 @@ def _add_tilt(
f"Tilt series {tilt_series} was previously thought complete but now {file_path} has been seen"
)
self._completed_tilt_series.remove(tilt_series)
rerun_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/rerun_tilt_series"
rerun_url = f"{str(environment.url.geturl())}{workflow_tomo_router.url_path_for('register_tilt_series_for_rerun', visit_name=environment.visit)}"
rerun_data = {
"session_id": environment.murfey_session,
"tag": tilt_series,
Expand All @@ -276,7 +280,7 @@ def _add_tilt(
if not self._tilt_series.get(tilt_series):
logger.info(f"New tilt series found: {tilt_series}")
self._tilt_series[tilt_series] = [file_path]
ts_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/tilt_series"
ts_url = f"{str(environment.url.geturl())}{workflow_tomo_router.url_path_for('register_tilt_series', visit_name=environment.visit)}"
ts_data = {
"session_id": environment.murfey_session,
"tag": tilt_series,
Expand Down Expand Up @@ -305,7 +309,7 @@ def _add_tilt(
self._tilt_series[tilt_series].append(file_path)

if environment:
tilt_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/tilt"
tilt_url = f"{str(environment.url.geturl())}{workflow_tomo_router.url_path_for('register_tilt', visit_name=environment.visit, session_id=environment.murfey_session)}"
tilt_data = {
"movie_path": str(file_transferred_to),
"tilt_series_tag": tilt_series,
Expand All @@ -316,7 +320,7 @@ def _add_tilt(
eer_fractionation_file = None
if environment.data_collection_parameters.get("num_eer_frames"):
response = requests.post(
f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/eer_fractionation_file",
f"{str(environment.url.geturl())}{file_manip_router.url_path_for('write_eer_fractionation_file', visit_name=environment.visit, session_id=environment.murfey_session)}",
json={
"num_frames": environment.data_collection_parameters[
"num_eer_frames"
Expand All @@ -331,7 +335,7 @@ def _add_tilt(
},
)
eer_fractionation_file = response.json()["eer_fractionation_file"]
preproc_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/tomography_preprocess"
preproc_url = f"{str(environment.url.geturl())}{workflow_tomo_router.url_path_for('request_tomography_preprocessing', visit_name=environment.visit, session_id=environment.murfey_session)}"
preproc_data = {
"path": str(file_transferred_to),
"description": "",
Expand Down Expand Up @@ -491,7 +495,7 @@ def post_transfer(

# Always update the tilt series length in the database after an mdoc
if environment.murfey_session is not None:
length_url = f"{str(environment.url.geturl())}/sessions/{environment.murfey_session}/tilt_series_length"
length_url = f"{str(environment.url.geturl())}{workflow_tomo_router.url_path_for('register_tile_series_length', session_id=environment.murfey_session)}"
capture_post(
length_url,
json={
Expand All @@ -508,7 +512,7 @@ def post_transfer(
f"The following tilt series are considered complete: {completed_tilts} "
f"after {transferred_file}"
)
complete_url = f"{str(environment.url.geturl())}/visits/{environment.visit}/{environment.murfey_session}/completed_tilt_series"
complete_url = f"{str(environment.url.geturl())}{workflow_tomo_router.url_path_for('register_completed_tilt_series', visit_name=environment.visit, session_id=environment.murfey_session)}"
capture_post(
complete_url,
json={
Expand Down Expand Up @@ -592,7 +596,7 @@ def gather_metadata(
binning_factor = 1
if environment:
server_config = requests.get(
f"{str(environment.url.geturl())}/instruments/{environment.instrument_name}/machine"
f"{str(environment.url.geturl())}{session_router.url_path_for('machine_info_by_instrument', instrument_name=environment.instrument_name)}"
).json()
if (
server_config.get("superres")
Expand Down
Loading
Loading