Migrate Airavata to Spring Boot, Simplify Data Models

[yasithdev]

…ecture

Migrate the Airavata platform from legacy Thrift/OpenJPA architecture to a modern Spring Boot application with clean service layer boundaries.

Key changes:

• Replace Thrift RPC with Spring Boot REST API (Swagger UI at /swagger-ui)
• Replace OpenJPA with Spring Data JPA (Hibernate) and MapStruct mappers
• Introduce interface-first service layer (FooService + DefaultFooService)
• Add generic CrudService/AbstractCrudService/EntityMapper infrastructure
• Restructure packages into domain boundaries: research/, execution/, compute/, storage/, status/, iam/, credential/, workflow/, protocol/
• Implement Temporal-based durable workflows (ProcessActivity: Pre/Post/Cancel)
• Add DAG-based task execution engine (ProcessDAGEngine)
• Consolidate compute backends into ComputeProvider interface (Slurm/AWS/Local)
• Consolidate storage operations into StorageClient interface (SFTP)
• Replace log4j with logback, Dozer with MapStruct
• Add Flyway for schema migrations
• Simplify distribution into single Spring Boot module
• Remove deprecated Python SDK, PHP SDK, and Thrift IDL definitions
• Streamline dev environment with Docker Compose and init scripts

In general, the fix is to explicitly define a permissions block that grants only the minimal required scopes to the GITHUB_TOKEN, instead of relying on repository/organization defaults. For a build-and-test-only Maven workflow that just checks out code and runs mvnw, contents: read is sufficient in most cases.

The best fix here, without changing existing functionality, is to add a permissions block at the workflow root (top level, alongside on: and jobs:) so it applies to all jobs that don’t override it. Based on the current steps (checkout and Maven build), the job only needs to read repository contents, so we set:

permissions:
  contents: read

Insert this block after the on: section and before jobs: in .github/workflows/maven-build.yml. No additional imports, methods, or definitions are needed.

In general, to fix this kind of issue you must not let arbitrary user input determine where your server connects. Instead, constrain the destination to a controlled set of hosts (and possibly ports), e.g. via an allow-list of known SFTP servers or by validating against a configuration-derived list. Also validate port numbers to ensure they’re within acceptable ranges.

For this specific code, the safest fix without changing the endpoint’s basic behavior is:

1. Introduce an allow-list of permitted SFTP hosts (e.g. from environment variables), and reject any host that does not match.
2. Strictly validate port:
   • Ensure it is a number.
   • Restrict it to a sensible range (for SFTP usually 22, or at least 1–65535 with the option to narrow further, e.g. only 22).
3. Keep using testPort but only after validation passes.

Concretely in airavata-portal/src/app/api/v1/connectivity-test/sftp/route.ts:

• Add a helper isHostAllowed(host: string): boolean that:
  • Loads an allow-list from an environment variable like SFTP_ALLOWED_HOSTS (comma-separated).
  • Normalizes hostnames (trim, toLowerCase).
  • Returns true only if the requested host is in that list.
• In the POST handler, after extracting host and port from body:
  • Validate typeof host === "string" and that isHostAllowed(host) is true; otherwise respond with 400.
  • Convert port to a number (if not already), ensure it’s an integer in [1, 65535], and optionally restrict to 22; otherwise respond with 400.
  • Use the validated validatedPort when calling testPort.
• Keep the testPort function unchanged except that it will now only receive validated values.

This keeps the existing functionality (testing SFTP connectivity) but removes the ability for arbitrary external callers to direct your server at arbitrary internal addresses.

In general, fix SSRF here by constraining which hosts and ports the API can connect to, instead of using user input directly. Common mitigations include:

• Using an allow‑list of permitted hosts (or patterns like domain suffixes).
• Restricting ports to a bounded, expected set (e.g., only 22 and 6817, or a small range).
• Optionally rejecting obvious internal/loopback addresses if this endpoint is publicly reachable.

For this specific code, the minimal fix without changing overall functionality is:

1. Add a small validation layer in the POST handler:
   • Validate host against a strict pattern (e.g., hostname or IP characters).
   • Restrict sshPort and slurmPort to a safe numeric range and, if appropriate, a small set of expected values.
2. Optionally, extract a validatePort helper to centralize port checks.

We only touch airavata-portal/src/app/api/v1/connectivity-test/slurm/route.ts. Concretely:

• After parsing body, validate host and ports:
  • Reject missing or invalid host.
  • Ensure ports are integers within 1–65535, and, if desired, only allow 22 and 6817 or a short allow‑list.
• Return 400 with an explanatory message when validation fails.
• Leave testPort and socket.connect unchanged; the taintedness is removed because we no longer pass untrusted values—only validated/sanitized ones.

This addresses both variants because the only data that reaches socket.connect will have passed validation.

In general, to fix this kind of issue you must prevent untrusted clients from arbitrarily controlling the target of outgoing connections. This is usually done by (a) restricting the host to an allowlist of known-safe values or domains, and/or (b) validating that the host is not an internal/loopback/multicast/etc. address; and (c) constraining the port to an expected, small range (e.g., 22 only, or safe SSH ports).

For this specific code, the minimal fix without changing overall behavior too much is:

• Validate that host is a syntactically valid hostname or IP address.
• Resolve host to an IP, and reject it if it’s loopback, private, link-local, multicast, or otherwise internal.
• Validate that port is an integer in an allowed range, e.g., 1–65535 and optionally only SSH-like ports (e.g., 22 and maybe a few others).
• Only then call testPort with the validated values.

We can implement this entirely within airavata-portal/src/app/api/v1/connectivity-test/ssh/route.ts by:

1. Adding a small helper isIpPrivate using Node’s built-in net module (already imported as net) and dns.promises.lookup to resolve hostnames to IP addresses.
2. Adding validation logic in POST:
   • Parse and sanitize port into a number and check its range.
   • Check that host is a string and passes basic hostname/IP regex.
   • Use dns.promises.lookup plus isIpPrivate to block private/loopback/etc. IPs.
3. Use the validated safeHost and safePort when calling testPort.

This directly addresses CodeQL’s complaint by ensuring the value flowing into socket.connect is not an arbitrary user-controlled network destination.

In general, this problem is fixed by not using AutoAddPolicy/WarningPolicy and instead either relying on Paramiko’s default RejectPolicy or explicitly using RejectPolicy, combined with ensuring that the host’s key is already known (e.g., via system/user known_hosts or by programmatically loading expected keys). This preserves host key verification and prevents silent acceptance of unknown hosts.

For this specific test, the least invasive fix is to stop using AutoAddPolicy and use RejectPolicy while ensuring host keys are loaded from the environment. Since these tests connect to containers started by the test harness, it is reasonable to assume standard SSH host key management on the test machine. Concretely, we can remove the explicit set_missing_host_key_policy call (letting Paramiko use its default RejectPolicy), and ensure the client loads known host keys by calling ssh.load_system_host_keys() and ssh.load_host_keys from the user’s known_hosts file before connecting. This keeps the functional behavior (connect to the container and run commands) while aligning with secure defaults; if the host is unknown, the test should fail explicitly rather than silently accepting the key.

Changes are confined to deployment/ansible/tests/test_base_role.py in the _test_base_role function around lines 64–71. We will:

• Add calls to ssh.load_system_host_keys() and ssh.load_host_keys(str(Path.home() / ".ssh" / "known_hosts")) after creating the SSHClient.
• Remove the ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) line so we no longer weaken the host key policy.
  No new imports are needed; Path is already imported from pathlib, and paramiko is already imported.

In general, the fix is to avoid AutoAddPolicy and ensure that Paramiko either (a) uses the secure default RejectPolicy or (b) explicitly sets RejectPolicy and loads known host keys (from a file or in-memory mapping). This preserves host authenticity checks and causes the connection attempt to fail if the host key is not recognized, instead of silently accepting a potentially spoofed host.

For this specific test, the minimal change that keeps behavior as close as possible while improving security is to stop using AutoAddPolicy. Since SSHClient already defaults to RejectPolicy, the simplest fix is to remove the set_missing_host_key_policy call entirely. If you want to be explicit, you can set paramiko.RejectPolicy() instead. Given the constraints (only editing the shown snippet, no assumptions about the rest of the project), the best fix is to replace ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy()) with ssh.set_missing_host_key_policy(paramiko.RejectPolicy()). This keeps the test logic unchanged except that it will now fail fast if the container’s SSH host key is not known/acceptable.

Concretely:

• In deployment/ansible/tests/test_database_role.py, at the creation of the SSHClient in _test_database_role, replace the AutoAddPolicy call with RejectPolicy.
• No new imports are necessary because the file already imports paramiko.
• No other code changes are required; the connection and command execution logic remain untouched.

To fix this, we should ensure that the value taken from UserContext.gatewayId() cannot cause the server to make arbitrary outbound requests. Because we cannot change the broader architecture, the least intrusive and clearest fix is to validate and normalize the gatewayId value inside fetchExperimentStorageFromAPI before using it in the URL. That keeps existing behavior for valid gateway IDs while preventing malformed or attacker-controlled values from being used as hostnames.

Concretely:

1. In AgentFileService.fetchExperimentStorageFromAPI:

   • Read UserContext.gatewayId() into a local variable.
   • Validate that it is non-null, non-empty, and matches a strict pattern for allowed gateway identifiers (e.g., lowercase letters, digits, and dashes only), and does not start or end with -. This prevents characters like ., /, :, etc., from influencing the hostname beyond the intended subdomain and avoids obvious malformed labels.
   • Optionally, also enforce a reasonable length limit.
   • If validation fails, throw an IllegalArgumentException (or similar) to stop the request rather than making an external HTTP call.
   • Use the validated value to construct the URL as before.

2. The code change is localized to airavata-api/modules/grpc-api/src/main/java/org/apache/airavata/grpc/service/AgentFileService.java, inside the fetchExperimentStorageFromAPI method. No external dependencies are needed; we can use String.matches with a regex or a simple manual check.

3. No changes are required in AuthzTokenFilter, AuthzToken, or UserContext to mitigate the SSRF; they may remain as-is, since we are sanitizing at the point where the value influences an HTTP request.

General approach: constrain the untrusted realm/gatewayId input before it is used to build the Keycloak URL, by (a) enforcing that it is a simple, safe identifier (no scheme/host, no slashes or whitespace), and ideally (b) restricting it to a set or pattern of allowed realm names (e.g., matching known gateways plus "master"). Since we cannot see or modify the broader gateway registry here, the minimal robust fix is to add a local validator in KeycloakRestClient that enforces a conservative pattern for realm names and use it before building the URL. This cuts the taint chain at the point where it matters for SSRF without changing external behavior for legitimate realm IDs.

Concrete single best fix:

• In KeycloakRestClient, add a private helper, e.g. sanitizeRealm(String realm), that:
  • Rejects null or blank values with an IamAdminServicesException.
  • Trims whitespace.
  • Ensures the realm matches a safe regex such as ^[A-Za-z0-9._-]+$ (letters, digits, dot, underscore, dash), preventing injection of /, :, ?, #, etc.
  • Optionally logs a warning when rejecting.
• In obtainAdminToken(String realm, PasswordCredential credentials):
  • Call sanitizeRealm(realm) at the start and use the sanitized value for both the cache key and path segment.
  • This ensures even if upstream code passes a malicious or malformed realm from token claims or request parameters, it cannot influence the URL beyond a constrained path segment on the configured Keycloak host.

This change is local to KeycloakRestClient.obtainAdminToken, which is the sink for all 14 variants. It doesn’t alter the configured serverUrl or the contract of upstream services; it only rejects invalid realm values early and with a clear error.

Implementation details:

• File to modify: airavata-api/modules/airavata-api/src/main/java/org/apache/airavata/iam/keycloak/KeycloakRestClient.java.

• Add the sanitizeRealm method near other private helpers (e.g., close to loadKeyStore / createTrustManagers), using IamAdminServicesException (already imported).

• Update obtainAdminToken to:

  public String obtainAdminToken(String realm, PasswordCredential credentials) throws IamAdminServicesException {
      String safeRealm = sanitizeRealm(realm);
      var cacheKey = safeRealm + ":" + credentials.getLoginUserName();
      ...
      var tokenUrl = UriComponentsBuilder.fromUriString(serverUrl)
              .pathSegment("realms", safeRealm, "protocol", "openid-connect", "token")
              .toUriString();
      ...
  }

• No new imports are needed; regex is done via String.matches, which is part of java.lang.String.

This single change ensures that all dataflow paths that reach obtainAdminToken (from request parameters, bodies, and token claims) are validated before being used to construct the outbound HTTP request, satisfying the SSRF mitigation guidance.

---

In general, the fix is to stop disabling CSRF globally and instead either (a) leave Spring’s default CSRF protection enabled, or (b) selectively ignore CSRF for endpoints that are strictly stateless API calls (e.g. /api/**) while keeping it for any browser-based or form-login flows. For a JWT-based stateless API, the usual pattern is to configure csrf with a matcher that ignores API paths, rather than calling disable().

The minimal change that preserves existing functionality is: replace .csrf(csrf -> csrf.disable()) with a configuration that tells Spring Security to ignore CSRF for the API endpoints, leaving CSRF protection in place for anything else (present or future) that might rely on cookies or form logins. For example, configure csrf using csrf.ignoringRequestMatchers("/api/**"). This keeps the current behaviour for the JWT-protected /api/v1/** endpoints (no CSRF token required, so clients won’t break) while removing the global disabling that CodeQL is flagging. No other logic in securityFilterChain needs to change, and no new methods or imports are strictly required, because ignoringRequestMatchers is available on Spring Security’s CsrfConfigurer and accepts String ant-style patterns.

Concretely, in airavata-api/modules/airavata-api/src/main/java/org/apache/airavata/config/WebSecurityConfiguration.java, update the line:

http.csrf(csrf -> csrf.disable())

to:

http.csrf(csrf -> csrf.ignoringRequestMatchers("/api/**"))

and leave the rest of the method as is. This confines the CSRF exemption to the API paths, allowing Spring’s CSRF mechanism to remain available for any non-API, browser-facing endpoints now or in the future.

In general, the fix is to stop using a custom X509TrustManager that does not validate certificates, and instead rely on the platform’s default trust managers or a TrustManager created from a KeyStore that contains only the certificates you explicitly trust. The checkServerTrusted implementation must perform real certificate chain validation, typically by delegating to a properly initialized TrustManagerFactory, rather than silently accepting all certificates.

The best fix here, without changing external behavior beyond removing the insecure mode, is to remove the anonymous trust‑all X509TrustManager and replace createTrustAllSSLContext with an implementation that builds an SSLContext using the default trust configuration. This uses TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm()), initializes it with a null keystore (meaning: use system/default trust store), obtains its trust managers, and passes them into SSLContext.init. This preserves functionality for normal, valid certificates while closing the vulnerability created by accepting any certificate. Concretely, within RestClientConfiguration.java, only lines 122–139 (the body of createTrustAllSSLContext) need to be replaced; no new imports are required because TrustManagerFactory and related classes are already imported at the top of the file.