Skip to content

[Bug]: salt-api times out with 504 on get jobs/id route in 3007.14 #69134

@akowalsk

Description

@akowalsk

What happened?

Starting in 3007.14 (works just fine in 3007.13), I am no longer able to get responses to the GET /jobs/{id} route for any job:

SALT_API_URL="http://localhost:8000"
MINION_ID=<id>
TOKEN=<token>

RESPONSE=$(curl -v -X POST "${SALT_API_URL}/minions" \
  -H "Content-Type: application/json" \
  -H "Accept: application/json" \
  -H "X-Auth-Token: ${TOKEN}" \
  -d "[{\"tgt\":\"${MINION_ID}\",\"fun\":\"test.ping\"}]")

## retrieve the ID
JID=$(echo "${RESPONSE}" | jq -r '.return[0].jid')

## This will hang until gateway timeout
curl -s -X GET "${SALT_API_URL}/jobs/${JID}" \
      -H "Accept: application/json" \
      -H "X-Auth-Token: ${TOKEN}"

The response is:

< HTTP/1.1 504 Gateway Timeout
< Content-Type: text/html;charset=utf-8
< Server: CherryPy/unknown
< Date: Tue, 12 May 2026 16:58:12 GMT
< Allow: GET, HEAD, POST
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: GET, POST
< Access-Control-Allow-Credentials: true
< Cache-Control: private
< Vary: Accept-Encoding
< Content-Length: 771
< Set-Cookie: session_id=b6616716fdcfeb60ac57724288e8d2119ad64210; expires=Wed, 13 May 2026 02:58:12 GMT; Max-Age=36000; Path=/
<
<!DOCTYPE html PUBLIC
"-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html>
<head>
    <meta http-equiv="Content-Type" content="text/html; charset=utf-8"></meta>
    <title>504 Gateway Timeout</title>
    <style type="text/css">
    #powered_by {
        margin-top: 20px;
        border-top: 2px solid black;
        font-style: italic;
    }

    #traceback {
        color: red;
    }
    </style>
</head>
    <body>
        <h2>504 Gateway Timeout</h2>
        <p>The gateway server did not receive a timely response</p>
        <pre id="traceback"></pre>
    <div id="powered_by">
      <span>
        Powered by <a href="http://www.cherrypy.dev">CherryPy unknown</a>
      </span>
    </div>
    </body>
</html>

It also happens for other states that have return values, not just the ping state. In 3007.13, the response would have looked like this;

{"info": [{"jid": "20260512164945826646", "Function": "test.ping", "Arguments": [], "Target": "<minion_id>", "Target-type": "glob", "User": "salt", "Minions": ["<minion_id>"], "StartTime": "2026, May 12 16:49:45.826646", "Result": {"<minion_id>": {"return": true, "retcode": 0, "success": true}}}], "return": [{"<minion_id>": true}]}

Type of salt install

Official deb

Major version

3007.x

What supported OS are you seeing the problem on? Can select multiple. (If bug appears on an unsupported OS, please open a GitHub Discussion instead)

ubuntu-24.04

salt --versions-report output

Salt Version:
          Salt: 3007.14

Python Version:
        Python: 3.10.19 (main, Feb 17 2026, 23:32:03) [GCC 11.2.0]

Dependency Versions:
          cffi: 2.0.0
      cherrypy: unknown
  cryptography: 46.0.5
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: 4.0.12
     gitpython: 3.1.46
        Jinja2: 3.1.6
       libgit2: 1.3.0
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 24.0
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: 1.7.0
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.22.4
         smmap: 5.0.2
       timelib: 0.3.0
       Tornado: 6.5.5
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: ubuntu 24.04.4 noble
        locale: utf-8
       machine: x86_64
       release: 6.17.0-1007-aws
        system: Linux
       version: Ubuntu 24.04.4 noble

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugbroken, incorrect, or confusing behaviorneeds-triage

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions