How to implement API auth?

[frederikheld]

Hey,

I have a question about how my Jobs get authorized to access my protected REST API.

My first approach was IP whitelisting that allows certain IP's to bypass auth. This turned out to be brittle: a wrong configuration can either block all access or allow the whole world to access it. It might also become difficult to handle if I start to scale runners and each has their own IP.

So now I'm going with service tokens. When a Job is enqueued by my backend, the backend mints a token that gets passed as parameter into the Job. The job can use this token to exchange it for a auth token. This allows me to treat the whole API as public and use the same auth-mechanism for services as I use for human users. The caveat is that - for security reasons - a service token can be exchanged into a access token exactly once.

This creates the issue that I can't re-run jobs from the Sidequest admin dashboard. The job itself is idempotent, but the auth mechanism is not.

It's not really a problem as I'm planning to implement a mechanism to re-run jobs in my Admin app.

But I'm still thinking if there's a better way to do this that is secure and convenient at the same time.

So how are you guys doing this?

[GiovaniGuizzo]

Hey @frederikheld. Not sure how your use case is, but maybe you can add to the job itself a step to get a new access token on every retry?
Then, even when you re-run jobs, they get a new access token and start anew.

[frederikheld]

I could do that. But the point of those tokens is that only the API server knows the secret to mint them and thus decides who gets access. If the runner can do that, I could have kept the whitelist approach.

Runner and API server are two entirely different entities in my distributed system.

[GiovaniGuizzo]

Got it. Then there is no way around it because Sidequest saves the parameters on the database. You could update the job with a new token, but not though the Dashboard.

[frederikheld]

Great idea to update the job instead of creating a new one with the same parameters 🤯 I didn't think about that.

I'm gonna need to create a dashboard that lists the jobs for each tenant anyway, so adding the re-run functionality won't be much extra work.

Thanks for the inspiration!

[GiovaniGuizzo]

NP :)