Skip to content

add proxy polling restriction, allow to query "/clients/host"#178

Open
peekjef72 wants to merge 1 commit intoprometheus-community:masterfrom
peekjef72:master
Open

add proxy polling restriction, allow to query "/clients/host"#178
peekjef72 wants to merge 1 commit intoprometheus-community:masterfrom
peekjef72:master

Conversation

@peekjef72
Copy link

@peekjef72 peekjef72 commented May 26, 2024 •
edited
Loading

Add in the proxy part the ability to specify which networks (or servers) can request the proxy.
This allows you to define two roles:

  • pollers: The pollers, generally the prometheus hosts, can scrap target through the proxy.
  • targets.: they can only register on the proxy and respond to requests from it.

The feature makes it possible to fill the security hole which allows in the current version two neighboring hosts, even if they cannot communicate directly, to obtain metrics from each other or the list of all hosts known by the proxy.

To enable the feature; add in the command line interface of the pushprox_proxy:

--scrape.pollers-ip 'ipV4/32,netV4#2/net,...

example: start pushproxy proxy and client on same server:

  • proxy listen on every IP on port 8080 (default), polling only authorized through loopback
  • client accessible through fqdn "localhost" connected to proxy on server ip (192.168.0.196:8080)
$ ./pushprox-proxy --scrape.pollers-ip 127.0.0.1/32 &
$ ./pushprox-client --fqdn=localhost --proxy-url=http://192.168.0.196:8080 &

Try to scrap an exporter listening on port 9321 on same server through server ip: reply "403 Forbidden" and in body "Not an authorized poller"

$ curl -v --proxy http://192.168.0.196:8080 -H "X-Prometheus-Scrape-Timeout-Seconds: 5" "http://localhost:9321"
* Rebuilt URL to: http://localhost:9321/
*   Trying 192.168.0.196...
* TCP_NODELAY set
* Connected to 192.168.0.196 (192.168.0.196) port 8080 (#0)
> GET http://localhost:9321/ HTTP/1.1
> Host: localhost:9321
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
> X-Prometheus-Scrape-Timeout-Seconds: 5
> 
ts=2024-05-26T09:53:37.089Z caller=main.go:288 level=error msg="can't reverse client address" err="lookup 196.0.168.192.in-addr.arpa. on 192.168.0.1:53: no such host"
< HTTP/1.1 403 Forbidden
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Sun, 26 May 2024 09:53:37 GMT
< Content-Length: 25
< 
Not an authorized poller
* Connection #0 to host 192.168.0.196 left intact

try to scrap httpapi_exporter through loopback ip :

$ curl -v --proxy http://127.0.0.1:8080 -H "X-Prometheus-Scrape-Timeout-Seconds: 5" "http://localhost:9321"
* Rebuilt URL to: http://localhost:9321/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET http://localhost:9321/ HTTP/1.1
> Host: localhost:9321
> User-Agent: curl/7.61.1
> Accept: */*
> Proxy-Connection: Keep-Alive
> X-Prometheus-Scrape-Timeout-Seconds: 5
> 
ts=2024-05-26T10:03:02.010Z caller=coordinator.go:122 level=info msg=DoScrape scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d url=http://localhost:9321/
ts=2024-05-26T10:03:02.010Z caller=main.go:208 level=info msg="Responded to /poll" url=http://localhost:9321/ scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d
ts=2024-05-26T10:03:02.010Z caller=main.go:206 level=info msg="Got scrape request" scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d url=http://localhost:9321/
ts=2024-05-26T10:03:02.011Z caller=coordinator.go:143 level=info msg=WaitForScrapeInstruction fqdn=localhost
ts=2024-05-26T10:03:02.011Z caller=main.go:140 level=info scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d msg="Retrieved scrape response"
ts=2024-05-26T10:03:02.012Z caller=main.go:189 level=info msg="Got /push" scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d
ts=2024-05-26T10:03:02.012Z caller=coordinator.go:175 level=info msg=ScrapeResult scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d
ts=2024-05-26T10:03:02.012Z caller=main.go:146 level=info scrape_id=cffa2eb0-e2d5-4b33-9753-cc430602723d msg="Pushed scrape result"
< HTTP/1.1 200 OK
< Content-Type: text/html; charset=utf-8
< Date: Sun, 26 May 2024 10:03:02 GMT
< X-Prometheus-Scrape-Timeout: 4.998953
< Transfer-Encoding: chunked
< 
<html>
      <head>
        <title>Prometheus httpapi_exporter</title>
        <style type="text/css">
          body { margin: 0; font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; font-size: 14px; line-height: 1.42857143; color: #333; background-color: #fff; }
          .navbar { display: flex; background-color: #222; margin: 0; border-width: 0 0 1px; border-style: solid; border-color: #080808; }
....

Add the feature to query if a specified fqdn can be found the the fqdn list of knwown hosts
two cases:

  • Poller role is defined: only pollers are authorized to access the full clients lists,
  • poller is not defined: as usuall the full list is available.

e.g.: poller role is not defined.

  • can query the full list /clients
  • can query a specfic fqdn /client/fqdn : if not found return 404
$ curl -v http://192.168.0.196:8080/clients/
*   Trying 192.168.0.196...
* TCP_NODELAY set
* Connected to 192.168.0.196 (192.168.0.196) port 8080 (#0)
> GET /clients/ HTTP/1.1
> Host: 192.168.0.196:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:22:05 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 192.168.0.196 left intact

look for fqdn "localhost":

$ curl -v http://192.168.0.196:8080/clients/localhost
*   Trying 192.168.0.196...
* TCP_NODELAY set
* Connected to 192.168.0.196 (192.168.0.196) port 8080 (#0)
> GET /clients/localhost HTTP/1.1
> Host: 192.168.0.196:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:22:36 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 192.168.0.196 left intact

poller role is defined ( 127.0.0.1:8080 only )

$ curl -v http://127.0.0.1:8080/clients/
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /clients/ HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:27:10 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 127.0.0.1 left intact

$ curl -v http://127.0.0.1:8080/clients/localhost
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /clients/localhost HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 200 OK
< Content-Type: application/json
< Date: Sun, 26 May 2024 10:27:19 GMT
< Content-Length: 42
< 
[{"targets":["localhost"],"labels":null}]
* Connection #0 to host 127.0.0.1 left intact


$ curl -v http://127.0.0.1:8080/clients/host1
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /clients/host1 HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.61.1
> Accept: */*
> 
< HTTP/1.1 404 Not Found
< Content-Type: text/plain; charset=utf-8
< X-Content-Type-Options: nosniff
< Date: Sun, 26 May 2024 10:27:24 GMT
< Content-Length: 1
< 

* Connection #0 to host 127.0.0.1 left intact

SuperQ
SuperQ requested changes Nov 8, 2024
View reviewed changes
Copy link
Contributor

@SuperQ SuperQ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove commented out code.

Also, this needs a rebase with the change to the new logging library.

@peekjef72
add proxy polling restriction
2ff6709 
Signed-off-by: peekjef72 <jfpik78@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@SuperQ SuperQ SuperQ requested changes

Requested changes must be addressed to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@peekjef72 @SuperQ