-
Notifications
You must be signed in to change notification settings - Fork 10
Go: memory leakage queries #39
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
ex0dus-0x
wants to merge
4
commits into
main
Choose a base branch
from
alan/go_memleak
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+675
−0
Open
Changes from all commits
Commits
Show all changes
4 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,32 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "fmt" | ||
| "os" | ||
| ) | ||
|
|
||
| // BAD: defer Close inside a loop leaks file descriptors | ||
| func badReadFiles(paths []string) { | ||
| for _, path := range paths { | ||
| f, err := os.Open(path) | ||
| if err != nil { | ||
| continue | ||
| } | ||
| defer f.Close() // not closed until function returns | ||
| fmt.Println(f.Name()) | ||
| } | ||
| } | ||
|
|
||
| // GOOD: extract into a function so defer runs per iteration | ||
| func goodReadFiles(paths []string) { | ||
| for _, path := range paths { | ||
| func() { | ||
| f, err := os.Open(path) | ||
| if err != nil { | ||
| return | ||
| } | ||
| defer f.Close() // closed at end of this closure | ||
| fmt.Println(f.Name()) | ||
| }() | ||
| } | ||
| } |
42 changes: 42 additions & 0 deletions
42
go/src/security/DeferReleaseInLoop/DeferReleaseInLoop.qhelp
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| <!DOCTYPE qhelp PUBLIC | ||
| "-//Semmle//qhelp//EN" | ||
| "qhelp.dtd"> | ||
| <qhelp> | ||
| <overview> | ||
| <p> | ||
| In Go, <code>defer</code> schedules a function call to run when the <em>enclosing function</em> | ||
| returns — not when the enclosing block or loop iteration ends. Deferring a resource release | ||
| call (such as <code>Close</code>, <code>Unlock</code>, or <code>Rollback</code>) inside a loop | ||
| means that cleanup calls accumulate and only execute after the loop finishes and the function | ||
| returns. | ||
| </p> | ||
|
|
||
| <p> | ||
| This can lead to resource exhaustion: file descriptors pile up, database connections are held | ||
| open, locks are held longer than intended, or transactions remain open across iterations. | ||
| </p> | ||
|
|
||
| </overview> | ||
| <recommendation> | ||
| <p> | ||
| Extract the loop body into a separate function or closure so that <code>defer</code> runs | ||
| at the end of each iteration: | ||
| </p> | ||
|
|
||
| <sample src="DeferReleaseInLoop.go" /> | ||
|
|
||
| <p> | ||
| Alternatively, call the cleanup function directly without <code>defer</code> at the | ||
| appropriate point in the loop body. | ||
| </p> | ||
|
|
||
| </recommendation> | ||
| <references> | ||
| <li> | ||
| <a href="https://go.dev/ref/spec#Defer_statements">Go Language Specification — Defer statements</a> | ||
| </li> | ||
| <li> | ||
| <a href="https://blog.learngoprogramming.com/gotchas-of-defer-in-go-1-8d070894cb01">Gotchas of Defer in Go</a> | ||
| </li> | ||
| </references> | ||
| </qhelp> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,93 @@ | ||
| /** | ||
| * @name Deferred resource release in loop | ||
| * @id tob/go/defer-release-in-loop | ||
| * @description Deferring a resource release (Close, Rollback, etc.) inside a loop delays cleanup until the enclosing function returns, causing resource leaks across iterations such as file descriptor exhaustion or connection pool starvation. | ||
| * @kind problem | ||
| * @tags security | ||
| * @problem.severity warning | ||
| * @precision medium | ||
| * @security-severity 3.0 | ||
| * @group security | ||
| */ | ||
|
|
||
| import go | ||
|
|
||
| /** | ||
| * A function that acquires a resource that could leak | ||
| */ | ||
| class ResourceAcquisition extends Function { | ||
| ResourceAcquisition() { | ||
| this.hasQualifiedName("os", ["Open", "OpenFile", "Create", "CreateTemp", "NewFile", "Pipe"]) | ||
| or | ||
| this.hasQualifiedName("net", ["Dial", "DialTimeout", "Listen", "ListenPacket"]) | ||
| or | ||
| this.hasQualifiedName("net", ["FileConn", "FileListener", "FilePacketConn"]) | ||
| or | ||
| this.(Method).hasQualifiedName("net", "Dialer", ["Dial", "DialContext"]) | ||
| or | ||
| this.hasQualifiedName("net/http", ["Get", "Post", "PostForm", "Head"]) | ||
| or | ||
| this.(Method).hasQualifiedName("net/http", "Client", ["Do", "Get", "Post", "PostForm", "Head"]) | ||
| or | ||
| this.hasQualifiedName("crypto/tls", ["Dial", "DialWithDialer", "Client", "Server"]) | ||
| or | ||
| this.(Method).hasQualifiedName("crypto/tls", "Dialer", "DialContext") | ||
| or | ||
| this.hasQualifiedName("compress/gzip", ["NewReader", "NewWriter", "NewWriterLevel"]) | ||
| or | ||
| this.hasQualifiedName("compress/zlib", | ||
| ["NewReader", "NewWriter", "NewWriterLevel", "NewWriterLevelDict"]) | ||
| or | ||
| this.hasQualifiedName("compress/flate", ["NewReader", "NewWriter"]) | ||
| or | ||
| this.hasQualifiedName("compress/lzw", ["NewReader", "NewWriter"]) | ||
| or | ||
| this.hasQualifiedName("archive/zip", "OpenReader") | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Holds if `inner` is a (transitive) child of `outer` without crossing | ||
| * a function literal boundary. | ||
| */ | ||
| predicate parentWithoutFuncLit(AstNode inner, AstNode outer) { | ||
| inner.getParent() = outer and not inner instanceof FuncLit | ||
| or | ||
| exists(AstNode mid | | ||
| parentWithoutFuncLit(inner, mid) and | ||
| parentWithoutFuncLit(mid, outer) | ||
| ) | ||
| } | ||
|
|
||
| /** Holds if `node` is inside the body of `loop`, not crossing closures. */ | ||
| predicate inLoopBody(AstNode node, LoopStmt loop) { | ||
| parentWithoutFuncLit(node, loop.(ForStmt).getBody()) | ||
| or | ||
| parentWithoutFuncLit(node, loop.(RangeStmt).getBody()) | ||
| } | ||
|
|
||
| /** | ||
| * Gets the innermost identifier used as the receiver in `defer x.Close()`. | ||
| * For `defer f.Close()`, this is `f`. | ||
| * For `defer resp.Body.Close()`, this is `resp`. | ||
| */ | ||
| DataFlow::Node deferCloseReceiverBase(DeferStmt d) { | ||
| d.getCall().getTarget().getName() = "Close" and | ||
| exists(Expr base | base = d.getCall().getCalleeExpr().(SelectorExpr).getBase() | | ||
| // defer f.Close() — base is an identifier | ||
| result.asExpr() = base.(Ident) | ||
| or | ||
| // defer resp.Body.Close() — base is a selector, take its base identifier | ||
| result.asExpr() = base.(SelectorExpr).getBase().(Ident) | ||
|
Comment on lines
+80
to
+81
Member
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. thinking out loud: can selectors be nested? does this work for e.g. i think this doesn't matter for resources in the current list. |
||
| ) | ||
| } | ||
|
|
||
| from DeferStmt deferStmt, DataFlow::CallNode acquisition, LoopStmt loop | ||
| where | ||
| acquisition.getTarget() instanceof ResourceAcquisition and | ||
| inLoopBody(acquisition.asExpr(), loop) and | ||
| inLoopBody(deferStmt, loop) and | ||
| DataFlow::localFlow(acquisition.getResult(0), deferCloseReceiverBase(deferStmt)) | ||
| select deferStmt, | ||
| "Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations.", | ||
| acquisition, acquisition.getTarget().getName() + "()" | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,19 @@ | ||
| package main | ||
|
|
||
| import ( | ||
| "io" | ||
| "net/http" | ||
| ) | ||
|
|
||
| // BAD: unbounded read of request body | ||
| func badHandler(w http.ResponseWriter, r *http.Request) { | ||
| body, _ := io.ReadAll(r.Body) // no size limit — OOM on large request | ||
| w.Write(body) | ||
| } | ||
|
|
||
| // GOOD: limit body size before reading | ||
| func goodHandler(w http.ResponseWriter, r *http.Request) { | ||
| r.Body = http.MaxBytesReader(w, r.Body, 1<<20) // 1 MB limit | ||
| body, _ := io.ReadAll(r.Body) | ||
| w.Write(body) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| <!DOCTYPE qhelp PUBLIC | ||
| "-//Semmle//qhelp//EN" | ||
| "qhelp.dtd"> | ||
| <qhelp> | ||
| <overview> | ||
| <p> | ||
| Reading an HTTP request body with <code>io.ReadAll</code> (or the deprecated | ||
| <code>ioutil.ReadAll</code>) allocates the entire body into memory with no upper bound. | ||
| A malicious client can send an arbitrarily large request body to exhaust server memory, | ||
| causing a denial-of-service condition. | ||
| </p> | ||
|
|
||
| </overview> | ||
| <recommendation> | ||
| <p> | ||
| Wrap the request body with a size-limiting reader before reading it: | ||
| </p> | ||
|
|
||
| <sample src="UnboundedIORead.go" /> | ||
|
|
||
| <p> | ||
| Prefer <code>http.MaxBytesReader</code> which also sets the appropriate error on the | ||
| response, or <code>io.LimitReader</code> for non-HTTP contexts. | ||
| </p> | ||
|
|
||
| </recommendation> | ||
| <references> | ||
| <li> | ||
| <a href="https://pkg.go.dev/net/http#MaxBytesReader">http.MaxBytesReader documentation</a> | ||
| </li> | ||
| <li> | ||
| <a href="https://pkg.go.dev/io#LimitReader">io.LimitReader documentation</a> | ||
| </li> | ||
| </references> | ||
| </qhelp> |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,101 @@ | ||
| /** | ||
| * @name Unbounded read of request body | ||
| * @id tob/go/unbounded-io-read | ||
| * @description Reading an HTTP request body with `io.ReadAll` or `ioutil.ReadAll` without a size limit allows a malicious client to exhaust server memory with an arbitrarily large request. | ||
| * @kind path-problem | ||
| * @tags security | ||
| * @problem.severity error | ||
| * @precision high | ||
| * @security-severity 7.5 | ||
| * @group security | ||
| */ | ||
|
|
||
| import go | ||
|
|
||
| /** | ||
| * An `io.ReadAll` or `ioutil.ReadAll` call — functions that read an entire | ||
| * reader into memory with no size bound. | ||
| */ | ||
| class UnboundedReadCall extends DataFlow::CallNode { | ||
| UnboundedReadCall() { | ||
| this.getTarget().hasQualifiedName("io", "ReadAll") | ||
| or | ||
| this.getTarget().hasQualifiedName("io/ioutil", "ReadAll") | ||
| or | ||
| this.getTarget().hasQualifiedName("ioutil", "ReadAll") | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * A source node: an HTTP request body that can be controlled by an attacker. | ||
| * | ||
| * Matches `r.Body` where `r` is an `*http.Request`. | ||
| */ | ||
| class HTTPRequestBodySource extends DataFlow::Node { | ||
| HTTPRequestBodySource() { | ||
| exists(Field f, SelectorExpr sel | | ||
| f.hasQualifiedName("net/http", "Request", "Body") and | ||
| sel.getSelector().refersTo(f) and | ||
| this.asExpr() = sel | ||
| ) | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * A call that wraps a reader with a size limit, acting as a sanitizer. | ||
| */ | ||
| class SizeLimitingCall extends DataFlow::CallNode { | ||
| SizeLimitingCall() { | ||
| this.getTarget().hasQualifiedName("net/http", "MaxBytesReader") | ||
| or | ||
| this.getTarget().hasQualifiedName("io", "LimitReader") | ||
| } | ||
| } | ||
|
|
||
| /** | ||
| * Holds if `r.Body` is reassigned from a size-limiting call in the same function | ||
| * that `bodyRead` is in, on the same request variable `r`. | ||
| */ | ||
| predicate bodyLimitedByReassignment(SelectorExpr bodyRead) { | ||
| exists(SizeLimitingCall limiter, Assignment assign, SelectorExpr lhs, Variable v | | ||
| // The assignment target is `v.Body` | ||
| assign.getLhs() = lhs and | ||
| lhs.getSelector().getName() = "Body" and | ||
| lhs.getBase().(Ident).refersTo(v) and | ||
| // The RHS is a MaxBytesReader/LimitReader call | ||
| assign.getRhs() = limiter.asExpr() and | ||
| // The body read is on the same variable: `v.Body` | ||
| bodyRead.getBase().(Ident).refersTo(v) and | ||
| // Both in the same function | ||
| assign.getEnclosingFunction() = bodyRead.getEnclosingFunction() | ||
| ) | ||
| } | ||
|
|
||
| module UnboundedReadConfig implements DataFlow::ConfigSig { | ||
| predicate isSource(DataFlow::Node source) { | ||
| source instanceof HTTPRequestBodySource and | ||
| // Exclude r.Body reads where r.Body was reassigned from a size limiter | ||
| not bodyLimitedByReassignment(source.asExpr()) | ||
| } | ||
|
|
||
| predicate isSink(DataFlow::Node sink) { | ||
| exists(UnboundedReadCall readAll | sink = readAll.getArgument(0)) | ||
| } | ||
|
|
||
| predicate isBarrier(DataFlow::Node node) { | ||
| // If the body passes through a size-limiting wrapper (io.LimitReader pattern), it is safe | ||
| node = any(SizeLimitingCall c).getResult(0) | ||
| or | ||
| node = any(SizeLimitingCall c).getResult() | ||
| } | ||
| } | ||
|
|
||
| module UnboundedReadFlow = DataFlow::Global<UnboundedReadConfig>; | ||
|
|
||
| import UnboundedReadFlow::PathGraph | ||
|
|
||
| from UnboundedReadFlow::PathNode source, UnboundedReadFlow::PathNode sink | ||
| where UnboundedReadFlow::flowPath(source, sink) | ||
| select sink.getNode(), source, sink, | ||
| "Unbounded read of HTTP request $@ with no size limit allows denial of service via memory exhaustion.", | ||
| source.getNode(), "body" |
14 changes: 14 additions & 0 deletions
14
go/test/query-tests/security/DeferReleaseInLoop/DeferReleaseInLoop.expected
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,14 @@ | ||
| | DeferReleaseInLoop.go:22:3:22:17 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:18:13:18:25 | call to Open | Open() | | ||
| | DeferReleaseInLoop.go:34:3:34:17 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:30:13:30:52 | call to Create | Create() | | ||
| | DeferReleaseInLoop.go:46:3:46:17 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:42:13:42:45 | call to OpenFile | OpenFile() | | ||
| | DeferReleaseInLoop.go:58:3:58:17 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:54:13:54:39 | call to CreateTemp | CreateTemp() | | ||
| | DeferReleaseInLoop.go:70:3:70:25 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:66:16:66:28 | call to Get | Get() | | ||
| | DeferReleaseInLoop.go:83:3:83:25 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:79:16:79:29 | call to Do | Do() | | ||
| | DeferReleaseInLoop.go:95:3:95:20 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:91:16:91:36 | call to Dial | Dial() | | ||
| | DeferReleaseInLoop.go:107:3:107:18 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:103:14:103:40 | call to Listen | Listen() | | ||
| | DeferReleaseInLoop.go:124:3:124:18 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:119:14:119:30 | call to NewReader | NewReader() | | ||
| | DeferReleaseInLoop.go:125:3:125:17 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:115:13:115:25 | call to Open | Open() | | ||
| | DeferReleaseInLoop.go:142:3:142:18 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:137:14:137:30 | call to NewReader | NewReader() | | ||
| | DeferReleaseInLoop.go:143:3:143:17 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:133:13:133:25 | call to Open | Open() | | ||
| | DeferReleaseInLoop.go:155:3:155:18 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:151:14:151:33 | call to OpenReader | OpenReader() | | ||
| | DeferReleaseInLoop.go:168:4:168:18 | defer statement | Deferred Close() of resource acquired from $@ in a loop will not execute until the function returns, leaking resources across iterations. | DeferReleaseInLoop.go:164:14:164:26 | call to Open | Open() | |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can this be parameterized with MaD?