using the auth library with grpc fails with Error: 7 PERMISSION_DENIED

[odedpeer]

this is a cross issue, see googleapis/google-auth-library-nodejs#2043

Link to the code that reproduces this issue.

A link to a public Github Repository or gist with a minimal reproduction.
https://gist.github.com/odedpeer/0aa4ca8b2efc928cb19806c5a129a5a4

A step-by-step description of how to reproduce the issue based on the linked reproduction.

• create a node project
• yarn add @googleapis/iam@28.0.1
• yarn add @google-cloud/logging@11.2.0
• create a service account with domain wide delegation as explained in https://developers.google.com/identity/protocols/oauth2/service-account#delegatingauthority
• set the service account email address and path to credentials file in the javascript file
• run the program

A clear and concise description of what the bug is, and what you expected to happen.

I am working on upgrading our Google SDK packages using the latest packages:

   "@googleapis/iam": "28.0.1",
   "google-auth-library": "10.1.0",
   "@google-cloud/logging": "11.2.0"

when I invoke the grpc call Logging.getEntries() from the @google-cloud/logging package it results in error
Error: 7 PERMISSION_DENIED: Method doesn't allow unregistered callers (callers without established identity). Please use API Key or other form of API consumer identity to call this API.

I expect to list all the log entries.

A clear and concise description WHY you expect this behavior

I was able to track this down to the following regression bug:

googleapis/google-auth-library-nodejs#1938 changed the getRequestMetadataAsync function from returning a JSON to returning an object
https://github.com/googleapis/google-auth-library-nodejs/pull/1938/files#diff-320fe75112224dd3e6d983b88d752cd3b6ab866ef7fcb8723cd30975b6a5f869R955

@grpc module converts the headers to a Metadata object by using Object.keys to fill the metadata object.
this used to work with JSON objects, but it does not work with Headers since the keys are not object properties.
https://github.com/grpc/grpc-node/blob/master/packages/grpc-js/src/call-credentials.ts#L130C42-L131C29

[odedpeer]

according to this discussion:

I think this is because @google-cloud/logging is still on auth 9 instead of 10 (it won't work with 10). https://github.com/googleapis does work with node 10. @cindy-peng, do you have an ETA of when logging will use 10?