Odd IAM issue using GCP Pub/Sub Ruby SDK to publish messages

Ruby is a programming language that is popular in Google Cloud community. Recently, I got an odd IAM issue from a client using its GCP Pub/Sub SDK when publishing messages.

Like always, I ran a lab to show you.

I wrote a Ruby script to publish a message to Pub/Sub:

require "google/cloud/pubsub"

pubsub = Google::Cloud::PubSub.new(
  project_id: "derrick-sandbox",
  credentials: "test-pubsub-sa.json"
)

# Retrieve a topic
topic = pubsub.topic "test-topic"

# Publish a new message
msg = topic.publish "new-message2"

puts 'Sent a message to PubSub!'

I have two test projects: airflow-talk and derrick-sandbox.
To simulate your case, I am running my script with a service account from airflow-talk, while I created a PubSub topic in derrick-sandbox and granted permissions to the PubSub topic there.

I created a PubSub topic and subscription in project derrick-sandbox:

I created a service account in project airflow-talk:

I then granted only Pub/Sub Publisher (roles/pubsub.publisher) to this service account, which in theory, is good enough to publish the message to Pub/Sub:

Now back to airflow-talk project, in Cloud Shell, I was able to use gcloud to publish a message to the topic:
(Note that I did authorize my gcloud using the service account first)

➜  gcloud auth activate-service-account test-pubsub@airflow-talk.iam.gserviceaccount.com  --key-file=test-pubsub-sa.json --project=derrick-sandbox

➜  gcloud pubsub topics publish projects/derrick-sandbox/topics/test-topic --message="Hello World"                                                
messageIds:
- '8303462186509395'

But when running the Ruby script, it got an error:

➜  ruby-pubsub ruby test-pubsub.rb
Traceback (most recent call last):
        11: from test-pubsub.rb:9:in `<main>'
        10: from /home/derrick/.gems/gems/google-cloud-pubsub-2.15.4/lib/google/cloud/pubsub/project.rb:171:in `topic'
         9: from /home/derrick/.gems/gems/google-cloud-pubsub-2.15.4/lib/google/cloud/pubsub/service.rb:109:in `get_topic'
         8: from /home/derrick/.gems/gems/google-cloud-pubsub-v1-0.18.0/lib/google/cloud/pubsub/v1/publisher/client.rb:584:in `get_topic'
         7: from /home/derrick/.gems/gems/gapic-common-0.20.0/lib/gapic/grpc/service_stub.rb:190:in `call_rpc'
         6: from /home/derrick/.gems/gems/gapic-common-0.20.0/lib/gapic/grpc/service_stub/rpc_call.rb:123:in `call'
         5: from /home/derrick/.gems/gems/grpc-1.58.0-x86_64-linux/src/ruby/lib/grpc/generic/client_stub.rb:173:in `block in request_response'
         4: from /home/derrick/.gems/gems/grpc-1.58.0-x86_64-linux/src/ruby/lib/grpc/generic/interceptors.rb:170:in `intercept!'
         3: from /home/derrick/.gems/gems/grpc-1.58.0-x86_64-linux/src/ruby/lib/grpc/generic/client_stub.rb:174:in `block (2 levels) in request_response'
         2: from /home/derrick/.gems/gems/grpc-1.58.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:377:in `request_response'
         1: from /home/derrick/.gems/gems/grpc-1.58.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:186:in `attach_status_results_and_complete_call'
/home/derrick/.gems/gems/grpc-1.58.0-x86_64-linux/src/ruby/lib/grpc/generic/active_call.rb:29:in `check_status': 7:User not authorized to perform this action.. debug_error_string:{UNKNOWN:Error received from peer ipv4:142.251.10.95:443 {grpc_message:"User not authorized to perform this action.", grpc_status:7, created_time:"2023-09-20T06:40:27.019799881+00:00"}} (GRPC::PermissionDenied)
        4: from test-pubsub.rb:9:in `<main>'
        3: from /home/derrick/.gems/gems/google-cloud-pubsub-2.15.4/lib/google/cloud/pubsub/project.rb:171:in `topic'
        2: from /home/derrick/.gems/gems/google-cloud-pubsub-2.15.4/lib/google/cloud/pubsub/service.rb:109:in `get_topic'
        1: from /home/derrick/.gems/gems/google-cloud-pubsub-v1-0.18.0/lib/google/cloud/pubsub/v1/publisher/client.rb:551:in `get_topic'
/home/derrick/.gems/gems/google-cloud-pubsub-v1-0.18.0/lib/google/cloud/pubsub/v1/publisher/client.rb:589:in `rescue in get_topic': 7:User not authorized to perform this action.. debug_error_string:{UNKNOWN:Error received from peer ipv4:142.251.10.95:443 {grpc_message:"User not authorized to perform this action.", grpc_status:7, created_time:"2023-09-20T06:40:27.019799881+00:00"}} (Google::Cloud::PermissionDeniedError)

This is very strange. After digging into the stacktrace, I found it is related to this line:

# Retrieve a topic
topic = pubsub.topic "test-topic"

Apparently, this will make an API call to check if the topic exists, so it requires a Pub/Sub Viewer (roles/pubsub.viewer) IAM role. In SDKs from other languages, this is not the case.

Back to the IAM page, I added Pub/Sub Viewer (roles/pubsub.viewer) to my service account:

The script worked!

➜  ruby-pubsub ruby test-pubsub.rb
Sent a message to PubSub!