Unresolved include directive in modules/ROOT/pages/_attributes.adoc - include::../_attributes.adoc[]

Configuring the Salesforce PubSub API Source Connector

Overview

The Salesforce PubSub API Source Connector configuration options can be divided into the following groups.

  • Authentication
    The settings to authenticate against the Salesforce instance and connect to the Pub/Sub API.

  • Topic Routing and Replay
    Settings to define which Salesforce topics to subscribe to and how to map them to Kafka topics.

  • Performance Tuning
    Settings to control internal buffers, batch sizes, and polling timeouts to tune throughput and latency.

  • Connection Reliability (gRPC)
    Low-level gRPC settings to manage connection liveliness and timeouts.

  • Advanced Configuration
    Advanced settings for schema caching and connection validation.

Authentication

These settings are required to establish a secure connection to Salesforce using OAuth 2.0 and to locate the correct Pub/Sub endpoint.

Key Type Mandatory Default Description

salesforce.client.id

String

Yes

-

The Salesforce OAuth Client ID (also known as the Connected App Consumer Key).

salesforce.client.secret

Password

Yes

-

The Salesforce OAuth Client Secret (also known as the Connected App Consumer Secret).

salesforce.instance.url

String

Yes

-

The Salesforce Instance URL (e.g., https://your-domain.my.salesforce.com).

salesforce.tenant.id

String

Yes

-

The Salesforce Organization ID (Tenant ID). This ID typically starts with 00D.

salesforce.pubsub.endpoint

String

No

api.pubsub.salesforce.com:7443

The Salesforce Pub/Sub API gRPC Endpoint in host:port format.

Topic Routing and Replay

These settings determine which data is ingested from Salesforce, how the Kafka topic names are generated, and where to start reading the stream.

Key Type Mandatory Default Description

salesforce.topics

List

Yes

-

A comma-separated list of Salesforce topics to subscribe to.
Example: /event/MyEvent__e,/data/Account. Note: The number of the topic should be less or equal to the number of the tasks as the connector is using i task per topic and is not able to use one task for two or more topics.

topic.prefix

String

Yes

-

The prefix to use for the destination Kafka topic names.
The Salesforce topic path is converted by replacing slashes with dots and prepending this prefix.
For example, if the prefix is salesforce. and the source is /event/MyEvente, the Kafka topic will be salesforce.event.MyEvente.

salesforce.replay.preset

String

No

EARLIEST

The starting position for new subscriptions (when no stored offset exists).
Valid values are:
* EARLIEST: Starts from the beginning of the retained event stream.
* LATEST: Starts from the current tip, skipping historical events.

Performance Tuning

These settings allow you to tune the internal behavior of the connector to balance memory usage, latency, and throughput.

Key Type Mandatory Default Description

salesforce.batch.size

Integer

No

100

The maximum number of records to return per poll() call.
Salesforce supports a maximum of 10,000 events per request.

salesforce.queue.capacity

Integer

No

1000

The maximum number of events to buffer in memory before pausing the gRPC consumer.
Increase this for high-volume topics, or decrease for memory-constrained environments.

salesforce.poll.timeout.ms

Long

No

1000

The maximum time in milliseconds to wait for events during a poll() operation.

Connection Reliability (gRPC)

These settings control the underlying gRPC channel behavior, specifically regarding keep-alive pings to prevent firewalls or load balancers from dropping idle connections.

Key Type Mandatory Default Description

salesforce.grpc.keepalive.time.seconds

Integer

No

30

The time interval in seconds between keep-alive pings sent to the server.

salesforce.grpc.keepalive.timeout.seconds

Integer

No

10

The time in seconds to wait for a keep-alive ping response before considering the connection dead.

salesforce.grpc.shutdown.timeout.seconds

Integer

No

5

The time in seconds to wait for a graceful channel shutdown before forcing termination.

Advanced Configuration

Key Type Mandatory Default Description

salesforce.schema.cache.size

Integer

No

100

The maximum number of Avro schemas to cache in memory.
Increase this if your environment uses a large number of distinct schema versions.

salesforce.validate.connection

Boolean

No

true

Whether to validate Salesforce connectivity during the connector configuration phase.
Set to false to skip this check if Salesforce is not reachable during startup/configuration.