Redpanda¶
Since v0.20.0
Introduction¶
Redpanda is a streaming data platform for developers. Kafka API compatible. 10x faster. No ZooKeeper. No JVM! This Testcontainers module provides three APIs:
- Kafka API
- Schema Registry API
- Redpanda Admin API
Adding this module to your project dependencies¶
Please run the following command to add the Redpanda module to your Go dependencies:
go get github.com/testcontainers/testcontainers-go/modules/redpanda
Usage example¶
ctx := context.Background()
redpandaContainer, err := redpanda.Run(ctx,
"docker.redpanda.com/redpandadata/redpanda:v23.3.3",
redpanda.WithEnableSASL(),
redpanda.WithEnableKafkaAuthorization(),
redpanda.WithEnableWasmTransform(),
redpanda.WithBootstrapConfig("data_transforms_per_core_memory_reservation", 33554432),
redpanda.WithBootstrapConfig("data_transforms_per_function_memory_limit", 16777216),
redpanda.WithNewServiceAccount("superuser-1", "test"),
redpanda.WithNewServiceAccount("superuser-2", "test"),
redpanda.WithNewServiceAccount("no-superuser", "test"),
redpanda.WithSuperusers("superuser-1", "superuser-2"),
redpanda.WithEnableSchemaRegistryHTTPBasicAuth(),
)
defer func() {
if err := testcontainers.TerminateContainer(redpandaContainer); err != nil {
log.Printf("failed to terminate container: %s", err)
}
}()
if err != nil {
log.Printf("failed to start container: %s", err)
return
}
Module Reference¶
Run function¶
- Since v0.32.0
Info
The RunContainer(ctx, opts...)
function is deprecated and will be removed in the next major release of Testcontainers for Go.
The Redpanda module exposes one entrypoint function to create the Redpanda container, and this function receives three parameters:
func Run(ctx context.Context, img string, opts ...testcontainers.ContainerCustomizer) (*RedpandaContainer, error)
context.Context
, the Go context.string
, the Docker image to use.testcontainers.ContainerCustomizer
, a variadic argument for passing options.
Image¶
Use the second argument in the Run
function to set a valid Docker image.
In example: Run(context.Background(), "docker.redpanda.com/redpandadata/redpanda:v23.1.7")
.
Container Options¶
When starting the Redpanda container, you can pass options in a variadic way to configure it.
TLS Encryption¶
If you need to enable TLS use WithTLS
with a valid PEM encoded certificate and key.
Additional Listener¶
- Since v0.28.0
There are scenarios where additional listeners are needed, for example if you want to consume/from another container in the same network
You can use the WithListener
option to add a listener to the Redpanda container.
ctr, err := redpanda.Run(ctx,
"redpandadata/redpanda:v23.2.18",
network.WithNetwork([]string{"redpanda-host"}, rpNetwork),
redpanda.WithListener("redpanda:29092"), redpanda.WithAutoCreateTopics(),
)
Container defined in the same network
kcat, err := testcontainers.GenericContainer(ctx, testcontainers.GenericContainerRequest{
ContainerRequest: testcontainers.ContainerRequest{
Image: "confluentinc/cp-kcat:7.4.1",
Networks: []string{
rpNetwork.Name,
},
Entrypoint: []string{
"sh",
},
Cmd: []string{
"-c",
"tail -f /dev/null",
},
},
Started: true,
})
Produce messages using the new registered listener
_, _, err = kcat.Exec(ctx, []string{"kcat", "-b", "redpanda:29092", "-t", "msgs", "-P", "-l", "/tmp/msgs.txt"})
Adding Service Accounts¶
- Since v0.20.0
It's possible to add service accounts to the Redpanda container using the WithNewServiceAccount
option, setting the service account name and its password.
E.g. WithNewServiceAccount("service-account", "password")
.
Adding Super Users¶
- Since v0.20.0
When a super user is needed, you can use the WithSuperusers
option, passing a variadic list of super users.
E.g. WithSuperusers("superuser-1", "superuser-2")
.
Enabling SASL¶
- Since v0.20.0
The WithEnableSASL()
option enables SASL scram sha 256 authentication. By default, no authentication (plaintext) is used.
When setting an authentication method, make sure to add users as well and authorize them using the WithSuperusers()
option.
WithEnableKafkaAuthorization¶
- Since v0.20.0
The WithEnableKafkaAuthorization
enables authorization for connections on the Kafka API.
WithEnableWasmTransform¶
- Since v0.28.0
The WithEnableWasmTransform
enables wasm transform.
Warning
Should not be used with RP versions before 23.3
WithEnableSchemaRegistryHTTPBasicAuth¶
- Since v0.20.0
The WithEnableSchemaRegistryHTTPBasicAuth
enables HTTP basic authentication for the Schema Registry.
WithAutoCreateTopics¶
- Since v0.22.0
The WithAutoCreateTopics
option enables the auto-creation of topics.
WithTLS¶
- Since v0.24.0
The WithTLS
option enables TLS encryption. It requires a valid PEM encoded certificate and key, passed as byte slices.
E.g. WithTLS([]byte(cert), []byte(key))
.
WithBootstrapConfig¶
- Since v0.33.0
WithBootstrapConfig
adds an arbitrary config key-value pair to the Redpanda container. Per the name, this config will be interpolated into the generated bootstrap
config file, which is particularly useful for configs requiring a restart when otherwise applied to a running Redpanda instance.
E.g. WithBootstrapConfig("config_key", config_value)
, where config_value
is of type any
.
WithAdminAPIAuthentication¶
- Since v0.36.0
Enables Admin API Authentication by setting admin_api_require_auth
cluster configuration property to true
.
It also configures a bootstrap superuser account via RP_BOOTSTRAP_USER
environment variable.
The following options are exposed by the testcontainers
package.
Basic Options¶
WithExposedPorts
Since v0.37.0WithEnv
Since v0.29.0WithWaitStrategy
Since v0.20.0WithAdditionalWaitStrategy
Not available until the next release mainWithWaitStrategyAndDeadline
Since v0.20.0WithAdditionalWaitStrategyAndDeadline
Not available until the next release mainWithEntrypoint
Since v0.37.0WithEntrypointArgs
Since v0.37.0WithCmd
Since v0.37.0WithCmdArgs
Since v0.37.0WithLabels
Since v0.37.0
Lifecycle Options¶
WithLifecycleHooks
Not available until the next release mainWithAdditionalLifecycleHooks
Not available until the next release mainWithStartupCommand
Since v0.25.0WithAfterReadyCommand
Since v0.28.0
Files & Mounts Options¶
WithFiles
Since v0.37.0WithMounts
Since v0.37.0WithTmpfs
Since v0.37.0WithImageMount
Since v0.37.0
Build Options¶
WithDockerfile
Since v0.37.0
Logging Options¶
WithLogConsumers
Since v0.28.0WithLogConsumerConfig
Not available until the next release mainWithLogger
Since v0.29.0
Image Options¶
WithAlwaysPull
Not available until the next release mainWithImageSubstitutors
Since v0.26.0WithImagePlatform
Not available until the next release main
Networking Options¶
WithNetwork
Since v0.27.0WithNetworkByName
Not available until the next release mainWithBridgeNetwork
Not available until the next release mainWithNewNetwork
Since v0.27.0
Advanced Options¶
WithHostPortAccess
Since v0.31.0WithConfigModifier
Since v0.20.0WithHostConfigModifier
Since v0.20.0WithEndpointSettingsModifier
Since v0.20.0CustomizeRequest
Since v0.20.0WithName
Not available until the next release mainWithNoStart
Not available until the next release main
Experimental Options¶
WithReuseByName
Since v0.37.0
Container Methods¶
The Redpanda container exposes the following methods:
KafkaSeedBroker¶
- Since v0.20.0
KafkaSeedBroker returns the seed broker that should be used for connecting to the Kafka API with your Kafka client. It'll be returned in the format: "host:port" - for example: "localhost:55687".
seedBroker, err := ctr.KafkaSeedBroker(ctx)
SchemaRegistryAddress¶
- Since v0.20.0
SchemaRegistryAddress returns the address to the schema registry API. This is an HTTP-based API and thus the returned format will be: http://host:port.
schemaRegistryURL, err := ctr.SchemaRegistryAddress(ctx)
AdminAPIAddress¶
- Since v0.20.0
AdminAPIAddress returns the address to the Redpanda Admin API. This is an HTTP-based API and thus the returned format will be: http://host:port.
adminAPIURL, err := ctr.AdminAPIAddress(ctx)