Connecting Trino to S3

You can specify S3 connection details directly inside the TrinoCatalog specification or by referring to an external S3Connection custom resource. This mechanism is used across the whole Stackable Data Platform, read the S3 concepts page to learn more.

Every S3 connection must contain a tls section, because Trino uses the native S3 client, which requires TLS.

Inline

To specify S3 connection details directly as part of the TrinoCatalog resource, you add an inline connection configuration as shown below:

s3: (1)
  inline:
    host: test-minio (2)
    port: 9000 (3)
    accessStyle: Path (4)
    credentials:
      secretClass: minio-credentials  (5)
    tls: (6)
      verification:
        server:
          caCert:
            secretClass: minio-tls-certificates

1	Entry point for the connection configuration
2	Connection host
3	Optional connection port
4	Optional access style, either `Path` or `VirtualHosted`; This defaults to `VirtualHosted`, which means virtual hosted-style URLs are used.
5	Name of the SecretClass providing the credentials. The Secret behind it is expected to contain the following keys: `accessKey` and `secretKey`
6	TLS settings for encrypted traffic. The `secretClass` can be provided by the Secret Operator or yourself. This section is required, see the note above.

A self provided S3 TLS secret can be specified like this:

---
apiVersion: secrets.stackable.tech/v1alpha1
kind: SecretClass
metadata:
  name: minio-tls-certificates
spec:
  backend:
    k8sSearch:
      searchNamespace:
        pod: {}
---
apiVersion: v1
kind: Secret
metadata:
  name: minio-tls-certificates
  labels:
    secrets.stackable.tech/class: minio-tls-certificates
data:
    ca.crt: <your-base64-encoded-ca>
    tls.crt: <your base64-encoded-public-key>
    tls.key: <your-base64-encoded-private-key>

Reference

It is also possible to configure the bucket connection details as a separate Kubernetes resource and only refer to that object from the TrinoCatalog specification like this:

s3:
  reference: my-connection-resource (1)

1	Name of the connection resource with connection details

The resource named my-connection-resource is then defined as shown below:

---
apiVersion: s3.stackable.tech/v1alpha1
kind: S3Connection
metadata:
  name: my-connection-resource
spec:
  host: test-minio
  port: 9000
  accessStyle: Path
  credentials:
    secretClass: minio-credentials
  tls:
    verification:
      server:
        caCert:
          secretClass: minio-tls-certificates

This has the advantage that the connection configuration is configured in a single place and can be shared across applications, reducing the cost of updating these details.