Configuration (Kubernetes)

This page serves as a reference to the configuration when services.type is set to Kubernetes.

The configuration is split into several files:

server.yml

Communication and internal database configuration for the Integration Module.

config.yml

Configuration related to your Kubernetes environment, services, etc.

Server configuration

refreshToken: "<token-goes-here>"
database:
  embedded: false
  username: postgres
  password: postgrespassword
  database: postgres
  ssl: false
  host:
    address: go-slurm-postgres

The server.yml file contains the refresh token, used to renew the access tokens for the communication from UCloud/Core, along with credentials for the internal Integration Module database, which contains the current state.

refreshToken

The refresh token is used to renew the (short-lived) access token, which is used by the UCloud/Core to make authorized calls to the UCloud Integration Module.

database optional

This section defines connection information for the database used by the UCloud Integration Module to store internal data. If this is not defined, the UCloud Integration Module will use its own embedded database.

embedded

Defines if the integration module should use its own internal (embedded) database for storing data or not. In case the database section is not defined, this will default to true. If set to false, the Integration Module will use the database and credentials defined by the following parameters.

username

The username for the database, in case embedded is set to false. Note that the user needs to have both read and write access to the database.

password

The password for the database, in case embedded is set to false.

database

The name of the database to connect to, in case embedded is set to false.

ssl

If set to true the Integration Module will only connect to the database using SSL.

If set to false the Integration Module will connect to the database without SSL.

The parameter is only used if embedded is set to false.

host

The host information used for connecting to the database. See Host information. The port will default to 5432 if not defined.

Provider and Services configuration (Kubernetes)

provider:
  id: my-k8s-provider

  providerBranding:
    title: My Provider title
    shortTitle: Short provider title
    shortDescription: My short provider description
    description: /etc/ucloud/description.md
    url: https://myprovider.com
    logo: "/etc/ucloud/images/provider_logo.png"
    sections:
      - description: /etc/ucloud/section1.md
        image: "/etc/ucloud/images/firsts-section.png"
      - description: /etc/ucloud/section2.md
        image: "/etc/ucloud/images/firsts-section.png"
    productDescription:
      - category: "u1-standard"
        shortDescription: u1-standard provides computation
        section:
          description: /etc/ucloud/aproduct.md
          image: "/etc/ucloud/images/a-cpu.png"
      - category: "storage"
        shortDescription: Provides storage
        section:
          description: /etc/ucloud/bproduct.md
          image: "/etc/ucloud/images/a-hardrive.png"

  hosts:
    ucloud:
      address: cloud.sdu.dk
      port: 443
      scheme: https
    self:
      address: provider.example.com
      port: 443
      scheme: https
      
  ipc:
    directory: /var/run/ucloud

  logs:
    directory: /var/log/ucloud
    rotation:
      enabled: true
      retentionPeriodInDays: 180

  envoy:
    directory: /var/run/ucloud/envoy
    executable: /usr/bin/envoy
    funceWrapper: false

services:
  type: Kubernetes

  fileSystem:
    name: "storage"
    mountPoint: "/mnt/storage"
    trashStagingArea: "/mnt/storage/trash"
    claimName: "ucloud-user-data"
    scanMethod:
      type: Walk

  compute:
    namespace: "ucloud-apps"
    estimatedContainerDownloadSpeed: 14.5

    inference:
      enabled: true
      provider: development
      development:
        provider: localai
        server: "http://localai:8080/v1"
      access:
        administrators: []
        testers: []

      # provider: dynamo
      # dynamo:
      #   namespace: ucloud-inference

    modules:
      tools:
        subPath: "tools"
        claimName: "shared-tools-pvc"
        # hostPath: "/srv/tools"     # exactly one of claimName/hostPath

    web:
      enabled: true
      prefix: "apps-"
      suffix: ".example.org"

    publicIps:
      enabled: true
      name: "public-ip"

    publicLinks:
      enabled: true
      name: "public-links"
      prefix: "app-"
      suffix: ".example.com"

    ssh:
      enabled: true
      ipAddress: "203.0.113.10"
      hostname: "ssh.example.com"
      portMin: 30000
      portMax: 31000

    syncthing:
      enabled: true
      ipAddress: "203.0.113.11"
      portMin: 32000
      portMax: 33000
      relaysEnabled: true
      developmentSourceCode: "/opt/ucloud/syncthing-dev" # optional

    integratedTerminal:
      enabled: true

    virtualMachines:
      enabled: true
      storage:
        type: "CsiStaticPv" # or "HostPath"
        csi:
          storageClassName: "csi-cephfs-sc"
          driver: "cephfs.csi.ceph.com"
          nodeStageSecretRef:
            name: "csi-cephfs-secret"
            namespace: "default"
          volumeAttributes:
            clusterID: "XXXX"
            fsName: "YYYY"
            rootPath: "/volumes/ucloud"
            staticVolume: "true"
          subpathField: "volumeAttributes.rootPath"

    machines:
      cpu-standard:
        payment:
          type: Resource
          unit: Cpu
          interval: Hourly
        systemReservedCpuMillis: 500 # category-level (default if omitted)
        # systemReservedCpuMillisPerCore: 42 # optional override, mutually exclusive with systemReservedCpuMillis

        groups:
          general:
            nameSuffix: CpuV2
            cpu: [2, 4, 8]
            memory: [8, 16, 32]
            cpuModel: "AMD EPYC"
            memoryModel: "DDR4"
            allowContainers: true
            allowVirtualMachines: false

      gpu-a10:
        payment:
          type: Money
          currency: "EUR"
          interval: Hourly

        groups:
          a10:
            nameSuffix: GpuV2
            gpuType: "nvidia.com/gpu"
            cpu: [8]
            memory: [64]
            gpu: [1, 2]
            price: [1.25, 2.50]
            gpuModel: "NVIDIA A10"
            allowContainers: true
            allowVirtualMachines: true
            customRuntime: "nvidia"

---

providerBranding

Defines the provider’s branding information and the services it exposes.

Many of the properties are self-explanatory, but some properties point at physical files on the hard drive, these properties are:

• description, points to a Markdown file on the disk.
• logo, points to a logo image file on the disk.
• image, points to an image file on the disk.

The provider is able to describe itself in more detail by using the following properties:

• sections, supplementary information about the provider.
• productDescription, information about the provider’s products.

---

services

type

Must be Kubernetes.

fileSystem

Configuration for the filesystem backing user/project files in the Kubernetes provider. See File system.

compute

Compute configuration, including machine catalog, networking features, optional SSH/syncthing exposure, and modules. See Compute.

---

File system

name

A name for the filesystem.

mountPoint

A folder path where the filesystem is mounted. Must exist and be readable/writable.

trashStagingArea

A folder path used as a staging area for trash/deletions. Must exist and be readable/writable.

claimName

The Kubernetes PVC claim name used for the filesystem.

scanMethod optional

Controls how the filesystem is scanned.

type

Possible values:

• Walk (default if scanMethod is omitted)
• Xattr
• Development

xattr (required if type is Xattr)

Name of the extended attribute used by the scanner.

---

Compute

namespace optional

Kubernetes namespace used for workloads. If omitted or empty, defaults to ucloud-apps.

estimatedContainerDownloadSpeed optional

A floating point number (MB/s). Defaults to 14.5.

imSourceCode optional

Optional path to Integration Module source code (used for development/diagnostics).

inference optional

Inference feature configuration. When enabled, IM stores inference model metadata in its internal model catalog and exposes model catalog management through the CLI.

enabled

Enable/disable inference features.

provider optional

Selects the inference backend integration. Defaults to development when omitted.

Allowed values:

• development
• dynamo

development (required when provider is development)

Development inference backend configuration.

provider

Development backend provider. Set to localai to enable LocalAI startup/bootstrap behavior in development mode.

server

Base URL of the OpenAI-compatible inference backend, for example http://localai:8080/v1.

For compatibility with older configuration files, backendServer and developmentProvider can still be set directly under inference when development is omitted.

dynamo (required when provider is dynamo)

Dynamo inference discovery configuration.

namespace

Kubernetes namespace where Dynamo frontend services are discovered. IM discovers services in this namespace with the -frontend suffix and registers models exposed by their OpenAI-compatible /v1/models endpoint.

access optional

Project-based inference access lists.

administrators

List of project IDs treated as inference administrators. Administrators can see all models in the UI/RPC surfaces intended for model management. Administrators are also automatically added to testers.

testers

List of project IDs used as the default allow-list for newly discovered non-public models.

modules optional

A dictionary of named module entries.

Each module entry:

subPath

Required. A sub-path within the volume source.

hostPath (exactly one of hostPath / claimName must be set)

Use a host path as the module’s volume source.

claimName (exactly one of hostPath / claimName must be set)

Use a PVC claim name as the module’s volume source.

Constraints:

• Module names must be unique.
• The configuration must set exactly one of hostPath and claimName.

machineImpersonation optional

A dictionary mapping one machine “name” to another. This is used to treat one machine SKU as another.

machines

A dictionary of machine categories (compute products). See Machines.

web optional

Controls web interfaces for applications.

enabled

Enable/disable web interfaces.

prefix (required if enabled)

Address prefix used when constructing web hostnames.

suffix (required if enabled)

Address suffix used when constructing web hostnames.

publicIps optional

Controls the public IP feature.

enabled

Enable/disable public IP support.

name optional

Defaults to public-ip if omitted.

publicLinks optional

Controls public links.

enabled

Enable/disable public link support.

name optional

Defaults to public-links if omitted.

prefix (required if enabled)

Address prefix used when constructing link hostnames.

suffix (required if enabled)

Address suffix used when constructing link hostnames.

ssh optional

Expose SSH access.

enabled

Enable/disable SSH feature.

ipAddress (required if enabled)

Must be a valid IP address string.

hostname optional

An optional hostname to associate with SSH.

portMin (required if enabled)

Minimum port (must be within valid TCP port range).

portMax (required if enabled)

Maximum port (must be within valid TCP port range).

syncthing optional

Exposes the Syncthing integration.

enabled

Enable/disable Syncthing feature.

ipAddress (required if enabled)

Must be a valid IP address string.

portMin (required if enabled)

Minimum port (must be within valid TCP port range).

portMax (required if enabled)

Maximum port (must be within valid TCP port range).

developmentSourceCode optional

Optional path used for development.

relaysEnabled optional

Boolean toggle for relays.

integratedTerminal optional

enabled

Enable/disable an integrated terminal feature.

virtualMachines optional

Controls virtual machine support.

enabled

Enable/disable virtual machines.

storage (required if enabled)

Virtual machine disk backend configuration.

type

Storage mode for VM disks.

Allowed values:

• HostPath
• CsiStaticPv

hostPath (required when type is HostPath)

Host path prefix used for creating VM disk PVs.

csi (required when type is CsiStaticPv)

CSI static PV configuration used when creating VM disk PVs.

storageClassName

Storage class name set on the generated PV and PVC.

driver

CSI driver name (for example `cephfs.csi.ceph.com`).

nodeStageSecretRef optional

Optional secret reference passed to csi.nodeStageSecretRef.

name

Secret name.

namespace

Secret namespace.

volumeAttributes

Dictionary of CSI volume attributes copied into the generated PV.

subpathField

Must be on the form volumeAttributes.<key> and must reference an existing non-empty key in volumeAttributes. IM appends the job-specific VM path suffix to this attribute when creating the PV.

---

Machines

compute.machines is a dictionary of machine categories. Each category has:

payment

Defines how this machine category is charged. See Payment.

Important constraints:

• If payment.type is Money, each machine configuration must provide price and it must be greater than 0.
• If payment.type is Resource, each machine configuration must not specify price.

groups optional

If omitted, the category itself is treated as a single implicit group.

If present, it must be a dictionary of groups. Each group defines compatible machine sizes and behavior.

systemReservedCpuMillis optional

Category-level CPU reservation for system overhead, in millicores. Default is 500 if neither reservation field is set.

systemReservedCpuMillisPerCore optional

Category-level per-core CPU reservation (millicores per virtual core). Overrides the computed per-core reservation behavior.

systemReservedCpuMillis and systemReservedCpuMillisPerCore are mutually exclusive.

Machine group options

Each group supports:

cpu, memory, gpu

Lists of supported sizes.

• cpu and memory are required and must be lists of the same size.
• gpu is optional, it must either be omitted or be the same size as cpu.
• If price is present, it must have the same length as cpu.

A “machine configuration” is formed by zipping the lists by index.

price optional

List of prices matching the cpu list length. Only valid/required when the machine category payment type is Money.

nameSuffix optional

Controls how machine names are generated. Possible values:

• Legacy: Cpu, Memory, Gpu
• V2: CpuV2, MemoryV2, GpuV2

Default:

• If gpu list is provided, defaults to Gpu
• Otherwise defaults to Cpu

cpuModel, memoryModel, gpuModel optional

Textual descriptions of underlying hardware models.

allowVirtualMachines optional

Enable/disable virtual machines for this group (defaults to `false` if omitted).

allowContainers optional

Enable/disable containers for this group (defaults to `true` if omitted).

gpuType optional

GPU resource type string (for example nvidia.com/gpu). Defaults to nvidia.com/gpu if omitted.

customRuntime optional

Optional runtime hint/name for custom container runtimes.

fraction optional

Fraction for the primary resource in this group. Defaults to 1 / 1 when omitted.

Can be specified either as:

• string shorthand: "1 / 7" (whitespace ignored)
• structured object:

fraction:
  numerator: 1
  denominator: 7

For GPU groups this is used for MIG/fractional GPU normalization and naming. For CPU groups this controls effective CPU allocation (cpu * numerator / denominator) for pod resource requests.

---

Payment

type

Possible values are Resource or Money.

price (required if type is Money)

currency (required if type is Money)

interval optional

Possible values are Minutely, Hourly and Daily.

unit

Possible values are GB, TB, PB, EB, GiB, TiB, PiB and EiB for storage products, and Cpu, Memory and Gpu for compute products.

---

Host information

Host information defines the address, port and scheme to a location. For example:

address: postgres
port: 8080
scheme: http

This states that the container named postgres is accessible over http on port 8080.

address

The name/address of the host.

port optional

The port number to use. If not defined, the Integration Module will attempt to use a reasonable default value.

scheme

The scheme to use. For example http, https, etc.