gitadmin/devops-infra-helm-charts-gcp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

added repo

Browse Source
This commit is contained in:
Your Name
2026-08-26 03:39:42 +05:30
parent 45c25a95af
commit b8575bb8b9
6889 changed files with 1217125 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
helm-templates/deepgram-onprem/CHANGELOG.md
+114
View File
@@ -0,0 +1,114 @@
# Changelog
All notable changes to this Helm chart will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
## [Unreleased]
## [0.5.0] - 2024-08-27
### Added
- Updated default container tags to August 2024 release. Refer to the [main Deepgram changelog](https://deepgram.com/changelog/deepgram-self-hosted-august-2024-release-240827) for additional details. Highlights include:
- GA support for entity detection for pre-recorded English audio
- GA support for improved redaction for pre-recorded English audio
### Fixed
- Fixed a misleading comment in the `03-basic-setup-onprem.yaml` sample file that wrongly suggested `engine.modelManager.volumes.customVolumeClaim.name` should be a `PersistentVolume` instead of a `PersistentVolumeClaim`
### Changed
- Deepgram's core products are available to host both on-premises and in the cloud. Official resources have been updated to refer to a ["self-hosted" product offering](https://deepgram.com/self-hosted), instead of an "onprem" product offering, to align the product name with industry naming standards. The Deepgram Quay image repository names have been updated to reflect this.
## [0.4.0] - 2024-07-25
### Added
- Introduced entity detection feature flag for API containers (`false` by default).
- Updated default container tags to July 2024 release. Refer to the [main Deepgram changelog](https://deepgram.com/changelog/deepgram-self-hosted-july-2024-release-240725) for additional details. Highlights include:
- Support for Deepgram's new English/Spanish multilingual code-switching model
- Beta support for entity detection for pre-recorded English audio
- Beta support for improved redaction for pre-recorded English audio
- Beta support for improved entity formatting for streaming English audio
### Removed
- Removed some items nested under `api.features` and `engine.features` sections in favor of opinionated defaults.
## [0.3.0] - 2024-07-18
### Added
- Allow specifying custom annotations for deployments.
## [0.2.3] - 2024-07-15
### Added
- Sample `values.yaml` file for on-premises/self-managed Kubernetes clusters.
### Fixed
- Resolves a mismatch between PVC and SC prefix naming convention.
- Resolves error when specifying custom service account names.
### Changed
- Make `imagePullSecrets` optional.
## [0.2.2-beta] - 2024-06-27
### Added
- Adds more verbose logging for audio content length.
- Keeps our software up-to-date.
- See the [changelog](https://deepgram.com/changelog/deepgram-on-premises-june-2024-release-240627) associated with this routine monthly release.
## [0.2.1-beta] - 2024-06-24
### Added
- Restart Deepgram containers automatically when underlying ConfigMaps have been modified.
## [0.2.0-beta] - 2024-06-20
### Added
- Support for managing node autoscaling with [cluster-autoscaler](https://github.com/kubernetes/autoscaler).
- Support for pod autoscaling of Deepgram components.
- Support for keeping the upstream Deepgram License server as a backup even when the License Proxy is deployed. See `licenseProxy.keepUpstreamServerAsBackup` for details.
### Changed
- Initial installation replica count values moved from `scaling.static.{api,engine}.replicas` to `scaling.replicas.{api,engine}`.
- License Proxy is no longer manually scaled. Instead, scaling can be indirectly controlled via `licenseProxy.{enabled,deploySecondReplica}`.
- Labels for Deepgram dedicated nodes in the sample `cluster-config.yaml` for AWS, and the `nodeAffinity` sections of the sample `values.yaml` files. The key has been renamed from `deepgram/nodeType` to `k8s.deepgram.com/node-type`, and the values are no longer prepended with `deepgram`.
- AWS EFS model download job hook delete policy changed to `before-hook-creation`.
- Concurrency limit moved from API (`api.concurrencyLimit.activeRequests`) to Engine level (`engine.concurrencyLimit.activeRequests`).
## [0.1.1-alpha] - 2024-06-03
### Added
- Various documentation improvements
## [0.1.0-alpha] - 2024-05-31
### Added
- Initial implementation of the Helm chart.
[unreleased]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.5.0...HEAD
[0.5.0]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.4.0...deepgram-self-hosted-0.5.0
[0.4.0]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.3.0...deepgram-self-hosted-0.4.0
[0.3.0]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.2.3...deepgram-self-hosted-0.3.0
[0.2.3]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.2.2-beta...deepgram-self-hosted-0.2.3
[0.2.2-beta]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.2.1-beta...deepgram-self-hosted-0.2.2-beta
[0.2.1-beta]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.2.0-beta...deepgram-self-hosted-0.2.1-beta
[0.2.0-beta]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.1.1-alpha...deepgram-self-hosted-0.2.0-beta
[0.1.1-alpha]: https://github.com/deepgram/self-hosted-resources/compare/deepgram-self-hosted-0.1.0-alpha...deepgram-self-hosted-0.1.1-alpha
[0.1.0-alpha]: https://github.com/deepgram/self-hosted-resources/releases/tag/deepgram-self-hosted-0.1.0-alpha
helm-templates/deepgram-onprem/Chart.lock
+15
View File
@@ -0,0 +1,15 @@
dependencies:
- name: gpu-operator
repository: https://helm.ngc.nvidia.com/nvidia
version: v24.3.0
- name: cluster-autoscaler
repository: https://kubernetes.github.io/autoscaler
version: 9.37.0
- name: kube-prometheus-stack
repository: https://prometheus-community.github.io/helm-charts
version: 60.5.0
- name: prometheus-adapter
repository: https://prometheus-community.github.io/helm-charts
version: 4.10.0
digest: sha256:38ff59bc00f2cd88c2f06a844f0309d3fea593dd0a8877a7667288fa9f5c53b8
generated: "2024-07-15T11:42:44.763681334-04:00"
helm-templates/deepgram-onprem/Chart.yaml
+42
View File
@@ -0,0 +1,42 @@
apiVersion: v2
name: deepgram-self-hosted
type: application
version: 0.5.0
appVersion: "release-240827"
description: A Helm chart for running Deepgram services in a self-hosted environment
home: "https://developers.deepgram.com/docs/self-hosted-introduction"
sources: ["https://github.com/deepgram/self-hosted-resources"]
kubeVersion: ">=1.28.0-0"
maintainers:
- name: Deepgram Self-Hosted
email: self.hosted@deepgram.com
icon: "https://www.dropbox.com/scl/fi/v4jtfbsrx881pbevcga3j/D-icon-black-square-250x250.png?rlkey=barv5jeuhd7t2lczz0m3nane7&dl=1"
keywords:
- voice ai
- text-to-speech
- tts
- aura
- speech-to-text
- stt
- asr
- nova
- voice agent
- self-hosted
dependencies:
- name: gpu-operator
version: "^24.3.0"
repository: "https://helm.ngc.nvidia.com/nvidia"
condition: gpu-operator.enabled
- name: cluster-autoscaler
version: "^9.37.0"
repository: "https://kubernetes.github.io/autoscaler"
condition: cluster-autoscaler.enabled
- name: kube-prometheus-stack
version: "^60.2.0"
repository: "https://prometheus-community.github.io/helm-charts"
condition: kube-prometheus-stack.includeDependency,scaling.auto.enabled
- name: prometheus-adapter
version: "^4.10.0"
repository: "https://prometheus-community.github.io/helm-charts"
condition: prometheus-adapter.includeDependency,scaling.auto.enabled
helm-templates/deepgram-onprem/README.md
+327
View File
@@ -0,0 +1,327 @@
# deepgram-self-hosted
![Version: 0.5.0](https://img.shields.io/badge/Version-0.5.0-informational?style=flat-square) ![Type: application](https://img.shields.io/badge/Type-application-informational?style=flat-square) ![AppVersion: release-240827](https://img.shields.io/badge/AppVersion-release--240827-informational?style=flat-square) [![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/deepgram-self-hosted)](https://artifacthub.io/packages/search?repo=deepgram-self-hosted)
A Helm chart for running Deepgram services in a self-hosted environment
**Homepage:** <https://developers.deepgram.com/docs/self-hosted-introduction>
**Deepgram Self-Hosted Kubernetes Guides:** <https://developers.deepgram.com/docs/kubernetes>
## Source Code
* <https://github.com/deepgram/self-hosted-resources>
## Requirements
Kubernetes: `>=1.28.0-0`
| Repository | Name | Version |
|------------|------|---------|
| https://helm.ngc.nvidia.com/nvidia | gpu-operator | ^24.3.0 |
| https://kubernetes.github.io/autoscaler | cluster-autoscaler | ^9.37.0 |
| https://prometheus-community.github.io/helm-charts | kube-prometheus-stack | ^60.2.0 |
| https://prometheus-community.github.io/helm-charts | prometheus-adapter | ^4.10.0 |
## Using the Chart
### Get Repository Info
```bash
helm repo add deepgram https://deepgram.github.io/self-hosted-resources
helm repo update
```
### Installing the Chart
The Deepgram self-hosted chart requires Helm 3.7+ in order to install successfully. Please check your helm release before installation.
You will need to provide your [self-service Deepgram licensing and credentials](https://developers.deepgram.com/docs/self-hosted-self-service-tutorial) information. See `global.deepgramSecretRef` and `global.pullSecretRef` in the [Values section](#values) for more details, and the [Deepgram Self-Hosted Kubernetes Guides](https://developers.deepgram.com/docs/kubernetes) for instructions on how to create these secrets.
You may also override any default configuration values. See [the Values section](#values) for a list of available options, and the [samples directory](./samples) for examples of a standard installation.
```
helm install -f my-values.yaml [RELEASE_NAME] deepgram/deepgram-self-hosted --atomic --timeout 45m
```
### Upgrade and Rollback Strategies
To upgrade the Deepgram components to a new version, follow these steps:
1. Update the various `image.tag` values in the `values.yaml` file to the desired version.
2. Run the Helm upgrade command:
```bash
helm upgrade -f my-values.yaml [RELEASE_NAME] deepgram/deepgram-self-hosted --atomic --timeout 60m
```
If you encounter any issues during the upgrade process, you can perform a rollback to the previous version:
```bash
helm rollback deepgram
```
Before upgrading, ensure that you have reviewed the release notes and any migration guides provided by Deepgram for the specific version you are upgrading to.
### Uninstalling the Chart
```bash
helm uninstall [RELEASE_NAME]
```
This removes all the Kubernetes components associated with the chart and deletes the release.
## Changelog
See the [chart CHANGELOG](./CHANGELOG.md) for a list of relevant changes for each version of the Helm chart.
For more details on changes to the underlying Deepgram resources, such as the container images or available models, see the [official Deepgram changelog](https://deepgram.com/changelog) ([RSS feed](https://deepgram.com/changelog.xml)).
## Chart Configuration
### Persistent Storage Options
The Deepgram Helm chart supports different persistent storage options for storing Deepgram models and data. The available options include:
- AWS Elastic File System (EFS)
- Google Cloud Persistent Disk (GPD)
- Custom PersistentVolumeClaim (PVC)
To configure a specific storage option, see the `engine.modelManager.volumes` [configuration values](#values). Make sure to provide the necessary configuration values for the selected storage option, such as the EFS file system ID or the GPD disk type and size.
For detailed instructions on setting up and configuring each storage option, refer to the [Deepgram self-hosted guides](https://developers.deepgram.com/docs/kubernetes) and the respective cloud provider's documentation.
### Autoscaling
Autoscaling your cluster's capacity to meet incoming traffic demands involves both node autoscaling and pod autoscaling. Node autoscaling for supported cloud providers is setup by default when using this Helm chart and creating your cluster with the [Deepgram self-hosted guides](https://developers.deepgram.com/docs/kubernetes). Pod autoscaling can be enabled via the `scaling.auto.enabled` configuration option in this chart.
#### Engine
The Engine component is the core of the Deepgram self-hosted platform, responsible for performing inference using your deployed models. Autoscaling increases the number of Engine replicas to maintain consistent performance for incoming traffic.
There are currently two primary ways to scale the Engine component: scaling with a hard request limit per Engine Pod, or scaling with a soft request limit per Engine pod.
To set a hard limit on which to scale, configure `engine.concurrencyLimit.activeRequests` and `scaling.auto.engine.metrics.requestCapacityRatio`. The `activeRequests` parameter will set a hard limit of how many requests any given Engine pod will accept, and the `requestCapacityRatio` will govern scaling the Engine deployment when a certain percentage of "available request slots" is filled. For example, a requestCapacityRatio of `0.8` will scale the Engine deployment when the current number of active requests is >=80% of the active request concurrency limit. If the cluster is not able to scale in time and current active requests hits 100% of the preset limit, additional client requests to the API will return a `429 Too Many Requests` HTTP response to clients. This hard limit means that if a request is accepted for inference, it will have consistent performance, as the cluster will refuse surplus requests that could overload the cluster and degrade performance, at the expense of possibly rejecting some incoming requests if capacity does not scale in time.
To set a soft limit on which to scale, configure `scaling.auto.engine.metrics.{speechToText,textToSpeech}.{batch,streaming}.requestsPerPod`, depending on the primary traffic source for your environment. The cluster will attempt to scale to meet this target for number of requests per Engine pod, but will not reject extra requests with a `429 Too Many Request` HTTP response like the hard limit will. If the number of extra requests increases faster than the cluster can scale additional capacity, all incoming requests will still be accepted, but the performance of individual requests may degrade.
> [!NOTE]
> Deepgram recommends provisioning separate environments for batch speech-to-text, streaming speech-to-text, and text-to-speech workloads because typical latency and throughput tradeoffs are different for each of those use cases.
There is also a `scaling.auto.engine.metrics.custom` configuration value available to define your own custom scaling metric, if needed.
#### API
The API component is responsible for accepting incoming requests and forming responses, delegating inference work to the Deepgram Engine as needed. A single API pod can typically handle delegating requests to multiple Engine pods, so it is more compute efficient to deploy fewer API pods relative to the number of Engine pods. The `scaling.auto.api.metrics.engineToApiRatio` configuration value defines the ratio between Engine to API pods. The default value is appropriate for most deployments.
There is also a `scaling.auto.api.metrics.custom` configuration value available to define your own custom scaling metric, if needed.
#### License Proxy
The [License Proxy](https://developers.deepgram.com/docs/license-proxy) is intended to be deployed as a fixed-scale deployment the proxies all licensing requests from your environment. It should not be upscaled with the traffic demands of your environment.
This chart deploys one License Proxy Pod per environment by default. If you wish to deploy a second License Proxy Pod for redundancy, set `licenseProxy.deploySecondReplica` to `true`.
### RBAC Configuration
Role-Based Access Control (RBAC) is used to control access to Kubernetes resources based on the roles and permissions assigned to users or service accounts. The Deepgram Helm chart includes default RBAC roles and bindings for the API, Engine, and License Proxy components.
To use custom RBAC roles and bindings based on your specific security requirements, you can individually specify pre-existing ServiceAccounts to bind to each deployment by specifying the following options in `values.yaml`:
```
{api|engine|licenseProxy}.serviceAccount.create=false
{api|engine|licenseProxy}.serviceAccount.name=<your-pre-existing-sa>
```
Make sure to review and adjust the RBAC configuration according to the principle of least privilege, granting only the necessary permissions for each component.
### Secret Management
The Deepgram Helm chart takes references to two existing secrets - one containing your distribution credentials to pull container images from Deepgram's image repository, and one containing your Deepgram self-hosted API key.
Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
## Getting Help
See the [Getting Help](../../README.md#getting-help) section in the root of this repository for a list of resources to help you troubleshoot and resolve issues.
### Troubleshooting
If you encounter issues while deploying or using Deepgram, consider the following troubleshooting steps:
1. Check the pod status and logs:
- Use `kubectl get pods` to check the status of the Deepgram pods.
- Use `kubectl logs <pod-name>` to view the logs of a specific pod.
2. Verify resource availability:
- Ensure that the cluster has sufficient CPU, memory, and storage resources to accommodate the Deepgram components.
- Check for any resource constraints or limits imposed by the namespace or the cluster.
3. Review the Kubernetes events:
- Use `kubectl get events` to view any events or errors related to the Deepgram deployment.
4. Check the network connectivity:
- Verify that the Deepgram components can communicate with each other and with the Deepgram license server (license.deepgram.com).
- Check the network policies and firewall rules to ensure that the necessary ports and protocols are allowed.
5. Collect diagnostic information:
- Gather relevant logs and metrics.
- Export your existing Helm chart values:
```bash
helm get values [RELEASE_NAME] > my-deployed-values.yaml
```
- Provide the collected diagnostic information to Deepgram for assistance.
## Values
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| api.additionalAnnotations | object | `nil` | Additional annotations to add to the API deployment |
| api.additionalLabels | object | `{}` | Additional labels to add to API resources |
| api.affinity | object | `{}` | [Affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) to apply for API pods. |
| api.driverPool | object | `` | driverPool configures the backend pool of speech engines (generically referred to as "drivers" here). The API will load-balance among drivers in the standard pool; if one standard driver fails, the next one will be tried. |
| api.driverPool.standard | object | `` | standard is the main driver pool to use. |
| api.driverPool.standard.maxResponseSize | string | `"1073741824"` | Maximum response to deserialize from Driver (in bytes). Default is 1GB, expressed in bytes. |
| api.driverPool.standard.retryBackoff | float | `1.6` | retryBackoff is the factor to increase the retrySleep by for each additional retry (for exponential backoff). |
| api.driverPool.standard.retrySleep | string | `"2s"` | retrySleep defines the initial sleep period (in humantime duration) before attempting a retry. |
| api.driverPool.standard.timeoutBackoff | float | `1.2` | timeoutBackoff is the factor to increase the timeout by for each additional retry (for exponential backoff). |
| api.features | object | `` | Enable ancillary features |
| api.features.diskBufferPath | string | `nil` | If API is receiving requests faster than Engine can process them, a request queue will form. By default, this queue is stored in memory. Under high load, the queue may grow too large and cause Out-Of-Memory errors. To avoid this, set a diskBufferPath to buffer the overflow on the request queue to disk. WARN: This is only to temporarily buffer requests during high load. If there is not enough Engine capacity to process the queued requests over time, the queue (and response time) will grow indefinitely. |
| api.features.entityDetection | bool | `false` | Enables entity detection on pre-recorded audio *if* a valid entity detection model is available. *WARNING*: Beta functionality. |
| api.features.entityRedaction | bool | `false` | Enables entity-based redaction on pre-recorded audio *if* a valid entity detection model is available. *WARNING*: Beta functionality. |
| api.image.path | string | `"quay.io/deepgram/self-hosted-api"` | path configures the image path to use for creating API containers. You may change this from the public Quay image path if you have imported Deepgram images into a private container registry. |
| api.image.pullPolicy | string | `"IfNotPresent"` | pullPolicy configures how the Kubelet attempts to pull the Deepgram API image |
| api.image.tag | string | `"release-240827"` | tag defines which Deepgram release to use for API containers |
| api.livenessProbe | object | `` | Liveness probe customization for API pods. |
| api.namePrefix | string | `"deepgram-api"` | namePrefix is the prefix to apply to the name of all K8s objects associated with the Deepgram API containers. |
| api.readinessProbe | object | `` | Readiness probe customization for API pods. |
| api.resolver | object | `` | Specify custom DNS resolution options. |
| api.resolver.maxTTL | int | `nil` | maxTTL sets the DNS TTL value if specifying a custom DNS nameserver. |
| api.resolver.nameservers | list | `[]` | nameservers allows for specifying custom domain name server(s). A valid list item's format is "{IP} {PORT} {PROTOCOL (tcp or udp)}", e.g. `"127.0.0.1 53 udp"`. |
| api.resources | object | `` | Configure resource limits per API container. See [Deepgram's documentation](https://developers.deepgram.com/docs/self-hosted-deployment-environments#api) for more details. |
| api.securityContext | object | `{}` | [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for API pods. |
| api.server | object | `` | Configure how the API will listen for your requests |
| api.server.callbackConnTimeout | string | `"1s"` | callbackConnTimeout configures how long to wait for a connection to a callback URL. See [Deepgram's callback documentation](https://developers.deepgram.com/docs/callback) for more details. The value should be a humantime duration. |
| api.server.callbackTimeout | string | `"10s"` | callbackTimeout configures how long to wait for a response from a callback URL. See [Deepgram's callback documentation](https://developers.deepgram.com/docs/callback) for more details. The value should be a humantime duration. |
| api.server.fetchConnTimeout | string | `"1s"` | fetchConnTimeout configures how long to wait for a connection to a fetch URL. The value should be a humantime duration. A fetch URL is a URL passed in an inference request from which a payload should be downloaded. |
| api.server.fetchTimeout | string | `"60s"` | fetchTimeout configures how long to wait for a response from a fetch URL. The value should be a humantime duration. A fetch URL is a URL passed in an inference request from which a payload should be downloaded. |
| api.server.host | string | `"0.0.0.0"` | host is the IP address to listen on. You will want to listen on all interfaces to interact with other pods in the cluster. |
| api.server.port | int | `8080` | port to listen on. |
| api.serviceAccount.create | bool | `true` | Specifies whether to create a default service account for the Deepgram API Deployment. |
| api.serviceAccount.name | string | `nil` | Allows providing a custom service account name for the API component. If left empty, the default service account name will be used. If specified, and `api.serviceAccount.create = true`, this defines the name of the default service account. If specified, and `api.serviceAccount.create = false`, this provides the name of a preconfigured service account you wish to attach to the API deployment. |
| api.tolerations | list | `[]` | [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to apply to API pods. |
| api.updateStrategy.rollingUpdate.maxSurge | int | `1` | The maximum number of extra API pods that can be created during a rollingUpdate, relative to the number of replicas. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge) for more details. |
| api.updateStrategy.rollingUpdate.maxUnavailable | int | `0` | The maximum number of API pods, relative to the number of replicas, that can go offline during a rolling update. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-unavailable) for more details. |
| cluster-autoscaler.autoDiscovery.clusterName | string | `nil` | Name of your AWS EKS cluster. Using the [Cluster Autoscaler](https://github.com/kubernetes/autoscaler) on AWS requires knowledge of certain cluster metadata. |
| cluster-autoscaler.awsRegion | string | `nil` | Region of your AWS EKS cluster. Using the [Cluster Autoscaler](https://github.com/kubernetes/autoscaler) on AWS requires knowledge of certain cluster metadata. |
| cluster-autoscaler.enabled | bool | `false` | Set to `true` to enable node autoscaling with AWS EKS. Note needed for GKE, as autoscaling is enabled by a [cli option on cluster creation](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-autoscaler#creating_a_cluster_with_autoscaling). |
| cluster-autoscaler.rbac.serviceAccount.annotations."eks.amazonaws.com/role-arn" | string | `nil` | Replace with the AWS Role ARN configured for the Cluster Autoscaler. See the [Deepgram AWS EKS guide](https://developers.deepgram.com/docs/aws-k8s#creating-a-cluster) or [Cluster Autoscaler AWS documentation](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#permissions) for details. |
| cluster-autoscaler.rbac.serviceAccount.name | string | `"cluster-autoscaler-sa"` | Name of the IAM Service Account with the [necessary permissions](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#permissions) |
| engine.additionalAnnotations | object | `nil` | Additional annotations to add to the Engine deployment |
| engine.additionalLabels | object | `{}` | Additional labels to add to Engine resources |
| engine.affinity | object | `{}` | [Affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) to apply for Engine pods. |
| engine.chunking | object | `` | chunking defines the size of audio chunks to process in seconds. Adjusting these values will affect both inference performance and accuracy of results. Please contact your Deepgram Account Representative if you want to adjust any of these values. |
| engine.chunking.speechToText.batch.maxDuration | float | `nil` | minDuration is the maximum audio duration for a STT chunk size for a batch request |
| engine.chunking.speechToText.batch.minDuration | float | `nil` | minDuration is the minimum audio duration for a STT chunk size for a batch request |
| engine.chunking.speechToText.streaming.maxDuration | float | `nil` | minDuration is the maximum audio duration for a STT chunk size for a streaming request |
| engine.chunking.speechToText.streaming.minDuration | float | `nil` | minDuration is the minimum audio duration for a STT chunk size for a streaming request |
| engine.chunking.speechToText.streaming.step | float | `1` | step defines how often to return interim results, in seconds. This value may be lowered to increase the frequency of interim results. However, this also causes a significant decrease in the number of concurrent streams supported by a single GPU. Please contact your Deepgram Account representative for more details. |
| engine.concurrencyLimit.activeRequests | int | `nil` | activeRequests limits the number of active requests handled by a single Engine container. If additional requests beyond the limit are sent, the API container forming the request will try a different Engine pod. If no Engine pods are able to accept the request, the API will return a 429 HTTP response to the client. The `nil` default means no limit will be set. |
| engine.halfPrecision.state | string | `"auto"` | Engine will automatically enable half precision operations if your GPU supports them. You can explicitly enable or disable this behavior with the state parameter which supports `"enable"`, `"disabled"`, and `"auto"`. |
| engine.image.path | string | `"quay.io/deepgram/self-hosted-engine"` | path configures the image path to use for creating Engine containers. You may change this from the public Quay image path if you have imported Deepgram images into a private container registry. |
| engine.image.pullPolicy | string | `"IfNotPresent"` | pullPolicy configures how the Kubelet attempts to pull the Deepgram Engine image |
| engine.image.tag | string | `"release-240827"` | tag defines which Deepgram release to use for Engine containers |
| engine.livenessProbe | object | `` | Liveness probe customization for Engine pods. |
| engine.metricsServer | object | `` | metricsServer exposes an endpoint on each Engine container for reporting inference-specific system metrics. See https://developers.deepgram.com/docs/metrics-guide#deepgram-engine for more details. |
| engine.metricsServer.host | string | `"0.0.0.0"` | host is the IP address to listen on for metrics requests. You will want to listen on all interfaces to interact with other pods in the cluster. |
| engine.metricsServer.port | int | `9991` | port to listen on for metrics requests |
| engine.modelManager.models.links | list | `[]` | Links to your Deepgram models, if automatically downloading into storage backing a persistent volume. **Automatic downloads are currently supported for AWS EFS volumes only.** Insert each model link provided to you by your Deepgram Account Representative. |
| engine.modelManager.volumes.aws.efs.enabled | bool | `false` | Whether to use an [AWS Elastic File Sytem](https://aws.amazon.com/efs/) to store Deepgram models for use by Engine containers. This option requires your cluster to be running in [AWS EKS](https://aws.amazon.com/eks/). |
| engine.modelManager.volumes.aws.efs.fileSystemId | string | `nil` | FileSystemId of existing AWS Elastic File System where Deepgram model files will be persisted. You can find it using the AWS CLI: ``` $ aws efs describe-file-systems --query "FileSystems[*].FileSystemId" ``` |
| engine.modelManager.volumes.aws.efs.forceDownload | bool | `false` | Whether to force a fresh download of all model links provided, even if models are already present in EFS. |
| engine.modelManager.volumes.aws.efs.namePrefix | string | `"dg-models"` | Name prefix for the resources associated with the model storage in AWS EFS. |
| engine.modelManager.volumes.customVolumeClaim.enabled | bool | `false` | You may manually create your own PersistentVolume and PersistentVolumeClaim to store and expose model files to the Deepgram Engine. Configure your storage beforehand, and enable here. Note: Make sure the PV and PVC accessMode are set to `readWriteMany` or `readOnlyMany` |
| engine.modelManager.volumes.customVolumeClaim.modelsDirectory | string | `"/"` | Name of the directory within your pre-configured PersistentVolume where the models are stored |
| engine.modelManager.volumes.customVolumeClaim.name | string | `nil` | Name of your pre-configured PersistentVolumeClaim |
| engine.modelManager.volumes.gcp.gpd.enabled | bool | `false` | Whether to use an [GKE Persistent Disks](https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes) to store Deepgram models for use by Engine containers. This option requires your cluster to be running in [GCP GKE](https://cloud.google.com/kubernetes-engine). See the GKE documentation on [using pre-existing persistent disks](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/preexisting-pd). |
| engine.modelManager.volumes.gcp.gpd.fsType | string | `"ext4"` | |
| engine.modelManager.volumes.gcp.gpd.namePrefix | string | `"dg-models"` | Name prefix for the resources associated with the model storage in GCP GPD. |
| engine.modelManager.volumes.gcp.gpd.storageCapacity | string | `"40G"` | The size of your pre-existing persistent disk. |
| engine.modelManager.volumes.gcp.gpd.storageClassName | string | `"standard-rwo"` | The storageClassName of the existing persistent disk. |
| engine.modelManager.volumes.gcp.gpd.volumeHandle | string | `""` | The identifier of your pre-existing persistent disk. The format is projects/{project_id}/zones/{zone_name}/disks/{disk_name} for Zonal persistent disks, or projects/{project_id}/regions/{region_name}/disks/{disk_name} for Regional persistent disks. |
| engine.namePrefix | string | `"deepgram-engine"` | namePrefix is the prefix to apply to the name of all K8s objects associated with the Deepgram Engine containers. |
| engine.readinessProbe | object | `` | Readiness probe customization for Engine pods. |
| engine.resources | object | `` | Configure resource limits per Engine container. See [Deepgram's documentation](https://developers.deepgram.com/docs/self-hosted-deployment-environments#engine) for more details. |
| engine.resources.limits.gpu | int | `1` | gpu maps to the nvidia.com/gpu resource parameter |
| engine.resources.requests.gpu | int | `1` | gpu maps to the nvidia.com/gpu resource parameter |
| engine.securityContext | object | `{}` | [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for API pods. |
| engine.server | object | `` | Configure Engine containers to listen for requests from API containers. |
| engine.server.host | string | `"0.0.0.0"` | host is the IP address to listen on for inference requests. You will want to listen on all interfaces to interact with other pods in the cluster. |
| engine.server.port | int | `8080` | port to listen on for inference requests |
| engine.serviceAccount.create | bool | `true` | Specifies whether to create a default service account for the Deepgram Engine Deployment. |
| engine.serviceAccount.name | string | `nil` | Allows providing a custom service account name for the Engine component. If left empty, the default service account name will be used. If specified, and `engine.serviceAccount.create = true`, this defines the name of the default service account. If specified, and `engine.serviceAccount.create = false`, this provides the name of a preconfigured service account you wish to attach to the Engine deployment. |
| engine.startupProbe | object | `` | The startupProbe combination of `periodSeconds` and `failureThreshold` allows time for the container to load all models and start listening for incoming requests. Model load time can be affected by hardware I/O speeds, as well as network speeds if you are using a network volume mount for the models. If you are hitting the failure threshold before models are finished loading, you may want to extend the startup probe. However, this will also extend the time it takes to detect a pod that can't establish a network connection to validate its license. |
| engine.startupProbe.failureThreshold | int | `60` | failureThreshold defines how many unsuccessful startup probe attempts are allowed before the container will be marked as Failed |
| engine.startupProbe.periodSeconds | int | `10` | periodSeconds defines how often to execute the probe. |
| engine.tolerations | list | `[]` | [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to apply to Engine pods. |
| engine.updateStrategy.rollingUpdate.maxSurge | int | `1` | The maximum number of extra Engine pods that can be created during a rollingUpdate, relative to the number of replicas. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge) for more details. |
| engine.updateStrategy.rollingUpdate.maxUnavailable | int | `0` | The maximum number of Engine pods, relative to the number of replicas, that can go offline during a rolling update. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-unavailable) for more details. |
| global.additionalLabels | object | `{}` | Additional labels to add to all Deepgram resources |
| global.deepgramSecretRef | string | `nil` | Name of the pre-configured K8s Secret containing your Deepgram self-hosted API key. See chart docs for more details. |
| global.outstandingRequestGracePeriod | int | `1800` | When an API or Engine container is signaled to shutdown via Kubernetes sending a SIGTERM signal, the container will stop listening on its port, and no new requests will be routed to that container. However, the container will continue to run until all existing batch or streaming requests have completed, after which it will gracefully shut down. Batch requests should be finished within 10-15 minutes, but streaming requests can proceed indefinitely. outstandingRequestGracePeriod defines the period (in sec) after which Kubernetes will forcefully shutdown the container, terminating any outstanding connections. 1800 / 60 sec/min = 30 mins |
| global.pullSecretRef | string | `nil` | If using images from the Deepgram Quay image repositories, or another private registry to which your cluster doesn't have default access, you will need to provide a pre-configured K8s Secret with image repository credentials. See chart docs for more details. |
| gpu-operator | object | `` | Passthrough values for [NVIDIA GPU Operator Helm chart](https://github.com/NVIDIA/gpu-operator/blob/master/deployments/gpu-operator/values.yaml) You may use the NVIDIA GPU Operator to manage installation of NVIDIA drivers and the container toolkit on nodes with attached GPUs. |
| gpu-operator.driver.enabled | bool | `true` | Whether to install NVIDIA drivers on nodes where a NVIDIA GPU is detected. If your Kubernetes nodes run a base image that comes with NVIDIA drivers pre-configured, disable this option, but keep the parent `gpu-operator` and sibling `toolkit` options enabled. |
| gpu-operator.driver.version | string | `"550.54.15"` | NVIDIA driver version to install. |
| gpu-operator.enabled | bool | `true` | Whether to install the NVIDIA GPU Operator to manage driver and/or container toolkit installation. See the list of [supported Operating Systems](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/platform-support.html#supported-operating-systems-and-kubernetes-platforms) to verify compatibility with your cluster/nodes. Disable this option if your cluster/nodes are not compatible. If disabled, you will need to self-manage NVIDIA software installation on all nodes where you want to schedule Deepgram Engine pods. |
| gpu-operator.toolkit.enabled | bool | `true` | Whether to install NVIDIA drivers on nodes where a NVIDIA GPU is detected. |
| gpu-operator.toolkit.version | string | `"v1.15.0-ubi8"` | NVIDIA container toolkit to install. The default `ubuntu` image tag for the toolkit requires a dynamic runtime link to a version of GLIBC that may not be present on nodes running older Linux distribution releases, such as Ubuntu 22.04. Therefore, we specify the `ubi8` image, which statically links the GLIBC library and avoids this issue. |
| kube-prometheus-stack | object | `` | Passthrough values for [Prometheus k8s stack Helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack). Prometheus (and its adapter) should be configured when scaling.auto is enabled. You may choose to use the installation/configuration bundled in this Helm chart, or you may configure an existing Prometheus installation in your cluster to expose the needed values. See source Helm chart for explanation of available values. Default values provided in this chart are used to provide pod autoscaling for Deepgram pods. |
| kube-prometheus-stack.includeDependency | bool | `nil` | Normally, this chart will be installed if `scaling.auto.enabled` is true. However, if you wish to manage the Prometheus adapter in your cluster on your own and not as part of the Deepgram Helm chart, you can force it to not be installed by setting this to `false`. |
| licenseProxy | object | `` | Configuration options for the optional [Deepgram License Proxy](https://developers.deepgram.com/docs/license-proxy). |
| licenseProxy.additionalAnnotations | object | `nil` | Additional annotations to add to the LicenseProxy deployment |
| licenseProxy.additionalLabels | object | `{}` | Additional labels to add to License Proxy resources |
| licenseProxy.affinity | object | `{}` | [Affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity) to apply for License Proxy pods. |
| licenseProxy.deploySecondReplica | bool | `false` | If the License Proxy is deployed, one replica should be sufficient to support many API/Engine pods. Highly available environments may wish to deploy a second replica to ensure uptime, which can be toggled with this option. |
| licenseProxy.enabled | bool | `false` | The License Proxy is optional, but highly recommended to be deployed in production to enable highly available environments. |
| licenseProxy.image.path | string | `"quay.io/deepgram/self-hosted-license-proxy"` | path configures the image path to use for creating License Proxy containers. You may change this from the public Quay image path if you have imported Deepgram images into a private container registry. |
| licenseProxy.image.pullPolicy | string | `"IfNotPresent"` | pullPolicy configures how the Kubelet attempts to pull the Deepgram License Proxy image |
| licenseProxy.image.tag | string | `"release-240827"` | tag defines which Deepgram release to use for License Proxy containers |
| licenseProxy.keepUpstreamServerAsBackup | bool | `true` | Even with a License Proxy deployed, API and Engine pods can be configured to keep the upstream `license.deepgram.com` license server as a fallback licensing option if the License Proxy is unavailable. Disable this option if you are restricting API/Engine Pod network access for security reasons, and only the License Proxy should send egress traffic to the upstream license server. |
| licenseProxy.livenessProbe | object | `` | Liveness probe customization for Proxy pods. |
| licenseProxy.namePrefix | string | `"deepgram-license-proxy"` | namePrefix is the prefix to apply to the name of all K8s objects associated with the Deepgram License Proxy containers. |
| licenseProxy.readinessProbe | object | `` | Readiness probe customization for License Proxy pods. |
| licenseProxy.resources | object | `` | Configure resource limits per License Proxy container. See [Deepgram's documentation](https://developers.deepgram.com/docs/license-proxy#system-requirements) for more details. |
| licenseProxy.securityContext | object | `{}` | [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for API pods. |
| licenseProxy.server | object | `` | Configure how the license proxy will listen for licensing requests. |
| licenseProxy.server.baseUrl | string | `"/"` | baseUrl is the prefix for incoming license verification requests. |
| licenseProxy.server.host | string | `"0.0.0.0"` | host is the IP address to listen on. You will want to listen on all interfaces to interact with other pods in the cluster. |
| licenseProxy.server.port | int | `8443` | port to listen on. |
| licenseProxy.server.statusPort | int | `8080` | statusPort is the port to listen on for the status/health endpoint. |
| licenseProxy.serviceAccount.create | bool | `true` | Specifies whether to create a default service account for the Deepgram License Proxy Deployment. |
| licenseProxy.serviceAccount.name | string | `nil` | Allows providing a custom service account name for the LicenseProxy component. If left empty, the default service account name will be used. If specified, and `licenseProxy.serviceAccount.create = true`, this defines the name of the default service account. If specified, and `licenseProxy.serviceAccount.create = false`, this provides the name of a preconfigured service account you wish to attach to the License Proxy deployment. |
| licenseProxy.tolerations | list | `[]` | [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/) to apply to License Proxy pods. |
| licenseProxy.updateStrategy.rollingUpdate | object | `` | For the LicenseProxy, we only expose maxSurge and not maxUnavailable. This is to avoid accidentally having all LicenseProxy nodes go offline during upgrades, which could impact the entire cluster's connection to the Deepgram License Server. |
| licenseProxy.updateStrategy.rollingUpdate.maxSurge | int | `1` | The maximum number of extra License Proxy pods that can be created during a rollingUpdate, relative to the number of replicas. See the [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge) for more details. |
| prometheus-adapter | object | `` | Passthrough values for [Prometheus Adapter Helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-adapter). Prometheus, and its adapter here, should be configured when scaling.auto is enabled. You may choose to use the installation/configuration bundled in this Helm chart, or you may configure an existing Prometheus installation in your cluster to expose the needed values. See source Helm chart for explanation of available values. Default values provided in this chart are used to provide pod autoscaling for Deepgram pods. |
| prometheus-adapter.includeDependency | string | `nil` | Normally, this chart will be installed if `scaling.auto.enabled` is true. However, if you wish to manage the Prometheus adapter in your cluster on your own and not as part of the Deepgram Helm chart, you can force it to not be installed by setting this to `false`. |
| scaling | object | `` | Configuration options for horizontal scaling of Deepgram services. Only one of `static` and `auto` options can be enabled. |
| scaling.auto | object | `` | Enable pod autoscaling based on system load/traffic. |
| scaling.auto.api.metrics.custom | list | `nil` | If you have custom metrics you would like to scale with, you may add them here. See the [k8s docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) for how to structure a list of metrics |
| scaling.auto.api.metrics.engineToApiRatio | int | `4` | Scale the API deployment to this Engine-to-Api pod ratio |
| scaling.auto.engine.behavior | object | "*See values.yaml file for default*" | [Configurable scaling behavior](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#configurable-scaling-behavior) |
| scaling.auto.engine.maxReplicas | int | `10` | Maximum number of Engine replicas. |
| scaling.auto.engine.metrics.custom | list | `[]` | If you have custom metrics you would like to scale with, you may add them here. See the [k8s docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/) for how to structure a list of metrics |
| scaling.auto.engine.metrics.requestCapacityRatio | string | `nil` | If `engine.concurrencyLimit.activeRequests` is set, this variable will define the ratio of current active requests to maximum active requests at which the Engine pods will scale. Setting this value too close to 1.0 may lead to a situation where the cluster is at max capacity and rejects incoming requests. Setting the ratio too close to 0.0 will over-optimistically scale your cluster and increase compute costs unnecessarily. |
| scaling.auto.engine.metrics.speechToText.batch.requestsPerPod | int | `nil` | Scale the Engine pods based on a static desired number of speech-to-text batch requests per pod |
| scaling.auto.engine.metrics.speechToText.streaming.requestsPerPod | int | `nil` | Scale the Engine pods based on a static desired number of speech-to-text streaming requests per pod |
| scaling.auto.engine.metrics.textToSpeech.batch.requestsPerPod | int | `nil` | Scale the Engine pods based on a static desired number of text-to-speech batch requests per pod |
| scaling.auto.engine.minReplicas | int | `1` | Minimum number of Engine replicas. |
| scaling.replicas | object | `` | Number of replicas to set during initial installation. |
## Maintainers
| Name | Email | Url |
| ---- | ------ | --- |
| Deepgram Self-Hosted | <self.hosted@deepgram.com> | |
helm-templates/deepgram-onprem/README.md.gotmpl
+168
View File
@@ -0,0 +1,168 @@
{{ template "chart.header" . }}
{{ template "chart.deprecationWarning" . }}
{{ template "chart.versionBadge" . }}{{ template "chart.typeBadge" . }}{{ template "chart.appVersionBadge" . }}[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/deepgram-self-hosted)](https://artifacthub.io/packages/search?repo=deepgram-self-hosted)
{{ template "chart.description" . }}
{{ template "chart.homepageLine" . }}
**Deepgram Self-Hosted Kubernetes Guides:** <https://developers.deepgram.com/docs/kubernetes>
{{ template "chart.sourcesSection" . }}
{{ template "chart.requirementsSection" . }}
## Using the Chart
### Get Repository Info
```bash
helm repo add deepgram https://deepgram.github.io/self-hosted-resources
helm repo update
```
### Installing the Chart
The Deepgram self-hosted chart requires Helm 3.7+ in order to install successfully. Please check your helm release before installation.
You will need to provide your [self-service Deepgram licensing and credentials](https://developers.deepgram.com/docs/self-hosted-self-service-tutorial) information. See `global.deepgramSecretRef` and `global.pullSecretRef` in the [Values section](#values) for more details, and the [Deepgram Self-Hosted Kubernetes Guides](https://developers.deepgram.com/docs/kubernetes) for instructions on how to create these secrets.
You may also override any default configuration values. See [the Values section](#values) for a list of available options, and the [samples directory](./samples) for examples of a standard installation.
```
helm install -f my-values.yaml [RELEASE_NAME] deepgram/deepgram-self-hosted --atomic --timeout 45m
```
### Upgrade and Rollback Strategies
To upgrade the Deepgram components to a new version, follow these steps:
1. Update the various `image.tag` values in the `values.yaml` file to the desired version.
2. Run the Helm upgrade command:
```bash
helm upgrade -f my-values.yaml [RELEASE_NAME] deepgram/deepgram-self-hosted --atomic --timeout 60m
```
If you encounter any issues during the upgrade process, you can perform a rollback to the previous version:
```bash
helm rollback deepgram
```
Before upgrading, ensure that you have reviewed the release notes and any migration guides provided by Deepgram for the specific version you are upgrading to.
### Uninstalling the Chart
```bash
helm uninstall [RELEASE_NAME]
```
This removes all the Kubernetes components associated with the chart and deletes the release.
## Changelog
See the [chart CHANGELOG](./CHANGELOG.md) for a list of relevant changes for each version of the Helm chart.
For more details on changes to the underlying Deepgram resources, such as the container images or available models, see the [official Deepgram changelog](https://deepgram.com/changelog) ([RSS feed](https://deepgram.com/changelog.xml)).
## Chart Configuration
### Persistent Storage Options
The Deepgram Helm chart supports different persistent storage options for storing Deepgram models and data. The available options include:
- AWS Elastic File System (EFS)
- Google Cloud Persistent Disk (GPD)
- Custom PersistentVolumeClaim (PVC)
To configure a specific storage option, see the `engine.modelManager.volumes` [configuration values](#values). Make sure to provide the necessary configuration values for the selected storage option, such as the EFS file system ID or the GPD disk type and size.
For detailed instructions on setting up and configuring each storage option, refer to the [Deepgram self-hosted guides](https://developers.deepgram.com/docs/kubernetes) and the respective cloud provider's documentation.
### Autoscaling
Autoscaling your cluster's capacity to meet incoming traffic demands involves both node autoscaling and pod autoscaling. Node autoscaling for supported cloud providers is setup by default when using this Helm chart and creating your cluster with the [Deepgram self-hosted guides](https://developers.deepgram.com/docs/kubernetes). Pod autoscaling can be enabled via the `scaling.auto.enabled` configuration option in this chart.
#### Engine
The Engine component is the core of the Deepgram self-hosted platform, responsible for performing inference using your deployed models. Autoscaling increases the number of Engine replicas to maintain consistent performance for incoming traffic.
There are currently two primary ways to scale the Engine component: scaling with a hard request limit per Engine Pod, or scaling with a soft request limit per Engine pod.
To set a hard limit on which to scale, configure `engine.concurrencyLimit.activeRequests` and `scaling.auto.engine.metrics.requestCapacityRatio`. The `activeRequests` parameter will set a hard limit of how many requests any given Engine pod will accept, and the `requestCapacityRatio` will govern scaling the Engine deployment when a certain percentage of "available request slots" is filled. For example, a requestCapacityRatio of `0.8` will scale the Engine deployment when the current number of active requests is >=80% of the active request concurrency limit. If the cluster is not able to scale in time and current active requests hits 100% of the preset limit, additional client requests to the API will return a `429 Too Many Requests` HTTP response to clients. This hard limit means that if a request is accepted for inference, it will have consistent performance, as the cluster will refuse surplus requests that could overload the cluster and degrade performance, at the expense of possibly rejecting some incoming requests if capacity does not scale in time.
To set a soft limit on which to scale, configure `scaling.auto.engine.metrics.{speechToText,textToSpeech}.{batch,streaming}.requestsPerPod`, depending on the primary traffic source for your environment. The cluster will attempt to scale to meet this target for number of requests per Engine pod, but will not reject extra requests with a `429 Too Many Request` HTTP response like the hard limit will. If the number of extra requests increases faster than the cluster can scale additional capacity, all incoming requests will still be accepted, but the performance of individual requests may degrade.
> [!NOTE]
> Deepgram recommends provisioning separate environments for batch speech-to-text, streaming speech-to-text, and text-to-speech workloads because typical latency and throughput tradeoffs are different for each of those use cases.
There is also a `scaling.auto.engine.metrics.custom` configuration value available to define your own custom scaling metric, if needed.
#### API
The API component is responsible for accepting incoming requests and forming responses, delegating inference work to the Deepgram Engine as needed. A single API pod can typically handle delegating requests to multiple Engine pods, so it is more compute efficient to deploy fewer API pods relative to the number of Engine pods. The `scaling.auto.api.metrics.engineToApiRatio` configuration value defines the ratio between Engine to API pods. The default value is appropriate for most deployments.
There is also a `scaling.auto.api.metrics.custom` configuration value available to define your own custom scaling metric, if needed.
#### License Proxy
The [License Proxy](https://developers.deepgram.com/docs/license-proxy) is intended to be deployed as a fixed-scale deployment the proxies all licensing requests from your environment. It should not be upscaled with the traffic demands of your environment.
This chart deploys one License Proxy Pod per environment by default. If you wish to deploy a second License Proxy Pod for redundancy, set `licenseProxy.deploySecondReplica` to `true`.
### RBAC Configuration
Role-Based Access Control (RBAC) is used to control access to Kubernetes resources based on the roles and permissions assigned to users or service accounts. The Deepgram Helm chart includes default RBAC roles and bindings for the API, Engine, and License Proxy components.
To use custom RBAC roles and bindings based on your specific security requirements, you can individually specify pre-existing ServiceAccounts to bind to each deployment by specifying the following options in `values.yaml`:
```
{api|engine|licenseProxy}.serviceAccount.create=false
{api|engine|licenseProxy}.serviceAccount.name=<your-pre-existing-sa>
```
Make sure to review and adjust the RBAC configuration according to the principle of least privilege, granting only the necessary permissions for each component.
### Secret Management
The Deepgram Helm chart takes references to two existing secrets - one containing your distribution credentials to pull container images from Deepgram's image repository, and one containing your Deepgram self-hosted API key.
Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
## Getting Help
See the [Getting Help](../../README.md#getting-help) section in the root of this repository for a list of resources to help you troubleshoot and resolve issues.
### Troubleshooting
If you encounter issues while deploying or using Deepgram, consider the following troubleshooting steps:
1. Check the pod status and logs:
- Use `kubectl get pods` to check the status of the Deepgram pods.
- Use `kubectl logs <pod-name>` to view the logs of a specific pod.
2. Verify resource availability:
- Ensure that the cluster has sufficient CPU, memory, and storage resources to accommodate the Deepgram components.
- Check for any resource constraints or limits imposed by the namespace or the cluster.
3. Review the Kubernetes events:
- Use `kubectl get events` to view any events or errors related to the Deepgram deployment.
4. Check the network connectivity:
- Verify that the Deepgram components can communicate with each other and with the Deepgram license server (license.deepgram.com).
- Check the network policies and firewall rules to ensure that the necessary ports and protocols are allowed.
5. Collect diagnostic information:
- Gather relevant logs and metrics.
- Export your existing Helm chart values:
```bash
helm get values [RELEASE_NAME] > my-deployed-values.yaml
```
- Provide the collected diagnostic information to Deepgram for assistance.
{{ template "chart.valuesSection" . }}
{{ template "chart.maintainersSection" . }}
helm-templates/deepgram-onprem/charts/cluster-autoscaler-9.37.0.tgz
BIN
View File
Binary file not shown.
helm-templates/deepgram-onprem/charts/gpu-operator-v24.3.0.tgz
BIN
View File
Binary file not shown.
helm-templates/deepgram-onprem/charts/kube-prometheus-stack-60.5.0.tgz
BIN
View File
Binary file not shown.
helm-templates/deepgram-onprem/charts/prometheus-adapter-4.10.0.tgz
BIN
View File
Binary file not shown.
helm-templates/deepgram-onprem/samples/01-basic-setup-aws.cluster-config.yaml
+79
View File
@@ -0,0 +1,79 @@
apiVersion: eksctl.io/v1alpha5
kind: ClusterConfig
metadata:
name: deepgram-self-hosted-cluster
region: us-west-2
version: "1.30"
iam:
withOIDC: true
serviceAccounts:
- metadata:
name: cluster-autoscaler-sa
namespace: dg-self-hosted
wellKnownPolicies:
autoScaler: true
roleName: cluster-autoscaler-role
roleOnly: true
- metadata:
name: efs-csi-controller-sa
namespace: kube-system
wellKnownPolicies:
efsCSIController: true
roleName: efs-csi-driver-role
roleOnly: true
managedNodeGroups:
- name: control-plane-node-group
minSize: 1
desiredCapacity: 1
maxSize: 3
instanceType: t3.large
amiFamily: Ubuntu2204
iam:
withAddonPolicies:
autoScaler: true
propagateASGTags: true
- name: engine-node-group
minSize: 0
desiredCapacity: 0
maxSize: 8
instanceType: g6.2xlarge
amiFamily: Ubuntu2204
labels:
k8s.deepgram.com/node-type: engine
k8s.amazonaws.com/accelerator: nvidia-l4
iam:
withAddonPolicies:
efs: true
autoScaler: true
taints:
- key: efs.csi.aws.com/agent-not-ready
value: "true"
effect: NoExecute
propagateASGTags: true
- name: api-node-group
minSize: 0
desiredCapacity: 0
maxSize: 2
instanceType: c5n.xlarge
amiFamily: Ubuntu2204
labels:
k8s.deepgram.com/node-type: api
iam:
withAddonPolicies:
autoScaler: true
propagateASGTags: true
- name: license-proxy-node-group
minSize: 0
desiredCapacity: 0
maxSize: 2
instanceType: t3.large
amiFamily: Ubuntu2204
labels:
k8s.deepgram.com/node-type: license-proxy
iam:
withAddonPolicies:
autoScaler: true
propagateASGTags: true
helm-templates/deepgram-onprem/samples/01-basic-setup-aws.values.yaml
+136
View File
@@ -0,0 +1,136 @@
# See the Chart [README](https://github.com/deepgram/self-hosted-resources/blob/main/charts/deepgram-self-hosted#values)
# for documentation on all available options.
global:
# pullSecretRef should refer to a K8s secret that
# must be created prior to installing this Chart.
# Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
#
# You can create a secret for your image pull credentials
# with the following commands:
# ```bash
# docker login quay.io
# kubectl create secret generic dg-regcred \
# --from-file=.dockerconfigjson=$HOME/.docker/config.json \
# --type=kubernetes.io/dockerconfigjson
# ```
pullSecretRef: "dg-regcred"
# deepgramSecretRef should refer to a K8s secret that
# must be created prior to installing this Chart.
# Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
#
# You can create a secret for your Deepgram self-hosted API key
# with the following command:
# ```bash
# kubectl create secret generic dg-self-hosted-api-key --from-literal=DEEPGRAM_API_KEY='<id>'
# ```
deepgramSecretRef: "dg-self-hosted-api-key"
scaling:
replicas:
api: 1
engine: 1
auto:
# Can toggle to true to enable autoscaling. Make sure to set a value for one of the available metrics
enabled: false
engine:
metrics:
speechToText:
batch:
requestsPerPod: # Discuss a reasonable value with your Deepgram Account Representative
streaming:
requestsPerPod: # Discuss a reasonable value with your Deepgram Account Representative
textToSpeech:
batch:
requestsPerPod: # Discuss a reasonable value with your Deepgram Account Representative
# Discuss a reasoanble value with your Deepgram Account Representative
# Must also set engine.concurrencyLimit.activeRequests if using request ratio for autoscaling
requestCapacityRatio:
api:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: k8s.deepgram.com/node-type
operator: In
values:
- api
resources:
requests:
memory: "4Gi"
cpu: "2000m"
limits:
memory: "8Gi"
cpu: "4000m"
engine:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: k8s.deepgram.com/node-type
operator: In
values:
- engine
resources:
requests:
memory: "28Gi"
cpu: "6000m"
gpu: 1
limits:
memory: "40Gi"
cpu: "8000m"
gpu: 1
# Discuss a reasonable value with your Deepgram Account Representative
# If not using autoscaling, can be left empty, but must be set if using
# autoscaling with scaling.auto.engine.metrics.requestCapacityRatio
concurrencyLimit:
activeRequests:
modelManager:
volumes:
aws:
efs:
enabled: true
fileSystemId: fs-xxxxxxxxxxxxxxxx # Replace with your EFS ID
models:
links:
- https://link-to-model-1.dg # Replace these links with those provided to you
- https://link-to-model-2.dg # by your Deepgram Account Representative.
- https://link-to-model-3.dg
- ...
licenseProxy:
enabled: true
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: k8s.deepgram.com/node-type
operator: In
values:
- license-proxy
resources:
requests:
memory: "6Gi"
cpu: "1500m"
limits:
memory: "8Gi"
cpu: "2000m"
cluster-autoscaler:
enabled: true
rbac:
serviceAccount:
name: "cluster-autoscaler-sa"
annotations:
# Replace with the AWS Role ARN configured for the Cluster Autoscaler
eks.amazonaws.com/role-arn: "arn:aws:iam::000000000000:role/MyRoleName"
autoDiscovery:
clusterName: "deepgram-self-hosted-cluster"
awsRegion: "us-west-2"
helm-templates/deepgram-onprem/samples/02-basic-setup-gcp.yaml
+123
View File
@@ -0,0 +1,123 @@
# See the Chart [README](https://github.com/deepgram/self-hosted-resources/blob/main/charts/deepgram-self-hosted#values)
# for documentation on all available options.
global:
# pullSecretRef should refer to a K8s secret that
# must be created prior to installing this Chart.
# Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
#
# You can create a secret for your image pull credentials
# with the following commands:
# ```bash
# docker login quay.io
# kubectl create secret generic dg-regcred \
# --from-file=.dockerconfigjson=$HOME/.docker/config.json \
# --type=kubernetes.io/dockerconfigjson
# ```
pullSecretRef: "dg-regcred"
# deepgramSecretRef should refer to a K8s secret that
# must be created prior to installing this Chart.
# Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
#
# You can create a secret for your Deepgram self-hosted API key
# with the following command:
# ```bash
# kubectl create secret generic dg-self-hosted-api-key --from-literal=DEEPGRAM_API_KEY='<id>'
# ```
deepgramSecretRef: "dg-self-hosted-api-key"
scaling:
replicas:
api: 1
engine: 1
auto:
# Can toggle to true to enable autoscaling. Make sure to set a value for one of the available metrics
enabled: false
engine:
metrics:
speechToText:
batch:
requestsPerPod: # Discuss a reasonable value with your Deepgram Account Representative
streaming:
requestsPerPod: # Discuss a reasonable value with your Deepgram Account Representative
textToSpeech:
batch:
requestsPerPod: # Discuss a reasonable value with your Deepgram Account Representative
# Discuss a reasoanble value with your Deepgram Account Representative
# Must also set engine.concurrencyLimit.activeRequests if using request ratio for autoscaling
requestCapacityRatio:
api:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: k8s.deepgram.com/node-type
operator: In
values:
- api
resources:
requests:
memory: "4Gi"
cpu: "2000m"
limits:
memory: "12Gi"
cpu: "4000m"
engine:
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: k8s.deepgram.com/node-type
operator: In
values:
- engine
resources:
requests:
memory: "24Gi"
cpu: "5000m"
gpu: 1
limits:
memory: "40Gi"
cpu: "8000m"
gpu: 1
# Discuss a reasonable value with your Deepgram Account Representative
# If not using autoscaling, can be left empty, but must be set if using
# autoscaling with scaling.auto.engine.metrics.requestCapacityRatio
concurrencyLimit:
activeRequests:
modelManager:
volumes:
gcp:
gpd:
enabled: true
# Replace with your Google disk handle
volumeHandle: "projects/<your-project-id>/zones/<your-zone>/disks/<your-disk>"
licenseProxy:
enabled: true
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: k8s.deepgram.com/node-type
operator: In
values:
- license-proxy
resources:
requests:
memory: "4Gi"
cpu: "1000m"
limits:
memory: "8Gi"
cpu: "2000m"
gpu-operator:
# GKE will manage the driver and toolkit installation for us by default.
enabled: false
helm-templates/deepgram-onprem/samples/03-basic-setup-onprem.yaml
+50
View File
@@ -0,0 +1,50 @@
# See the Chart [README](https://github.com/deepgram/self-hosted-resources/blob/main/charts/deepgram-self-hosted#values)
# for documentation on all available options.
global:
# pullSecretRef should refer to a K8s secret that
# must be created prior to installing this Chart.
# Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
#
# You can create a secret for your image pull credentials
# with the following commands:
# ```bash
# docker login quay.io
# kubectl create secret generic dg-regcred \
# --from-file=.dockerconfigjson=$HOME/.docker/config.json \
# --type=kubernetes.io/dockerconfigjson
# ```
pullSecretRef: "dg-regcred"
# deepgramSecretRef should refer to a K8s secret that
# must be created prior to installing this Chart.
# Consult the [official Kubernetes documentation](https://kubernetes.io/docs/concepts/configuration/secret/) for best practices on configuring Secrets for use in your cluster.
#
# You can create a secret for your Deepgram self-hosted API key
# with the following command:
# ```bash
# kubectl create secret generic dg-self-hosted-api-key --from-literal=DEEPGRAM_API_KEY='<id>'
# ```
deepgramSecretRef: "dg-self-hosted-api-key"
scaling:
replicas:
api: 1
engine: 1
auto:
# Most on-premises clusters will have a static number of servers, where autoscaling may not be desirable.
enabled: false
engine:
modelManager:
volumes:
customVolumeClaim:
enabled: true
name: deepgram-models-pvc # Replace with the name of a PersistentVolumeClaim that maps to the PersistentVolume you have created
modelsDirectory: "/"
licenseProxy:
enabled: true
gpu-operator:
enabled: true
helm-templates/deepgram-onprem/samples/README.md
+9
View File
@@ -0,0 +1,9 @@
# Samples
This directory contains examples of how to use the Deepgram Helm chart in various configurations and with various cloud providers. The samples are not meant to be an exhaustive demonstration of all available options; please see the chart [README](../README.md) and [values.yaml](../values.yaml) for more information.
## AWS EKS Samples
See the [Deepgram AWS EKS guide](https://developers.deepgram.com/docs/aws-k8s) for detailed instructions on how to deploy Deepgram services in a managed Kubernetes cluster in AWS.
## GCP GKE Samples
See the [Deepgram GCP GKE guide](https://developers.deepgram.com/docs/gcp-k8s) for detailed instructions on how to deploy Deepgram services in a managed Kubernetes cluster in GCP.
helm-templates/deepgram-onprem/templates/NOTES.txt
+21
View File
@@ -0,0 +1,21 @@
This is an beta Helm chart for Deepgram self-hosted deployments.
Please refer to Deepgram's self-hosted documentation for further details:
* https://developers.deepgram.com/docs/self-hosted-introduction
This software includes the following third-party libraries:
NVIDIA GPU Operator
Copyright 2024 NVIDIA Corporation
Licensed under the Apache-2.0 license.
See https://github.com/NVIDIA/gpu-operator for more information.
Kubernetes Autoscaler
Copyright 2024 The Linux Foundation
Licensed under the Apache-2.0 license.
See https://github.com/kubernetes/autoscaler for more information.
Prometheus Helm Charts
Copyright 2024 The Linux Foundation
Licensed under the Apache-2.0 license.
See https://github.com/prometheus-community/helm-charts for more information.
helm-templates/deepgram-onprem/templates/_helpers.tpl
+29
View File
@@ -0,0 +1,29 @@
{{/*
Create chart name and version as used by the chart label.
*/}}
{{- define "deepgram-self-hosted.chart" -}}
{{- printf "%s-%s" .Chart.Name .Chart.Version | replace "+" "_" | trunc 63 | trimSuffix "-" }}
{{- end }}
{{/*
Common labels
*/}}
{{- define "deepgram-self-hosted.labels" -}}
app.kubernetes.io/name: "deepgram-self-hosted"
helm.sh/chart: {{ include "deepgram-self-hosted.chart" . }}
{{ include "deepgram-self-hosted.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- range $key, $val := .Values.global.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
{{- end }}
{{/*
Selector labels
*/}}
{{- define "deepgram-self-hosted.selectorLabels" -}}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
helm-templates/deepgram-onprem/templates/api/api.config.yaml
+62
View File
@@ -0,0 +1,62 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.api.namePrefix }}-config
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
data:
api.toml: |
[license]
server_url = [
{{- if .Values.licenseProxy.enabled }}
"https://{{ .Values.licenseProxy.namePrefix }}-internal:{{ .Values.licenseProxy.server.port}}{{ .Values.licenseProxy.server.baseUrl }}"
{{- end }}
{{- if or (not .Values.licenseProxy.enabled) .Values.licenseProxy.keepUpstreamServerAsBackup }}
{{- if .Values.licenseProxy.enabled -}},{{- end -}}"https://license.deepgram.com"
{{- end }}
]
[server]
base_url = "{{ .Values.api.server.baseUrl }}"
host = "{{ .Values.api.server.host }}"
port = {{ .Values.api.server.port }}
callback_conn_timeout = "{{ .Values.api.server.callbackConnTimeout }}"
callback_timeout = "{{ .Values.api.server.callbackTimeout }}"
fetch_conn_timeout = "{{ .Values.api.server.fetchConnTimeout }}"
fetch_timeout = "{{ .Values.api.server.fetchTimeout }}"
[resolver]
{{- if .Values.api.resolver.nameservers }}
{{- if gt (len .Values.api.resolver.nameservers) 0 }}
nameservers = [
{{- range $index, $element := .Values.api.resolver.nameservers }}
{{- if ne $index 0}},{{ end }}"{{ $element }}"
{{- end -}}
]
{{- end }}
{{- end }}
{{- if .Values.api.resolver.maxTTL }}
max_ttl = {{ .Values.api.resolver.maxTTL }}
{{- end }}
[features]
topic_detection = true
summarization = true
entity_detection = {{ .Values.api.features.entityDetection }}
entity_redaction = {{ .Values.api.features.entityRedaction }}
{{- if .Values.api.features.diskBufferPath }}
disk_buffer_path = "{{ .Values.api.features.diskBufferPath }}"
{{- end }}
[[driver_pool.standard]]
url = "https://{{ .Values.engine.namePrefix }}-internal:{{ .Values.engine.server.port}}/v2"
timeout_backoff = {{ .Values.api.driverPool.standard.timeoutBackoff }}
retry_sleep = "{{ .Values.api.driverPool.standard.retrySleep }}"
retry_backoff = {{ .Values.api.driverPool.standard.retryBackoff }}
max_response_size = {{ int .Values.api.driverPool.standard.maxResponseSize }}
helm-templates/deepgram-onprem/templates/api/api.deployment.yaml
+86
View File
@@ -0,0 +1,86 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.api.namePrefix }}
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
app: deepgram-api
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
{{- with .Values.api.additionalAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
selector:
matchLabels:
app: deepgram-api
{{ include "deepgram-self-hosted.selectorLabels" . }}
replicas: {{ .Values.scaling.replicas.api }}
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: {{ .Values.api.updateStrategy.rollingUpdate.maxUnavailable }}
maxSurge: {{ .Values.api.updateStrategy.rollingUpdate.maxSurge }}
template:
metadata:
labels: *labels
annotations:
checksum/config: {{ include (print $.Template.BasePath "/api/api.config.yaml") . | sha256sum }}
spec:
terminationGracePeriodSeconds: {{ .Values.global.outstandingRequestGracePeriod }}
{{- if .Values.global.pullSecretRef }}
imagePullSecrets:
- name: {{ .Values.global.pullSecretRef }}
{{- end }}
affinity:
{{- toYaml .Values.api.affinity | nindent 8 }}
tolerations:
{{- toYaml .Values.api.tolerations | nindent 8 }}
securityContext:
{{- toYaml .Values.licenseProxy.securityContext | nindent 8 }}
{{- if or .Values.api.serviceAccount.create .Values.api.serviceAccount.name }}
serviceAccountName: {{ default (printf "%s-sa" .Values.api.namePrefix) .Values.api.serviceAccount.name }}
{{- end }}
containers:
- name: {{ .Values.api.namePrefix }}
image: {{ .Values.api.image.path }}:{{ .Values.api.image.tag }}
imagePullPolicy: {{ .Values.api.image.pullPolicy }}
envFrom:
- secretRef:
name: {{ required "Missing Deepgram self-hosted API key - see `global.deepgramSecretRef`" .Values.global.deepgramSecretRef }}
command: [ "stem" ]
args: ["-v", "serve", "/etc/config/api.toml"]
resources:
requests:
memory: "{{ .Values.api.resources.requests.memory }}"
cpu: "{{ .Values.api.resources.requests.cpu }}"
limits:
memory: "{{ .Values.api.resources.limits.memory }}"
cpu: "{{ .Values.api.resources.limits.cpu }}"
volumeMounts:
- name: api-config-volume
mountPath: /etc/config
ports:
- name: primary
containerPort: {{ .Values.api.server.port }}
livenessProbe:
tcpSocket:
port: {{ .Values.api.server.port }}
initialDelaySeconds: {{ .Values.api.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.api.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.api.livenessProbe.timeoutSeconds }}
readinessProbe:
exec:
command:
- sh
- -c
- curl --silent --insecure http://localhost:{{ .Values.api.server.port }}{{ if not (regexMatch `^\/` .Values.api.server.baseUrl) }}/{{ end }}{{ .Values.api.server.baseUrl}}/status/engine | grep --quiet -e '^{\"engine_connection_status\"\:\"Connected\".*}$'
initialDelaySeconds: {{ .Values.api.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.api.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.api.readinessProbe.timeoutSeconds }}
volumes:
- name: api-config-volume
configMap:
name: {{ .Values.api.namePrefix }}-config
helm-templates/deepgram-onprem/templates/api/api.hpa.yaml
+31
View File
@@ -0,0 +1,31 @@
{{- if .Values.scaling.auto.enabled -}}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ .Values.api.namePrefix }}-hpa
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.api.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ .Values.api.namePrefix }}
minReplicas: {{ (divf .Values.scaling.auto.engine.minReplicas .Values.scaling.auto.api.metrics.engineToApiRatio) | ceil | int64 }}
maxReplicas: {{ (divf .Values.scaling.auto.engine.maxReplicas .Values.scaling.auto.api.metrics.engineToApiRatio) | ceil | int64 }}
metrics:
- type: External
external:
metric:
name: engine_to_api_pod_ratio
target:
type: Value
value: {{ .Values.scaling.auto.api.metrics.engineToApiRatio }}
{{- if .Values.scaling.auto.api.metrics.custom }}
{{- toYaml .Values.scaling.auto.api.metrics.custom | nindent 4 }}
{{- end }}
behavior:
{{- toYaml .Values.scaling.auto.api.behavior | nindent 4 }}
{{- end -}}
helm-templates/deepgram-onprem/templates/api/api.ingress.yaml
+33
View File
@@ -0,0 +1,33 @@
{{- if .Values.api.ingress.enabled -}}
{{- if .Values.api.createContourGateway -}}
{{- if or ( eq "contour-internal" .Values.api.ingress.ingressClassName ) ( eq "contour-external" .Values.api.ingress.ingressClassName ) ( eq "contour-internal-0" .Values.api.ingress.ingressClassName ) ( eq "contour-internal-1" .Values.api.ingress.ingressClassName ) ( eq "contour-external-0" .Values.api.ingress.ingressClassName ) ( eq "contour-external-1" .Values.api.ingress.ingressClassName ) }}
{{- $servicePortNumber := .Values.api.ingress.servicePortNumber -}}
{{- $pathType := .Values.api.ingress.pathType -}}
{{- $namespace := .Values.api.ingress.namespace -}}
{{- $ingressClassName := .Values.api.ingress.ingressClassName -}}
{{ $count := 0 | int }}
{{- range .Values.api.ingress.hosts }}
apiVersion: projectcontour.io/v1
kind: HTTPProxy
metadata:
namespace: {{ $namespace }}
name: {{ $namespace }}-{{ $count }}
annotations:
projectcontour.io/ingress.class: {{ $.Values.api.ingress.ingressClassName }}
spec:
ingressClassName: {{ $ingressClassName }}
routes:
- conditions:
- prefix: /
enableWebsockets: true
services:
- name: deepgram-api-external
port: 80
virtualhost:
fqdn: {{ .host }}
---
{{- end -}}
{{- end -}}
{{- end -}}
{{- end -}}
helm-templates/deepgram-onprem/templates/api/api.rbac.yaml
+42
View File
@@ -0,0 +1,42 @@
{{- if .Values.api.serviceAccount.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ default (printf "%s-sa" .Values.api.namePrefix) .Values.api.serviceAccount.name }}
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ .Values.api.namePrefix }}-role
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
rules:
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["{{ .Values.api.namePrefix }}-config"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["{{ .Values.global.pullSecretRef }}", "{{ .Values.global.deepgramSecretRef }}"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: ["{{ .Values.engine.namePrefix }}-internal"{{- if .Values.licenseProxy.enabled -}}, "{{ .Values.licenseProxy.namePrefix }}-internal"{{- end -}}]
verbs: ["get"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ .Values.api.namePrefix }}-rolebinding
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
subjects:
- kind: ServiceAccount
name: {{ default (printf "%s-sa" .Values.api.namePrefix) .Values.api.serviceAccount.name }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ .Values.api.namePrefix }}-role
{{- end }}
helm-templates/deepgram-onprem/templates/api/api.scaledobject.yaml
+55
View File
@@ -0,0 +1,55 @@
{{- if .Values.apiAutoscaling.enabled }}
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
namespace: {{ .Values.apiAutoscaling.namespace }}
name: deepgram-api-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ .Values.apiAutoscaling.targetName }}
pollingInterval: {{ .Values.apiAutoscaling.pollingInterval }}
{{- if ( default false (.Values.disasterRecovery).enabled ) }}
minReplicaCount: 1
{{- else }}
minReplicaCount: {{ .Values.apiAutoscaling.minReplicas }}
{{- end }}
maxReplicaCount: {{ .Values.apiAutoscaling.maxReplicas }}
advanced:
horizontalPodAutoscalerConfig:
behavior:
scaleDown:
stabilizationWindowSeconds: {{ .Values.apiAutoscaling.scaledown.stabilizationWindowSeconds }}
policies:
{{- range .Values.apiAutoscaling.scaledown.policies }}
- type: {{ .type }}
value: {{ .value }}
periodSeconds: {{ .periodseconds }}
{{- end }}
selectPolicy: {{ .Values.apiAutoscaling.scaledown.selectpolicy }}
scaleUp:
stabilizationWindowSeconds: {{ .Values.apiAutoscaling.scaleup.stabilizationWindowSeconds }}
policies:
{{- range .Values.apiAutoscaling.scaleup.policies }}
- type: {{ .type }}
value: {{ .value }}
periodSeconds: {{ .periodseconds }}
{{- end }}
selectPolicy: {{ .Values.apiAutoscaling.scaleup.selectpolicy }}
triggers:
{{- if ( default false (.Values.disasterRecovery).enabled ) }}
{{- range $.Values.apiAutoscaling.triggers }}
{{- if or (eq .type "cpu") (eq .type "memory") }}
- metadata:
{{- toYaml .metadata | nindent 8 }}
type: {{ .type }}
metricType: "Utilization"
{{- end }}
{{- end }}
{{- else }}
{{- toYaml .Values.apiAutoscaling.triggers | nindent 2 }}
{{ end }}
{{- end }}
helm-templates/deepgram-onprem/templates/api/api.service.yaml
+19
View File
@@ -0,0 +1,19 @@
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.api.namePrefix }}-external
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
selector:
app: deepgram-api
{{ include "deepgram-self-hosted.selectorLabels" . }}
ports:
- name: "primary"
port: 80
protocol: TCP
targetPort: {{ .Values.api.server.port }}
helm-templates/deepgram-onprem/templates/engine/engine.config.yaml
+68
View File
@@ -0,0 +1,68 @@
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.engine.namePrefix }}-config
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
data:
engine.toml: |
{{- if .Values.engine.concurrencyLimit.activeRequests }}
max_active_requests = {{ .Values.engine.concurrencyLimit.activeRequests }}
{{- end }}
[license]
server_url = [
{{- if .Values.licenseProxy.enabled }}
"https://{{ .Values.licenseProxy.namePrefix }}-internal:{{ .Values.licenseProxy.server.port}}{{ .Values.licenseProxy.server.baseUrl }}"
{{- end }}
{{- if or (not .Values.licenseProxy.enabled) .Values.licenseProxy.keepUpstreamServerAsBackup }}
{{- if .Values.licenseProxy.enabled -}},{{- end -}}"https://license.deepgram.com"
{{- end }}
]
[server]
host = "{{ .Values.engine.server.host }}"
port = {{ .Values.engine.server.port }}
[metrics_server]
host = "{{ .Values.engine.metricsServer.host }}"
port = {{ .Values.engine.metricsServer.port }}
[model_manager]
search_paths = [
"/models
{{- if .Values.engine.modelManager.volumes.customVolumeClaim.enabled -}}
{{- if not (regexMatch `^\/` .Values.engine.modelManager.volumes.customVolumeClaim.modelsDirectory) -}}
/
{{- end -}}
{{- .Values.engine.modelManager.volumes.customVolumeClaim.modelsDirectory -}}
{{- end -}}"
]
[features]
multichannel = true
language_detection = true
[chunking.batch]
{{- if .Values.engine.chunking.speechToText.batch.minDuration }}
min_duration = {{ .Values.engine.chunking.speechToText.batch.minDuration }}
{{- end }}
{{- if .Values.engine.chunking.speechToText.batch.maxDuration }}
max_duration = {{ .Values.engine.chunking.speechToText.batch.maxDuration }}
{{- end }}
[chunking.streaming]
{{- if .Values.engine.chunking.speechToText.streaming.minDuration }}
min_duration = {{ .Values.engine.chunking.speechToText.streaming.minDuration }}
{{- end }}
{{- if .Values.engine.chunking.speechToText.streaming.maxDuration }}
max_duration = {{ .Values.engine.chunking.speechToText.streaming.maxDuration }}
{{- end }}
step = {{ .Values.engine.chunking.speechToText.streaming.step }}
[half_precision]
state = "{{ .Values.engine.halfPrecision.state }}"
helm-templates/deepgram-onprem/templates/engine/engine.deployment.yaml
+144
View File
@@ -0,0 +1,144 @@
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.engine.namePrefix }}
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
app: deepgram-engine
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
{{- with .Values.engine.additionalAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
selector:
matchLabels:
app: deepgram-engine
{{ include "deepgram-self-hosted.selectorLabels" . }}
{{- if not .Values.engineAutoscaling.enabled }}
replicas: {{ .Values.scaling.replicas.engine }}
{{- end }}
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: {{ .Values.engine.updateStrategy.rollingUpdate.maxUnavailable }}
maxSurge: {{ .Values.engine.updateStrategy.rollingUpdate.maxSurge }}
template:
metadata:
labels: *labels
annotations:
checksum/config: {{ include (print $.Template.BasePath "/engine/engine.config.yaml") . | sha256sum }}
spec:
nodeSelector:
{{- toYaml .Values.engine.nodeSelector | nindent 8 }}
terminationGracePeriodSeconds: {{ .Values.global.outstandingRequestGracePeriod }}
{{- if .Values.global.pullSecretRef }}
imagePullSecrets:
- name: {{ .Values.global.pullSecretRef }}
{{- end }}
affinity:
{{- toYaml .Values.engine.affinity | nindent 8 }}
tolerations:
{{- toYaml .Values.engine.tolerations | nindent 8 }}
securityContext:
{{- toYaml .Values.licenseProxy.securityContext | nindent 8 }}
{{- if or .Values.engine.serviceAccount.create .Values.engine.serviceAccount.name }}
serviceAccountName: {{ default (printf "%s-sa" .Values.engine.namePrefix) .Values.engine.serviceAccount.name }}
{{- end }}
containers:
- name: {{ .Values.engine.namePrefix }}
image: {{ .Values.engine.image.path }}:{{ .Values.engine.image.tag }}
imagePullPolicy: {{ .Values.engine.image.pullPolicy }}
envFrom:
- secretRef:
name: {{ required "Missing Deepgram self-hosted API key - see `global.deepgramSecretRef`" .Values.global.deepgramSecretRef }}
{{- if le (int .Values.engine.resources.requests.gpu) 0 }}
env:
- name: NVIDIA_VISIBLE_DEVICES
value: "void"
{{- end }}
command: [ "impeller" ]
args: ["-v", "serve", "/etc/config/engine.toml"]
resources:
requests:
memory: "{{ .Values.engine.resources.requests.memory }}"
cpu: "{{ .Values.engine.resources.requests.cpu }}"
{{- if gt (int .Values.engine.resources.requests.gpu) 0 }}
nvidia.com/gpu: {{ .Values.engine.resources.requests.gpu }}
{{- end }}
limits:
memory: "{{ .Values.engine.resources.limits.memory }}"
cpu: "{{ .Values.engine.resources.limits.cpu }}"
{{- if gt (int .Values.engine.resources.limits.gpu) 0 }}
nvidia.com/gpu: {{ .Values.engine.resources.limits.gpu }}
{{- end }}
volumeMounts:
- name: engine-config-volume
mountPath: /etc/config
- name: models-volume
mountPath: /models
ports:
- name: primary
containerPort: {{ .Values.engine.server.port }}
- name: metrics
containerPort: {{ .Values.engine.metricsServer.port }}
startupProbe:
tcpSocket:
port: {{ .Values.engine.server.port }}
periodSeconds: {{ .Values.engine.startupProbe.periodSeconds }}
failureThreshold: {{ .Values.engine.startupProbe.failureThreshold }}
livenessProbe:
tcpSocket:
port: {{ .Values.engine.server.port }}
initialDelaySeconds: {{ .Values.engine.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.engine.livenessProbe.periodSeconds }}
failureThreshold: {{ .Values.engine.livenessProbe.failureThreshold }}
readinessProbe:
tcpSocket:
port: {{ .Values.engine.server.port }}
initialDelaySeconds: {{ .Values.engine.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.engine.readinessProbe.periodSeconds }}
failureThreshold: {{ .Values.engine.readinessProbe.failureThreshold }}
{{- if .Values.engine.lifecycle }}
lifecycle:
{{- toYaml .Values.engine.lifecycle | nindent 10 }}
{{- end }}
volumes:
- name: engine-config-volume
configMap:
name: {{ .Values.engine.namePrefix }}-config
- name: models-volume
persistentVolumeClaim:
{{- $customClaimEnabled := .Values.engine.modelManager.volumes.customVolumeClaim.enabled }}
{{- $customClaimName := .Values.engine.modelManager.volumes.customVolumeClaim.name }}
{{- $awsEfsEnabled := .Values.engine.modelManager.volumes.aws.efs.enabled }}
{{- $gcpGpdEnabled := .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
{{- $nova3Enabled := .Values.engine.modelManager.volumes.nova3.enabled }}
{{- $nova3MultilingualEnabled := .Values.engine.modelManager.volumes.nova3.multilingual.enabled }}
{{- $enabledCount := (int $customClaimEnabled) | add (int $awsEfsEnabled) | add (int $gcpGpdEnabled) }}
{{- if eq $enabledCount 0 }}
{{- fail "Error: At least one of customVolumeClaim.enabled, aws.efs.enabled, or gcp.gpd.enabled must be set to true." }}
{{- else if gt $enabledCount 1 }}
{{- fail "Error: Only one of customVolumeClaim.enabled, aws.efs.enabled, or gcp.gpd.enabled can be set to true." }}
{{- end }}
{{- if $customClaimEnabled }}
{{- if not $customClaimName }}
{{- fail "Error: customVolumeClaim.name must be set when customVolumeClaim.enabled is true." }}
{{- else }}
claimName: {{ $customClaimName }}
{{- end }}
{{- else if $awsEfsEnabled }}
claimName: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-pvc
{{- else if and $gcpGpdEnabled $nova3Enabled }}
claimName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-nova3
{{- else if and $gcpGpdEnabled $nova3MultilingualEnabled }}
claimName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-nova3-multilingual
{{- else }}
claimName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-v1
{{- end }}
helm-templates/deepgram-onprem/templates/engine/engine.hpa.yaml
+73
View File
@@ -0,0 +1,73 @@
{{- if .Values.scaling.auto.enabled -}}
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: {{ .Values.engine.namePrefix }}-hpa
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ .Values.engine.namePrefix }}
minReplicas: {{ .Values.scaling.auto.engine.minReplicas }}
maxReplicas: {{ .Values.scaling.auto.engine.maxReplicas }}
metrics:
{{- $sttBatchScaling := .Values.scaling.auto.engine.metrics.speechToText.batch.requestsPerPod }}
{{- $sttStreamingScaling := .Values.scaling.auto.engine.metrics.speechToText.streaming.requestsPerPod }}
{{- $ttsBatchScaling := .Values.scaling.auto.engine.metrics.textToSpeech.batch.requestsPerPod }}
{{- $requestCapacityScaling := .Values.scaling.auto.engine.metrics.requestCapacityRatio }}
{{- $customScaling := .Values.scaling.auto.engine.metrics.custom }}
{{- if and (not $sttBatchScaling) (not $sttStreamingScaling) (not $ttsBatchScaling) (not $requestCapacityScaling) (not $customScaling) }}
{{- fail "Error: When autoscaling is enabled, at least one scaling metric in scaling.auto.engine.metrics must be defined." }}
{{- end }}
{{- if .Values.scaling.auto.engine.metrics.speechToText.batch.requestsPerPod }}
- type: External
external:
metric:
name: engine_active_requests_stt_batch
target:
type: Value
value: {{ .Values.scaling.auto.engine.metrics.speechToText.batch.requestsPerPod }}
{{- end }}
{{- if .Values.scaling.auto.engine.metrics.speechToText.streaming.requestsPerPod }}
- type: External
external:
metric:
name: engine_active_requests_stt_streaming
target:
type: Value
value: {{ .Values.scaling.auto.engine.metrics.speechToText.streaming.requestsPerPod }}
{{- end }}
{{- if .Values.scaling.auto.engine.metrics.textToSpeech.batch.requestsPerPod }}
- type: External
external:
metric:
name: engine_active_requests_tts_batch
target:
type: Value
value: {{ .Values.scaling.auto.engine.metrics.textToSpeech.batch.requestsPerPod }}
{{- end }}
{{- if .Values.scaling.auto.engine.metrics.requestCapacityRatio }}
{{- if not .Values.engine.concurrencyLimit.activeRequests }}
{{- fail "Error: engine.concurrencyLimit.activeRequests must be set to autoscale using requestCapacityRatio." }}
{{- end }}
- type: External
external:
metric:
name: engine_requests_active_to_max_ratio
target:
type: Value
value: {{ .Values.scaling.auto.engine.metrics.requestCapacityRatio }}
{{- end }}
{{- if .Values.scaling.auto.engine.metrics.custom }}
{{- toYaml .Values.scaling.auto.engine.metrics.custom | nindent 4 }}
{{- end }}
behavior:
{{- toYaml .Values.scaling.auto.engine.behavior | nindent 4 }}
{{- end -}}
helm-templates/deepgram-onprem/templates/engine/engine.rbac.yaml
+49
View File
@@ -0,0 +1,49 @@
{{- if .Values.engine.serviceAccount.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ default (printf "%s-sa" .Values.engine.namePrefix) .Values.engine.serviceAccount.name }}
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ .Values.engine.namePrefix }}-role
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
rules:
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["{{ .Values.engine.namePrefix }}-config"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["{{ .Values.global.pullSecretRef }}", "{{ .Values.global.deepgramSecretRef }}"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["services"]
resourceNames: [{{- if .Values.licenseProxy.enabled -}}"{{ .Values.licenseProxy.namePrefix }}-internal"{{- end -}}]
verbs: ["get"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
resourceNames: ["{{ .Values.engine.modelManager.volumes.customVolumeClaim.name }}", "{{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-pvc", "{{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc" ]
verbs: ["get", "list", "watch"]
- apiGroups: ["nvidia.com"]
resources: ["*"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ .Values.engine.namePrefix }}-rolebinding
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
subjects:
- kind: ServiceAccount
name: {{ default (printf "%s-sa" .Values.engine.namePrefix) .Values.engine.serviceAccount.name }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ .Values.engine.namePrefix }}-role
{{- end }}
helm-templates/deepgram-onprem/templates/engine/engine.scaledobject.yaml
+55
View File
@@ -0,0 +1,55 @@
{{- if .Values.engineAutoscaling.enabled }}
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata:
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
namespace: {{ .Values.engineAutoscaling.namespace }}
name: deepgram-engine-hpa
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: {{ .Values.engineAutoscaling.targetName }}
pollingInterval: {{ .Values.engineAutoscaling.pollingInterval }}
{{- if ( default false (.Values.disasterRecovery).enabled ) }}
minReplicaCount: 1
{{- else }}
minReplicaCount: {{ .Values.engineAutoscaling.minReplicas }}
{{- end }}
maxReplicaCount: {{ .Values.engineAutoscaling.maxReplicas }}
advanced:
horizontalPodAutoscalerConfig:
behavior:
scaleDown:
stabilizationWindowSeconds: {{ .Values.engineAutoscaling.scaledown.stabilizationWindowSeconds }}
policies:
{{- range .Values.engineAutoscaling.scaledown.policies }}
- type: {{ .type }}
value: {{ .value }}
periodSeconds: {{ .periodseconds }}
{{- end }}
selectPolicy: {{ .Values.engineAutoscaling.scaledown.selectpolicy }}
scaleUp:
stabilizationWindowSeconds: {{ .Values.engineAutoscaling.scaleup.stabilizationWindowSeconds }}
policies:
{{- range .Values.engineAutoscaling.scaleup.policies }}
- type: {{ .type }}
value: {{ .value }}
periodSeconds: {{ .periodseconds }}
{{- end }}
selectPolicy: {{ .Values.engineAutoscaling.scaleup.selectpolicy }}
triggers:
{{- if ( default false (.Values.disasterRecovery).enabled ) }}
{{- range $.Values.engineAutoscaling.triggers }}
{{- if or (eq .type "cpu") (eq .type "memory") }}
- metadata:
{{- toYaml .metadata | nindent 8 }}
type: {{ .type }}
metricType: "Utilization"
{{- end }}
{{- end }}
{{- else }}
{{- toYaml .Values.engineAutoscaling.triggers | nindent 2 }}
{{ end }}
{{- end }}
helm-templates/deepgram-onprem/templates/engine/engine.service.yaml
+38
View File
@@ -0,0 +1,38 @@
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.engine.namePrefix }}-metrics
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
selector:
app: deepgram-engine
{{ include "deepgram-self-hosted.selectorLabels" . }}
ports:
- name: "metrics"
port: {{ .Values.engine.metricsServer.port }}
targetPort: {{ .Values.engine.metricsServer.port }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.engine.namePrefix }}-internal
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.engine.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
selector:
app: deepgram-engine
{{ include "deepgram-self-hosted.selectorLabels" . }}
ports:
- name: "primary"
port: {{ .Values.engine.server.port }}
targetPort: {{ .Values.engine.server.port }}
helm-templates/deepgram-onprem/templates/license-proxy/license-proxy.config.yaml
+18
View File
@@ -0,0 +1,18 @@
{{- if .Values.licenseProxy.enabled -}}
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ .Values.licenseProxy.namePrefix }}-config
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.licenseProxy.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
data:
license-proxy.toml: |
[server]
host = "{{ .Values.licenseProxy.server.host }}"
port = {{ .Values.licenseProxy.server.port }}
base_url = "{{ .Values.licenseProxy.server.baseUrl }}"
status_port = {{ .Values.licenseProxy.server.statusPort }}
{{- end -}}
helm-templates/deepgram-onprem/templates/license-proxy/license-proxy.deployment.yaml
+89
View File
@@ -0,0 +1,89 @@
{{- if .Values.licenseProxy.enabled -}}
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.licenseProxy.namePrefix }}
labels: &labels
{{ include "deepgram-self-hosted.labels" . | indent 4}}
app: deepgram-license-proxy
{{- range $key, $val := .Values.licenseProxy.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end }}
{{- with .Values.licenseProxy.additionalAnnotations }}
annotations:
{{- toYaml . | nindent 4 }}
{{- end }}
spec:
selector:
matchLabels:
app: deepgram-license-proxy
{{ include "deepgram-self-hosted.selectorLabels" . }}
replicas: {{ if .Values.licenseProxy.deploySecondReplica }}2{{ else }}1{{ end }}
strategy:
type: RollingUpdate
rollingUpdate:
maxUnavailable: 0
maxSurge: {{ .Values.licenseProxy.updateStrategy.rollingUpdate.maxSurge }}
template:
metadata:
labels: *labels
annotations:
checksum/config: {{ include (print $.Template.BasePath "/license-proxy/license-proxy.config.yaml") . | sha256sum }}
spec:
{{- if .Values.global.pullSecretRef }}
imagePullSecrets:
- name: {{ .Values.global.pullSecretRef }}
{{- end }}
affinity:
{{- toYaml .Values.licenseProxy.affinity | nindent 8 }}
tolerations:
{{- toYaml .Values.licenseProxy.tolerations | nindent 8 }}
securityContext:
{{- toYaml .Values.licenseProxy.securityContext | nindent 8 }}
{{- if or .Values.licenseProxy.serviceAccount.create .Values.licenseProxy.serviceAccount.name }}
serviceAccountName: {{ default (printf "%s-sa" .Values.licenseProxy.namePrefix) .Values.licenseProxy.serviceAccount.name }}
{{- end }}
containers:
- name: {{ .Values.licenseProxy.namePrefix }}
image: {{ .Values.licenseProxy.image.path }}:{{ .Values.licenseProxy.image.tag }}
imagePullPolicy: {{ .Values.licenseProxy.image.pullPolicy }}
envFrom:
- secretRef:
name: {{ required "Missing Deepgram self-hosted API key - see `global.deepgramSecretRef`" .Values.global.deepgramSecretRef }}
command: [ "hermes" ]
args: ["-v", "serve", "/etc/config/license-proxy.toml"]
resources:
requests:
memory: "{{ .Values.licenseProxy.resources.requests.memory }}"
cpu: "{{ .Values.licenseProxy.resources.requests.cpu }}"
limits:
memory: "{{ .Values.licenseProxy.resources.limits.memory }}"
cpu: "{{ .Values.licenseProxy.resources.limits.cpu }}"
volumeMounts:
- name: license-proxy-config-volume
mountPath: /etc/config
ports:
- name: primary
containerPort: {{ .Values.licenseProxy.server.port }}
- name: status
containerPort: {{ .Values.licenseProxy.server.statusPort }}
livenessProbe:
tcpSocket:
port: {{ .Values.licenseProxy.server.statusPort }}
initialDelaySeconds: {{ .Values.licenseProxy.livenessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.licenseProxy.livenessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.licenseProxy.livenessProbe.timeoutSeconds }}
readinessProbe:
exec:
command:
- sh
- -c
- curl --silent http://localhost:{{ .Values.licenseProxy.server.statusPort }}/v1/status | grep --quiet -e '^{.*\"state\"\:\"\(Connected\|TrustBased\)\".*}$'
initialDelaySeconds: {{ .Values.licenseProxy.readinessProbe.initialDelaySeconds }}
periodSeconds: {{ .Values.licenseProxy.readinessProbe.periodSeconds }}
timeoutSeconds: {{ .Values.licenseProxy.readinessProbe.timeoutSeconds }}
volumes:
- name: license-proxy-config-volume
configMap:
name: {{ .Values.licenseProxy.namePrefix }}-config
{{- end -}}
helm-templates/deepgram-onprem/templates/license-proxy/license-proxy.rbac.yaml
+38
View File
@@ -0,0 +1,38 @@
{{- if and .Values.licenseProxy.enabled .Values.licenseProxy.serviceAccount.create }}
apiVersion: v1
kind: ServiceAccount
metadata:
name: {{ default (printf "%s-sa" .Values.licenseProxy.namePrefix) .Values.licenseProxy.serviceAccount.name }}
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
---
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: {{ .Values.licenseProxy.namePrefix }}-role
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
rules:
- apiGroups: [""]
resources: ["configmaps"]
resourceNames: ["{{ .Values.licenseProxy.namePrefix }}-config"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["secrets"]
resourceNames: ["{{ .Values.global.pullSecretRef }}", "{{ .Values.global.deepgramSecretRef }}"]
verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: {{ .Values.licenseProxy.namePrefix }}-rolebinding
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
subjects:
- kind: ServiceAccount
name: {{ default (printf "%s-sa" .Values.licenseProxy.namePrefix) .Values.licenseProxy.serviceAccount.name }}
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: {{ .Values.licenseProxy.namePrefix }}-role
{{- end }}
helm-templates/deepgram-onprem/templates/license-proxy/license-proxy.service.yaml
+40
View File
@@ -0,0 +1,40 @@
{{- if .Values.licenseProxy.enabled -}}
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.licenseProxy.namePrefix }}-status
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.licenseProxy.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
selector:
app: deepgram-license-proxy
{{ include "deepgram-self-hosted.selectorLabels" . }}
type: NodePort
ports:
- name: "status"
port: {{ .Values.licenseProxy.server.statusPort }}
targetPort: {{ .Values.licenseProxy.server.statusPort }}
---
apiVersion: v1
kind: Service
metadata:
name: {{ .Values.licenseProxy.namePrefix }}-internal
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
{{- range $key, $val := .Values.licenseProxy.additionalLabels }}
{{ $key }}: {{ $val | quote }}
{{- end}}
spec:
selector:
app: deepgram-license-proxy
{{ include "deepgram-self-hosted.selectorLabels" . }}
ports:
- name: "primary"
port: {{ .Values.licenseProxy.server.port }}
targetPort: {{ .Values.licenseProxy.server.port }}
{{- end -}}
helm-templates/deepgram-onprem/templates/volumes/aws/efs-model-download.job.yaml
+57
View File
@@ -0,0 +1,57 @@
{{- if .Values.engine.modelManager.volumes.aws.efs.enabled }}
{{- if .Values.engine.modelManager.models.links }}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-models-download
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
annotations:
"helm.sh/hook": post-install,post-upgrade
"helm.sh/hook-delete-policy": before-hook-creation
spec:
template:
spec:
affinity:
{{- toYaml .Values.engine.affinity | nindent 8 }}
containers:
- name: model-download
image: alpine
command:
- /bin/sh
- -c
- |
LINKS=$(cat <<EOF
{{- range .Values.engine.modelManager.models.links }}
{{ . }}
{{- end }}
EOF
)
TIMESTAMP_FORMAT="%Y-%m-%dT%H:%M:%SZ"
echo "$LINKS" | while IFS= read -r link; do
FILE_NAME=$(basename "$link")
if [ "{{ .Values.engine.modelManager.volumes.aws.efs.forceDownload }}" == "true" ]; then
printf "[%s] [INFO] Force downloading model file: %s\n" "$(date -u +$TIMESTAMP_FORMAT)" "$FILE_NAME"
wget -O "/mnt/efs/$FILE_NAME" "$link"
printf "[%s] [INFO] Downloaded model file (force download): %s\n" "$(date -u +$TIMESTAMP_FORMAT)" "$FILE_NAME"
elif [ ! -f "/mnt/efs/$FILE_NAME" ]; then
printf "[%s] [INFO] Downloading model file: %s\n" "$(date -u +$TIMESTAMP_FORMAT)" "$FILE_NAME"
wget -O "/mnt/efs/$FILE_NAME" "$link"
printf "[%s] [INFO] Downloaded model file: %s\n" "$(date -u +$TIMESTAMP_FORMAT)" "$FILE_NAME"
else
printf "[%s] [INFO] Model file already exists in EFS storage: %s\n" "$(date -u +$TIMESTAMP_FORMAT)" "$FILE_NAME"
fi
done
volumeMounts:
- name: aws-efs-volume
mountPath: /mnt/efs
restartPolicy: OnFailure
volumes:
- name: aws-efs-volume
persistentVolumeClaim:
claimName: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-pvc
{{- end }}
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/aws/efs.pv.yaml
+24
View File
@@ -0,0 +1,24 @@
{{- if .Values.engine.modelManager.volumes.aws.efs.enabled }}
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-pv
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
capacity:
# EFS will grow and shrink as files are added.
# This is just a starting request.
storage: 5Gi
volumeMode: Filesystem
accessModes:
- ReadWriteMany
claimRef:
name: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-pvc
namespace: {{ .Release.Namespace }}
storageClassName: aws-efs-sc
persistentVolumeReclaimPolicy: Retain
csi:
driver: efs.csi.aws.com
volumeHandle: {{ required "Missing AWS EFS fileSystemId, see `.engine.modelManager.volumes.aws.efs`" .Values.engine.modelManager.volumes.aws.efs.fileSystemId }}
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/aws/efs.pvc.yaml
+15
View File
@@ -0,0 +1,15 @@
{{- if .Values.engine.modelManager.volumes.aws.efs.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-pvc
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
accessModes:
- ReadWriteMany
storageClassName: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-sc
resources:
requests:
storage: 5Gi
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/aws/efs.storageClass.yaml
+9
View File
@@ -0,0 +1,9 @@
{{- if .Values.engine.modelManager.volumes.aws.efs.enabled }}
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: {{ .Values.engine.modelManager.volumes.aws.efs.namePrefix }}-aws-efs-sc
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
provisioner: efs.csi.aws.com
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pv.nova3.multilingual.yaml
+22
View File
@@ -0,0 +1,22 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv-nova3-multilingual
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
capacity:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
accessModes:
- ReadOnlyMany
claimRef:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-nova3-multilingual
namespace: {{ .Release.Namespace }}
csi:
driver: pd.csi.storage.gke.io
volumeHandle: {{ .Values.engine.modelManager.volumes.gcp.gpd.volumeHandle }}
fsType: {{ .Values.engine.modelManager.volumes.gcp.gpd.fsType }}
readOnly: true
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pv.nova3.yaml
+22
View File
@@ -0,0 +1,22 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv-nova3
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
capacity:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
accessModes:
- ReadOnlyMany
claimRef:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-nova3
namespace: {{ .Release.Namespace }}
csi:
driver: pd.csi.storage.gke.io
volumeHandle: {{ .Values.engine.modelManager.volumes.gcp.gpd.volumeHandle }}
fsType: {{ .Values.engine.modelManager.volumes.gcp.gpd.fsType }}
readOnly: true
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pv.v1.yaml
+22
View File
@@ -0,0 +1,22 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv-v1
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
capacity:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
accessModes:
- ReadOnlyMany
claimRef:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-v1
namespace: {{ .Release.Namespace }}
csi:
driver: pd.csi.storage.gke.io
volumeHandle: {{ .Values.engine.modelManager.volumes.gcp.gpd.volumeHandle }}
fsType: {{ .Values.engine.modelManager.volumes.gcp.gpd.fsType }}
readOnly: true
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pv.yaml
+22
View File
@@ -0,0 +1,22 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolume
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
capacity:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
accessModes:
- ReadOnlyMany
claimRef:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc
namespace: {{ .Release.Namespace }}
csi:
driver: pd.csi.storage.gke.io
volumeHandle: {{ .Values.engine.modelManager.volumes.gcp.gpd.volumeHandle }}
fsType: {{ .Values.engine.modelManager.volumes.gcp.gpd.fsType }}
readOnly: true
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pvc.nova3.multilingual.yaml
+16
View File
@@ -0,0 +1,16 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-nova3-multilingual
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
volumeName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv-nova3-multilingual
accessModes:
- ReadOnlyMany
resources:
requests:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pvc.nova3.yaml
+16
View File
@@ -0,0 +1,16 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-nova3
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
volumeName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv-nova3
accessModes:
- ReadOnlyMany
resources:
requests:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pvc.v1.yaml
+16
View File
@@ -0,0 +1,16 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc-v1
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
volumeName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv-v1
accessModes:
- ReadOnlyMany
resources:
requests:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
{{- end }}
helm-templates/deepgram-onprem/templates/volumes/gcp/gpd.pvc.yaml
+16
View File
@@ -0,0 +1,16 @@
{{- if .Values.engine.modelManager.volumes.gcp.gpd.enabled }}
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pvc
labels:
{{ include "deepgram-self-hosted.labels" . | indent 4}}
spec:
storageClassName: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageClassName }}
volumeName: {{ .Values.engine.modelManager.volumes.gcp.gpd.namePrefix }}-gcp-gpd-pv
accessModes:
- ReadOnlyMany
resources:
requests:
storage: {{ .Values.engine.modelManager.volumes.gcp.gpd.storageCapacity }}
{{- end }}
helm-templates/deepgram-onprem/values.yaml
+811
View File
@@ -0,0 +1,811 @@
global:
# -- (string) If using images from the Deepgram Quay image repositories,
# or another private registry to which your cluster doesn't have default access,
# you will need to provide a pre-configured K8s Secret
# with image repository credentials. See chart docs for more details.
pullSecretRef: dg-regcred
# -- (string) Name of the pre-configured K8s Secret containing your Deepgram
# self-hosted API key. See chart docs for more details.
deepgramSecretRef: dg-self-hosted-api-key
# -- Additional labels to add to all Deepgram resources
additionalLabels: {}
# -- When an API or Engine container is signaled to shutdown via Kubernetes sending a SIGTERM
# signal, the container will stop listening on its port, and no new requests will be routed
# to that container. However, the container will continue to run until all existing
# batch or streaming requests have completed, after which it will gracefully shut down.
#
# Batch requests should be finished within 10-15 minutes, but streaming requests can proceed indefinitely.
#
# outstandingRequestGracePeriod defines the period (in sec) after which Kubernetes will forcefully
# shutdown the container, terminating any outstanding connections. 1800 / 60 sec/min = 30 mins
outstandingRequestGracePeriod: 1800
# -- Configuration options for horizontal scaling of Deepgram
# services. Only one of `static` and `auto` options can be enabled.
# @default -- ``
scaling:
# -- Number of replicas to set during initial installation.
# @default -- ``
replicas:
api: 1
engine: 1
# -- Enable pod autoscaling based on system load/traffic.
# @default -- ``
auto:
enabled: false
api:
metrics:
# -- Scale the API deployment to this Engine-to-Api pod ratio
engineToApiRatio: 4
# -- (list) If you have custom metrics you would like to scale with, you may add them here.
# See the [k8s docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)
# for how to structure a list of metrics
custom:
# -- [Configurable scaling behavior](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#configurable-scaling-behavior)
# @default -- "*See values.yaml file for default*"
behavior:
scaleDown:
policies:
- type: Pods
value: 1
periodSeconds: 60
- type: Percent
value: 25
periodSeconds: 60
engine:
# -- Minimum number of Engine replicas.
minReplicas: 1
# -- Maximum number of Engine replicas.
maxReplicas: 10
metrics:
# -- If `engine.concurrencyLimit.activeRequests` is set, this variable will
# define the ratio of current active requests to maximum active requests at which
# the Engine pods will scale. Setting this value too close to 1.0 may lead to a situation where
# the cluster is at max capacity and rejects incoming requests. Setting the ratio too close to 0.0
# will over-optimistically scale your cluster and increase compute costs unnecessarily.
requestCapacityRatio:
speechToText:
batch:
# -- (int) Scale the Engine pods based on a static desired number of speech-to-text batch requests per pod
requestsPerPod:
streaming:
# -- (int) Scale the Engine pods based on a static desired number of speech-to-text streaming requests per pod
requestsPerPod:
textToSpeech:
batch:
# -- (int) Scale the Engine pods based on a static desired number of text-to-speech batch requests per pod
requestsPerPod:
# -- If you have custom metrics you would like to scale with, you may add them here.
# See the [k8s docs](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/)
# for how to structure a list of metrics
custom: []
# -- [Configurable scaling behavior](https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#configurable-scaling-behavior)
# @default -- "*See values.yaml file for default*"
behavior:
scaleDown:
policies:
- type: Pods
value: 1
periodSeconds: 60
- type: Percent
value: 25
periodSeconds: 60
api:
# -- namePrefix is the prefix to apply to the name of all K8s objects
# associated with the Deepgram API containers.
namePrefix: "deepgram-api"
image:
# -- path configures the image path to use for creating API containers.
# You may change this from the public Quay image path if you have imported
# Deepgram images into a private container registry.
path: quay.io/deepgram/self-hosted-api
# -- pullPolicy configures how the Kubelet attempts to pull the Deepgram API image
pullPolicy: IfNotPresent
# -- tag defines which Deepgram release to use for API containers
tag: release-240827
# -- Additional labels to add to API resources
additionalLabels: {}
# -- (object) Additional annotations to add to the API deployment
additionalAnnotations:
updateStrategy:
rollingUpdate:
# -- The maximum number of API pods, relative to the number of replicas,
# that can go offline during a rolling update. See the
# [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-unavailable)
# for more details.
maxUnavailable: 0
# -- The maximum number of extra API pods that can be created during a rollingUpdate,
# relative to the number of replicas. See the
# [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge)
# for more details.
maxSurge: 1
# -- Configure resource limits per API container. See
# [Deepgram's documentation](https://developers.deepgram.com/docs/self-hosted-deployment-environments#api)
# for more details.
# @default -- ``
resources:
requests:
memory: "4Gi"
cpu: "2000m"
limits:
memory: "8Gi"
cpu: "4000m"
# -- Readiness probe customization for API pods.
# @default -- ``
readinessProbe:
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 1
# -- Liveness probe customization for API pods.
# @default -- ``
livenessProbe:
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 3
# -- [Affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
# to apply for API pods.
affinity: {}
# -- [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)
# to apply to API pods.
tolerations:
- effect: NoSchedule
key: dedicated
operator: Equal
value: deepgram-api-pool
# -- [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for API pods.
securityContext: {}
serviceAccount:
# -- Specifies whether to create a default service account for the Deepgram API Deployment.
create: true
# -- (string) Allows providing a custom service account name for the API component.
# If left empty, the default service account name will be used.
# If specified, and `api.serviceAccount.create = true`, this defines the name of the default service account.
# If specified, and `api.serviceAccount.create = false`, this provides the name of a preconfigured service account
# you wish to attach to the API deployment.
name:
# -- Configure how the API will listen for your requests
# @default -- ``
server:
# baseUrl is the prefix requests to the API.
baseUrl: "/v1"
# -- host is the IP address to listen on. You will want to listen
# on all interfaces to interact with other pods in the cluster.
host: "0.0.0.0"
# -- port to listen on.
port: 8080
# -- callbackConnTimeout configures how long to wait for a connection to a callback URL.
# See [Deepgram's callback documentation](https://developers.deepgram.com/docs/callback)
# for more details. The value should be a humantime duration.
callbackConnTimeout: "1s"
# -- callbackTimeout configures how long to wait for a response from a callback URL.
# See [Deepgram's callback documentation](https://developers.deepgram.com/docs/callback)
# for more details. The value should be a humantime duration.
callbackTimeout: "10s"
# -- fetchConnTimeout configures how long to wait for a connection to a fetch URL.
# The value should be a humantime duration.
# A fetch URL is a URL passed in an inference request from which a payload should be
# downloaded.
fetchConnTimeout: "1s"
# -- fetchTimeout configures how long to wait for a response from a fetch URL.
# The value should be a humantime duration.
# A fetch URL is a URL passed in an inference request from which a payload should be
# downloaded.
fetchTimeout: "60s"
# -- Specify custom DNS resolution options.
# @default -- ``
resolver:
# -- nameservers allows for specifying custom domain name server(s).
# A valid list item's format is "{IP} {PORT} {PROTOCOL (tcp or udp)}",
# e.g. `"127.0.0.1 53 udp"`.
nameservers: []
# -- (int) maxTTL sets the DNS TTL value if specifying a custom DNS nameserver.
maxTTL:
# -- Enable ancillary features
# @default -- ``
features:
# -- Enables entity detection on pre-recorded audio
# *if* a valid entity detection model is available.
# *WARNING*: Beta functionality.
entityDetection: false
# -- Enables entity-based redaction on pre-recorded audio
# *if* a valid entity detection model is available.
# *WARNING*: Beta functionality.
entityRedaction: false
# -- If API is receiving requests faster than Engine can process them, a request
# queue will form. By default, this queue is stored in memory. Under high load,
# the queue may grow too large and cause Out-Of-Memory errors. To avoid this,
# set a diskBufferPath to buffer the overflow on the request queue to disk.
#
# WARN: This is only to temporarily buffer requests during high load.
# If there is not enough Engine capacity to process the queued requests over time,
# the queue (and response time) will grow indefinitely.
diskBufferPath:
# -- driverPool configures the backend pool of speech engines (generically referred to as
# "drivers" here). The API will load-balance among drivers in the standard
# pool; if one standard driver fails, the next one will be tried.
# @default -- ``
driverPool:
# -- standard is the main driver pool to use.
# @default -- ``
standard:
# -- timeoutBackoff is the factor to increase the timeout by
# for each additional retry (for exponential backoff).
timeoutBackoff: 1.2
# -- retrySleep defines the initial sleep period (in humantime duration)
# before attempting a retry.
retrySleep: "2s"
# -- retryBackoff is the factor to increase the retrySleep
# by for each additional retry (for exponential backoff).
retryBackoff: 1.6
# -- Maximum response to deserialize from Driver (in bytes).
# Default is 1GB, expressed in bytes.
maxResponseSize: "1073741824"
engineAutoscaling:
enabled: false
engine:
# -- namePrefix is the prefix to apply to the name of all K8s objects
# associated with the Deepgram Engine containers.
namePrefix: "deepgram-engine"
image:
# -- path configures the image path to use for creating Engine containers.
# You may change this from the public Quay image path if you have imported
# Deepgram images into a private container registry.
path: quay.io/deepgram/self-hosted-engine
# -- pullPolicy configures how the Kubelet attempts to pull the Deepgram Engine image
pullPolicy: IfNotPresent
# -- tag defines which Deepgram release to use for Engine containers
tag: release-240827
# -- Additional labels to add to Engine resources
additionalLabels: {}
# -- (object) Additional annotations to add to the Engine deployment
additionalAnnotations:
updateStrategy:
rollingUpdate:
# -- The maximum number of Engine pods, relative to the number of replicas,
# that can go offline during a rolling update. See the
# [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-unavailable)
# for more details.
maxUnavailable: 0
# -- The maximum number of extra Engine pods that can be created during a rollingUpdate,
# relative to the number of replicas. See the
# [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge)
# for more details.
maxSurge: 1
# -- Configure resource limits per Engine container. See
# [Deepgram's documentation](https://developers.deepgram.com/docs/self-hosted-deployment-environments#engine)
# for more details.
# @default -- ``
resources:
requests:
memory: "30Gi"
cpu: "4000m"
# -- gpu maps to the nvidia.com/gpu resource parameter
gpu: 1
limits:
memory: "40Gi"
cpu: "8000m"
# -- gpu maps to the nvidia.com/gpu resource parameter
gpu: 1
# -- The startupProbe combination of `periodSeconds` and `failureThreshold` allows
# time for the container to load all models and start listening for incoming requests.
#
# Model load time can be affected by hardware I/O speeds, as well as network speeds
# if you are using a network volume mount for the models.
#
# If you are hitting the failure threshold before models are finished loading, you may
# want to extend the startup probe. However, this will also extend the time it takes
# to detect a pod that can't establish a network connection to validate its license.
# @default -- ``
startupProbe:
# -- periodSeconds defines how often to execute the probe.
periodSeconds: 10
# -- failureThreshold defines how many unsuccessful startup probe attempts
# are allowed before the container will be marked as Failed
failureThreshold: 60
# -- Readiness probe customization for Engine pods.
# @default -- ``
readinessProbe:
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 1
# -- Liveness probe customization for Engine pods.
# @default -- ``
livenessProbe:
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 3
# -- [Container lifecycle hooks](https://kubernetes.io/docs/concepts/containers/container-lifecycle-hooks/)
# for Engine containers. Use this to configure a preStop hook for graceful shutdown.
# @default -- ``
# Example:
# lifecycle:
# preStop:
# exec:
# command:
# - /bin/sh
# - -c
# - "sleep 30"
lifecycle: {}
# -- [Affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
# to apply for Engine pods.
affinity: {}
# -- [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)
# to apply to Engine pods.
tolerations:
- effect: NoSchedule
key: dedicated
operator: Equal
value: deepgram-engine-pool
# -- [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for API pods.
securityContext: {}
serviceAccount:
# -- Specifies whether to create a default service account for the Deepgram Engine Deployment.
create: true
# -- (string) Allows providing a custom service account name for the Engine component.
# If left empty, the default service account name will be used.
# If specified, and `engine.serviceAccount.create = true`, this defines the name of the default service account.
# If specified, and `engine.serviceAccount.create = false`, this provides the name of a preconfigured service account
# you wish to attach to the Engine deployment.
name:
concurrencyLimit:
# -- (int) activeRequests limits the number of active requests handled by
# a single Engine container.
# If additional requests beyond the limit are sent, the API container
# forming the request will try a different Engine pod. If no Engine pods
# are able to accept the request, the API will return a 429 HTTP response
# to the client. The `nil` default means no limit will be set.
activeRequests:
# -- Configure Engine containers to listen for requests from API containers.
# @default -- ``
server:
# -- host is the IP address to listen on for inference requests.
# You will want to listen on all interfaces to interact with
# other pods in the cluster.
host: "0.0.0.0"
# -- port to listen on for inference requests
port: 8080
# -- metricsServer exposes an endpoint on each Engine container
# for reporting inference-specific system metrics.
# See https://developers.deepgram.com/docs/metrics-guide#deepgram-engine
# for more details.
# @default -- ``
metricsServer:
# -- host is the IP address to listen on for metrics requests.
# You will want to listen on all interfaces to interact with
# other pods in the cluster.
host: "0.0.0.0"
# -- port to listen on for metrics requests
port: 9991
modelManager:
volumes:
customVolumeClaim:
# -- You may manually create your own PersistentVolume and PersistentVolumeClaim to store and
# expose model files to the Deepgram Engine. Configure your storage beforehand,
# and enable here.
# Note: Make sure the PV and PVC accessMode are set to `readWriteMany` or `readOnlyMany`
enabled: false
# -- (string) Name of your pre-configured PersistentVolumeClaim
name:
# -- Name of the directory within your pre-configured PersistentVolume
# where the models are stored
modelsDirectory: "/"
aws:
efs:
# -- Whether to use an [AWS Elastic File Sytem](https://aws.amazon.com/efs/)
# to store Deepgram models for use by Engine containers.
# This option requires your cluster to be running in
# [AWS EKS](https://aws.amazon.com/eks/).
enabled: false
# -- Name prefix for the resources associated with the model storage in AWS EFS.
namePrefix: dg-models
# -- (string) FileSystemId of existing AWS Elastic File System where
# Deepgram model files will be persisted.
# You can find it using the AWS CLI:
# ```
# $ aws efs describe-file-systems --query "FileSystems[*].FileSystemId"
# ```
fileSystemId:
# -- Whether to force a fresh download of all model links provided,
# even if models are already present in EFS.
forceDownload: false
gcp:
gpd:
# -- Whether to use an [GKE Persistent Disks](https://cloud.google.com/kubernetes-engine/docs/concepts/persistent-volumes)
# to store Deepgram models for use by Engine containers.
# This option requires your cluster to be running in
# [GCP GKE](https://cloud.google.com/kubernetes-engine).
# See the GKE documentation on
# [using pre-existing persistent disks](https://cloud.google.com/kubernetes-engine/docs/how-to/persistent-volumes/preexisting-pd).
enabled: true
# -- Name prefix for the resources associated with the model storage in GCP GPD.
namePrefix: dg-models
# -- The storageClassName of the existing persistent disk.
storageClassName: "standard-rwo"
# -- The size of your pre-existing persistent disk.
storageCapacity: "40G"
# -- The identifier of your pre-existing persistent disk.
# The format is projects/{project_id}/zones/{zone_name}/disks/{disk_name} for Zonal persistent disks,
# or projects/{project_id}/regions/{region_name}/disks/{disk_name} for Regional persistent disks.
volumeHandle: "projects/meesho-supply-prd-0622/zones/asia-southeast1-a/disks/deepgram-model-storage"
fsType: "ext4"
models:
# -- Links to your Deepgram models, if automatically downloading
# into storage backing a persistent volume.
# **Automatic downloads are currently supported for AWS EFS volumes only.**
# Insert each model link provided to you by your Deepgram
# Account Representative.
links: []
# -- chunking defines the size of audio chunks to process in seconds.
# Adjusting these values will affect both inference performance and accuracy
# of results. Please contact your Deepgram Account Representative if you
# want to adjust any of these values.
# @default -- ``
chunking:
speechToText:
batch:
# -- (float) minDuration is the minimum audio duration for a STT chunk size for a batch request
minDuration:
# -- (float) minDuration is the maximum audio duration for a STT chunk size for a batch request
maxDuration:
streaming:
# -- (float) minDuration is the minimum audio duration for a STT chunk size for a streaming request
minDuration:
# -- (float) minDuration is the maximum audio duration for a STT chunk size for a streaming request
maxDuration:
# -- step defines how often to return interim results, in seconds.
# This value may be lowered to increase the frequency of interim results.
# However, this also causes a significant decrease in the number of concurrent
# streams supported by a single GPU. Please contact your Deepgram Account
# representative for more details.
step: 1.0
halfPrecision:
# -- Engine will automatically enable half precision operations if your GPU supports
# them. You can explicitly enable or disable this behavior with the state parameter
# which supports `"enable"`, `"disabled"`, and `"auto"`.
state: "auto"
# -- Configuration options for the optional
# [Deepgram License Proxy](https://developers.deepgram.com/docs/license-proxy).
# @default -- ``
licenseProxy:
# -- The License Proxy is optional, but highly recommended to be deployed in production
# to enable highly available environments.
enabled: false
# -- If the License Proxy is deployed, one replica should be sufficient to
# support many API/Engine pods.
# Highly available environments may wish to deploy a second replica to ensure
# uptime, which can be toggled with this option.
deploySecondReplica: false
# -- Even with a License Proxy deployed, API and Engine pods can be configured to keep the
# upstream `license.deepgram.com` license server as a fallback licensing option if the
# License Proxy is unavailable.
# Disable this option if you are restricting API/Engine Pod network access for security reasons,
# and only the License Proxy should send egress traffic to the upstream license server.
keepUpstreamServerAsBackup: true
# -- namePrefix is the prefix to apply to the name of all K8s objects
# associated with the Deepgram License Proxy containers.
namePrefix: "deepgram-license-proxy"
image:
# -- path configures the image path to use for creating License Proxy containers.
# You may change this from the public Quay image path if you have imported
# Deepgram images into a private container registry.
path: quay.io/deepgram/self-hosted-license-proxy
# -- tag defines which Deepgram release to use for License Proxy containers
tag: release-240827
# -- pullPolicy configures how the Kubelet attempts to pull the Deepgram
# License Proxy image
pullPolicy: IfNotPresent
# -- Additional labels to add to License Proxy resources
additionalLabels: {}
# -- (object) Additional annotations to add to the LicenseProxy deployment
additionalAnnotations:
updateStrategy:
# -- For the LicenseProxy, we only expose maxSurge and not maxUnavailable.
# This is to avoid accidentally having all LicenseProxy nodes go offline during upgrades,
# which could impact the entire cluster's connection to the Deepgram License Server.
# @default -- ``
rollingUpdate:
# -- The maximum number of extra License Proxy pods that can be created during a rollingUpdate,
# relative to the number of replicas. See the
# [Kubernetes documentation](https://kubernetes.io/docs/concepts/workloads/controllers/deployment/#max-surge)
# for more details.
maxSurge: 1
# -- Configure resource limits per License Proxy container. See
# [Deepgram's documentation](https://developers.deepgram.com/docs/license-proxy#system-requirements)
# for more details.
# @default -- ``
resources:
requests:
memory: "1Gi"
cpu: "1000m"
limits:
memory: "8Gi"
cpu: "2000m"
# -- Readiness probe customization for License Proxy pods.
# @default -- ``
readinessProbe:
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 1
# -- Liveness probe customization for Proxy pods.
# @default -- ``
livenessProbe:
initialDelaySeconds: 5
periodSeconds: 10
failureThreshold: 3
# -- [Affinity and anti-affinity](https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#affinity-and-anti-affinity)
# to apply for License Proxy pods.
affinity: {}
# -- [Tolerations](https://kubernetes.io/docs/concepts/scheduling-eviction/taint-and-toleration/)
# to apply to License Proxy pods.
tolerations:
- effect: NoSchedule
key: dedicated
operator: Equal
value: deepgram-proxy-pool
# -- [Security context](https://kubernetes.io/docs/tasks/configure-pod-container/security-context/) for API pods.
securityContext: {}
serviceAccount:
# -- Specifies whether to create a default service account for the Deepgram License Proxy Deployment.
create: true
# -- (string) Allows providing a custom service account name for the LicenseProxy component.
# If left empty, the default service account name will be used.
# If specified, and `licenseProxy.serviceAccount.create = true`, this defines the name of the default service account.
# If specified, and `licenseProxy.serviceAccount.create = false`, this provides the name of a preconfigured service account
# you wish to attach to the License Proxy deployment.
name:
# -- Configure how the license proxy will listen for licensing requests.
# @default -- ``
server:
# --host is the IP address to listen on. You will want to listen
# on all interfaces to interact with other pods in the cluster.
host: "0.0.0.0"
# -- port to listen on.
port: 8443
# -- baseUrl is the prefix for incoming license verification requests.
baseUrl: "/"
# -- statusPort is the port to listen on for the status/health endpoint.
statusPort: 8080
# -- Passthrough values for [NVIDIA GPU Operator Helm chart](https://github.com/NVIDIA/gpu-operator/blob/master/deployments/gpu-operator/values.yaml)
# You may use the NVIDIA GPU Operator to manage installation of NVIDIA drivers and the container toolkit on nodes with attached GPUs.
# @default -- ``
gpu-operator:
# -- Whether to install the NVIDIA GPU Operator to manage driver and/or container toolkit installation.
# See the list of [supported Operating Systems](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/latest/platform-support.html#supported-operating-systems-and-kubernetes-platforms)
# to verify compatibility with your cluster/nodes. Disable this option if your cluster/nodes are not compatible.
# If disabled, you will need to self-manage NVIDIA software installation on all nodes where you want
# to schedule Deepgram Engine pods.
enabled: true
driver:
# -- Whether to install NVIDIA drivers on nodes where a NVIDIA GPU is detected.
# If your Kubernetes nodes run a base image that comes with NVIDIA drivers pre-configured,
# disable this option, but keep the parent `gpu-operator` and sibling `toolkit`
# options enabled.
enabled: true
# -- NVIDIA driver version to install.
version: "550.54.15"
toolkit:
# -- Whether to install NVIDIA drivers on nodes where a NVIDIA GPU is detected.
enabled: true
# -- NVIDIA container toolkit to install. The default `ubuntu` image tag for the
# toolkit requires a dynamic runtime link to a version of GLIBC that may not be
# present on nodes running older Linux distribution releases, such as Ubuntu 22.04.
# Therefore, we specify the `ubi8` image, which statically links the GLIBC library
# and avoids this issue.
version: v1.15.0-ubi8
cluster-autoscaler:
# -- Set to `true` to enable node autoscaling with AWS EKS. Note needed for GKE, as autoscaling is enabled by a
# [cli option on cluster creation](https://cloud.google.com/kubernetes-engine/docs/how-to/cluster-autoscaler#creating_a_cluster_with_autoscaling).
enabled: false
rbac:
serviceAccount:
# -- Name of the IAM Service Account with the [necessary permissions](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#permissions)
name: cluster-autoscaler-sa
annotations:
# -- (string) Replace with the AWS Role ARN configured for the Cluster Autoscaler.
# See the [Deepgram AWS EKS guide](https://developers.deepgram.com/docs/aws-k8s#creating-a-cluster)
# or [Cluster Autoscaler AWS documentation](https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md#permissions)
# for details.
eks.amazonaws.com/role-arn:
autoDiscovery:
# -- (string) Name of your AWS EKS cluster. Using the [Cluster Autoscaler](https://github.com/kubernetes/autoscaler)
# on AWS requires knowledge of certain cluster metadata.
clusterName:
# -- (string) Region of your AWS EKS cluster. Using the [Cluster Autoscaler](https://github.com/kubernetes/autoscaler)
# on AWS requires knowledge of certain cluster metadata.
awsRegion:
# -- Passthrough values for [Prometheus k8s stack Helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/kube-prometheus-stack).
# Prometheus (and its adapter) should be configured when scaling.auto is enabled.
# You may choose to use the installation/configuration bundled in this Helm chart,
# or you may configure an existing Prometheus installation in your cluster to expose
# the needed values.
# See source Helm chart for explanation of available values. Default values provided in this chart are used
# to provide pod autoscaling for Deepgram pods.
# @default -- ``
kube-prometheus-stack:
# -- (bool) Normally, this chart will be installed if `scaling.auto.enabled` is true. However, if you wish
# to manage the Prometheus adapter in your cluster on your own and not as part of the Deepgram Helm chart,
# you can force it to not be installed by setting this to `false`.
includeDependency:
fullnameOverride: "dg-prometheus-stack"
prometheus:
prometheusSpec:
additionalScrapeConfigs:
- job_name: "dg_engine_metrics"
scrape_interval: "2s"
kubernetes_sd_configs:
- role: endpoints
namespaces:
names:
- "{{ .Release.Namespace }}"
relabel_configs:
- source_labels: [__meta_kubernetes_service_name]
regex: "(.*)-metrics"
action: keep
- source_labels: [__meta_kubernetes_endpoint_port_name]
regex: "metrics"
action: keep
- source_labels: [__meta_kubernetes_namespace]
target_label: namespace
- source_labels: [__meta_kubernetes_service_name]
target_label: service
- source_labels: [__meta_kubernetes_pod_name]
target_label: pod
prometheusOperator:
enabled: true
alertmanager:
enabled: false
grafana:
enabled: true
nodeExporter:
enabled: false
kube-state-metrics:
enabled: true
metricLabelsAllowlist:
- namespaces=[{{ .Release.Namespace }}],deployments=[app]
# -- Passthrough values for [Prometheus Adapter Helm chart](https://github.com/prometheus-community/helm-charts/tree/main/charts/prometheus-adapter).
# Prometheus, and its adapter here, should be configured when scaling.auto is enabled.
# You may choose to use the installation/configuration bundled in this Helm chart,
# or you may configure an existing Prometheus installation in your cluster to expose
# the needed values.
# See source Helm chart for explanation of available values. Default values provided in this chart are used
# to provide pod autoscaling for Deepgram pods.
# @default -- ``
prometheus-adapter:
# -- Normally, this chart will be installed if `scaling.auto.enabled` is true. However, if you wish
# to manage the Prometheus adapter in your cluster on your own and not as part of the Deepgram Helm chart,
# you can force it to not be installed by setting this to `false`.
includeDependency:
prometheus:
url: http://dg-prometheus-stack-prometheus.{{ .Release.Namespace }}.svc
rules:
default: false
external:
- name:
as: "engine_active_requests_stt_streaming"
seriesQuery: 'engine_active_requests{kind="stream"}'
metricsQuery: 'avg(engine_active_requests{kind="stream"})'
resources:
overrides:
namespace: { resource: "namespace" }
pod: { resource: "pod" }
service: { resource: "service" }
- name:
as: "engine_active_requests_stt_batch"
seriesQuery: 'engine_active_requests{kind="batch"}'
metricsQuery: 'avg(engine_active_requests{kind="batch"})'
resources:
overrides:
namespace: { resource: "namespace" }
pod: { resource: "pod" }
service: { resource: "service" }
- name:
as: "engine_active_requests_tts_batch"
seriesQuery: 'engine_active_requests{kind="tts"}'
metricsQuery: 'avg(engine_active_requests{kind="tts"})'
resources:
overrides:
namespace: { resource: "namespace" }
pod: { resource: "pod" }
service: { resource: "service" }
- name:
as: "engine_estimated_stream_capacity"
seriesQuery: 'engine_active_requests{kind="stream"}'
metricsQuery: 'avg_over_time((sum(engine_active_requests{kind="stream"}) / sum(engine_estimated_stream_capacity) * 100)[1m:1m])'
resources:
overrides:
namespace: { resource: "namespace" }
pod: { resource: "pod" }
service: { resource: "service" }
- name:
as: "engine_requests_active_to_max_ratio"
seriesQuery: "engine_max_active_requests"
metricsQuery: "avg_over_time((sum(engine_active_requests) / sum(engine_max_active_requests) * 100)[1m:1m])"
resources:
overrides:
namespace: { resource: "namespace" }
pod: { resource: "pod" }
service: { resource: "service" }
- name:
as: "engine_to_api_pod_ratio"
seriesQuery: 'kube_deployment_labels{label_app="deepgram-engine"}'
metricsQuery: '(sum(kube_deployment_status_replicas and on(deployment) kube_deployment_labels{label_app="deepgram-engine"})) / (sum(kube_deployment_status_replicas and on(deployment) kube_deployment_labels{label_app="deepgram-api"}))'
resources:
overrides:
namespace: { resource: "namespace" }
pod: { resource: "pod" }