Compare commits
12
Commits
c16c36b6a6
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 Mukul SharmaandClaude Opus 5
|
6d743cbe48 | ||
|
Mukul SharmaandClaude Opus 5
|
a9119d7b8e | ||
|
Mukul SharmaandClaude Opus 5
|
3e09eecbfe | ||
|
Mukul SharmaandClaude Opus 5
|
43ca78e83e | ||
|
Mukul Sharma
|
f448f204e2 | ||
|
Mukul Sharma
|
47a60b7d95 | ||
|
Mukul Sharma
|
63eef9014b | ||
|
Mukul Sharma
|
4b908150e4 | ||
|
Mukul Sharma
|
e0572db0f4 | ||
|
Mukul Sharma
|
3b48d51b18 | ||
|
Mukul Sharma
|
3ffa2d8444 | ||
|
Mukul Sharma
|
dad5d9f9f2 |
@@ -1,67 +1,123 @@
|
||||
# devops-lib
|
||||
# devops-lib (GKE)
|
||||
|
||||
---
|
||||
Jenkins Shared Library for the CI/CD pipeline on the GKE cluster. The GCP
|
||||
counterpart of the homelab repo of the same name, and a copy rather than a
|
||||
shared repo because several values here are cluster-specific in ways that
|
||||
would break the other cluster if crossed over.
|
||||
|
||||
## Parameters
|
||||
Most of the functionality depends on the parameters provided by the users in form of groovy map of key and value pairs. The supported parameters are as below:
|
||||
Register it in Jenkins under the name `devops-lib`, exactly as on the
|
||||
homelab. Consuming repos then need no change at all — the same two-line
|
||||
Jenkinsfile works on either cluster, and which library it resolves to is a
|
||||
property of the Jenkins it runs on.
|
||||
|
||||
### Required parameters
|
||||
**repo_name**: The key repo_name is required for checking out the code in a subdirectory. The value is the repository name that you want to checkout
|
||||
**What differs from the homelab copy**, all of it a consequence of GKE
|
||||
being a real cloud rather than one VM:
|
||||
|
||||
**build_tool**: This parameter is required to identify which build_tool to use in the pipeline. The supported values are *maven*, *gradle*, *docker*, *python*, *node*, *go*, *php* (and their prefixed variants such as *maven-3.3-jdk-17*, *python-3*, *node-16*, *go1.21*)
|
||||
- **The registry hostname** is `harbor.infra.deployshed.com`, in the push
|
||||
target (`harbor_registry`, see the table below) and in all five fallback
|
||||
Dockerfiles.
|
||||
- **Harbor speaks TLS, with a public certificate.** The homelab's dind
|
||||
passes `--insecure-registry`; here nothing is needed at all, because
|
||||
Let's Encrypt issues for the real domain and dockerd already trusts it.
|
||||
- **`helm_repo_url` uses cluster DNS**, since the clone happens inside a
|
||||
build pod. The homelab points it at an ingress hostname.
|
||||
- **`build-tools` is not in this repo.** It lives in
|
||||
`devops-base-images-gcp` alongside the mirrored base images, and is
|
||||
referenced here only by tag.
|
||||
|
||||
**maintainer** : This parameter is required to send the notification in the slack channel *#ci-cd-status*. Please provide your slack username here
|
||||
Everything else — the stage flow, the fallback templates, the hooks
|
||||
contract — is unchanged. The library was already adapted from a much
|
||||
larger, company-wide one; see git history for what was removed.
|
||||
|
||||
### Optional parameter
|
||||
## Using it in a service repo
|
||||
|
||||
`devops-lib` is Homelab's Jenkins Shared Library that provides a unified CI/CD pipeline for all microservices across the organisation. Consumer repos load it via `@Library('devops-lib@main')` and call a single `eksCICD(repo)` entry point — the library handles language-specific building (Maven, Go, Gradle, Node.js, Python, PHP), code quality gates (Sonar), Docker image publishing to GAR/ECR, Helm chart updates, and ArgoCD-based deployment to GKE/EKS clusters. Build status and deployment metadata are reported back to Ringmaster and Slack.
|
||||
|
||||
**Stack:** Groovy (Jenkins Shared Library) · ArgoCD · Helm · GCP (GKE, GAR, GCS, Vault, Sonar) · AWS (EKS, ECR, S3)
|
||||
|
||||
## Dependencies
|
||||
|
||||
**push_to_jfrog**: By default master, main, gcp-main, and gcp-master branches push artifacts to jfrog/s3 repository, set this parameter to true to push artifacts from non-master branches
|
||||
|
||||
---
|
||||
|
||||
## config.yaml schema (consumer services)
|
||||
|
||||
Every service that uses this library must provide a `config.yaml`:
|
||||
|
||||
| Key | Required | Description |
|
||||
|-----|----------|-------------|
|
||||
| `repo_name` | yes | GitHub repo slug — must match exactly |
|
||||
| `build_tool` | yes | `maven`, `go`, `gradle`, `node-*`, `python-*`, `php`, `docker` |
|
||||
| `dockerBuildVersion` | yes | Drives Dockerfile template: `maven-21`, `go-1.22`, `node-20`, etc. |
|
||||
| `team` | yes | Team slug — validated against `buTeamMapping` |
|
||||
| `bu` | yes | Business unit: `supply`, `demand`, `central`, `dataengg`, `datascience`, `mcache`, `infra` |
|
||||
| `maintainer` | yes | GitHub handle for Slack notifications |
|
||||
| `deployment_order` | yes | List of ArgoCD application names to deploy |
|
||||
| `notify_channel` | no | Slack channel (default: `ci-cd-status`) |
|
||||
| `skip_sonar` | no | Whitelist-gated; see `constructParam.groovy` |
|
||||
| `deployArgo` | no | Set `false` to skip ArgoCD sync |
|
||||
| `appConfigEnabled` | no | Required `true` for `stg`; whitelist-gated |
|
||||
| `skip_test` | no | Skip unit tests (Maven) |
|
||||
| `push_to_jfrog` | no | Publish JAR to JFrog Artifactory |
|
||||
| `push_to_s3` | no | Push artifact to S3 |
|
||||
| `build_packages` | no | System development packages required while compiling (currently consumed by Rust builds; for example `libpq-dev`) |
|
||||
| `runtime_packages` | no | System runtime libraries required by the compiled binary (currently consumed by Rust builds; for example `libpq5`) |
|
||||
|
||||
## Adding this library to a new service
|
||||
The entire Jenkinsfile is 2 lines:
|
||||
|
||||
```groovy
|
||||
// Jenkinsfile
|
||||
@Library('devops-lib@main') _
|
||||
|
||||
eksCICD([
|
||||
repo_name: 'my-service'
|
||||
])
|
||||
@Library('devops-lib') _
|
||||
homelabPipeline(repo_name: 'my-service')
|
||||
```
|
||||
|
||||
Place `config.yaml` at the repo root with the required fields above.
|
||||
`repo_name` is the only required key. Everything else has a sensible
|
||||
default — override any of them by passing extra keys to `homelabPipeline`,
|
||||
or by committing a `config.yaml` to the service repo's own root (merged
|
||||
in after checkout; repo-committed values win over the Jenkinsfile call).
|
||||
|
||||
## Adding a new build stage
|
||||
| Key | Default | Notes |
|
||||
|---|---|---|
|
||||
| `service_name` | `repo_name` | Second path segment under `devops-helm-charts/values/` |
|
||||
| `argo_app_name` | `repo_name` | Must match the ArgoCD Application's `metadata.name` |
|
||||
| `harbor_registry` | `harbor.infra.deployshed.com` | The registry hostname images are pushed to and pulled from. Must be spelled identically here, in the `dockerconfigjson` `auths` key, and in Harbor's `externalURL` — docker matches both stored credentials and TLS trust by exact hostname, so a mismatch fails as `unauthorized` rather than as a name problem |
|
||||
| `harbor_project` | `apps-registry` | Must already exist in Harbor, and be public unless you also wire an `imagePullSecret` — the app values assume anonymous pull. A push to a missing project fails as `unauthorized: project <name> not found` |
|
||||
| `helm_repo_url` | `devops-helm-charts-gcp`, over cluster DNS | `http://gitea-http.gitea.svc.cluster.local:3000/gitadmin/…` — pod-to-pod, so it never leaves the cluster and comes back through the ingress |
|
||||
| `image_tag_yq_path` | `.deployment.image.tag` | **Override this if the app's chart isn't `1.0.0`** — e.g. `sts-2.0.0` uses `.podtemplate.image.tag` instead. Getting this wrong doesn't fail loudly: `yq -i` creates the path if missing rather than erroring, silently leaving the real field un-bumped. |
|
||||
| `dockerBuildVersion` | none | Only read when the repo has **no Dockerfile of its own** — picks a fallback template (see below). No default; either ship a Dockerfile or set this. |
|
||||
|
||||
1. Create `src/com/homelab/stages/build<Lang>.groovy` implementing `def run(Map config)`.
|
||||
2. Add a `case` in `src/com/homelab/stages/buildObjHelper.groovy`.
|
||||
3. Add a Dockerfile template in `resources/com/homelab/<lang>-Dockerfile` if needed.
|
||||
## Pipeline stages
|
||||
|
||||
`checkOut → loadConfig → runHooks(pre_build) → buildDocker →
|
||||
runHooks(post_build) → updateHelmTag → syncArgoApp → notify`, all inside
|
||||
a `podTemplate` (`resources/org/homelab/dind-pod.yaml`) via
|
||||
`node(POD_LABEL) { ... }`.
|
||||
|
||||
- **`loadConfig`** — if the repo has a `config.yaml` at its root, its
|
||||
keys are merged into the pipeline config (repo values win).
|
||||
- **`runHooks`** — reads `config.yaml`'s `hooks.pre_build`/`hooks.post_build`
|
||||
lists, each `{name, script, interpreter, requirements, blocking,
|
||||
timeout_seconds}`. Blocking by default; `blocking: false` demotes a
|
||||
failure to advisory (log + continue). Script paths must be
|
||||
repo-relative (no `..`, no absolute paths).
|
||||
- **`buildDocker`** — uses the repo's own `Dockerfile` if present;
|
||||
otherwise renders one from `resources/com/homelab/<lang>-Dockerfile`
|
||||
based on `dockerBuildVersion` (e.g. `go-1.22`, `node-20`,
|
||||
`python-3.12`, `java-21`, `php-8.3`). All fallback templates pull base
|
||||
images from Harbor's `base-images` project (mirrored via the separate
|
||||
`devops-base-images-gcp` repo), not Docker Hub directly. Only versions
|
||||
actually mirrored there resolve — an unmirrored tag fails the build
|
||||
rather than silently falling back to Docker Hub.
|
||||
- **`updateHelmTag`** — clones `devops-helm-charts`, bumps the image tag
|
||||
via `yq` at `image_tag_yq_path`, commits, pushes to `main`.
|
||||
- **`syncArgoApp`** — calls the ArgoCD REST API to sync `argo_app_name`.
|
||||
|
||||
## Adding a new language's fallback template
|
||||
|
||||
1. Add the base image to `devops-base-images/images.txt`, re-mirror it
|
||||
into Harbor.
|
||||
2. Add `resources/com/homelab/<lang>-Dockerfile`, parametrized by
|
||||
`${version}` (rendered via `constructTemplate.groovy`'s
|
||||
`SimpleTemplateEngine` wrapper).
|
||||
3. Add a case for it in `buildDocker.groovy`'s `templates` map.
|
||||
|
||||
## Build-tools image
|
||||
|
||||
The `docker-cli` container runs
|
||||
`harbor.infra.deployshed.com/base-images/build-tools:1`, which bakes in
|
||||
git, yq, bash, python3 with pip and venv, and curl, so nothing is installed
|
||||
on demand on every build.
|
||||
|
||||
**It is not built here.** The Dockerfile lives in `devops-base-images-gcp`,
|
||||
next to the mirrored base images, because it is the same kind of artefact:
|
||||
built by hand, occasionally, and pushed to Harbor. A Jenkins job could not
|
||||
build it anyway — it is the image Jenkins builds *in*.
|
||||
|
||||
`dind-pod.yaml` pins the tag, so rebuilding the image rolls nothing out
|
||||
until that pin is bumped. Bump the tag rather than overwriting one.
|
||||
|
||||
## Registry trust
|
||||
|
||||
Nothing to configure. Harbor is reached at `harbor.infra.deployshed.com`,
|
||||
which carries a publicly trusted Let's Encrypt certificate, so both
|
||||
containerd on the node (pulls) and dockerd in the build pod (pushes) accept
|
||||
it out of the box.
|
||||
|
||||
This used to be a real piece of setup, and the history is worth keeping
|
||||
because reintroducing a nip.io registry name would bring it all back. A
|
||||
nip.io address cannot have a public certificate — it is not on the public
|
||||
suffix list, and every `*.nip.io` certificate on the internet shares one
|
||||
rate limit — so cert-manager signed Harbor from a private CA instead. The
|
||||
node pool was told to trust that CA at creation, which covered pulls; the
|
||||
push came from dockerd inside the build pod, a separate client with its own
|
||||
trust store, so `dind-pod.yaml` had to mount the CA at
|
||||
`/etc/docker/certs.d/<registry host>/ca.crt` as well. The failure when that
|
||||
mount was missing was thoroughly confusing: pushes failed TLS verification
|
||||
while pulls of the very same image succeeded.
|
||||
|
||||
@@ -1,38 +0,0 @@
|
||||
plugins {
|
||||
id 'groovy'
|
||||
}
|
||||
|
||||
repositories {
|
||||
maven { url 'https://repo.jenkins-ci.org/releases/' }
|
||||
maven { url 'https://repo.jenkins-ci.org/public/' }
|
||||
mavenCentral()
|
||||
}
|
||||
|
||||
sourceSets {
|
||||
// Main sources are Jenkins Shared Library scripts loaded at runtime by JenkinsPipelineUnit.
|
||||
// They depend on Jenkins API and are not pre-compiled — loadScript() handles them at test time.
|
||||
main {
|
||||
groovy { srcDirs = [] }
|
||||
java { srcDirs = [] }
|
||||
}
|
||||
test {
|
||||
// test/unit — actual test classes
|
||||
// test/stubs — minimal stub implementations of Jenkins stage classes used by dispatch tests
|
||||
groovy { srcDirs = ['test/unit', 'test/stubs'] }
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
testImplementation 'com.lesfurets:jenkins-pipeline-unit:1.22'
|
||||
// Match the Groovy version bundled by JenkinsPipelineUnit
|
||||
testImplementation 'org.codehaus.groovy:groovy-all:2.4.21'
|
||||
testImplementation 'junit:junit:4.13.2'
|
||||
}
|
||||
|
||||
test {
|
||||
systemProperty 'user.dir', rootDir.absolutePath
|
||||
testLogging {
|
||||
events 'passed', 'skipped', 'failed'
|
||||
exceptionFormat 'full'
|
||||
}
|
||||
}
|
||||
Vendored
BIN
Binary file not shown.
-9
@@ -1,9 +0,0 @@
|
||||
distributionBase=GRADLE_USER_HOME
|
||||
distributionPath=wrapper/dists
|
||||
distributionUrl=https\://services.gradle.org/distributions/gradle-9.5.0-bin.zip
|
||||
networkTimeout=10000
|
||||
retries=0
|
||||
retryBackOffMs=500
|
||||
validateDistributionUrl=true
|
||||
zipStoreBase=GRADLE_USER_HOME
|
||||
zipStorePath=wrapper/dists
|
||||
@@ -1,248 +0,0 @@
|
||||
#!/bin/sh
|
||||
|
||||
#
|
||||
# Copyright © 2015 the original authors.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# https://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
#
|
||||
# SPDX-License-Identifier: Apache-2.0
|
||||
#
|
||||
|
||||
##############################################################################
|
||||
#
|
||||
# Gradle start up script for POSIX generated by Gradle.
|
||||
#
|
||||
# Important for running:
|
||||
#
|
||||
# (1) You need a POSIX-compliant shell to run this script. If your /bin/sh is
|
||||
# noncompliant, but you have some other compliant shell such as ksh or
|
||||
# bash, then to run this script, type that shell name before the whole
|
||||
# command line, like:
|
||||
#
|
||||
# ksh Gradle
|
||||
#
|
||||
# Busybox and similar reduced shells will NOT work, because this script
|
||||
# requires all of these POSIX shell features:
|
||||
# * functions;
|
||||
# * expansions «$var», «${var}», «${var:-default}», «${var+SET}»,
|
||||
# «${var#prefix}», «${var%suffix}», and «$( cmd )»;
|
||||
# * compound commands having a testable exit status, especially «case»;
|
||||
# * various built-in commands including «command», «set», and «ulimit».
|
||||
#
|
||||
# Important for patching:
|
||||
#
|
||||
# (2) This script targets any POSIX shell, so it avoids extensions provided
|
||||
# by Bash, Ksh, etc; in particular arrays are avoided.
|
||||
#
|
||||
# The "traditional" practice of packing multiple parameters into a
|
||||
# space-separated string is a well documented source of bugs and security
|
||||
# problems, so this is (mostly) avoided, by progressively accumulating
|
||||
# options in "$@", and eventually passing that to Java.
|
||||
#
|
||||
# Where the inherited environment variables (DEFAULT_JVM_OPTS, JAVA_OPTS,
|
||||
# and GRADLE_OPTS) rely on word-splitting, this is performed explicitly;
|
||||
# see the in-line comments for details.
|
||||
#
|
||||
# There are tweaks for specific operating systems such as AIX, CygWin,
|
||||
# Darwin, MinGW, and NonStop.
|
||||
#
|
||||
# (3) This script is generated from the Groovy template
|
||||
# https://github.com/gradle/gradle/blob/3d91ce3b8caaf77ad09f381f43615b715b53f72c/platforms/jvm/plugins-application/src/main/resources/org/gradle/api/internal/plugins/unixStartScript.txt
|
||||
# within the Gradle project.
|
||||
#
|
||||
# You can find Gradle at https://github.com/gradle/gradle/.
|
||||
#
|
||||
##############################################################################
|
||||
|
||||
# Attempt to set APP_HOME
|
||||
|
||||
# Resolve links: $0 may be a link
|
||||
app_path=$0
|
||||
|
||||
# Need this for daisy-chained symlinks.
|
||||
while
|
||||
APP_HOME=${app_path%"${app_path##*/}"} # leaves a trailing /; empty if no leading path
|
||||
[ -h "$app_path" ]
|
||||
do
|
||||
ls=$( ls -ld "$app_path" )
|
||||
link=${ls#*' -> '}
|
||||
case $link in #(
|
||||
/*) app_path=$link ;; #(
|
||||
*) app_path=$APP_HOME$link ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# This is normally unused
|
||||
# shellcheck disable=SC2034
|
||||
APP_BASE_NAME=${0##*/}
|
||||
# Discard cd standard output in case $CDPATH is set (https://github.com/gradle/gradle/issues/25036)
|
||||
APP_HOME=$( cd -P "${APP_HOME:-./}" > /dev/null && printf '%s\n' "$PWD" ) || exit
|
||||
|
||||
# Use the maximum available, or set MAX_FD != -1 to use that value.
|
||||
MAX_FD=maximum
|
||||
|
||||
warn () {
|
||||
echo "$*"
|
||||
} >&2
|
||||
|
||||
die () {
|
||||
echo
|
||||
echo "$*"
|
||||
echo
|
||||
exit 1
|
||||
} >&2
|
||||
|
||||
# OS specific support (must be 'true' or 'false').
|
||||
cygwin=false
|
||||
msys=false
|
||||
darwin=false
|
||||
nonstop=false
|
||||
case "$( uname )" in #(
|
||||
CYGWIN* ) cygwin=true ;; #(
|
||||
Darwin* ) darwin=true ;; #(
|
||||
MSYS* | MINGW* ) msys=true ;; #(
|
||||
NONSTOP* ) nonstop=true ;;
|
||||
esac
|
||||
|
||||
|
||||
|
||||
# Determine the Java command to use to start the JVM.
|
||||
if [ -n "$JAVA_HOME" ] ; then
|
||||
if [ -x "$JAVA_HOME/jre/sh/java" ] ; then
|
||||
# IBM's JDK on AIX uses strange locations for the executables
|
||||
JAVACMD=$JAVA_HOME/jre/sh/java
|
||||
else
|
||||
JAVACMD=$JAVA_HOME/bin/java
|
||||
fi
|
||||
if [ ! -x "$JAVACMD" ] ; then
|
||||
die "ERROR: JAVA_HOME is set to an invalid directory: $JAVA_HOME
|
||||
|
||||
Please set the JAVA_HOME variable in your environment to match the
|
||||
location of your Java installation."
|
||||
fi
|
||||
else
|
||||
JAVACMD=java
|
||||
if ! command -v java >/dev/null 2>&1
|
||||
then
|
||||
die "ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
|
||||
|
||||
Please set the JAVA_HOME variable in your environment to match the
|
||||
location of your Java installation."
|
||||
fi
|
||||
fi
|
||||
|
||||
# Increase the maximum file descriptors if we can.
|
||||
if ! "$cygwin" && ! "$darwin" && ! "$nonstop" ; then
|
||||
case $MAX_FD in #(
|
||||
max*)
|
||||
# In POSIX sh, ulimit -H is undefined. That's why the result is checked to see if it worked.
|
||||
# shellcheck disable=SC2039,SC3045
|
||||
MAX_FD=$( ulimit -H -n ) ||
|
||||
warn "Could not query maximum file descriptor limit"
|
||||
esac
|
||||
case $MAX_FD in #(
|
||||
'' | soft) :;; #(
|
||||
*)
|
||||
# In POSIX sh, ulimit -n is undefined. That's why the result is checked to see if it worked.
|
||||
# shellcheck disable=SC2039,SC3045
|
||||
ulimit -n "$MAX_FD" ||
|
||||
warn "Could not set maximum file descriptor limit to $MAX_FD"
|
||||
esac
|
||||
fi
|
||||
|
||||
# Collect all arguments for the java command, stacking in reverse order:
|
||||
# * args from the command line
|
||||
# * the main class name
|
||||
# * -classpath
|
||||
# * -D...appname settings
|
||||
# * --module-path (only if needed)
|
||||
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and GRADLE_OPTS environment variables.
|
||||
|
||||
# For Cygwin or MSYS, switch paths to Windows format before running java
|
||||
if "$cygwin" || "$msys" ; then
|
||||
APP_HOME=$( cygpath --path --mixed "$APP_HOME" )
|
||||
|
||||
JAVACMD=$( cygpath --unix "$JAVACMD" )
|
||||
|
||||
# Now convert the arguments - kludge to limit ourselves to /bin/sh
|
||||
for arg do
|
||||
if
|
||||
case $arg in #(
|
||||
-*) false ;; # don't mess with options #(
|
||||
/?*) t=${arg#/} t=/${t%%/*} # looks like a POSIX filepath
|
||||
[ -e "$t" ] ;; #(
|
||||
*) false ;;
|
||||
esac
|
||||
then
|
||||
arg=$( cygpath --path --ignore --mixed "$arg" )
|
||||
fi
|
||||
# Roll the args list around exactly as many times as the number of
|
||||
# args, so each arg winds up back in the position where it started, but
|
||||
# possibly modified.
|
||||
#
|
||||
# NB: a `for` loop captures its iteration list before it begins, so
|
||||
# changing the positional parameters here affects neither the number of
|
||||
# iterations, nor the values presented in `arg`.
|
||||
shift # remove old arg
|
||||
set -- "$@" "$arg" # push replacement arg
|
||||
done
|
||||
fi
|
||||
|
||||
|
||||
# Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
|
||||
DEFAULT_JVM_OPTS='"-Xmx64m" "-Xms64m"'
|
||||
|
||||
# Collect all arguments for the java command:
|
||||
# * DEFAULT_JVM_OPTS, JAVA_OPTS, and optsEnvironmentVar are not allowed to contain shell fragments,
|
||||
# and any embedded shellness will be escaped.
|
||||
# * For example: A user cannot expect ${Hostname} to be expanded, as it is an environment variable and will be
|
||||
# treated as '${Hostname}' itself on the command line.
|
||||
|
||||
set -- \
|
||||
"-Dorg.gradle.appname=$APP_BASE_NAME" \
|
||||
-jar "$APP_HOME/gradle/wrapper/gradle-wrapper.jar" \
|
||||
"$@"
|
||||
|
||||
# Stop when "xargs" is not available.
|
||||
if ! command -v xargs >/dev/null 2>&1
|
||||
then
|
||||
die "xargs is not available"
|
||||
fi
|
||||
|
||||
# Use "xargs" to parse quoted args.
|
||||
#
|
||||
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
|
||||
#
|
||||
# In Bash we could simply go:
|
||||
#
|
||||
# readarray ARGS < <( xargs -n1 <<<"$var" ) &&
|
||||
# set -- "${ARGS[@]}" "$@"
|
||||
#
|
||||
# but POSIX shell has neither arrays nor command substitution, so instead we
|
||||
# post-process each arg (as a line of input to sed) to backslash-escape any
|
||||
# character that might be a shell metacharacter, then use eval to reverse
|
||||
# that process (while maintaining the separation between arguments), and wrap
|
||||
# the whole thing up as a single "set" statement.
|
||||
#
|
||||
# This will of course break if any of these variables contains a newline or
|
||||
# an unmatched quote.
|
||||
#
|
||||
|
||||
eval "set -- $(
|
||||
printf '%s\n' "$DEFAULT_JVM_OPTS $JAVA_OPTS $GRADLE_OPTS" |
|
||||
xargs -n1 |
|
||||
sed ' s~[^-[:alnum:]+,./:=@_]~\\&~g; ' |
|
||||
tr '\n' ' '
|
||||
)" '"$@"'
|
||||
|
||||
exec "$JAVACMD" "$@"
|
||||
Vendored
-82
@@ -1,82 +0,0 @@
|
||||
@rem
|
||||
@rem Copyright 2015 the original author or authors.
|
||||
@rem
|
||||
@rem Licensed under the Apache License, Version 2.0 (the "License");
|
||||
@rem you may not use this file except in compliance with the License.
|
||||
@rem You may obtain a copy of the License at
|
||||
@rem
|
||||
@rem https://www.apache.org/licenses/LICENSE-2.0
|
||||
@rem
|
||||
@rem Unless required by applicable law or agreed to in writing, software
|
||||
@rem distributed under the License is distributed on an "AS IS" BASIS,
|
||||
@rem WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
@rem See the License for the specific language governing permissions and
|
||||
@rem limitations under the License.
|
||||
@rem
|
||||
@rem SPDX-License-Identifier: Apache-2.0
|
||||
@rem
|
||||
|
||||
@if "%DEBUG%"=="" @echo off
|
||||
@rem ##########################################################################
|
||||
@rem
|
||||
@rem Gradle startup script for Windows
|
||||
@rem
|
||||
@rem ##########################################################################
|
||||
|
||||
@rem Set local scope for the variables, and ensure extensions are enabled
|
||||
setlocal EnableExtensions
|
||||
|
||||
set DIRNAME=%~dp0
|
||||
if "%DIRNAME%"=="" set DIRNAME=.
|
||||
@rem This is normally unused
|
||||
set APP_BASE_NAME=%~n0
|
||||
set APP_HOME=%DIRNAME%
|
||||
|
||||
@rem Resolve any "." and ".." in APP_HOME to make it shorter.
|
||||
for %%i in ("%APP_HOME%") do set APP_HOME=%%~fi
|
||||
|
||||
@rem Add default JVM options here. You can also use JAVA_OPTS and GRADLE_OPTS to pass JVM options to this script.
|
||||
set DEFAULT_JVM_OPTS="-Xmx64m" "-Xms64m"
|
||||
|
||||
@rem Find java.exe
|
||||
if defined JAVA_HOME goto findJavaFromJavaHome
|
||||
|
||||
set JAVA_EXE=java.exe
|
||||
%JAVA_EXE% -version >NUL 2>&1
|
||||
if %ERRORLEVEL% equ 0 goto execute
|
||||
|
||||
echo. 1>&2
|
||||
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH. 1>&2
|
||||
echo. 1>&2
|
||||
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
|
||||
echo location of your Java installation. 1>&2
|
||||
|
||||
"%COMSPEC%" /c exit 1
|
||||
|
||||
:findJavaFromJavaHome
|
||||
set JAVA_HOME=%JAVA_HOME:"=%
|
||||
set JAVA_EXE=%JAVA_HOME%/bin/java.exe
|
||||
|
||||
if exist "%JAVA_EXE%" goto execute
|
||||
|
||||
echo. 1>&2
|
||||
echo ERROR: JAVA_HOME is set to an invalid directory: %JAVA_HOME% 1>&2
|
||||
echo. 1>&2
|
||||
echo Please set the JAVA_HOME variable in your environment to match the 1>&2
|
||||
echo location of your Java installation. 1>&2
|
||||
|
||||
"%COMSPEC%" /c exit 1
|
||||
|
||||
:execute
|
||||
@rem Setup the command line
|
||||
|
||||
|
||||
|
||||
@rem Execute Gradle
|
||||
@rem endlocal doesn't take effect until after the line is parsed and variables are expanded
|
||||
@rem which allows us to clear the local environment before executing the java command
|
||||
endlocal & "%JAVA_EXE%" %DEFAULT_JVM_OPTS% %JAVA_OPTS% %GRADLE_OPTS% "-Dorg.gradle.appname=%APP_BASE_NAME%" -jar "%APP_HOME%\gradle\wrapper\gradle-wrapper.jar" %* & call :exitWithErrorLevel
|
||||
|
||||
:exitWithErrorLevel
|
||||
@rem Use "%COMSPEC%" /c exit to allow operators to work properly in scripts
|
||||
"%COMSPEC%" /c exit %ERRORLEVEL%
|
||||
@@ -1,33 +0,0 @@
|
||||
# This sample, non-production-ready template describes an Amazon EC2 instance and an Elastic Load Balancer.
|
||||
# © 2020 Amazon Web Services, Inc. or its affiliates. All Rights Reserved.
|
||||
# This AWS Content is provided subject to the terms of the AWS Customer Agreement available at
|
||||
# http://aws.amazon.com/agreement or other written agreement between Customer and either
|
||||
# Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both.
|
||||
# ARG ACCOUNT_ID=766380763301
|
||||
|
||||
FROM ${buildRegistry}/build/java:8-jdk-slim-secure_v1.0
|
||||
#FROM asia-southeast1-docker.pkg.dev/supply-poc-351106/homelab-devops/java:8
|
||||
ARG artifactId=sample
|
||||
ARG XMS=2G
|
||||
ARG XMX=2G
|
||||
ARG target
|
||||
|
||||
ADD https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.15.0/jmx_prometheus_javaagent-0.15.0.jar /opt/jmx_exporter.jar
|
||||
#ADD https://github.com/open-telemetry/opentelemetry-java-instrumentation/releases/download/v1.17.0/opentelemetry-javaagent.jar /opt/opentelemetry-javaagent.jar
|
||||
|
||||
### Config added through configmap
|
||||
# COPY config.yaml /opt/config.yaml
|
||||
EXPOSE 8880 8010
|
||||
|
||||
COPY ${artifactId}/target/*.jar /opt/target/${artifactId}.jar
|
||||
RUN mkdir -p /var/log/${artifactId} && touch /var/log/${artifactId}/gc.log
|
||||
|
||||
WORKDIR /opt/target
|
||||
|
||||
CMD ["${artifactId}.jar", "-javaagent:/opt/jmx_exporter.jar=8880:/opt/config/jmx-config.yaml", \
|
||||
"-XX:MinRAMPercentage=50.0", "-XX:MaxRAMPercentage=80.0", \
|
||||
"-XX:+UseParallelGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails", \
|
||||
"-XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime", "-XX:+PrintHeapAtGC", \
|
||||
"-Xloggc:/var/log/${artifactId}/gc.log", \
|
||||
"-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=9000k", \
|
||||
"-Xms${XMS}", "-Xmx${XMX}"]
|
||||
@@ -1,31 +0,0 @@
|
||||
apiVersion: argoproj.io/v1alpha1
|
||||
kind: Application
|
||||
metadata:
|
||||
name: ${env_ns}-${app_name}
|
||||
namespace: ${argoAppNS}
|
||||
labels:
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
app_name: ${app_name}
|
||||
service: ${app_name}
|
||||
env: ${environment}
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
commit_id: ${commit_id}
|
||||
spec:
|
||||
destination:
|
||||
namespace: ${env_ns}-${app_name}
|
||||
<% if (CLOUD_PROVIDER == 'AWS') { print "server: ${clusterName}" } %>
|
||||
<% if (CLOUD_PROVIDER == 'GCP') { print "name: ${clusterName}" } %>
|
||||
project: ${buini}-${teamini}
|
||||
source:
|
||||
helm:
|
||||
valueFiles:
|
||||
- ../${helm_values_path}/values.yaml
|
||||
path: ${helm_version}
|
||||
repoURL: https://github.com/Homelab/devops-helm-charts.git
|
||||
targetRevision: ${branch_name}
|
||||
syncPolicy:
|
||||
syncOptions:
|
||||
- CreateNamespace=true
|
||||
@@ -1,14 +0,0 @@
|
||||
---
|
||||
|
||||
repo_name: ${repo_name}
|
||||
maintainer: ${maintainer}
|
||||
skip_sonar: true
|
||||
build_tool: docker
|
||||
dockerBuildVersion: ${dockerBuildVersion}
|
||||
arch: ${arch}
|
||||
<% if (excludedMoudles){ println 'excludedModules:';for(module in excludedMoudles){ println ' - '+module }} %>team: ${team}
|
||||
bu: ${bu}
|
||||
deployArgo: true
|
||||
deployment_order:
|
||||
<% for (deployment in deployment_order){println ' - '+deployment} %><% if (branch_params){ println 'branch_params:'; branch_params.each {entry -> println " $entry.key:";entry.value.each{ val_e -> println " $val_e.key:$val_e.value" }}} %>
|
||||
notify_channel: ${slack_channel}
|
||||
@@ -1,99 +0,0 @@
|
||||
repoName: ${repo_name}
|
||||
applicationName: ${app_name}
|
||||
appType: ${dockerBuildVersion}
|
||||
deployment:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
canary:
|
||||
enabled: false
|
||||
autoscaling:
|
||||
enabled: false
|
||||
podDisruptionBudget:
|
||||
enabled: false
|
||||
maxUnavailable: 100%
|
||||
minAvailable: ""
|
||||
cron:
|
||||
enabled: true
|
||||
concurrencyPolicy: ${concurrencyPolicy}
|
||||
failedJobsHistoryLimit: ${failedJobsHistoryLimit}
|
||||
successfulJobsHistoryLimit: ${successfulJobsHistoryLimit}
|
||||
suspend: false
|
||||
startingDeadlineSeconds: ${startingDeadlineSeconds}
|
||||
backoffLimit: ${backoffLimit}
|
||||
restartPolicy: ${restartPolicy}
|
||||
env:
|
||||
- name: PRISMSDK_ENVIRONMENT
|
||||
value: ${prismsdk_environment}
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module+'' %>
|
||||
tag: ${tag}
|
||||
podAnnotations:
|
||||
<% if (appMetrics && dockerBuildVersion.contains("maven")) { print 'jmx.io/path: /metrics' %>
|
||||
<% print 'jmx.io/port: "8880"' %>
|
||||
<% print 'jmx.io/scrape: "true"'} %>
|
||||
<% if (nodeSelector.contains("arm64") && (environment=="int" || environment=="prd")) { print 'telegraf.influxdata.com/image: 847438129436.dkr.ecr.ap-southeast-1.amazonaws.com/telegraf:1.24.4-arm64' } %>
|
||||
<% if ( CLOUD_PROVIDER == "GCP" ) { print 'telegraf.influxdata.com/image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/sre/telegraf:1.24.4' } %>
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> if(v instanceof String) { println " ${k}: '${v}'"} else { println " ${k}: ${v}" }};} else {print ''} %>
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
<% println ' jobs:'; for(val in jobs){ val.each{k,v -> println " ${k}:"; println " resources:"; println " limits:"; println " cpu: ${v.cpu_limit}"; println " memory: ${v.memory_limit}i"; println " requests:"; println " cpu: ${v.cpu_request}"; println " memory: ${v.memory_request}i"; println " schedule: \"${v.schedule}\""; if (v.command instanceof List){ println ' command:'; for (com in v.command ) { if (com != v.command.last()){ println " - ${com}" } else {println " - ${com};touch /tmp/podtermination/telegraf-termination"} }}; if(v.args){ if (v.args instanceof List){ println ' args:'; for (arg in v.args ) { println " - ${arg}" } }}}} %>
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
ingress:
|
||||
enabled: false
|
||||
jmxconfig:
|
||||
enabled: true
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
nameOverride: ""
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
enabled: false
|
||||
|
||||
|
||||
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
@@ -1,203 +0,0 @@
|
||||
---
|
||||
- name: Playbook to deploy Jar
|
||||
hosts: all
|
||||
gather_facts: false
|
||||
become: yes
|
||||
become_method: sudo
|
||||
|
||||
tasks:
|
||||
- name: Check if region is configured
|
||||
shell: "aws configure get default.region"
|
||||
register: aws_cli_region_reponse
|
||||
|
||||
- name: "ERROR: AWS region missing"
|
||||
vars:
|
||||
msg: |
|
||||
#############################################################################################################################
|
||||
####### AWS REGION IS NOT CONFIGURED PROPERLY. CURRENT REGION: {{ aws_cli_region_reponse.stdout | upper }}, SHOULD BE: AP-SOUTHEAST-1 #######
|
||||
#############################################################################################################################
|
||||
debug:
|
||||
msg: "{{ msg.split('\n') }}"
|
||||
when: aws_cli_region_reponse.stdout != "ap-southeast-1"
|
||||
failed_when:
|
||||
- aws_cli_region_reponse.stdout != "ap-southeast-1"
|
||||
|
||||
- name: Check if CICD profile is configured
|
||||
shell: "aws configure get region --profile cicd"
|
||||
register: aws_profile_cli_region_reponse
|
||||
|
||||
- name: "ERROR: AWS region missing in profile"
|
||||
vars:
|
||||
msg: |
|
||||
#############################################################################################################################
|
||||
####### AWS REGION IS NOT CONFIGURED PROPERLY. CURRENT REGION: {{ aws_cli_region_reponse.stdout | upper }}, SHOULD BE: AP-SOUTHEAST-1 #######
|
||||
#############################################################################################################################
|
||||
debug:
|
||||
msg: "{{ msg.split('\n') }}"
|
||||
when: aws_profile_cli_region_reponse.stdout != "ap-southeast-1"
|
||||
failed_when:
|
||||
- aws_profile_cli_region_reponse.stdout != "ap-southeast-1"
|
||||
|
||||
- name: Check if IAM role is attached
|
||||
shell: "aws sts get-caller-identity"
|
||||
register: aws_cli_gci_reponse
|
||||
|
||||
- name: "ERROR: IAM role missing"
|
||||
vars:
|
||||
msg: |
|
||||
#############################################################################################################################
|
||||
############################################### CICD ROLE IS NOT ATTACHED ###############################################
|
||||
#############################################################################################################################
|
||||
debug:
|
||||
msg: "{{ msg.split('\n') }}"
|
||||
when:
|
||||
- aws_cli_gci_reponse.stdout is not search("assumed-role/cicd")
|
||||
failed_when:
|
||||
- aws_cli_gci_reponse.stdout is not search("assumed-role/cicd")
|
||||
|
||||
- name: Check if systemd service exists
|
||||
stat:
|
||||
path: "/etc/systemd/system/{{ app_name }}.service"
|
||||
register: service_status
|
||||
|
||||
- name: Systemd service status
|
||||
vars:
|
||||
msg: |
|
||||
#############################################################################################################################
|
||||
################################ SYSTEMD SERVICE {{ app_name | upper }} DOES NOT EXIST. EXITING. ##################################
|
||||
#############################################################################################################################
|
||||
debug:
|
||||
msg: "{{ msg.split('\n') }}"
|
||||
when: service_status is defined and not service_status.stat.exists
|
||||
failed_when:
|
||||
- not service_status.stat.exists
|
||||
|
||||
- name: Check application directory structure
|
||||
block:
|
||||
|
||||
- name: Check application directory structure
|
||||
stat:
|
||||
path: "/home/ubuntu/{{ app_name }}"
|
||||
register: app_dir_status
|
||||
|
||||
- name: Application directory does not exist
|
||||
vars:
|
||||
msg: |
|
||||
#############################################################################################################################
|
||||
###################### APPLICATION DIRECTORY: /home/ubuntu/{{ app_name }} DOES NOT EXIST. EXITING. #######################
|
||||
#############################################################################################################################
|
||||
debug:
|
||||
msg: "{{ msg.split('\n') }}"
|
||||
when: app_dir_status is defined and not app_dir_status.stat.exists
|
||||
failed_when:
|
||||
- not app_dir_status.stat.exists
|
||||
|
||||
tags:
|
||||
- artifact_deployment
|
||||
|
||||
- name: Ensure jq is installed
|
||||
apt:
|
||||
name: "jq"
|
||||
state: present
|
||||
force_apt_get: yes
|
||||
|
||||
- name: Checking if environment file exists, pre-deployment
|
||||
stat:
|
||||
path: "/etc/sysconfig/{{ app_name }}"
|
||||
register: env_file
|
||||
|
||||
- name: Backing up current properties
|
||||
copy:
|
||||
src: "/etc/sysconfig/{{ app_name }}"
|
||||
dest: "/etc/sysconfig/rollback-{{ app_name }}"
|
||||
owner: ubuntu
|
||||
group: ubuntu
|
||||
mode: '0664'
|
||||
remote_src: yes
|
||||
force: yes
|
||||
when: env_file.stat.exists
|
||||
|
||||
- name: Checking if current symlink exists, pre-deployment
|
||||
stat:
|
||||
path: "/home/ubuntu/{{ app_name }}/{{ app_name }}-current.jar"
|
||||
register: current_symlink
|
||||
|
||||
- name: Show output
|
||||
debug: msg= "{{ current_symlink }}"
|
||||
|
||||
- name: Checking if current artifact exists, pre-deployment
|
||||
stat:
|
||||
path: "{{ current_symlink.stat.lnk_source }}"
|
||||
when:
|
||||
- current_symlink.stat.exists
|
||||
- current_symlink.stat.islnk
|
||||
register: current_artifact
|
||||
|
||||
- name: Show output
|
||||
debug: msg= "{{ current_artifact }}"
|
||||
|
||||
- name: Stopping systemd service
|
||||
systemd:
|
||||
name: '{{ app_name }}'
|
||||
state: stopped
|
||||
|
||||
- name: Download latest properties
|
||||
shell: "pull-env '{{ env }}' '{{ app_name }}'"
|
||||
ignore_errors: yes
|
||||
|
||||
- name: Download latest artifact
|
||||
copy:
|
||||
src: "{{ item }}"
|
||||
dest: "/home/ubuntu/{{ app_name }}/"
|
||||
owner: ubuntu
|
||||
group: ubuntu
|
||||
with_fileglob:
|
||||
- "{{ repo_name }}/target/*.jar"
|
||||
|
||||
- name: Setting latest_artifact_name
|
||||
set_fact:
|
||||
latest_artifact_name: '{{ item.split("/")[-1] }}'
|
||||
with_fileglob:
|
||||
- "{{ repo_name }}/target/*.jar"
|
||||
|
||||
- name: Creating rollback symlink
|
||||
file:
|
||||
src: "{{ current_artifact.stat.path }}"
|
||||
dest: "/home/ubuntu/{{ app_name }}/rollback"
|
||||
state: link
|
||||
|
||||
- name: Removing current symlink
|
||||
file:
|
||||
path: "/home/ubuntu/{{ app_name }}/{{ app_name }}-current.jar"
|
||||
state: absent
|
||||
|
||||
- name: Creating current symlink to latest artifact
|
||||
file:
|
||||
src: "/home/ubuntu/{{ app_name }}/{{ latest_artifact_name }}"
|
||||
dest: "/home/ubuntu/{{ app_name }}/{{ app_name }}-current.jar"
|
||||
state: link
|
||||
|
||||
- name: Starting systemd service
|
||||
systemd:
|
||||
name: '{{ app_name }}'
|
||||
state: started
|
||||
daemon_reload: yes
|
||||
|
||||
- name: Trying healthcheck
|
||||
uri:
|
||||
url: "http://localhost:{{ app_port }}{{ healthcheck_api }}"
|
||||
method: GET
|
||||
register: healthcheck_response
|
||||
until: healthcheck_response.status == 200
|
||||
retries: 60
|
||||
delay: 1
|
||||
ignore_errors: False
|
||||
|
||||
- name: Healthcheck response
|
||||
vars:
|
||||
msg: |
|
||||
##############################################################################################################################
|
||||
HEALTH-CHECK ({{ healthcheck_api }}) RESPONSE: {{ healthcheck_response.status }}
|
||||
##############################################################################################################################
|
||||
debug:
|
||||
msg: "{{ msg.split('\n') }}"
|
||||
@@ -1,103 +0,0 @@
|
||||
---
|
||||
|
||||
app_name: ${app_name}
|
||||
app_port: ${app_port}
|
||||
health_check: ${health_check}
|
||||
module: module_less
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
priority: ${priority}
|
||||
priority_v2: ${priority_v2}
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
replica_count: ${replica_count}
|
||||
environment:
|
||||
ftr:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=dev
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
stg:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=dev
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
int:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=int
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
prd:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=prd
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
@@ -4,14 +4,23 @@
|
||||
# kafka-specific CGO toggle. Assumes a standard single-binary repo layout
|
||||
# (main package at the repo root) — a repo with a different structure
|
||||
# should just bring its own Dockerfile, same as demo-go-app does.
|
||||
FROM golang:${version}-alpine AS build
|
||||
# Both stages pulled from Harbor's base-images project (mirrored from
|
||||
# Docker Hub via devops-base-images), not Docker Hub directly — see that
|
||||
# repo's README for the one-off mirror setup and why (build-time
|
||||
# dependency on an external registry, plus wanting to pick the leanest
|
||||
# variant of each deliberately rather than accept whatever a public tag
|
||||
# defaults to). Only versions actually mirrored there resolve — passing
|
||||
# a dockerBuildVersion whose tag isn't in devops-base-images/images.txt
|
||||
# yet needs that added and re-mirrored first, unlike pulling straight
|
||||
# from Docker Hub where any tag "just worked".
|
||||
FROM harbor.infra.deployshed.com/base-images/golang:${version}-alpine AS build
|
||||
WORKDIR /src
|
||||
COPY go.mod go.sum* ./
|
||||
RUN go mod download 2>/dev/null || true
|
||||
COPY . .
|
||||
RUN CGO_ENABLED=0 go build -o /app .
|
||||
|
||||
FROM alpine:3.20
|
||||
FROM harbor.infra.deployshed.com/base-images/alpine:3.20
|
||||
COPY --from=build /app /app
|
||||
EXPOSE 8080
|
||||
ENTRYPOINT ["/app"]
|
||||
|
||||
@@ -1,65 +0,0 @@
|
||||
---
|
||||
|
||||
app_name: ${app_name}
|
||||
app_port: ${app_port}
|
||||
health_check: ${health_check}
|
||||
module: module_less
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
priority: ${priority}
|
||||
priority_v2: ${priority_v2}
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
replica_count: ${replica_count}
|
||||
environment:
|
||||
ftr:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args: false
|
||||
command: /app/server
|
||||
|
||||
dev:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args: false
|
||||
command: /app/server
|
||||
int:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args: false
|
||||
command: /app/server
|
||||
prd:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args: false
|
||||
command: /app/server
|
||||
|
||||
@@ -1,202 +0,0 @@
|
||||
<% if (deploymentStrategy == 'canary'){ print "canary:\n enabled: true\n slackChannel: ${canary.slackChannel}\n enableManualPromotion: ${enableManualPromotion}\n skipAnalysis: ${canary.skipAnalysis}\n service:\n port: 80\n targetPort: ${(grpc_host || grpc_hosts) ? app_port : primary_port}\n progressDeadlineSeconds: ${canary.progressDeadlineSeconds}\n minCanaryReplicas: ${minCanaryReplicas}\n maxCanaryReplicas: ${maxCanaryReplicas}\n analysisInterval: ${canary.analysisInterval}\n analysisThreshold: ${canary.analysisThreshold}\n analysisMaxWeight: ${canary.analysisMaxWeight}\n analysisStepWeight: ${canary.analysisStepWeight}\n analysisMetrics:\n thresholdRangeMin: ${canary.analysisMetrics.thresholdRangeMin}\n interval: ${canary.analysisMetrics.interval}\n" } else { print "canary:\n enabled: false\n" } %>
|
||||
repoName: ${repo_name}
|
||||
cron:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
applicationName: ${app_name}
|
||||
autoscaling:
|
||||
enabled: ${as_enabled}
|
||||
maxReplicas: ${as_max}
|
||||
minReplicas: ${as_min}
|
||||
pollingInterval: ${as_poll}
|
||||
scaledown:
|
||||
policies:
|
||||
- periodseconds: ${as_down_period}
|
||||
type: Pods
|
||||
value: ${as_down_pod_count}
|
||||
selectpolicy: Min
|
||||
stabilizationWindowSeconds: ${as_down_stable_window}
|
||||
scaleup:
|
||||
policies:
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Pods
|
||||
value: ${as_up_pod_count}
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Percent
|
||||
value: ${as_up_pod_percentage}
|
||||
selectpolicy: Max
|
||||
stabilizationWindowSeconds: ${as_up_stable_window}
|
||||
<% if(!triggers){print " triggers:\n - metadata:\n value: \"${as_trigger_value}\"\n metricType: ${as_trigger_type}\n type: ${as_trigger_metric}"} else {println ' triggers:'; for(val in triggers){if(val instanceof Map){ val.each{k,v -> if (v instanceof Map) { println ' - '+k+':'; v.each{a,b -> if(b.isNumber()){println " ${a}: \"${b}\"" } else {println " ${a}: ${b}" }}} else {println" ${k}: ${v}"}}}}} %>
|
||||
replicaCount: ${replica_count}
|
||||
deployment:
|
||||
affinity: {}
|
||||
podDistributionSkew: ${podDistributionSkew}
|
||||
<% if (deployment_args){println ' args:'; for (arg in deployment_args){ println ' - '+arg}} %>
|
||||
command:
|
||||
- ${command}
|
||||
enabled: true
|
||||
env:
|
||||
- name: GOMAXPROCS
|
||||
value: ${activeProcessorCount}
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
<% if(lifecycle) { println ' lifecycle:\n preStop:\n exec:\n command:'; for(val in lifecycle.preStop.exec.command){ println " - $val" }} else {print " lifecycle:\n preStop:\n exec:\n command:\n - /bin/bash\n - -c\n - kill -SIGQUIT 1 ; /bin/sleep 120\n "} %>
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (nodeSelector.contains("arm64") && (environment=="int" || environment=="prd")) { print 'telegraf.influxdata.com/image: 847438129436.dkr.ecr.ap-southeast-1.amazonaws.com/telegraf:1.24.4-arm64' } %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> println " ${k}: '${v}'"};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
- containerPort: 8880
|
||||
name: metric
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
updateStrategy:
|
||||
strategy:
|
||||
<% if (deploymentStrategy == 'recreate') { print " type: Recreate"} else { print " type: RollingUpdate\n rollingUpdate:\n maxUnavailable: 0%\n maxSurge: ${maxSurge}%" } %>
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
fullnameOverride: ""
|
||||
ingress:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/use-regex: "true"
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/server-snippet: |
|
||||
location ~* "^/api/1.0/search/recent" {
|
||||
default_type application/json;
|
||||
return 200 '{"recent_searches": [],"recent_suggestions": [],"limit": 5,"autosuggest_use_recent": true,"show_recent_header": false}';
|
||||
}
|
||||
location ~* "^/api/1.0/anonymous/search/recent" {
|
||||
default_type application/json;
|
||||
return 200 '{"recent_searches": [],"recent_suggestions": [],"limit": 5,"autosuggest_use_recent": true,"show_recent_header": false}';
|
||||
}
|
||||
location ~* "^/search-queries/recent" {
|
||||
default_type application/json;
|
||||
return 200 '{"recent_searches": [],"recent_suggestions": [],"limit": 5,"autosuggest_use_recent": true,"show_recent_header": false}';
|
||||
}
|
||||
<% if (ingress_annotations) {ingress_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
<% if(grpc_host){println " grpc_hosts:\n - host: ${grpc_host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else if (grpc_hosts) {println " grpc_hosts:";for(host_arr in grpc_hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %>
|
||||
<% if(host){println " hosts:\n - host: ${host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else {println " hosts:";for(host_arr in hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %> ingressClassName: ${ingress_class}
|
||||
servicePort: http
|
||||
enableWebsocket: ${enableWebsocket}
|
||||
slowStart:
|
||||
enabled: <% if (slowStartWindow) { print "true" } else { print "false" } %>
|
||||
window: <% if (slowStartWindow) { print "${slowStartWindow}" } else { print "120s" } %>
|
||||
aggression: <% if (slowStartAggression) { print "${slowStartAggression}" } else { print "1.0" } %>
|
||||
minPercent: <% if (slowStartMinPercent) { print "${slowStartMinPercent}" } else { print "10" } %>
|
||||
jmxconfig:
|
||||
enabled: false
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
nameOverride: ""
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podDisruptionBudget:
|
||||
enabled: <% if (pdbMaxUnavailable) { print "true" } else { print "false" } %>
|
||||
maxUnavailable: ${pdbMaxUnavailable}
|
||||
minAvailable: ${pdbMinAvailable}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
<% if (service_annotations) { println " annotations:"; service_annotations.each{k,v -> println " ${k}: \"${v}\""};} else { println " annotations: null" } %>
|
||||
enabled: true
|
||||
<% if (addon_ports) { for (p in addon_ports) { println " addons:"; println " - name: ${p.name}"; println " targetPort: ${p.targetPort}"; println " type: ${p.type}"; }} else { println " addon_ports: []" } %>
|
||||
<% if (grpc_port && app_port && (grpc_host || grpc_hosts)) {
|
||||
println " ports:"
|
||||
println " - name: http"
|
||||
println " port: 80"
|
||||
println " protocol: TCP"
|
||||
println " targetPort: ${app_port}"
|
||||
println " grpc_ports:"
|
||||
println " - name: grpc"
|
||||
println " port: 80"
|
||||
println " protocol: TCP"
|
||||
println " targetPort: ${grpc_port}"
|
||||
} else {
|
||||
println " ports:"
|
||||
println " - name: http"
|
||||
println " port: 80"
|
||||
println " protocol: TCP"
|
||||
println " targetPort: ${primary_port}"
|
||||
}
|
||||
%>
|
||||
type: ClusterIP
|
||||
|
||||
createContourGateway: <% if (createContourGateway) { print "${createContourGateway}" } else { print "false" } %>
|
||||
contourResponseTimeout: ${contourResponseTimeout}
|
||||
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
@@ -1,103 +0,0 @@
|
||||
---
|
||||
|
||||
app_name: ${app_name}
|
||||
app_port: ${app_port}
|
||||
health_check: ${health_check}
|
||||
module: module_less
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
priority: ${priority}
|
||||
priority_v2: ${priority_v2}
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
replica_count: ${replica_count}
|
||||
environment:
|
||||
ftr:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=dev
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
stg:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=dev
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
int:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=int
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
prd:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- -Dspring.profiles.active=prd
|
||||
- -XX:+UseG1GC
|
||||
- -XX:+PrintGCDateStamps
|
||||
- -XX:+PrintGCDetails
|
||||
- -XX:+PrintGCApplicationStoppedTime
|
||||
- -XX:+PrintGCApplicationConcurrentTime
|
||||
- -XX:+PrintHeapAtGC
|
||||
- -Xloggc:/var/log/gc.log
|
||||
- -XX:+UseGCLogFileRotation
|
||||
- -XX:NumberOfGCLogFiles=5
|
||||
- -XX:GCLogFileSize=9000k
|
||||
@@ -1,15 +1,21 @@
|
||||
# Fallback only — see go-Dockerfile's header comment for the general
|
||||
# rule. Simplified from the real java-Dockerfile: no JFrog artifact
|
||||
# rule, including the Harbor base-images sourcing (also applies here).
|
||||
# Simplified from the real java-Dockerfile: no JFrog artifact
|
||||
# resolution, no Homelab-internal Maven mirror. Assumes a standard Maven
|
||||
# repo producing a single runnable jar under target/.
|
||||
FROM maven:3-eclipse-temurin-${version} AS build
|
||||
# repo producing a single runnable jar under target/. Switched both
|
||||
# stages from their Debian defaults to the -alpine variant — smaller,
|
||||
# still keeps a shell (not distroless). Maven itself still reaches out
|
||||
# to Maven Central for plugins/dependencies during the build regardless
|
||||
# of base image — this only removes the Docker Hub dependency for the
|
||||
# base image layer, not package-registry traffic during the build.
|
||||
FROM harbor.infra.deployshed.com/base-images/maven:3-eclipse-temurin-${version}-alpine AS build
|
||||
WORKDIR /src
|
||||
COPY pom.xml .
|
||||
RUN mvn -B dependency:go-offline
|
||||
COPY . .
|
||||
RUN mvn -B package -DskipTests
|
||||
|
||||
FROM eclipse-temurin:${version}-jre
|
||||
FROM harbor.infra.deployshed.com/base-images/eclipse-temurin:${version}-jre-alpine
|
||||
WORKDIR /app
|
||||
COPY --from=build /src/target/*.jar app.jar
|
||||
EXPOSE 8080
|
||||
|
||||
@@ -1,15 +1,18 @@
|
||||
# Fallback only — see go-Dockerfile's header comment for the general
|
||||
# rule. Simplified from the real node-Dockerfile: no PBAC registry sync,
|
||||
# rule, including the Harbor base-images sourcing (also applies here).
|
||||
# Simplified from the real node-Dockerfile: no PBAC registry sync,
|
||||
# no inline Sonar/coverage stage, no .npmrc-across-subdirectories dance.
|
||||
# Assumes a standard `npm run build` + `npm start` repo.
|
||||
FROM node:${version}-slim AS build
|
||||
# Assumes a standard `npm run build` + `npm start` repo. Switched from
|
||||
# node:*-slim (Debian) to node:*-alpine for both stages — smaller, still
|
||||
# keeps a shell for kubectl exec debugging (not distroless).
|
||||
FROM harbor.infra.deployshed.com/base-images/node:${version}-alpine AS build
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm ci
|
||||
COPY . .
|
||||
RUN npm run build --if-present
|
||||
|
||||
FROM node:${version}-slim
|
||||
FROM harbor.infra.deployshed.com/base-images/node:${version}-alpine
|
||||
WORKDIR /app
|
||||
COPY --from=build /app .
|
||||
ENV NODE_ENV=production
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
---
|
||||
|
||||
app_name: ${app_name}
|
||||
app_port: ${app_port}
|
||||
health_check: ${health_check}
|
||||
module: module_less
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
priority: ${priority}
|
||||
priority_v2: ${priority_v2}
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
replica_count: ${replica_count}
|
||||
environment:
|
||||
ftr:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- pm2-config.json
|
||||
stg:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- pm2-config.json
|
||||
int:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- pm2-config.json
|
||||
prd:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
- pm2-config.json
|
||||
|
||||
@@ -1,184 +0,0 @@
|
||||
<% if (deploymentStrategy == 'canary'){ print "canary:\n enabled: true\n slackChannel: ${canary.slackChannel}\n enableManualPromotion: ${enableManualPromotion}\n skipAnalysis: ${canary.skipAnalysis}\n service:\n port: 80\n targetPort: ${primary_port}\n progressDeadlineSeconds: ${canary.progressDeadlineSeconds}\n minCanaryReplicas: ${minCanaryReplicas}\n maxCanaryReplicas: ${maxCanaryReplicas}\n analysisInterval: ${canary.analysisInterval}\n analysisThreshold: ${canary.analysisThreshold}\n analysisMaxWeight: ${canary.analysisMaxWeight}\n analysisStepWeight: ${canary.analysisStepWeight}\n analysisMetrics:\n thresholdRangeMin: ${canary.analysisMetrics.thresholdRangeMin}\n interval: ${canary.analysisMetrics.interval}\n" } else { print "canary:\n enabled: false\n" } %>repoName: ${repo_name}
|
||||
cron:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
applicationName: ${app_name}
|
||||
autoscaling:
|
||||
enabled: ${as_enabled}
|
||||
maxReplicas: ${as_max}
|
||||
minReplicas: ${as_min}
|
||||
pollingInterval: ${as_poll}
|
||||
scaledown:
|
||||
policies:
|
||||
- periodseconds: ${as_down_period}
|
||||
type: Pods
|
||||
value: ${as_down_pod_count}
|
||||
selectpolicy: Min
|
||||
stabilizationWindowSeconds: ${as_down_stable_window}
|
||||
scaleup:
|
||||
policies:
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Pods
|
||||
value: ${as_up_pod_count}
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Percent
|
||||
value: ${as_up_pod_percentage}
|
||||
selectpolicy: Max
|
||||
stabilizationWindowSeconds: ${as_up_stable_window}
|
||||
<% if(!triggers){print " triggers:\n - metadata:\n value: \"${as_trigger_value}\"\n metricType: ${as_trigger_type}\n type: ${as_trigger_metric}"} else {println ' triggers:'; for(val in triggers){if(val instanceof Map){ val.each{k,v -> if (v instanceof Map) { println ' - '+k+':'; v.each{a,b -> if(b.isNumber()){println " ${a}: \"${b}\"" } else {println " ${a}: ${b}" }}} else {println" ${k}: ${v}"}}}}} %>
|
||||
deployment:
|
||||
affinity: {}
|
||||
podDistributionSkew: ${podDistributionSkew}
|
||||
args:
|
||||
<% for (arg in deployment_args){ println ' - '+arg}%>
|
||||
command:
|
||||
- ${command}
|
||||
enabled: true
|
||||
env: null
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
lifecycle:
|
||||
preStop:
|
||||
exec:
|
||||
command:
|
||||
- /bin/bash
|
||||
- -c
|
||||
- kill -SIGQUIT 1 ; /bin/sleep 60
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
<% if (appMetrics) { print 'prometheus.io/path: "/actuator/prometheus"' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> println " ${k}: \"${v}\""};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
- containerPort: 9209
|
||||
name: pm2-metrics
|
||||
protocol: TCP
|
||||
- containerPort: 9200
|
||||
name: metrics
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
replicaCount: ${replica_count}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
updateStrategy:
|
||||
strategy:
|
||||
<% if (deploymentStrategy == 'recreate') { print " type: Recreate"} else { print " type: RollingUpdate\n rollingUpdate:\n maxUnavailable: 0%\n maxSurge: ${maxSurge}%" } %>
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
fullnameOverride: ""
|
||||
ingress:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/use-regex: "true"
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "false"
|
||||
<% if (ingress_annotations) {ingress_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
<% if(grpc_host){println " grpc_hosts:\n - host: ${grpc_host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else if (grpc_hosts) {println " grpc_hosts:";for(host_arr in grpc_hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %>
|
||||
<% if(host){println " hosts:\n - host: ${host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else {println " hosts:";for(host_arr in hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %> ingressClassName: ${ingress_class}
|
||||
servicePort: http
|
||||
enableWebsocket: ${enableWebsocket}
|
||||
slowStart:
|
||||
enabled: <% if (slowStartWindow) { print "true" } else { print "false" } %>
|
||||
window: <% if (slowStartWindow) { print "${slowStartWindow}" } else { print "120s" } %>
|
||||
aggression: <% if (slowStartAggression) { print "${slowStartAggression}" } else { print "1.0" } %>
|
||||
minPercent: <% if (slowStartMinPercent) { print "${slowStartMinPercent}" } else { print "10" } %>
|
||||
jmxconfig:
|
||||
enabled: false
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
node_prom_client_enabled: true
|
||||
nameOverride: ""
|
||||
replicaCount: ${replica_count}
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podDisruptionBudget:
|
||||
enabled: <% if (pdbMaxUnavailable) { print "true" } else { print "false" } %>
|
||||
maxUnavailable: ${pdbMaxUnavailable}
|
||||
minAvailable: ${pdbMinAvailable}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
<% if (service_annotations) { println " annotations:"; service_annotations.each{k,v -> println " ${k}: \"${v}\""};} else { println " annotations: null" } %>
|
||||
enabled: true
|
||||
<% if (addon_ports) { for (p in addon_ports) { println " addons:"; println " - name: ${p.name}"; println " targetPort: ${p.targetPort}"; println " type: ${p.type}"; }} else { println " addon_ports: []" } %>
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: TCP
|
||||
targetPort: ${primary_port}
|
||||
type: ClusterIP
|
||||
|
||||
createContourGateway: <% if (createContourGateway) { print "${createContourGateway}" } else { print "false" } %>
|
||||
contourResponseTimeout: ${contourResponseTimeout}
|
||||
|
||||
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
@@ -1,12 +1,52 @@
|
||||
# Fallback only — see go-Dockerfile's header comment for the general
|
||||
# rule. Simplified from the real php-Dockerfile. Assumes a standard
|
||||
# composer-based repo served by Apache.
|
||||
FROM php:${version}-apache
|
||||
# rule, including the Harbor base-images sourcing (also applies here).
|
||||
# Simplified from the real php-Dockerfile. Assumes a standard
|
||||
# composer-based repo.
|
||||
#
|
||||
# Was php:*-apache (Debian, full Apache httpd) — dropped Apache
|
||||
# entirely in favor of php:*-cli-alpine + PHP's own built-in dev server
|
||||
# (`php -S`). Genuinely minimal (no httpd, no extra process, Alpine
|
||||
# base) and brings this language in line with every other one here on
|
||||
# port 8080 instead of PHP's special-cased 80. Trade-off, stated
|
||||
# plainly: PHP's own docs call the built-in server "not designed to be
|
||||
# a full-featured web server" for production — perfectly fine for a
|
||||
# homelab/demo app, would need revisiting (php-fpm + nginx, two
|
||||
# processes/containers) for anything serving real production traffic.
|
||||
#
|
||||
# docker-php-ext-install needs PHPIZE_DEPS present to compile
|
||||
# extensions on Alpine (unlike the Debian image, which had them
|
||||
# preinstalled) — installed as a virtual package and removed again
|
||||
# right after, so the final image doesn't carry build tooling.
|
||||
#
|
||||
# NOTE ON DOLLAR SIGNS IN THIS FILE (read before editing anything
|
||||
# below, comments included): the whole file — every line, comments
|
||||
# included — is fed through constructTemplate.groovy's
|
||||
# SimpleTemplateEngine before it becomes a real Dockerfile. That
|
||||
# engine treats any dollar-sign character as the start of a Groovy
|
||||
# interpolation, whether or not a human reading it would call it
|
||||
# "code". A dollar sign followed by a letter or underscore gets
|
||||
# looked up in the render binding (only `version` exists there) and
|
||||
# throws a MissingPropertyException if not found there; a dollar sign
|
||||
# followed by anything else (punctuation, space, end of line) can't
|
||||
# even be parsed as an interpolation attempt and throws a harder
|
||||
# syntax error instead. The only dollar sign meant to reach the shell
|
||||
# below (in the PHPIZE_DEPS line) is escaped with a leading backslash
|
||||
# for exactly this reason. Because a stray dollar sign in prose is
|
||||
# this easy to reintroduce by accident (an earlier revision of this
|
||||
# very comment did so), new edits to this header should avoid typing
|
||||
# the character at all — write "dollar sign" in words instead of using
|
||||
# the glyph.
|
||||
FROM harbor.infra.deployshed.com/base-images/php:${version}-cli-alpine
|
||||
WORKDIR /var/www/html
|
||||
RUN docker-php-ext-install pdo pdo_mysql
|
||||
RUN apk add --no-cache --virtual .build-deps \$PHPIZE_DEPS \
|
||||
&& docker-php-ext-install pdo pdo_mysql \
|
||||
&& apk del .build-deps
|
||||
COPY . .
|
||||
RUN if [ -f composer.json ]; then \
|
||||
apk add --no-cache --virtual .composer-deps curl && \
|
||||
curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer && \
|
||||
composer install --no-dev --optimize-autoloader; \
|
||||
composer install --no-dev --optimize-autoloader && \
|
||||
apk del .composer-deps; \
|
||||
fi
|
||||
EXPOSE 80
|
||||
EXPOSE 8080
|
||||
CMD ["php", "-S", "0.0.0.0:8080", "-t", "."]
|
||||
|
||||
@@ -1,55 +0,0 @@
|
||||
---
|
||||
|
||||
app_name: ${app_name}
|
||||
app_port: ${app_port}
|
||||
health_check: ${health_check}
|
||||
module: module_less
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
priority: ${priority}
|
||||
priority_v2: ${priority_v2}
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
replica_count: ${replica_count}
|
||||
environment:
|
||||
ftr:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
stg:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
int:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
prd:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
@@ -1,169 +0,0 @@
|
||||
<% if (deploymentStrategy == 'canary'){ print "canary:\n enabled: true\n slackChannel: ${canary.slackChannel}\n enableManualPromotion: ${enableManualPromotion}\n skipAnalysis: ${canary.skipAnalysis}\n service:\n port: 80\n targetPort: ${primary_port}\n progressDeadlineSeconds: ${canary.progressDeadlineSeconds}\n minCanaryReplicas: ${minCanaryReplicas}\n maxCanaryReplicas: ${maxCanaryReplicas}\n analysisInterval: ${canary.analysisInterval}\n analysisThreshold: ${canary.analysisThreshold}\n analysisMaxWeight: ${canary.analysisMaxWeight}\n analysisStepWeight: ${canary.analysisStepWeight}\n analysisMetrics:\n thresholdRangeMin: ${canary.analysisMetrics.thresholdRangeMin}\n interval: ${canary.analysisMetrics.interval}\n" } else { print "canary:\n enabled: false\n" } %>repoName: ${repo_name}
|
||||
cron:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
applicationName: ${app_name}
|
||||
appType: php
|
||||
autoscaling:
|
||||
enabled: ${as_enabled}
|
||||
maxReplicas: ${as_max}
|
||||
minReplicas: ${as_min}
|
||||
pollingInterval: ${as_poll}
|
||||
scaledown:
|
||||
policies:
|
||||
- periodseconds: ${as_down_period}
|
||||
type: Pods
|
||||
value: ${as_down_pod_count}
|
||||
selectpolicy: Min
|
||||
stabilizationWindowSeconds: 300
|
||||
scaleup:
|
||||
policies:
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Pods
|
||||
value: ${as_up_pod_count}
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Percent
|
||||
value: ${as_up_pod_percentage}
|
||||
selectpolicy: Max
|
||||
stabilizationWindowSeconds: ${as_up_stable_window}
|
||||
<% if(!triggers){print " triggers:\n - metadata:\n value: \"${as_trigger_value}\"\n metricType: ${as_trigger_type}\n type: ${as_trigger_metric}"} else {println ' triggers:'; for(val in triggers){if(val instanceof Map){ val.each{k,v -> if (v instanceof Map) { println ' - '+k+':'; v.each{a,b -> if(b.isNumber()){println " ${a}: \"${b}\"" } else {println " ${a}: ${b}" }}} else {println" ${k}: ${v}"}}}}} %>
|
||||
deployment:
|
||||
affinity: {}
|
||||
podDistributionSkew: ${podDistributionSkew}
|
||||
command:
|
||||
- apache2-foreground
|
||||
enabled: true
|
||||
env: null
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
<% if(lifecycle) { println ' lifecycle:\n preStop:\n exec:\n command:'; for(val in lifecycle.preStop.exec.command){ println " - $val" }} else {print " lifecycle:\n preStop:\n exec:\n command:\n - /bin/bash\n - -c\n - kill -SIGQUIT 1 ; /bin/sleep 120\n "} %>
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
<% if (nodeSelector.contains("arm64") && (environment=="int" || environment=="prd")) { print 'telegraf.influxdata.com/image: 847438129436.dkr.ecr.ap-southeast-1.amazonaws.com/telegraf:1.24.4-arm64' } %>
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
replicaCount: ${replica_count}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
updateStrategy:
|
||||
strategy:
|
||||
<% if (deploymentStrategy == 'recreate') { print " type: Recreate"} else { print " type: RollingUpdate\n rollingUpdate:\n maxUnavailable: 0%\n maxSurge: ${maxSurge}%" } %>
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
fullnameOverride: ""
|
||||
replicaCount: ${replica_count}
|
||||
ingress:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "false"
|
||||
<% if (ingress_annotations) {ingress_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
<% if(grpc_host){println " grpc_hosts:\n - host: ${grpc_host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else if (grpc_hosts) {println " grpc_hosts:";for(host_arr in grpc_hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %>
|
||||
<% if(host){println " hosts:\n - host: ${host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else {println " hosts:";for(host_arr in hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %> ingressClassName: ${ingress_class}
|
||||
servicePort: http
|
||||
enableWebsocket: ${enableWebsocket}
|
||||
slowStart:
|
||||
enabled: <% if (slowStartWindow) { print "true" } else { print "false" } %>
|
||||
window: <% if (slowStartWindow) { print "${slowStartWindow}" } else { print "120s" } %>
|
||||
aggression: <% if (slowStartAggression) { print "${slowStartAggression}" } else { print "1.0" } %>
|
||||
minPercent: <% if (slowStartMinPercent) { print "${slowStartMinPercent}" } else { print "10" } %>
|
||||
jmxconfig:
|
||||
enabled: false
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
nameOverride: ""
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podDisruptionBudget:
|
||||
enabled: <% if (pdbMaxUnavailable) { print "true" } else { print "false" } %>
|
||||
maxUnavailable: ${pdbMaxUnavailable}
|
||||
minAvailable: ${pdbMinAvailable}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
<% if (service_annotations) { println " annotations:"; service_annotations.each{k,v -> println " ${k}: \"${v}\""};} else { println " annotations: null" } %>
|
||||
enabled: true
|
||||
<% if (addon_ports) { for (p in addon_ports) { println " addons:"; println " - name: ${p.name}"; println " targetPort: ${p.targetPort}"; println " type: ${p.type}"; }} else { println " addon_ports: []" } %>
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: TCP
|
||||
targetPort: ${primary_port}
|
||||
type: ClusterIP
|
||||
|
||||
createContourGateway: <% if (createContourGateway) { print "${createContourGateway}" } else { print "false" } %>
|
||||
contourResponseTimeout: ${contourResponseTimeout}
|
||||
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
@@ -1,29 +0,0 @@
|
||||
FROM ${buildRegistry}/build/python:2.7-${arch}-0.8.2 as build-system
|
||||
|
||||
COPY ${modules_requirements_file} /app/${modules_requirements_file}
|
||||
|
||||
FROM build-system as intermediate
|
||||
# add credentials on build
|
||||
COPY id_github_jenkins /root/.ssh/id_rsa
|
||||
|
||||
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts && \
|
||||
chmod -R 600 /root/.ssh/
|
||||
|
||||
# temp-package python dependencies
|
||||
RUN pip download -r /app/requirements.txt -d /temp-package/python
|
||||
|
||||
### create the runtime image ###
|
||||
FROM build-system as runtime
|
||||
|
||||
# install temp-packageed python dependencies
|
||||
COPY --from=intermediate /temp-package/python /temp-package/python
|
||||
RUN pip install /temp-package/python/* && \
|
||||
rm -rf /temp-package
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY ./ /app/
|
||||
<% if (copy_file) { print "COPY copied_files/* $copy_target"} %>
|
||||
<% add_files.each { print "COPY ${it.path} ${it.target}\n" } %>
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
@@ -1,31 +0,0 @@
|
||||
FROM ${buildRegistry}/build/python:3.10.12-${arch} as build-system
|
||||
|
||||
COPY ${modules_requirements_file} /app/${modules_requirements_file}
|
||||
|
||||
FROM build-system as intermediate
|
||||
# add credentials on build
|
||||
COPY id_github_jenkins /root/.ssh/id_rsa
|
||||
|
||||
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts && \
|
||||
chmod -R 600 /root/.ssh/
|
||||
|
||||
# temp-package python dependencies
|
||||
RUN pip download -r /app/requirements.txt -d /temp-package/python
|
||||
|
||||
### create the runtime image ###
|
||||
FROM build-system as runtime
|
||||
|
||||
# install temp-packageed python dependencies
|
||||
COPY --from=intermediate /temp-package/python /temp-package/python
|
||||
RUN pip install /temp-package/python/* && \
|
||||
rm -rf /temp-package && \
|
||||
pip install uwsgi && \
|
||||
python3 --version
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY ./ /app/
|
||||
<% if (copy_file) { print "COPY copied_files/* $copy_target"} %>
|
||||
<% add_files.each { print "COPY ${it.path} ${it.target}\n" } %>
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
@@ -1,31 +0,0 @@
|
||||
FROM ${buildRegistry}/build/python:3.13-${arch} as build-system
|
||||
|
||||
COPY ${modules_requirements_file} /app/${modules_requirements_file}
|
||||
|
||||
FROM build-system as intermediate
|
||||
# add credentials on build
|
||||
COPY id_github_jenkins /root/.ssh/id_rsa
|
||||
|
||||
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts && \
|
||||
chmod -R 600 /root/.ssh/
|
||||
|
||||
# temp-package python dependencies
|
||||
RUN pip download -r /app/requirements.txt -d /temp-package/python
|
||||
|
||||
### create the runtime image ###
|
||||
FROM build-system as runtime
|
||||
|
||||
# install temp-packageed python dependencies
|
||||
COPY --from=intermediate /temp-package/python /temp-package/python
|
||||
RUN pip install /temp-package/python/* && \
|
||||
rm -rf /temp-package && \
|
||||
pip install uwsgi && \
|
||||
python3 --version
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY ./ /app/
|
||||
<% if (copy_file) { print "COPY copied_files/* $copy_target"} %>
|
||||
<% add_files.each { print "COPY ${it.path} ${it.target}\n" } %>
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
@@ -1,29 +0,0 @@
|
||||
FROM ${buildRegistry}/build/python:3.7-${arch}-0.8.2 as build-system
|
||||
|
||||
COPY ${modules_requirements_file} /app/${modules_requirements_file}
|
||||
|
||||
FROM build-system as intermediate
|
||||
# add credentials on build
|
||||
COPY id_github_jenkins /root/.ssh/id_rsa
|
||||
|
||||
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts && \
|
||||
chmod -R 600 /root/.ssh/
|
||||
|
||||
# temp-package python dependencies
|
||||
RUN pip download -r /app/requirements.txt -d /temp-package/python
|
||||
|
||||
### create the runtime image ###
|
||||
FROM build-system as runtime
|
||||
|
||||
# install temp-packageed python dependencies
|
||||
COPY --from=intermediate /temp-package/python /temp-package/python
|
||||
RUN pip install /temp-package/python/* && \
|
||||
rm -rf /temp-package
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY ./ /app/
|
||||
<% if (copy_file) { print "COPY copied_files/* $copy_target"} %>
|
||||
<% add_files.each { print "COPY ${it.path} ${it.target}\n" } %>
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
@@ -1,9 +1,15 @@
|
||||
# Fallback only — see go-Dockerfile's header comment for the general
|
||||
# rule. Simplified from the real python-*-Dockerfile set (which had four
|
||||
# rule, including the Harbor base-images sourcing (also applies here).
|
||||
# Simplified from the real python-*-Dockerfile set (which had four
|
||||
# separate version-pinned files, 2.7/3.7/3.10.12/3.13) into one
|
||||
# version-parametrized template. Assumes a standard requirements.txt +
|
||||
# app.py (Flask/FastAPI-style `app:app` target for gunicorn) repo.
|
||||
FROM python:${version}-slim
|
||||
# Switched from python:*-slim (Debian) to python:*-alpine — smaller,
|
||||
# still keeps a shell (not distroless). Caveat: pip packages with C
|
||||
# extensions that only ship glibc wheels may need musl-dev/gcc added
|
||||
# here to build from source on Alpine — fine for this repo's pure-Python
|
||||
# deps, worth knowing if a future repo's requirements.txt needs more.
|
||||
FROM harbor.infra.deployshed.com/base-images/python:${version}-alpine
|
||||
WORKDIR /app
|
||||
COPY requirements.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
@@ -1,64 +0,0 @@
|
||||
---
|
||||
|
||||
app_name: ${app_name}
|
||||
app_port: ${app_port}
|
||||
health_check: ${health_check}
|
||||
module: module_less
|
||||
bu: ${bu}
|
||||
team: ${team}
|
||||
priority: ${priority}
|
||||
priority_v2: ${priority_v2}
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
replica_count: ${replica_count}
|
||||
environment:
|
||||
ftr:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
supervisord_config_path: /app/configurations/supervisord/supervisord.conf
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
stg:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
supervisord_config_path: /app/configurations/supervisord/supervisord.conf
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
int:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
supervisord_config_path: /app/configurations/supervisord/supervisord.conf
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
prd:
|
||||
as_enabled: ${as_enabled}
|
||||
as_min: ${as_min}
|
||||
as_max: ${as_max}
|
||||
supervisord_config_path: /app/configurations/supervisord/supervisord.conf
|
||||
cpu_limit: ${cpu_limit}
|
||||
cpu_request: ${cpu_request}
|
||||
memory_limit: ${memory_limit}
|
||||
memory_request: ${memory_request}
|
||||
deploymentStrategy: ${deploymentStrategy}
|
||||
appMetrics: ${appMetrics}
|
||||
deployment_args:
|
||||
|
||||
@@ -1,242 +0,0 @@
|
||||
<% if (deploymentStrategy == 'canary'){ print "canary:\n enabled: true\n slackChannel: ${canary.slackChannel}\n enableManualPromotion: ${enableManualPromotion}\n skipAnalysis: ${canary.skipAnalysis}\n service:\n port: 80\n targetPort: ${primary_port}\n progressDeadlineSeconds: ${canary.progressDeadlineSeconds}\n minCanaryReplicas: ${minCanaryReplicas}\n maxCanaryReplicas: ${maxCanaryReplicas}\n analysisInterval: ${canary.analysisInterval}\n analysisThreshold: ${canary.analysisThreshold}\n analysisMaxWeight: ${canary.analysisMaxWeight}\n analysisStepWeight: ${canary.analysisStepWeight}\n analysisMetrics:\n thresholdRangeMin: ${canary.analysisMetrics.thresholdRangeMin}\n interval: ${canary.analysisMetrics.interval}\n" } else { print "canary:\n enabled: false\n" } %>repoName: ${repo_name}
|
||||
cron:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
applicationName: ${app_name}
|
||||
appType: ${dockerBuildVersion}
|
||||
autoscaling:
|
||||
enabled: ${as_enabled}
|
||||
maxReplicas: ${as_max}
|
||||
minReplicas: ${as_min}
|
||||
pollingInterval: ${as_poll}
|
||||
scaledown:
|
||||
policies:
|
||||
- periodseconds: ${as_down_period}
|
||||
type: Pods
|
||||
value: ${as_down_pod_count}
|
||||
selectpolicy: Min
|
||||
stabilizationWindowSeconds: ${as_down_stable_window}
|
||||
scaleup:
|
||||
policies:
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Pods
|
||||
value: ${as_up_pod_count}
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Percent
|
||||
value: ${as_up_pod_percentage}
|
||||
selectpolicy: Max
|
||||
stabilizationWindowSeconds: ${as_up_stable_window}
|
||||
<% if(!triggers){print " triggers:\n - metadata:\n value: \"${as_trigger_value}\"\n metricType: ${as_trigger_type}\n type: ${as_trigger_metric}"} else {println ' triggers:'; for(val in triggers){if(val instanceof Map){ val.each{k,v -> if (v instanceof Map) { println ' - '+k+':'; v.each{a,b -> if(b.isNumber()){println " ${a}: \"${b}\"" } else {println " ${a}: ${b}" }}} else {println" ${k}: ${v}"}}}}} %>
|
||||
deployment:
|
||||
enabled: <% if (kind == "deployment" || kind == "Deployment") { print "true" } else { print "false" } %>
|
||||
updateStrategy:
|
||||
strategy:
|
||||
rollingUpdate:
|
||||
maxUnavailable: 0%
|
||||
maxSurge: <% print "${maxSurge}%" %>
|
||||
type: RollingUpdate
|
||||
affinity: {}
|
||||
podDistributionSkew: ${podDistributionSkew}
|
||||
env:
|
||||
- name: SUPERVISORD_CONFIG_FILE
|
||||
value: ${supervisord_config_path}
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
<% if(lifecycle) { println ' lifecycle:\n preStop:\n exec:\n command:'; for(val in lifecycle.preStop.exec.command){ println " - $val" }} else {print " lifecycle:\n preStop:\n exec:\n command:\n - /bin/bash\n - -c\n - kill -SIGQUIT 1 ; /bin/sleep 60"} %>
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
- containerPort: 9901
|
||||
name: metric
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
replicaCount: ${replica_count}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
fullnameOverride: ""
|
||||
replicaCount: ${replica_count}
|
||||
ingress:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/rewrite-target: /
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "false"
|
||||
enabled: true
|
||||
<% if(grpc_host){println " grpc_hosts:\n - host: ${grpc_host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else if (grpc_hosts) {println " grpc_hosts:";for(host_arr in grpc_hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %>
|
||||
<% if(host){println " hosts:\n - host: ${host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else {println " hosts:";for(host_arr in hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %> ingressClassName: ${ingress_class}
|
||||
servicePort: http
|
||||
enableWebsocket: ${enableWebsocket}
|
||||
slowStart:
|
||||
enabled: <% if (slowStartWindow) { print "true" } else { print "false" } %>
|
||||
window: <% if (slowStartWindow) { print "${slowStartWindow}" } else { print "120s" } %>
|
||||
aggression: <% if (slowStartAggression) { print "${slowStartAggression}" } else { print "1.0" } %>
|
||||
minPercent: <% if (slowStartMinPercent) { print "${slowStartMinPercent}" } else { print "10" } %>
|
||||
jmxconfig:
|
||||
enabled: true
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
nameOverride: ""
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podDisruptionBudget:
|
||||
enabled: false
|
||||
maxUnavailable: 100%
|
||||
minAvailable: ${pdbMinAvailable}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
<% if (service_annotations) { println " annotations:"; service_annotations.each{k,v -> println " ${k}: \"${v}\""};} else { println " annotations: null" } %>
|
||||
enabled: true
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: TCP
|
||||
targetPort: ${app_port}
|
||||
type: ClusterIP
|
||||
statefulset:
|
||||
enabled: <% if (kind == "statefulset" || kind == "StatefulSet" || kind == "statefulSet") { print "true" } else { print "false" } %>
|
||||
updateStrategy: <% if (statefulset.updateStrategy == "RollingUpdate" || statefulset.updateStrategy == "rollingUpdate" || statefulset.updateStrategy == "rollingupdate") { print "RollingUpdate" } else { print "null" } %>
|
||||
volumeType: ${statefulset.volumeType}
|
||||
<% if(statefulset.volumeType == "static" || statefulset.volumeType == "Static") { println " staticVolume:"; statefulset.staticVolume.each{k,v -> println " ${k}: ${v}"};} else {print " staticVolume:\n accessMode: ReadWriteMany\n mountPath: /opt/data\n size: 5Gi\n storageClass: homelab-gp2\n volumeHandle: fs-0e2a97a38b01857d1::fsap-07b146e3bc7aef280\n csiDriver: efs.csi.aws.com"} %>
|
||||
<% if(statefulset.volumeType == "dynamic" || statefulset.volumeType == "Dynamic") { println " dynamicVolume:"; statefulset.dynamicVolume.each{k,v -> println " ${k}: ${v}"};} else {print " dynamicVolume:\n accessMode: ReadWriteMany\n mountPath: /opt/data\n size: 5Gi\n storageClass: homelab-gp2"} %>
|
||||
createContourGateway: <% if (createContourGateway) { print "${createContourGateway}" } else { print "false" } %>
|
||||
contourResponseTimeout: ${contourResponseTimeout}
|
||||
podtemplate:
|
||||
affinity: {}
|
||||
env:
|
||||
- name: SUPERVISORD_CONFIG_FILE
|
||||
value: ${supervisord_config_path}
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
<% if(lifecycle) { println ' lifecycle:\n preStop:\n exec:\n command:'; for(val in lifecycle.preStop.exec.command){ println " - $val" }} else {print " lifecycle:\n preStop:\n exec:\n command:\n - /bin/bash\n - -c\n - kill -SIGQUIT 1 ; /bin/sleep 60"} %>
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
- containerPort: 9901
|
||||
name: metric
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
replicaCount: ${replica_count}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
serviceAccount:
|
||||
annotations: null
|
||||
enabled: false
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
@@ -1,58 +0,0 @@
|
||||
FROM ${buildRegistry}/build/rust:${version} AS builder
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
protobuf-compiler \
|
||||
cmake \
|
||||
libprotobuf-dev \
|
||||
libssl-dev \
|
||||
pkg-config \
|
||||
openssh-client \
|
||||
ca-certificates \
|
||||
libsasl2-2 \
|
||||
libsasl2-dev \
|
||||
clang \
|
||||
libclang-dev \
|
||||
<% build_packages.each { print " ${it} \\\n" } %>
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /usr/src/app
|
||||
|
||||
|
||||
COPY id_github_jenkins /root/.ssh/id_rsa
|
||||
|
||||
RUN ssh-keyscan github.com >> /root/.ssh/known_hosts && \
|
||||
chmod -R 600 /root/.ssh/ && \
|
||||
git config --global url."git@github.com:".insteadOf "https://github.com/"
|
||||
|
||||
# Copy source code
|
||||
COPY . .
|
||||
|
||||
# Build release binary for the native architecture (amd64)
|
||||
RUN cargo build --workspace --release
|
||||
|
||||
# Clean up SSH key from builder stage for security
|
||||
RUN rm -rf /root/.ssh/
|
||||
|
||||
FROM asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/build/debian:trixie-slim
|
||||
|
||||
# Install runtime dependencies if your app needs them (e.g., SSL)
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
libssl3t64 \
|
||||
ca-certificates \
|
||||
libsasl2-2 \
|
||||
<% runtime_packages.each { print " ${it} \\\n" } %>
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Set workdir for the binary
|
||||
WORKDIR /app
|
||||
|
||||
# Copy the compiled amd64 binary from the builder
|
||||
COPY --from=builder /usr/src/app/target/release/${binary_name} ./server
|
||||
<% add_files.each { print "COPY ${it.path} ${it.target}\n" } %>
|
||||
|
||||
# Expose port if your app listens on it
|
||||
EXPOSE 8080
|
||||
|
||||
# Start the application
|
||||
CMD ["./server"]
|
||||
@@ -1,206 +0,0 @@
|
||||
<% if (deploymentStrategy == 'canary'){ print "canary:\n enabled: true\n slackChannel: ${canary.slackChannel}\n enableManualPromotion: ${enableManualPromotion}\n skipAnalysis: ${canary.skipAnalysis}\n service:\n port: 80\n targetPort: ${(grpc_host || grpc_hosts) ? app_port : primary_port}\n progressDeadlineSeconds: ${canary.progressDeadlineSeconds}\n minCanaryReplicas: ${minCanaryReplicas}\n maxCanaryReplicas: ${maxCanaryReplicas}\n analysisInterval: ${canary.analysisInterval}\n analysisThreshold: ${canary.analysisThreshold}\n analysisMaxWeight: ${canary.analysisMaxWeight}\n analysisStepWeight: ${canary.analysisStepWeight}\n analysisMetrics:\n thresholdRangeMin: ${canary.analysisMetrics.thresholdRangeMin}\n interval: ${canary.analysisMetrics.interval}\n" } else { print "canary:\n enabled: false\n" } %>
|
||||
repoName: ${repo_name}
|
||||
cron:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
applicationName: ${app_name}
|
||||
autoscaling:
|
||||
enabled: ${as_enabled}
|
||||
maxReplicas: ${as_max}
|
||||
minReplicas: ${as_min}
|
||||
pollingInterval: ${as_poll}
|
||||
scaledown:
|
||||
policies:
|
||||
- periodseconds: ${as_down_period}
|
||||
type: Pods
|
||||
value: ${as_down_pod_count}
|
||||
selectpolicy: Min
|
||||
stabilizationWindowSeconds: ${as_down_stable_window}
|
||||
scaleup:
|
||||
policies:
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Pods
|
||||
value: ${as_up_pod_count}
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Percent
|
||||
value: ${as_up_pod_percentage}
|
||||
selectpolicy: Max
|
||||
stabilizationWindowSeconds: ${as_up_stable_window}
|
||||
<% if(!triggers){print " triggers:\n - metadata:\n value: \"${as_trigger_value}\"\n metricType: ${as_trigger_type}\n type: ${as_trigger_metric}"} else {println ' triggers:'; for(val in triggers){if(val instanceof Map){ val.each{k,v -> if (v instanceof Map) { println ' - '+k+':'; v.each{a,b -> if(b.isNumber()){println " ${a}: \"${b}\"" } else {println " ${a}: ${b}" }}} else {println" ${k}: ${v}"}}}}} %>
|
||||
replicaCount: ${replica_count}
|
||||
deployment:
|
||||
affinity: {}
|
||||
podDistributionSkew: ${podDistributionSkew}
|
||||
<% if (deployment_args){println ' args:'; for (arg in deployment_args){ println ' - '+arg}} %>
|
||||
command:
|
||||
- ${command}
|
||||
enabled: true
|
||||
env:
|
||||
- name: GOMAXPROCS
|
||||
value: ${activeProcessorCount}
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
<% if(lifecycle) { println ' lifecycle:\n preStop:\n exec:\n command:'; for(val in lifecycle.preStop.exec.command){ println " - $val" }} else {print " lifecycle:\n preStop:\n exec:\n command:\n - /bin/bash\n - -c\n - kill -SIGQUIT 1 ; /bin/sleep 120\n "} %>
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (nodeSelector.contains("arm64") && (environment=="int" || environment=="prd")) { print 'telegraf.influxdata.com/image: 847438129436.dkr.ecr.ap-southeast-1.amazonaws.com/telegraf:1.24.4-arm64' } %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> println " ${k}: '${v}'"};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
- containerPort: 8880
|
||||
name: metric
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
# - effect: NoSchedule
|
||||
# key: kubernetes.io/arch
|
||||
# operator: Equal
|
||||
# value: arm64
|
||||
updateStrategy:
|
||||
strategy:
|
||||
<% if (deploymentStrategy == 'recreate') { print " type: Recreate"} else { print " type: RollingUpdate\n rollingUpdate:\n maxUnavailable: 0%\n maxSurge: ${maxSurge}%" } %>
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
fullnameOverride: ""
|
||||
ingress:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/use-regex: "true"
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/server-snippet: |
|
||||
location ~* "^/api/1.0/search/recent" {
|
||||
default_type application/json;
|
||||
return 200 '{"recent_searches": [],"recent_suggestions": [],"limit": 5,"autosuggest_use_recent": true,"show_recent_header": false}';
|
||||
}
|
||||
location ~* "^/api/1.0/anonymous/search/recent" {
|
||||
default_type application/json;
|
||||
return 200 '{"recent_searches": [],"recent_suggestions": [],"limit": 5,"autosuggest_use_recent": true,"show_recent_header": false}';
|
||||
}
|
||||
location ~* "^/search-queries/recent" {
|
||||
default_type application/json;
|
||||
return 200 '{"recent_searches": [],"recent_suggestions": [],"limit": 5,"autosuggest_use_recent": true,"show_recent_header": false}';
|
||||
}
|
||||
<% if (ingress_annotations) {ingress_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
<% if(grpc_host){println " grpc_hosts:\n - host: ${grpc_host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else if (grpc_hosts) {println " grpc_hosts:";for(host_arr in grpc_hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %>
|
||||
<% if(host){println " hosts:\n - host: ${host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else {println " hosts:";for(host_arr in hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %> ingressClassName: ${ingress_class}
|
||||
servicePort: http
|
||||
enableWebsocket: ${enableWebsocket}
|
||||
slowStart:
|
||||
enabled: <% if (slowStartWindow) { print "true" } else { print "false" } %>
|
||||
window: <% if (slowStartWindow) { print "${slowStartWindow}" } else { print "120s" } %>
|
||||
aggression: <% if (slowStartAggression) { print "${slowStartAggression}" } else { print "1.0" } %>
|
||||
minPercent: <% if (slowStartMinPercent) { print "${slowStartMinPercent}" } else { print "10" } %>
|
||||
jmxconfig:
|
||||
enabled: false
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
nameOverride: ""
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podDisruptionBudget:
|
||||
enabled: <% if (pdbMaxUnavailable) { print "true" } else { print "false" } %>
|
||||
maxUnavailable: ${pdbMaxUnavailable}
|
||||
minAvailable: ${pdbMinAvailable}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
<% if (service_annotations) { println " annotations:"; service_annotations.each{k,v -> println " ${k}: \"${v}\""};} else { println " annotations: null" } %>
|
||||
enabled: true
|
||||
<% if (addon_ports) { for (p in addon_ports) { println " addons:"; println " - name: ${p.name}"; println " targetPort: ${p.targetPort}"; println " type: ${p.type}"; }} else { println " addon_ports: []" } %>
|
||||
<% if (grpc_port && app_port && (grpc_host || grpc_hosts)) {
|
||||
println " ports:"
|
||||
println " - name: http"
|
||||
println " port: 80"
|
||||
println " protocol: TCP"
|
||||
println " targetPort: ${app_port}"
|
||||
println " grpc_ports:"
|
||||
println " - name: grpc"
|
||||
println " port: 80"
|
||||
println " protocol: TCP"
|
||||
println " targetPort: ${grpc_port}"
|
||||
} else {
|
||||
println " ports:"
|
||||
println " - name: http"
|
||||
println " port: 80"
|
||||
println " protocol: TCP"
|
||||
println " targetPort: ${primary_port}"
|
||||
}
|
||||
%>
|
||||
type: ClusterIP
|
||||
|
||||
createContourGateway: <% if (createContourGateway) { print "${createContourGateway}" } else { print "false" } %>
|
||||
contourResponseTimeout: ${contourResponseTimeout}
|
||||
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
@@ -1,202 +0,0 @@
|
||||
<% if (deploymentStrategy == 'canary'){ print "canary:\n enabled: true\n slackChannel: ${canary.slackChannel}\n enableManualPromotion: ${enableManualPromotion}\n skipAnalysis: ${canary.skipAnalysis}\n service:\n port: 80\n targetPort: ${primary_port}\n" ; print " progressDeadlineSeconds: ${canary.progressDeadlineSeconds}\n minCanaryReplicas: ${minCanaryReplicas}\n maxCanaryReplicas: ${maxCanaryReplicas}\n analysisInterval: ${canary.analysisInterval}\n analysisThreshold: ${canary.analysisThreshold}\n analysisMaxWeight: ${canary.analysisMaxWeight}\n analysisStepWeight: ${canary.analysisStepWeight}\n analysisMetrics:\n thresholdRangeMin: ${canary.analysisMetrics.thresholdRangeMin}\n interval: ${canary.analysisMetrics.interval}\n" } else { print "canary:\n enabled: false\n" } %>repoName: ${repo_name}
|
||||
cron:
|
||||
enabled: false
|
||||
serviceAccount:
|
||||
enabled: false
|
||||
applicationName: ${app_name}
|
||||
autoscaling:
|
||||
enabled: ${as_enabled}
|
||||
maxReplicas: ${as_max}
|
||||
minReplicas: ${as_min}
|
||||
pollingInterval: ${as_poll}
|
||||
scaledown:
|
||||
policies:
|
||||
- periodseconds: ${as_down_period}
|
||||
type: Pods
|
||||
value: ${as_down_pod_count}
|
||||
selectpolicy: Min
|
||||
stabilizationWindowSeconds: ${as_down_stable_window}
|
||||
scaleup:
|
||||
policies:
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Pods
|
||||
value: ${as_up_pod_count}
|
||||
- periodseconds: ${as_up_period}
|
||||
type: Percent
|
||||
value: ${as_up_pod_percentage}
|
||||
selectpolicy: Max
|
||||
stabilizationWindowSeconds: ${as_up_stable_window}
|
||||
<% if(!triggers){print " triggers:\n - metadata:\n value: \"${as_trigger_value}\"\n metricType: ${as_trigger_type}\n type: ${as_trigger_metric}"} else {println ' triggers:'; for(val in triggers){if(val instanceof Map){ val.each{k,v -> if (v instanceof Map) { println ' - '+k+':'; v.each{a,b -> if(b.isNumber()){println " ${a}: \"${b}\"" } else {println " ${a}: ${b}" }}} else {println" ${k}: ${v}"}}}}} %>
|
||||
replicaCount: ${replica_count}
|
||||
otel_enabled: ${otel_enabled}
|
||||
metrics_mode: ${metrics_mode}
|
||||
deployment:
|
||||
affinity: {}
|
||||
podDistributionSkew: ${podDistributionSkew}
|
||||
args:
|
||||
<% for (arg in deployment_args){ if (arg.contains("Xms") || arg.contains("Xmx")) {println ''} else {println ' - '+arg}} %>
|
||||
<% if (appConfigEnabled) { %>- -Dspring.profiles.active=${environment},${app_name},dyn-${environment} <% } %>
|
||||
<% if (appConfigEnabled) { %>- -Dspring.config.additional-location=/opt/config/application-${environment}.yml,/opt/config/application-dyn-${environment}.yml<% } %>
|
||||
- -Xms${xms}
|
||||
- -Xmx${xmx}
|
||||
- -XX:ActiveProcessorCount=${activeProcessorCount}
|
||||
- -javaagent:/opt/jmx_exporter.jar=8880:/jmx/jmx-config.yaml
|
||||
<% if (telegraf_metrics) { %>- -Dtelegraf-metrics-enabled=true<% } else { %>- -Dtelegraf-metrics-enabled=false<% } %>
|
||||
<% if (otel_enabled) { %>
|
||||
- -javaagent:/opt/opentelemetry-javaagent.jar
|
||||
- -Dotel.exporter.otlp.protocol=grpc
|
||||
- -Dotel.logs.exporter=none
|
||||
- -Dotel.metrics.exporter=none
|
||||
- -Dotel.traces.exporter=otlp
|
||||
- -Dotel.traces.sampler=traceidratio
|
||||
<% if (environment != "stg") { %>
|
||||
- -Dotel.traces.sampler.arg=${otel_traces_sampler_arg}
|
||||
<% } %>
|
||||
- -Dotel.resource.attributes=service.name=${app_name},application.name=${app_name},api.name=${app_name},cx.application.name=${app_name},cx.subsystem.name=${app_name}
|
||||
<% } %>
|
||||
- -jar
|
||||
- <% print module=='module_less'?repo_name:module %>.jar
|
||||
command:
|
||||
- java
|
||||
enabled: true
|
||||
env:
|
||||
- name: PRISMSDK_ENVIRONMENT
|
||||
value: ${prismsdk_environment}
|
||||
envFrom:
|
||||
secretRef: ${app_name}
|
||||
image:
|
||||
pullPolicy: IfNotPresent
|
||||
pullSecret: ""
|
||||
repository: ${registry}/${environment}/${build_team}/<% print module=='module_less'?repo_name.toLowerCase():repo_name.toLowerCase()+'/'+module %>
|
||||
tag: ${tag}
|
||||
<% if(lifecycle) { println ' lifecycle:\n preStop:\n exec:\n command:'; for(val in lifecycle.preStop.exec.command){ println " - $val" }} else {print " lifecycle:\n preStop:\n exec:\n command:\n - /bin/bash\n - -c\n - kill -SIGQUIT 1 ; /bin/sleep 120\n "} %>
|
||||
minReadySeconds: 10
|
||||
podAnnotations:
|
||||
jmx.io/path: /metrics
|
||||
jmx.io/port: "8880"
|
||||
jmx.io/scrape: "true"
|
||||
<% if (nodeSelector.contains("arm64") && (environment=="int" || environment=="prd")) { print 'telegraf.influxdata.com/image: 847438129436.dkr.ecr.ap-southeast-1.amazonaws.com/telegraf:1.24.4-arm64' } %>
|
||||
<% if (appMetrics) { print 'prometheus.io/path: /actuator/prometheus' %>
|
||||
<% print 'prometheus.io/port: "'+app_port+'"' %>
|
||||
<% print 'prometheus.io/scrape: "true"'} %>
|
||||
<% if (pod_annotations) {pod_annotations.each{k,v -> if(v instanceof String) { println " ${k}: '${v}'"} else { println " ${k}: ${v}" }};} else {print ''} %>
|
||||
ports:
|
||||
- containerPort: ${app_port}
|
||||
name: http
|
||||
protocol: TCP
|
||||
- containerPort: 8880
|
||||
name: metric
|
||||
protocol: TCP
|
||||
probes:
|
||||
liveness:
|
||||
failureThreshold: ${liveness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${liveness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${liveness_success_threshold}
|
||||
timeoutSeconds: ${liveness_timeout_seconds}
|
||||
readiness:
|
||||
failureThreshold: ${readiness_failure_threshold}
|
||||
initialDelaySeconds: ${initialDelaySeconds}
|
||||
path: ${health_check}
|
||||
periodSeconds: ${readiness_period_seconds}
|
||||
port: http
|
||||
scheme: HTTP
|
||||
successThreshold: ${readiness_success_threshold}
|
||||
timeoutSeconds: ${readiness_timeout_seconds}
|
||||
resources:
|
||||
limits:
|
||||
cpu: ${cpu_limit}
|
||||
memory: ${memory_limit}i
|
||||
requests:
|
||||
cpu: ${cpu_request}
|
||||
memory: ${memory_request}i
|
||||
revisionHistoryLimit: 6
|
||||
<% if(serviceAccount){println ' serviceAccount:\n annotations:';serviceAccount.annotations.each{k,v -> println " ${k}: ${v}"};println " enabled: ${serviceAccount.enabled}"} else {print ' serviceAccount:\n annotations: null\n enabled: false'} %>
|
||||
nodeSelector:
|
||||
${nodeSelector}: ${nodeSelectorValue}
|
||||
<% if (hostAliases){println ' hostAliases:';for(arr in hostAliases){println ' - ip: '+arr.get("ip"); println ' hostnames:'; for (ele in arr.get("hostnames")) { println ' - '+ele } }} %>
|
||||
tolerations:
|
||||
- effect: NoSchedule
|
||||
key: ${nodeSelector}
|
||||
operator: Equal
|
||||
value: ${nodeSelectorValue}
|
||||
updateStrategy:
|
||||
strategy:
|
||||
<% if (deploymentStrategy == 'recreate') { print " type: Recreate"} else { print " type: RollingUpdate\n rollingUpdate:\n maxUnavailable: 0%\n maxSurge: ${maxSurge}%" } %>
|
||||
externalSecret:
|
||||
annotations:
|
||||
<% if (external_secrets_annotations) {external_secrets_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
path: homelab/${vault_env}/${bu}/${team}/${app_name}
|
||||
version: ${tag}
|
||||
name: ${env_ns}-${app_name}
|
||||
target: ${app_name}
|
||||
fullnameOverride: ""
|
||||
ingress:
|
||||
annotations:
|
||||
nginx.ingress.kubernetes.io/force-ssl-redirect: "false"
|
||||
nginx.ingress.kubernetes.io/ssl-redirect: "false"
|
||||
<% if (ingress_annotations) {ingress_annotations.each{k,v -> println " ${k}: ${v}"};} else {print ''} %>
|
||||
enabled: true
|
||||
<% if(grpc_host){println " grpc_hosts:\n - host: ${grpc_host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else if (grpc_hosts) {println " grpc_hosts:";for(host_arr in grpc_hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %>
|
||||
<% if(host){println " hosts:\n - host: ${host}\n paths:\n - pathType: ImplementationSpecific\n path: /"} else {println " hosts:";for(host_arr in hosts){println " - host: ${host_arr.host}\n paths:";for(path_arr in host_arr.paths){println " - pathType: ${path_arr.pathType}\n path: ${path_arr.path}"; if(path_arr.targetService) { println " targetService: ${path_arr.targetService}" } }}} %> ingressClassName: ${ingress_class}
|
||||
servicePort: http
|
||||
enableWebsocket: ${enableWebsocket}
|
||||
slowStart:
|
||||
enabled: <% if (slowStartWindow) { print "true" } else { print "false" } %>
|
||||
window: <% if (slowStartWindow) { print "${slowStartWindow}" } else { print "120s" } %>
|
||||
aggression: <% if (slowStartAggression) { print "${slowStartAggression}" } else { print "1.0" } %>
|
||||
minPercent: <% if (slowStartMinPercent) { print "${slowStartMinPercent}" } else { print "10" } %>
|
||||
jmxconfig:
|
||||
enabled: true
|
||||
labels:
|
||||
priority: <% print priority?:'p1' %>
|
||||
priority_v2: <% print priority_v2?:'cp3' %>
|
||||
primary_owner: ${primary_owner}
|
||||
secondary_owner: ${secondary_owner}
|
||||
env: ${environment_norm}
|
||||
team: ${team_norm}
|
||||
bu: ${bu_norm}
|
||||
<% if (service_type_norm) { println "service_type: ${service_type_norm}" } %>
|
||||
commit_id: ${commit_id}
|
||||
nameOverride: ""
|
||||
namespace: ${env_ns}-${app_name}
|
||||
podDisruptionBudget:
|
||||
enabled: <% if (pdbMaxUnavailable) { print "true" } else { print "false" } %>
|
||||
maxUnavailable: ${pdbMaxUnavailable}
|
||||
minAvailable: ${pdbMinAvailable}
|
||||
podSecurityContext:
|
||||
fsGroup: 65534
|
||||
runAsGroup: 65534
|
||||
runAsUser: 65534
|
||||
service:
|
||||
<% if (service_annotations) { println " annotations:"; service_annotations.each{k,v -> println " ${k}: \"${v}\""};} else { println " annotations: null" } %>
|
||||
enabled: true
|
||||
<% if (addon_ports) { for (p in addon_ports) { println " addons:"; println " - name: ${p.name}"; println " targetPort: ${p.targetPort}"; println " type: ${p.type}"; }} else { println " addon_ports: []" } %>
|
||||
ports:
|
||||
- name: http
|
||||
port: 80
|
||||
protocol: TCP
|
||||
targetPort: ${primary_port}
|
||||
type: ClusterIP
|
||||
|
||||
createContourGateway: <% if (createContourGateway) { print "${createContourGateway}" } else { print "false" } %>
|
||||
contourResponseTimeout: ${contourResponseTimeout}
|
||||
|
||||
|
||||
appConfig:
|
||||
enabled: <% if(appConfigEnabled) { print "true" } else { print "false"} %>
|
||||
env: ${environment}
|
||||
<% if(appConfigEnabled) {%>
|
||||
staticAppConfig:
|
||||
data: |
|
||||
${staticAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
dynamicAppConfig:
|
||||
data: |
|
||||
${dynamicAppConfigData.trim().replaceAll("(?m)^", " ")}
|
||||
<% }
|
||||
%>
|
||||
|
||||
@@ -1,25 +0,0 @@
|
||||
---
|
||||
|
||||
as_poll: ${as_poll}
|
||||
as_down_period: ${as_down_period}
|
||||
as_up_period: ${as_up_period}
|
||||
as_up_stable_window: ${as_up_stable_window}
|
||||
ingress_class: ${ingress_class}
|
||||
host: ${host}
|
||||
hostAliases: false
|
||||
nodeSelector: <% if(arch == 'arm64'){print environment=='dev' || environment=='ftr' ? bu+'-'+arch: environment == 'int' ? team+"-"+arch:nodeSelector } else { print environment == 'dev' || environment == 'ftr' ? bu: environment == 'int' ? bu+"-int" : nodeSelector} %>
|
||||
triggers:
|
||||
- metadata:
|
||||
value: "${as_trigger_value}"
|
||||
metricType: ${as_trigger_type}
|
||||
type: ${as_trigger_metric}
|
||||
canary:
|
||||
progressDeadlineSeconds: 300
|
||||
analysisInterval: 120s
|
||||
analysisThreshold: 5
|
||||
analysisMaxWeight: 5
|
||||
analysisStepWeight: 5
|
||||
analysisMetrics:
|
||||
thresholdRangeMin: 0.99
|
||||
interval: 1m
|
||||
skipAnalysis: false
|
||||
@@ -1,47 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
bu: "infra"
|
||||
team: "devops"
|
||||
service: "jenkins-dev"
|
||||
env: "dev"
|
||||
priority: "p0"
|
||||
type: "jenkins"
|
||||
component: "jenkins-agent"
|
||||
spec:
|
||||
serviceAccountName: jenkins-dev-agent
|
||||
containers:
|
||||
- name: devops-tools
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/build-tools:lunar-v2.0.22
|
||||
imagePullPolicy: Always
|
||||
resources:
|
||||
requests:
|
||||
memory: "6G"
|
||||
cpu: "2"
|
||||
limits:
|
||||
memory: "1000G"
|
||||
cpu: "100"
|
||||
volumeMounts:
|
||||
- mountPath: "/root"
|
||||
name: "cache"
|
||||
readOnly: false
|
||||
env:
|
||||
- name: TZ
|
||||
value: Asia/Kolkata
|
||||
- name: DOCKER_HOST
|
||||
value: dind-dev-svc
|
||||
command:
|
||||
- cat
|
||||
tty: true
|
||||
nodeSelector:
|
||||
dedicated: jenkins
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "jenkins"
|
||||
effect: "NoSchedule"
|
||||
volumes:
|
||||
- name: cache
|
||||
persistentVolumeClaim:
|
||||
claimName: pvc-jenkins-agents-cache-dev
|
||||
@@ -15,16 +15,27 @@ spec:
|
||||
image: docker:27-dind
|
||||
securityContext:
|
||||
privileged: true
|
||||
# Harbor's harbor-core Service serves plain HTTP internally (TLS is
|
||||
# disabled cluster-wide by design — see claude.md's "everything is
|
||||
# plain HTTP" note). Docker still defaults to attempting HTTPS
|
||||
# against any bare registry hostname regardless of whether the
|
||||
# network path actually involves TLS anywhere — that default isn't
|
||||
# about routing through Contour/Ingress, it's just the client's own
|
||||
# convention. Without this flag, `docker push` hangs doing a TLS
|
||||
# handshake against a server that's only ever spoken HTTP.
|
||||
args:
|
||||
- "--insecure-registry=harbor-core.harbor.svc.cluster.local"
|
||||
# No --insecure-registry and no mounted CA: Harbor is reached at
|
||||
# harbor.infra.deployshed.com, which carries a publicly trusted Let's
|
||||
# Encrypt certificate, so dockerd's own trust store already accepts it
|
||||
# with nothing configured.
|
||||
#
|
||||
# This previously mounted the private registry CA here, because the
|
||||
# registry was a nip.io name that no public CA can issue for (nip.io is
|
||||
# not on the public suffix list, and every *.nip.io certificate shares
|
||||
# one rate limit), so cert-manager signed it from a private CA instead.
|
||||
# A pull was fine — the node pool was told to trust that CA and
|
||||
# containerd performs pulls — but a push is a different client with its
|
||||
# own trust store, which is why dockerd needed the CA at
|
||||
# /etc/docker/certs.d/<registry host>/ca.crt. Moving to a real domain
|
||||
# removes the whole arrangement rather than repointing it.
|
||||
#
|
||||
# The registry is still named by its ingress hostname rather than
|
||||
# harbor-core.harbor.svc.cluster.local, for a reason that still holds:
|
||||
# cluster DNS resolves from this pod but not from the node's containerd
|
||||
# doing the real Deployment pull, and Docker matches both stored
|
||||
# credentials and trust by exact hostname — so push and pull have to
|
||||
# name the registry identically.
|
||||
env:
|
||||
- name: DOCKER_TLS_CERTDIR
|
||||
value: ""
|
||||
@@ -32,7 +43,18 @@ spec:
|
||||
- name: docker-graph-storage
|
||||
mountPath: /var/lib/docker
|
||||
- name: docker-cli
|
||||
image: docker:27-cli
|
||||
# Custom image, built and pushed by hand from devops-base-images-gcp
|
||||
# (build-tools.Dockerfile there) — bakes in git/yq/bash/python3 with
|
||||
# pip and venv/curl so nothing needs installing on every single build,
|
||||
# which was slow and quietly undermined reproducibility.
|
||||
#
|
||||
# It lives in the base-images project rather than homelab because that
|
||||
# project is public: this pod pulls the image before any credential is
|
||||
# available to it.
|
||||
#
|
||||
# Versioned tag, never :latest — rebuilding the tools image must not
|
||||
# roll out until this pin is bumped deliberately.
|
||||
image: harbor.infra.deployshed.com/base-images/build-tools:1
|
||||
command: ["cat"]
|
||||
tty: true
|
||||
env:
|
||||
|
||||
@@ -1,49 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
bu: "infra"
|
||||
team: "devops"
|
||||
service: "jenkins-prd"
|
||||
env: "prd"
|
||||
priority: "p0"
|
||||
type: "jenkins"
|
||||
component: "jenkins-agent"
|
||||
spec:
|
||||
serviceAccountName: jenkins-prd-agent
|
||||
containers:
|
||||
- name: devops-tools
|
||||
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/build-tools:lunar-v2.0.22
|
||||
|
||||
imagePullPolicy: Always
|
||||
resources:
|
||||
requests:
|
||||
memory: "6G"
|
||||
cpu: "2"
|
||||
limits:
|
||||
memory: "1000G"
|
||||
cpu: "100"
|
||||
volumeMounts:
|
||||
- mountPath: "/root"
|
||||
name: "cache"
|
||||
readOnly: false
|
||||
env:
|
||||
- name: TZ
|
||||
value: Asia/Kolkata
|
||||
- name: DOCKER_HOST
|
||||
value: dind-prd-svc
|
||||
command:
|
||||
- cat
|
||||
tty: true
|
||||
nodeSelector:
|
||||
dedicated: jenkins
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "jenkins"
|
||||
effect: "NoSchedule"
|
||||
volumes:
|
||||
- name: cache
|
||||
persistentVolumeClaim:
|
||||
claimName: pvc-jenkins-agents-cache-prd-new
|
||||
@@ -1,68 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
bu: "infra"
|
||||
team: "devops"
|
||||
service: "jenkins-prd"
|
||||
env: "prd"
|
||||
priority: "p0"
|
||||
type: "jenkins"
|
||||
component: "jenkins-agent"
|
||||
spec:
|
||||
serviceAccountName: jenkins-prd-agent
|
||||
containers:
|
||||
- name: devops-tools
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/build-tools:lunar-v2.0.21
|
||||
imagePullPolicy: Always
|
||||
resources:
|
||||
requests:
|
||||
memory: "6G"
|
||||
cpu: "2"
|
||||
limits:
|
||||
memory: "1000G"
|
||||
cpu: "100"
|
||||
volumeMounts:
|
||||
- mountPath: "/root"
|
||||
name: "cache"
|
||||
readOnly: false
|
||||
env:
|
||||
- name: TZ
|
||||
value: Asia/Kolkata
|
||||
- name: DOCKER_HOST
|
||||
value: dind-prd-svc
|
||||
command:
|
||||
- cat
|
||||
tty: true
|
||||
|
||||
- name: dind
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/docker:28-dind
|
||||
securityContext:
|
||||
privileged: true
|
||||
env:
|
||||
- name: DOCKER_TLS_CERTDIR
|
||||
value: ""
|
||||
resources:
|
||||
requests:
|
||||
cpu: "1"
|
||||
memory: "2Gi"
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "2Gi"
|
||||
volumeMounts:
|
||||
- name: dind-storage
|
||||
mountPath: /var/lib/docker
|
||||
nodeSelector:
|
||||
dedicated: jenkins
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "jenkins"
|
||||
effect: "NoSchedule"
|
||||
volumes:
|
||||
- name: cache
|
||||
persistentVolumeClaim:
|
||||
claimName: pvc-jenkins-agents-cache-prd-new
|
||||
- name: dind-storage
|
||||
emptyDir: {}
|
||||
|
||||
@@ -1,48 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
bu: "infra"
|
||||
team: "devops"
|
||||
service: "jenkins-stg"
|
||||
env: "stg"
|
||||
priority: "p0"
|
||||
type: "jenkins"
|
||||
component: "jenkins-agent"
|
||||
spec:
|
||||
serviceAccountName: jenkins-dev-agent
|
||||
containers:
|
||||
- name: devops-tools
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/build-tools:lunar-v2.0.22
|
||||
imagePullPolicy: Always
|
||||
resources:
|
||||
requests:
|
||||
memory: "6G"
|
||||
cpu: "2"
|
||||
limits:
|
||||
memory: "1000G"
|
||||
cpu: "100"
|
||||
volumeMounts:
|
||||
- mountPath: "/root"
|
||||
name: "cache"
|
||||
readOnly: false
|
||||
env:
|
||||
- name: TZ
|
||||
value: Asia/Kolkata
|
||||
- name: DOCKER_HOST
|
||||
value: dind-dev-new-svc.jenkins-new.svc.cluster.local
|
||||
command:
|
||||
- cat
|
||||
tty: true
|
||||
|
||||
nodeSelector:
|
||||
dedicated: jenkins
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "jenkins"
|
||||
effect: "NoSchedule"
|
||||
volumes:
|
||||
- name: cache
|
||||
persistentVolumeClaim:
|
||||
claimName: pvc-jenkins-agents-cache-dev
|
||||
@@ -1,67 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
bu: "infra"
|
||||
team: "devops"
|
||||
service: "jenkins-stg"
|
||||
env: "stg"
|
||||
priority: "p0"
|
||||
type: "jenkins"
|
||||
component: "jenkins-agent"
|
||||
spec:
|
||||
serviceAccountName: jenkins-dev-agent
|
||||
containers:
|
||||
- name: devops-tools
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/build-tools:lunar-v2.0.21
|
||||
imagePullPolicy: Always
|
||||
resources:
|
||||
requests:
|
||||
memory: "6G"
|
||||
cpu: "2"
|
||||
limits:
|
||||
memory: "1000G"
|
||||
cpu: "100"
|
||||
volumeMounts:
|
||||
- mountPath: "/root"
|
||||
name: "cache"
|
||||
readOnly: false
|
||||
env:
|
||||
- name: TZ
|
||||
value: Asia/Kolkata
|
||||
- name: DOCKER_HOST
|
||||
value: dind-dev-new-svc.jenkins-new.svc.cluster.local
|
||||
command:
|
||||
- cat
|
||||
tty: true
|
||||
|
||||
- name: dind
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/docker:28-dind
|
||||
securityContext:
|
||||
privileged: true
|
||||
env:
|
||||
- name: DOCKER_TLS_CERTDIR
|
||||
value: "" # Disable TLS so we can connect via HTTP on port 2375
|
||||
resources:
|
||||
requests:
|
||||
cpu: "1"
|
||||
memory: "2Gi"
|
||||
limits:
|
||||
cpu: "1"
|
||||
memory: "2Gi"
|
||||
volumeMounts:
|
||||
- name: dind-storage
|
||||
mountPath: /var/lib/docker
|
||||
nodeSelector:
|
||||
dedicated: jenkins
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "jenkins"
|
||||
effect: "NoSchedule"
|
||||
volumes:
|
||||
- name: cache
|
||||
persistentVolumeClaim:
|
||||
claimName: pvc-jenkins-agents-cache-dev
|
||||
- name: dind-storage
|
||||
emptyDir: {}
|
||||
@@ -1,84 +0,0 @@
|
||||
#!/bin/bash
|
||||
prepareDockerfileWithBuild(){
|
||||
cat << EOF > Dockerfile-${artifactId}
|
||||
# This sample, non-production-ready template describes an Amazon EC2 instance and an Elastic Load Balancer.
|
||||
# © 2020 Amazon Web Services, Inc. or its affiliates. All Rights Reserved.
|
||||
# This AWS Content is provided subject to the terms of the AWS Customer Agreement available at
|
||||
# http://aws.amazon.com/agreement or other written agreement between Customer and either
|
||||
# Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both.
|
||||
FROM 766380763301.dkr.ecr.ap-south-1.amazonaws.com/build/maven:3.3-jdk-8 as BUILD
|
||||
WORKDIR /usr/src/app
|
||||
COPY . /usr/src/app
|
||||
RUN mvn -s sandbox-settings.xml clean install -DskipTests
|
||||
RUN mkdir -p /var/log/${artifactId} && touch /var/log/${artifactId}/gc.log
|
||||
FROM 766380763301.dkr.ecr.ap-south-1.amazonaws.com/build/java:8-jdk-slim-secure_v1.0
|
||||
ADD https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.15.0/jmx_prometheus_javaagent-0.15.0.jar /opt/jmx_exporter.jar
|
||||
### Config added through configmap
|
||||
# COPY config.yaml /opt/config.yaml
|
||||
EXPOSE 8880 8010
|
||||
COPY --from=BUILD /usr/src/app/target/${artifactId}-${version}.jar /opt/target/${artifactId}.jar
|
||||
COPY --from=BUILD /var/log/${artifactId} /var/log/${artifactId}
|
||||
WORKDIR /opt/target
|
||||
CMD ["${artifactId}.jar", "-javaagent:/opt/jmx_exporter.jar=8880:/opt/config/jmx-config.yaml", \
|
||||
"-XX:MinRAMPercentage=50.0", "-XX:MaxRAMPercentage=80.0", \
|
||||
"-XX:+UseParallelGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails", \
|
||||
"-XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime", "-XX:+PrintHeapAtGC", \
|
||||
"-Xloggc:/var/log/${artifactId}/gc.log", \
|
||||
"-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=9000k", \
|
||||
"-Xms2G", "-Xmx2G"]
|
||||
EOF
|
||||
|
||||
}
|
||||
|
||||
prepareDockerfileNoBuild(){
|
||||
cat << EOF > Dockerfile-${artifactId}
|
||||
# This sample, non-production-ready template describes an Amazon EC2 instance and an Elastic Load Balancer.
|
||||
# © 2020 Amazon Web Services, Inc. or its affiliates. All Rights Reserved.
|
||||
# This AWS Content is provided subject to the terms of the AWS Customer Agreement available at
|
||||
# http://aws.amazon.com/agreement or other written agreement between Customer and either
|
||||
# Amazon Web Services, Inc. or Amazon Web Services EMEA SARL or both.
|
||||
FROM 766380763301.dkr.ecr.ap-south-1.amazonaws.com/build/maven:3.3-jdk-8 as BUILD
|
||||
#FROM asia-southeast1-docker.pkg.dev/supply-poc-351106/homelab-devops/maven:3.3-jdk-8 as BUILD
|
||||
RUN mkdir -p /var/log/${artifactId} && touch /var/log/${artifactId}/gc.log
|
||||
FROM 766380763301.dkr.ecr.ap-south-1.amazonaws.com/build/java:8-jdk-slim-secure_v1.0
|
||||
#FROM asia-southeast1-docker.pkg.dev/supply-poc-351106/homelab-devops/java:8
|
||||
ADD https://repo1.maven.org/maven2/io/prometheus/jmx/jmx_prometheus_javaagent/0.15.0/jmx_prometheus_javaagent-0.15.0.jar /opt/jmx_exporter.jar
|
||||
### Config added through configmap
|
||||
# COPY config.yaml /opt/config.yaml
|
||||
EXPOSE 8880 8010
|
||||
COPY ${artifactId}/target/*.jar /opt/target/${artifactId}.jar
|
||||
COPY --from=BUILD /var/log/${artifactId} /var/log/${artifactId}
|
||||
WORKDIR /opt/target
|
||||
CMD ["${artifactId}.jar", "-javaagent:/opt/jmx_exporter.jar=8880:/opt/config/jmx-config.yaml", \
|
||||
"-XX:MinRAMPercentage=50.0", "-XX:MaxRAMPercentage=80.0", \
|
||||
"-XX:+UseParallelGC -XX:+PrintGCDateStamps -XX:+PrintGCDetails", \
|
||||
"-XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime", "-XX:+PrintHeapAtGC", \
|
||||
"-Xloggc:/var/log/${artifactId}/gc.log", \
|
||||
"-XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=5 -XX:GCLogFileSize=9000k", \
|
||||
"-Xms2G", "-Xmx2G"]
|
||||
EOF
|
||||
|
||||
}
|
||||
|
||||
main(){
|
||||
if [[ $# == 4 ]];then
|
||||
export artifactId=$1
|
||||
export version=$2
|
||||
export doBuild=$3
|
||||
export repoName=$4
|
||||
if [[ $doBuild == "YES" ]];then
|
||||
rm -f Dockerfile-${artifactId}
|
||||
prepareDockerfileWithBuild
|
||||
else
|
||||
rm -f Dockerfile-${artifactId}
|
||||
prepareDockerfileNoBuild
|
||||
fi
|
||||
ls -latr
|
||||
cat Dockerfile-${artifactId}
|
||||
else
|
||||
echo "Please provide required parameters"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
main $@
|
||||
@@ -1,50 +0,0 @@
|
||||
#!/bin/bash
|
||||
prepareDockerfile(){
|
||||
cat << EOF > Dockerfile-${artifactId}
|
||||
|
||||
ARG ACCOUNT_ID=766380763301
|
||||
|
||||
FROM \${ACCOUNT_ID}.dkr.ecr.ap-southeast-1.amazonaws.com/build/node:12.22.1-slim as build-env
|
||||
|
||||
WORKDIR /usr/src/app
|
||||
|
||||
COPY package*.json ./
|
||||
|
||||
RUN npm install
|
||||
|
||||
FROM \${ACCOUNT_ID}.dkr.ecr.ap-southeast-1.amazonaws.com/build/node:12.22.1-alpine
|
||||
|
||||
# for health checks
|
||||
# RUN apk add --update --no-cache curl=7.74.0-r1
|
||||
|
||||
USER node:1000
|
||||
|
||||
COPY --chown=node:1000 --from=build-env /usr/src/app /app
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
EXPOSE 2020
|
||||
|
||||
COPY --chown=node:1000 . .
|
||||
|
||||
CMD ["npm", "start"]
|
||||
|
||||
EOF
|
||||
|
||||
}
|
||||
|
||||
main(){
|
||||
if [[ $# == 1 ]];then
|
||||
export artifactId=$1
|
||||
rm -f Dockerfile-${artifactId}
|
||||
prepareDockerfile
|
||||
ls -latr
|
||||
cat Dockerfile-${artifactId}
|
||||
else
|
||||
echo "Please provide required parameters"
|
||||
exit 1
|
||||
fi
|
||||
}
|
||||
|
||||
main $@
|
||||
|
||||
@@ -1,47 +0,0 @@
|
||||
apiVersion: v1
|
||||
kind: Pod
|
||||
metadata:
|
||||
labels:
|
||||
bu: "infra"
|
||||
team: "toolchain"
|
||||
service: "jenkins-toolchain"
|
||||
env: "stg"
|
||||
priority: "p0"
|
||||
type: "jenkins"
|
||||
component: "jenkins-agent"
|
||||
spec:
|
||||
serviceAccountName: jenkins-toolchain-dev-agent
|
||||
containers:
|
||||
- name: devops-tools
|
||||
image: asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/build-tools:lunar-v2.0.22
|
||||
imagePullPolicy: Always
|
||||
resources:
|
||||
requests:
|
||||
memory: "6G"
|
||||
cpu: "2"
|
||||
limits:
|
||||
memory: "1000G"
|
||||
cpu: "100"
|
||||
volumeMounts:
|
||||
- mountPath: "/root"
|
||||
name: "cache"
|
||||
readOnly: false
|
||||
env:
|
||||
- name: TZ
|
||||
value: Asia/Kolkata
|
||||
- name: DOCKER_HOST
|
||||
value: toolchain-dind-dev-svc.jenkins-toolchain.svc.cluster.local
|
||||
command:
|
||||
- cat
|
||||
tty: true
|
||||
nodeSelector:
|
||||
dedicated: toolchain-jenkins
|
||||
tolerations:
|
||||
- key: "dedicated"
|
||||
operator: "Equal"
|
||||
value: "toolchain-jenkins"
|
||||
effect: "NoSchedule"
|
||||
volumes:
|
||||
- name: cache
|
||||
persistentVolumeClaim:
|
||||
claimName: pvc-toolchain-jenkins-agents-cache-dev
|
||||
@@ -1,68 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.stages.deployJar
|
||||
|
||||
|
||||
def run(Map config){
|
||||
// get required variables from config
|
||||
def automation_repo_name = config.run_automation.repo_name
|
||||
def branch = config.run_automation.branch
|
||||
def deployObj = new deployJar()
|
||||
|
||||
//deploy service(farmiso) on specified machine
|
||||
deployObj.run(config)
|
||||
|
||||
|
||||
checkoutAutomationRepo(automation_repo_name, branch)
|
||||
|
||||
runAutomationSuite(automation_repo_name)
|
||||
|
||||
}
|
||||
|
||||
def checkoutAutomationRepo(String automation_repo_name, String branch){
|
||||
try{
|
||||
stage('Checkout automation repo'){
|
||||
sh "rm -rf ${automation_repo_name}; git clone git@github.com:Homelab/${automation_repo_name}.git -b ${branch}"
|
||||
echo "automation repo cloned"
|
||||
}
|
||||
}
|
||||
catch( Exception e) {
|
||||
env.msg = "Error cloning automation repo. Please check console output for more details."
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def runAutomationSuite(String automation_repo_name){
|
||||
try{
|
||||
stage('Run Automation Suite'){
|
||||
dir("$automation_repo_name"){
|
||||
run_cmd = "mvn test" //specific to Farmiso as of now.
|
||||
sh "$run_cmd"
|
||||
}
|
||||
}
|
||||
}
|
||||
catch( Exception e) {
|
||||
env.msg = "Error running automation suite. Please check console output for more details."
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
finally{
|
||||
//publish HTML test report
|
||||
dir("$automation_repo_name"){
|
||||
archiveArtifacts artifacts: "test-output/*.*"
|
||||
|
||||
|
||||
publishHTML (target: [
|
||||
allowMissing: false,
|
||||
alwaysLinkToLastBuild: false,
|
||||
keepAll: true,
|
||||
reportDir: 'test-output',
|
||||
reportFiles: 'index.html',
|
||||
reportName: "Farmiso-Test-Automation-Report"
|
||||
])
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -20,7 +20,35 @@ import com.homelab.utilities.constructTemplate
|
||||
def run(Map config) {
|
||||
def tag = "${env.BUILD_NUMBER}-${env.GIT_COMMIT?.take(7) ?: 'dev'}"
|
||||
env.TAG = tag
|
||||
def image = "harbor-core.harbor.svc.cluster.local/${config.harbor_project}/${config.repo_name}:${tag}"
|
||||
// The registry ingress hostname, not harbor-core.harbor.svc.cluster.local.
|
||||
// Cluster DNS would work for this push, which runs inside a pod, but the
|
||||
// Deployment's image PULL is performed by containerd on the node, using
|
||||
// the node's own resolver, which has no route to *.svc.cluster.local at
|
||||
// all — that namespace exists only in CoreDNS. This is the one place the
|
||||
// project's "always use cluster DNS between services" rule cannot apply:
|
||||
// the puller is not a service.
|
||||
//
|
||||
// It also has to be spelled identically everywhere, because Docker
|
||||
// matches stored credentials and TLS trust by exact hostname: here, the
|
||||
// dockerconfigjson auths key and Harbor's externalURL.
|
||||
//
|
||||
// Configurable rather than hardcoded, which is what this used to be. A
|
||||
// literal here meant the registry could not move without editing the
|
||||
// shared library itself, and every consumer moving in the same commit —
|
||||
// the failure being a push to a hostname nothing serves, several minutes
|
||||
// into a build.
|
||||
//
|
||||
// The DEFAULT lives in homelabPipeline.groovy beside harbor_project and
|
||||
// every other key, not here. Repeating the literal in both places would
|
||||
// leave two defaults free to disagree, and the one that lost would only
|
||||
// show up as a push to the wrong registry. Failing loudly beats
|
||||
// defaulting quietly: an unset value would otherwise build an image
|
||||
// named "null/<project>/<repo>", which docker accepts as a hostname and
|
||||
// then fails to resolve, pointing nowhere near the cause.
|
||||
if (!config.harbor_registry) {
|
||||
error("buildDocker: config.harbor_registry is not set. homelabPipeline normally defaults it; if this stage is being called directly, pass harbor_registry (e.g. 'harbor.infra.deployshed.com').")
|
||||
}
|
||||
def image = "${config.harbor_registry}/${config.harbor_project}/${config.repo_name}:${tag}"
|
||||
try {
|
||||
stage(stageName('Build & push image')) {
|
||||
container('docker-cli') {
|
||||
@@ -90,8 +118,42 @@ def renderDockerfile(Map config) {
|
||||
def (templateFile, defaultVersion) = entry
|
||||
def resolvedVersion = version ?: defaultVersion
|
||||
|
||||
validateRepoStructure(lang)
|
||||
|
||||
log.info("buildDocker: no Dockerfile in repo — rendering ${templateFile} for ${lang} ${resolvedVersion}")
|
||||
def constructObj = new constructTemplate()
|
||||
constructObj.renderTemplate([version: resolvedVersion], templateFile, 'Dockerfile')
|
||||
sh 'cat Dockerfile'
|
||||
}
|
||||
|
||||
// Each fallback template assumes one specific, canonical repo layout — it is
|
||||
// a fixed Dockerfile per language, not a detector across the many layouts a
|
||||
// real repo might actually use (a subdirectory build, Gradle instead of
|
||||
// Maven, a different entrypoint name). That is a real limitation, not just
|
||||
// this check's — see docs/PRODUCT-ARCHITECTURE.md's buildpacks item for the
|
||||
// actual fix. Until then, this at least turns a missing file into a specific
|
||||
// message naming the file and the fix, in place of a raw `docker build`
|
||||
// COPY failure that never says what the fallback expected in the first
|
||||
// place, several minutes into a build someone was told needs no Dockerfile.
|
||||
//
|
||||
// Deliberately only checks for a file's existence, never its contents (e.g.
|
||||
// not whether requirements.txt's app matches gunicorn's `app:app` target) —
|
||||
// that would need language-aware parsing this stage has no business doing,
|
||||
// and a wrong guess would be a worse failure than no check at all.
|
||||
def validateRepoStructure(String lang) {
|
||||
def required = [
|
||||
go : ['go.mod', 'a Go module — run `go mod init <module-name>` at the repo root'],
|
||||
node : ['package.json', 'an npm project — run `npm init` at the repo root, with a "start" script'],
|
||||
python: ['requirements.txt', 'your dependencies, and expects the app itself as `app:app` (Flask/FastAPI-style) for gunicorn'],
|
||||
java : ['pom.xml', 'a Maven project — Gradle repos need their own Dockerfile for now'],
|
||||
maven : ['pom.xml', 'a Maven project — Gradle repos need their own Dockerfile for now'],
|
||||
]
|
||||
def check = required[lang]
|
||||
if (!check) {
|
||||
return // php has no required file — composer.json is used only if present
|
||||
}
|
||||
def (file, help) = check
|
||||
if (!fileExists(file)) {
|
||||
error("buildDocker: no ${file} found at the repo root. The '${lang}' fallback build expects ${help}. If your repository has a different layout, add your own Dockerfile instead — that always takes priority over this fallback and can build however you like.")
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,412 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.utilities.addSSHKey
|
||||
import com.homelab.utilities.gitActions
|
||||
import com.homelab.utilities.constructParam
|
||||
import com.homelab.utilities.dockerUtilities
|
||||
|
||||
|
||||
def buildDckr(Map config) {
|
||||
env.GOPRIVATE = 'github.com/Homelab'
|
||||
def buTeamMappingObj = new buTeamMapping()
|
||||
def constructObj = new constructTemplate()
|
||||
def dockerParamObj = new getDockerParams()
|
||||
def addSshKeyObj = new addSSHKey()
|
||||
def dockerUtilObj = new dockerUtilities()
|
||||
|
||||
def team = buTeamMappingObj.get_team_initials(config.team)
|
||||
def modules = config.modules ?: ['module_less']
|
||||
def repoName = config.repo_name
|
||||
def dockerRepository = "${env.cicd_environment}/${team}/${repoName.toLowerCase()}"
|
||||
def tag = dockerParamObj.getTag(repoName)
|
||||
def dockerBindings = [:]
|
||||
def version = config.dockerBuildVersion.split('-')[-1]
|
||||
def constructParamObj = new constructParam()
|
||||
def repoType = config.repo_type ?: 'microservice'
|
||||
boolean skipSonarAndQualityGate = constructParamObj.skipSonarCheckForGo(config)
|
||||
echo "skipSonarAndQualityGate: ${skipSonarAndQualityGate}"
|
||||
String goVersion = config.goVersion ?: version
|
||||
boolean shouldDeployArgo = (config.deployArgo ?: false).toBoolean()
|
||||
boolean isConfigOnlyChange = false
|
||||
|
||||
// Ensure goVersion is in x.x.x format
|
||||
echo "goVersion: ${goVersion}"
|
||||
if (goVersion.split('\\.').size() == 2) {
|
||||
goVersion += '.0'
|
||||
}
|
||||
if (env.INFRA_ENV == 'toolchain') {
|
||||
tag = dockerParamObj.getTag(repoName)
|
||||
boolean allImagesExist = true
|
||||
|
||||
for (module in modules) {
|
||||
def moduleName = (module instanceof LinkedHashMap) ? module.keySet()[0] : module
|
||||
def modulePath = (moduleName == 'module_less') ? dockerRepository : "${dockerRepository}/${moduleName}"
|
||||
//TODO: building everything if one is missing -> create list of modules unbuild will only build those
|
||||
if (!dockerUtilObj.imageExists(env.registry, modulePath, tag)) {
|
||||
log.info("Toolchain: Image missing for ${moduleName} at ${modulePath}:${tag}. Proceeding with build.")
|
||||
allImagesExist = false
|
||||
break
|
||||
}
|
||||
}
|
||||
if (allImagesExist) {
|
||||
log.info("Toolchain: All images found in registry for tag ${tag}. Skipping build step.")
|
||||
return [tag, shouldDeployArgo]
|
||||
}
|
||||
}
|
||||
stage(stageName('Check Config Changes and Get Latest Image Tag')) {
|
||||
(isConfigOnlyChange, shouldDeployArgo) = is_config_only_change_and_should_deploy_argo(repoName, shouldDeployArgo)
|
||||
log.info("Is just application config change? $isConfigOnlyChange")
|
||||
def param = new constructParam()
|
||||
if (env.CHANGE_ID && repoType == 'microservice') {
|
||||
def shouldValidateConfig = param.ValidateCacConfigForRepo(false, repoName)
|
||||
if (shouldValidateConfig) {
|
||||
log.info('************ Validate Config for CAC application.yml files ************')
|
||||
dir("$repoName") {
|
||||
writeFile file: 'validate_configs.py', text: libraryResource('com/homelab/validate_configs_v2.py')
|
||||
sh 'python3 validate_configs.py'
|
||||
}
|
||||
} else {
|
||||
log.info('************ Skipping Validation of CAC Config ************')
|
||||
}
|
||||
} else {
|
||||
log.info("Skipping Validation of CAC Config for ${repoType} repo type")
|
||||
log.info('************ Skipping Validation of CAC Config ************')
|
||||
}
|
||||
|
||||
if (isConfigOnlyChange) {
|
||||
log.info('Skipping the build, since it is just application config change')
|
||||
log.info('Getting latest image tag from GAR')
|
||||
def firstModule = modules[0]
|
||||
def firstModuleName = (firstModule instanceof LinkedHashMap) ? firstModule.keySet()[0] : firstModule
|
||||
def moduleDockerRepository = (firstModuleName == 'module_less') ? dockerRepository : "${dockerRepository}/${firstModuleName}"
|
||||
try {
|
||||
def latest_image = sh(
|
||||
script: "gcloud container images list-tags ${env.registry}/${moduleDockerRepository} --format='value(tags)' | sed '/^\$/d' | awk -F'-' '{print \$NF}' | sort | tail -1",
|
||||
returnStdout: true
|
||||
).trim()
|
||||
echo "latest_image: ${latest_image}"
|
||||
if (latest_image == '') {
|
||||
log.info("No existing images found for ${env.registry}/${moduleDockerRepository}. Using generated tag: ${tag}")
|
||||
} else {
|
||||
tag = sh(
|
||||
script: "gcloud container images list-tags ${env.registry}/${moduleDockerRepository} --format='value(tags)' | tr ',' '\n' | grep ${latest_image}",
|
||||
returnStdout: true
|
||||
).trim()
|
||||
}
|
||||
} catch (Exception e) {
|
||||
env.msg = 'Error getting latest docker image from GAR'
|
||||
env.error_msg_to_db = env.msg
|
||||
log.error("${env.msg}\n${e.toString()}")
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
return [tag, shouldDeployArgo]
|
||||
}
|
||||
log.info('Proceeding with the build')
|
||||
}
|
||||
|
||||
stage(stageName('Scanning Sonar and Quality Gate')) {
|
||||
dir(repoName) {
|
||||
sonar_scan(repoName, skipSonarAndQualityGate, goVersion)
|
||||
}
|
||||
}
|
||||
if (currentBuild.result == 'UNSTABLE')
|
||||
{ return [tag, false] }
|
||||
|
||||
if (repoType != 'microservice') {
|
||||
log.info("Skipping Docker build for ${repoType} repo type")
|
||||
return [tag, shouldDeployArgo]
|
||||
}
|
||||
|
||||
stage(stageName('Building docker images')) {
|
||||
try {
|
||||
sh(script: 'gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet')
|
||||
} catch (Exception e) {
|
||||
env.msg = "[Failure] Can't login to gcloud docker registry."
|
||||
env.error_msg_to_db = env.msg
|
||||
log.error("${env.msg}. Error: ${e.toString()}")
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
dockerBindings['version'] = (version == 'go') ? '1.24.4' : version
|
||||
dockerBindings['base_dir'] = config.base_dir ?: false
|
||||
dockerBindings['build_registry'] = env.buildRegistry
|
||||
dockerBindings['go_proxy'] = env.goProxyUrl
|
||||
dockerBindings['repo_name'] = repoName
|
||||
if (config.containsKey('copy_file')) {
|
||||
String recursive = config.copy_file.recursive ? ' -r' : ''
|
||||
dir(repoName) {
|
||||
dir('copied_files') {
|
||||
sh(script: "gsutil cp${recursive} ${config.copy_file.path} .")
|
||||
}
|
||||
}
|
||||
dockerBindings['copy_file'] = true
|
||||
dockerBindings['copy_target'] = config.copy_file.target ?: '/app/'
|
||||
dockerBindings['base_dir'] = config.base_dir ?: false
|
||||
} else {
|
||||
dockerBindings['copy_file'] = false
|
||||
}
|
||||
|
||||
try {
|
||||
dir(repoName) {
|
||||
addSshKeyObj.create()
|
||||
if (!fileExists('Dockerfile')) {
|
||||
def moduleBuilds = [:]
|
||||
for (m in modules) {
|
||||
def moduleRef = m
|
||||
def moduleName = (moduleRef instanceof LinkedHashMap) ? moduleRef.keySet()[0] : moduleRef
|
||||
moduleBuilds["build-${moduleName}"] = {
|
||||
def localBindings = dockerBindings.clone()
|
||||
localBindings['module_property'] = (moduleRef instanceof LinkedHashMap) ? moduleRef[moduleName] : [:]
|
||||
localBindings['kafka'] = (localBindings['module_property'].kafka) ? '-kafka' : ''
|
||||
localBindings['module'] = moduleName
|
||||
constructObj.renderTemplate(localBindings, 'go-Dockerfile', "Dockerfile-${moduleName}")
|
||||
sh "cat Dockerfile-${moduleName}"
|
||||
def moduleDockerRepository = (moduleRef == 'module_less') ? dockerRepository : "${dockerRepository}/${moduleName}"
|
||||
sh(script: 'tar -cf only-mods.tar $(git ls-files "go.mod" "go.sum" "**/go.mod" "**/go.sum" 2>/dev/null || find . -name go.mod -o -name go.sum)')
|
||||
sh(script: "set +x && docker build --tag ${env.registry}/${moduleDockerRepository}:${tag} -f Dockerfile-${moduleName} .")
|
||||
if (env.cicd_environment != 'ftr' || env.INFRA_ENV == 'toolchain') {
|
||||
dockerUtilObj.retryDockerPush("docker push ${env.registry}/${moduleDockerRepository}:${tag}")
|
||||
} else {
|
||||
log.info("Skipping Docker Push - ${env.cicd_environment} env")
|
||||
}
|
||||
sh(script: "rm -rf Dockerfile-${moduleName}")
|
||||
}
|
||||
}
|
||||
parallel moduleBuilds
|
||||
} else {
|
||||
sh 'cat Dockerfile'
|
||||
def moduleDockerRepository = dockerRepository // default for repo root
|
||||
if (modules && modules[0] != 'module_less') {
|
||||
def firstModule = modules[0]
|
||||
def firstModuleName = (firstModule instanceof LinkedHashMap) ? firstModule.keySet()[0] : firstModule
|
||||
moduleDockerRepository = dockerRepository + '/' + firstModuleName
|
||||
}
|
||||
def pushOrNot = (env.cicd_environment != 'ftr' || env.INFRA_ENV == 'toolchain') ? '--push' : ''
|
||||
if (!pushOrNot) {
|
||||
log.info("Skipping Docker Push - ${env.cicd_environment} env")
|
||||
}
|
||||
sh(script: "docker build --tag ${env.registry}/${moduleDockerRepository}:${tag} ${pushOrNot} .")
|
||||
}
|
||||
}
|
||||
} catch (Exception e) {
|
||||
env.msg = 'Error in building DockerFile Or Pushing To ECR'
|
||||
env.error_msg_to_db = env.msg
|
||||
log.error("${env.msg}. Error: ${e.toString()}")
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
return [tag, shouldDeployArgo]
|
||||
}
|
||||
}
|
||||
|
||||
def is_config_only_change_and_should_deploy_argo(String repoName, boolean deployArgo) {
|
||||
if (env.INFRA_ENV == 'toolchain') {
|
||||
return [false, deployArgo]
|
||||
}
|
||||
def gitObj = new gitActions()
|
||||
def configFiles = []
|
||||
boolean shouldDeployArgoResult = deployArgo
|
||||
boolean isConfigOnlyChange = false
|
||||
|
||||
def changedFiles = env.CHANGE_ID ?
|
||||
gitObj.fetchDiffFilesForPullRequest(repoName, env.CHANGE_TARGET) :
|
||||
gitObj.fetchDiffFilesForPushRequest(repoName)
|
||||
|
||||
if (!changedFiles) {
|
||||
return [isConfigOnlyChange, shouldDeployArgoResult]
|
||||
}
|
||||
|
||||
for (file in changedFiles.split('\n')) {
|
||||
if (!file.startsWith('configs/')) {
|
||||
isConfigOnlyChange = false
|
||||
return [isConfigOnlyChange, shouldDeployArgoResult]
|
||||
}
|
||||
isConfigOnlyChange = true
|
||||
configFiles << file
|
||||
}
|
||||
|
||||
if (configFiles && deployArgo && isConfigOnlyChange) {
|
||||
def hasEnvironmentConfig = configFiles.any { it.contains(env.cicd_environment) }
|
||||
if (!hasEnvironmentConfig) {
|
||||
shouldDeployArgoResult = false
|
||||
echo "Config changes don't contain environment: ${env.cicd_environment}"
|
||||
}
|
||||
}
|
||||
return [isConfigOnlyChange, shouldDeployArgoResult]
|
||||
}
|
||||
|
||||
/**
|
||||
* Function to execute sonar scan
|
||||
* @param repoName: repository name
|
||||
* @param skipSonarAndQualityGate: boolean parameter to skip sonar scan
|
||||
* @param goVersion: Go version string
|
||||
*/
|
||||
def sonar_scan(String repoName, boolean skipSonarAndQualityGate, String goVersion) {
|
||||
if (skipSonarAndQualityGate) {
|
||||
log.info('Sonar scan is skipped. Marking this stage as passed.')
|
||||
return
|
||||
}
|
||||
stage(stageName('Running sonar scan')) {
|
||||
try {
|
||||
withSonarQubeEnv(env.sonarEnv) {
|
||||
echo "env.sonarEnv: ${env.sonarEnv}"
|
||||
echo "env.BRANCH_NAME: ${env.BRANCH_NAME}"
|
||||
def exclusions = ""
|
||||
if (!fileExists('sonar-project.properties')) {
|
||||
exclusions = " -Dsonar.exclusions=**/*_test.go,**/mock_*.go,**/mock.go,**/*.pb.go,**/*.proto,**/model.go"
|
||||
}else{
|
||||
exclusions = " -Dproject.settings=`pwd`/sonar-project.properties"
|
||||
}
|
||||
|
||||
def scannerCommand = "sonar-scanner -Dsonar.projectKey=${repoName} -Dsonar.go.coverage.reportPaths=./cov.out -Dsonar.branch.name=${env.BRANCH_NAME} -Dsonar.ws.timeout=120 ${exclusions}"
|
||||
echo "env.SONAR_HOST_URL: ${env.SONAR_HOST_URL}"
|
||||
env.PATH = "/usr/local/sonar-scanner/sonar-scanner-5.0.1.3006-linux/bin:${env.PATH}"
|
||||
def response = sh(
|
||||
script: "curl -s -w '\n%{http_code}' -u ${env.SONAR_AUTH_TOKEN}: ${env.SONAR_HOST_URL}/api/navigation/component?component=${repoName}",
|
||||
returnStdout: true
|
||||
).trim()
|
||||
|
||||
def responseLines = response.split('\n')
|
||||
def statusCode = responseLines[-1]
|
||||
def responseBody = responseLines[0..-2].join('\n')
|
||||
|
||||
log.info("SonarQube API Response Code: ${statusCode}")
|
||||
log.info("SonarQube API Response Body: ${responseBody}")
|
||||
|
||||
def projectExists = (statusCode == '200')
|
||||
if (!projectExists && env.CHANGE_ID) {
|
||||
log.info("Project ${repoName} does not exist. Creating it via a bootstrap scan on the target branch.")
|
||||
|
||||
def targetBranch = env.CHANGE_TARGET ?: 'develop'
|
||||
echo "targetBranch: ${targetBranch}"
|
||||
sh "sonar-scanner \
|
||||
-Dsonar.projectKey=${repoName} \
|
||||
-Dsonar.projectName=${repoName} \
|
||||
-Dsonar.branch.name=${targetBranch} \
|
||||
-Dsonar.sources=. \
|
||||
-Dsonar.scm.disabled=true \
|
||||
-Dsonar.ws.timeout=120"
|
||||
log.info("Bootstrap complete. Project created.")
|
||||
}else{
|
||||
echo "Project ${repoName} already exists. Skipping bootstrap scan."
|
||||
}
|
||||
|
||||
if (env.CHANGE_ID) {
|
||||
echo "env.CHANGE_ID: ${env.CHANGE_ID}"
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh "git fetch origin ${env.CHANGE_TARGET}:refs/remotes/origin/${env.CHANGE_TARGET}"
|
||||
}
|
||||
scannerCommand = "sonar-scanner -Dsonar.projectKey=${repoName} -Dsonar.pullrequest.provider=GitHub -Dsonar.pullrequest.github.repository=Homelab/${repoName} -Dsonar.pullrequest.key=${env.CHANGE_ID} -Dsonar.pullrequest.branch=${env.CHANGE_BRANCH} -Dsonar.pullrequest.base=${env.CHANGE_TARGET} -Dsonar.go.coverage.reportPaths=./cov.out -Dsonar.ws.timeout=120${exclusions}"
|
||||
echo "scannerCommand: ${scannerCommand}"
|
||||
}
|
||||
downloadGoFromJFrog(goVersion)
|
||||
sh(script: "GOPROXY=${env.goProxyUrl},direct && go mod tidy")
|
||||
int testExitCode = sh(script: 'go test -short -coverprofile=./cov.out ./...', returnStatus: true)
|
||||
if (testExitCode != 0) {
|
||||
error('Go tests failed.')
|
||||
}
|
||||
echo "scannerCommand: ${scannerCommand}"
|
||||
sh(script: scannerCommand)
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("Error in running sonar scan: ${e}")
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
stage(stageName('Quality Gate')) {
|
||||
try {
|
||||
if (!env.CHANGE_ID || env.cicd_environment == 'int') {
|
||||
log.info('Skipping quality gate check on Branches/Pre-Prod. Marking this stage as passed.')
|
||||
} else {
|
||||
timeout(time: 600, unit: 'SECONDS') {
|
||||
def qg = waitForQualityGate()
|
||||
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
|
||||
if (qg.status != 'OK') {
|
||||
log.warning("Quality gate failed: ${qg.status}")
|
||||
error "Stopping pipeline due to quality gate failure."
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}catch (Exception e) {
|
||||
echo "Error in quality gate: ${e.message}"
|
||||
unstable('Quality Gate Failed !')
|
||||
throw e
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
/**
|
||||
* Function to download Go binary from JFrog
|
||||
* @param goVersion: Go version string
|
||||
*/
|
||||
def downloadGoFromJFrog(String goVersion) {
|
||||
try {
|
||||
echo "env.cicd_environment: ${env.cicd_environment}"
|
||||
|
||||
def jfrogUrl = (env.cicd_environment == 'prd' || env.cicd_environment == 'int') ?
|
||||
'https://jfrog-prd.homelabgcp.in' :
|
||||
'https://jfrog-dev.homelabgcp.in'
|
||||
|
||||
def repo = 'devops-tools-local'
|
||||
def goTarball = "go${goVersion}.linux-amd64.tar.gz"
|
||||
def jfrogPath = "${jfrogUrl}/artifactory/${repo}/go/${goTarball}"
|
||||
|
||||
def jfrogUser = ''
|
||||
def jfrogPass = ''
|
||||
def credentialId = (env.cicd_environment == 'prd' || env.cicd_environment == 'int') ?
|
||||
'jfrog-prd-credentials' : 'jfrog-stg-credentials'
|
||||
echo "Using credential ID: ${credentialId}"
|
||||
withCredentials([usernamePassword(credentialsId: credentialId,
|
||||
usernameVariable: 'JFROG_USER',
|
||||
passwordVariable: 'JFROG_PASS')]) {
|
||||
jfrogUser = env.JFROG_USER
|
||||
jfrogPass = env.JFROG_PASS
|
||||
|
||||
log.info("Attempting to download Go ${goVersion} from JFrog: ${jfrogPath}")
|
||||
|
||||
def downloadStatus = sh(script: """
|
||||
curl -u "${env.JFROG_USER}:${env.JFROG_PASS}" \
|
||||
-fLO "${jfrogPath}" \
|
||||
--fail --silent --show-error
|
||||
""", returnStatus: true)
|
||||
|
||||
if (downloadStatus == 0 && fileExists(goTarball)) {
|
||||
log.info("Successfully downloaded Go ${goVersion} from JFrog")
|
||||
} else {
|
||||
echo("WARN: Go ${goVersion} not found in JFrog. Falling back to go.dev...")
|
||||
sh(script: "curl -LO https://go.dev/dl/${goTarball}")
|
||||
|
||||
log.info("Uploading downloaded Go ${goVersion} to JFrog for future use")
|
||||
sh(script: """
|
||||
curl -u "${env.JFROG_USER}:${env.JFROG_PASS}" \
|
||||
-T "${goTarball}" \
|
||||
"${jfrogPath}" \
|
||||
--fail --silent --show-error || echo "Upload to JFrog failed, but continuing..."
|
||||
""")
|
||||
}
|
||||
}
|
||||
|
||||
sh(script: "tar -xvzf ${goTarball} -C /usr/local", returnStdout: true)
|
||||
sh(script: "rm -Rf ${goTarball}")
|
||||
env.PATH = "/usr/local/go/bin:${env.PATH}"
|
||||
|
||||
// Verify installation
|
||||
sh(script: "go version")
|
||||
|
||||
} catch (Exception e) {
|
||||
echo("ERROR: Failed to download Go from JFrog: ${e.message}")
|
||||
|
||||
// Final fallback
|
||||
sh(script: "curl -LO https://go.dev/dl/go${goVersion}.linux-amd64.tar.gz")
|
||||
sh(script: "tar -xvzf go${goVersion}.linux-amd64.tar.gz -C /usr/local", returnStdout: true)
|
||||
sh(script: "rm -Rf go${goVersion}.linux-amd64.tar.gz")
|
||||
env.PATH = "/usr/local/go/bin:${env.PATH}"
|
||||
}
|
||||
}
|
||||
@@ -1,542 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.stages.checkOut
|
||||
|
||||
/*
|
||||
Function to define the flow of entire build, this function will call different stages related to maven build
|
||||
*/
|
||||
def run(Map config) {
|
||||
// get required variables from config
|
||||
def checkoutObj = new checkOut()
|
||||
def repo_name = config.repo_name
|
||||
def args = config.build_args ?: ''
|
||||
def skip_test = config.skip_test ?: false
|
||||
def skip_sonar = config.skip_sonar ?: false
|
||||
def push_to_jfrog = config.push_to_jfrog ?: false
|
||||
def push_to_s3 = config.push_to_s3 ?: false
|
||||
def branch_name = "${env.BRANCH_NAME}"
|
||||
if (branch_name == 'gcp-main' || branch_name == 'gcp-master') {
|
||||
push_to_jfrog = config.containsKey('push_to_jfrog') ? config.push_to_jfrog : false
|
||||
}
|
||||
def version = getVersion("${repo_name}")
|
||||
|
||||
if (env.hot_fix) {
|
||||
skip_test = true
|
||||
skip_sonar = true
|
||||
push_to_jfrog = false
|
||||
}
|
||||
|
||||
// call the stages
|
||||
checkS3(repo_name, branch_name, version, push_to_s3)
|
||||
build(repo_name, skip_test, args)
|
||||
sonar_scan(repo_name, skip_sonar)
|
||||
if (!env.CHANGE_ID) {
|
||||
pushArtifactToJFrog(repo_name, push_to_jfrog)
|
||||
pushArtifactToS3(repo_name, branch_name, push_to_s3)
|
||||
}
|
||||
else {
|
||||
log.info('Artifact Push is disabled for Pull Requests')
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Function to check if the artifact exist in s3 or not
|
||||
input argument
|
||||
*/
|
||||
def checkS3(String repo_name, String branch_name, String version, boolean push_to_s3) {
|
||||
stage(stageName('Checking if artifact already exists')) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
if (env.hot_fix) {
|
||||
env.TAG = "v${version}-HOT"
|
||||
}
|
||||
else {
|
||||
if (("${branch_name}" == 'master' || "${branch_name}" == 'main' || push_to_s3) && !env.CHANGE_ID) {
|
||||
env.TAG = "v${version}"
|
||||
log.info('########################## Checking if Artifact Already Exists. ###########################')
|
||||
artifact_exists = sh(returnStdout: true, script: "aws s3 ls \"s3://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/\" 2>/dev/null || echo ''").trim()
|
||||
log.info("${artifact_exists}")
|
||||
if ( artifact_exists ) {
|
||||
env.msg = "CI for this\n version - ${version}\n branch - ${branch_name}\n TAG - ${TAG}\nis already done. Please proceed with CD."
|
||||
log.error(env.msg)
|
||||
currentBuild.result = 'FAILURE'
|
||||
throw new Exception(env.msg)
|
||||
}
|
||||
else {
|
||||
log.info("Proceeding with building artifact for TAG - ${TAG}.")
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info('Skipping - Building artifact')
|
||||
}
|
||||
}
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if (env.hot_fix) {
|
||||
env.TAG = "v${version}-HOT"
|
||||
}
|
||||
else {
|
||||
if (("${branch_name}" == 'master' || "${branch_name}" == 'main' || "${branch_name}" == 'gcp-main' || "${branch_name}" == 'gcp-master' || push_to_s3) && !env.CHANGE_ID) {
|
||||
env.TAG = "v${version}"
|
||||
log.info('########################## Checking if Artifact Already Exists. ###########################')
|
||||
artifact_exists = sh(returnStdout: true, script: "gsutil ls \"gs://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/\" 2>/dev/null || echo ''").trim()
|
||||
log.info("${artifact_exists}")
|
||||
if ( artifact_exists ) {
|
||||
env.msg = "CI for this\n version - ${version}\n branch - ${branch_name}\n TAG - ${TAG}\nis already done. Please proceed with CD."
|
||||
log.error(env.msg)
|
||||
currentBuild.result = 'FAILURE'
|
||||
throw new Exception(env.msg)
|
||||
}
|
||||
else {
|
||||
log.info("Proceeding with building artifact for TAG - ${TAG}.")
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info('Skipping - Building artifact.')
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Function to build the maven package
|
||||
input arguments:
|
||||
repo_name: repository name for changing dirctory
|
||||
skip_test: boolen value to skip unit tests
|
||||
skip_sonar: boolen value to skip sonar quality gate
|
||||
args: string parameter to provide additional arguments to build cmd, example: '-U'
|
||||
*/
|
||||
def build(String repo_name, boolean skip_test, String args) {
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
stage(stageName('Building gradle package')) {
|
||||
String build_cmd = 'gradle clean build'
|
||||
dir("$repo_name") {
|
||||
sh(script:"cp ~/.m2/settings.xml .;${build_cmd} ${args}")
|
||||
}
|
||||
}
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
stage(stageName('Building gradle package')) {
|
||||
String build_cmd = 'gradle clean build'
|
||||
dir("$repo_name") {
|
||||
sh(script:"gradle -v;cp ~/.m2/settings.xml .;${build_cmd} ${args}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch ( Exception e) {
|
||||
env.msg = "Error Building the maven package. Please check console output for more details - ${e}"
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
/*
|
||||
Fuction to execute sonar scan
|
||||
Input Arguments:
|
||||
repo_name: repository name
|
||||
skip_sonar: boolena parameter to skip sonar scan
|
||||
*/
|
||||
def sonar_scan(String repo_name, boolean skip_sonar) {
|
||||
try {
|
||||
stage(stageName('Running sonar scan')) {
|
||||
if (!skip_sonar) {
|
||||
dir("$repo_name") {
|
||||
withSonarQubeEnv(env.sonarEnv) {
|
||||
if (env.CHANGE_ID) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "gradle sonar:sonar -Dsonar.pullrequest.provider=GitHub -Dsonar.pullrequest.github.repository=Homelab/${repo_name} -Dsonar.pullrequest.key=${env.CHANGE_ID} -Dsonar.pullrequest.branch=${env.CHANGE_BRANCH} -Dsonar.pullrequest.base=${env.CHANGE_TARGET}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script: "gradle sonar:sonar -Dsonar.pullrequest.provider=GitHub -Dsonar.pullrequest.github.repository=Homelab/${repo_name} -Dsonar.pullrequest.key=${env.CHANGE_ID} -Dsonar.pullrequest.branch=${env.CHANGE_BRANCH} -Dsonar.pullrequest.base=${env.CHANGE_TARGET}")
|
||||
}
|
||||
}
|
||||
else {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "gradle sonar:sonar -Dsonar.branch.name=${env.BRANCH_NAME}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script: "gradle sonar:sonar -Dsonar.branch.name=${env.BRANCH_NAME}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info('Skipping - Sonar Scan')
|
||||
}
|
||||
}
|
||||
// if (!skip_sonar) {
|
||||
// stage('Quality Gate') {
|
||||
// timeout(time: 300, unit: 'SECONDS') {
|
||||
// def qg = waitForQualityGate()
|
||||
// catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE') {
|
||||
// if (qg.status != 'OK') {
|
||||
// error "stage failed due to quality gate failure: ${qg.status}"
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
}
|
||||
catch (Exception e) {
|
||||
dir("$repo_name") {
|
||||
withSonarQubeEnv(env.sonarEnv) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: 'gradle sonar:sonar')
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script: 'gradle sonar:sonar')
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
/*
|
||||
Function to get the version from build.gradle
|
||||
input arguments:
|
||||
repo_name: String parameter to change the directory where build.gradle is located
|
||||
*/
|
||||
def getVersion(String repo_name) {
|
||||
try {
|
||||
dir("$repo_name") {
|
||||
return sh(returnStdout: true, script: "grep version build.gradle | head -1 | cut -d \"'\" -f2").trim()
|
||||
}
|
||||
}
|
||||
catch ( Exception e) {
|
||||
env.msg = 'Error while getting the version from build.gradle . Please check console output for more details.'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Function to get the modules from build.gradle
|
||||
input arguments:
|
||||
repo_name: String parameter to change the directory where build.gradle is located
|
||||
*/
|
||||
def getModules(String repo_name) {
|
||||
try {
|
||||
dir("$repo_name") {
|
||||
modules = sh(returnStdout: true, script: 'xq -r .project.modules.module[] build.gradle 2>/dev/null || xq -r .project.modules.module build.gradle 2>/dev/null || echo empty').trim()
|
||||
if (modules == 'empty' || modules == 'null') {
|
||||
modules = 'module_less'
|
||||
}
|
||||
modules = modules.split('\n') as List
|
||||
return modules
|
||||
}
|
||||
}
|
||||
catch ( Exception e) {
|
||||
env.msg = 'Error getting the modules from build.gradle . Please check console output for more details.'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Function to push artifacts to jfrog artifactory
|
||||
input arguments:
|
||||
repo_name: repository name
|
||||
push_to_jfrog: boolen argument to push_to_jfrog
|
||||
*/
|
||||
def pushArtifactToJFrog(String repo_name, boolean push_to_jfrog) {
|
||||
try {
|
||||
stage(stageName('Deploying to JFrog')) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
branch_name = 'repo'
|
||||
if (branch_name == 'master' || branch_name == 'main' || push_to_jfrog) {
|
||||
dir("${repo_name}") {
|
||||
profiles = sh(returnStdout:true, script: 'xq -r .project.profiles build.gradle ').trim()
|
||||
block_dist = sh(returnStdout:true, script: 'xq -r .project.distributionManagement build.gradle ').trim()
|
||||
// Check for profiles tag in the build.gradle
|
||||
if (profiles != 'null') {
|
||||
// Check if profiles has distiributionManagement defined
|
||||
profile_dist = sh(returnStdout:true, script: 'xq -r .project.profiles.profile[].distributionManagement build.gradle 2>/dev/null || xq -r .project.profiles.profile.distributionManagement build.gradle 2>/dev/null || echo null').trim()
|
||||
if (profile_dist != 'null' ) {
|
||||
publish_repo = (branch_name == 'master' || branch_name == 'main' || branch_name == 'gcp-main' || branch_name == 'gcp-master') ? 'useProdRepo' : 'useTestRepo'
|
||||
log.info('########################### Pushing artifact to Jfrog. ###########################')
|
||||
sh "gradle package deploy -DskipTests=true -D${publish_repo}=true"
|
||||
}
|
||||
else {
|
||||
log.info('distributionManagement is not defined in the build.gradle . Skipping - Push to Jfrog Artifactory')
|
||||
}
|
||||
}
|
||||
else if (block_dist != 'null') {
|
||||
// Check if distributionManagement is defined without profiles
|
||||
log.info('########################### Pushing artifact to Jfrog. ###########################')
|
||||
sh 'gradle package deploy -DskipTests=true'
|
||||
}
|
||||
else {
|
||||
log.info('distributionManagement is not defined in the build.gradle . Skipping - Push to Jfrog Artifactory')
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info('########################### Skipping - Push to Jfrog Artifactory. ###########################')
|
||||
}
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
branch_name = 'repo'
|
||||
if (branch_name == 'master' || branch_name == 'main' || push_to_jfrog) {
|
||||
dir("${repo_name}") {
|
||||
profiles = sh(returnStdout:true, script: 'xq -r .project.profiles build.gradle ').trim()
|
||||
block_dist = sh(returnStdout:true, script: 'xq -r .project.distributionManagement build.gradle ').trim()
|
||||
// Check for profiles tag in the build.gradle
|
||||
if (profiles != 'null') {
|
||||
// Check if profiles has distiributionManagement defined
|
||||
profile_dist = sh(returnStdout:true, script: 'xq -r .project.profiles.profile[].distributionManagement build.gradle 2>/dev/null || xq -r .project.profiles.profile.distributionManagement build.gradle 2>/dev/null || echo null').trim()
|
||||
if (profile_dist != 'null' ) {
|
||||
publish_repo = (branch_name == 'master' || branch_name == 'main' || branch_name == 'gcp-main' || branch_name == 'gcp-master') ? 'useProdRepo' : 'useTestRepo'
|
||||
log.info('########################### Pushing artifact to Jfrog. ###########################')
|
||||
sh "gradle package deploy -DskipTests=true -D${publish_repo}=true"
|
||||
echo 'execute gradle dependency'
|
||||
sh 'gradle dependency:list > dependency_tree.txt'
|
||||
echo ' uploading gradle dependency'
|
||||
sh 'ls -ltr'
|
||||
echo 'lets run copy command'
|
||||
try {
|
||||
sh "gsutil cp dependency_tree.txt 'gs://${env.objBucket}/common-dependencies/${repo_name}/'"
|
||||
}
|
||||
catch ( Exception e ) {
|
||||
echo "Skipping - copying txt file to GCS - ${e}"
|
||||
}
|
||||
echo 'uploading gradle dependency'
|
||||
}
|
||||
else {
|
||||
log.info('distributionManagement is not defined in the build.gradle . Skipping - Push to Jfrog Artifactory')
|
||||
}
|
||||
}
|
||||
else if (block_dist != 'null') {
|
||||
// Check if distributionManagement is defined without profiles
|
||||
log.info('########################### Pushing artifact to Jfrog. ###########################')
|
||||
sh 'gradle package deploy -DskipTests=true'
|
||||
}
|
||||
else {
|
||||
log.info('distributionManagement is not defined in the build.gradle . Skipping - Push to Jfrog Artifactory')
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info('########################### Skipping - Push to Jfrog Artifactory. ###########################')
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch ( Exception e ) {
|
||||
env.msg = 'Error in pushing artifacts to jfrog . Please check console output for more details.'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Function to push artifacts to s3
|
||||
input arguments:
|
||||
repo_name: repository name
|
||||
push_to_s3: boolen argument to push to s3
|
||||
*/
|
||||
def pushArtifactToS3(String repo_name, String branch_name, boolean push_to_s3) {
|
||||
try {
|
||||
stage(stageName('Pushing artifacts to object storage')) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
if ("${branch_name}" == 'master' || "${branch_name}" == 'main' || push_to_s3) {
|
||||
def modules = getModules("${repo_name}")
|
||||
log.info('}########################### Pushing artifacts to S3. ###########################')
|
||||
j = 0
|
||||
for (module in modules) {
|
||||
j += 1
|
||||
sh """
|
||||
echo "${j}. ${module}"
|
||||
if [ -f ${repo_name}/${module}/target/*.jar ]
|
||||
then
|
||||
ls -al ${repo_name}/${module}/target/*.jar
|
||||
echo "Uploading artifacts to - ${repo_name}/${branch_name}/${TAG}"
|
||||
aws s3 cp ${repo_name}/${module}/target/*.jar "s3://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/"
|
||||
echo "Listing ${TAG} artifacts -"
|
||||
aws s3 ls "s3://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/" || echo Nothing to display here.
|
||||
elif [ -f ${repo_name}/target/*.jar ] && [ ${module} = 'module_less' ]
|
||||
then
|
||||
ls -al ${repo_name}/target/*.jar
|
||||
echo "Uploading artifacts to - ${repo_name}/${branch_name}/${TAG}"
|
||||
aws s3 cp ${repo_name}/target/*.jar "s3://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/"
|
||||
echo "Uploading Archived Code to - ${repo_name}/${branch_name}/${TAG}"
|
||||
echo "Listing ${TAG} artifacts -"
|
||||
aws s3 ls "s3://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/" || echo Nothing to display here.
|
||||
else
|
||||
echo "No jar file found."
|
||||
fi
|
||||
"""
|
||||
}
|
||||
log.info("TAG for CD - ${TAG}")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping - Artifact push. As it is not supported for ${env.BRANCH_NAME}")
|
||||
}
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if ("${branch_name}" == 'master' || "${branch_name}" == 'main' || "${branch_name}" == 'gcp-main' || branch_name == 'gcp-master' || push_to_s3) {
|
||||
def modules = getModules("${repo_name}")
|
||||
log.info('}########################### Pushing artifacts to S3. ###########################')
|
||||
j = 0
|
||||
for (module in modules) {
|
||||
j += 1
|
||||
sh """
|
||||
echo "${j}. ${module}"
|
||||
if [ -f ${repo_name}/${module}/target/*.jar ]
|
||||
then
|
||||
ls -al ${repo_name}/${module}/target/*.jar
|
||||
echo "Uploading artifacts to - ${repo_name}/${branch_name}/${TAG}"
|
||||
gsutil cp ${repo_name}/${module}/target/*.jar "gs://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/"
|
||||
echo "Listing ${TAG} artifacts -"
|
||||
gsutil ls "gs://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/" || echo Nothing to display here.
|
||||
elif [ -f ${repo_name}/target/*.jar ] && [ ${module} = 'module_less' ]
|
||||
then
|
||||
ls -al ${repo_name}/target/*.jar
|
||||
echo "Uploading artifacts to - ${repo_name}/${branch_name}/${TAG}"
|
||||
gsutil cp ${repo_name}/target/*.jar "gs://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/"
|
||||
echo "Uploading Archived Code to - ${repo_name}/${branch_name}/${TAG}"
|
||||
echo "Listing ${TAG} artifacts -"
|
||||
gsutil ls "gs://${env.objBucket}/${repo_name}/${branch_name}/${TAG}/${module}/" || echo Nothing to display here.
|
||||
else
|
||||
echo "No jar file found."
|
||||
fi
|
||||
"""
|
||||
}
|
||||
log.info("TAG for CD - ${TAG}")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping - Artifact push. As it is not supported for ${env.BRANCH_NAME}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch ( Exception e ) {
|
||||
env.msg = "Error in pushing artifacts to s3 bucket. Please check console output for more details - ${e}"
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Fuction for java version
|
||||
*/
|
||||
def getDockerBuildVersion(String java_version) {
|
||||
switch (java_version) {
|
||||
case 'gradle': return '8-jdk-slim-secure-multiarch_v3.0'
|
||||
}
|
||||
}
|
||||
|
||||
/*
|
||||
Fuction to build maven docker repo
|
||||
*/
|
||||
|
||||
def buildDckr(Map config) {
|
||||
def btObj = new buTeamMapping()
|
||||
def dparam_obj = new getDockerParams()
|
||||
def constructObj = new constructTemplate()
|
||||
|
||||
def team = btObj.get_team_initials(config.team)
|
||||
def repo_name = config.repo_name
|
||||
def tag = dparam_obj.getTag(repo_name)
|
||||
def skip_test = config.skip_test ?: false
|
||||
def modules = getModules("${repo_name}")
|
||||
def excludedModules = config.excludedModules ?: []
|
||||
def deployArgo = config.deployArgo ?: false
|
||||
def repoType = config.repo_type ?: 'microservice'
|
||||
def docker_repo = "${env.cicd_environment}/${team}/${repo_name.toLowerCase()}"
|
||||
env.JAVA_HOME = '/usr/lib/jvm/java-8-openjdk-amd64/'
|
||||
env.PATH = "/opt/gradle/gradle-5.6.1/bin:${env.PATH}"
|
||||
|
||||
def docker_bindings = [
|
||||
'repo_name': repo_name,
|
||||
'buildRegistry': env.buildRegistry
|
||||
]
|
||||
if (config.containsKey('copy_file')) {
|
||||
// if (env.CLOUD_PROVIDER == "AWS"){
|
||||
// def recursive = config.copy_file.recursive ? ' --recursive' : ''
|
||||
// }
|
||||
// else if (env.CLOUD_PROVIDER == "GCP"){
|
||||
// def recursive = config.copy_file.recursive ? ' -r' : ''
|
||||
// }
|
||||
def recursive = config.copy_file.recursive ? ' --recursive' : ''
|
||||
dir(repo_name) {
|
||||
dir('copied_files') {
|
||||
sh(script:"aws s3 cp${recursive} ${config.copy_file.path} .")
|
||||
}
|
||||
}
|
||||
docker_bindings['copy_file'] = true
|
||||
docker_bindings['copy_target'] = config.copy_file.target ?: '/opt/target'
|
||||
}
|
||||
else {
|
||||
docker_bindings['copy_file'] = false
|
||||
}
|
||||
|
||||
if (modules != 'module_less') {
|
||||
excludedModules.each { modules.removeElement(it) }
|
||||
}
|
||||
else {
|
||||
modules = ['module_less']
|
||||
}
|
||||
docker_bindings['arch'] = config.arch
|
||||
def java_version = config.dockerBuildVersion
|
||||
dockerBuildVersion = getDockerBuildVersion(java_version)
|
||||
docker_bindings['dockerBuildVersion'] = dockerBuildVersion
|
||||
run(config)
|
||||
if (docker_bindings.copy_file) {
|
||||
dir(repo_name) {
|
||||
dir('copied_files') {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
// delete any old data
|
||||
sh(script:'rm -rf *')
|
||||
sh(script:"aws s3 cp${docker_bindings.recursive} ${docker_bindings.copy_file_path} .")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
// delete any old data
|
||||
docker_bindings['recursive'] = '-r'
|
||||
sh(script:'rm -rf *')
|
||||
sh(script:"gsutil cp ${docker_bindings.recursive} ${docker_bindings.copy_file_path}* .")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
stage(stageName('Building docker images')) {
|
||||
// Login to docker
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr get-login-password --region ${env.region} | docker login --username AWS --password-stdin ${env.registry}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script:'gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet')
|
||||
}
|
||||
dir(repo_name) {
|
||||
for (module in modules) {
|
||||
docker_bindings['module'] = module
|
||||
constructObj.renderTemplate(docker_bindings,'java-Dockerfile','Dockerfile-' + module)
|
||||
def module_repo = (module == 'module_less') ? docker_repo : docker_repo + '/' + module
|
||||
if ( module == 'module_less') {
|
||||
sh(script: 'mkdir target;cp build/libs/*jar target')
|
||||
}
|
||||
else {
|
||||
sh(script: "mkdir -p ${module}/target;cp build/libs/*jar ${module}/target")
|
||||
}
|
||||
if (env.cicd_environment != 'ftr' && repoType == 'microservice') {
|
||||
sh(script: "ls target; docker build --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module} . && docker push ${env.registry}/${module_repo}:${tag}")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker Build - ${env.cicd_environment} env")
|
||||
log.info("Skipping Docker Build - ${repoType} repo type")
|
||||
}
|
||||
//Remove dockerfile
|
||||
sh(script: "rm -rf Dockerfile-${module}")
|
||||
}
|
||||
}
|
||||
return [tag , deployArgo]
|
||||
}
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -1,568 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.utilities.addSSHKey
|
||||
import com.homelab.utilities.dockerUtilities
|
||||
import com.homelab.utilities.getYamlParameter
|
||||
|
||||
def removePackageLock() {
|
||||
sh(script: 'rm -rf package-lock.json')
|
||||
}
|
||||
|
||||
def buildNode(def build_cmd) {
|
||||
try {
|
||||
if (fileExists('package-lock.json')) {
|
||||
sh(script: 'npm ci')
|
||||
}
|
||||
else {
|
||||
sh(script: 'npm install')
|
||||
}
|
||||
sh(script: build_cmd)
|
||||
}
|
||||
catch ( Exception e ) {
|
||||
env.msg = 'Error in building node packages . Please check console output for more details.'
|
||||
env.error_msg_to_db = 'Error Building Node Packages'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def getArtifactId(String repo_name) {
|
||||
dir("$repo_name") {
|
||||
if (fileExists('package.json')) {
|
||||
return sh(returnStdout: true, script: 'jq -r .name package.json').trim()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def getAwsSecret(def secret_name, def destination_file, def team, def bu) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws secretsmanager get-secret-value --secret-id ${secret_name} --query SecretString --output text > ${destination_file}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
getVaultSecret("vault kv get -format=json homelab/${env.cicd_environment}/${bu}/${team}/${secret_name} | jq -r .data.data > ${destination_file}")
|
||||
}
|
||||
}
|
||||
|
||||
def getNpmRc(def secret_name, def team, def bu) {
|
||||
def npmrc_file = secret_name + '-npmrc-' + env.cicd_environment
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws secretsmanager get-secret-value --secret-id ${npmrc_file} --query SecretString --output text| jq -r .HOMELAB_NPMRC_SECRET > .npmrc")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
getVaultSecret("vault kv get -format=json homelab/${env.cicd_environment}/${bu}/${team}/${npmrc_file} | jq -r .data.data.HOMELAB_NPMRC_SECRET > .npmrc")
|
||||
}
|
||||
}
|
||||
|
||||
def getPemFile(def secret_name, def team, def bu) {
|
||||
def pem_secret_name = secret_name + '-secrets-' + env.cicd_environment
|
||||
if (env.BUILD_ENV == 'stage') {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws secretsmanager get-secret-value --secret-id ${pem_secret_name} --query SecretString --output text| jq -r .public_secret_dev > 1_public_secret_dev.pem")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
getVaultSecret("vault kv get -format=json homelab/${env.cicd_environment}/${bu}/${team}/${pem_secret_name} | jq -r .data.data.public_secret_dev > 1_public_secret_dev.pem")
|
||||
}
|
||||
sh(script:"cat 1_public_secret_dev.pem | sed -e 's/-----BEGIN PUBLIC KEY-----/& \\n/' -e 's/-----END PUBLIC KEY-----/\\n-----END PUBLIC KEY-----/g' > public_secret_dev.pem")
|
||||
}
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws secretsmanager get-secret-value --secret-id ${pem_secret_name} --query SecretString --output text| jq -r .public_secret_prod > 1_public_secret_prod.pem")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
getVaultSecret("vault kv get -format=json homelab/${env.cicd_environment}/${bu}/${team}/${pem_secret_name} | jq -r .data.data.public_secret_prod > 1_public_secret_prod.pem")
|
||||
}
|
||||
|
||||
sh(script:"cat 1_public_secret_prod.pem | sed -e 's/-----BEGIN PUBLIC KEY-----/& \\n/' -e 's/-----END PUBLIC KEY-----/\\n-----END PUBLIC KEY-----/g' > public_secret_prod.pem")
|
||||
}
|
||||
|
||||
def getEnvFile(def secret_name, def team, def bu, boolean useCacPath = false) {
|
||||
def env_file = secret_name + '-env-' + env.cicd_environment
|
||||
def gcp_env_file = secret_name
|
||||
def destination_file = '.env'
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws secretsmanager get-secret-value --secret-id ${env_file} | jq --raw-output '.SecretString' | jq '.' | jq -r 'to_entries|map(\"\\(.key)=\\(.value|tostring)\")|.[]' > .env")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
def vault_path = useCacPath
|
||||
? "homelab/${env.cicd_environment}-cac/${bu}/${team}/${gcp_env_file}-client"
|
||||
: "homelab/${env.cicd_environment}/${bu}/${team}/${gcp_env_file}"
|
||||
getVaultSecret("vault kv get -format=json ${vault_path} | jq -r '.data.data | to_entries|map(\"\\(.key)=\\(.value|tostring)\")|.[]' > .env")
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Read appConfigEnabled from deployment yaml(s) in repo (deployments/<name>.yaml).
|
||||
* Uses config.deployment_order for deployment names, or repo_name if not set.
|
||||
* Aligns with deploy/Helm which reads appConfigEnabled from the same files.
|
||||
*/
|
||||
def isAppConfigEnabledFromDeployments(String repo_name, Map config) {
|
||||
def deploymentNames = (config.deployment_order instanceof List && !config.deployment_order.isEmpty())
|
||||
? config.deployment_order
|
||||
: [repo_name]
|
||||
def yamlObj = new getYamlParameter()
|
||||
for (def deployment in deploymentNames) {
|
||||
try {
|
||||
def depYaml = yamlObj.getParam(repo_name, "deployments/${deployment}.yaml")
|
||||
def enabled = depYaml?.appConfigEnabled
|
||||
if (enabled == true || enabled?.toString()?.equalsIgnoreCase('true')) {
|
||||
return true
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.debug("No appConfigEnabled in ${repo_name}/deployments/${deployment}.yaml or file missing: ${e.message}")
|
||||
continue
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
def getManifestJson(def secret_name, def team, def bu) {
|
||||
def manifest_file = secret_name + '-manifest-' + env.cicd_environment
|
||||
def destination_file = 'public/manifest.json'
|
||||
getAwsSecret(manifest_file, destination_file, team, bu)
|
||||
}
|
||||
|
||||
def generateExcludePattern(String excludeS3Files) {
|
||||
def files = excludeS3Files.split(/\s*,\s*/) // Split the string by comma and trim whitespace
|
||||
return '|' + files.join('|') // Join file names with '|' as an "or" operator in regex
|
||||
}
|
||||
|
||||
def shouldSkipPm2Metrics(Map config) {
|
||||
return config.skip_pm2_metrics?.toString()?.toBoolean() ?: false
|
||||
}
|
||||
|
||||
/*
|
||||
Fuction to build maven docker repo
|
||||
*/
|
||||
|
||||
def getCommitSHA(String repo_name) {
|
||||
dir(repo_name) {
|
||||
return sh(returnStdout: true, script: 'git log -1 --format=%H').trim()
|
||||
}
|
||||
}
|
||||
|
||||
def buildDckr(Map config) {
|
||||
// // Block all Node.js builds - throwing exception
|
||||
// throw new Exception("Node.js builds are currently blocked, As a precautionary measure. Due to some packages that got hacked.")
|
||||
def btObj = new buTeamMapping()
|
||||
def constructObj = new constructTemplate()
|
||||
def dparam_obj = new getDockerParams()
|
||||
def addSSHKey = new addSSHKey()
|
||||
def dockerUtilObj = new dockerUtilities()
|
||||
|
||||
def team = btObj.get_team_initials(config.team)
|
||||
def bu = btObj.get_bu_initials(config.bu)
|
||||
def repo_name = config.repo_name
|
||||
// CAC: read appConfigEnabled from deployment folder (deployments/<name>.yaml), same source as deploy/Helm
|
||||
def useCacPath = isAppConfigEnabledFromDeployments(repo_name, config)
|
||||
def build_cmd = config.build_cmd ?: 'npm run build'
|
||||
// Resolve secret_name from deployment YAML app_name.
|
||||
// For multi-deployment repos, env/secret files are kept in sync so the first valid app_name is used.
|
||||
def secret_name = repo_name
|
||||
def keep_package_lock = config.keep_package_lock != null ? config.keep_package_lock : true
|
||||
def skip_npmrc = config.skip_npmrc != null ? config.skip_npmrc : true
|
||||
if (config.secret_name) {
|
||||
secret_name = config.secret_name
|
||||
} else {
|
||||
try {
|
||||
def yamlObj = new getYamlParameter()
|
||||
def deploymentsPath = "${repo_name}/deployments"
|
||||
def yamlFilesOutput = sh(
|
||||
script: "ls ${deploymentsPath}/*.yaml 2>/dev/null | xargs -r -n1 basename",
|
||||
returnStdout: true
|
||||
).trim()
|
||||
if (yamlFilesOutput) {
|
||||
def yamlFiles = yamlFilesOutput.split('\n').collect { it.trim() }.findAll { it }
|
||||
for (yamlFile in yamlFiles) {
|
||||
def appConfig = yamlObj.getParam(deploymentsPath, yamlFile)
|
||||
def appName = appConfig?.app_name?.toString()?.trim()
|
||||
if (appName) {
|
||||
echo "Resolved secret_name to ${appName} from ${deploymentsPath}/${yamlFile}"
|
||||
secret_name = appName
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
if (secret_name == repo_name) {
|
||||
echo "No app_name found in deployment yamls; using repo_name: ${repo_name}"
|
||||
}
|
||||
} catch (Exception e) {
|
||||
echo "Error reading deployment yaml: ${e.getMessage()}; using repo_name: ${repo_name}"
|
||||
}
|
||||
}
|
||||
def require_mainfest = config.require_mainfest ?: false
|
||||
def pbac_enabled = config.pbac_enabled ?: false
|
||||
def pbac_scope_name = config.pbac_scope_name ?: ''
|
||||
def npm_install_arg = config.npm_install_arg ?: ''
|
||||
if (!npm_install_arg) {
|
||||
if (fileExists("${repo_name}/pnpm-lock.yaml") || build_cmd.contains('pnpm')) {
|
||||
npm_install_arg = 'npm install -g pnpm@10.33.0 && pnpm install --frozen-lockfile'
|
||||
}
|
||||
else if (!keep_package_lock || !fileExists("${repo_name}/package-lock.json")) {
|
||||
npm_install_arg = 'npm install'
|
||||
}
|
||||
// else: package-lock.json exists + keep_package_lock=true → Dockerfile uses `npm ci`
|
||||
}
|
||||
def require_pemfiles = config.require_pemfiles ?: false
|
||||
def deployArgo = config.deployArgo ?: false
|
||||
def artifactId = getArtifactId("${repo_name}")
|
||||
// def region = dparam_obj.getRegion(env.cicd_environment)
|
||||
// def registry = dparam_obj.getRegistry(env.cicd_environment)
|
||||
def docker_repo = "${env.cicd_environment}/${team}/${repo_name.toLowerCase()}"
|
||||
def push_to_s3 = config.push_to_s3 ?: false
|
||||
def s3_path = config.s3_path ?: "homelab-${env.BUILD_ENV}-artifacts/${repo_name}/${env.BRANCH_NAME}"
|
||||
if (env.INFRA_ENV == 'toolchain' && env.TOOLCHAIN_ENV) {
|
||||
docker_repo = "${env.cicd_environment}/${env.TOOLCHAIN_ENV}/${team}/${repo_name.toLowerCase()}"
|
||||
log.info("Toolchain: Overriding docker_repo to ${docker_repo}")
|
||||
echo "docker_repo: ${docker_repo}"
|
||||
|
||||
if (config.s3_path && config.push_to_s3) {
|
||||
echo" {env.cicd_environment} should be stg due ot ovveride"
|
||||
s3_path = s3_path.replaceFirst("/${env.cicd_environment}/", "/${env.TOOLCHAIN_ENV}/")
|
||||
log.info("TOOLCHAIN OVERRIDE:")
|
||||
log.info(" Original S3 Path: ${config.s3_path}")
|
||||
log.info(" New Toolchain S3 Path: ${s3_path}")
|
||||
log.info(" Toolchain Env ID: ${env.TOOLCHAIN_ENV}")
|
||||
|
||||
} else if (!config.push_to_s3) {
|
||||
log.info("Toolchain: push_to_s3 is false, skipping s3_path override as it wont be pushed")
|
||||
} else if (!config.s3_path && config.push_to_s3) {
|
||||
log.info("Toolchain: s3_path is not set but push_to_s3 is true, skipping s3_path override as it wont be pushed")
|
||||
error("CRITICAL: 's3_path' is missing in config.yaml for the ${env.cicd_environment} environment. Toolchain builds require an s3_path.")
|
||||
}
|
||||
}
|
||||
if (env.TOOLCHAIN_ENV) {
|
||||
log.info("TOOLCHAIN_ENV: ${env.TOOLCHAIN_ENV}")
|
||||
}else{
|
||||
log.info("TOOLCHAIN_ENV: not set")
|
||||
}
|
||||
def local_path = config.local_path ?: 'build/'
|
||||
def acl = config.acl ? ' --acl ' + config.acl : ''
|
||||
def custom_pm2_metrics = config.custom_pm2_metrics ?: false
|
||||
def skip_pm2_metrics = shouldSkipPm2Metrics(config)
|
||||
def include_s3_files = ''
|
||||
def exclude_s3_files = ''
|
||||
if (!s3_path.endsWith('/')) {
|
||||
s3_path += '/'
|
||||
}
|
||||
if (!local_path.endsWith('/')) {
|
||||
local_path += '/'
|
||||
}
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
if (config.include_s3_files) {
|
||||
include_s3_files = ' --exclude "*"'
|
||||
def include_file_map = config.include_s3_files.split(',')
|
||||
for (pattern in include_file_map) {
|
||||
include_s3_files += ' --include "'+pattern+'"'
|
||||
}
|
||||
}
|
||||
exclude_s3_files = config.exclude_s3_files ? ' --exclude "'+config.exclude_s3_files+'"' : ''
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if (config.include_s3_files) {
|
||||
def includeFileMap = config.include_s3_files.split(',')
|
||||
def file_types = ''
|
||||
def i = 0
|
||||
includeFileMap.each { pattern ->
|
||||
if ( i == 0 ) {
|
||||
file_types = "${pattern.replaceAll('\\*.', '')}"
|
||||
}
|
||||
else {
|
||||
file_types += "|${pattern.replaceAll('\\*.', '')}"
|
||||
}
|
||||
i++
|
||||
}
|
||||
if ( file_types == '*' ) {
|
||||
include_s3_files = '^(?!.*\\.*$)'
|
||||
}
|
||||
else {
|
||||
include_s3_files = "^(?!.*\\.(${file_types})\$)"
|
||||
}
|
||||
echo "include_s3_files is ${include_s3_files}"
|
||||
}
|
||||
else {
|
||||
include_s3_files = '^(?!.*\\.*$)'
|
||||
}
|
||||
|
||||
exclude_s3_files = config.exclude_s3_files != null && config.exclude_s3_files.trim()? generateExcludePattern(config.exclude_s3_files.trim()) : ''
|
||||
|
||||
}
|
||||
def phantomjs = config.phantomjs ?: false
|
||||
def version = config.dockerBuildVersion.split('-')[-1]
|
||||
def tag = dparam_obj.getTag(repo_name)
|
||||
// TODO: create value binding for dockerfile render
|
||||
|
||||
def skip_sonar = config.skip_sonar ?: false
|
||||
if (env.hot_fix || env.INFRA_ENV == 'toolchain') {
|
||||
skip_sonar = true
|
||||
}
|
||||
def testCMD = config.testCMD ?: 'test-report'
|
||||
def scmType = 'branch'
|
||||
def commit_sha = getCommitSHA(config.repo_name)
|
||||
def docker_bindings = [
|
||||
'buildRegistry': env.buildRegistry,
|
||||
'version': version,
|
||||
'build_cmd': build_cmd,
|
||||
'push_to_s3': push_to_s3,
|
||||
's3_path': s3_path,
|
||||
'local_path': local_path,
|
||||
'include_s3_files': include_s3_files,
|
||||
'exclude_s3_files': exclude_s3_files,
|
||||
'acl': acl,
|
||||
'phantomjs': phantomjs,
|
||||
'skip_sonar': skip_sonar,
|
||||
'testCMD': testCMD,
|
||||
'CLOUD_PROVIDER': env.CLOUD_PROVIDER,
|
||||
'npm_install_arg': npm_install_arg
|
||||
]
|
||||
docker_bindings['arch'] = config.arch
|
||||
docker_bindings['pbac_enabled'] = pbac_enabled
|
||||
docker_bindings['pbac_scope_name'] = pbac_scope_name ?: ''
|
||||
// Map BUILD_ENV to pbac format (stg/int/prd)
|
||||
def pbac_env = env.cicd_environment
|
||||
docker_bindings['pbac_env'] = pbac_env
|
||||
docker_bindings['useCacPath'] = useCacPath
|
||||
|
||||
// Validate pbac configuration
|
||||
if (pbac_enabled && !pbac_scope_name) {
|
||||
error('pbac_scope_name is required when pbac_enabled is true in config.yaml')
|
||||
}
|
||||
|
||||
stage(stageName('Creating build files')) {
|
||||
dir(repo_name) {
|
||||
if (!keep_package_lock) {
|
||||
removePackageLock()
|
||||
}
|
||||
if (!skip_npmrc) {
|
||||
getNpmRc(secret_name, team, bu)
|
||||
}
|
||||
|
||||
getEnvFile(secret_name, team, bu, useCacPath)
|
||||
|
||||
if (require_pemfiles) {
|
||||
getPemFile(secret_name, team, bu)
|
||||
}
|
||||
if (require_mainfest) {
|
||||
getManifestJson(secret_name, team, bu)
|
||||
}
|
||||
def npm_registry = ''
|
||||
if (fileExists('.npmrc')) {
|
||||
npm_registry = sh(
|
||||
returnStdout: true,
|
||||
script: "grep '^@homelab:registry=' .npmrc | head -1 | cut -d'=' -f2 | tr -d '\\r\\n'"
|
||||
).trim()
|
||||
echo "Detected npm registry: ${npm_registry ?: 'default (none found)'}"
|
||||
} else {
|
||||
echo ".npmrc not found, skipping registry extraction"
|
||||
}
|
||||
docker_bindings['npm_registry'] = npm_registry
|
||||
}
|
||||
}
|
||||
|
||||
if (skip_sonar){
|
||||
stage(stageName('Checking quality gate')){
|
||||
log.info("Skipping the quality gate check as sonar scan is skipped or this is a hotfix ")
|
||||
}
|
||||
}
|
||||
stage(stageName('Building docker images')) {
|
||||
// Login to docker
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr get-login-password --region ${env.region} | docker login --username AWS --password-stdin ${env.registry}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script:'gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet')
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in Docker login'
|
||||
env.error_msg_to_db = 'Error in Docker login'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr describe-repositories --region ${env.region} --repository-names ${docker_repo} || aws ecr create-repository --region ${env.region} --repository-name ${docker_repo}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
echo 'Skipping - Registry Creation in GCP.'
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "Error in creating ECR repository ${docker_repo}"
|
||||
env.error_msg_to_db = "Error in creating ECR repository ${docker_repo}"
|
||||
currentBuild.result = 'FAILURE'
|
||||
}
|
||||
try {
|
||||
dir(repo_name) {
|
||||
addSSHKey.create()
|
||||
withCredentials([usernamePassword(credentialsId: "svc-devops-homelab-token", usernameVariable: 'GIT_USER', passwordVariable: 'GIT_TOKEN')]) {
|
||||
sh """
|
||||
echo "GITHUB_TOKEN=${GIT_TOKEN}" >> .env
|
||||
echo "GIT_COMMIT_SHA='${commit_sha}'" >> .env
|
||||
"""
|
||||
}
|
||||
withCredentials([string(credentialsId: env.sonarToken, variable: 'TOKEN')]) {
|
||||
sh """
|
||||
echo "SONAR_HOST_URL='${env.sonarURL}'" >> .env
|
||||
echo "SONAR_TOKEN='${TOKEN}'" >> .env
|
||||
echo "SONAR_WS_TIMEOUT=120" >> .env
|
||||
"""
|
||||
if (env.CHANGE_ID) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh "git fetch origin ${env.CHANGE_TARGET}:refs/remotes/origin/${env.CHANGE_TARGET}"
|
||||
}
|
||||
sh """
|
||||
echo "SONAR_CHANGE_ID='${env.CHANGE_ID}'" >> .env
|
||||
echo "SONAR_CHANGE_BRANCH='${env.CHANGE_BRANCH}'" >> .env
|
||||
echo "SONAR_CHANGE_TARGET='${env.CHANGE_TARGET}'" >> .env
|
||||
"""
|
||||
scmType = 'pr'
|
||||
}
|
||||
else {
|
||||
sh """
|
||||
echo "SONAR_BRANCH_NAME='${env.BRANCH_NAME}'" >> .env
|
||||
"""
|
||||
}
|
||||
echo 'Make sure to update script section in package.json for Sonar Analysis to be successfull.'
|
||||
docker_bindings['scmType'] = scmType
|
||||
}
|
||||
if (!fileExists('Dockerfile')) {
|
||||
constructObj.renderTemplate(docker_bindings, 'node-Dockerfile', 'Dockerfile-' + artifactId)
|
||||
sh "cat Dockerfile-${artifactId}"
|
||||
if (push_to_s3) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh """
|
||||
echo 'FROM amazon/aws-cli:2.2.0 as push_env' >> Dockerfile-${artifactId}
|
||||
echo 'COPY --from=build-env /usr/src/app /app' >> Dockerfile-${artifactId}
|
||||
echo 'RUN ls -al && aws s3 cp /app/${local_path} s3://${s3_path}${include_s3_files}${exclude_s3_files} --recursive --cache-control max-age=31536000,public${acl}' >> Dockerfile-${artifactId}
|
||||
"""
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP' ) {
|
||||
sh """
|
||||
echo 'FROM asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/google-cloud-sdk:458.0.0-alpine as push_env' >> Dockerfile-${artifactId}
|
||||
echo 'COPY --from=build-env /usr/src/app /app' >> Dockerfile-${artifactId}
|
||||
# echo 'RUN ls -al /app && find /app/${local_path} -type f \\( -name "*.js" -o -name "*.map" \\) && find /app/${local_path} -type f \\( -name "*.js" -o -name "*.map" \\) | wc -l && gsutil -m cp -r /app/${local_path}* gs://${s3_path}' >> Dockerfile-${artifactId}
|
||||
echo "RUN ls -al /app && gsutil -m rsync -r -x \'${include_s3_files} ${exclude_s3_files}\' /app/${local_path} gs://${s3_path}" >> Dockerfile-${artifactId}
|
||||
echo "RUN if [ -f /app/${local_path}index.html ]; then gsutil cp /app/${local_path}index.html gs://${s3_path}; fi" >> Dockerfile-${artifactId}
|
||||
"""
|
||||
}
|
||||
}
|
||||
|
||||
if (phantomjs) {
|
||||
sh """
|
||||
echo 'FROM build-env' >> Dockerfile-${artifactId}
|
||||
echo 'RUN apt-get update && apt-get install -y libfontconfig' >> Dockerfile-${artifactId}
|
||||
echo 'WORKDIR /usr/src/app' >> Dockerfile-${artifactId}
|
||||
echo 'RUN npm install pm2 -g' >> Dockerfile-${artifactId}
|
||||
echo 'RUN pm2 install pm2-metrics' >> Dockerfile-${artifactId}
|
||||
if [ "${custom_pm2_metrics}" = "true" ]; then
|
||||
echo 'RUN pm2 install pm2-prom-module' >> Dockerfile-${artifactId}
|
||||
echo 'RUN pm2 set pm2-prom-module:port 9200' >> Dockerfile-${artifactId}
|
||||
echo 'RUN pm2 restart pm2-prom-module' >> Dockerfile-${artifactId}
|
||||
fi
|
||||
echo 'RUN rm -rf /root/.ssh/id_rsa && apt-get remove -y git openssh-client bzip2' >> Dockerfile-${artifactId}
|
||||
"""
|
||||
}
|
||||
else {
|
||||
sh """
|
||||
echo 'FROM ${env.buildRegistry}/build/node:${version}-alpine-secure-multiarch_v1.0' >> Dockerfile-${artifactId}
|
||||
echo 'RUN npm install pm2 -g' >> Dockerfile-${artifactId}
|
||||
if [ "${skip_pm2_metrics}" != "true" ]; then
|
||||
echo 'RUN pm2 install pm2-metrics' >> Dockerfile-${artifactId}
|
||||
fi
|
||||
if [ "${custom_pm2_metrics}" = "true" ]; then
|
||||
echo 'RUN pm2 install pm2-prom-module' >> Dockerfile-${artifactId}
|
||||
echo 'RUN pm2 set pm2-prom-module:port 9200' >> Dockerfile-${artifactId}
|
||||
echo 'RUN pm2 restart pm2-prom-module' >> Dockerfile-${artifactId}
|
||||
fi
|
||||
echo 'WORKDIR /app' >> Dockerfile-${artifactId}
|
||||
echo 'COPY --from=build-env /usr/src/app /app' >> Dockerfile-${artifactId}
|
||||
"""
|
||||
}
|
||||
if (docker_bindings['useCacPath']) {
|
||||
sh "echo 'RUN truncate -s 0 .env' >> Dockerfile-${artifactId}"
|
||||
}
|
||||
sh "cat Dockerfile-${artifactId}"
|
||||
def imageList = "${env.buildRegistry}/build/node:${version}-alpine-secure-multiarch_v1.0 " +
|
||||
"asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin/devops/google-cloud-sdk:458.0.0-alpine " +
|
||||
"${env.buildRegistry}/build/node:${version}-slim-secure-multiarch_v2.0"
|
||||
|
||||
sh """
|
||||
max_attempts=5
|
||||
images=\"$imageList\"
|
||||
|
||||
for image in \$images; do
|
||||
attempt=1
|
||||
until docker pull \"\$image\"; do
|
||||
if [ \$attempt -eq \$max_attempts ]; then
|
||||
echo \"Failed to pull \$image after \$attempt attempts.\"
|
||||
exit 1
|
||||
fi
|
||||
echo \"Pull failed for \$image, retrying in 2 seconds... (Attempt \$attempt/\$max_attempts)\"
|
||||
attempt=\$((attempt + 1))
|
||||
sleep 2
|
||||
done
|
||||
done
|
||||
"""
|
||||
if (env.cicd_environment != 'ftr' || env.INFRA_ENV == 'toolchain') {
|
||||
def docker_cmd = "export DOCKER_BUILDKIT=0;docker build --tag ${env.registry}/${docker_repo}:${tag} -f Dockerfile-${artifactId} . "
|
||||
sh(script: docker_cmd)
|
||||
dockerUtilObj.retryDockerPush("docker push ${env.registry}/${docker_repo}:${tag}")
|
||||
}
|
||||
else {
|
||||
def docker_cmd = "export DOCKER_BUILDKIT=0;docker build --tag ${env.registry}/${docker_repo}:${tag} -f Dockerfile-${artifactId} ."
|
||||
sh(script: docker_cmd)
|
||||
}
|
||||
//Remove dockerfile
|
||||
sh(script: "rm -rf Dockerfile-${artifactId}")
|
||||
}
|
||||
else {
|
||||
if (env.cicd_environment != 'ftr' || env.INFRA_ENV == 'toolchain') {
|
||||
sh(script: "export DOCKER_BUILDKIT=0;docker build --tag ${env.registry}/${docker_repo}:${tag} . ")
|
||||
dockerUtilObj.retryDockerPush("docker push ${env.registry}/${docker_repo}:${tag}")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker Push - ${env.cicd_environment} env")
|
||||
sh(script: "export DOCKER_BUILDKIT=0;docker build --tag ${env.registry}/${docker_repo}:${tag} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in building DockerFile Or Pushing To ECR'
|
||||
env.error_msg_to_db = env.msg
|
||||
log.error(env.msg + '\n' + e.toString())
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
return [tag , deployArgo]
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
def getVaultSecret(String vault_cmd) {
|
||||
log.info("Fetching secrets from Vault with vault-${env.cicd_environment}-token with CMD - ${vault_cmd}")
|
||||
if (env.INFRA_ENV == 'toolchain' && env.TOOLCHAIN_ENV) {
|
||||
boolean skipToolchainOverride = vault_cmd.contains("npmrc") || vault_cmd.contains("-secrets-")
|
||||
if (!skipToolchainOverride) {
|
||||
vault_cmd = vault_cmd.replace("homelab/${env.cicd_environment}-cac/", "homelab/toolchain/${env.TOOLCHAIN_ENV}/${env.cicd_environment}-cac/")
|
||||
vault_cmd = vault_cmd.replace("homelab/${env.cicd_environment}/", "homelab/toolchain/${env.TOOLCHAIN_ENV}/${env.cicd_environment}/")
|
||||
log.info("TOOLCHAIN OVERRIDE (Application Config):")
|
||||
log.info(" New Toolchain Vault CMD: ${vault_cmd}")
|
||||
} else {
|
||||
log.info("TOOLCHAIN: Reading from standard (non-toolchain) Vault path for npmrc / pem secret.")
|
||||
}
|
||||
}
|
||||
|
||||
withCredentials([string(credentialsId: "${env.vaultToken}", variable: 'TOKEN')]) {
|
||||
env.VAULT_ADDR = "${env.vaultURL}"
|
||||
env.VAULT_TOKEN = "${TOKEN}"
|
||||
sh(script:"${vault_cmd}")
|
||||
env.VAULT_TOKEN = 'empty'
|
||||
env.VAULT_ADDR = env.VAULT_TOKEN
|
||||
}
|
||||
}
|
||||
@@ -1,42 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
def run(String build_tool){
|
||||
try {
|
||||
switch (build_tool) {
|
||||
case 'maven':
|
||||
return new buildMaven()
|
||||
break;
|
||||
case 'docker':
|
||||
return new buildDocker()
|
||||
break;
|
||||
case ~/^maven-.*/:
|
||||
return new buildMaven()
|
||||
break;
|
||||
case ~/^python-.*/:
|
||||
return new buildPython()
|
||||
break;
|
||||
case ~/^node-.*/:
|
||||
return new buildNode()
|
||||
break;
|
||||
case ~/^rust.*/:
|
||||
return new buildRust()
|
||||
break;
|
||||
case ~/^go.*/:
|
||||
return new buildGo()
|
||||
break;
|
||||
case 'gradle':
|
||||
return new buildGradle()
|
||||
break;
|
||||
case 'php':
|
||||
return new buildPhp()
|
||||
default:
|
||||
return defaultBuild()
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in selecting the build tool (check the spell) Error: ' + e.toString()
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
@@ -1,104 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.getDockerParams
|
||||
|
||||
def buildDckr(Map config){
|
||||
def btObj = new buTeamMapping()
|
||||
def constructObj = new constructTemplate()
|
||||
def dparam_obj = new getDockerParams()
|
||||
|
||||
def team = btObj.get_team_initials(config.team)
|
||||
def repo_name = config.repo_name
|
||||
def deployArgo = config.deployArgo ?: false
|
||||
// def region = dparam_obj.getRegion(env.cicd_environment)
|
||||
// def registry = dparam_obj.getRegistry(env.cicd_environment)
|
||||
def docker_repo = "${env.cicd_environment}/${team}/${repo_name.toLowerCase()}"
|
||||
def tag = dparam_obj.getTag(repo_name)
|
||||
def docker_bindings = [
|
||||
"repo_name": repo_name,
|
||||
'buildRegistry': env.buildRegistry
|
||||
|
||||
]
|
||||
if (config.containsKey("copy_file")){
|
||||
if (env.CLOUD_PROVIDER == 'AWS'){
|
||||
def recursive = config.copy_file.recursive ? " --recursive" : ""
|
||||
dir(repo_name){
|
||||
dir('copied_files'){
|
||||
sh(script:"aws s3 cp${recursive} ${config.copy_file.path} .")
|
||||
}
|
||||
}
|
||||
} else if (env.CLOUD_PROVIDER == 'GCP'){
|
||||
def recursive = config.copy_file.recursive ? " -r" : ""
|
||||
dir(repo_name){
|
||||
dir('copied_files'){
|
||||
sh(script:"gsutil cp${recursive} ${config.copy_file.path} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
docker_bindings["copy_file"] = true
|
||||
docker_bindings["copy_target"] = config.copy_file.target ?: "/opt/target"
|
||||
}
|
||||
else{
|
||||
docker_bindings["copy_file"] = false
|
||||
}
|
||||
stage(stageName('Building docker images')){
|
||||
// Login to docker
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr get-login-password --region ${env.region} | docker login --username AWS --password-stdin ${env.registry}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script:'gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet')
|
||||
}
|
||||
}
|
||||
catch(Exception e) {
|
||||
env.msg = "Error in Docker login"
|
||||
env.error_msg_to_db = "Error in Docker login"
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
try{
|
||||
if (env.CLOUD_PROVIDER == "AWS"){
|
||||
sh(script:"aws ecr describe-repositories --region ${env.region} --repository-names ${docker_repo} || aws ecr create-repository --region ${env.region} --repository-name ${docker_repo}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP'){
|
||||
echo "Skipping - Registry Creation in GCP."
|
||||
}
|
||||
}
|
||||
catch(Exception e){
|
||||
env.msg = "Error in creating ECR repository ${docker_repo}"
|
||||
env.error_msg_to_db = "Error in creating ECR repository ${docker_repo}"
|
||||
currentBuild.result = "FAILURE"
|
||||
}
|
||||
try {
|
||||
dir(repo_name){
|
||||
if(!fileExists("Dockerfile")){
|
||||
constructObj.renderTemplate(docker_bindings,'php-Dockerfile','Dockerfile-php')
|
||||
if (env.cicd_environment != 'ftr') {
|
||||
sh(script: "docker build --tag ${env.registry}/${docker_repo}:${tag} -f Dockerfile-php . && docker push ${env.registry}/${docker_repo}:${tag}")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker builds for PHP in ${env.cicd_environment} env")
|
||||
}
|
||||
|
||||
//Remove dockerfile
|
||||
sh(script: "rm -rf Dockerfile-php")
|
||||
}
|
||||
else {
|
||||
sh(script: "docker build --tag ${env.registry}/${docker_repo}:${tag} . && docker push ${env.registry}/${docker_repo}:${tag}")
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "Error in building DockerFile Or Pushing To artifact registry"
|
||||
env.error_msg_to_db = "Error in building DockerFile Or Pushing To image registry"
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
return [tag , deployArgo]
|
||||
}
|
||||
@@ -1,221 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.utilities.addSSHKey
|
||||
import com.homelab.utilities.dockerUtilities
|
||||
/*
|
||||
Function to get the version from pom.xml
|
||||
input arguments:
|
||||
repo_name: String parameter to change the directory where pom.xml is located
|
||||
*/
|
||||
// def getArtifactId(String repo_name){
|
||||
// dir("$repo_name"){
|
||||
// if (fileExists("package.json")) {
|
||||
// return sh(returnStdout: true, script: 'jq -r .name package.json').trim()
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
|
||||
// def getAwsSecret(def secret_name, def destination_file){
|
||||
// sh (script:"aws secretsmanager get-secret-value --secret-id ${secret_name} --query SecretString --output text > ${destination_file}")
|
||||
// }
|
||||
|
||||
// def getCommitid(String repo_name){
|
||||
// dir("$repo_name"){
|
||||
// return sh(returnStdout: true, script: 'git log -1 --format=%h').trim()
|
||||
// }
|
||||
// }
|
||||
|
||||
// def getNpmRc(def secret_name){
|
||||
// def npmrc_file = secret_name + "-npmrc"
|
||||
// sh (script:"aws secretsmanager get-secret-value --secret-id ${npmrc_file} --query SecretString --output text| jq -r .HOMELAB_NPMRC_SECRET > .npmrc")
|
||||
// }
|
||||
|
||||
// def getPemFile(def secret_name){
|
||||
// def pem_secret_name = secret_name + "-secrets"
|
||||
// if(env.BUILD_ENV == 'stage'){
|
||||
// sh (script:"aws secretsmanager get-secret-value --secret-id ${pem_secret_name} --query SecretString --output text| jq -r .public_secret_dev > 1_public_secret_dev.pem")
|
||||
// sh (script:"cat 1_public_secret_dev.pem | sed -e 's/-----BEGIN PUBLIC KEY-----/& \\n/' -e 's/-----END PUBLIC KEY-----/\\n-----END PUBLIC KEY-----/g' > public_secret_dev.pem")
|
||||
// }
|
||||
// sh (script:"aws secretsmanager get-secret-value --secret-id ${pem_secret_name} --query SecretString --output text| jq -r .public_secret_prod > 1_public_secret_prod.pem")
|
||||
// sh (script:"cat 1_public_secret_prod.pem | sed -e 's/-----BEGIN PUBLIC KEY-----/& \\n/' -e 's/-----END PUBLIC KEY-----/\\n-----END PUBLIC KEY-----/g' > public_secret_prod.pem")
|
||||
// }
|
||||
|
||||
// def getEnvFile(def secret_name){
|
||||
// def env_file = secret_name + "-env"
|
||||
// def destination_file = ".env"
|
||||
// sh(script:"aws secretsmanager get-secret-value --secret-id ${env_file} | jq --raw-output '.SecretString' | jq '.' | jq -r 'to_entries|map(\"\\(.key)=\\(.value|tostring)\")|.[]' > .env")
|
||||
// }
|
||||
|
||||
// def getManifestJson(def secret_name){
|
||||
// def manifest_file = secret_name+"-manifest"
|
||||
// def destination_file = "public/manifest.json"
|
||||
// getAwsSecret(manifest_file,destination_file)
|
||||
// }
|
||||
/*
|
||||
Fuction to build maven docker repo
|
||||
*/
|
||||
|
||||
def buildDckr(Map config) {
|
||||
def btObj = new buTeamMapping()
|
||||
def constructObj = new constructTemplate()
|
||||
def dparam_obj = new getDockerParams()
|
||||
def addSSHKey = new addSSHKey()
|
||||
|
||||
def team = btObj.get_team_initials(config.team)
|
||||
def repo_name = config.repo_name
|
||||
def deployArgo = config.deployArgo ?: false
|
||||
def modules_requirements_file = config.modules_requirements_file ?: 'requirements.txt'
|
||||
def docker_repo = "${env.cicd_environment}/${team}/${repo_name.toLowerCase()}"
|
||||
def tag = dparam_obj.getTag(repo_name)
|
||||
if (env.INFRA_ENV == 'toolchain') {
|
||||
def dockerUtilObj = new dockerUtilities()
|
||||
if (dockerUtilObj.imageExists(env.registry, docker_repo, tag)) {
|
||||
log.info("Toolchain: Image found in registry for ${docker_repo}:${tag}. Skipping build step.")
|
||||
return [tag, deployArgo]
|
||||
} else {
|
||||
log.info("Toolchain: Image missing for ${docker_repo}:${tag}. Proceeding with build.")
|
||||
}
|
||||
}
|
||||
// TODO: create value binding for dockerfile render
|
||||
def docker_bindings = [
|
||||
'buildRegistry': env.buildRegistry,
|
||||
'modules_requirements_file': modules_requirements_file,
|
||||
'arch': config.arch,
|
||||
'buildRegistry': env.buildRegistry
|
||||
]
|
||||
docker_bindings['arch'] = config.arch
|
||||
// stage("Create build files"){
|
||||
// dir(repo_name){
|
||||
// getNpmRc(secret_name)
|
||||
// getEnvFile(secret_name)
|
||||
// if (require_pemfiles){getPemFile(secret_name)}
|
||||
// if (require_mainfest){getManifestJson(secret_name)}
|
||||
// }
|
||||
// }
|
||||
stage(stageName('Building docker images')) {
|
||||
// Login to docker
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr get-login-password --region ${env.region} | docker login --username AWS --password-stdin ${env.registry}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script:'gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet')
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in Docker login'
|
||||
env.error_msg_to_db = 'Error in Docker login'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
if (config.containsKey('copy_file')) {
|
||||
def recursive = config.copy_file.recursive ? ' --recursive' : ''
|
||||
docker_bindings['copy_file_path'] = config.copy_file.path
|
||||
dir(repo_name) {
|
||||
dir('copied_files') {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
// delete any old data
|
||||
sh(script:'rm -rf *')
|
||||
sh(script:"aws s3 cp${recursive} ${config.copy_file.path} .")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
// delete any old data
|
||||
docker_bindings['recursive'] = '-r'
|
||||
sh(script:'rm -rf *')
|
||||
sh(script:"gsutil cp ${docker_bindings.recursive} ${docker_bindings.copy_file_path} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
docker_bindings['copy_file'] = true
|
||||
docker_bindings['copy_target'] = config.copy_file.target ?: '/app/'
|
||||
}
|
||||
else {
|
||||
docker_bindings['copy_file'] = false
|
||||
}
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr describe-repositories --region ${env.region} --repository-names ${docker_repo} || aws ecr create-repository --region ${env.region} --repository-name ${docker_repo}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
echo 'Skipping - Registry Creation in GCP.'
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "Error in creating ECR repository ${docker_repo}"
|
||||
env.error_msg_to_db = "Error in creating ECR repository ${docker_repo}"
|
||||
currentBuild.result = env.FAILURE
|
||||
}
|
||||
try {
|
||||
dir(repo_name) {
|
||||
addSSHKey.create()
|
||||
if (!fileExists('Dockerfile')) {
|
||||
// print(config)
|
||||
constructObj.renderTemplate(docker_bindings,config.dockerBuildVersion+'-Dockerfile','Dockerfile-'+repo_name)
|
||||
sh "cat Dockerfile-${repo_name}"
|
||||
// withCredentials([string(credentialsId: 'homelab-github-ssh-prv-key', variable: 'SSH_PRIVATE_KEY_S')]) {
|
||||
// withCredentials(bindings: [sshUserPrivateKey(credentialsId: 'homelab-ssh-github-key', \
|
||||
// keyFileVariable: 'SSH_PRIVATE_KEY', \
|
||||
// passphraseVariable: '', \
|
||||
// usernameVariable: '')]) {
|
||||
//withCredentials([string(credentialsId: 'git_private_key', variable: 'gitkey')]) {
|
||||
// sh """
|
||||
// set +x
|
||||
// docker build --tag ${env.registry}/${docker_repo}:${tag} --build-arg SSH_PRIVATE_KEY="\$(cat ~/.ssh/id_github_jenkins)" -f "Dockerfile-${repo_name}" . && docker push ${env.registry}/${docker_repo}:${tag}
|
||||
// set -x
|
||||
// """
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh """
|
||||
set +x
|
||||
docker build --tag ${env.registry}/${docker_repo}:${tag} --build-arg SSH_PRIVATE_KEY="\$(cat ~/.ssh/id_github_jenkins)" -f "Dockerfile-${repo_name}" .
|
||||
docker push ${env.registry}/${docker_repo}:${tag}
|
||||
set -x
|
||||
"""
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if (env.cicd_environment != 'ftr' || env.INFRA_ENV == 'toolchain') {
|
||||
sh """
|
||||
set +x
|
||||
docker build --tag ${env.registry}/${docker_repo}:${tag} --build-arg SSH_PRIVATE_KEY="\$(cat ~/.ssh/id_github_jenkins)" -f "Dockerfile-${repo_name}" .
|
||||
docker push ${env.registry}/${docker_repo}:${tag}
|
||||
set -x
|
||||
"""
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker builds for Python in ${env.cicd_environment} env")
|
||||
}
|
||||
}
|
||||
// }
|
||||
// }
|
||||
//Remove dockerfile
|
||||
sh(script: "rm -rf Dockerfile-${repo_name}")
|
||||
}
|
||||
else {
|
||||
sh "cat Dockerfile"
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "docker buildx build --platform linux/arm64,linux/amd64 --tag ${env.registry}/${docker_repo}:${tag} --push .")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh """
|
||||
set +x
|
||||
docker build --tag ${env.registry}/${docker_repo}:${tag} --build-arg SSH_PRIVATE_KEY="\$(cat ~/.ssh/id_github_jenkins)" -f "Dockerfile" .
|
||||
docker push ${env.registry}/${docker_repo}:${tag}
|
||||
set -x
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "Error in building DockerFile Or Pushing To ECR. For Full Error Details - ${e}"
|
||||
env.error_msg_to_db = 'Error in building DockerFile Or Pushing To ECR'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
return [tag , deployArgo]
|
||||
}
|
||||
}
|
||||
@@ -1,315 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.utilities.addSSHKey
|
||||
|
||||
|
||||
|
||||
def buildDckr(Map config) {
|
||||
env.RUSTPRIVATE = 'github.com/Homelab'
|
||||
def btObj = new buTeamMapping()
|
||||
def constructObj = new constructTemplate()
|
||||
def dparam_obj = new getDockerParams()
|
||||
def addSSHKey = new addSSHKey()
|
||||
def deployArgo = config.deployArgo ?: true
|
||||
|
||||
def team = btObj.get_team_initials(config.team)
|
||||
def modules = config.modules ?: ['module_less']
|
||||
def repo_name = config.repo_name
|
||||
def docker_repo = "${env.cicd_environment}/${team}/${repo_name.toLowerCase()}"
|
||||
def tag = dparam_obj.getTag(repo_name)
|
||||
def buildx = config.containsKey('buildx') ? config.buildx : true
|
||||
def docker_bindings = [:]
|
||||
def skip_sonar = config.skip_sonar ?: false
|
||||
def version = config.dockerBuildVersion.split('-')[-1]
|
||||
def repoType = config.repo_type ?: 'microservice'
|
||||
docker_bindings['version'] = version
|
||||
docker_bindings['base_dir'] = config.base_dir ?: false
|
||||
docker_bindings['buildRegistry'] = env.buildRegistry
|
||||
docker_bindings['build_packages'] = getSystemPackages(config, 'build_packages')
|
||||
docker_bindings['runtime_packages'] = getSystemPackages(config, 'runtime_packages')
|
||||
stage('Build docker images') {
|
||||
|
||||
|
||||
// Login to docker
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr get-login-password --region ${env.region} | docker login --username AWS --password-stdin ${env.registry}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
|
||||
sh(script:'gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet')
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in Docker login'
|
||||
env.error_msg_to_db = env.msg
|
||||
log.error(env.msg + '. Error: ' + e.toString())
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
if (config.containsKey('copy_file')) {
|
||||
def recursive = config.copy_file.recursive ? ' --recursive' : ''
|
||||
dir(repo_name) {
|
||||
dir('copied_files') {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws s3 cp${recursive} ${config.copy_file.path} .")
|
||||
} else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script:"gsutil cp${recursive} ${config.copy_file.path} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
docker_bindings['copy_file'] = true
|
||||
docker_bindings['copy_target'] = config.copy_file.target ?: '/app/'
|
||||
docker_bindings['base_dir'] = config.base_dir ?: false
|
||||
}
|
||||
else {
|
||||
docker_bindings['copy_file'] = false
|
||||
}
|
||||
|
||||
try {
|
||||
dir(repo_name) {
|
||||
sonar_scan(repo_name, skip_sonar, version )
|
||||
|
||||
if (repoType != 'microservice') {
|
||||
log.info("Skipping Docker build for ${repoType} repo type")
|
||||
return [tag, deployArgo]
|
||||
}
|
||||
|
||||
addSSHKey.create()
|
||||
if (!fileExists('Dockerfile')) {
|
||||
for (module in modules) {
|
||||
def module_name = (module instanceof LinkedHashMap) ? module.keySet()[0] : module
|
||||
docker_bindings['module_property'] = (module instanceof LinkedHashMap) ? module[module_name] : [ : ]
|
||||
docker_bindings['module'] = module_name
|
||||
docker_bindings['binary_name'] = (module == 'module_less') ? repo_name : module_name
|
||||
constructObj.renderTemplate(docker_bindings, 'rust-Dockerfile', 'Dockerfile-' + module_name)
|
||||
sh "cat Dockerfile-${module_name}"
|
||||
module_repo = (module == 'module_less') ? docker_repo : docker_repo + '/' + module_name
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script:"aws ecr describe-repositories --region ${env.region} --repository-names ${module_repo} || aws ecr create-repository --region ${env.region} --repository-name ${module_repo}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
echo 'Skipping - Registry Creation in GCP.'
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "Error in creating ECR repository ${module_repo}"
|
||||
env.error_msg_to_db = "Error in creating ECR repository ${module_repo}"
|
||||
currentBuild.result = 'FAILURE'
|
||||
}
|
||||
if (buildx) {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "set +x && docker buildx build --platform linux/arm64,linux/amd64 --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module_name} --push .")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if (env.cicd_environment != 'ftr') {
|
||||
sh(script: "set +x && docker build --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module_name} . && docker push ${env.registry}/${module_repo}:${tag}")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker Push - ${env.cicd_environment} env")
|
||||
sh(script: "set +x && docker build --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module_name} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
else {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "set +x && docker build --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module_name} . && docker push ${env.registry}/${module_repo}:${tag}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if (env.cicd_environment != 'ftr') {
|
||||
sh(script: """
|
||||
set +x
|
||||
# Setup buildx for ARM64 builds
|
||||
docker buildx rm mybuilder || true
|
||||
docker buildx create --name mybuilder --driver docker-container --bootstrap
|
||||
docker buildx use mybuilder
|
||||
|
||||
# Build for ARM64
|
||||
docker buildx build --platform linux/arm64 --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module_name} --push .
|
||||
""")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker Push - ${env.cicd_environment} env")
|
||||
sh(script: "set +x && docker build --tag ${env.registry}/${module_repo}:${tag} -f Dockerfile-${module_name} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
//Remove dockerfile
|
||||
sh(script: "rm -rf Dockerfile-${module_name}")
|
||||
}
|
||||
}
|
||||
else {
|
||||
sh "cat Dockerfile"
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "docker buildx build --platform linux/arm64,linux/amd64 --tag ${env.registry}/${docker_repo}:${tag} --push .")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script: "docker buildx build --platform linux/arm64,linux/amd64 --tag ${env.registry}/${docker_repo}:${tag} --push .")
|
||||
if (env.cicd_environment != 'ftr') {
|
||||
sh(script: "docker buildx build --platform linux/arm64,linux/amd64 --tag ${env.registry}/${docker_repo}:${tag} --push .")
|
||||
}
|
||||
else {
|
||||
log.info("Skipping Docker Push - ${env.cicd_environment} env")
|
||||
sh(script: "docker buildx build --platform linux/arm64,linux/amd64 --tag ${env.registry}/${docker_repo}:${tag} .")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in building DockerFile Or Pushing To ECR'
|
||||
env.error_msg_to_db = env.msg
|
||||
log.error(env.msg + '. Error: ' + e.toString())
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
return [tag, deployArgo]
|
||||
}
|
||||
}
|
||||
|
||||
// Services may need native libraries that are not part of a shared base image
|
||||
// (for example, database client headers used by a crate's build script). The
|
||||
// keys are language-neutral for future reuse; currently only the Rust builder
|
||||
// consumes them. Restrict values to Debian package names so config data cannot
|
||||
// alter the rendered Dockerfile instruction.
|
||||
def getSystemPackages(Map config, String key) {
|
||||
def configuredPackages = config[key]
|
||||
if (configuredPackages == null) {
|
||||
return []
|
||||
}
|
||||
if (!(configuredPackages instanceof List)) {
|
||||
throw new IllegalArgumentException("${key} must be a YAML list of system package names")
|
||||
}
|
||||
|
||||
def packages = configuredPackages.collect { packageName -> packageName?.toString()?.trim() }
|
||||
if (packages.any { packageName -> !packageName || !(packageName ==~ /^[a-z0-9][a-z0-9+.-]*$/) }) {
|
||||
throw new IllegalArgumentException("${key} contains an invalid system package name")
|
||||
}
|
||||
return packages.unique()
|
||||
}
|
||||
|
||||
|
||||
/*
|
||||
Fuction to execute sonar scan
|
||||
Input Arguments:
|
||||
repo_name: repository name
|
||||
skip_sonar: boolena parameter to skip sonar scan
|
||||
*/
|
||||
|
||||
def sonar_scan(String repo_name, boolean skip_sonar , String version ) {
|
||||
try {
|
||||
stage('Run sonar scan') {
|
||||
if (!skip_sonar) {
|
||||
|
||||
withSonarQubeEnv('sonarqube-test') {
|
||||
if (env.CHANGE_ID) {
|
||||
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh "git fetch origin ${env.CHANGE_TARGET}:refs/remotes/origin/${env.CHANGE_TARGET}"
|
||||
}
|
||||
sh(script: "mvn sonar:sonar -Dsonar.pullrequest.provider=GitHub -Dsonar.pullrequest.github.repository=Homelab/${repo_name} -Dsonar.pullrequest.key=${env.CHANGE_ID} -Dsonar.pullrequest.branch=${env.CHANGE_BRANCH} -Dsonar.pullrequest.base=${env.CHANGE_TARGET}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh "git fetch origin ${env.CHANGE_TARGET}:refs/remotes/origin/${env.CHANGE_TARGET}"
|
||||
}
|
||||
|
||||
sh(script: "curl -LO https://go.dev/dl/go${go_version}.linux-amd64.tar.gz")
|
||||
sh(script: "tar -xvzf go${go_version}.linux-amd64.tar.gz -C /usr/local",returnStdout: true)
|
||||
sh(script:"rm -Rf go${go_version}.linux-amd64.tar.gz")
|
||||
// Set Go environment variables
|
||||
env.PATH = "/usr/local/go/bin:${env.PATH}"
|
||||
|
||||
// Run Go mod tidy and tests
|
||||
sh (script: "go mod tidy")
|
||||
int testExitCode = sh(script: "go test -short -coverprofile=./cov.out ./...", returnStatus: true)
|
||||
if (testExitCode != 0) {
|
||||
sh(script: "echo Go tests failed, but the pipeline will continue.")
|
||||
}
|
||||
|
||||
// Download and extract Sonar Scanner
|
||||
sh(script: "curl -LO https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-5.0.1.3006-linux.zip")
|
||||
sh(script: "unzip -o sonar-scanner-cli-5.0.1.3006-linux.zip -d /usr/local/sonar-scanner")
|
||||
sh(script: "rm -Rf sonar-scanner-cli-5.0.1.3006-linux.zip")
|
||||
env.PATH = "/usr/local/sonar-scanner/sonar-scanner-5.0.1.3006-linux/bin:${env.PATH}"
|
||||
|
||||
// Run Sonar Scanner
|
||||
sh(script:"sonar-scanner -Dsonar.pullrequest.provider=GitHub -Dsonar.pullrequest.github.repository=Homelab/${repo_name} -Dsonar.pullrequest.key=${env.CHANGE_ID} -Dsonar.pullrequest.branch=${env.CHANGE_BRANCH} -Dsonar.pullrequest.base=${env.CHANGE_TARGET} -Dsonar.go.coverage.reportPaths=./cov.out ")
|
||||
}
|
||||
}
|
||||
else {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: "mvn sonar:sonar -Dsonar.branch.name=${env.BRANCH_NAME}")
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
|
||||
sh(script: "curl -LO https://go.dev/dl/go${go_version}.linux-amd64.tar.gz")
|
||||
sh(script: "tar -xvzf go${go_version}.linux-amd64.tar.gz -C /usr/local",returnStdout: true)
|
||||
sh(script:"rm -Rf go${go_version}.linux-amd64.tar.gz")
|
||||
// Set Go environment variables
|
||||
env.PATH = "/usr/local/go/bin:${env.PATH}"
|
||||
|
||||
// Run Go mod tidy and tests
|
||||
sh (script: "go mod tidy")
|
||||
int testExitCode = sh(script: "go test -short -coverprofile=./cov.out ./...", returnStatus: true)
|
||||
if (testExitCode != 0) {
|
||||
sh(script: "echo Go tests failed, but the pipeline will continue.")
|
||||
}
|
||||
|
||||
// Download and extract Sonar Scanner
|
||||
sh(script: "curl -LO https://binaries.sonarsource.com/Distribution/sonar-scanner-cli/sonar-scanner-cli-5.0.1.3006-linux.zip")
|
||||
sh(script: "unzip -o sonar-scanner-cli-5.0.1.3006-linux.zip -d /usr/local/sonar-scanner")
|
||||
sh(script: "rm -Rf sonar-scanner-cli-5.0.1.3006-linux.zip")
|
||||
env.PATH = "/usr/local/sonar-scanner/sonar-scanner-5.0.1.3006-linux/bin:${env.PATH}"
|
||||
|
||||
// Run Sonar Scanner
|
||||
sh(script:"sonar-scanner -Dsonar.go.coverage.reportPaths=./cov.out -Dproject.settings=`pwd`/sonar-project.properties -Dsonar.branch.name=${env.BRANCH_NAME}")
|
||||
|
||||
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
else {
|
||||
log.info('Skipping - Sonar Scan')
|
||||
}
|
||||
}
|
||||
stage("Quality Gate"){
|
||||
|
||||
if (skip_sonar){
|
||||
log.info("Sonar scan is skipped. Marking this stage as passed.")
|
||||
}
|
||||
else if (!env.CHANGE_ID){
|
||||
log.info("Skipping quality gate check on Branches. Marking this stage as passed.")
|
||||
}
|
||||
else{
|
||||
timeout(time: 600, unit: 'SECONDS') {
|
||||
def qg = waitForQualityGate()
|
||||
catchError(buildResult: 'UNSTABLE', stageResult: 'FAILURE'){
|
||||
if (qg.status != 'OK') {
|
||||
log.warn("Quality gate failed: ${qg.status}, but continuing pipeline execution.")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
dir("$repo_name") {
|
||||
withSonarQubeEnv('sonarqube-test') {
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
sh(script: 'mvn sonar:sonar')
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
sh(script: "Quality Gate Failed !")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,906 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.cwctravel.hudson.plugins.extended_choice_parameter.ExtendedChoiceParameterDefinition
|
||||
|
||||
import com.homelab.utilities.getYamlParameter
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.gitActions
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.constructParam
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.utilities.nodePoolSelection
|
||||
|
||||
def run(String repo_name, def deployment_order, def tag, def build_team, def dockerBuildVersion, String notify_channel) {
|
||||
def userInput = ''
|
||||
// def helm_repo_name = 'devops-helm-charts'
|
||||
// def argo_repo_name = 'devops-argo-config'
|
||||
|
||||
def gitObj = new gitActions()
|
||||
def constructParam = new constructParam()
|
||||
def yamlObj = new getYamlParameter()
|
||||
def branch_name = env.BRANCH_NAME
|
||||
def branch_param_map = ['(master|main|gcp-main|gcp-master|farmiso-main)': ['branch':'main', 'envrn':'prd'],
|
||||
'(develop|gcp-dev)':['branch':'develop', 'envrn':'stg']]
|
||||
|
||||
if (env.CHANGE_ID) {
|
||||
branch_param_map = [
|
||||
'(develop|gcp-dev)':['branch':'feature', 'envrn':'ftr'],
|
||||
'(master|main|gcp-main|gcp-master|farmiso-main)':['branch':'pre-prod', 'envrn':'int']]
|
||||
branch_name = env.CHANGE_TARGET
|
||||
}
|
||||
def branch_param = branch_param_map.collectEntries { key, value -> branch_name.matches(key) ? value : [ : ] }
|
||||
if (branch_param == [:]) {
|
||||
branch_param = ['branch':'feature', 'envrn':'ftr']
|
||||
}
|
||||
def helm_branch_name = branch_param.branch
|
||||
def argo_branch_name = branch_param.branch
|
||||
def envrn = branch_param.envrn
|
||||
if (env.CHANGE_ID && env.CHANGE_BRANCH != "develop" && env.CHANGE_TARGET == "main" ) {
|
||||
def allowedNonDevelopPrDeploymentToInt = constructParam.allowedNonDevelopPrDeploymentToIntRepos(repo_name)
|
||||
if (!allowedNonDevelopPrDeploymentToInt) {
|
||||
log.error("******** ONLY DEVELOP BRANCH PR ALLOWED FOR PRE-PROD ENV DEPLOYMENT ********")
|
||||
return
|
||||
}
|
||||
}
|
||||
log.info('Please provide apps to deploy')
|
||||
timeout(unit: 'SECONDS', time: 300) {
|
||||
userInput = wait_for_user_input(deployment_order)
|
||||
}
|
||||
|
||||
// change deployment order according to user input
|
||||
|
||||
if (userInput == '') {
|
||||
log.info('No Deployments selected. Running remaining steps.')
|
||||
return
|
||||
}
|
||||
else if (!userInput.contains('All')) {
|
||||
deployment_order = userInput.split(',') as List
|
||||
}
|
||||
|
||||
commit_branch_name = env.CHANGE_ID ? env.CHANGE_BRANCH : env.BRANCH_NAME
|
||||
def commit_id = gitObj.fetchLatestCommitId(repo_name, commit_branch_name)
|
||||
env.COMMIT_ID = commit_id
|
||||
|
||||
env.SERVICES = deployment_order
|
||||
//Show selected applications to deploy
|
||||
log.info('Following services will be deployed:\n' + deployment_order.join('\n'))
|
||||
// Clone the repo just once for all the deployments
|
||||
gitObj.clone("${WORKSPACE}", "${env.helm_repo_name}", "${helm_branch_name}")
|
||||
|
||||
// Clone the repo just once for all the deployments
|
||||
gitObj.clone("${WORKSPACE}", "${env.argo_repo_name}", "${argo_branch_name}")
|
||||
|
||||
for (deployment in deployment_order) {
|
||||
try {
|
||||
|
||||
def isMultizoneEnabled = constructParam.isMultizoneEnabled(deployment)
|
||||
log.info(" Multi-zone enabled for ${deployment} - ${isMultizoneEnabled}")
|
||||
if (isMultizoneEnabled) {
|
||||
def multizone = "Multi-zone enabled for this deployable ${deployment}. Please use Ringmaster for deployment."
|
||||
currentBuild.result = env.FAILURE
|
||||
throw new Exception(multizone)
|
||||
}
|
||||
def value_binding = yamlObj.getParam("${repo_name}/deployments", "${deployment}.yaml")
|
||||
constructParam.perDeploymentVars(value_binding)
|
||||
// if(envrn == 'prd') {
|
||||
// stage("${deployment}: Downscale Pods in Preprod Env") {
|
||||
// preprod_downscale(repo_name, deployment,notify_channel)
|
||||
// }
|
||||
// }
|
||||
stage("${deployment}: Update Argo App") {
|
||||
update_argo_repo(repo_name, deployment, argo_branch_name, envrn)
|
||||
}
|
||||
stage('Update Argo App of apps') {
|
||||
refresh_app_of_apps(envrn)
|
||||
}
|
||||
stage("${deployment}: Update Helm Repo") {
|
||||
update_helm_repo(repo_name, deployment, tag, build_team, helm_branch_name, envrn, dockerBuildVersion, notify_channel)
|
||||
}
|
||||
stage("${deployment}: Refresh & Sync App in argoCD") {
|
||||
refresh_and_sync(repo_name, deployment, envrn)
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
log.error(e)
|
||||
currentBuild.result = env.FAILURE
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// def preprod_downscale(String repo_name, String deployment, String notify_channel){
|
||||
// def yamlobj = new getYamlParameter()
|
||||
// def deploy_values = yamlobj.getParam("${repo_name}/deployments","${deployment}.yaml")
|
||||
// def app_name = deploy_values.app_name
|
||||
// def bu = deploy_values.bu
|
||||
// def slack_channel = notify_channel
|
||||
// def as_enabled = deploy_values.environment.'int'.as_enabled
|
||||
// log.info("########################### Invoking Jenkins Job to Downscale Pods in Preprod Environment. ###########################")
|
||||
// build wait: false, job: 'downscale-preprod-eks', parameters:[string(name:'app_name', value:"${app_name}"),
|
||||
// string(name:'bu',value:"${bu}"),
|
||||
// string(name:'as_enabled',value:"${as_enabled}"),
|
||||
// string(name:'slack_channel',value:"${slack_channel}")]
|
||||
// }
|
||||
|
||||
def wait_for_user_input(def deployments) {
|
||||
def userInput = ''
|
||||
String choices = 'All,' + deployments.join(',')
|
||||
int visibleItemCount = 1 + deployments.size()
|
||||
def multiSelect = new ExtendedChoiceParameterDefinition('deployments', //name
|
||||
'PT_CHECKBOX', // parameter type
|
||||
choices, //values
|
||||
'', //projectName
|
||||
'', //propertyFile
|
||||
'', //groovyScript
|
||||
'', //groovyScriptFile
|
||||
'', //bindings
|
||||
'', //groovyClasspath
|
||||
'', //propertyKey
|
||||
'', //defaultValue
|
||||
'', //defaultPropertyFile
|
||||
'', //defaultGroovyScript
|
||||
'', //defaultGroovyScriptFile
|
||||
'', //defaultBindings
|
||||
'', //defaultGroovyClasspath
|
||||
'', //defaultPropertyKey
|
||||
'', //descriptionPropertyValue
|
||||
'', //descriptionPropertyFile
|
||||
'', //descriptionGroovyScript
|
||||
'', //descriptionGroovyScriptFile
|
||||
'', //descriptionBindings
|
||||
'', //descriptionGroovyClasspath
|
||||
'', //descriptionPropertyKey
|
||||
'', //javascriptFile
|
||||
'', //javascript
|
||||
false, //saveJSONParameterToFile
|
||||
false, //quoteValue
|
||||
visibleItemCount, //visibleItemCount
|
||||
'Choose Deployments', //description
|
||||
',') //multiSelectDelimiter
|
||||
|
||||
stage('wait for user input') {
|
||||
try {
|
||||
echo "Skipping User Input - ${env.skip_user_input}"
|
||||
if (env.skip_user_input.toBoolean()) {
|
||||
userInput = 'All'
|
||||
}
|
||||
else {
|
||||
userInput = input message: 'Choose applications to deploy', ok: 'Deploy', parameters: [multiSelect]
|
||||
}
|
||||
return userInput
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in taking userInput' + e.toString()
|
||||
env.error_msg_to_db = 'Error Taking User Input for Deployment of Applications'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def update_helm_repo(String repo_name, String deployment, String tag, String build_team, String helm_branch_name, String envrn, def dockerBuildVersion, String notify_channel) {
|
||||
def yamlobj = new getYamlParameter()
|
||||
def btObj = new buTeamMapping()
|
||||
def gitObj = new gitActions()
|
||||
def templateObj = new constructTemplate()
|
||||
def nodePoolSelection = new nodePoolSelection()
|
||||
// def helm_repo_name = 'devops-helm-charts'
|
||||
|
||||
def value_yaml_file = get_value_yaml_file(dockerBuildVersion)
|
||||
def value_binding1 = yamlobj.getParam("${repo_name}/deployments", "${deployment}.yaml")
|
||||
if (value_binding1.containsKey('cron')) {
|
||||
if (value_binding1['cron']) {
|
||||
value_yaml_file = 'cron-values.yaml'
|
||||
}
|
||||
}
|
||||
|
||||
def env_norm = ['prd':'prod', 'int':'pre-prod', 'stg':'stg', 'ftr':'feature']
|
||||
def pr_num = ''
|
||||
def commit_status = 0
|
||||
|
||||
try {
|
||||
// Get values from deployments/deployment.yaml
|
||||
isfeatureDeployment = (envrn == 'ftr') ? true : false
|
||||
value_binding1 = yamlobj.getParam("${repo_name}/deployments", "${deployment}.yaml")
|
||||
def memory_request = value_binding1.environment[env.cicd_environment].memory_request
|
||||
def cpu_request = value_binding1.environment[env.cicd_environment].cpu_request
|
||||
def priority_v2 = value_binding1.priority_v2
|
||||
def commit_id = env.COMMIT_ID
|
||||
if ( env.CLOUD_PROVIDER == 'GCP' && !value_binding1.containsKey('cron') ) {
|
||||
nodeSelectorValue = nodePoolSelection.run(memory_request, cpu_request.toString(),priority_v2)
|
||||
echo "Node Selector Value is ${nodeSelectorValue}"
|
||||
value_binding1['nodeSelectorValue'] = nodeSelectorValue
|
||||
}
|
||||
def xms = ''
|
||||
def xmx = ''
|
||||
if (!value_binding1.containsKey('cron')) {
|
||||
if (dockerBuildVersion.contains('maven') || dockerBuildVersion.contains('gradle') || dockerBuildVersion.contains('go')) {
|
||||
value_binding1['activeProcessorCount'] = calculate_active_processors(cpu_request.toString())
|
||||
}
|
||||
if (dockerBuildVersion.contains('maven') || dockerBuildVersion.contains('gradle')) {
|
||||
def memory_limit = value_binding1.environment[env.cicd_environment].memory_limit
|
||||
def deployment_args = value_binding1.environment[env.cicd_environment].deployment_args
|
||||
echo "memory_limit is ${memory_limit}"
|
||||
def memory_string = memory_limit
|
||||
echo "performing operations on this memory string - ${memory_string}"
|
||||
|
||||
def memory_value = ''
|
||||
|
||||
if ( memory_string.contains('M') ) {
|
||||
memory_value = memory_string.replaceAll('Mi', '')
|
||||
memory_value = memory_value.replaceAll('M', '')
|
||||
try {
|
||||
memory_value = memory_value.toInteger()
|
||||
}
|
||||
catch (NumberFormatException e) {
|
||||
memory_value = memory_value.toDouble()
|
||||
memory_value = memory_value.toInteger()
|
||||
}
|
||||
}
|
||||
else if ( memory_string.contains('G') ) {
|
||||
memory_value = memory_string.replaceAll('Gi', '')
|
||||
memory_value = memory_value.replaceAll('G', '')
|
||||
try {
|
||||
memory_value = memory_value.toInteger() * 1024
|
||||
}
|
||||
catch (NumberFormatException e) {
|
||||
memory_value = memory_value.toDouble() * 1024
|
||||
memory_value = memory_value.toInteger()
|
||||
}
|
||||
}
|
||||
|
||||
memory_value = memory_value * 0.5
|
||||
memory_value = memory_value.toInteger()
|
||||
xms = "${memory_value}M"
|
||||
xmx = "${memory_value}M"
|
||||
|
||||
echo "xms and xmx from memory value - ${memory_value} are - ${xms} ${xmx}"
|
||||
for (arg in deployment_args) {
|
||||
if (arg.contains('Xms')) {
|
||||
xms = arg.replaceAll('.*Xms', '')
|
||||
echo "xms from deployment_args - ${xms}"
|
||||
}
|
||||
if (arg.contains('Xmx')) {
|
||||
xmx = arg.replaceAll('.*Xmx', '')
|
||||
echo "xmx from deployment_args - ${xmx}"
|
||||
}
|
||||
}
|
||||
echo "final xms and xmx - ${xms} ${xmx}"
|
||||
}
|
||||
}
|
||||
|
||||
value_binding1['dockerBuildVersion'] = dockerBuildVersion == 'python-3.10.12' ? 'python-3.7' : dockerBuildVersion
|
||||
deployEnv = !value_binding1['environment'].containsKey(envrn) && isfeatureDeployment ? 'stg' : envrn
|
||||
Map value_binding = value_binding1['environment'].collectEntries { key, value -> deployEnv.matches(key) ? value : [ : ] }
|
||||
value_binding1.remove('environment')
|
||||
value_binding1.putAll(value_binding)
|
||||
|
||||
value_binding1['repo_name'] = repo_name
|
||||
value_binding1['environment'] = envrn
|
||||
value_binding1['environment_norm'] = env_norm[envrn]
|
||||
value_binding1['tag'] = tag
|
||||
value_binding1['repo_name'] = repo_name
|
||||
value_binding1['build_team'] = build_team
|
||||
value_binding1['commit_id'] = commit_id
|
||||
|
||||
def buini = btObj.get_bu_initials(value_binding1.bu)
|
||||
def bu = btObj.get_bu_initials(value_binding1.bu)
|
||||
def teamini = btObj.get_team_initials(value_binding1.team)
|
||||
def team = btObj.get_team_initials(value_binding1.team)
|
||||
def app_name = value_binding1.app_name
|
||||
def app_branch = app_name + '-' + helm_branch_name
|
||||
|
||||
def app_helm_repo = "${env.helm_repo_name}/${env.helmChartsPath}/${buini}/${teamini}/${app_name}"
|
||||
def ingress_val = value_binding1.ingress_val ?: (env.CHANGE_ID) ? "pr-${CHANGE_ID}" : "${BRANCH_NAME}"
|
||||
|
||||
sh "chmod -R 777 ${app_helm_repo}"
|
||||
|
||||
sh "yq . ${app_helm_repo}/values_properties.yaml -y > a.yaml;mv a.yaml ${app_helm_repo}/values_properties.yaml"
|
||||
sh "cat ${app_helm_repo}/values_properties.yaml"
|
||||
if (isfeatureDeployment) {
|
||||
env.ingress_val = ingress_val
|
||||
value_binding1['env_ns'] = ingress_val
|
||||
value_binding1['vault_env'] = value_binding1['create_vault_path'] ? ingress_val : 'stg'
|
||||
sh(returnStdout: true, script: """
|
||||
mkdir -p ${app_helm_repo}/${ingress_val}
|
||||
sed "s/INGRESS_PR_NUMBER/${ingress_val}/g" ${app_helm_repo}/values_properties.yaml > ${app_helm_repo}/${ingress_val}/values_properties.yaml
|
||||
""")
|
||||
value_binding2 = yamlobj.getParam("${app_helm_repo}/${ingress_val}", 'values_properties.yaml')
|
||||
app_branch = ingress_val + '-' + app_branch
|
||||
}
|
||||
else {
|
||||
value_binding1['vault_env'] = envrn
|
||||
value_binding1['env_ns'] = envrn
|
||||
value_binding2 = yamlobj.getParam("${app_helm_repo}", 'values_properties.yaml')
|
||||
echo '722 Printing Value Binding 2'
|
||||
print value_binding2
|
||||
}
|
||||
|
||||
value_binding1.putAll(value_binding2)
|
||||
value_binding1['bu_norm'] = value_binding1['bu']
|
||||
value_binding1['team_norm'] = value_binding1['team']
|
||||
value_binding1['bu'] = buini
|
||||
value_binding1['team'] = teamini
|
||||
|
||||
|
||||
def serviceTypes = ["httpstateless", "consumer", "producer", "scheduler", "worker", "grpc", "web", "websocket", "cache", "database"]
|
||||
//validate service type params
|
||||
if (value_binding1['service_type']){
|
||||
// Validating the parameter type
|
||||
if(!(value_binding1['service_type'] instanceof List) || value_binding1['service_type'].isEmpty()){
|
||||
env.msg = 'You have not specified service_type correctly. Exiting the pipeline.'
|
||||
log.error(env.msg)
|
||||
sh 'exit 1'
|
||||
}
|
||||
// Check if value_binding1['service_type'] contains any service type not in servicesType
|
||||
def invalidServiceTypes = value_binding1['service_type'].findAll { !serviceTypes.contains(it) }
|
||||
if (!invalidServiceTypes.isEmpty()) {
|
||||
log.error("Invalid service type(s): ${invalidServiceTypes.join(',')}. Allowed service_types are: ${serviceTypes.join(',')}")
|
||||
sh 'exit 1'
|
||||
}
|
||||
value_binding1['service_type_norm'] = value_binding1['service_type'].join(',')
|
||||
}
|
||||
else{
|
||||
value_binding1['service_type_norm']=''
|
||||
}
|
||||
|
||||
// Enable backward compatibility for missing keys
|
||||
enable_backward_compatibility(value_binding1)
|
||||
value_binding1['prismsdk_environment'] = 'PRODUCTION' // THIS KEY WILL CHANGE ONCE 3RD CONFIRM TO STANDERIZE
|
||||
if (envrn == 'stg'){
|
||||
value_binding1['otel_enabled'] = true
|
||||
value_binding1['prismsdk_environment'] = 'SANDBOX' // THIS KEY WILL CHANGE ONCE 3RD CONFIRM TO STANDERIZE
|
||||
}
|
||||
// Override hot fix and notify_channel
|
||||
value_binding1['canary']['skipAnalysis'] = (env.hot_fix) ? true : value_binding1['canary']['skipAnalysis']
|
||||
value_binding1['canary']['slackChannel'] = notify_channel
|
||||
value_binding1['xms'] = xms
|
||||
value_binding1['xmx'] = xmx
|
||||
value_binding1['CLOUD_PROVIDER'] = env.CLOUD_PROVIDER
|
||||
def appConfig = value_binding1['appConfigEnabled']
|
||||
if (appConfig) {
|
||||
def configModule = (value_binding1["module"] == 'module_less') ? repo_name : value_binding1["module"]
|
||||
// read the static config from configs directory
|
||||
def value_binding3 = yamlobj.getParamAsString("${repo_name}/configs/${configModule}", "application-${envrn}.yml")
|
||||
// to update in values.yaml
|
||||
value_binding1['staticAppConfigData'] = value_binding3
|
||||
//read the dynamic config from config directory
|
||||
def value_binding4 = yamlobj.getParamAsString("${repo_name}/configs/${configModule}", "application-dyn-${envrn}.yml")
|
||||
// to update in values.yaml
|
||||
value_binding1['dynamicAppConfigData'] = value_binding4
|
||||
echo 'printing the values_binding3 and values_binding4'
|
||||
print value_binding3
|
||||
print value_binding4
|
||||
value_binding1['vault_env'] = value_binding1['vault_env'] + '-cac'
|
||||
|
||||
}
|
||||
|
||||
|
||||
gitObj.preDeleteBranch(env.helm_repo_name, helm_branch_name, app_branch)
|
||||
gitObj.branchCheckOut(env.helm_repo_name, app_branch)
|
||||
|
||||
echo "Helm Step - Value Binding 1 - ${value_binding1}"
|
||||
if (isfeatureDeployment) {
|
||||
echo 'It is feature deployment'
|
||||
templateObj.renderTemplate(value_binding1, value_yaml_file, "${app_helm_repo}/${ingress_val}/values.yaml")
|
||||
echo 'Before YAML Linting'
|
||||
sh "cat ${app_helm_repo}/${ingress_val}/values.yaml"
|
||||
sh "yq . ${app_helm_repo}/${ingress_val}/values.yaml -y > a.yaml;mv a.yaml ${app_helm_repo}/${ingress_val}/values.yaml"
|
||||
echo 'After YAML Linting'
|
||||
sh "cat ${app_helm_repo}/${ingress_val}/values.yaml"
|
||||
gitObj.add(env.helm_repo_name, "${env.helmChartsPath}/${buini}/${teamini}/${app_name}/${ingress_val}/values.yaml")
|
||||
}
|
||||
else {
|
||||
echo 'Not a feature deployment'
|
||||
templateObj.renderTemplate(value_binding1, value_yaml_file, "${app_helm_repo}/values.yaml")
|
||||
echo 'Before YAML Linting'
|
||||
sh "cat ${app_helm_repo}/values.yaml"
|
||||
|
||||
// Perform YAML linting
|
||||
sh "yq . ${app_helm_repo}/values.yaml -y > a.yaml; mv a.yaml ${app_helm_repo}/values.yaml"
|
||||
echo 'After YAML Linting'
|
||||
sh "cat ${app_helm_repo}/values.yaml"
|
||||
|
||||
// creating a map from final values.yaml to check canary enforcement conditions
|
||||
def valuesMap = yamlobj.getParam("${app_helm_repo}", "values.yaml")
|
||||
|
||||
// Canary enforcement for sp0 services
|
||||
// Also will have to check for cron, worker and scheduler services
|
||||
def enforceCanary = false
|
||||
if (!app_helm_repo.contains("cron") && !app_helm_repo.contains("worker") && !app_helm_repo.contains("scheduler") && !app_helm_repo.contains("consumer") && (valuesMap["labels"]["priority_v2"] == "sp0" || valuesMap["labels"]["priority_v2"] == "up0") && !value_binding1["addHeadless"] && !value_binding1['dockerBuildVersion'].contains("node") && envrn == "prd") {
|
||||
enforceCanary = true
|
||||
}
|
||||
|
||||
def proceed = false
|
||||
def deploymentFailureErrorMessage
|
||||
if (enforceCanary) {
|
||||
if (valuesMap["canary"]["enabled"] == true) {
|
||||
if (valuesMap["canary"]["skipAnalysis"] == false) {
|
||||
if (valuesMap["canary"]["enableManualPromotion"] == true) {
|
||||
proceed = true
|
||||
} else {
|
||||
error("Error: Update the enableManualPromotion parameter")
|
||||
}
|
||||
} else {
|
||||
error("Error: Update the skipAnalysis parameter")
|
||||
|
||||
}
|
||||
} else {
|
||||
error("Error: Enable canary and retry")
|
||||
}
|
||||
} else {
|
||||
proceed = true
|
||||
}
|
||||
|
||||
// Check if priority labels and environment conditions require critical dependency check
|
||||
def criticalPriorities = ["sp0", "up0", "cp0", "sp1", "up1", "cp1"]
|
||||
def priority = valuesMap["labels"]["priority_v2"]
|
||||
if (criticalPriorities.contains(priority) && (envrn == "prd")) {
|
||||
isDependabotCritcal = dependabotCriticalCheck(repo_name)
|
||||
if (isDependabotCritcal) {
|
||||
error("Error: Critical vulnerabilities found in repo: " + repo_name + " \nPlease resolve the alerts marked with CRITICAL here and retry: https://github.com/Homelab/" + repo_name + "/security/dependabot and Retry.")
|
||||
}
|
||||
}
|
||||
|
||||
def param = new constructParam()
|
||||
log.info("checking if appConfig is enabled")
|
||||
def isAppConfigDisabled = param.appConfigDisabledForbidden(appConfig, repo_name, envrn, dockerBuildVersion)
|
||||
if (isAppConfigDisabled){
|
||||
error("Error: appConfig is disabled , onboard your application with config-as-code changes")
|
||||
}
|
||||
|
||||
sh "cat ${app_helm_repo}/values.yaml"
|
||||
gitObj.add(env.helm_repo_name, "${env.helmChartsPath}/${buini}/${teamini}/${app_name}/values.yaml")
|
||||
|
||||
}
|
||||
|
||||
|
||||
commit_status = gitObj.codeCommit(env.helm_repo_name, app_branch, 'Generating values yaml file')
|
||||
if (commit_status == 0) {
|
||||
gitObj.codePush(env.helm_repo_name, app_branch)
|
||||
pr_num = gitObj.createPR(app_name, env.helm_repo_name, helm_branch_name, app_branch, 'Merge helm values file')
|
||||
gitObj.mergePR(env.helm_repo_name, pr_num, app_branch)
|
||||
gitObj.deleteBranch(env.helm_repo_name, helm_branch_name, app_branch)
|
||||
}
|
||||
}
|
||||
catch (FileNotFoundException e) {
|
||||
env.msg = 'Error Updating Helm Repo ' + e.toString()
|
||||
env.error_msg_to_db = 'File Not Found'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error Updating Helm Repo ' + e.toString()
|
||||
env.error_msg_to_db += 'Error Updating in Helm repo. Git error message - ' + env.error_part_msg_to_db
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def refresh_and_sync(String repo_name, String deployment, String envrn) {
|
||||
def yamlobj = new getYamlParameter()
|
||||
def deploy_values = yamlobj.getParam("${repo_name}/deployments", "${deployment}.yaml")
|
||||
def app_name = deploy_values.app_name
|
||||
// def argoUrl = (env.BRANCH_NAME == 'master' || env.BRANCH_NAME == 'main' || env.CHANGE_TARGET == 'master' || env.CHANGE_TARGET == 'main') ? 'prod-ops-argocd.homelab.com' : 'stg-dev-argocd.homelabtest.in'
|
||||
def argoEnv = (envrn == 'ftr') ? env.ingress_val : envrn
|
||||
|
||||
log.info('########################### Pulling latest changes in ArgoCD. ###########################')
|
||||
withCredentials([usernamePassword(credentialsId: env.argoCreds, passwordVariable: 'ARGO_PASSWORD', usernameVariable: 'ARGO_USERNAME')]) {
|
||||
try {
|
||||
sh """
|
||||
set +x
|
||||
argocd login ${env.argoURL}:443 --username ${ARGO_USERNAME} --password ${ARGO_PASSWORD} --grpc-web
|
||||
argocd app get --hard-refresh ${argoEnv}-${app_name} --grpc-web
|
||||
"""
|
||||
}
|
||||
catch (Exception e) { //added try catch block here
|
||||
env.msg = "Error in hard refresh of app ${app_name} full error: ${e}"
|
||||
env.error_msg_to_db += "Error in hard refresh of app ${app_name};"
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
log.info('########################### Syncing latest changes in ArgoCD. ###########################')
|
||||
return_value = sh(returnStatus: true, script: "argocd app sync ${argoEnv}-${app_name} --grpc-web --http-retry-max 3 --retry-backoff-duration 1m") as Integer
|
||||
if (return_value == 0) {
|
||||
log.info('App synced succesfully.')
|
||||
} else {
|
||||
env.msg = 'App sync failed. Please check in ArgoCD UI.'
|
||||
env.error_msg_to_db += "Error Argo App sync failed ${app_name};"
|
||||
log.error(env.msg)
|
||||
error "${env.msg}"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def refresh_app_of_apps(String envrn) {
|
||||
try {
|
||||
appofapps = env.argoIncubator
|
||||
log.info('########################### Pulling latest changes in ArgoCD for App of Apps. ###########################')
|
||||
withCredentials([usernamePassword(credentialsId: env.argoCreds, passwordVariable: 'ARGO_PASSWORD', usernameVariable: 'ARGO_USERNAME')]) {
|
||||
sh """
|
||||
set +x
|
||||
argocd login ${env.argoURL}:443 --username ${ARGO_USERNAME} --password ${ARGO_PASSWORD} --grpc-web
|
||||
argocd app sync ${appofapps} --grpc-web --http-retry-max 3 --retry-backoff-duration 1m || true
|
||||
"""
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = 'Error in App Sync' + e.toString()
|
||||
env.error_msg_to_db += 'Error syncing app of app;'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def update_argo_repo(String repo_name, String deployment, String argoBranch, String envrn) {
|
||||
def yamlobj = new getYamlParameter()
|
||||
def btObj = new buTeamMapping()
|
||||
def gitObj = new gitActions()
|
||||
def templateObj = new constructTemplate()
|
||||
// def argo_repo_name = 'devops-argo-config'
|
||||
def pr_num = ''
|
||||
def commit_status = 0
|
||||
def commit_id = env.COMMIT_ID
|
||||
|
||||
value_binding1 = yamlobj.getParam("${repo_name}/deployments", "${deployment}.yaml")
|
||||
value_binding1['CLOUD_PROVIDER'] = env.CLOUD_PROVIDER
|
||||
isfeatureDeployment = (envrn == 'ftr') ? true : false
|
||||
deployEnv = !value_binding1['environment'].containsKey(envrn) && isfeatureDeployment ? 'stg' : envrn
|
||||
Map value_binding = value_binding1['environment'].collectEntries { key, value -> deployEnv.matches(key) ? value : [ : ] }
|
||||
value_binding1.remove('environment')
|
||||
value_binding1.putAll(value_binding)
|
||||
value_binding1['environment'] = envrn
|
||||
|
||||
def buini = btObj.get_bu_initials(value_binding1.bu)
|
||||
def bu = value_binding1.bu
|
||||
|
||||
def teamini = btObj.get_team_initials(value_binding1.team)
|
||||
def team = value_binding1.team
|
||||
def app_name = value_binding1.app_name
|
||||
def filename = "${teamini}-${app_name}.yaml"
|
||||
def helm_values_path = "${env.helmChartsPath}/${buini}/${teamini}/${app_name}"
|
||||
def app_branch = app_name + '-' + argoBranch
|
||||
def ingress_val = value_binding1.ingress_val ?: (env.CHANGE_ID) ? "pr-${CHANGE_ID}" : "${BRANCH_NAME}"
|
||||
value_binding1['helm_version'] = value_binding1.helm_version ?: env.defaultHelmChartVersion
|
||||
|
||||
if (isfeatureDeployment) {
|
||||
value_binding1['env_ns'] = ingress_val
|
||||
value_binding1['helm_values_path'] = "${env.helmChartsPath}/${buini}/${teamini}/${app_name}/${ingress_val}"
|
||||
filename = ingress_val + '-' + "${app_name}.yaml"
|
||||
app_branch = ingress_val + '-' + app_branch
|
||||
}
|
||||
else {
|
||||
value_binding1['env_ns'] = envrn
|
||||
value_binding1['helm_values_path'] = "${env.helmChartsPath}/${buini}/${teamini}/${app_name}"
|
||||
filename = "${teamini}-${app_name}.yaml"
|
||||
}
|
||||
|
||||
value_binding1['server'] = env.clusterName
|
||||
value_binding1['clusterName'] = env.clusterName
|
||||
|
||||
value_binding1['branch_name'] = argoBranch
|
||||
value_binding1['buini'] = buini
|
||||
value_binding1['teamini'] = teamini
|
||||
|
||||
value_binding1['priority_v2'] = value_binding1.priority_v2 ?: 'cp3'
|
||||
value_binding1['primary_owner'] = value_binding1.primary_owner != null ? value_binding1.primary_owner.split('@')[0] : value_binding1.team
|
||||
value_binding1['secondary_owner'] = value_binding1.secondary_owner != null ? value_binding1.secondary_owner.split('@')[0] : value_binding1.team
|
||||
value_binding1['argoAppNS'] = env.argoAppNS
|
||||
value_binding1['commit_id'] = commit_id
|
||||
log.info(commit_id)
|
||||
gitObj.preDeleteBranch(env.argo_repo_name, argoBranch, app_branch)
|
||||
gitObj.branchCheckOut(env.argo_repo_name, app_branch)
|
||||
echo "Argo Step - Value Binding 1 - ${value_binding1}"
|
||||
|
||||
dir(env.argo_repo_name) {
|
||||
def dirExists = sh(script: "cat ${env.argoAppsPath}/${filename}", returnStatus: true)
|
||||
if ( dirExists != 0 ) {
|
||||
sh "mkdir -p ${env.argoAppsPath}"
|
||||
sh "touch ${env.argoAppsPath}/${filename}"
|
||||
}
|
||||
sh "chmod -R 777 ${env.argoAppsPath}/${filename}"
|
||||
templateObj.renderTemplate(value_binding1, 'argoApp.yaml', "${env.argoAppsPath}/${filename}")
|
||||
sh "yq . ${env.argoAppsPath}/${filename} -y > a.yaml;mv a.yaml ${env.argoAppsPath}/${filename}"
|
||||
sh "cat ${env.argoAppsPath}/${filename}"
|
||||
}
|
||||
|
||||
gitObj.add(env.argo_repo_name, "${env.argoAppsPath}/${filename}")
|
||||
commit_status = gitObj.codeCommit(env.argo_repo_name, app_branch, "onboarding ${deployment} app to ARGO")
|
||||
if (commit_status == 0) {
|
||||
gitObj.codePush(env.argo_repo_name, app_branch)
|
||||
pr_num = gitObj.createPR(app_name, env.argo_repo_name, argoBranch, app_branch, 'Merge argo application configuration')
|
||||
gitObj.mergePR(env.argo_repo_name, pr_num, app_branch)
|
||||
gitObj.deleteBranch(env.argo_repo_name, argoBranch, app_branch)
|
||||
}
|
||||
}
|
||||
|
||||
def calculate_active_processors(String cpu_request) {
|
||||
// If lowercase m is present
|
||||
if (cpu_request.contains('m')) {
|
||||
// Remove m
|
||||
cpu_request = cpu_request.replaceAll('m', '')
|
||||
|
||||
// If this fails means invalid input is given with m, so we can let the pipeline fail here
|
||||
// Convert to double, divide by thousand to get another double, round it off and then convert to integer.
|
||||
return Math.ceil(cpu_request.toDouble() / 1000).toInteger()
|
||||
}
|
||||
|
||||
return Math.ceil(cpu_request.toDouble()).toInteger()
|
||||
}
|
||||
|
||||
def enable_backward_compatibility(def bindings) {
|
||||
// def primaryEmailInitial = bindings.primary_owner.tokenize( '@' )[0]: null
|
||||
// def secondaryEmailInitial = bindings.secondary_owner.tokenize( '@' )[0]: null
|
||||
def dparam_obj = new getDockerParams()
|
||||
bindings['registry'] = env.registry
|
||||
def canary_default = [
|
||||
'progressDeadlineSeconds': 300,
|
||||
'analysisInterval': '120s',
|
||||
'analysisThreshold': 5,
|
||||
'analysisMaxWeight': 5,
|
||||
'analysisStepWeight': 5,
|
||||
'analysisMetrics':[
|
||||
'thresholdRangeMin': 0.99,
|
||||
'interval': '1m'],
|
||||
'skipAnalysis': false
|
||||
]
|
||||
def statefulset_default = [
|
||||
'updateStrategy': 'RollingUpdate',
|
||||
'volumeType': 'dynamic',
|
||||
'dynamicVolume':[
|
||||
'accessMode': 'ReadWriteMany',
|
||||
'mountPath': '/opt/data',
|
||||
'size': '5Gi',
|
||||
'storageClass': ''],
|
||||
'staticVolume':[
|
||||
'accessMode': 'ReadWriteMany',
|
||||
'size': '5Gi',
|
||||
'mountPath': '/opt/data',
|
||||
'storageClass': '',
|
||||
'volumeHandle': '',
|
||||
'csiDriver': '']
|
||||
]
|
||||
bindings['hostAliases'] = bindings.hostAliases ?: false
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
bindings['nodeSelector'] = bindings.nodeSelector ?: (env.cicd_environment == 'stg' || env.cicd_environment == 'ftr') ? bindings.bu : ( env.cicd_environment == 'int' ? bindings.bu + '-int' : bindings.team )
|
||||
bindings['nodeSelectorValue'] = 'dedicated'
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
if (env.cicd_environment == 'int') {
|
||||
bindings['nodeSelector'] = bindings.nodeSelector ?: 'cloud.google.com/compute-class'
|
||||
} else {
|
||||
bindings['nodeSelector'] = bindings.nodeSelector ?: 'dedicated'
|
||||
}
|
||||
bindings['nodeSelectorValue'] = bindings.nodeSelectorValue ?: 'megatetra'
|
||||
}
|
||||
bindings['triggers'] = bindings.triggers ?: false
|
||||
bindings['host'] = bindings.host ?: false
|
||||
bindings['hosts'] = bindings.hosts ?: false
|
||||
bindings['grpc_host'] = bindings.grpc_host ?: false
|
||||
bindings['grpc_hosts'] = bindings.grpc_hosts ?: false
|
||||
bindings['serviceAccount'] = bindings.serviceAccount ?: false
|
||||
bindings['ingress_annotations'] = bindings.ingress_annotations ?: ''
|
||||
bindings['canary'] = bindings.canary ?: canary_default
|
||||
bindings['minCanaryReplicas'] = bindings.canary?.minCanaryReplicas ?: bindings.minCanaryReplicas ?: bindings.as_min
|
||||
bindings['maxCanaryReplicas'] = bindings.canary?.maxCanaryReplicas ?: bindings.maxCanaryReplicas ?: bindings.as_max
|
||||
bindings['enableManualPromotion'] = bindings.canary?.enableManualPromotion ?: bindings.enableManualPromotion ?: false
|
||||
bindings['cpu_limit'] = bindings.cpu_limit ?: bindings.cpu_request
|
||||
bindings['slowStartWindow'] = bindings.slowStartWindow ?: false
|
||||
bindings['slowStartAggression'] = bindings.slowStartAggression ?: '1'
|
||||
bindings['slowStartMinPercent'] = bindings.slowStartMinPercent ?: '10'
|
||||
bindings['lifecycle'] = bindings.lifecycle ?: false
|
||||
bindings['maxSurge'] = bindings.maxSurge ?: '50'
|
||||
bindings['as_down_pod_count'] = bindings.as_down_pod_count ?: '2'
|
||||
bindings['as_up_pod_count'] = bindings.as_up_pod_count ?: '2'
|
||||
bindings['as_up_pod_percentage'] = bindings.as_up_pod_percentage ?: '10'
|
||||
bindings['createContourGateway'] = bindings.createContourGateway ?: false
|
||||
bindings['service_annotations'] = bindings.service_annotations ?: false
|
||||
bindings['pod_annotations'] = bindings.pod_annotations ?: false
|
||||
bindings['kind'] = bindings.kind ?: 'deployment'
|
||||
bindings['statefulset'] = bindings.statefulset ?: statefulset_default
|
||||
bindings['contourResponseTimeout'] = bindings.contourResponseTimeout ?: false
|
||||
bindings['pdbMinAvailable'] = bindings.pdbMinAvailable ?: ''
|
||||
bindings['pdbMaxUnavailable'] = bindings.pdbMaxUnavailable ?: '10%'
|
||||
|
||||
// Just to keep compatibility for services which still use grpc_port
|
||||
// If someone has supplied primary_port, then it is used
|
||||
// Else we check for grpc_port, and that is used
|
||||
// If none of the above is supplied, then app_port is used just like normal flow
|
||||
bindings['primary_port'] = bindings.primary_port ?: bindings.grpc_port ?: bindings.app_port
|
||||
bindings['grpc_port'] = bindings.grpc_port ?: false
|
||||
// bindings['xmx'] = bindings.xmx ?: '50.0'
|
||||
// bindings['xms'] = bindings.xms ?: '50.0'
|
||||
bindings['enableWebsocket'] = bindings.enableWebsocket ?: false
|
||||
bindings['external_secrets_annotations'] = bindings.external_secrets_annotations ?: ''
|
||||
bindings['liveness_failure_threshold'] = bindings.liveness_failure_threshold ?: '5'
|
||||
bindings['liveness_period_seconds'] = bindings.liveness_period_seconds ?: bindings.team_norm == 'ml-platform' ? '5' : '10'
|
||||
bindings['liveness_success_threshold'] = bindings.liveness_success_threshold ?: '1'
|
||||
bindings['liveness_timeout_seconds'] = bindings.liveness_timeout_seconds ?: '2'
|
||||
bindings['readiness_failure_threshold'] = bindings.readiness_failure_threshold ?: '5'
|
||||
bindings['readiness_period_seconds'] = bindings.liveness_period_seconds ?: bindings.team_norm == 'ml-platform' ? '5' : '10'
|
||||
bindings['readiness_success_threshold'] = bindings.liveness_success_threshold ?: '1'
|
||||
bindings['readiness_timeout_seconds'] = bindings.liveness_timeout_seconds ?: '2'
|
||||
bindings['addon_ports'] = bindings.addon_ports ?: false
|
||||
|
||||
validateRequiredMetadata(bindings)
|
||||
|
||||
bindings['priority_v2'] = bindings.priority_v2 ?: 'cp3'
|
||||
bindings['primary_owner'] = bindings.primary_owner != null ? bindings.primary_owner.split('@')[0] : bindings.team_norm
|
||||
bindings['secondary_owner'] = bindings.secondary_owner != null ? bindings.secondary_owner.split('@')[0] : bindings.team_norm
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
if (bindings.otel_enabled == null) {
|
||||
bindings['otel_enabled'] = (bindings.priority_v2 == 'cp1' || bindings.priority_v2 == 'up1') ? true : false
|
||||
}
|
||||
else {
|
||||
bindings['otel_enabled'] = bindings.otel_enabled
|
||||
}
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
echo "Checking for otel value - ${bindings.otel_enabled}"
|
||||
if (bindings.otel_enabled == null) {
|
||||
bindings['otel_enabled'] = true
|
||||
}
|
||||
else {
|
||||
bindings['otel_enabled'] = bindings.otel_enabled
|
||||
}
|
||||
|
||||
// Setting metrics mode -> avaliable options: telegraf | otel | dual
|
||||
echo "Setting metrics mode - ${bindings.metrics_mode}"
|
||||
// Setting default values
|
||||
bindings['telegraf_metrics'] = false
|
||||
bindings['otel_metrics'] = false
|
||||
bindings['metrics_mode'] = bindings.metrics_mode?.toLowerCase()
|
||||
def samplerArg = bindings.otel_traces_sampler_arg
|
||||
def isValidSamplerArg = false
|
||||
if (samplerArg != null && (samplerArg instanceof Float || samplerArg instanceof Double)) {
|
||||
if (samplerArg >= 0.0 && samplerArg <= 1.0) {
|
||||
isValidSamplerArg = true
|
||||
}
|
||||
}
|
||||
bindings['otel_traces_sampler_arg'] = isValidSamplerArg ? samplerArg : '0.1'
|
||||
log.info("1 - otel_traces_sampler_arg - ${bindings.otel_traces_sampler_arg}")
|
||||
log.info("2 - ${bindings['otel_traces_sampler_arg']}")
|
||||
// Enabling metrics based on mode
|
||||
if (bindings.metrics_mode == 'telegraf') {
|
||||
bindings['telegraf_metrics'] = true
|
||||
}
|
||||
else if (bindings.metrics_mode == 'otel') {
|
||||
bindings['otel_metrics'] = true
|
||||
}
|
||||
else if (bindings.metrics_mode == 'dual') {
|
||||
bindings['telegraf_metrics'] = true
|
||||
bindings['otel_metrics'] = true
|
||||
}
|
||||
else {
|
||||
bindings['metrics_mode'] = 'telegraf'
|
||||
bindings['telegraf_metrics'] = true
|
||||
}
|
||||
}
|
||||
bindings['command'] = bindings.command
|
||||
if (bindings.command == null) {
|
||||
bindings['command'] = get_default_command(bindings['dockerBuildVersion'])
|
||||
}
|
||||
bindings['as_down_stable_window'] = bindings.as_down_stable_window ?: '1800'
|
||||
bindings['podDistributionSkew'] = bindings.podDistributionSkew ?: false
|
||||
bindings["appConfigEnabled"] = bindings.appConfigEnabled ?: false
|
||||
bindings["addHeadless"] = bindings.addHeadless?: false
|
||||
}
|
||||
|
||||
def get_value_yaml_file(def dockerBuildVersion) {
|
||||
switch (dockerBuildVersion) {
|
||||
case ~/^maven-.*/: return 'values.yaml'
|
||||
case ~/^node-.*/: return 'node-values.yaml'
|
||||
case ~/^python-.*/: return 'python-values.yaml'
|
||||
case ~/^go.*/: return 'go-values.yaml'
|
||||
case 'php': return 'php-values.yaml'
|
||||
case 'gradle': return 'values.yaml'
|
||||
}
|
||||
}
|
||||
|
||||
def get_default_command(def dockerBuildVersion) {
|
||||
switch (dockerBuildVersion) {
|
||||
case ~/^maven-.*/: return 'java'
|
||||
case ~/^node-.*/: return 'pm2-runtime'
|
||||
case ~/^go.*/: return '/app/server'
|
||||
case 'php': return 'apache2-foreground'
|
||||
case 'gradle': return 'java'
|
||||
default: return null
|
||||
}
|
||||
}
|
||||
|
||||
def validateRequiredMetadata(def bindings) {
|
||||
def requiredParams = ['primary_owner', 'secondary_owner', 'priority_v2', 'service_type']
|
||||
def serviceOwners = ['primary_owner', 'secondary_owner']
|
||||
// Validate if required parameters are present
|
||||
for (param in requiredParams) {
|
||||
if (bindings[param] == null || bindings[param] == '') {
|
||||
env.msg = 'You have not supplied ' + param + '. Exiting the pipeline.'
|
||||
log.error(env.msg)
|
||||
sh 'exit 1'
|
||||
}
|
||||
}
|
||||
|
||||
// Validate if the owners are valid or not
|
||||
if (env.cicd_environment == 'dev' || env.cicd_environment == 'ftr' || env.cicd_environment == 'stg') {
|
||||
return
|
||||
}
|
||||
|
||||
for (param in serviceOwners) {
|
||||
final String owner = bindings[param]
|
||||
final String url = "https://pulse.homelabgcp.in/api/anonymous-User/userexist?email=${owner}"
|
||||
final def(String response, String code) = sh(returnStdout: true, script: """
|
||||
set +x
|
||||
curl -s -X GET -w '\n%{response_code}' $url
|
||||
set -x
|
||||
""").trim().tokenize('\n')
|
||||
|
||||
if (code != "200") {
|
||||
// Let's not break the pipeline in case the API fails
|
||||
log.info("Received ${code} code from Pulse while checking for user. Skipping the checks further and letting the pipeline proceed.")
|
||||
} else {
|
||||
// Check if the user exists
|
||||
def jqCommand = "echo '${response}' | jq -r '.exists'"
|
||||
def userExists = sh(returnStdout: true, script: """
|
||||
set +x
|
||||
${jqCommand}
|
||||
set -x
|
||||
""").trim()
|
||||
if (userExists != "true") {
|
||||
env.msg = "Invalid value provided in ${param}. Check if the user ${owner} exists"
|
||||
log.error(env.msg)
|
||||
sh 'exit 1'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def dependabotCriticalCheck(def repoName) {
|
||||
def repo = repoName
|
||||
|
||||
def alerts
|
||||
withCredentials([usernamePassword(credentialsId: "${env.GITHUB_CRED}", usernameVariable:'user', passwordVariable: 'token')]) {
|
||||
validateDependabot = httpRequest httpMode: 'GET',
|
||||
customHeaders: [
|
||||
[name: 'Accept', value: 'application/vnd.github+json'],
|
||||
[maskValue: true, name: 'Authorization', value: 'Bearer ' + token]
|
||||
],
|
||||
url: "https://api.github.com/repos/homelab/${repo}/dependabot/alerts?state=open&per_page=100",
|
||||
validResponseCodes: '200',
|
||||
timeout: 10
|
||||
alerts = readJSON(text: validateDependabot.content)
|
||||
|
||||
}
|
||||
|
||||
// if (!alerts) {
|
||||
// println "Failed to fetch alerts for ${repo}."
|
||||
// return false
|
||||
// }
|
||||
|
||||
// Parse the JSON response
|
||||
if (!alerts) {
|
||||
println "No alerts found for ${repo}."
|
||||
return false
|
||||
}
|
||||
|
||||
def criticalCount = 0
|
||||
alerts.each { alert ->
|
||||
def severity = alert?.security_vulnerability?.severity
|
||||
|
||||
if (severity == "critical") {
|
||||
criticalCount++
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
if (criticalCount > 0) {
|
||||
println "Critical vulnerabilities found in ${repo}: ${criticalCount}"
|
||||
return true // Critical vulnerabilities found
|
||||
} else {
|
||||
println "No critical vulnerabilities in ${repo}."
|
||||
return false // No critical vulnerabilities found
|
||||
}
|
||||
}
|
||||
@@ -1,32 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
def run(Map param){
|
||||
//get the jar for deployment, get it from the params module
|
||||
//run the ansible playbook
|
||||
//create the inventory file with given ip in parameters
|
||||
def server_ip = param.run_automation.server_ip
|
||||
def app_name = param.run_automation.app_name
|
||||
def healthcheck_api = param.run_automation.healthcheck_api
|
||||
def app_port = param.run_automation.app_port
|
||||
def repo_name = param.repo_name
|
||||
|
||||
deploy(server_ip,repo_name,app_name,healthcheck_api,app_port)
|
||||
}
|
||||
|
||||
def deploy(String server_ip, String repo_name, String app_name, String healthcheck_api, String app_port){
|
||||
try{
|
||||
stage('Deploying JAR'){
|
||||
sh "echo '$server_ip' > host_file.txt"
|
||||
echo "inventory created"
|
||||
def playbook_content = libraryResource 'com/homelab/deployJar.yaml'
|
||||
writeFile file:"deployJar.yaml", text: playbook_content
|
||||
sh(returnStatus: true, script: "ansible-playbook -i host_file.txt -u 'ubuntu' -e 'env=stage' -e 'app_name=${app_name}' -e 'repo_name=${repo_name}' -e 'healthcheck_api=${healthcheck_api}' -e 'app_port=${app_port}' deployJar.yaml -v")
|
||||
}
|
||||
}
|
||||
catch( Exception e) {
|
||||
env.msg = "Error while deploying JAR. Please check console output for more details."
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
@@ -1,144 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.getYamlParameter
|
||||
import java.time.ZonedDateTime
|
||||
import java.time.format.DateTimeFormatterBuilder
|
||||
|
||||
def run(String repo_name, def deployment_order, def tag, def build_team, def dockerBuildVersion, String notify_channel){
|
||||
def yamlobj = new getYamlParameter()
|
||||
def applicationNames = []
|
||||
def jobStatusMap=["SUCCESS": "BUILD_STATUS_COMPLETED","FAILURE": "BUILD_STATUS_FAILED","UNSTABLE":"BUILD_STATUS_FAILED"]
|
||||
def build_user = currentBuild.rawBuild.getCause(Cause.UserIdCause).getUserId()
|
||||
|
||||
for (deployment in deployment_order){
|
||||
def value_binding = yamlobj.getParam("${repo_name}/deployments","${deployment}.yaml")
|
||||
applicationNames.add(value_binding['app_name'])
|
||||
}
|
||||
def end_time = getDateTime()
|
||||
def working_env = (env.CLOUD_PROVIDER == "GCP")? "gcp_${env.cicd_environment}" : env.cicd_environment
|
||||
|
||||
def jsonMap = [:]
|
||||
jsonMap["hot_fix"] = env.hot_fix ? true : false
|
||||
jsonMap["job_name"] = env.JOB_NAME ? env.JOB_NAME.split('/')[0] : "unknown"
|
||||
jsonMap["build_no"] = env.BUILD_NUMBER
|
||||
jsonMap["image"] = tag
|
||||
jsonMap["applications"] = applicationNames
|
||||
jsonMap["job_status"] = jobStatusMap[currentBuild?.currentResult]
|
||||
jsonMap["err_msg"] = env.error_msg_to_db
|
||||
jsonMap["end_time"] = end_time
|
||||
jsonMap["email"] = currentBuild?.rawBuild?.getCause(Cause.UserIdCause)?.getUserId()
|
||||
jsonMap["start_time"] = env.STARTTIME
|
||||
jsonMap["build_team"] = build_team
|
||||
jsonMap["docker_build_version"] = dockerBuildVersion
|
||||
jsonMap["notify_channel"] = notify_channel
|
||||
jsonMap["branch_name"] = env.CHANGE_ID ? env.CHANGE_BRANCH : env.BRANCH_NAME
|
||||
jsonMap["commit_id"] = env.commit_id
|
||||
jsonMap["deploy_argo"] = env.deployArgo != null ? env.deployArgo.toBoolean() : false
|
||||
jsonMap["pr_number"] = env.CHANGE_ID ? env.CHANGE_ID.toInteger() : 0
|
||||
|
||||
final String baseUrl
|
||||
|
||||
switch (env.cicd_environment) {
|
||||
case 'prd':
|
||||
case 'int':
|
||||
baseUrl = 'https://ringmaster-api.homelabgcp.in/api/v1/key/cicd/cd/update'
|
||||
break
|
||||
default:
|
||||
baseUrl = 'https://ringmaster-api.admin.homelabgcp.in/api/v1/key/cicd/cd/update'
|
||||
}
|
||||
final String url = "${baseUrl}?workingEnv=${working_env}"
|
||||
final String header = "Content-Type: application/json"
|
||||
final String jsonData = writeJSON returnText: true, json: jsonMap
|
||||
|
||||
|
||||
if (build_user == "ringmaster-bot"){
|
||||
callApi(url, header, jsonData)
|
||||
} else{
|
||||
def newCICD_Payload = [:]
|
||||
newCICD_Payload["repo_name"] = repo_name
|
||||
newCICD_Payload["source_branch"] = env.CHANGE_ID ? env.CHANGE_BRANCH : env.BRANCH_NAME
|
||||
newCICD_Payload["pull_request_number"] = env.CHANGE_ID ? env.CHANGE_ID.toInteger() : 0
|
||||
newCICD_Payload["env"] = env.cicd_environment
|
||||
newCICD_Payload["job_name"] = env.JOB_NAME ? env.JOB_NAME.split('/')[0] : "UNKNOWN"
|
||||
newCICD_Payload["sub_job_name"] = env.JOB_NAME ? env.JOB_NAME.split('/')[1] : "UNKNOWN"
|
||||
newCICD_Payload["build_number"] = env.BUILD_NUMBER.toInteger()
|
||||
newCICD_Payload["image_tag"] = tag
|
||||
newCICD_Payload["build_detailed_error"] = env.error_msg_to_db
|
||||
switch (env.cicd_environment) {
|
||||
case 'prd':
|
||||
case 'int':
|
||||
cicdBaseUrl = 'http://turbo-turtle.homelabgcp.in'
|
||||
break
|
||||
default:
|
||||
cicdBaseUrl = 'http://turbo-turtle.admin.homelabgcp.in'
|
||||
}
|
||||
final String newCICD_JSON = writeJSON returnText: true, json: newCICD_Payload
|
||||
//log.info("New CICD JSON - ${newCICD_JSON}")
|
||||
final String newCICD_URL = cicdBaseUrl + "/api/v1/ci/jenkins/callback"
|
||||
final String newCICD_Header = "Content-Type: application/json"
|
||||
// Use a temporary file to store the JSON payload
|
||||
// This avoids shell quoting issues completely
|
||||
final String jsonFilePath = "cicd_payload_${env.BUILD_NUMBER}_${System.currentTimeMillis()}.json"
|
||||
|
||||
try {
|
||||
// 1. Write the JSON payload to a temporary file
|
||||
// The writeJSON step ensures the content is valid JSON, escaping internal characters
|
||||
writeFile(file: jsonFilePath, text: writeJSON(returnText: true, json: newCICD_Payload))
|
||||
final String newCICD_JSON_log = readFile(file: jsonFilePath)
|
||||
log.info("New CICD JSON (from file) - ${newCICD_JSON_log.take(500)}...") // Log a snippet
|
||||
final def(String response, String code) = sh(
|
||||
returnStdout: true,
|
||||
script: """
|
||||
curl -s -X POST \\
|
||||
-H '$newCICD_Header' \\
|
||||
-w '\\n%{response_code}' \\
|
||||
$newCICD_URL \\
|
||||
-d @$jsonFilePath
|
||||
"""
|
||||
).trim().tokenize("\n")
|
||||
|
||||
if (code != "200") {
|
||||
log.error("CICD Application API call failed with error code - ${code}, response - ${response}")
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("CICD Application API call failed - " + "Error: " + e.toString())
|
||||
throw e
|
||||
} finally {
|
||||
sh(script: "rm -f ${jsonFilePath}", returnStatus: true)
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
def callApi(String url, String header, String jsonData){
|
||||
try{
|
||||
withCredentials([usernamePassword(credentialsId: "ringmaster-token", usernameVariable:'user', passwordVariable: 'token')]){
|
||||
final def(String response, String code) = sh(returnStdout: true, script: "curl -s -X POST -H '$header' -H 'Authorization: $token' -w '\\n%{response_code}' $url -d '$jsonData'").trim().tokenize("\n")
|
||||
log.info("HTTP response status code : ${code}")
|
||||
if(code != "200"){
|
||||
log.error("API call failed with error code - ${code}, response - ${response}")
|
||||
currentBuild.result = 'FAILURE'
|
||||
}
|
||||
}
|
||||
}
|
||||
catch ( Exception e) {
|
||||
log.error("API call failed")
|
||||
currentBuild.result = 'FAILURE'
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
@NonCPS
|
||||
def getDateTime() {
|
||||
// Get the current date and time in IST
|
||||
def currentDateTime = ZonedDateTime.now()
|
||||
// Create a formatter for the desired pattern
|
||||
def formatter = new DateTimeFormatterBuilder()
|
||||
.appendPattern("yyyy-MM-dd'T'HH:mm:ss")
|
||||
.appendOffset("+HH:mm", "+00:00")
|
||||
.toFormatter()
|
||||
// Format the current date and time using the formatter
|
||||
def formattedDateTime = currentDateTime.format(formatter)
|
||||
return formattedDateTime;
|
||||
}
|
||||
|
||||
@@ -1,185 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.utilities.gitActions
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.getDockerParams
|
||||
import com.homelab.stages.multiBranchPipeline
|
||||
|
||||
def run(Map params) {
|
||||
def btObj = new buTeamMapping()
|
||||
def gitObj = new gitActions()
|
||||
|
||||
def environments = ['stg':'develop', 'int':'pre-prod', 'prd':'main', 'ftr':'feature']
|
||||
final Map<?, ?> modifiedParams = new HashMap<>(params)
|
||||
|
||||
// def helm_repo_name = 'devops-helm-charts'
|
||||
// def argoRepo = "devops-argo-config"
|
||||
|
||||
def deployment_order = modifiedParams.app_names.replaceAll('\\s', '').split(', ') as List
|
||||
|
||||
modifiedParams['deployment_order'] = deployment_order
|
||||
|
||||
stage('Create Jenkinsfile') {
|
||||
jenkinsfileCreate(modifiedParams)
|
||||
}
|
||||
|
||||
modifiedParams['bu'] = btObj.get_bu_initials(modifiedParams['bu'])
|
||||
modifiedParams['team'] = btObj.get_team_initials(modifiedParams['team'])
|
||||
log.info(modifiedParams)
|
||||
|
||||
stage('Update Helm charts') {
|
||||
gitObj.clone("${WORKSPACE}", "${env.helm_repo_name}", null)
|
||||
environments.each {
|
||||
modifiedParams['environment'] = it.key
|
||||
def helm_branch_name = it.value
|
||||
for (deployment in deployment_order) {
|
||||
modifiedParams['app_name'] = deployment
|
||||
modifiedParams['host'] = helm_branch_name == 'feature' ? "INGRESS_PR_NUMBER-${deployment}.dev.internal.homelabtest.in" : "${deployment}.${it.key}.internal.homelabtest.in"
|
||||
modifiedParams['host'] = (helm_branch_name == 'pre-prod') ? "${deployment}.${modifiedParams.bu}.internal.homelab.co" : modifiedParams.host
|
||||
//If branch is main, then assuming the env as prod
|
||||
//Making nodeSelector changes to hypercore or hypermem & based on arch, so that it moves to common node pool
|
||||
if (helm_branch_name == 'main') {
|
||||
def cpu = modifiedParams['cpu_request']
|
||||
def memory = modifiedParams['memory_request']
|
||||
def nodeSelector = 'hypercore'
|
||||
|
||||
if (cpu.contains('m')) {
|
||||
def numericValue = cpu.replaceAll('\\D+', '').toDouble()
|
||||
def gb = numericValue / 1000
|
||||
cpu = gb.toString()
|
||||
}
|
||||
def mem_dgt = memory.replaceAll('\\D+', '').toDouble()
|
||||
memory = memory.contains('Mi') ? mem_dgt / 1024 : mem_dgt
|
||||
def ratio = cpu.toDouble() / memory.toDouble()
|
||||
def arch = modifiedParams['arch']
|
||||
def graviton_required = false
|
||||
if (arch == 'arm64') {
|
||||
graviton_required = true
|
||||
}
|
||||
if (ratio <= 1 / 3) {
|
||||
if (graviton_required) {
|
||||
nodeSelector = 'hypermem-arm64'
|
||||
} else {
|
||||
nodeSelector = 'hypermem'
|
||||
}
|
||||
} else {
|
||||
if (graviton_required) {
|
||||
nodeSelector = 'hypercore-arm64'
|
||||
} else {
|
||||
nodeSelector = 'hypercore'
|
||||
}
|
||||
}
|
||||
modifiedParams['nodeSelector'] = nodeSelector
|
||||
}
|
||||
updateHelmRepo(modifiedParams, env.helm_repo_name, helm_branch_name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// stage("Update Argo Application"){
|
||||
// gitObj.clone("${WORKSPACE}", "${argoRepo}", null)
|
||||
// environments.each{
|
||||
// modifiedParams["environment"] = it.key
|
||||
// def argo_branch_name = it.value
|
||||
// for (deployment in deployment_order){
|
||||
// modifiedParams["app_name"] = deployment
|
||||
// updateArgoRepo(modifiedParams,argoRepo,argo_branch_name)
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
|
||||
stage('Create ECR repo') {
|
||||
def buildDockerObj = new getDockerParams()
|
||||
def modules = buildDockerObj.getModules(modifiedParams['repo_name'])
|
||||
if (modules == null) {
|
||||
modules = ['module_less']
|
||||
}
|
||||
for (module in modules) {
|
||||
def ecr_repo_name = (module == 'module_less') ? "${modifiedParams.team}/${modifiedParams.repo_name.toLowerCase()}" : "${modifiedParams.team}/${modifiedParams.repo_name.toLowerCase()}/${module}"
|
||||
try {
|
||||
sh "aws ecr create-repository --repository-name ${ecr_repo_name}"
|
||||
}
|
||||
catch (Exceptione) {
|
||||
log.info('repository already present')
|
||||
}
|
||||
}
|
||||
}
|
||||
stage('Create Multibranch pipeline') {
|
||||
def multiBranchPipelineObj = new multiBranchPipeline()
|
||||
multiBranchPipelineObj.applicationOnboard(params['bu'], params['repo_name'])
|
||||
}
|
||||
}
|
||||
|
||||
def jenkinsfileCreate(Map config) {
|
||||
def constructObj = new constructTemplate()
|
||||
def gitObj = new gitActions()
|
||||
def repo_name = config.repo_name
|
||||
def deployment_order = config.deployment_order
|
||||
def repo_branch_name = 'eks_onboarding'
|
||||
config['branch_params'] = ''
|
||||
config['excludedMoudles'] = config.excludedMoudles.replaceAll('\\s', '').split(', ') as List
|
||||
gitObj.clone("${WORKSPACE}", "${config.repo_name}", null)
|
||||
gitObj.branchCheckOut(config.repo_name, repo_branch_name)
|
||||
deployment_yaml_file = get_deployment_yaml_file(config['dockerBuildVersion'])
|
||||
dir(repo_name) {
|
||||
sh(script:'mkdir -p deployments/')
|
||||
sh 'chmod -R 777 .'
|
||||
constructObj.renderTemplate(config, 'Jenkinsfile', 'Jenkinsfile')
|
||||
gitObj.add('.', 'Jenkinsfile')
|
||||
|
||||
constructObj.renderTemplate(config, 'config.yaml', 'config.yaml')
|
||||
gitObj.add('.', 'config.yaml')
|
||||
|
||||
dir('deployments') {
|
||||
for (deployment in deployment_order) {
|
||||
config['app_name'] = deployment
|
||||
constructObj.renderTemplate(config, deployment_yaml_file, deployment + '.yaml')
|
||||
gitObj.add('.', deployment + '.yaml')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
gitObj.codeCommit(repo_name, repo_branch_name, 'Generating build and deployments files')
|
||||
gitObj.codePush(repo_name, repo_branch_name)
|
||||
}
|
||||
|
||||
def updateHelmRepo(Map config, String helm_repo_name, String helm_branch_name) {
|
||||
def constructObj = new constructTemplate()
|
||||
def gitObj = new gitActions()
|
||||
def filepath = "${env.helmChartsPath}/${config.bu}/${config.team}/${config.app_name}"
|
||||
def file_name = 'values_properties.yaml'
|
||||
def app_branch = config.app_name + '-' + helm_branch_name + '-helm'
|
||||
def pr_num = ''
|
||||
def commit_status = 0
|
||||
|
||||
gitObj.branchCheckOut(env.helm_repo_name, helm_branch_name)
|
||||
gitObj.branchCheckOut("${env.helm_repo_name}", "${app_branch}")
|
||||
// create filepath
|
||||
dir(env.helm_repo_name) {
|
||||
sh(script:"mkdir -p ${filepath}" ,returnStdout:true)
|
||||
sh 'chmod -R 777 .'
|
||||
constructObj.renderTemplate(config, file_name, filepath + '/' + file_name)
|
||||
}
|
||||
|
||||
// git commit and push
|
||||
gitObj.add(env.helm_repo_name, filepath + '/' + file_name)
|
||||
commit_status = gitObj.codeCommit(env.helm_repo_name, app_branch, 'Generating properties values yaml file')
|
||||
if (commit_status == 0) {
|
||||
gitObj.codePush(env.helm_repo_name, app_branch)
|
||||
pr_num = gitObj.createPR(config.app_name, env.helm_repo_name, helm_branch_name, app_branch, 'Merge helm values properties')
|
||||
gitObj.mergePR(env.helm_repo_name, pr_num, app_branch)
|
||||
gitObj.deleteBranch(env.helm_repo_name, helm_branch_name, app_branch)
|
||||
}
|
||||
}
|
||||
|
||||
def get_deployment_yaml_file(def dockerBuildVersion) {
|
||||
switch(dockerBuildVersion) {
|
||||
case ~/^maven-.*/: return 'deployment.yaml'
|
||||
case ~/^node-.*/: return 'node-deployment.yaml'
|
||||
case ~/^python-.*/: return 'python-deployment.yaml'
|
||||
case ~/^go.*/: return 'go-deployment.yaml'
|
||||
case 'gradle': return 'gradle-deployment.yaml'
|
||||
case 'php': return 'php-deployment.yaml'
|
||||
}
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
def run(String repo_name){
|
||||
stage(stageName("Check for Hot Fix")){
|
||||
dir(repo_name){
|
||||
if (env.CHANGE_ID) {
|
||||
def source_branch_name = env.CHANGE_BRANCH
|
||||
log.info("Source-Branch : ${source_branch_name}")
|
||||
source_branch_name = source_branch_name.toLowerCase()
|
||||
if (source_branch_name.contains("hotfix_")){
|
||||
env.hot_fix = true
|
||||
log.info("***************** Enabling Hot-fix workflow *****************")
|
||||
}
|
||||
else if (source_branch_name.matches('^revert-\\d+-.+$')){
|
||||
env.hot_fix = true
|
||||
log.info("***************** Enabling Hot-fix workflow for revert branch *****************")
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info("**** Branch: ${env.BRANCH_NAME}, So Skipping Hot-fix check ****")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,73 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import jxl.*
|
||||
import hudson.util.PersistedList
|
||||
import jenkins.model.Jenkins
|
||||
import jenkins.branch.*
|
||||
import jenkins.plugins.git.*
|
||||
import org.jenkinsci.plugins.workflow.multibranch.*
|
||||
|
||||
import com.cloudbees.hudson.plugins.folder.*
|
||||
import org.jenkinsci.plugins.github_branch_source.*
|
||||
import org.jenkinsci.plugins.workflow.libs.*
|
||||
import hudson.scm.SCM
|
||||
import hudson.plugins.git.*
|
||||
import net.gleske.scmfilter.impl.trait.*
|
||||
|
||||
def applicationOnboard(String foldername, String repo_name) {
|
||||
// Bring some values in from ansible using the jenkins_script modules wierd "args" approach (these are not gstrings)
|
||||
String folderName = "${foldername}"
|
||||
String repoName = "${repo_name}"
|
||||
String scriptPath = "Jenkinsfile"
|
||||
String gitRepo = "https://github.com/Homelab/${repo_name}.git"
|
||||
String mBPName = "${repo_name}-cicd"
|
||||
String credentialsId = env.GITHUB_CRED
|
||||
|
||||
Jenkins jenkins = Jenkins.instance // saves some typing
|
||||
|
||||
// Get the folder where this job should be
|
||||
// def folder = jenkins.getItem(folderName)
|
||||
// //Create the folder if it doesn't exist
|
||||
// if (folder == null) {
|
||||
// folder = jenkins.createProject(Folder.class, folderName)
|
||||
// }
|
||||
|
||||
// Multibranch creation/update
|
||||
WorkflowMultiBranchProject mbp
|
||||
def view = jenkins.getView(foldername)
|
||||
Item item = jenkins.getItem(mBPName)
|
||||
if ( item != null ) {
|
||||
// Update case
|
||||
mbp = (WorkflowMultiBranchProject) item
|
||||
} else {
|
||||
// Create case
|
||||
mbp = jenkins.createProject(WorkflowMultiBranchProject.class, mBPName)
|
||||
}
|
||||
|
||||
// Configure the script this MBP uses
|
||||
mbp.getProjectFactory().setScriptPath(scriptPath)
|
||||
|
||||
def implicit = false
|
||||
def defaultVersion = "master"
|
||||
def traits = []
|
||||
|
||||
GitHubSCMSource gitHubSCMSource = new GitHubSCMSource("Homelab", repoName, gitRepo, implicit)
|
||||
gitHubSCMSource.credentialsId = credentialsId
|
||||
|
||||
BranchDiscoveryTrait branchDiscoveryTrait = new BranchDiscoveryTrait(3)
|
||||
OriginPullRequestDiscoveryTrait pullRequestTrait = new OriginPullRequestDiscoveryTrait(1)
|
||||
WildcardSCMHeadFilterTrait wildcardSCMHeadFilterTrait = new WildcardSCMHeadFilterTrait('gcp-main*','','','*')
|
||||
traits.add(branchDiscoveryTrait)
|
||||
traits.add(pullRequestTrait)
|
||||
traits.add(wildcardSCMHeadFilterTrait)
|
||||
gitHubSCMSource.setTraits(traits)
|
||||
BranchSource branchSource = new BranchSource(gitHubSCMSource)
|
||||
NoTriggerBranchProperty noTriggerBranchProperty = new NoTriggerBranchProperty()
|
||||
BranchProperty[] ntbp = [noTriggerBranchProperty]
|
||||
branchSource.setStrategy(new DefaultBranchPropertyStrategy(ntbp))
|
||||
|
||||
PersistedList sources = mbp.getSourcesList()
|
||||
sources.clear()
|
||||
sources.add(branchSource)
|
||||
view.add(mbp)
|
||||
}
|
||||
@@ -93,21 +93,15 @@ def buildRunCommand(String script, String interpreter, String requirements) {
|
||||
}
|
||||
|
||||
boolean isPython = (interp == 'python3' || interp == 'python' || script.endsWith('.py'))
|
||||
// docker:27-cli is minimal Alpine — only `sh` is guaranteed present.
|
||||
// bash/python3 need installing on demand, unlike the real system's
|
||||
// pod images which already bundle a full toolchain.
|
||||
String installLine = ''
|
||||
if (isPython) {
|
||||
installLine = 'apk add --no-cache python3 py3-pip py3-virtualenv >/dev/null'
|
||||
} else if (interp == 'bash') {
|
||||
installLine = 'apk add --no-cache bash >/dev/null'
|
||||
}
|
||||
// bash/python3+pip/venv are baked into the docker-cli image (see
|
||||
// build-tools.Dockerfile) — used to apk-install these on demand on
|
||||
// every single hook invocation, unlike the real system's pod images
|
||||
// which already bundle a full toolchain.
|
||||
|
||||
if (requirements && isPython) {
|
||||
String py = interp ?: 'python3'
|
||||
return """
|
||||
set -e
|
||||
${installLine}
|
||||
${py} -m venv .hook_venv
|
||||
. .hook_venv/bin/activate
|
||||
pip install --quiet --disable-pip-version-check -r ${requirements}
|
||||
@@ -116,7 +110,7 @@ def buildRunCommand(String script, String interpreter, String requirements) {
|
||||
}
|
||||
|
||||
if (interp) {
|
||||
return "set -e\n${installLine}\n${interp} ${script}"
|
||||
return "set -e\n${interp} ${script}"
|
||||
}
|
||||
// No interpreter resolved — rely on the script's shebang.
|
||||
return "set -e\nchmod +x ${script}\n./${script}"
|
||||
|
||||
@@ -1,24 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
def run(Map config) {
|
||||
try {
|
||||
def repo_name = config.repo_name
|
||||
stage('Security scan') {
|
||||
if (config.skip_security_scan) {
|
||||
log.info('Skipping - Security Scan')
|
||||
}
|
||||
else {
|
||||
final String url = '172.31.5.29:63232/scans'
|
||||
final def(String response, String code) = sh(returnStdout: true, script: "curl -s -X POST -H 'Content-Type: application/json' -w '\\n%{response_code}' $url -d '{\"reponame\":\"$repo_name\",\"branch\":\"$BRANCH_NAME\"}'").trim().tokenize('\n')
|
||||
log.info("HTTP response status code : ${code}")
|
||||
log.info("Response: ${response}")
|
||||
}
|
||||
}
|
||||
}
|
||||
catch ( Exception e) {
|
||||
env.msg = 'failed in security scan . Please check console output for more details.'
|
||||
log.error(env.msg)
|
||||
currentBuild.result = env.FAILURE
|
||||
throw e
|
||||
}
|
||||
}
|
||||
@@ -16,18 +16,29 @@ package com.homelab.stages
|
||||
// empty (bootstrap token not generated yet), fail loudly here with a
|
||||
// clear message rather than a confusing curl auth error.
|
||||
//
|
||||
// The emptiness check has to happen inside the shell script, not as a
|
||||
// Groovy `env.ARGOCD_TOKEN` check before it — env.X in Groovy is
|
||||
// Jenkins' own pipeline-level environment map (build parameters,
|
||||
// environment{} blocks, withEnv, etc.), which a container-scoped env:
|
||||
// entry in a podTemplate YAML never populates. The container's real OS
|
||||
// environment does have it (visible to sh, which inherits the
|
||||
// container's actual process environment) — checking env.ARGOCD_TOKEN
|
||||
// in Groovy was always going to see null regardless of whether the
|
||||
// Secret/ESO/Vault chain was correctly wired, which is exactly what
|
||||
// happened: every fix to the secret chain made no difference because
|
||||
// the check itself was looking in the wrong place.
|
||||
//
|
||||
// Expects in config:
|
||||
// argo_app_name the Application's metadata.name, e.g. demo-go-app
|
||||
// argo_server_url e.g. http://argocd-admin-prd-server.argocd.svc.cluster.local
|
||||
def run(Map config) {
|
||||
stage(stageName('Sync ArgoCD Application')) {
|
||||
container('docker-cli') {
|
||||
if (!env.ARGOCD_TOKEN?.trim()) {
|
||||
log.error('ARGOCD_TOKEN is empty — the jenkins-ci account token has not been generated yet. See secretstores/argocd-jenkins-ci-token.yaml for the one-time bootstrap steps.')
|
||||
error('Skipping ArgoCD sync: no token available.')
|
||||
}
|
||||
sh """
|
||||
apk add --no-cache curl >/dev/null
|
||||
if [ -z "\$ARGOCD_TOKEN" ]; then
|
||||
echo "ARGOCD_TOKEN is empty — the jenkins-ci account token has not been generated yet. See secretstores/argocd-jenkins-ci-token.yaml for the one-time bootstrap steps." >&2
|
||||
exit 1
|
||||
fi
|
||||
curl -sf -X POST \\
|
||||
-H "Authorization: Bearer \$ARGOCD_TOKEN" \\
|
||||
-H "Content-Type: application/json" \\
|
||||
|
||||
@@ -16,7 +16,9 @@ package com.homelab.stages
|
||||
// service_name second path segment; same as repo_name for a
|
||||
// single-service repo, different for a monorepo with
|
||||
// several services sharing one repo
|
||||
// helm_repo_url e.g. http://gitea.192.168.1.7.nip.io/mukul/devops-helm-charts.git
|
||||
// helm_repo_url e.g. http://gitea-http.gitea.svc.cluster.local:3000/gitadmin/devops-helm-charts-gcp.git
|
||||
// (cluster DNS — this clone runs in a build pod, so it
|
||||
// never goes out through the ingress and back)
|
||||
// image_tag_yq_path yq path to the tag field, e.g. .deployment.image.tag
|
||||
// gitea_cred Jenkins credential ID for a Gitea push-capable token (default: gitea-ci-credentials)
|
||||
//
|
||||
@@ -25,46 +27,67 @@ package com.homelab.stages
|
||||
def run(Map config) {
|
||||
def valuesFile = "values/${config.repo_name}/${config.service_name}/values.yaml"
|
||||
stage(stageName('Update Helm chart image tag')) {
|
||||
withCredentials([usernamePassword(credentialsId: config.gitea_cred ?: 'gitea-ci-credentials', usernameVariable: 'GIT_USER', passwordVariable: 'GIT_PASS')]) {
|
||||
dir('helm-chart-repo') {
|
||||
deleteDir()
|
||||
// Plain split, not replaceFirst — replaceFirst's *replacement*
|
||||
// argument is also parsed as regex-replacement syntax, where
|
||||
// ${name} means "substitute named capture group", not literal
|
||||
// text. Since \${GIT_USER}/\${GIT_PASS} are meant to stay
|
||||
// literal here (so the *shell* expands them from the
|
||||
// credential-bound env vars at sh-step time, not Groovy —
|
||||
// otherwise the secret value would land in a Groovy-processed
|
||||
// string and defeat withCredentials' masking), that collided
|
||||
// with a pattern that has no such named group and threw
|
||||
// `IllegalArgumentException: named capturing group is
|
||||
// missing trailing '}'`.
|
||||
def urlParts = config.helm_repo_url.split('://', 2)
|
||||
def authedUrl = "${urlParts[0]}://\${GIT_USER}:\${GIT_PASS}@${urlParts[1]}"
|
||||
sh """
|
||||
git clone ${authedUrl} .
|
||||
# This sh step isn't wrapped in container('docker-cli')
|
||||
# (unlike runHooks/buildDocker), so it runs in the
|
||||
# auto-injected jnlp agent container by default — a
|
||||
# Debian-based jenkins/inbound-agent image, not Alpine,
|
||||
# confirmed by `apk: not found` right after git clone
|
||||
# worked fine in the same step. No package manager
|
||||
# assumption is safe here since the underlying
|
||||
# container/distro isn't pinned — fetch the static
|
||||
# mikefarah/yq binary directly instead, to /tmp (always
|
||||
# writable, unlike /usr/local/bin under a non-root
|
||||
# agent user).
|
||||
if ! command -v yq >/dev/null 2>&1; then
|
||||
curl -sL -o /tmp/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
|
||||
chmod +x /tmp/yq
|
||||
fi
|
||||
YQ=\$(command -v yq || echo /tmp/yq)
|
||||
\$YQ -i '${config.image_tag_yq_path} = "${env.TAG}"' ${valuesFile}
|
||||
git config user.email 'jenkins-ci@homelab.local'
|
||||
git config user.name 'jenkins-ci'
|
||||
git commit -am 'ci: bump ${config.repo_name}/${config.service_name} image tag to ${env.TAG}'
|
||||
git push origin main
|
||||
"""
|
||||
// git and yq are both baked into the docker-cli image now (see
|
||||
// build-tools.Dockerfile) — this used to run unwrapped, defaulting
|
||||
// to the auto-injected jnlp agent container (which has git but no
|
||||
// yq, confirmed by `apk: not found` when trying to install yq
|
||||
// on demand there), needing a curl-downloaded yq fallback to
|
||||
// /tmp every single build. Wrapping in container('docker-cli')
|
||||
// means both tools are simply already there.
|
||||
container('docker-cli') {
|
||||
withCredentials([usernamePassword(credentialsId: config.gitea_cred ?: 'gitea-ci-credentials', usernameVariable: 'GIT_USER', passwordVariable: 'GIT_PASS')]) {
|
||||
dir('helm-chart-repo') {
|
||||
deleteDir()
|
||||
// Plain split, not replaceFirst — replaceFirst's *replacement*
|
||||
// argument is also parsed as regex-replacement syntax, where
|
||||
// ${name} means "substitute named capture group", not literal
|
||||
// text. Since \${GIT_USER}/\${GIT_PASS} are meant to stay
|
||||
// literal here (so the *shell* expands them from the
|
||||
// credential-bound env vars at sh-step time, not Groovy —
|
||||
// otherwise the secret value would land in a Groovy-processed
|
||||
// string and defeat withCredentials' masking), that collided
|
||||
// with a pattern that has no such named group and threw
|
||||
// `IllegalArgumentException: named capturing group is
|
||||
// missing trailing '}'`.
|
||||
def urlParts = config.helm_repo_url.split('://', 2)
|
||||
def authedUrl = "${urlParts[0]}://\${GIT_USER}:\${GIT_PASS}@${urlParts[1]}"
|
||||
// set -e added after a build showed "Cloning into
|
||||
// '.'..." with no further output, then a later git
|
||||
// command failing "not in a git directory" — without
|
||||
// set -e, a failed clone wouldn't have stopped the
|
||||
// script, letting later commands run against
|
||||
// whatever partial state was left behind and fail
|
||||
// with a confusing, disconnected error instead of
|
||||
// pointing straight at the clone.
|
||||
//
|
||||
// safe.directory added for the same failure: this
|
||||
// stage started running in container('docker-cli')
|
||||
// (root, docker:27-cli base has no non-root USER)
|
||||
// right when it broke, while deleteDir() just before
|
||||
// it runs via the Jenkins agent's own JNLP process
|
||||
// (a different, non-root UID) — modern git refuses
|
||||
// to trust a repo directory owned by a different UID
|
||||
// than the current process, and that refusal can
|
||||
// surface as an unrelated-looking "not a git
|
||||
// directory" on a *later* command instead of a clear
|
||||
// ownership error on the clone itself. Safe to trust
|
||||
// unconditionally here: this workspace is a
|
||||
// throwaway, container-local checkout for one build.
|
||||
// `git status` right after clone is a cheap
|
||||
// diagnostic that'll make the actual state obvious
|
||||
// if something else is still wrong.
|
||||
sh """
|
||||
set -e
|
||||
git config --global --add safe.directory '*'
|
||||
git clone ${authedUrl} .
|
||||
git status
|
||||
yq -i '${config.image_tag_yq_path} = "${env.TAG}"' ${valuesFile}
|
||||
git config user.email 'jenkins-ci@homelab.local'
|
||||
git config user.name 'jenkins-ci'
|
||||
git commit -am 'ci: bump ${config.repo_name}/${config.service_name} image tag to ${env.TAG}'
|
||||
git push origin main
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,18 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
|
||||
def run(String bu, String team,String module){
|
||||
stage("Validate BU and Team"){
|
||||
def vObj = new buTeamMapping()
|
||||
if (vObj.validate(bu,team)){
|
||||
log.info("Correct team and BU values")
|
||||
}
|
||||
else{
|
||||
error "Incorrect BU and team values provided"
|
||||
}
|
||||
if(module == ""){
|
||||
error "Module can't be empty. if there is no module, Please provide the parameter value as module_less"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
package com.homelab.utilities
|
||||
|
||||
def create() {
|
||||
withCredentials([file(credentialsId: 'ssh-private-key', variable: 'FILE')]) {
|
||||
sh """
|
||||
cat ${FILE} > ./id_github_jenkins
|
||||
chmod 600 ./id_github_jenkins
|
||||
|
||||
# Ensure .ssh directory has correct permissions
|
||||
chmod 700 /root/.ssh
|
||||
|
||||
# Fix SSH config file permissions if it exists
|
||||
if [ -f /root/.ssh/config ]; then
|
||||
chmod 600 /root/.ssh/config
|
||||
chown root:root /root/.ssh/config
|
||||
fi
|
||||
|
||||
# Fix existing SSH private key permissions if it exists
|
||||
if [ -f /root/.ssh/id_rsa ]; then
|
||||
chmod 600 /root/.ssh/id_rsa
|
||||
chown root:root /root/.ssh/id_rsa
|
||||
fi
|
||||
|
||||
# Fix any other SSH key files that might exist
|
||||
find /root/.ssh -type f -name "id_*" -exec chmod 600 {} \\; 2>/dev/null || true
|
||||
"""
|
||||
}
|
||||
}
|
||||
@@ -1,125 +0,0 @@
|
||||
/*
|
||||
Purpose: Utility function to return BU and their respective teams
|
||||
Author: Avinash kumar Lodhi
|
||||
*/
|
||||
package com.homelab.utilities
|
||||
|
||||
def validate(String bu, String team) {
|
||||
def bu_team_map = ['supply':['supplier-ads', 'supplier-ads-frontend', 'experience', 'fulfilment', 'fulfilment-frontend','financial-services', 'cataloging', 'cataloging-frontend', 'payout', 'payout-frontend', 'supplier-acquisition-activation', 'supplier-service', 'returns', 'supply-shared', 'display-ads', 'offers','transact', 'supplier-live-commerce'],
|
||||
'demand':['comms-platform', 'live-commerce', 'shopping-platform', 'product-feed', 'search', 'product-meta', 'user-growth', 'web', 'transact', 'communications', 'discovery-platform', 'offers', 'android-platform', 'ios', 'demand-shared', 'discovery-ranking'],
|
||||
'farmiso':['farmiso'],
|
||||
'admin':['devops'],
|
||||
'central':['shared', 'devops', 'psec', 'dbe'],
|
||||
'dataengg':['data-platform', 'dataengg-shared', 'data-intelligence', 'data-platform-consumption', 'data-platform-ingestion', 'data-platform-nrt', 'data-platform-prism-frmw', 'data-platform-experimentation'],
|
||||
'datascience':['data-science', 'ml-platform', 'for-you', 'recommendation', 'catalog-listing-page', 'search', 'advertisement', 'explore', 'pricing', 'product-match', 'catalog-taxonomy', 'brand-infringment', 'fds', 'return-reimbursements', 'fullfilment', 'ugc-moderation-analysis', 'home-page', 'core', 'usergrowth', 'demand-forecast', 'catalog-qc'],
|
||||
'mcache':['mcache', 'mcache-shared', 'supplier-service'],
|
||||
'infra':['devops', 'dbe']
|
||||
]
|
||||
if (bu == null || team == null) {
|
||||
return null
|
||||
}
|
||||
return bu_team_map[bu].contains(team)
|
||||
}
|
||||
|
||||
def get_initials(String targetMap, String targetString) {
|
||||
def bu_initials = ['supply':'supl',
|
||||
'demand':'dmnd',
|
||||
'farmiso':'farm',
|
||||
'admin':'admn',
|
||||
'central':'cntr',
|
||||
'dataengg':'deng',
|
||||
'datascience':'dsci',
|
||||
'mcache':'mche',
|
||||
'infra':'infr'
|
||||
]
|
||||
def team_initials = ['supplier-ads':'ads',
|
||||
'comms-platform': 'cplat',
|
||||
'supplier-ads-frontend':'fads',
|
||||
'experience':'xp',
|
||||
'fulfilment':'fnf',
|
||||
'fulfilment-frontend':'ffnf',
|
||||
'financial-services':'fsvc',
|
||||
'cataloging':'ctlng',
|
||||
'cataloging-frontend':'fctlg',
|
||||
'payout':'pay',
|
||||
'payout-frontend':'fpay',
|
||||
'supplier-acquisition-activation':'saa',
|
||||
'supplier-service':'ssvc',
|
||||
'seller-services': 'sis',
|
||||
'live-commerce':'lcom',
|
||||
'shopping-platform':'splat',
|
||||
'product-feed':'pfeed',
|
||||
'search':'srch',
|
||||
'product-meta':'pmeta',
|
||||
'user-growth':'grwth',
|
||||
'web':'web',
|
||||
'transact':'trnst',
|
||||
'communications':'comms',
|
||||
'discovery-platform':'dplat',
|
||||
'farmiso':'farm',
|
||||
'devops':'devop',
|
||||
'offers':'offer',
|
||||
'returns':'retrn',
|
||||
'android-platform':'andrd',
|
||||
'ios':'ios',
|
||||
'shared':'xcntr',
|
||||
'supply-shared':'xsupl',
|
||||
'demand-shared':'xdmnd',
|
||||
'data-platform':'dp',
|
||||
'data-science':'ds',
|
||||
'ml-platform':'ml',
|
||||
'dataengg-shared':'xdeng',
|
||||
'datascience-shared':'xdsci',
|
||||
'data-intelligence':'di',
|
||||
'recommendation':'rcmnd',
|
||||
'catalog-listing-page':'ctllp',
|
||||
'advertisement':'adv',
|
||||
'explore':'explr',
|
||||
'pricing':'price',
|
||||
'product-match':'patch',
|
||||
'catalog-taxonomy':'ctltx',
|
||||
'brand-infringment':'brndi',
|
||||
'fds':'fds',
|
||||
'return-reimbursements':'retrr',
|
||||
'fullfilment':'flfmt',
|
||||
'ugc-moderation-analysis':'umdra',
|
||||
'home-page':'hpage',
|
||||
'usergrowth':'ugrwt',
|
||||
'demand-forecast':'dmndf',
|
||||
'catalog-qc':'ctlqc',
|
||||
'data-platform-consumption':'dpcon',
|
||||
'data-platform-ingestion':'dping',
|
||||
'data-platform-nrt':'dpnrt',
|
||||
'data-platform-prism-frmw':'dpprf',
|
||||
'data-platform-experimentation':'dpexp',
|
||||
'display-ads':'dplay',
|
||||
'discovery-ranking':'drank',
|
||||
'mcache':'mche',
|
||||
'mcache-shared':'xmche',
|
||||
'supplier-live-commerce':'slcom',
|
||||
'trust-and-safety': 'tns',
|
||||
'valmo': 'vlm',
|
||||
'psec':'psec',
|
||||
'dbe':'dbe',
|
||||
'dev-productivity':'devprd']
|
||||
|
||||
if (targetString == null) {
|
||||
return null
|
||||
}
|
||||
switch (targetMap) {
|
||||
case 'bu_initials':
|
||||
return bu_initials[targetString]
|
||||
case 'team_initials':
|
||||
return team_initials[targetString]
|
||||
default:
|
||||
return 'Undefined option'
|
||||
}
|
||||
}
|
||||
|
||||
def get_team_initials(String team) {
|
||||
return get_initials('team_initials', team)
|
||||
}
|
||||
|
||||
def get_bu_initials(String bu) {
|
||||
return get_initials('bu_initials', bu)
|
||||
}
|
||||
@@ -1,391 +0,0 @@
|
||||
|
||||
package com.homelab.utilities
|
||||
|
||||
def getWhitelistedRepos(fileName){
|
||||
dir('whitelist'){
|
||||
git(
|
||||
url: "https://github.com/Homelab/whitelists.git",
|
||||
branch: "main",
|
||||
credentialsId: 'cicd-github-app',
|
||||
)}
|
||||
def yaml = readYaml file: "whitelist/${fileName}.yaml"
|
||||
return yaml.get("repos", []) as Set
|
||||
}
|
||||
|
||||
def getWhitelistedDeployable(fileName, keyName){
|
||||
dir('whitelist'){
|
||||
git(
|
||||
url: "https://github.com/Homelab/whitelists.git",
|
||||
branch: "main",
|
||||
credentialsId: 'cicd-github-app',
|
||||
)}
|
||||
def yaml = readYaml file: "whitelist/${fileName}.yaml"
|
||||
return yaml.get(keyName, []) as Set
|
||||
}
|
||||
|
||||
/*
|
||||
* return `true` if we should not proceed
|
||||
*/
|
||||
def isMultizoneEnabled( String deployable){
|
||||
def WHITELIST = getWhitelistedDeployable("multizone-enabled-repos" , "multizone_enabled_deployables")
|
||||
if (WHITELIST.contains(deployable)) {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/*
|
||||
* return `true` if we should not proceed
|
||||
*/
|
||||
def skipSonarCheckForbidden(Map config, Map environment_map) {
|
||||
def branch = env.BRANCH_NAME
|
||||
def environment = environment_map.getOrDefault(branch, "int")
|
||||
|
||||
def WHITELIST = getWhitelistedRepos("skip-sonar-whitelist")
|
||||
|
||||
// returns 'true' if we aren't skipping sonar
|
||||
// or if we're allowed to skip sonar
|
||||
def build_version = config['dockerBuildVersion']
|
||||
def repo_name = config['repo_name']
|
||||
|
||||
// if not a maven build OR if it a hotfix we exit early and don’t care what skip_sonar is
|
||||
if (WHITELIST.contains(repo_name)|| !(build_version.contains("maven")) || environment != "prd" || branch.contains("hotfix")){
|
||||
return false;
|
||||
}
|
||||
|
||||
return config['skip_sonar'];
|
||||
}
|
||||
|
||||
def skipSonarCheckForGo(Map config) {
|
||||
def branch = env.BRANCH_NAME
|
||||
def WHITELIST = getWhitelistedRepos("skip-sonar-whitelist")
|
||||
def build_version = config['dockerBuildVersion']
|
||||
def repo_name = config['repo_name']
|
||||
echo "env.INFRA_ENV: ${env.INFRA_ENV}"
|
||||
if (WHITELIST.contains(repo_name)|| branch.contains("hotfix")|| env.INFRA_ENV == "toolchain"){
|
||||
return true;
|
||||
}
|
||||
return false;
|
||||
}
|
||||
/*
|
||||
* return `true` if we should not proceed
|
||||
*/
|
||||
def appConfigDisabledForbidden(boolean appConfigEnabled, String repo_name, String environment, String build_version){
|
||||
def branch = env.CHANGE_TARGET
|
||||
def WHITELIST = getWhitelistedRepos("app-config-disabled")
|
||||
if (WHITELIST.contains(repo_name) || environment!="stg"){
|
||||
return false
|
||||
}
|
||||
if (!(build_version.contains("maven") || build_version.contains("gradle"))){
|
||||
return false
|
||||
}
|
||||
return !appConfigEnabled
|
||||
}
|
||||
|
||||
/*
|
||||
* return `true` if we should not proceed
|
||||
*/
|
||||
def allowedNonDevelopPrDeploymentToIntRepos( String repo_name){
|
||||
def WHITELIST = getWhitelistedRepos("allowedNonDevelopPrDeploymentToInt")
|
||||
if (WHITELIST.contains(repo_name)) {
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
/*
|
||||
* return `true` if we should not proceed
|
||||
*/
|
||||
def ValidateCacConfigForRepo(boolean ValidateConfig, String repo_name ){
|
||||
def branch = env.CHANGE_TARGET
|
||||
def WHITELIST = getWhitelistedRepos("ValidateCacConfig")
|
||||
if (WHITELIST.contains(repo_name)) {
|
||||
return true
|
||||
}
|
||||
return ValidateConfig
|
||||
}
|
||||
|
||||
def getToolchainEnv() {
|
||||
def paramsAction = currentBuild.rawBuild.getAction(hudson.model.ParametersAction.class)
|
||||
if (paramsAction) {
|
||||
echo "paramsAction: ${paramsAction}"
|
||||
def p = paramsAction.getParameter("TOOLCHAIN_ENV")
|
||||
echo "p: ${p}"
|
||||
if (p) {
|
||||
return p.getValue()?.toString()
|
||||
}
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
def run(Map config) {
|
||||
def branch_name = env.BRANCH_NAME
|
||||
if (env.INFRA_ENV == 'toolchain' && (config.build_tool?.startsWith('node-') || config.dockerBuildVersion?.startsWith('node-'))) {
|
||||
branch_name = 'develop' // develop maps to stg in the environment_map
|
||||
def tcEnv = getToolchainEnv()
|
||||
env.TOOLCHAIN_ENV = tcEnv
|
||||
log.info("Successfully extracted TOOLCHAIN_ENV from trigger cause: ${env.TOOLCHAIN_ENV}")
|
||||
}
|
||||
def environment_map = ['master':'prd', 'main':'prd', 'develop':'stg', 'gcp-main':'prd', 'farmiso-main':'prd', 'gcp-master':'prd', 'gcp-dev':'stg']
|
||||
env.skip_user_input = config.skip_user_input ?: false
|
||||
|
||||
// don't allow skip_sonar
|
||||
if (skipSonarCheckForbidden(config, environment_map)){
|
||||
throw new Exception("Not allowed to skip sonar (skip_sonar in config.yaml)")
|
||||
}
|
||||
|
||||
if (env.CHANGE_ID) {
|
||||
branch_name = env.CHANGE_TARGET
|
||||
environment_map = ['master':'int', 'main':'int', 'gcp-main':'int', 'farmiso-main':'int', 'gcp-master':'int', 'develop':'ftr', 'gcp-dev':'ftr']
|
||||
}
|
||||
environment_map[branch_name] = environment_map[branch_name] ?: 'ftr'
|
||||
|
||||
if (config.containsKey('branch_params')) {
|
||||
Map branch_config = config['branch_params'].collectEntries { key, value -> branch_name.matches(key) ? value : [ : ] }
|
||||
config.remove('branch_params')
|
||||
config.putAll(branch_config)
|
||||
}
|
||||
if (config.containsKey('environment')) {
|
||||
Map envrionment_config = config['environment'].collectEntries { key, value -> environment_map[branch_name].matches(key) ? value : [ : ] }
|
||||
config.remove('environment')
|
||||
config.putAll(envrionment_config)
|
||||
}
|
||||
|
||||
env.GITHUB_CRED = 'svc-devops-homelab'
|
||||
env.cicd_environment = environment_map[branch_name]
|
||||
env.helm_repo_name = 'devops-helm-charts'
|
||||
env.argo_repo_name = 'devops-argo-config'
|
||||
|
||||
echo "Branch Name - ${branch_name} and Environment - ${env.cicd_environment}"
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
def prodAccountID = '847438129436'
|
||||
def prodRegion = 'ap-southeast-1'
|
||||
def prodObjBucket = 'homelab-prod-artifacts'
|
||||
def devAccountID = '766380763301'
|
||||
def devObjBucket = 'homelab-stg-artifacts'
|
||||
def devRegion = 'ap-south-1'
|
||||
def accountDetails = [
|
||||
'prd': [
|
||||
'accountID': prodAccountID,
|
||||
'region': prodRegion,
|
||||
'objBucket': prodObjBucket
|
||||
],
|
||||
'int': [
|
||||
'accountID': prodAccountID,
|
||||
'region': prodRegion,
|
||||
'objBucket': prodObjBucket
|
||||
],
|
||||
'stg': [
|
||||
'accountID': devAccountID,
|
||||
'region': devRegion,
|
||||
'objBucket': devObjBucket
|
||||
],
|
||||
'ftr': [
|
||||
'accountID': devAccountID,
|
||||
'region': devRegion,
|
||||
'objBucket': devObjBucket
|
||||
]
|
||||
]
|
||||
env.accountID = accountDetails[env.cicd_environment]['accountID']
|
||||
env.region = accountDetails[env.cicd_environment]['region']
|
||||
env.registry = "${env.accountID}.dkr.ecr.${env.region}.amazonaws.com"
|
||||
env.buildRegistry = env.registry
|
||||
env.helmChartsPath = 'charts'
|
||||
env.defaultHelmChartVersion = '1.0.10'
|
||||
env.objBucket = accountDetails[env.cicd_environment]['objBucket']
|
||||
env.skip_notify = false
|
||||
echo "${env.accountID}.dkr.ecr.${env.region}.amazonaws.com"
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
def prodVaultURL = 'https://vault-prd.homelabgcp.in'
|
||||
def prodVaultToken = 'vault-prd-token'
|
||||
def prodSonarURL = 'https://sonarqube-prd.homelabgcp.in'
|
||||
def prodSonarToken = 'sonar-token-prod'
|
||||
def prodSonarEnv = 'sonarqube-test'
|
||||
def prodGoProxyUrl = 'https://athens-prd.homelabgcp.in'
|
||||
def prdDockerHost = 'dind-prd-svc'
|
||||
def preProdDockerHost = 'dind-int-svc'
|
||||
def prodGCPProject = "homelab-${config.bu}-prd-0622"
|
||||
def preprodGCPProject = "homelab-shared-int-0525"
|
||||
def devVaultURL = 'https://vault-dev.homelabgcp.in'
|
||||
def devVaultToken = 'vault-dev-token'
|
||||
def devSonarURL = "https://sonarqube-${config.bu}-dev.homelabgcp.in"
|
||||
def devSonarToken = "sonar-token-${config.bu}-dev"
|
||||
def devSonarEnv = "sonar-${config.bu}-dev"
|
||||
def devGoProxyUrl = 'https://athens-dev.homelabgcp.in'
|
||||
def devGCPProject = "homelab-${config.bu}-dev-0622"
|
||||
def devDockerHost = 'dind-dev-new-svc.jenkins-new.svc.cluster.local'
|
||||
def toolchainDockerHost = 'toolchain-dind-dev-svc.jenkins-toolchain.svc.cluster.local'
|
||||
def accountDetails = [
|
||||
'prd': [
|
||||
'vaultURL': prodVaultURL,
|
||||
'vaultToken': prodVaultToken,
|
||||
'sonarURL': prodSonarURL,
|
||||
'sonarToken': prodSonarToken,
|
||||
'GCPProject': prodGCPProject,
|
||||
'sonarEnv': prodSonarEnv,
|
||||
'GCPLBProject': prodGCPProject,
|
||||
'goProxyUrl': prodGoProxyUrl,
|
||||
'dockerHost': prdDockerHost
|
||||
],
|
||||
'int': [
|
||||
'vaultURL': prodVaultURL,
|
||||
'vaultToken': prodVaultToken,
|
||||
'sonarURL': prodSonarURL,
|
||||
'sonarToken': prodSonarToken,
|
||||
'GCPProject': preprodGCPProject,
|
||||
'sonarEnv': prodSonarEnv,
|
||||
'GCPLBProject': prodGCPProject,
|
||||
'goProxyUrl': prodGoProxyUrl,
|
||||
'dockerHost': preProdDockerHost
|
||||
],
|
||||
'stg': [
|
||||
'vaultURL': devVaultURL,
|
||||
'vaultToken': devVaultToken,
|
||||
'sonarURL': devSonarURL,
|
||||
'sonarToken': devSonarToken,
|
||||
'GCPProject': devGCPProject,
|
||||
'sonarEnv': devSonarEnv,
|
||||
'GCPLBProject': devGCPProject,
|
||||
'goProxyUrl': devGoProxyUrl,
|
||||
'dockerHost': devDockerHost
|
||||
],
|
||||
'ftr': [
|
||||
'vaultURL': devVaultURL,
|
||||
'vaultToken': devVaultToken,
|
||||
'sonarURL': devSonarURL,
|
||||
'sonarToken': devSonarToken,
|
||||
'GCPProject': devGCPProject,
|
||||
'sonarEnv': devSonarEnv,
|
||||
'GCPLBProject': devGCPProject,
|
||||
'goProxyUrl': devGoProxyUrl,
|
||||
'dockerHost': devDockerHost ]
|
||||
]
|
||||
env.GCPProject = accountDetails[env.cicd_environment]['GCPProject']
|
||||
env.GCPLBProject = accountDetails[env.cicd_environment]['GCPLBProject']
|
||||
env.registry = 'asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622'
|
||||
if (env.INFRA_ENV == 'toolchain') {
|
||||
env.registry = 'asia-southeast1-docker.pkg.dev/homelab-central-dev-0622/toolchain'
|
||||
}
|
||||
env.buildRegistry = 'asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/admin'
|
||||
env.helmChartsPath = env.cicd_environment == 'prd' ? 'values_v3' : 'values_v2'
|
||||
env.defaultHelmChartVersion = '2.0.0'
|
||||
env.objBucket = "gcs-infr-dvps-homelab-artifacts-${env.cicd_environment}"
|
||||
env.vaultURL = accountDetails[env.cicd_environment]['vaultURL']
|
||||
env.vaultToken = accountDetails[env.cicd_environment]['vaultToken']
|
||||
env.sonarURL = accountDetails[env.cicd_environment]['sonarURL']
|
||||
env.sonarToken = accountDetails[env.cicd_environment]['sonarToken']
|
||||
env.sonarEnv = accountDetails[env.cicd_environment]['sonarEnv']
|
||||
env.goProxyUrl = accountDetails[env.cicd_environment]['goProxyUrl']
|
||||
env.skip_notify = true
|
||||
env.DOCKER_HOST = accountDetails[env.cicd_environment]['dockerHost']
|
||||
if (env.INFRA_ENV == 'toolchain') {
|
||||
env.DOCKER_HOST = toolchainDockerHost
|
||||
}
|
||||
}
|
||||
echo "Bucket and Image Repo Details - ${env.registry} ${env.buildRegistry} ${env.objBucket}"
|
||||
echo "Docker Host - ${env.DOCKER_HOST}"
|
||||
}
|
||||
|
||||
def perDeploymentVars(Map value_binding) {
|
||||
env.BU = value_binding.bu
|
||||
echo "${env.BU}"
|
||||
|
||||
if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
def inClusterName = 'https://kubernetes.default.svc'
|
||||
def prodArgoURL = 'prod-ops-argocd.homelab.com'
|
||||
def devArgoURL = 'stg-dev-argocd.homelabtest.in'
|
||||
def prodK8sCluster = [
|
||||
'supply': 'https://211689C65F4496AAA76FE19B29E24B6E.yl4.ap-southeast-1.eks.amazonaws.com',
|
||||
'demand': 'https://9059D138B6277A0EA592BA7F4B680CEC.gr7.ap-southeast-1.eks.amazonaws.com',
|
||||
'dataengg': 'https://806ADE97231CA65D2A0FFB780352630D.yl4.ap-southeast-1.eks.amazonaws.com',
|
||||
'datascience': 'https://E34D119516F751AFD1A61043B0514726.yl4.ap-southeast-1.eks.amazonaws.com',
|
||||
'central': 'https://C95FCEDF7CEE890F531E1D0488BEC6C3.gr7.ap-southeast-1.eks.amazonaws.com',
|
||||
'mcache': 'https://2FFADF214AD8BE58769C5F2797987E50.gr7.ap-southeast-1.eks.amazonaws.com'
|
||||
]
|
||||
def devK8sCluster = [
|
||||
'supply': inClusterName,
|
||||
'demand': inClusterName,
|
||||
'dataengg': inClusterName,
|
||||
'datascience': inClusterName,
|
||||
'central': inClusterName,
|
||||
'mcache': inClusterName
|
||||
]
|
||||
def accountDetails = [
|
||||
'prd': [
|
||||
'argoURL': prodArgoURL,
|
||||
'argoIncubator': 'prod-app-of-apps',
|
||||
'serverMap': prodK8sCluster
|
||||
],
|
||||
'int': [
|
||||
'argoURL': prodArgoURL,
|
||||
'argoIncubator': 'int-app-of-app',
|
||||
'serverMap': prodK8sCluster
|
||||
],
|
||||
'stg': [
|
||||
'argoURL': devArgoURL,
|
||||
'argoIncubator': 'app-of-apps',
|
||||
'serverMap': devK8sCluster
|
||||
],
|
||||
'ftr': [
|
||||
'argoURL': devArgoURL,
|
||||
'argoIncubator': 'ftr-app-of-apps',
|
||||
'serverMap': devK8sCluster
|
||||
]
|
||||
]
|
||||
env.clusterName = accountDetails[env.cicd_environment]['serverMap'][env.BU]
|
||||
env.argoAppsPath = 'applications'
|
||||
env.argoURL = accountDetails[env.cicd_environment]['argoURL']
|
||||
env.argoCreds = 'argocd-jenkins'
|
||||
env.argoIncubator = accountDetails[env.cicd_environment]['argoIncubator']
|
||||
env.argoAppNS = 'argocd'
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
def prodArgoURL = "argocd-${env.BU}-prd.homelabgcp.in"
|
||||
def prodArgoCreds = "argocd-${env.BU}-prd-creds"
|
||||
def preprodArgoURL = "argocd-shared-int.homelabgcp.in"
|
||||
def preprodArgoCreds = "argocd-shared-int-creds"
|
||||
def devArgoURL = 'argocd-dev.homelabgcp.in'
|
||||
def devArgoCreds = 'argocd-dev-creds'
|
||||
def accountDetails = [
|
||||
'prd': [
|
||||
'argoURL': prodArgoURL,
|
||||
'argoCreds': prodArgoCreds,
|
||||
'argoAppNS': "argocd-${env.BU}-prd",
|
||||
'clusterName': "k8s-${env.BU}-prd-ase1"
|
||||
],
|
||||
'int': [
|
||||
'argoURL': preprodArgoURL,
|
||||
'argoCreds': preprodArgoCreds,
|
||||
'argoAppNS': "argocd-shared-int",
|
||||
'clusterName': "k8s-shared-int-ase1"
|
||||
],
|
||||
'stg': [
|
||||
'argoURL': devArgoURL,
|
||||
'argoCreds': devArgoCreds,
|
||||
'argoAppNS': "argocd-dev",
|
||||
'clusterName': "k8s-${env.BU}-stg-ase1"
|
||||
],
|
||||
'ftr': [
|
||||
'argoURL': devArgoURL,
|
||||
'argoCreds': devArgoCreds,
|
||||
'argoAppNS': "argocd-dev",
|
||||
'clusterName': "k8s-${env.BU}-stg-ase1"
|
||||
]
|
||||
]
|
||||
|
||||
env.clusterName = accountDetails[env.cicd_environment]['clusterName']
|
||||
|
||||
if (env.cicd_environment == 'int') {
|
||||
env.argoAppsPath = "applications_v2/k8s-${env.BU}-int-ase1"
|
||||
} else {
|
||||
env.argoAppsPath = "applications_v2/${env.clusterName}"
|
||||
}
|
||||
|
||||
env.argoURL = accountDetails[env.cicd_environment]['argoURL']
|
||||
env.argoCreds = accountDetails[env.cicd_environment]['argoCreds']
|
||||
env.argoAppNS = accountDetails[env.cicd_environment]['argoAppNS']
|
||||
env.argoIncubator = "incubator-apps-k8s-${env.BU}-${env.cicd_environment}-ase1"
|
||||
}
|
||||
echo "${env.clusterName} ${env.argoURL} ${env.argoIncubator}"
|
||||
}
|
||||
@@ -1,52 +0,0 @@
|
||||
package com.homelab.utilities
|
||||
|
||||
def retryDockerPush(String cmd) {
|
||||
int maxAttempts = 5
|
||||
int attempt = 1
|
||||
while (attempt <= maxAttempts) {
|
||||
try {
|
||||
sh cmd
|
||||
break
|
||||
} catch (err) {
|
||||
if (attempt == maxAttempts) {
|
||||
error("Command failed after ${maxAttempts} attempts: ${err}")
|
||||
}
|
||||
echo "Command failed, retrying... (${attempt}/${maxAttempts})"
|
||||
sleep 3
|
||||
attempt++
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def imageExists(String registry, String repoName, String tag) {
|
||||
int maxAttempts = 5
|
||||
int attempt = 1
|
||||
|
||||
while (attempt <= maxAttempts) {
|
||||
try {
|
||||
if (env.CLOUD_PROVIDER == 'GCP') {
|
||||
def result = sh(
|
||||
script: "gcloud container images list-tags ${registry}/${repoName} --filter='tags:${tag}' --format='get(tags)'",
|
||||
returnStdout: true
|
||||
).trim()
|
||||
return result != ""
|
||||
}
|
||||
else if (env.CLOUD_PROVIDER == 'AWS') {
|
||||
echo "AWS not supported."
|
||||
return false
|
||||
}
|
||||
} catch (Exception e) {
|
||||
echo "Attempt ${attempt}/${maxAttempts} failed: Error checking image existence: ${e.toString()}"
|
||||
|
||||
if (attempt == maxAttempts) {
|
||||
echo "Max attempts reached. Assuming image does not exist or service is down."
|
||||
return false
|
||||
}
|
||||
|
||||
echo "Retrying in 5 seconds..."
|
||||
sleep 5
|
||||
attempt++
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -1,61 +0,0 @@
|
||||
package com.homelab.utilities
|
||||
|
||||
def getCommitid(String repo_name) {
|
||||
dir(repo_name) {
|
||||
def gitCmd = env.INFRA_ENV == 'toolchain' ? 'git -c safe.directory="$(pwd)"' : 'git'
|
||||
def commitID = sh(returnStdout: true, script: "${gitCmd} log -1 --format=%h").trim()
|
||||
env.commit_id = sh(returnStdout: true, script: "${gitCmd} log -1 --format=%H").trim()
|
||||
return commitID
|
||||
}
|
||||
}
|
||||
|
||||
def getVersion(String repo_name) {
|
||||
dir(repo_name) {
|
||||
if (fileExists('pom.xml')) {
|
||||
return sh(returnStdout: true, script: 'xq -r .project.version pom.xml').trim()
|
||||
}
|
||||
else if (fileExists('package.json')) {
|
||||
return sh(returnStdout: true, script: 'jq -r .version package.json').trim()
|
||||
}
|
||||
else {
|
||||
return '1.0'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def getModules(String repo_name) {
|
||||
dir(repo_name) {
|
||||
if (fileExists('pom.xml')) {
|
||||
modules = sh(returnStdout: true, script: 'xq -r .project.modules.module[] pom.xml 2>/dev/null || xq -r .project.modules.module pom.xml 2>/dev/null || echo empty').trim()
|
||||
if (modules == 'empty' || modules == 'null') {
|
||||
return null
|
||||
}
|
||||
modules = modules.split('\n') as List
|
||||
return modules
|
||||
}
|
||||
else {
|
||||
return null
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def getTag(String repo_name) {
|
||||
def version = getVersion(repo_name)
|
||||
def commitID = getCommitid(repo_name)
|
||||
def date = new Date()
|
||||
def timesha = date.getTime()
|
||||
def tag = "v${version}-${commitID}-${timesha}"
|
||||
if (env.INFRA_ENV == 'toolchain') {
|
||||
tag = "v${version}-${commitID}"
|
||||
}
|
||||
|
||||
return tag
|
||||
}
|
||||
|
||||
def getTagShort(String repo_name) {
|
||||
def version = getVersion(repo_name)
|
||||
def commitID = getCommitid(repo_name)
|
||||
def tagShort = "v${version}-${commitID}"
|
||||
|
||||
return tagShort
|
||||
}
|
||||
@@ -1,16 +0,0 @@
|
||||
package com.homelab.utilities
|
||||
|
||||
def getParam(String wd, String fileName = 'config.yaml') {
|
||||
dir(wd) {
|
||||
def config = readYaml file: fileName
|
||||
return config
|
||||
}
|
||||
}
|
||||
//read as string
|
||||
def getParamAsString(String wd, String fileName = 'config.yaml') {
|
||||
dir(wd) {
|
||||
// Read the entire file content as a string
|
||||
def yamlContent = readFile(file: fileName)
|
||||
return yamlContent
|
||||
}
|
||||
}
|
||||
@@ -1,342 +0,0 @@
|
||||
package com.homelab.utilities
|
||||
|
||||
def clone(String path, String repo_name, String branch_name) {
|
||||
log.info("Cloning repo - ${repo_name}, branch - ${branch_name} in path - ${path}")
|
||||
dir(path) {
|
||||
try {
|
||||
sh "rm -rf ${repo_name}"
|
||||
}
|
||||
catch (Exception e) {
|
||||
log.info('Repo not present, proceeding to clone the repo.')
|
||||
}
|
||||
try {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
if (branch_name) {
|
||||
sh "git clone -b ${branch_name} https://github.com/Homelab/${repo_name}.git"
|
||||
}
|
||||
else {
|
||||
sh "git clone https://github.com/Homelab/${repo_name}.git"
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg += "\n\nFAILED -\n ```Unable to clone repo - ${repo_name}, branch - ${branch_name} in path - ${path}.\n Full Erroror Details - ${e}```"
|
||||
env.error_msg_to_db += "Unable to Clone Repo ${repo_name};"
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def branchCheckOut(String path, String branch_name) {
|
||||
log.info("Checking out to branch - ${branch_name} in path - ${path}")
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh 'git fetch'
|
||||
try {
|
||||
sh "git checkout ${branch_name}"
|
||||
sh 'git pull --ff-only'
|
||||
}
|
||||
catch (Exception e) {
|
||||
log.info('Creating new branch')
|
||||
sh "git checkout -b ${branch_name}"
|
||||
}
|
||||
sh 'git branch'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def add(String path, String git_add_file) {
|
||||
log.info("Adding files - ${git_add_file} to git, in path - ${path}")
|
||||
try {
|
||||
dir(path) {
|
||||
sh "git add ${git_add_file}"
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg += "\n\nFAILED -\n ```Unable to add files - ${git_add_file} to git, in path - ${path}.\n Full Erroror Details - ${e}```"
|
||||
env.error_msg_to_db += 'Unable to Add Files;'
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def createTag(path, git_tag, tag_message) {
|
||||
log.info("Creating tag - ${git_tag} in git, in path - ${path}")
|
||||
try {
|
||||
dir(path) {
|
||||
sh "git tag ${git_tag} -m '${tag_message}'"
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg += "\n\nFAILED -\n ```Unable to create tag - ${git_tag} in git, in path - ${path}.\n Full Erroror Details - ${e}```"
|
||||
env.error_msg_to_db += "Unable to Create Tag ${git_tag};"
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def codeCommit(String path, String branch_name, String commit_message) {
|
||||
log.info("Commiting code with message - ${commit_message} in git, in path - ${path}")
|
||||
def global_arg = '--global'
|
||||
def gitName = 'svc-devops-homelab'
|
||||
def gitEmail = 'devops@homelab.com'
|
||||
def gitEnv = [
|
||||
"GIT_AUTHOR_NAME=${gitName}",
|
||||
"GIT_AUTHOR_EMAIL=${gitEmail}",
|
||||
"GIT_COMMITTER_NAME=${gitName}",
|
||||
"GIT_COMMITTER_EMAIL=${gitEmail}",
|
||||
]
|
||||
dir(path) {
|
||||
withEnv(gitEnv) {
|
||||
if (sh(returnStatus: true, script: "git diff-index --quiet ${branch_name} 2>/dev/null")) {
|
||||
try {
|
||||
sh "git commit -m '${commit_message}'"
|
||||
return 0
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg += "\n\nFAILED -\n ```Unable to commit code with message - ${commit_message} to git in path - ${path}.\n Full Erroror Details - ${e}```"
|
||||
env.error_msg_to_db += 'Unable to Commit;'
|
||||
log.error("${env.msg}")
|
||||
return 1
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info("No changes were made in branch - ${branch_name}")
|
||||
return 1
|
||||
// log.error("${env.msg}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def codePush(String path, String branch_name) {
|
||||
log.info("Pushing code in branch - ${branch_name} in path - ${path}")
|
||||
try {
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
try {
|
||||
sh "git push -f --set-upstream origin ${branch_name}"
|
||||
}
|
||||
catch (Exception e) {
|
||||
sh 'git pull --ff-only'
|
||||
sh 'git push -f'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg += "\n\nFAILED -\n ```Unable to push code in branch - ${branch_name} in path - ${path}.\n Full Erroror Details - ${e}```"
|
||||
env.error_msg_to_db += "Unable to push code in branch - ${branch_name};"
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def tagPush(String path) {
|
||||
log.info("Pushing tag in path - ${path}")
|
||||
try {
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh 'git push --tags'
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg += "\n\nFAILED -\n ```Unable to push tag from path - ${path}.\n Full Erroror Details - ${e}```"
|
||||
env.error_msg_to_db += 'Unable to Push tag;'
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def createPR(String app_name, String repo_name, String base_branch, String target_branch, String pr_message) {
|
||||
log.info("Creating PR with message - ${pr_message} in Github for repo ${repo_name}")
|
||||
def body = "{\"title\":\"${app_name} ${base_branch} Onboarding\",\"body\":\"${pr_message}\",\"head\":\"${target_branch}\",\"base\":\"${base_branch}\"}"
|
||||
def pr_num = 'empty'
|
||||
|
||||
try {
|
||||
withCredentials([usernamePassword(credentialsId: "${env.GITHUB_CRED}", usernameVariable:'user', passwordVariable: 'token')]) {
|
||||
create_pr = httpRequest httpMode: 'POST',
|
||||
customHeaders: [
|
||||
[name: 'Accept', value: 'application/vnd.github+json'],
|
||||
[maskValue: true, name: 'Authorization', value: 'Bearer ' + token]
|
||||
],
|
||||
requestBody: body,
|
||||
url: "https://api.github.com/repos/Homelab/${repo_name}/pulls",
|
||||
validResponseCodes: '201',
|
||||
timeout: 10
|
||||
def create_pr_json = readJSON(text: create_pr.content)
|
||||
try {
|
||||
error_filter = create_pr_json.errors.message[0]
|
||||
}
|
||||
catch (Exception e) {
|
||||
error_filter = 'empty'
|
||||
}
|
||||
if ( error_filter.contains('No commits between') ) {
|
||||
log.info("No changes were made in branch - ${target_branch}. Skipping - PR Creation...\n${create_pr_json}")
|
||||
}
|
||||
else if ( error_filter.contains('A pull request already exists') ) {
|
||||
log.info(error_filter)
|
||||
}
|
||||
else {
|
||||
pr_num = create_pr_json.number
|
||||
pr_url = create_pr_json.url
|
||||
log.info("PR Number - ${pr_num} and URL - ${pr_url}")
|
||||
}
|
||||
return pr_num.toString()
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "\n\nFAILED -\n Unable to create PR with message - ${pr_message}.\n Full Error Details - ${e}"
|
||||
def error_code = "${e}".split('Status code')[1]
|
||||
error_code = error_code.toString()
|
||||
error_code = error_code.split(' ')
|
||||
error_code = error_code[1]
|
||||
env.error_part_msg_to_db = "Unable to Create PR for base branch - ${base_branch}, target branch - ${target_branch}, Repo Name - ${repo_name}. Failed with status code ${error_code};" //Akshay has asked to remove it
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
|
||||
def mergePR(String repo_name, String pr_num, String target_branch) {
|
||||
log.info("Merging PR ${pr_num} in repo - ${repo_name}")
|
||||
if ( "${pr_num}" == 'empty' || pr_num == null) {
|
||||
log.info("No changes were made in branch - ${target_branch}. Skipping - PR Merge...")
|
||||
}
|
||||
else {
|
||||
try {
|
||||
withCredentials([usernamePassword(credentialsId: "${env.GITHUB_CRED}", usernameVariable:'user', passwordVariable: 'token')]) {
|
||||
merge_pr = httpRequest httpMode: 'PUT',
|
||||
customHeaders: [
|
||||
[name: 'Accept', value: 'application/vnd.github+json'],
|
||||
[maskValue: true, name: 'Authorization', value: 'Bearer ' + token]
|
||||
],
|
||||
url: "https://api.github.com/repos/Homelab/${repo_name}/pulls/${pr_num}/merge",
|
||||
validResponseCodes: '200',
|
||||
timeout: 10
|
||||
|
||||
def merge_pr_json = readJSON(text: merge_pr.content)
|
||||
if ( merge_pr_json.message.contains('not mergeable') ) {
|
||||
env.msg += "\n\nFAILED -\n Unable to merge PR - ${pr_num}. PR URL - https://github.com/Homelab/${repo_name}/pulls/${pr_num}. ${merge_pr_json.message}"
|
||||
log.error("${env.msg}")
|
||||
error("${env.msg}")
|
||||
}
|
||||
else {
|
||||
log.info("PR Merge was SUCCESSFUL. Message - ${merge_pr_json.message}")
|
||||
}
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "\n\nFAILED -\n Unable to merge PR - ${pr_num} \n Full Error Details - ${e}"
|
||||
def error_code = "${e}".split('Status code')[1]
|
||||
error_code = error_code.toString()
|
||||
error_code = error_code.split(' ')
|
||||
error_code = error_code[1]
|
||||
env.error_part_msg_to_db = "Unable to Merge PR - ${pr_num}, Repo Name - ${repo_name} status code ${error_code};" //Akshay has asked to remove it
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def deleteBranch(String path, String base_branch, String target_branch) {
|
||||
log.info("Deleting branch - ${target_branch}")
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh 'git fetch'
|
||||
try {
|
||||
sh "git checkout ${base_branch}"
|
||||
sh "git branch -d ${target_branch}"
|
||||
sh "git push -d origin ${target_branch}"
|
||||
}
|
||||
catch (Exception e) {
|
||||
env.msg = "\n\nFAILED -\n Unable to delete branch - ${target_branch}.\n Full Error Details - ${e}"
|
||||
env.error_msg_to_db = "Unable to Delete Branch ${target_branch};"
|
||||
log.error("${env.msg}")
|
||||
throw e
|
||||
}
|
||||
sh 'git branch'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def preDeleteBranch(String path, String base_branch, String target_branch) {
|
||||
log.info("Pre Deleting branch - ${target_branch}")
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
try {
|
||||
sh 'git fetch'
|
||||
sh "git checkout ${base_branch}"
|
||||
sh "git push -d origin ${target_branch}"
|
||||
}
|
||||
catch (Exception e) {
|
||||
echo "Pre Delete Branch - Unable to delete branch ${target_branch}"
|
||||
}
|
||||
sh 'git branch'
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def fetchDiffFilesForPullRequest(String path , String target_branch){
|
||||
log.info("check diff from target branch - ${target_branch}")
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
try {
|
||||
sh "git fetch origin ${target_branch}:${target_branch}"
|
||||
sh "git branch"
|
||||
def changedFiles = sh(script: "git diff --name-only HEAD ${target_branch}", returnStdout: true).trim()
|
||||
echo "${changedFiles}"
|
||||
return changedFiles
|
||||
}
|
||||
catch (Exception e) {
|
||||
echo "failed to get the diff files from ${target_branch}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
def fetchDiffFilesForPushRequest(String path ){
|
||||
log.info("check diff of current and previous commit for ${path}")
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
try {
|
||||
sh "git fetch origin"
|
||||
|
||||
// Get the latest commit hash and the previous commit hash
|
||||
def currentCommit = sh(script: "git rev-parse HEAD", returnStdout: true).trim()
|
||||
def previousCommit = sh(script: "git rev-parse HEAD~1", returnStdout: true).trim()
|
||||
|
||||
// Capture the list of changed files in a Groovy variable
|
||||
def changedFiles = sh(script: "git diff --name-only ${previousCommit} ${currentCommit}", returnStdout: true).trim()
|
||||
|
||||
return changedFiles
|
||||
}
|
||||
catch (Exception e) {
|
||||
echo "failed to get the diff files for ${path}"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def fetchLatestCommitId(String path, String branch) {
|
||||
log.info("Checking the latest commit for branch '${branch}' in path '${path}'")
|
||||
dir(path) {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
try {
|
||||
// Fetch the latest changes for the specific branch
|
||||
sh "git fetch origin ${branch}:${branch}"
|
||||
|
||||
// Check out the specified branch
|
||||
sh "git checkout ${branch}"
|
||||
|
||||
// Get the latest commit hash for the branch
|
||||
def currentCommit = sh(script: "git rev-parse ${branch}", returnStdout: true).trim()
|
||||
|
||||
return currentCommit
|
||||
} catch (Exception e) {
|
||||
echo "Failed to get the latest commit for branch '${branch}' in path '${path}': ${e.message}"
|
||||
return null
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,141 +0,0 @@
|
||||
package com.homelab.utilities
|
||||
|
||||
def run(String memory_request, String cpu_request, String priority_v2) {
|
||||
echo 'Code to select node pool based on environment'
|
||||
switch (env.cicd_environment) {
|
||||
case 'prd':
|
||||
echo 'Code to select node pool based on memory and cpu request in prd'
|
||||
def mem_req_part = memory_request
|
||||
def cpu_req_part = cpu_request
|
||||
def priority = priority_v2
|
||||
echo "mem_req_part is ${mem_req_part}"
|
||||
echo "cpu_req_part is ${cpu_req_part}"
|
||||
if ( mem_req_part.contains('M') ) {
|
||||
mem_req = mem_req_part.replaceAll('Mi', '')
|
||||
mem_req = mem_req.replaceAll('M', '')
|
||||
try {
|
||||
mem_req = mem_req.toDouble()
|
||||
}
|
||||
catch (NumberFormatException e) {
|
||||
mem_req = mem_req.toDouble()
|
||||
//mem_req = mem_req.toInteger()
|
||||
}
|
||||
}
|
||||
else if ( mem_req_part.contains('G') ) {
|
||||
mem_req = mem_req_part.replaceAll('Gi', '')
|
||||
mem_req = mem_req.replaceAll('G', '')
|
||||
try {
|
||||
mem_req = mem_req.toDouble() * 1024
|
||||
}
|
||||
catch (NumberFormatException e) {
|
||||
mem_req = mem_req.toDouble() * 1024
|
||||
//mem_req = mem_req.toInteger()
|
||||
}
|
||||
}
|
||||
echo "memory_request is ${memory_request} ${mem_req}"
|
||||
|
||||
if ( cpu_req_part.contains('m') ) {
|
||||
cpu_req = cpu_req_part.replaceAll('m', '')
|
||||
try {
|
||||
cpu_req = cpu_req.toDouble()
|
||||
}
|
||||
catch (NumberFormatException e) {
|
||||
cpu_req = cpu_req.toDouble()
|
||||
//cpu_req = cpu_req.toInteger()
|
||||
}
|
||||
}
|
||||
else {
|
||||
try {
|
||||
cpu_req = cpu_req_part.toDouble() * 1000
|
||||
}
|
||||
catch (NumberFormatException e) {
|
||||
cpu_req = cpu_req_part.toDouble() * 1000
|
||||
//cpu_req = cpu_req.toInteger()
|
||||
}
|
||||
}
|
||||
echo "cpu_request is ${cpu_request} ${cpu_req}"
|
||||
|
||||
def ratio = (mem_req / cpu_req).toDouble()
|
||||
if ( cpu_req > mem_req ) {
|
||||
ratio = 2
|
||||
}
|
||||
//ratio = ratio.toInteger()
|
||||
echo "Ratio - ${ratio}"
|
||||
if(priority.equalsIgnoreCase('cp1')||priority.equalsIgnoreCase('cp2')||priority.equalsIgnoreCase('cp3')||priority.equalsIgnoreCase('up1')||priority.equalsIgnoreCase('up2')||priority.equalsIgnoreCase('up3')||priority.equalsIgnoreCase('sp1')||priority.equalsIgnoreCase('sp2')||priority.equalsIgnoreCase('sp3')){
|
||||
low_priority="lite"
|
||||
if ( ratio >= 2.5 ) {
|
||||
ratiovalue = "tetra"
|
||||
}
|
||||
else {
|
||||
ratiovalue = "duo"
|
||||
}
|
||||
if ( cpu_req >=2200 ) {
|
||||
nodename = "sumo"
|
||||
}
|
||||
else{
|
||||
nodename= "mega"
|
||||
}
|
||||
nodeSelectorvalue = "${nodename}${ratiovalue}${low_priority}"
|
||||
println nodeSelectorvalue
|
||||
break
|
||||
}
|
||||
else{
|
||||
if ( ratio > 5.5 ) {
|
||||
ratiovalue = 'octa'
|
||||
}
|
||||
else if ( ratio >= 2.5 ) {
|
||||
ratiovalue = 'tetra'
|
||||
}
|
||||
else {
|
||||
ratiovalue = 'duo'
|
||||
}
|
||||
|
||||
if ( cpu_req >= 2200) {
|
||||
nodevalue = 'sumo'
|
||||
}
|
||||
else if ( cpu_req < 2200 && cpu_req >= 1000) {
|
||||
nodevalue = 'mega'
|
||||
}
|
||||
else {
|
||||
nodevalue = 'compact'
|
||||
}
|
||||
|
||||
switch ( env.BU?.toLowerCase() ) {
|
||||
case 'supply':
|
||||
if (nodevalue == 'sumo' && (ratiovalue == 'hexa' || ratiovalue == 'octa')){
|
||||
ratiovalue = 'tetra'
|
||||
}
|
||||
else if (nodevalue == 'mega' && (ratiovalue == 'quad' || ratiovalue == 'octa')){
|
||||
ratiovalue = 'tetra'
|
||||
}
|
||||
else if (nodevalue == 'compact' && ratiovalue == 'trio'){
|
||||
ratiovalue = 'tetra'
|
||||
}
|
||||
break
|
||||
case 'demand':
|
||||
if (nodevalue == 'mega' && ratiovalue == 'quad'){
|
||||
ratiovalue = 'tetra'
|
||||
}
|
||||
else if (nodevalue == 'compact' && (ratiovalue == 'octa' || ratiovalue == 'trio')){
|
||||
ratiovalue = 'tetra'
|
||||
}
|
||||
break
|
||||
}
|
||||
|
||||
nodeSelectorvalue = "${nodevalue}${ratiovalue}"
|
||||
break
|
||||
|
||||
}
|
||||
case 'int':
|
||||
echo 'Shared node pool for int/pre-prod'
|
||||
nodeSelectorvalue = "preprod-cost-optimized"
|
||||
break
|
||||
case ['dev', 'ftr', 'stg']:
|
||||
echo 'Shared node pool for dev and ftr'
|
||||
nodeSelectorvalue = "${env.BU}-shared"
|
||||
break
|
||||
default:
|
||||
log.error('Unable to fetch environment')
|
||||
}
|
||||
return nodeSelectorvalue
|
||||
}
|
||||
@@ -1,164 +0,0 @@
|
||||
package com.homelab.stages
|
||||
|
||||
def run(Map config){
|
||||
stage('Build docker image'){
|
||||
container('dockerpush') {
|
||||
def env = "dev" //write if condition according to branch if branch is dev/staging or any other branch then master it will be dev if master prod
|
||||
def ecrImagePod = config.ecrImagePod
|
||||
def repo_name = config.repo_name
|
||||
def dockerBuildVersion = config.dockerBuildVersion ?: ""
|
||||
def version = getVersion()
|
||||
def artifactId = getArtifactId()
|
||||
def modules = getModules()
|
||||
def excludedModules = config.excludedModules ?: []
|
||||
j = 0
|
||||
for (module in modules){
|
||||
if (excludedModules.contains(module)){
|
||||
echo "===== NOT Building container for Module - ${module} ====="
|
||||
}else {
|
||||
def dockerfileRef = getDockerfile("${dockerBuildVersion}","${repo_name}","${version}","${module}","NO")
|
||||
j+=1
|
||||
sh """
|
||||
echo "===== ${j}. Creating Container for Module - ${module} ======"
|
||||
if [ -f ${module}/target/*.jar ]
|
||||
then
|
||||
echo "docker build -t ${ecrImagePod}/${repo_name}/${module} ${dockerfileRef}; rm -f Dockerfile-${module}"
|
||||
else
|
||||
echo "No jar file found."
|
||||
exit 1
|
||||
fi
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def getCommitid(){
|
||||
return sh(returnStdout: true, script: 'git log -1 --format=%h').trim()
|
||||
}
|
||||
|
||||
def registry(String buildenv){
|
||||
switch(env) {
|
||||
case "prod":
|
||||
return 'asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/prod/'
|
||||
break;
|
||||
case "dev":
|
||||
return 'asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622/dev/'
|
||||
break;
|
||||
default:
|
||||
return 'asia-southeast1-docker.pkg.dev/homelab-devops-admin-0622'
|
||||
}
|
||||
}
|
||||
|
||||
def getVersion(){
|
||||
// if (fileExists("pom.xml")) {
|
||||
// return sh(returnStdout: true, script: 'xq -r .project.version pom.xml').trim()
|
||||
// }else if (fileExists("package.json")) {
|
||||
// return sh(returnStdout: true, script: 'jq -r .version package.json').trim()
|
||||
// }
|
||||
return "2.0"
|
||||
}
|
||||
|
||||
def getModules(){
|
||||
// if (fileExists("pom.xml")) {
|
||||
// modules = sh(returnStdout: true, script: 'xq -r .project.modules.module[] pom.xml 2>/dev/null || xq -r .project.modules.module pom.xml 2>/dev/null || echo empty').trim()
|
||||
// if (modules == 'empty' || modules == 'null') {
|
||||
// return null
|
||||
// }
|
||||
// modules = modules.split("\n")
|
||||
// return modules
|
||||
// }
|
||||
// else {
|
||||
// return null
|
||||
// }
|
||||
return ["server"]
|
||||
}
|
||||
|
||||
|
||||
def getArtifactId(){
|
||||
// if (fileExists("pom.xml")) {
|
||||
// return sh(returnStdout: true, script: 'xq -r .project.artifactId pom.xml').trim()
|
||||
// } else if (fileExists("package.json")) {
|
||||
// return sh(returnStdout: true, script: 'jq -r .name package.json').trim()
|
||||
// }
|
||||
return "external-payment-gateway"
|
||||
}
|
||||
|
||||
def getTag(){
|
||||
def buildenv = "${env.BUILD_ENV}"
|
||||
echo "========get version========"
|
||||
def version = getVersion()
|
||||
echo "========commitID========"
|
||||
def commitID = getCommitid()
|
||||
echo "========tag========"
|
||||
def tag = "v${version}-${commitID}"
|
||||
return ${tag}
|
||||
}
|
||||
|
||||
|
||||
|
||||
def release(Map config){
|
||||
stage('Push docker image'){
|
||||
def repo_name = config.repo_name
|
||||
def branch_name = "${env.BRANCH_NAME}"
|
||||
def ecrImagePod = config.ecrImagePod
|
||||
def bu = config.BU
|
||||
def buildenv = "${env.BUILD_ENV}"
|
||||
echo "========registry========"
|
||||
def registry = registry("${buildenv}")
|
||||
echo "========get version========"
|
||||
def version = getVersion()
|
||||
echo "========commitID========"
|
||||
def commitID = getCommitid()
|
||||
echo "========tag========"
|
||||
echo "registry is ${registry}"
|
||||
def tag = "v${version}-${commitID}"
|
||||
echo "${registry}"
|
||||
echo "tag is ${tag}"
|
||||
def artifactId = getArtifactId()
|
||||
def modules = getModules()
|
||||
def excludedModules = config.excludedModules ?: []
|
||||
|
||||
log.info("########################### Pushing Docker Images ###########################")
|
||||
if (modules == null){
|
||||
echo "docker tag ${ecrImagePod}/${repo_name} ${registry}/${bu}/${ecrImagePod}/${repo_name}:${tag}"
|
||||
echo "gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet"
|
||||
echo "docker push ${registry}/${bu}/${ecrImagePod}/${repo_name}:${tag}"
|
||||
}else {
|
||||
j = 0
|
||||
for (module in modules){
|
||||
if (excludedModules.contains(module)){
|
||||
echo "===== NOT Pushing container for Module - ${common} ====="
|
||||
}else {
|
||||
j+=1
|
||||
sh """
|
||||
echo "===== ${j}. Pushing Container for Module - ${module} ======"
|
||||
echo "docker tag ${ecrImagePod}/${repo_name}/${module} ${registry}/${bu}/${ecrImagePod}/${repo_name}/${module}:${tag}"
|
||||
echo "gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet"
|
||||
echo "docker push ${registry}/${bu}/${ecrImagePod}/${repo_name}/${module}:${tag}"
|
||||
"""
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
def getDockerfile(String dockerBuildVersion, String repo_name, String version, String module, String doBuild) {
|
||||
if (dockerBuildVersion) {
|
||||
if(dockerBuildVersion == "maven-3.3-jdk-8"){
|
||||
def scriptcontents = libraryResource "org/homelab/templates/${dockerBuildVersion}.sh"
|
||||
writeFile file: "${dockerBuildVersion}.sh", text: scriptcontents
|
||||
sh "chmod a+x ./${dockerBuildVersion}.sh;./${dockerBuildVersion}.sh ${module} ${version} ${doBuild} ${repo_name}"
|
||||
return "-f Dockerfile-${module} ."
|
||||
}else if(dockerBuildVersion == "node-12.22"){
|
||||
def scriptcontents = libraryResource "org/homelab/templates/${dockerBuildVersion}.sh"
|
||||
writeFile file: "${dockerBuildVersion}.sh", text: scriptcontents
|
||||
sh "chmod a+x ./${dockerBuildVersion}.sh;./${dockerBuildVersion}.sh ${module}"
|
||||
return "-f Dockerfile-${module} ."
|
||||
}
|
||||
} else {
|
||||
return "-f Dockerfile ."
|
||||
}
|
||||
}
|
||||
@@ -1,43 +0,0 @@
|
||||
import com.homelab.stages.checkOut
|
||||
import com.homelab.stages.buildObjHelper
|
||||
import com.homelab.stages.notify
|
||||
import com.homelab.stages.securityScan
|
||||
import com.homelab.stages.automationTest
|
||||
import com.homelab.utilities.constructParam
|
||||
|
||||
def call(Map param) {
|
||||
def constructParam = new constructParam()
|
||||
def checkObj = new checkOut()
|
||||
def buildObjHelper = new buildObjHelper()
|
||||
def buildObj = buildObjHelper.run(param.build_tool)
|
||||
def notify = new notify()
|
||||
def securityScan = new securityScan()
|
||||
def automationTest = new automationTest()
|
||||
node('slave02') {
|
||||
env.msg = 'Job Passed'
|
||||
timestamps {
|
||||
ansiColor('xterm') {
|
||||
try {
|
||||
constructParam.run(param)
|
||||
log.info(param)
|
||||
checkObj.run(param)
|
||||
// securityScan.run(param)
|
||||
buildObj.run(param)
|
||||
if (param.containsKey('run_automation') && env.CHANGE_ID) {
|
||||
automationTest.run(param)
|
||||
}
|
||||
}
|
||||
catch (Exception e) {
|
||||
if (env.msg == 'Job Passed') {
|
||||
log.error(e.toString())
|
||||
currentBuild.result = env.FAILURE
|
||||
env.msg = 'Job Failed. Error: ' + e.toString()
|
||||
}
|
||||
}
|
||||
finally {
|
||||
notify.run(param)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,195 +0,0 @@
|
||||
// cdHookRunner — standalone Jenkins job that runs turbo-turtle deploy-phase
|
||||
// CI/CD hooks (pre_deploy / post_deploy) in an isolated agent pod.
|
||||
//
|
||||
// turbo-turtle triggers this parameterized job during the CD workflow (see
|
||||
// TriggerCdHookPipelineActivity). It checks out the target repo, reads the
|
||||
// deploy-phase hooks declared under `environment.<TT_ENV>.hooks.<HOOK_PHASE>` in
|
||||
// the per-app deployment file (deployments/<APP_NAME>.yaml), executes them via
|
||||
// the shared runHooks execution core, and POSTs an aggregate completion callback
|
||||
// to /api/v1/cd/hook/callback which signals the waiting workflow. A blocking hook
|
||||
// failure makes success=false so the workflow can fail the deploy; advisory
|
||||
// (blocking:false) failures keep success=true.
|
||||
//
|
||||
// Required parameters (Jenkins job params, read from params/env):
|
||||
// REPO_URL, TT_REPO, APP_NAME, BRANCH, COMMIT, TT_ENV, HOOK_PHASE,
|
||||
// IMAGE_TAG, TT_IS_HOTFIX, TT_PR_NUMBER, TT_WORKFLOW_ID, TT_RUN_ID
|
||||
def call(Map jobParams = [:]) {
|
||||
String repoUrl = param('REPO_URL', jobParams)
|
||||
String repoName = param('TT_REPO', jobParams)
|
||||
String appName = param('APP_NAME', jobParams)
|
||||
String branch = param('BRANCH', jobParams)
|
||||
String commit = param('COMMIT', jobParams)
|
||||
String ttEnv = param('TT_ENV', jobParams)
|
||||
String phase = param('HOOK_PHASE', jobParams)
|
||||
String workflowID = param('TT_WORKFLOW_ID', jobParams)
|
||||
String runID = param('TT_RUN_ID', jobParams)
|
||||
|
||||
if (!repoUrl && repoName) {
|
||||
repoUrl = "https://github.com/Homelab/${repoName}"
|
||||
}
|
||||
// TT_WORKFLOW_ID / TT_RUN_ID are required: the completion callback correlates
|
||||
// to the waiting Temporal run by them. If either is missing (e.g. incomplete
|
||||
// job-param registration), fail before checkout/hook execution rather than run
|
||||
// hooks whose callback can never be matched (workflow would wait to timeout).
|
||||
if (!repoName || !appName || !ttEnv || !phase || !workflowID || !runID) {
|
||||
error("cdHookRunner: TT_REPO, APP_NAME, TT_ENV, HOOK_PHASE, TT_WORKFLOW_ID and TT_RUN_ID are required parameters.")
|
||||
}
|
||||
// Make env available to stageName / the hook context.
|
||||
env.cicd_environment = ttEnv
|
||||
|
||||
String podyaml = "org/homelab/${env.INFRA_ENV ?: 'prd'}-pod.yaml"
|
||||
podTemplate(yaml: libraryResource(podyaml)) {
|
||||
node(POD_LABEL) {
|
||||
container('devops-tools') {
|
||||
boolean success = true
|
||||
String errMsg = ''
|
||||
try {
|
||||
stage(stageName("cd-hook checkout: ${repoName}")) {
|
||||
deleteDir()
|
||||
dir(repoName) {
|
||||
checkout([
|
||||
$class: 'GitSCM',
|
||||
branches: [[name: commit ?: "*/${branch}"]],
|
||||
userRemoteConfigs: [[url: repoUrl, credentialsId: 'svc-devops-homelab']],
|
||||
extensions: [[$class: 'CloneOption', shallow: false, noTags: false]],
|
||||
])
|
||||
}
|
||||
}
|
||||
|
||||
def hooks = resolveHooks(repoName, appName, ttEnv, phase)
|
||||
if (!hooks) {
|
||||
log.info("cdHookRunner: no ${phase} hooks declared for env ${ttEnv} in ${repoName}/deployments/${appName}.yaml — nothing to run.")
|
||||
} else {
|
||||
List<String> ctxEnv = deployPhaseContextEnv(repoName, ttEnv, jobParams)
|
||||
runHooks.executeHooks(repoName, hooks, phase, ctxEnv)
|
||||
}
|
||||
} catch (Exception e) {
|
||||
success = false
|
||||
errMsg = e.toString()
|
||||
log.error("cdHookRunner: ${phase} hook run failed for ${repoName} (env ${ttEnv}): ${errMsg}")
|
||||
} finally {
|
||||
postCdHookCallback(repoName, ttEnv, phase, success, errMsg, workflowID, runID)
|
||||
}
|
||||
if (!success) {
|
||||
// Surface as a build failure too (the workflow reads the callback,
|
||||
// but a red build aids debugging).
|
||||
currentBuild.result = 'FAILURE'
|
||||
error("cdHookRunner: ${phase} hooks failed for ${repoName}: ${errMsg}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// resolveHooks reads the per-app deployment file (deployments/<appName>.yaml) and
|
||||
// returns the deploy-phase hook list for env+phase (or null). Deploy-phase hooks
|
||||
// live in deployment.yaml, not config.yaml.
|
||||
def resolveHooks(String repoName, String appName, String ttEnv, String phase) {
|
||||
dir(repoName) {
|
||||
String deployFile = "deployments/${appName}.yaml"
|
||||
if (!fileExists(deployFile)) {
|
||||
error("cdHookRunner: ${deployFile} not found in ${repoName}.")
|
||||
}
|
||||
def deployment = readYaml file: deployFile
|
||||
def envBlock = deployment?.environment?.get(ttEnv)
|
||||
def hooksBlock = envBlock?.hooks
|
||||
if (!(hooksBlock instanceof Map)) {
|
||||
return null
|
||||
}
|
||||
def hooks = hooksBlock[phase]
|
||||
return (hooks instanceof List && !hooks.isEmpty()) ? hooks : null
|
||||
}
|
||||
}
|
||||
|
||||
// deployPhaseContextEnv builds the TT_* context contract from the job params.
|
||||
def deployPhaseContextEnv(String repoName, String ttEnv, Map jobParams) {
|
||||
return [
|
||||
"TT_REPO_NAME=${repoName}",
|
||||
"TT_ENV=${ttEnv}",
|
||||
"TT_EVENT=push",
|
||||
"TT_IS_HOTFIX=${param('TT_IS_HOTFIX', jobParams) ?: 'false'}",
|
||||
"TT_BRANCH=${param('BRANCH', jobParams) ?: ''}",
|
||||
"TT_TARGET_BRANCH=",
|
||||
"TT_PR_NUMBER=${param('TT_PR_NUMBER', jobParams) ?: ''}",
|
||||
"TT_COMMIT_SHA=${param('COMMIT', jobParams) ?: ''}",
|
||||
"TT_IMAGE_TAG=${param('IMAGE_TAG', jobParams) ?: ''}",
|
||||
]
|
||||
}
|
||||
|
||||
// postCdHookCallback POSTs the aggregate result to turbo-turtle, which signals
|
||||
// the waiting CD workflow. Env-routed exactly like the Jenkins CI callback.
|
||||
def postCdHookCallback(String repoName, String ttEnv, String phase, boolean success, String errMsg, String workflowID, String runID) {
|
||||
String baseUrl
|
||||
switch (ttEnv) {
|
||||
case 'prd':
|
||||
case 'int':
|
||||
baseUrl = 'http://turbo-turtle.homelabgcp.in'
|
||||
break
|
||||
default:
|
||||
baseUrl = 'http://turbo-turtle.admin.homelabgcp.in'
|
||||
}
|
||||
Map payload = [
|
||||
repo_name : repoName,
|
||||
env : ttEnv,
|
||||
phase : phase,
|
||||
success : success,
|
||||
error : errMsg,
|
||||
build_url : env.BUILD_URL ?: '',
|
||||
workflow_id: workflowID,
|
||||
run_id : runID,
|
||||
]
|
||||
String url = baseUrl + "/api/v1/cd/hook/callback"
|
||||
String jsonFilePath = "cd_hook_callback_${env.BUILD_NUMBER}.json"
|
||||
int maxAttempts = 3
|
||||
// The callback is turbo-turtle's ONLY completion signal for the waiting run.
|
||||
// Retry delivery (bounded, with connect/request timeouts) and, if every attempt
|
||||
// fails, fail the job so the failure is visible — otherwise the job goes green
|
||||
// while the workflow waits until its backstop timeout. Duplicate delivery is
|
||||
// safe: turbo-turtle correlates callbacks by workflow/run/repo/env/phase.
|
||||
try {
|
||||
writeFile(file: jsonFilePath, text: writeJSON(returnText: true, json: payload))
|
||||
for (int attempt = 1; attempt <= maxAttempts; attempt++) {
|
||||
boolean delivered = false
|
||||
try {
|
||||
final def(String response, String code) = sh(
|
||||
returnStdout: true,
|
||||
script: """
|
||||
curl -s --connect-timeout 10 --max-time 30 -X POST \\
|
||||
-H 'Content-Type: application/json' \\
|
||||
-w '\\n%{response_code}' \\
|
||||
$url \\
|
||||
-d @$jsonFilePath
|
||||
"""
|
||||
).trim().tokenize("\n")
|
||||
if (code == "200") {
|
||||
log.info("cdHookRunner: callback delivered (attempt ${attempt}/${maxAttempts})")
|
||||
delivered = true
|
||||
} else {
|
||||
log.warn("cdHookRunner: callback attempt ${attempt}/${maxAttempts} failed code=${code} response=${response}")
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.warn("cdHookRunner: callback attempt ${attempt}/${maxAttempts} error: ${e}")
|
||||
}
|
||||
if (delivered) {
|
||||
return
|
||||
}
|
||||
if (attempt < maxAttempts) {
|
||||
sleep(time: attempt * 5, unit: 'SECONDS')
|
||||
}
|
||||
}
|
||||
error("cdHookRunner: callback POST to ${url} failed after ${maxAttempts} attempts; turbo-turtle will not receive completion for repo=${repoName} env=${ttEnv} phase=${phase}")
|
||||
} finally {
|
||||
sh(script: "rm -f ${jsonFilePath}", returnStatus: true)
|
||||
}
|
||||
}
|
||||
|
||||
// param reads a job parameter, preferring an explicit map, then params, then env.
|
||||
def param(String key, Map jobParams) {
|
||||
if (jobParams?.containsKey(key)) {
|
||||
return jobParams[key]?.toString()
|
||||
}
|
||||
if (params?.containsKey(key) && params[key] != null) {
|
||||
return params[key].toString()
|
||||
}
|
||||
return env[key]?.toString()
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
|
||||
pipeline {
|
||||
agent none
|
||||
|
||||
environment {
|
||||
K8S_LABEL = 'cloud-function-cicd-agent'
|
||||
GITHUB_CRED = 'cicd-github-app'
|
||||
}
|
||||
|
||||
podTemplate(yaml: libraryResource('org/homelab/pod-cloud-function.yaml')) {
|
||||
node(POD_LABEL) {
|
||||
container('devops-tools') {
|
||||
cloudFunctionCICDFlow()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def cloudFunctionCICDFlow() {
|
||||
stages {
|
||||
stage {
|
||||
steps {
|
||||
script {
|
||||
sh 'ls -al'
|
||||
echo 'Hello World'
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,24 +0,0 @@
|
||||
import com.homelab.stages.helmGenerator
|
||||
import com.homelab.stages.validateBuTeam
|
||||
|
||||
def call(Map params) {
|
||||
podTemplate(yaml: libraryResource("org/homelab/${env.INFRA_ENV}-pod.yaml")) {
|
||||
node(POD_LABEL) {
|
||||
container('devops-tools') {
|
||||
timestamps {
|
||||
ansiColor('xterm') {
|
||||
env.GITHUB_CRED = 'svc-devops-homelab'
|
||||
// Initialize objects
|
||||
def validateObj = new validateBuTeam()
|
||||
def hemlGenObj = new helmGenerator()
|
||||
|
||||
// validate parameters
|
||||
validateObj.run(params.bu, params.team, params.module)
|
||||
// Create helm file
|
||||
hemlGenObj.run(params)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,155 +0,0 @@
|
||||
import com.homelab.stages.hotFix
|
||||
import com.homelab.stages.checkOut
|
||||
import com.homelab.stages.buildObjHelper
|
||||
import com.homelab.stages.notify
|
||||
import com.homelab.utilities.getYamlParameter
|
||||
import com.homelab.utilities.constructParam
|
||||
import java.time.ZonedDateTime
|
||||
import java.time.format.DateTimeFormatterBuilder
|
||||
|
||||
def call(Map repo) {
|
||||
ansiColor('xterm') {
|
||||
def userId = null
|
||||
|
||||
try {
|
||||
def cause = currentBuild.rawBuild.getCause(hudson.model.Cause.UserIdCause)
|
||||
if (cause) {
|
||||
userId = cause.getUserId()
|
||||
}
|
||||
} catch (Exception e) {
|
||||
log.error("Failed to get user ID: ${e.message}")
|
||||
}
|
||||
|
||||
// Per-user allowlist removed — it hardcoded real people's emails
|
||||
// from the original org and doesn't apply to a solo homelab.
|
||||
// This whole eksCICD path is unused legacy code anyway (nothing
|
||||
// in homelabPipeline.groovy calls it); the branch below is now
|
||||
// permanently skipped rather than deleted, to avoid hand-editing
|
||||
// the escape-sequence-heavy echo blocks it guards.
|
||||
if (false) {
|
||||
echo "\u001B[1;31m========================================\n[ERROR] Build triggered by unauthorized user: ${userId}\n\nPlease use Ringmaster to trigger builds and deployments: https://ringmaster.homelabgcp.in/applications/cicd/home\n========================================\u001B[0m"
|
||||
error("Build triggered by unauthorized user: ${userId}.")
|
||||
} else if (!userId) {
|
||||
echo "\u001B[1;31m========================================\n[ERROR] Could not determine the user who triggered the build\n\nThis might be a scheduled or system-triggered build.\n========================================\u001B[0m"
|
||||
error("Could not determine the user who triggered the build")
|
||||
} else {
|
||||
echo "Starting the build....."
|
||||
}
|
||||
}
|
||||
|
||||
env.STARTTIME = getDateTime()
|
||||
env.FAILURE = 'FAILURE'
|
||||
echo "CLOUD_PROVIDER : ${env.CLOUD_PROVIDER}"
|
||||
echo "INFRA_ENV : ${env.INFRA_ENV}"
|
||||
switch (env.CLOUD_PROVIDER) {
|
||||
case 'GCP':
|
||||
echo 'Running on GCP'
|
||||
gcpInfra(repo)
|
||||
break
|
||||
case 'AWS':
|
||||
echo 'Running on AWS'
|
||||
awsInfra(repo)
|
||||
break
|
||||
default:
|
||||
log.error('Not running on AWS or GCP')
|
||||
currentBuild.result = env.FAILURE
|
||||
def msg = 'Job Failed. Error: Not running on AWS or GCP'
|
||||
notify.run(msg)
|
||||
}
|
||||
}
|
||||
|
||||
def gcpInfra (Map repo) {
|
||||
def isSidecarNeeded = repo.get('useSidecar', false)
|
||||
def yamlName = isSidecarNeeded ? "${env.INFRA_ENV}-sidecar-pod.yaml" : "${env.INFRA_ENV}-pod.yaml"
|
||||
def podyaml = "org/homelab/${yamlName}"
|
||||
echo "Architecture Check: useSidecar=${isSidecarNeeded}. Loading ${podyaml}"
|
||||
podTemplate(yaml: libraryResource(podyaml)) {
|
||||
node(POD_LABEL) {
|
||||
container('devops-tools') {
|
||||
commonCICDFlow(repo)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def awsInfra (Map repo) {
|
||||
node('EKS') {
|
||||
commonCICDFlow(repo)
|
||||
}
|
||||
}
|
||||
|
||||
def commonCICDFlow (Map repo) {
|
||||
def constructParam = new constructParam()
|
||||
def ymlObj = new getYamlParameter()
|
||||
def checkObj = new checkOut()
|
||||
def buildObjHelper = new buildObjHelper()
|
||||
def hotFixObj = new hotFix()
|
||||
def notify = new notify()
|
||||
def param = [:]
|
||||
def msg = 'Job Passed'
|
||||
env.msg = msg
|
||||
env.error_msg_to_db = ''
|
||||
|
||||
timestamps {
|
||||
ansiColor('xterm') {
|
||||
try {
|
||||
env.deploymentStartTime = new Date().format('yyyy-MM-dd HH:mm:ss')
|
||||
checkObj.run(repo)
|
||||
param = ymlObj.getParam(repo.repo_name)
|
||||
def maintainer = param.maintainer ?: 'jenkins-user'
|
||||
def buildObj = buildObjHelper.run(param.build_tool)
|
||||
constructParam.run(param)
|
||||
param['skip_notify'] = env.skip_notify
|
||||
log.info(param)
|
||||
hotFixObj.run(param.repo_name)
|
||||
// Pre-build hooks: user scripts declared under
|
||||
// environment.<env>.hooks.pre_build in the repo's config.yaml.
|
||||
// constructParam has merged the resolved env block into param,
|
||||
// so param.hooks holds the current environment's hooks. A
|
||||
// blocking hook failure fails the build before the docker build.
|
||||
runHooks(param, 'pre_build')
|
||||
buildObj.run(param)
|
||||
// Post-build hooks: run after a successful build.
|
||||
runHooks(param, 'post_build')
|
||||
// Auto-trigger the standalone ai-blitz-jobs (coverage-only) job
|
||||
// once the per-repo CI has succeeded. Internally gated on
|
||||
// PR / hot-fix / toolchain / mainline-branch (see helper), and
|
||||
// fire-and-forget — coverage observability must never block
|
||||
// or fail this build. Repos opt out via skip_coverage_trigger
|
||||
// in config.yaml.
|
||||
triggerCoverageOnly(param)
|
||||
}
|
||||
catch (org.jenkinsci.plugins.workflow.steps.FlowInterruptedException e) {
|
||||
currentBuild.result = 'ABORTED'
|
||||
log.info(currentBuild.result)
|
||||
env.msg = 'Job Aborted'
|
||||
env.error_msg_to_db = 'Job ABORTED by the User'
|
||||
}
|
||||
catch (Exception e) {
|
||||
if (env.msg == msg) {
|
||||
log.error(e.toString())
|
||||
currentBuild.result = env.FAILURE
|
||||
env.msg = 'Job Failed. Error: ' + e.toString()
|
||||
}
|
||||
}
|
||||
finally {
|
||||
env.deploymentEndTime = new Date().format('yyyy-MM-dd HH:mm:ss')
|
||||
notify.run(param)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@NonCPS
|
||||
def getDateTime() {
|
||||
// Get the current date and time in IST
|
||||
def currentDateTime = ZonedDateTime.now()
|
||||
// Create a formatter for the desired pattern
|
||||
def formatter = new DateTimeFormatterBuilder()
|
||||
.appendPattern("yyyy-MM-dd'T'HH:mm:ss")
|
||||
.appendOffset('+HH:mm', '+00:00')
|
||||
.toFormatter()
|
||||
// Format the current date and time using the formatter
|
||||
def formattedDateTime = currentDateTime.format(formatter)
|
||||
return formattedDateTime
|
||||
}
|
||||
@@ -1,278 +0,0 @@
|
||||
import com.homelab.utilities.gitActions
|
||||
import com.homelab.utilities.buTeamMapping
|
||||
import com.homelab.utilities.getYamlParameter
|
||||
import com.homelab.utilities.constructTemplate
|
||||
import com.homelab.stages.multiBranchPipeline
|
||||
import com.homelab.stages.deployArgoCD
|
||||
|
||||
def call(Map params){
|
||||
podTemplate(yaml: libraryResource('org/homelab/pod.yaml')) {
|
||||
node(POD_LABEL) {
|
||||
container('devops-tools') {
|
||||
timestamps{
|
||||
ansiColor("xterm"){
|
||||
env.GITHUB_CRED = 'svc-devops-homelab'
|
||||
def gitObj = new gitActions()
|
||||
def bu_team_obj = new buTeamMapping()
|
||||
def constructObj = new constructTemplate()
|
||||
def application_info_map = [:]
|
||||
def repo_bu = ""
|
||||
stage('App Repo: cloning the repo'){
|
||||
gitObj.clone("${WORKSPACE}", params.repo_name, params.repo_main_branch)
|
||||
}
|
||||
|
||||
stage('APP Repo: Create Jenkinsfile(gcp-main)'){
|
||||
if (!params.skip_jenkinsfile){
|
||||
def repo_branch_name = "gcp-main"
|
||||
def config = ["repo_name":params.repo_name]
|
||||
gitObj.branchCheckOut(params.repo_name, repo_branch_name)
|
||||
dir(params.repo_name){
|
||||
sh "chmod 777 Jenkinsfile"
|
||||
constructObj.renderTemplate(config, 'Jenkinsfile', 'Jenkinsfile')
|
||||
gitObj.add('.', 'Jenkinsfile')
|
||||
}
|
||||
def commit_status = gitObj.codeCommit(repo_name, repo_branch_name, 'Generating build and deployments files')
|
||||
if (commit_status == 0) {
|
||||
gitObj.codePush(repo_name, repo_branch_name)
|
||||
}
|
||||
}
|
||||
else {
|
||||
log.info("skipping this stage")
|
||||
}
|
||||
}
|
||||
|
||||
stage('Helm Repo: Update values_property file'){
|
||||
def yaml_obj = new getYamlParameter()
|
||||
gitObj.clone("${WORKSPACE}","devops-helm-charts","main")
|
||||
def helm_repo_name = "devops-helm-charts"
|
||||
def helm_checkout_branch = "gcp-migration-${params.repo_name}-${env.BUILD_NUMBER}"
|
||||
def repo_branch_name = "gcp-main"
|
||||
gitObj.branchCheckOut("devops-helm-charts", helm_checkout_branch)
|
||||
gitObj.branchCheckOut(params.repo_name, repo_branch_name)
|
||||
def build_config = yaml_obj.getParam(params.repo_name)
|
||||
|
||||
if (build_config['copy_file'] || (build_config['environment'] && build_config['environment']['prd']['copy_file'])){
|
||||
// voilating naming convention willing to avoid confilct with deployment variables
|
||||
log.info("Checking copy_file path")
|
||||
def buSrt = bu_team_obj.get_bu_initials(build_config["bu"])
|
||||
def teamSrt = bu_team_obj.get_team_initials(build_config["team"])
|
||||
def copy_path = (build_config['copy_file'])? build_config.copy_file.path : build_config.environment.prd.copy_file.path
|
||||
if (copy_path.startsWith("gs:")){
|
||||
log.info("Copy file path already pointing to gcs")
|
||||
} else {
|
||||
def file_name = copy_path.split('/')[-1]
|
||||
def new_path = "gs://gcs-${buSrt}-${teamSrt}-config-prd/${params.repo_name}/${file_name}"
|
||||
if(build_config.copy_file){
|
||||
build_config.copy_file.path = new_path
|
||||
} else {
|
||||
build_config.environment.prd.copy_file.path = new_path
|
||||
}
|
||||
dir(params.repo_name){
|
||||
sh "chmod 777 config.yaml"
|
||||
writeYaml file: 'config.yaml', data: build_config, overwrite: true
|
||||
}
|
||||
gitObj.add(params.repo_name, 'config.yaml')
|
||||
def config_commit = gitObj.codeCommit(repo_name, repo_branch_name, 'Changed copy_file path to gcs')
|
||||
if (config_commit == 0) {
|
||||
gitObj.codePush(repo_name, repo_branch_name)
|
||||
}
|
||||
log.warning("Path for copy_file has been changed to gcs. Please upload the contents to ${new_path} from ${copy_path}")
|
||||
}
|
||||
}
|
||||
if (build_config.containsKey('environment')) {
|
||||
Map envrionment_config = build_config['environment'].collectEntries { key, value -> "prd".matches(key)? value: [:]}
|
||||
build_config.remove('environment')
|
||||
build_config.putAll(envrionment_config)
|
||||
}
|
||||
repo_bu = build_config.bu
|
||||
env.skip_user_input = params.skip_userinput
|
||||
def wait_obj = new deployArgoCD()
|
||||
def deployment_order = build_config.deployment_order
|
||||
timeout(unit: 'SECONDS', time: 300) {
|
||||
userInput = wait_obj.wait_for_user_input(build_config.deployment_order)
|
||||
}
|
||||
if (userInput == '') {
|
||||
log.info('No Deployments selected. Running remaining steps.')
|
||||
return
|
||||
}
|
||||
else if (!userInput.contains('All')) {
|
||||
deployment_order = userInput.split(',') as List
|
||||
}
|
||||
|
||||
for (deployment in deployment_order){
|
||||
def application_config = yaml_obj.getParam(params.repo_name,"deployments/${deployment}.yaml")
|
||||
def bu_short = bu_team_obj.get_bu_initials(application_config["bu"])
|
||||
def BU = application_config["bu"]
|
||||
def team_short = bu_team_obj.get_team_initials(application_config["team"])
|
||||
//def app_name_short = application_config.app_name.substring(0, Math.min(originalString.length(), 15))
|
||||
def app_name_short = application_config.app_name
|
||||
def values_version = env.cicd_environment == 'prd' ? 'values_v3' : 'values_v2'
|
||||
def value_properties_path = "${values_version}/${bu_short}/${team_short}/${deployment}/values_properties.yaml"
|
||||
def value_properties_content = yaml_obj.getParam(helm_repo_name,value_properties_path)
|
||||
def canary_default = ['progressDeadlineSeconds': 300,'analysisInterval': '120s','analysisThreshold': 5,'analysisMaxWeight': 5,'analysisStepWeight': 5,'analysisMetrics':['thresholdRangeMin': 0.99,'interval': '1m'],'skipAnalysis': true]
|
||||
|
||||
// Set/change value_properties values for migration
|
||||
value_properties_content['as_min']= 1
|
||||
log.info("Setting canary skipAnalysis to true")
|
||||
if (value_properties_content['canary']){
|
||||
value_properties_content['canary']['skipAnalysis'] = true
|
||||
} else {
|
||||
value_properties_content['canary'] = canary_default
|
||||
}
|
||||
log.info("Canary skipAnalysis has been set to true")
|
||||
log.info("set min and nodeselector")
|
||||
if (value_properties_content['ingress_class']!= "contour-external"){
|
||||
if(value_properties_content['host']){
|
||||
value_properties_content.host = application_config['app_name']+".prd.homelab.int"
|
||||
}
|
||||
else if(value_properties_content.hosts){
|
||||
for (host_arr in value_properties_content.hosts){
|
||||
host_arr.host = application_config['app_name']+".prd.homelab.int"
|
||||
}
|
||||
}
|
||||
if (application_config.priority_v2 == "up0" || application_config.priority_v2 == "sp0" || application_config.priority_v2 == "cp0"){
|
||||
value_properties_content.ingress_class = "contour-internal-0"
|
||||
}
|
||||
else{
|
||||
value_properties_content.ingress_class = "contour-internal-1"
|
||||
}
|
||||
}
|
||||
else if (value_properties_content['ingress_class'] == "external"){
|
||||
value_properties_content.ingress_class = "contour-external"
|
||||
}
|
||||
log.info("set host")
|
||||
if (value_properties_content.serviceAccount){
|
||||
value_properties_content.serviceAccount.annotations.remove('eks.amazonaws.com/role-arn')
|
||||
value_properties_content.serviceAccount.annotations['iam.gke.io/gcp-service-account'] = "sa-${bu_short}-prd-${app_name_short}@homelab-${BU}-prd-0622.iam.gserviceaccount.com"
|
||||
}
|
||||
log.info("set service account")
|
||||
dir(helm_repo_name){
|
||||
sh "chmod 777 ${value_properties_path}"
|
||||
writeYaml file: value_properties_path, data: value_properties_content, overwrite: true
|
||||
}
|
||||
|
||||
def host = (value_properties_content.host)? value_properties_content.host : value_properties_content.hosts[0].host
|
||||
log.info(host)
|
||||
application_info_map[application_config.app_name]=[
|
||||
"bu": application_config.bu,
|
||||
"team": application_config.team,
|
||||
"host": host,
|
||||
"ingress_class": value_properties_content.ingress_class
|
||||
]
|
||||
log.info(application_info_map)
|
||||
gitObj.add(helm_repo_name,value_properties_path)
|
||||
}
|
||||
commit_status = gitObj.codeCommit(helm_repo_name, helm_checkout_branch, "Modified all application of ${params.repo_name} repo")
|
||||
if (commit_status == 0) {
|
||||
try{
|
||||
gitObj.codePush(helm_repo_name, helm_checkout_branch)
|
||||
}
|
||||
catch(Exception e){
|
||||
log.error("Error in code push")
|
||||
return
|
||||
}
|
||||
try{
|
||||
pr_num = gitObj.createPR(params.repo_name, helm_repo_name, "main", helm_checkout_branch, "Merge ${params.repo_name} changes")
|
||||
gitObj.mergePR(helm_repo_name, pr_num, helm_checkout_branch)
|
||||
gitObj.deleteBranch(helm_repo_name, "main", helm_checkout_branch)
|
||||
}
|
||||
catch(Exception ex){
|
||||
log.error("error in code push")
|
||||
log.error(ex)
|
||||
log.info("Deleting the branch")
|
||||
gitObj.deleteBranch(helm_repo_name, "main", helm_checkout_branch)
|
||||
}
|
||||
}
|
||||
}
|
||||
stage('Jenkins: Create Jenkins job'){
|
||||
def job_obj = new multiBranchPipeline()
|
||||
job_obj.applicationOnboard(repo_bu,params.repo_name)
|
||||
log.info("Jenkins job created")
|
||||
}
|
||||
// stage('GCP: Create DNS record'){
|
||||
// def zone="pvt-homelab-admin-prd-homelab-int"
|
||||
// def project="homelab-admin-prd-0622"
|
||||
// def app_name_list = application_info_map.collect{ it.key }
|
||||
// log.info(app_name_list)
|
||||
// for (app_name in app_name_list) {
|
||||
// def host=application_info_map[app_name].host
|
||||
// def ingress_class=application_info_map[app_name].ingress_class
|
||||
// def BU=application_info_map[app_name].bu
|
||||
// def dns_status= sh(script: "gcloud dns --project=${project} record-sets describe ${host} --zone=${zone} --type='CNAME' ",returnStatus: true)
|
||||
// if( dns_status != 0 ){
|
||||
// dns_status = sh(script: "gcloud dns --project=${project} record-sets create ${host} --zone=${zone} --type=CNAME --ttl='300' --rrdatas='${ingress_class}.${BU}.prd.prd.homelab.int.' ",returnStatus: true)
|
||||
// if(dns_status == 0 ){
|
||||
// log.info("DNS recordSet ${host} created successfully")
|
||||
// }
|
||||
// else{
|
||||
// log.info("DNS recordset failed with status ${dns_status}")
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
|
||||
// }
|
||||
// stage('GCP: Create workload identity'){
|
||||
// def bu_short=bu_team_obj.get_bu_initials(application_info_map[app_name].bu)
|
||||
// def BU = application_info_map[app_name].bu
|
||||
// def project="homelab-${BU}-prd-0622"
|
||||
// def app_name_list = application_info_map.collect{ it.key }
|
||||
// for (app_name in app_name_list){
|
||||
// if (params.provide_service_role){
|
||||
// //def app_name_short= app_name.substring(0, Math.min(originalString.length(), 15))
|
||||
// def app_name_short=app_name
|
||||
// def sa = "sa-${bu_short}-prd-${app_name_short}"
|
||||
// sh(script: "gcloud iam service-accounts create ${sa} --display-name='service account for ${app_name}' --project=${project} ")
|
||||
// sh(script: "gcloud iam service-accounts add-iam-policy-binding --role roles/iam.workloadIdentityUser --member 'serviceAccount:${sa}@homelab-${BU}-prd-0622.iam.gserviceaccount.com' --project=homelab-${BU}-prd-0622")
|
||||
// sh(script: "gcloud iam service-accounts add-iam-policy-binding --role roles/${params.SA_role} --member 'serviceAccount:${sa}@homelab-${BU}-prd-0622.iam.gserviceaccount.com' --project=homelab-${BU}-prd-0622")
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
stage('CoreDNS: Add entry for svc-svc'){
|
||||
def yaml_obj = new getYamlParameter()
|
||||
def coredns_repo_name = "devops-infra-helm-charts"
|
||||
def coredns_checkout_branch = "gcp-migration-${params.repo_name}-${env.BUILD_NUMBER}"
|
||||
gitObj.clone("${WORKSPACE}",coredns_repo_name,"main")
|
||||
gitObj.branchCheckOut(coredns_repo_name, coredns_checkout_branch)
|
||||
def coredns_path= "helm-templates/coredns/values.yaml"
|
||||
def coredns_content = yaml_obj.getParam(coredns_repo_name,coredns_path)
|
||||
def app_name_list = application_info_map.collect{ it.key }
|
||||
for (app_name in app_name_list) {
|
||||
def host=application_info_map[app_name].host
|
||||
def ingress_class=application_info_map[app_name].ingress_class
|
||||
def BU=application_info_map[app_name].bu
|
||||
coredns_content.rewrites[host]="${ingress_class}-${BU}-prd"
|
||||
log.info("coredns entry for ${host} is set")
|
||||
dir(coredns_repo_name){
|
||||
sh "chmod 777 ${coredns_path}"
|
||||
writeYaml file: coredns_path, data: coredns_content, overwrite: true
|
||||
}
|
||||
gitObj.add(coredns_repo_name,coredns_path)
|
||||
}
|
||||
commit_status = gitObj.codeCommit(coredns_repo_name, coredns_checkout_branch, "Modified all application of ${params.repo_name} repo")
|
||||
if (commit_status == 0) {
|
||||
try{
|
||||
gitObj.codePush(coredns_repo_name, coredns_checkout_branch)
|
||||
}
|
||||
catch(Exception e){
|
||||
log.error("Error in code push")
|
||||
return
|
||||
}
|
||||
try{
|
||||
pr_num = gitObj.createPR(params.repo_name, coredns_repo_name, "main", coredns_checkout_branch, "Merge ${params.repo_name} changes")
|
||||
gitObj.mergePR(coredns_repo_name, pr_num, coredns_checkout_branch)
|
||||
gitObj.deleteBranch(coredns_repo_name, "main", coredns_checkout_branch)
|
||||
}
|
||||
catch(Exception ex){
|
||||
log.error("error in code push")
|
||||
log.error(ex)
|
||||
log.info("Deleting the branch")
|
||||
gitObj.deleteBranch(coredns_repo_name, "main", coredns_checkout_branch)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,202 +0,0 @@
|
||||
import com.homelab.stages.buildDocker
|
||||
import com.homelab.stages.notify
|
||||
|
||||
|
||||
def Codecall(Map stepParams) {
|
||||
def buildDocker = new buildDocker()
|
||||
def notify = new notify()
|
||||
checkOutCode()
|
||||
config = readYaml file: "${stepParams.file}"
|
||||
def maintainer = config.maintainer ?: "jenkins-user"
|
||||
def msg = "Job Passed"
|
||||
try{
|
||||
mvnBuild()
|
||||
dockerBuild(config)
|
||||
}
|
||||
catch(Exception e){
|
||||
log.error(e.toString())
|
||||
currentBuild.result = env.FAILURE
|
||||
msg = "Job Failed. Error: "+ e.toString()
|
||||
}
|
||||
finally{
|
||||
log.info(msg)
|
||||
notify.run(msg,maintainer)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
def checkOutCode () {
|
||||
stage('Checking Out Code') {
|
||||
checkout scm
|
||||
}
|
||||
}
|
||||
|
||||
def mvnBuild() {
|
||||
stage('Maven Build') {
|
||||
container('maven') {
|
||||
sh "mvn clean install -DskipTests"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def Podcall(Map stepParams) {
|
||||
podTemplate(
|
||||
yaml: libraryResource('org/homelab/pod.yaml')) {
|
||||
node(POD_LABEL) {
|
||||
Codecall(file: "$stepParams.file")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def dockerBuild(Map config) {
|
||||
// container('dockerpush') {
|
||||
buildDockerGroovyGke.run(config)
|
||||
// buildDockerGroovyGke.release(config)
|
||||
// }
|
||||
}
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
|
||||
// container('tools') {
|
||||
// stage('Dockerfile creation') {
|
||||
// if (config.dockerfilePresent == "Yes"){
|
||||
// sh ''' docker build -t "${config.gcrAddress}/${config.repo_name} -f ${config.dockerFileName}" '''
|
||||
|
||||
// }
|
||||
// else{
|
||||
// def artifactId = config.artifactId
|
||||
// def version = config.version
|
||||
// def doBuild = "NO"
|
||||
// sh "ls -al"
|
||||
// echo "chmod after creating dockerfile"
|
||||
// sh "cd server ; ls"
|
||||
// // sh "cp /root/.m2/repository/com/homelab/payment/gateway/server/2.9.1/server-2.9.1.jar ."
|
||||
// def dockerfile = buildDocker.getDockerfile(config.dockerBuildVersion, config.repo_name, version, artifactId, doBuild)
|
||||
// echo "ls after creating dockerfile"
|
||||
// sh "ls "
|
||||
|
||||
// def tag = config.version
|
||||
|
||||
// sh "gcloud auth configure-docker asia-southeast1-docker.pkg.dev --quiet"
|
||||
// sh "docker build -t asia-southeast1-docker.pkg.dev/supply-poc-351106/homelab-devops/external-payment-gateway:${tag} ${dockerfile}"
|
||||
// sh "docker push asia-southeast1-docker.pkg.dev/supply-poc-351106/homelab-devops/external-payment-gateway:${tag}"
|
||||
|
||||
|
||||
|
||||
// // echo "docker build -t ${config.gcrImagePod}/${config.repo_name} ${dockerfile}"
|
||||
// // echo "docker tag ${config.gcrImagePod}/${config.repo_name} ${config.gcrAddress}/${config.gcrImagePod}/${config.repo_name}:${tag}"
|
||||
// // echo "docker push ${config.gcrAddress}/${config.gcrImagePod}/${config.repo_name}:${tag}"
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
// }
|
||||
@@ -23,8 +23,27 @@
|
||||
def call(Map config) {
|
||||
config.service_name = config.service_name ?: config.repo_name
|
||||
config.argo_app_name = config.argo_app_name ?: config.repo_name
|
||||
config.harbor_project = config.harbor_project ?: 'homelab'
|
||||
config.helm_repo_url = config.helm_repo_url ?: 'http://gitea.192.168.1.7.nip.io/mukul/devops-helm-charts.git'
|
||||
// The Harbor project images are pushed to. Named apps-registry on this
|
||||
// cluster, not the homelab's "homelab" — the project has to already
|
||||
// exist, and Harbor rejects a push to a missing one with
|
||||
// "unauthorized: project <name> not found", which reads like a
|
||||
// credentials problem rather than a missing project.
|
||||
config.harbor_project = config.harbor_project ?: 'apps-registry'
|
||||
// The registry hostname images are pushed to and pulled from. This is
|
||||
// the ONLY default for it — buildDocker.groovy deliberately errors
|
||||
// rather than carrying a second copy, since two defaults for one value
|
||||
// are free to disagree and the loser only shows up as a push to the
|
||||
// wrong registry.
|
||||
//
|
||||
// It must be spelled identically here, in the dockerconfigjson auths
|
||||
// key, and in Harbor's own externalURL: docker matches both stored
|
||||
// credentials and TLS trust by exact hostname, so a mismatch fails as
|
||||
// "unauthorized" rather than as anything resembling a name problem.
|
||||
config.harbor_registry = config.harbor_registry ?: 'harbor.infra.deployshed.com'
|
||||
// Cluster DNS, not the ingress hostname: this clone happens from a build
|
||||
// pod, so it is pod-to-pod traffic and has no business leaving the
|
||||
// cluster and coming back in through Contour.
|
||||
config.helm_repo_url = config.helm_repo_url ?: 'http://gitea-http.gitea.svc.cluster.local:3000/gitadmin/devops-helm-charts-gcp.git'
|
||||
config.image_tag_yq_path = config.image_tag_yq_path ?: '.deployment.image.tag'
|
||||
|
||||
// Every stage file (both the ones adapted for this homelab and the
|
||||
|
||||
@@ -1,115 +0,0 @@
|
||||
pipeline {
|
||||
agent {
|
||||
kubernetes {
|
||||
yamlFile "resources/org/homelab/${env.INFRA_ENV}-pod.yaml"
|
||||
defaultContainer 'devops-tools'
|
||||
}
|
||||
}
|
||||
|
||||
environment {
|
||||
GITHUB_CRED = 'svc-devops-homelab'
|
||||
}
|
||||
|
||||
parameters {
|
||||
string(name: 'repo_name', defaultValue: '', trim: true, description: 'Please enter repo name.')
|
||||
string(name: 'branch_name', defaultValue: 'main', trim: true, description: 'Please enter branch name.')
|
||||
choice(name: 'jdk_version', choices: ['jdk8', 'jdk11', 'jdk17', 'jdk21', 'jdk25'], description: 'Please select JDK Version. For adding new JDK Version, please contact DevOps Team.')
|
||||
booleanParam(name: 'skip_tests', defaultValue: true, description: 'This option will skip tests while mvn build.')
|
||||
booleanParam(name: 'sonar_scan', defaultValue: true, description: 'This option will perform static analysis of code via SonarQube.')
|
||||
booleanParam(name: 'sub_modules', defaultValue: false, description: 'This option will enable the use of Git submodules.')
|
||||
}
|
||||
|
||||
stages {
|
||||
stage('Validate Params') {
|
||||
steps {
|
||||
script {
|
||||
echo '####################### Validating Parameters ###########################'
|
||||
echo "Repo Name: ${params.repo_name}"
|
||||
echo "Branch Name: ${params.branch_name}"
|
||||
echo "JDK Version: ${params.jdk_version}"
|
||||
echo "Skip Tests: ${params.skip_tests}"
|
||||
echo "Sonar Scan: ${params.sonar_scan}"
|
||||
echo "Sub module: ${params.sub_modules}"
|
||||
}
|
||||
}
|
||||
}
|
||||
stage('Build') {
|
||||
steps {
|
||||
ansiColor('xterm') {
|
||||
script {
|
||||
sh 'whoami'
|
||||
def extra_args = ''
|
||||
if (env.INFRA_ENV == 'prd') {
|
||||
jfrog_repo = '-DuseProdRepo=true'
|
||||
}
|
||||
else {
|
||||
jfrog_repo = '-DuseTestRepo=true'
|
||||
}
|
||||
if (skip_tests.toBoolean()) {
|
||||
extra_args = '-DskipTests'
|
||||
}
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh "git clone -b ${branch_name} https://github.com/Homelab/${repo_name}.git"
|
||||
}
|
||||
if (sub_modules.toBoolean()) {
|
||||
dir("${repo_name}") {
|
||||
withCredentials([gitUsernamePassword(credentialsId: "${env.GITHUB_CRED}", gitToolName: 'git-tool')]) {
|
||||
sh(script:'git submodule init && git submodule update && git submodule update --recursive --remote')
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
def effective_jdk = params.jdk_version
|
||||
dir("${repo_name}") {
|
||||
if (fileExists('config.yaml')) {
|
||||
def config = readYaml file: 'config.yaml'
|
||||
def dv = config?.dockerBuildVersion?.toString() ?: ''
|
||||
def matcher = (dv =~ /maven-.*-jdk-(\d+)/)
|
||||
if (matcher) {
|
||||
effective_jdk = "jdk${matcher[0][1]}"
|
||||
echo "Using JDK from config.yaml dockerBuildVersion: ${dv} -> ${effective_jdk}"
|
||||
}
|
||||
}
|
||||
}
|
||||
if (effective_jdk == 'jdk25') {
|
||||
env.JAVA_HOME = '/usr/lib/jvm/java-25-openjdk-amd64/'
|
||||
}
|
||||
else if (effective_jdk == 'jdk21') {
|
||||
env.JAVA_HOME = '/usr/lib/jvm/java-21-openjdk-amd64/'
|
||||
}
|
||||
else if (effective_jdk == 'jdk17') {
|
||||
env.JAVA_HOME = '/usr/lib/jvm/java-17-openjdk-amd64/'
|
||||
}
|
||||
else if (effective_jdk == 'jdk11') {
|
||||
env.JAVA_HOME = '/usr/lib/jvm/java-11-openjdk-amd64/'
|
||||
}
|
||||
else {
|
||||
env.JAVA_HOME = '/usr/lib/jvm/java-8-openjdk-amd64/'
|
||||
}
|
||||
echo "Resolved JAVA_HOME: ${env.JAVA_HOME} (effective_jdk: ${effective_jdk})"
|
||||
dir("${repo_name}") {
|
||||
echo '####################### Building Artifacts ###########################'
|
||||
sh """
|
||||
pwd
|
||||
ls -al
|
||||
mvn -v
|
||||
mvn clean install ${extra_args}
|
||||
"""
|
||||
if (sonar_scan.toBoolean()) {
|
||||
withSonarQubeEnv('sonarqube-test') {
|
||||
echo '####################### Performing Sonar Scan ###########################'
|
||||
sh(script: "JAVA_HOME='${env.JAVA_HOME}' mvn sonar:sonar -Dsonar.branch.name=${branch_name} -Dsonar.projectName=${repo_name}")
|
||||
}
|
||||
}
|
||||
else {
|
||||
echo '####################### Skipping Sonar Scan ###########################'
|
||||
}
|
||||
echo '####################### Pushing Artifacts to Jfrog ###########################'
|
||||
sh(script: "mvn package deploy -DskipTests=${skip_tests} ${jfrog_repo}")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1,160 +0,0 @@
|
||||
// runHooks — generic per-environment CI/CD hook runner (execution core).
|
||||
//
|
||||
// Executes user-provided scripts (shell or python) that live in the service
|
||||
// repo and are declared under `environment.<env>.hooks.<phase>` in the repo's
|
||||
// config.yaml. The platform stays generic: it only locates, contextualises
|
||||
// (via TT_* env vars) and runs each script — all use-case logic lives in the
|
||||
// script itself.
|
||||
//
|
||||
// Build-phase entry (call from commonCICDFlow, repo already checked out):
|
||||
// runHooks(param, 'pre_build') // before docker build
|
||||
// runHooks(param, 'post_build') // after a successful build
|
||||
//
|
||||
// Deploy-phase re-use (from cdHookRunner, after it checks out the repo):
|
||||
// runHooks.executeHooks(repoName, hooks, phase, ctxEnv)
|
||||
//
|
||||
// By the time the build-phase entry runs, constructParam has merged the
|
||||
// resolved environment block into `param` top-level, so the current env's hooks
|
||||
// are at `param.hooks`. The repo is checked out into `${repoName}`, so scripts
|
||||
// run with CWD at the repo root and can reach any repo file.
|
||||
def call(Map param, String phase) {
|
||||
def hooksBlock = param?.hooks
|
||||
if (!(hooksBlock instanceof Map)) {
|
||||
return
|
||||
}
|
||||
def hooks = hooksBlock[phase]
|
||||
if (!(hooks instanceof List) || hooks.isEmpty()) {
|
||||
return
|
||||
}
|
||||
|
||||
String repoName = param?.repo_name
|
||||
if (!repoName) {
|
||||
error("runHooks: param.repo_name is missing — cannot locate hook scripts.")
|
||||
}
|
||||
|
||||
executeHooks(repoName, hooks, phase, buildPhaseContextEnv(repoName, phase))
|
||||
}
|
||||
|
||||
// executeHooks runs each hook spec in `hooks` inside the repo working directory,
|
||||
// injecting `ctxEnv` (the TT_* context contract) plus a per-hook TT_HOOK_NAME.
|
||||
// Reused by both the build-phase entry and the deploy-phase cdHookRunner.
|
||||
def executeHooks(String repoName, def hooks, String phase, List<String> ctxEnv) {
|
||||
hooks.eachWithIndex { hook, idx ->
|
||||
runOneHook(repoName, phase, idx, hook, ctxEnv)
|
||||
}
|
||||
}
|
||||
|
||||
// runOneHook executes a single hook spec inside the repo working directory.
|
||||
def runOneHook(String repoName, String phase, int idx, def hook, List<String> ctxEnv) {
|
||||
String name = (hook?.name ?: "${phase}-${idx}").toString()
|
||||
String script = hook?.script?.toString()
|
||||
String interpreter = hook?.interpreter?.toString()
|
||||
String requirements = hook?.requirements?.toString()
|
||||
// Blocking by default; only an explicit `blocking: false` demotes to advisory.
|
||||
boolean blocking = !(hook?.blocking?.toString() == 'false')
|
||||
int timeoutSeconds = 600
|
||||
if (hook?.timeout_seconds) {
|
||||
try { timeoutSeconds = hook.timeout_seconds.toString().toInteger() } catch (ignored) { timeoutSeconds = 600 }
|
||||
}
|
||||
|
||||
if (!script) {
|
||||
error("runHooks: ${phase}[${idx}] '${name}' has no 'script' path in config.yaml.")
|
||||
}
|
||||
assertRepoRelative(script, "${phase}[${idx}] '${name}' script")
|
||||
if (requirements) {
|
||||
assertRepoRelative(requirements, "${phase}[${idx}] '${name}' requirements")
|
||||
}
|
||||
|
||||
stage(stageName("${phase}: ${name}")) {
|
||||
dir(repoName) {
|
||||
if (!fileExists(script)) {
|
||||
error("runHooks: ${phase} hook '${name}' script not found in repo: ${script}")
|
||||
}
|
||||
|
||||
List<String> hookEnv = []
|
||||
hookEnv.addAll(ctxEnv)
|
||||
hookEnv.add("TT_HOOK_PHASE=${phase}")
|
||||
hookEnv.add("TT_HOOK_NAME=${name}")
|
||||
String runCmd = buildRunCommand(script, interpreter, requirements)
|
||||
|
||||
log.info("runHooks: executing ${phase} hook '${name}' (${script}), blocking=${blocking}, timeout=${timeoutSeconds}s")
|
||||
withEnv(hookEnv) {
|
||||
try {
|
||||
timeout(time: timeoutSeconds, unit: 'SECONDS') {
|
||||
sh(script: runCmd)
|
||||
}
|
||||
} catch (Exception e) {
|
||||
if (blocking) {
|
||||
log.error("runHooks: blocking ${phase} hook '${name}' failed: ${e}")
|
||||
throw e
|
||||
}
|
||||
// Advisory hook — record and continue without failing the build.
|
||||
log.warning("runHooks: advisory ${phase} hook '${name}' failed (non-blocking): ${e}")
|
||||
currentBuild.description = (currentBuild.description ? currentBuild.description + " | " : "") + "hook(${name}) advisory-failed"
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// buildRunCommand resolves the interpreter (explicit > extension > shebang) and,
|
||||
// when a python requirements file is given, provisions an ephemeral venv.
|
||||
def buildRunCommand(String script, String interpreter, String requirements) {
|
||||
String interp = interpreter
|
||||
if (!interp) {
|
||||
if (script.endsWith('.py')) {
|
||||
interp = 'python3'
|
||||
} else if (script.endsWith('.sh')) {
|
||||
interp = 'bash'
|
||||
}
|
||||
}
|
||||
|
||||
boolean isPython = (interp == 'python3' || interp == 'python' || script.endsWith('.py'))
|
||||
if (requirements && isPython) {
|
||||
String py = interp ?: 'python3'
|
||||
return """
|
||||
set -e
|
||||
${py} -m venv .tt_hook_venv
|
||||
. .tt_hook_venv/bin/activate
|
||||
pip install --quiet --disable-pip-version-check -r ${requirements}
|
||||
${py} ${script}
|
||||
""".stripIndent().trim()
|
||||
}
|
||||
|
||||
if (interp) {
|
||||
return "set -e\n${interp} ${script}"
|
||||
}
|
||||
// No interpreter resolved — rely on the script's shebang.
|
||||
return "set -e\nchmod +x ${script}\n./${script}"
|
||||
}
|
||||
|
||||
// buildPhaseContextEnv builds the standard TT_* env-var contract for build-phase
|
||||
// hooks from the pipeline env already populated by constructParam / the GitHub
|
||||
// Branch Source plugin.
|
||||
def buildPhaseContextEnv(String repoName, String phase) {
|
||||
boolean isPR = (env.CHANGE_ID ? true : false)
|
||||
String branch = isPR ? (env.CHANGE_BRANCH ?: env.BRANCH_NAME ?: '') : (env.BRANCH_NAME ?: '')
|
||||
boolean isHotfix = (env.hot_fix == 'true' || env.hot_fix == true)
|
||||
return [
|
||||
"TT_REPO_NAME=${repoName}",
|
||||
"TT_ENV=${env.cicd_environment ?: ''}",
|
||||
"TT_EVENT=${isPR ? 'pr' : 'push'}",
|
||||
"TT_IS_HOTFIX=${isHotfix}",
|
||||
"TT_BRANCH=${branch}",
|
||||
"TT_TARGET_BRANCH=${env.CHANGE_TARGET ?: ''}",
|
||||
"TT_PR_NUMBER=${env.CHANGE_ID ?: ''}",
|
||||
"TT_COMMIT_SHA=${env.GIT_COMMIT ?: ''}",
|
||||
"TT_IMAGE_TAG=${env.image_tag ?: ''}",
|
||||
]
|
||||
}
|
||||
|
||||
// assertRepoRelative rejects absolute paths and parent-directory traversal so a
|
||||
// hook can only execute code that lives inside the checked-out repo.
|
||||
def assertRepoRelative(String p, String what) {
|
||||
if (p.startsWith('/')) {
|
||||
error("runHooks: ${what} path '${p}' must be repo-relative, not absolute.")
|
||||
}
|
||||
if (p == '..' || p.startsWith('../') || p.contains('/../')) {
|
||||
error("runHooks: ${what} path '${p}' must not traverse outside the repo ('..').")
|
||||
}
|
||||
}
|
||||
@@ -1,92 +0,0 @@
|
||||
// Best-effort fire of the standalone ai-blitz-jobs (coverage-only) Jenkins
|
||||
// job after a successful per-repo CI build. ai-blitz-jobs runs the
|
||||
// coverage-only.Jenkinsfile from devops-lib@coverage-only-pipeline against
|
||||
// the repo+branch we just built, so coverage and test-quality metrics flow
|
||||
// to novaviz without anyone having to trigger the job manually.
|
||||
//
|
||||
// Gates — skip when triggering would waste compute or pollute the data:
|
||||
// - PR builds (CHANGE_ID set; coverage on PRs is noise)
|
||||
// - hot-fix builds (env.hot_fix == true)
|
||||
// - toolchain image rebuilds (env.INFRA_ENV == 'toolchain')
|
||||
// - explicit opt-out (skip_coverage_trigger: true in config.yaml)
|
||||
// - non-develop branches (current policy: ONLY 'develop' fires. main /
|
||||
// master / gcp-main / gcp-master are
|
||||
// deploy-only branches at Homelab — tests
|
||||
// already ran on the develop merge that
|
||||
// produced their content, so re-running
|
||||
// coverage on them would be duplicate
|
||||
// spend. Expand this allowlist if a repo
|
||||
// ships from a non-develop branch.)
|
||||
//
|
||||
// Failure is swallowed — observability must never block prod CI. The
|
||||
// ai-blitz-jobs Jenkins job being renamed/disabled/missing logs a warning
|
||||
// and continues; the parent build stays green.
|
||||
def call(Map param) {
|
||||
if (env.CHANGE_ID) {
|
||||
log.info("triggerCoverageOnly: skipping PR build (CHANGE_ID=${env.CHANGE_ID}).")
|
||||
return
|
||||
}
|
||||
if (env.hot_fix == 'true' || env.hot_fix == true) {
|
||||
log.info("triggerCoverageOnly: skipping hot-fix build.")
|
||||
return
|
||||
}
|
||||
if ((env.INFRA_ENV ?: '') == 'toolchain') {
|
||||
log.info("triggerCoverageOnly: skipping toolchain (image-rebuild) build.")
|
||||
return
|
||||
}
|
||||
if (param?.skip_coverage_trigger?.toString() == 'true') {
|
||||
log.info("triggerCoverageOnly: skipping — repo opted out via config.yaml skip_coverage_trigger=true.")
|
||||
return
|
||||
}
|
||||
|
||||
String branch = (env.BRANCH_NAME ?: '').trim()
|
||||
Set<String> allowed = ['develop']
|
||||
if (!allowed.contains(branch)) {
|
||||
log.info("triggerCoverageOnly: skipping branch '${branch}' (only ${allowed} triggers coverage today).")
|
||||
return
|
||||
}
|
||||
|
||||
String repoName = param?.repo_name
|
||||
if (!repoName) {
|
||||
log.warning("triggerCoverageOnly: param.repo_name missing — cannot construct REPO_URL. Skipping.")
|
||||
return
|
||||
}
|
||||
// Homelab convention: every backend service lives at github.com/Homelab/<repo>.
|
||||
// ai-blitz-jobs' coverage-only.Jenkinsfile takes the full URL as REPO_URL.
|
||||
String repoUrl = "https://github.com/Homelab/${repoName}"
|
||||
|
||||
// Absolute path. Per-repo CI jobs may live inside Jenkins folders
|
||||
// (e.g. /Homelab/order-service); a relative 'ai-blitz-jobs' would try
|
||||
// the parent folder first and 404 there. The leading slash anchors at
|
||||
// the Jenkins root, where ai-blitz-jobs lives (confirmed by the URL
|
||||
// shape http://jenkins-dev.../job/ai-blitz-jobs/<N>/).
|
||||
String targetJob = '/ai-blitz-jobs'
|
||||
|
||||
try {
|
||||
log.info("triggerCoverageOnly: launching ${targetJob} for ${repoName} @ ${branch} (fire-and-forget).")
|
||||
// The `build` step is in-process Jenkins RPC — no API tokens or
|
||||
// credentials needed. It runs with the parent build's identity
|
||||
// (UpstreamCause), and ai-blitz-jobs' coverage-only.Jenkinsfile
|
||||
// calls coverageOnly() directly (NOT eksCICD), so the
|
||||
// allowedUsers gate in eksCICD doesn't apply to this path. Bot
|
||||
// and upstream-triggered builds both work.
|
||||
// wait:false → parent build queues the downstream and moves on
|
||||
// immediately. It never reads the downstream result, so `propagate`
|
||||
// is a no-op here and intentionally omitted. The try/catch below is
|
||||
// what shields the parent from step-level errors (job not found,
|
||||
// bad params, queue full).
|
||||
build(
|
||||
job: targetJob,
|
||||
parameters: [
|
||||
string(name: 'REPO_URL', value: repoUrl),
|
||||
string(name: 'BRANCH', value: branch)
|
||||
],
|
||||
wait: false,
|
||||
quietPeriod: 0
|
||||
)
|
||||
} catch (Exception e) {
|
||||
// ai-blitz-jobs renamed/disabled/queue-blocked/whatever — log and
|
||||
// continue. We never block the parent build on observability.
|
||||
log.warning("triggerCoverageOnly: build(job: '${targetJob}', …) threw: ${e}. Continuing — parent CI stays green.")
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user