We have a Jenkins instance that is running jobs in Docker containers on its host.
Eventually, we’ve faced with an issue when the current AWS Ec2 instance t2.2xlarge (8 CPU, 32 RAM) during peak workload periods was too overloaded — not enough CPU time, not enough memory.
So, the first solution could be to proceed with its vertical scaling for example up to c5.9large, and proceed running builds on the master-host, or by moving some jobs to external workers.
At this time, we have three such workers — Android-builds are running on a PC in our office with Android studio…
We have a PHP application running with Kubernetes in pods with two dedicated containers — NGINX и PHP-FPM.
The problem is that during downscaling clients get 502 errors. E.g. when a pod is stopping, its containers can not correctly close existing connections.
So, in this post, we will take a closer look at the pods’ termination process in general, and NGINX and PHP-FPM containers in particular.
Testing will be performed on the AWS Elastic Kubernetes Service by the Yandex.Tank utility.
Ingress resource will create an AWS Application Load Balancer with the AWS ALB Ingress Controller.
Для управления контейнерами на Kubernetes…
It’s used by our QA team and now it’s time for me to take a closer look on it to test one issue with our application running on a Kubernetes cluster.
In this post a short overview of its capabilities and configuration.
In contrast to the Apache Bench, Yandex.Tank displays response codes statistics and is much more simple in running and configuration the JMeter, plus it has a nice Autostop feature for a case when “Huston, we have a problem” (с)
The Yandex Tank core…
Locales is a set of environment variables that are used to determine how to display data and time (for example, first of the week), symbols encoding (for example, how to display cyrillic symbols), default files order when one executing the
ls command, and so on.
Those variables are:
LANG: Determines the default locale in the absence of other locale related environment variables
LANGUAGE: List of fallback message translation languages
LC_CTYPE: Character classification and case conversion
LC_NUMERIC: Numeric formatting
LC_TIME: Date and time formats
LC_COLLATE: Collation (sort) order
LC_MONETARY: Monetary formatting
LC_MESSAGES: Format of interactive words and responses
LC_PAPER: Default paper…
The problem we faced is that those pods are consuming too much CPU — up to 3000 millicpu, while our WorkerNodes has only 4 cores, e.g. 4000 millicpu.
So, to solve this issue I’ve decided to search for similar log collectors and the second thing to do is was to able to deploy them with…
Active alerts sending frequency via Alertmanager is configured via the
repeat_interval in the
We have this interval set to 15 minutes, and as result, we have notifications about alerts in our Slack each fifteen minutes.
Still, some alerts are such a “known issues”, when we already started the investigation or fixing it, but the alert is repeatedly sent to Slack.
To mute those alerts to prevent them to be sent over and over they can be disabled by marking them as “silenced”.
An alert can be silenced with the Web UI of the Alertmanager, see the documentation.
In the previous post ArgoCD: an overview, SSL configuration, and an application deploy we did a quick overview on how to work with the ArgoCD in general, and now let’s try to deploy a Helm chart.
The most interesting part of this is how to enable the Helm Secrets. Had some pain with this, but finally, it’s working as expected.
ArgoCD helps to deliver applications to Kubernetes by using the GitOps approach, i.e. when a Git-repository is used as a source of trust, thus all manifest, configs and other data are stored in a repository.
ArgoCD spins up its controller in the cluster and watches for changes in a repository to compare it with resources deployed in the cluster, synchronizing their states.
For the authentification and authorization, Kubernetes has such notions as User Accounts and Service Accounts.
User Accounts — common user profiles used to access a cluster from the outside, while Service Accounts are used to grant access from inside of the cluster.
ServiceAccounts are intended to provide an identity for a Kubernetes Pod to be used by its container to authenticate and authorize them when performing API-requests to the Kubernetes API-server.
Every Kubernetes Namespace has its own default ServiceAccount…
We’d like to have the ability to add a DNS-record on the AWS Route53 when a Kubernetes Ingress resource is deployed and point this record to the URL of an AWS Load Balancer which is created by the ALB Ingress controller.
To achieve this, the ExternalDNS can be used which will make API-requests to the AWS Route53 to add appropriate records.
AWS installation is described in its documentation>>>.
First, need to create an IAM policy. For the…