Details on Tales Of Details

AI Insights

stelis@systematize.it (stelis) — Thu, 16 Apr 2026 00:00:00 +0000

Sure, Claude Code is nothing short of magic… until you run out of tokens :-P
But here’s a thought I found interesting.

By now, we know that given a specific deliverable, your chances of getting it from the LLM increase if

you provide as many details as you can about what you need done and how you’d prefer to have it done
you’re in a position to evaluate the output and ask for amendments
actually take the time to evaluate the output and ask for amendments

Well, as it turns out

Mid-term alignment

stelis@systematize.it (stelis) — Wed, 28 Jan 2026 00:00:00 +0000

Selfishness is usually frowned upon.

Largely because it’s taken to mean looking after my own interests to the detriment of others.
I find this definition a bit too simplistic.

All relationships, all interactions contain a transaction.
While it’s not at all romantic to voice as much ( ask me how I know :-P ), there is something to gain in everything we do.

Maintaining a friendship because you get a sympathetic ear.
Saying hi to your neighbor so it’s not awkward when you ask them for some coffee.

Sometimes the motive is evident and hurts others (so it’s labeled selfish).
Sometimes it’s more subtle and it avoids detection by this crude filter.

My best idea yet, and I came a distant second

stelis@systematize.it (stelis) — Mon, 26 Jan 2026 00:00:00 +0000

I’ve been using traefik in various contexts over the years.
It’s a bit much at first glance but once you get to know it, it can do anything ( at least anything I’ve thrown at it) and it’s extensible via plugins.
The winning feature it had when first introduced was letsencrypt support.
You could have the ingress controller manage your certificates and inject them in any ingress you instructed it to. I’ve been using that feature for a long time.

Intuition is great... and dangerous

stelis@systematize.it (stelis) — Tue, 20 Jan 2026 00:00:00 +0000

In my formative years, I would try to apply logic to any and all matters.

How do I decide what to do?
Why should I do this?
What does this impact? etc etc.

Any kind of intuition of predisposition seemed dangerous as I had no way of knowing why I leaning that way (biases, misinformation etc)

This analytical process was lengthy and exhausting but I like to think it helped shape a way of thinking and form a (mostly) reliable intuition.
I can still remember the exhilarating feeling of hearing a question, leaning one way about it, then analyzing it and discovering I still liked my original inclination.
I could now trust my gut and stop the (over)analysis, right?

Cache-Control

stelis@systematize.it (stelis) — Sun, 28 Dec 2025 00:00:00 +0000

There are two main problems in programming:

Invalidating Cache
Naming things
Off-by-one errors

Let’s do cache in the web today :-)

Suppose you have a static frontend that you need to serve.
And you care about having globally available with low latency.
That’s the prime use case for a CDN, right?
So you deploy a CDN ( Cloudflare, Cloudfront etc).
And you point it to a source ( an S3 bucket with your static files or a publicly accessible web-server serving your files).

Certificates, revisited

stelis@systematize.it (stelis) — Sat, 13 Dec 2025 00:00:00 +0000

In the never-ending quest for env parity, I ran into another interesting issue.
mkcert does not deal with client certificates, it only creates server certs.
What if we want to do mutual TLS and need them?
Well, buckle up, we’ll take openssl for a ride.
We’ll need the three files shown below in a folder named ca and the scripts that follow.

# ca/ca.cnf
[ policy_match ]
countryName = match
stateOrProvinceName = match
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional

[ req ]
prompt = no
distinguished_name = dn
default_md = sha256
default_bits = 4096
x509_extensions = v3_ca

[ dn ]
countryName = GR
organizationName = MyCompany
localityName = Athens
commonName = mycompany-ca

[ v3_ca ]
subjectKeyIdentifier=hash
basicConstraints = critical,CA:true
authorityKeyIdentifier=keyid:always,issuer:always
keyUsage = critical,keyCertSign,cRLSign

# ca/client.cnf
[req]
prompt = no
distinguished_name = dn
default_md = sha256
default_bits = 4096
req_extensions = v3_req

[ dn ]
countryName = GR
organizationName = MyCompnay
localityName = Athens
commonName = mycompany-ca
commonName=*.random.k8s

[ v3_ca ]
subjectKeyIdentifier=hash
basicConstraints = critical,CA:true
authorityKeyIdentifier=keyid:always,issuer:always
keyUsage = critical,keyCertSign,cRLSign

[ v3_req ]
subjectKeyIdentifier = hash
basicConstraints = CA:FALSE
nsComment = "OpenSSL Generated Certificate"
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = clientAuth
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = *.random.k8s
DNS.2 = random.k8s
DNS.3 = localhost
DNS.4 = 127.0.0.1
DNS.5 = myservice
DNS.6 = myotherservice
DNS.7 = *.infra
DNS.8 = *.app
DNS.9 = *.somedomain.iwantotouse

# ca/server.cnf
[req]
prompt = no
distinguished_name = dn
default_md = sha256
default_bits = 4096
req_extensions = v3_req

[ dn ]
countryName = GR
organizationName = mycompany
localityName = Athens
commonName = mycompany-ca
commonName=*.random.k8s

[ v3_ca ]
subjectKeyIdentifier=hash
basicConstraints = critical,CA:true
authorityKeyIdentifier=keyid:always,issuer:always
keyUsage = critical,keyCertSign,cRLSign

[ v3_req ]
subjectKeyIdentifier = hash
basicConstraints = CA:FALSE
nsComment = "OpenSSL Generated Certificate"
keyUsage = critical, digitalSignature, keyEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names

[ alt_names ]
DNS.1 = *.random.k8s
DNS.2 = random.k8s
DNS.3 = localhost
DNS.4 = 127.0.0.1
DNS.5 = myservice
DNS.6 = myotherservice
DNS.7 = *.infra
DNS.8 = *.app
DNS.9 = *.somedomain.iwantotouse

function create_certs() {
  local CERTS_DIR="${CERTS_DIR:-"/opt/my_certs"}"
  local STORE_PASS="${STORE_PASS:-changeit}"

  # uninstall any existing CAs handled by mkcert
  mkcert -uninstall

  # then delete the CA files
  # and existing client/server certs
  rm -rf $(mkcert -CAROOT)/*
  rm -rf ${CERTS_DIR}/*

  # create new ca
  openssl req -new -nodes \
    -x509 \
    -days 3650 \
    -newkey rsa:2048 \
    -keyout "${CERTS_DIR}"/ca.key \
    -out "${CERTS_DIR}"/ca.crt \
    -config ./ca/ca.cnf

  # create ca pem from crt and key
  cat "${CERTS_DIR}"/ca.crt "${CERTS_DIR}"/ca.key > "${CERTS_DIR}"/ca.pem

  # create ca p12 from existing java trust store
  docker run --rm -it -v "${CERTS_DIR}":/certs  \
    eclipse-temurin:17-jdk -- keytool -importkeystore \
   -srckeystore/opt/java/openjdk/lib/security/cacerts \
   -srcstoretype PKCS12 \
   -srcstorepass changeit \
   -destkeystore /certs/ca.p12  \
   -deststoretype PKCS12 \
   -deststorepass "${STORE_PASS}"

  # change files to have proper user permissions
  sudo chown $(id -u):$(id -g) "${CERTS_DIR}"/ca.p12

  # add our new ca to newly created store
  docker run --rm -it -v ${CERTS_DIR}:/certs  \
    eclipse-temurin:17-jdk \
           -- keytool -importcert -alias my_ca_cert \
                      -file /certs/ca.pem \
                      -keystore /certs/ca.p12 \
                      -deststorepass "${STORE_PASS}" \
                      -noprompt

  # create key and csr for server EKU
  openssl req -new \
      -newkey rsa:2048 \
      -keyout "${CERTS_DIR}"/server.key \
      -out "${CERTS_DIR}"/server.csr \
      -config ./ca/server.cnf \
      -nodes

  # sign the server csr and create cert
  openssl x509 -req \
      -days 3650 \
      -in "${CERTS_DIR}"/server.csr \
      -CA "${CERTS_DIR}"/ca.crt \
      -CAkey "${CERTS_DIR}"/ca.key \
      -CAcreateserial \
      -out "${CERTS_DIR}"/server.crt \
      -extfile ./ca/server.cnf \
      -extensions v3_req

  # create server pem from crt and key
  cat "${CERTS_DIR}"/server.crt "${CERTS_DIR}"/server.key > "${CERTS_DIR}"/server.pem

  # create server p12 keystore
  openssl pkcs12 -export \
      -in "${CERTS_DIR}"/server.crt \
      -inkey "${CERTS_DIR}"/server.key \
      -chain \
      -CAfile "${CERTS_DIR}"/ca.pem \
      -name server \
      -out "${CERTS_DIR}"/server.p12 \
      -password pass:"${STORE_PASS}"

  # change files to have proper user permissions
  sudo chown $(id -u):$(id -g) "${CERTS_DIR}"/server.p12

  # import server p12 to store if needed
  docker run --rm -it -v ${CERTS_DIR}:/certs  \
    eclipse-temurin:17-jdk -- keytool \
         -importcert -alias server_cert \
         -file /certs/server.pem \
         -keystore /certs/ca.p12 \
         -deststorepass "${STORE_PASS}" \
         -noprompt

  # create key and csr for client EKU
  openssl req -new \
      -newkey rsa:2048 \
      -keyout "${CERTS_DIR}"/client.key \
      -out "${CERTS_DIR}"/client.csr \
      -config ./ca/client.cnf \
      -nodes

  # sign the client csr and create cert
  openssl x509 -req \
      -days 3650 \
      -in "${CERTS_DIR}"/client.csr \
      -CA "${CERTS_DIR}"/ca.crt \
      -CAkey "${CERTS_DIR}"/ca.key \
      -CAcreateserial \
      -out "${CERTS_DIR}"/client.crt \
      -extfile ./ca/client.cnf \
      -extensions v3_req

  # create client pem from crt and key
  cat "${CERTS_DIR}"/client.crt "${CERTS_DIR}"/client.key > "${CERTS_DIR}"/client.pem

  # create client p12 keystore
  openssl pkcs12 -export \
      -in "${CERTS_DIR}"/client.crt \
      -inkey "${CERTS_DIR}"/client.key \
      -chain \
      -CAfile "${CERTS_DIR}"/ca.pem \
      -name client \
      -out "${CERTS_DIR}"/client.p12 \
      -password pass:"${STORE_PASS}"

  # change files created to have proper user permissions
  sudo chown $(id -u):$(id -g) "${CERTS_DIR}"/client.p12

  # import client p12 to store if needed
  docker run --rm -it -v ${PWD}:/certs \
         eclipse-temurin:17-jdk \
           -- keytool -importcert -alias client_cert \
           -file /certs/client.pem \
           -keystore /certs/ca.p12 \
           -deststorepass "${STORE_PASS}" \
           -noprompt

  # list certs in store to verify everything imported
  docker run --rm -it -v ${PWD}:/certs  \
         eclipse-temurin:17-jdk \
           -- keytool -list -v -keystore /certs/ca.p12 \
           -storepass "${STORE_PASS}"

  # cleanup files we don't need
  rm -rf "${CERTS_DIR}"/client.csr
  rm -rf "${CERTS_DIR}"/server.csr

  # copy ca files to mkcert folder
  cp "${CERTS_DIR}"/ca.key $(mkcert -CAROOT)/rootCA-key.pem
  cp "${CERTS_DIR}"/ca.crt $(mkcert -CAROOT)/rootCA.pem
  cp "${CERTS_DIR}"/ca.srl $(mkcert -CAROOT)/rootCA.srl

  # trust the ca for your system
  mkcert -install

  # concatenate server cert + ca.cert to use for ingress
  cat ${CERTS_DIR}/server.crt ${CERTS_DIR}/ca.crt > ${CERTS_DIR}/full.pem

}

function create_tls_secrets() {
  kubectl create namespace traefik
  kubectl create namespace infra
  kubectl create namespace app

  # create ingress certificate
  kubectl -n traefik create secret tls my-root-ca-tls \
          --key ${CERTS_DIR}/server.key  \
          --cert ${CERTS_DIR}/server.crt \
          --dry-run=client -o yaml \
          | kubectl apply -f -

  # create secret for plain certificates in infra + app namespaces
  kubectl -n infra create secret generic certificates \
        --from-file=ca.crt=${CERTS_DIR}/ca.crt \
        --from-file=server.crt=${CERTS_DIR}/server.crt \
        --from-file=server.key=${CERTS_DIR}/server.key \
        --from-file=client.crt=${CERTS_DIR}/client.crt \
        --from-file=client.key=${CERTS_DIR}/client.key \
        --dry-run=client -o yaml \
        | kubectl apply -f -

  kubectl -n app create secret generic certificates \
        --from-file=ca.crt=${CERTS_DIR}/ca.crt \
        --from-file=server.crt=${CERTS_DIR}/server.crt \
        --from-file=server.key=${CERTS_DIR}/server.key \
        --from-file=client.crt=${CERTS_DIR}/client.crt \
        --from-file=client.key=${CERTS_DIR}/client.key \
        --dry-run=client -o yaml \
        | kubectl apply -f -

  # create secret for keystores in infra + app namespaces
  kubectl -n infra create secret generic keystores \
        --from-file=ca.p12=${CERTS_DIR}/ca.p12\
        --from-file=server.p12=${CERTS_DIR}/server.p12 \
        --from-file=client.p12=${CERTS_DIR}/client.p12 \
        --from-literal=password="${STORE_PASS}" \
        --dry-run=client -o yaml \
        | kubectl apply -f -

  kubectl -n app create secret generic keystores \
        --from-file=ca.p12=${CERTS_DIR}/ca.p12\
        --from-file=server.p12=${CERTS_DIR}/server.p12 \
        --from-file=client.p12=${CERTS_DIR}/client.p12 \
        --from-literal=password="${STORE_PASS}" \
        --dry-run=client -o yaml \
        | kubectl apply -f -
}

Now we’ve got 2 secrets per namespace.
One contains certificates, the other keystores.
We can mount them to our apps as needed to be used for mutual TLS.

Dev Ready Bare Metal k8s

stelis@systematize.it (stelis) — Sat, 13 Dec 2025 00:00:00 +0000

We’re building a shared development environment in k8s
The typical cloud managed k8s services are not available.
We need ample resources and low-as-possible price for k8s.

Requirements? We have them

Private access. Given private access to the machine’s local IP address, it needs to be reachable and publish services to that interface.
Public Access. Given a public IP assigned to its second interface, we need to be able to selectively publish services to the internet.
Data is not important. We care about them as far as they are needed for the services to keep running properly.

Wanna try a Bare Metal server for this?

If my java memory serves me well...

stelis@systematize.it (stelis) — Sat, 13 Dec 2025 00:00:00 +0000

You’ve got a backend in Java deployed in k8s and every once in a while it OOMs.
What a surprise :-P

How do you go about fixing this?
Well, give it more memory since it’s out!
About half the time this will help.
But in case you’re not lucky and you need to fix the underlying issue, let’s take the engineering way around.
It’s full of twists, turns and wasted time.
But if you need it, it’s indispensable.

You're allowed to not know everything

stelis@systematize.it (stelis) — Sat, 13 Dec 2025 00:00:00 +0000

Another not, strictly, technical post.
But one that’s very dear to me and has been instrumental to my development as a person and a professional.

The IT sector is a hard skills sector with lots of depth.
The more you know on your subject, the more valuable you are, the more respect you can get from your peers and compensation from your employer.

But what once might have been a difficult quest ( know everything in my field) is now practically impossible.
You cannot know everything, you should not be trying to know everything and you should not be expecting anyone else to know everything either.
Even if you define everything as everything within my specific role or everything about a specific technology or tool, this is still a fool’s errand.

Imposter Syndrome

stelis@systematize.it (stelis) — Sun, 09 Nov 2025 00:00:00 +0000

Wanted to save this quote somewhere visible.

Impostor syndrome doesn’t mean you’re unqualified.
It means you’re growing faster than your confidence can catch up.

Sourced here, from engineering career coach Taha Hussain

The Triangle of Conscience

stelis@systematize.it (stelis) — Fri, 07 Nov 2025 00:00:00 +0000

Conscience is like a triangle in your heart.

When you do something wrong it spins and hurts you a bit.

If it spins for too long, its edges fade and it stops hurting.

( This metaphor is supposedly inspired by a Navajo teaching. I could not find reputable sources about the origin or the usage of the .metaphor. But the idea seems potent enough to stand on its own.)

Extreme Env Parity

stelis@systematize.it (stelis) — Fri, 13 Jun 2025 00:00:00 +0000

You need/want a local k8s environment to develop your container images, helm charts etc.
And you’re pedantic about env parity. Your envs should differ only where they absolutely need to.
So that you can re-use your manifests/charts as usual and when you eventually push to a shared env, you do so with confidence.

You don’t expect issues with the workloads. You can build or pull the containers.
Persistence may be an issue but your local k8s solution should already have a storage class for you to use.

Cheap Thrills

stelis@systematize.it (stelis) — Sat, 19 Apr 2025 00:00:00 +0000

Consider the case of a hobby project or a startup with no funding or clients for the foreseeable future.
We need dirt cheap infra for development and demos.

What’s the lowest price for:

somewhere I can run a couple of containers of my services
a db to save my data, managed if possible
a Load Balancer to expose my app to the world
DNS so I can slap a name on the thing.

AWS Free Tier (or equivalent) is always on the table.
But then you need to be vigilant of usage and attentive to details (and who’s got time for the details, right? :-P)
A simple VPC + subnets + NATs in AWS ends up costing somewhere near 15$ / month when accounting for Public IPs and traffic.
And then we factor in our apps’ needs.

Empathy for Engineers

stelis@systematize.it (stelis) — Sat, 15 Mar 2025 00:00:00 +0000

I’ve been in a couple of leadership seminars. One of the concepts that arose as central for good leadership was empathy.

But what’s empathy? Let’s use this definition.

…the ability to sense other people’s emotions, coupled with the ability to imagine what someone else might be thinking or feeling.

If someone says I'm sad and I hear that, am I now empathetic towards them?
Empathy is not superficial, but an in-depth understanding of both the state (sad) and its underlying causes, ideally coupled with the ability to predict future behavior.

Was this piece of code successful?

stelis@systematize.it (stelis) — Fri, 04 Oct 2024 00:00:00 +0000

When it comes to a unit of code ( let’s define it as a project, service, function, script etc that performs some task) how do you define success?

An easy answer would be It fulfils its primary purpose
Each unit’s purpose, of course, differs.
But, excluding learning projects, PoCs and similar endeavors, most of our work is commissioned.
And it usually includes support and maintenance, right?

Most of us, hopefully, enjoy the building part but merely tolerate the support/maintenance part.

Managed Services Woes, RabbitMQ Edition

stelis@systematize.it (stelis) — Sat, 28 Sep 2024 00:00:00 +0000

I love managed services. You need a message broker, I’ll have one ready for you in 15’.
We trade some money for ease of use and focus on the core deliverable, great.

But then we need a simple change. Make the message queue’s endpoint public.
If you’re using AWS MQ RabbitMQ and had created a private instance, then tried to flick a switch to make it public…
Let’s start a support group, I’ll bring the cookies and frustration :-P

macos setup as code

stelis@systematize.it (stelis) — Sat, 21 Sep 2024 00:00:00 +0000

I try to keep track of everything that goes into my work machine.
I, of course, start missing things after a couple of months.
But the value is in trying, right? Let’s hope so.

In any case, I recently setup a macos machine from scratch. It was intended for devops/infra work ( k8s, terraform, aws etc).
It also includes java + node as I’ll occasionally need to build the apps before running them.

ConfigMap Confusion

stelis@systematize.it (stelis) — Mon, 16 Sep 2024 00:00:00 +0000

This has happened to me more than a few times…

We’re mounting a ConfigMap to our container and using it to configure some aspect of the app inside it.
Then the ConfigMap changes.
But the app is unaware and does not react/reconfigure itself. Which component is at fault here and how can we fix it?

Until this bit me, I was under the impression that Configmaps mounted in containers do NOT auto-update their contents in the container.
At least not without restarting the container itself.
There are even projects that undertake parts of this responsibility.
Like Reloader or SpringBoot Config Watcher.

My most used bash functions

stelis@systematize.it (stelis) — Sun, 15 Sep 2024 00:00:00 +0000

There’s problems you’d like to solve.
And problems you like solving.

What you put in the second category is almost uncomfortably revealing of who you are.
Anything you consider trivial fits right into the first category and is a candidate for automation.

Here are some of these automations I bring with me to all new laptops I work on.

get a random string of variable length

function randpw() {
 # e.g randpw 20
  local pwlength="${1}"
  cat /dev/urandom | tr -dc 'a-zA-Z0-9!@#$%^&*()-[]{}:;\|,.<>/?' | fold -w "${pwlength:-50}" | head -n 1
}

full update for ubuntu distros

function fu() {
    sudo apt update
    sudo apt -y upgrade
    sudo apt -y dist-upgrade
    sudo apt -y autoremove
    sudo ubuntu-drivers install
    brew update
    brew upgrade
}

update all repos in your code directory

function getallprojects() {
  find "${YOUR_PROJECTS_DIR}" -maxdepth 1 \
            -mindepth 1 \
            -type d \
            -printf '%f\n' \
            | grep -v .vscode
}
function getdefaultbranch() {
  git remote show origin | grep 'HEAD branch' | cut -d' ' -f5
}
function refreshprojects() {

  local LIGHTGREEN="\e[92m"
  local ENDCOLOR="\e[0m"

  echo "---------------"
  for project in "$(getallprojects)";
  do
    echo -e "Refreshing ${LIGHTGREEN}${project}${ENDCOLOR}"
    pushd "${project}" > /dev/null || exit

    git fetch --prune
    git checkout "$(getdefaultbranch)"
    git pull
    popd > /dev/null || exit
    echo "---------------"
  done
}

hack to get a urlencoded string

function urlenc() {
  local str_to_encode="${1}"
  encoded_str=$(python3 -c "import urllib.parse; print(urllib.parse.quote('''${str_to_encode}'''))")
  echo "${encoded_str}"
  # why did I need it?
  # verify connectivity to rabbitmq, password had special characters
  # rabbitmq-dump-queue -uri="amqps://mq_user:$(urlenc "${mq_password}")@my.mq.endpoint.tld:5671/" -queue=myqueue -max-messages=1 -output-dir=./
}

Is it worth the time? It depends

Can I get Free Email for my custom Domain, please?

stelis@systematize.it (stelis) — Sat, 14 Sep 2024 00:00:00 +0000

At some point you bought a domain. You made something of it, or (usually) not. Now you’d like an email address on that domain.
It looks much nicer than your typical xyz@gmail.com, right?

Especially if you’re a business, Google Workspace or Microsoft 365 is a no-brainer.
You get your custom domain email, storage, collaboration apps, SSO and other tools for something like 5-10$/person/month.

But if you’re like me (i.e you like the vanity of the thing but will not allow yourself the expense unless you know you’ll actually use it or it’ll make its money back somehow) you may be looking for something without monthly subscriptions, preferably free.

Get the env vars your app started with

stelis@systematize.it (stelis) — Sat, 14 Sep 2024 00:00:00 +0000

The setup should be familiar. You have an app that is ready to ship, so you put it in a docker image.
Any config after that is done mostly via env vars.

For anything non-trivial like e.g - mounting and parsing secrets to env vars - making sure certain default values are overriden

you need an init script that calculates the var’s desired value, then exports it like

export MY_CONFIG_VAR=calculated_value

And once done, you’d start the app with your equivalent of

Meet your Heroes!

stelis@systematize.it (stelis) — Tue, 10 Sep 2024 00:00:00 +0000

There’s this quote that I hate

Never meet your heroes

The assumption being that our heroes should be seated on pedestals and admired from afar.
Meeting them will uncover their human flaws and thus they’ll lose their glory and primary function.

That’s why I love Colossus’ quote from the first Deadpool movie

Everyone thinks it’s a full-time job. Wake up a hero. Brush your teeth a hero. Go to work a hero. Not true. Over a lifetime, there are only four or five moments that really matter. Moments when you’re offered a choice - to make a sacrifice, conquer a flaw, save a friend, spare an enemy. In these moments, everything else falls away.

Linguistic Details

stelis@systematize.it (stelis) — Sun, 08 Sep 2024 00:00:00 +0000

Language is a terrific tool to get your message across.
But it exactly that, a tool.
Everyone uses it for their purposes.
And if it succeeds we should be happy.
After all, conveying a nuanced message is hard enough in itself.

This is why I’ve stopped being pedantic about small errors in syntax or grammar, unless the sender has explicitly asked for comments on that.

On the other hand, it’s almost magical when you notice a skilled speaker masterfully choosing the correct word to relay their message.
In just as many words, they’ve communicated volumes more.
But to understand and appreciate that you need to be prepared.

Port-forward non k8s service via k8s

stelis@systematize.it (stelis) — Sun, 08 Sep 2024 00:00:00 +0000

So, suppose my app that runs on k8s needs a postgres db to write to.
I’ll set up an RDS that is reachable only within my VPC.
Then a dev needs to connect to that DB from their local to troubleshoot some weird issue.
What do I do?
The tried-and-true solution is of course some VPN.
If you have the time and resources to do it right, you’re golden.
If not, a VPN is a surefire way to headaches. (new authentication pool, 2FA , maintenance, split DNS, slow connection speeds etc)

What's the most you can make of an olivetin?

stelis@systematize.it (stelis) — Sun, 08 Sep 2024 00:00:00 +0000

No, not that kind of olivetin, this one :-P

OliveTin is a simple web UI that can be configured via yaml to run cli commands. You add a section such as

actions:
- title: Restart backend
  icon: restart
  shell: docker restart main-backend

and in its UI you’ll find an icon with your title that, once pressed, will execute the shell command you specifed. You can even add arguments

- title: Ping host
  shell: ping {{ host }}
  icon: ping
  arguments:
    - name: host
      title: host
      type: ascii_identifier
      default: example.com

My only slight complaint is that while it sort of supports authorization it does not support authentication. If you’re deploying it to be publicly accessible, you’d need to configure your ingress to authenticate the user and only then redirect them to the app.

Why did Ancient Greeks not have Probability Theory?

stelis@systematize.it (stelis) — Sun, 08 Sep 2024 00:00:00 +0000

Ancient Greeks’ contribution to the foundation of sciences in undeniable. Euclidian Geometry, that accurately describes the space around us,the concept of “mathematical proof”, Aristarchus’ heliocentric theory, the foundation of medicine by Hippocrates, philosophy as expressed by Plato, Socrates and Aristotle are few examples to support this. It’s usual and familiar, whenever tracing back to the foundation of a scientific field, to find references to Ancient Greece. So it might surprise a scholar of the history of Probability Theory that its origins date back to the 16th century. And they might rightfully wonder why did Ancient Greeks not have Probability Theory?

S3 can be persistent

stelis@systematize.it (stelis) — Sat, 07 Sep 2024 00:00:00 +0000

I’ve always hated adding persistence in k8s workloads.
There are valid reasons to avoid it, right?
Adding a PV means people can store stuff there that they expect to find, regardless of where the app is deployed/redeployed (which could be in another region/zone/cluster/node etc where the original pv is not present)
Thus adding another step in the deployment process that comes usually as an afterthought and may be easily overlooked and poorly documented.

DIY internet connectivity notification

stelis@systematize.it (stelis) — Tue, 27 Aug 2024 00:00:00 +0000

WFH is great!
But every once in a while, internet goes down mid-workday and you’re not sure when it will be back up.
You’re forced to leave in a hurry for some nearby cafe.
How can you know it’s back up so you can return?
I solved this once and while I’m not proud of how it worked, it worked!

Prerequisite: A host in that network and a Slack workspace or other service you’re allowed to create a webhook for.

S3 Object Count shenanigans

stelis@systematize.it (stelis) — Tue, 27 Aug 2024 00:00:00 +0000

Suppose you have access to an S3 bucket in some AWS account.
You need to get the files to another bucket (same or different AWS account I think makes no difference to my eventual point)

You do your due diligence and count the objects in the origin bucket

aws s3 ls s3://originBucket --recursive --summarize | grep "Total Objects:"

You get the total count and start your sync

aws s3 sync s3://originBucket s3://destinationBucket --delete --exact-timestamps

Great. It completed successfully. Now let’s count the objects in the destination bucket

AWS, why do you hate me?

stelis@systematize.it (stelis) — Mon, 26 Aug 2024 00:00:00 +0000

It’s generally a good practice to differentiate between application endpoints than need to be accessible via Internet and endpoints that need to be accessible only internally via your private VPC.
But what if I need one app’s endpoint temporarily accessible to my CICD runner that’s not in the VPC?
e.g a /health to know if the deployment succeeded.

If you’re on AWS, you may be correctly thinking PrivateLink, VPC Endpoints etc
But that may be overkill for a short-lived connection from our CICD.
The apps I’m interested in run mostly in kubernetes.
To access an internal endpoint from my local, I’d just kubectl port-forward

Bash rocks! It also bombs :-D

stelis@systematize.it (stelis) — Mon, 26 Aug 2024 00:00:00 +0000

Ran into a nice little bit of nostalgia today.
Have any of you heard/remember the bash bomb ?
It’s Denial of Service attack in 13 characters!

: (){ : |: & };:

Here’s how

Why quote environment variable values in kubernetes manifests?

stelis@systematize.it (stelis) — Mon, 26 Aug 2024 00:00:00 +0000

This a another discussion I have frequently.

Why bother changing something, however small, if it works as it is?
Well, it’s a best practice. If we can spend a very limited amount of time to make trivial changes that will protect us from mishaps, carelessness and changes beyond our control, is it not worth it?

This argument, on its own, is usually not very effective against a stubborn engineer in a hurry.
So, whenever possible, I try to give more concrete examples.
Here’s one that I was happy to find just in time for such a discussion.

What's in a detail that makes it worth a tale?

stelis@systematize.it (stelis) — Tue, 20 Aug 2024 00:00:00 +0000

I’ve always been fascinated with details.
They’re subtle, easy to ignore but every once in a while they turn out to be important.

I have gotten in very interesting (and often heated) discussions about this fascination (mainly at work).
Taking the time to go over the details, learn, understand and digest takes time.
And startups are always on a countdown clock and push to get things faster.
People too. If something seems like it can be ignored, we tend to ignore it.
I suppose most of our minds are already at capacity, way before someone starts a long-winded argument about things that seem trivial and inconsequential.