Background

We rely on OpenSearch’s k-NN plugin (built on FAISS) for vector search.
Each index contains high-dimensional embeddings (~1536 dimensions), and we ingest data in bulk batches of 256 documents at a time.

Our setup:

• Engine: OpenSearch_2.19
• Instance type: or2.large.search
• Data nodes: 6 → later scaled to 9
• Breaker limit: 60% (default)
• Memory usage pattern: One index (mab_b2f97...) consistently dominated memory.

The Circuit Breaker Event

During a bulk ingestion job, we started seeing errors like this:

"caused_by": {
  "type": "knn_circuit_breaker_exception",
  "reason": "Parsing the created knn vector fields prior to indexing has failed as the circuit breaker triggered. This indicates that the cluster is low on memory resources and cannot index more documents at the moment. Check _plugins/_knn/stats for the circuit breaker status."
}

Running the diagnostic command:

GET _plugins/_knn/stats?pretty

produced the following key result:

“circuit_breaker_triggered”: true

Two nodes had excessive graph memory usage:

Node ID Graph Memory Usage Cache Full Top Index Usage % rvr4O1HFRiiZpcO1BctWK61 98.27% True mab_b2f97… 85.95% knev23f3029vhivh13fjevn 91.21% False mab_b2f97… 84.24%

This confirmed that the breaker was triggered by one overloaded index that consumed almost all available native memory on a single data node.

Troubleshooting Process

We built a small Python diagnostic script to confirm the issue programmatically.

• Checking k-NN Stats import requests import boto3, json

region = ‘us-west-2’ service = ‘es’

endpoint = “https://vpc-main-xxxx.us-west-2.es.amazonaws.com” url = f”{endpoint}/_plugins/_knn/stats?pretty”

resp = requests.get(url) print(resp.status_code) print(json.dumps(resp.json(), indent=2))

This returned detailed per-node memory usage, helping us pinpoint the culprit.

Verifying Breaker Limit url = f”{endpoint}/_cluster/settings?include_defaults=true&filter_path=**.indices.breaker.request.limit” resp = requests.get(url) print(json.dumps(resp.json(), indent=2))

Result:

{ “defaults”: { “indices”: { “breaker”: { “request”: { “limit”: “60%” } } } } }

So, we were indeed running close to the default 60% circuit breaker threshold.

• Checking Cluster Health url = f”{endpoint}/_cluster/health?pretty” resp = requests.get(url) print(json.dumps(resp.json(), indent=2))

Output:

“status”: “green”, “relocating_shards”: 2, “active_shards_percent_as_number”: 100.0

Everything was healthy — except for the memory issue.

Diagnosing Shard Distribution

We then checked how the heavy index shards were distributed across nodes:

url = f”{endpoint}/_cat/shards?v&format=json” response = requests.get(url) shards = [s for s in response.json() if “mab_b2f97” in s[“index”]]

for s in shards: print(f”Index: {s[‘index’]} | Shard: {s[‘shard’]} | Node: {s[‘node’]}”)

Output:

Found 10 shards for index pattern ‘mab_b2f97’: Shard 0 → 234rsvdsvde43t36r2b14b686b19523f, b464vwerbwvev393849t8vb97ff1f554 Shard 1 → 234rsvdsvde43t36r2b14b686b19523f, 4289fhf83fb3vb3839c36b17afaf590e Shard 2 → 234rsvdsvde43t36r2b14b686b19523f, 9b90871abd6e7r3if32iv209vnooino9 …

⚠️ Observation: One node was hosting four shards of the heavy index. That node was exactly the one showing 98% memory usage.

Fix Attempt 1 – Scaling Data Nodes

We increased the number of data nodes from 6 → 9. The cluster started redistributing shards (relocating_shards: 2). However, after waiting ~30 minutes, the breaker still remained tripped — because the overloaded node reloaded the same k-NN graphs into memory.

Fix Attempt 2 – Reboot Node

We followed AWS docs and rebooted the data node via the console. Result: Node restarted successfully, Circuit breaker still tripped.

Rebooting cleared the JVM heap, but as soon as the node rejoined, OpenSearch reloaded the same graphs, instantly hitting the memory limit again.

Fix Attempt 3 – Close and Reopen Index

Finally, this worked perfectly.

We used the following Python script:

import requests, boto3, json, time

endpoint = “https://vpc-main-xxxx.us-west-2.es.amazonaws.com” index_name = “mab_b2f97…”

Close index

print(f”Closing index: {index_name}”) resp = requests.post(f”{endpoint}/{index_name}/_close”) print(“Status:”, resp.status_code)

time.sleep(30)

Reopen index

print(f”Reopening index: {index_name}”) resp = requests.post(f”{endpoint}/{index_name}/_open”, auth=awsauth) print(“Status:”, resp.status_code)

Check health

resp = requests.get(f”{endpoint}/_cluster/health/{index_name}?pretty”, auth=awsauth) print(json.dumps(resp.json(), indent=2))

Result:

Index closed successfully. Index reopened successfully. Checking index health… “status”: “yellow” → “green”

Then rerunning:

GET _plugins/_knn/stats?pretty

showed:

“circuit_breaker_triggered”: false No nodes above 80% graph memory usage.

The breaker reset cleanly, and indexing resumed.

Root Cause

One vector index (mab_b2f97…) consumed ~85% of native graph memory on a single node.

The k-NN circuit breaker tripped at 60% limit.

Adding data nodes helped distribute shards but didn’t clear the memory state automatically.

Rebooting nodes reloaded the same graphs.

Closing and reopening the index successfully flushed and rebalanced the k-NN cache.

• Outcome

Circuit breaker cleared

No nodes > 80% memory usage

Indexing & search restored

Cluster status: green

Lessons Learned

Monitor _plugins/_knn/stats regularly. It gives visibility into node-level memory pressure.

Circuit breakers protect your cluster. They prevent out-of-memory crashes but can pause indexing.

Closing & reopening an index is a safe AWS-native reset. It flushes the FAISS graphs from native memory.

Scaling helps, but placement matters. Use _cat/shards to ensure balanced shard distribution.

Default circuit breaker limit (60%) is conservative. It’s safer to scale horizontally than tweak that threshold.

Next Steps

We’ll continue to:

Monitor k-NN stats and shard placement daily.

Automate alerts if any node exceeds 80% graph memory usage.

Optimize ingestion batch size to reduce vector memory bursts.

Document this process in our internal runbook.

– Siddharth

Update: July 22, 2025

We received the root cause analysis from AWS support. They confirmed that the issue was indeed caused by the simultaneous scaling and security changes. The domain was stuck in a modifying state due to simultaneous configuration changes: enabling both remote store and FGAC. They confirmed that this combination can cause a red index status for one of the system indices. The issue was resolved by deleting the affected system index and retrying the process. They said they will be implementing fixes to prevent similar incidents in the future.

This was a huge relief as it confirmed our suspicions and provided us with a clear path forward. We will be implementing the recommended changes in our future deployments.

“Until then, we recommend following this sequence: Enable FGAC first and then proceed with remote store migration if needed.” ~ AWS Support

This is honestly a good recommendation and I should have followed it with or without the AWS support confirmation. During any migrations or major changes, it is always a good idea to follow a sequence of steps that are known to work. This will help you avoid any unexpected issues and ensure that your changes are applied successfully. I am glad that the AWS support team was able to get us out of this situation quickly and efficiently with no data loss.

Update: July 5, 2025

AWS support confirmed that the our cluster has fully recovered and is now in a healthy state. They also confirmed that the migration was now fully completed and all indices are accessible. We were able to confirm the same by visually looking at the cluster status. We now needed to test the cluster and make a decision if we wanted to point our pipelines to this cluster or not.

Waiting for AWS support to tell us if they can sync the two clusters for us - our old one and the new one. If they can, we will be able to point our pipelines to the new cluster without losing any data. If not, we will have to reingest the data from S3 again. I also needed approvals from my Director if we wanted to point our pipelines back to the old cluster or not.

So we wait…

Update: July 4, 2025

AWS support identified a stuck snapshot blocking shard relocation and requested approval to perform a rolling restart of data nodes in the blue environment. After getting our approval for potential outages (since we had a backup domain which was now being used actively), they successfully restarted the nodes and confirmed the cluster returned to a healthy green state with all indices accessible.

Upon viewing the cluster status, I confirmed that the cluster, although in the green state, was still trying to apply some changes.

TL;DR

• Incident Date: July 3, 2025
• Duration: ~5 hours
• Impact: Production OpenSearch cluster became inaccessible due to simultaneous scaling and security changes.
• Speculations: Without knowing for sure or AWS confirming, I will be mentioning if something is a pure speculation from my end. I have asked AWS for a root cause analysis and will update this post once I receive it.

How It Started

During the week of June 30th, we noticed extreme JVM pressure and high CPU utilization in our Test environment. Our setup had:

• 5 shards per index (default)
• 3 nodes with 1 dedicated master node
• Warm storage enabled
• Document ingestion rate: ~100 docs/sec

Our initial configuration looked like this:

cluster_config = {
  instance_count           = 3
  dedicated_master_enabled = true
  dedicated_master_type    = "c7g.large.search"
  instance_type            = "r7g.large.search"
  warm_count               = 3
  warm_enabled             = true
  warm_type                = "ultrawarm1.large.search"
  cold_storage_options = {
    enabled = false
  }
  zone_awareness_config = {
    availability_zone_count = 3
  }
  zone_awareness_enabled = true
}
ebs_options = {
  ebs_enabled = true
  iops        = 3000
  throughput  = 250
  volume_type = "gp3"
  volume_size = 512
}
encrypt_at_rest = {
  enabled = true
}

This setup, while memory-optimized, was not ready for our load tests (~100 documents in short bursts). JVM pressure spiked and we needed to scale — fast.

Scaling Up

I rolled out a new configuration on July 1st, increasing node count, instance size, and enabling cold storage:

cluster_config = {
  instance_count           = 9
  dedicated_master_enabled = true
  dedicated_master_count   = 3
  dedicated_master_type    = "r7g.large.search"
  instance_type            = "or2.2xlarge.search"
  warm_count               = 3
  warm_enabled             = true
  warm_type                = "ultrawarm1.large.search"
  cold_storage_options = {
    enabled = true
  }
  zone_awareness_config = {
    availability_zone_count = 3
  }
  zone_awareness_enabled = true
}
ebs_options = {
  ebs_enabled = true
  iops        = 3000
  throughput  = 250
  volume_type = "gp3"
  volume_size = 1024
}

This resulted in a major performance improvement: JVM pressure dropped, CPU stabilized, and ingestion was smooth.

The Setup in Production

Confident in the new setup, I decided to apply the same configuration to production on July 3rd. However, there was one (SPECULATION: there might be more than one difference) critical difference:

• In test, we applied configuration and security changes separately.
• In production, we applied both at the same time.

Why Security Changes?

We had observed random index creations in test and wanted to enable AUDIT_LOGS to track user activity. That required enabling advanced security, which in turn required a master user. Advanced security also provides features like fine-grained access control (FGAC) and internal user database management.

advanced_security_options = {
  enabled                        = true
  internal_user_database_enabled = false
  anonymous_auth_enabled         = false
  master_user_options = {
    master_user_arn = "arn:aws:iam::xxxxxxxxxx:role/user-role"
  }
}

In the test environment, this change caused a temporary “RED” cluster state for ~10 minutes, which reverted to “GREEN” successfully. Per AWS docs:

“The change triggers a blue/green deployment during which the cluster health becomes red, but all cluster operations remain unaffected.”
— AWS Docs on FGAC

So, I felt confident rolling the same changes into prod.

The Deployment – What Went Wrong?

Once approved, we deployed changes in prod — both the infrastructure scale-up and advanced security enablement — simultaneously. This was the beginning of the outage (SPECULATION: we do not know for sure if this was the beginning). Below is an image of our cluster migration status during the deployment.

It showed that new nodes had been added and traffic routing was successful. It had now reached the point where it was supposed to copy the shards to the new nodes. This is when we started noticing issues. The cluster status turned to “RED”. We were unable to access the cluster and the OpenSearch dashboard was not loading. Any index or search operation was failing with a “503 Service Unavailable” error.

A quick google search led me to this forum post on AWS. https://repost.aws/knowledge-center/opensearch-domain-stuck-processing The post basically mentioned that the cluster was stuck in a state where it was trying to copy the shards to the new nodes, but it was unable to do so. It asked us to monitor the current migration of the shard with the following API call:

GET /DOMAIN_ENDPOINT/_cat/recovery?active_only=true

We hit this error:

{
  "error": {
    "type": "security_exception",
    "reason": "OpenSearch Security not initialized for indices:monitor/recovery"
  },
  "status": 503
}

That’s when I realized: the security plugin hadn’t initialized properly.

I attempted master-user access via IAM — still failed.

This was around 3:00 pm.

AWS Support Weighs In

At 6:15 pm I contacted AWS support. They were able to confirm that the cluster was stuck in a state where it was trying to copy the shards to the new nodes, but it was unable to do so. They initially mentioned that this is a normal behavior and the cluster would eventually recover. However, after waiting for a few hours, the cluster was still in the same state and we were unable to access it.

AWS confirmed:

• The cluster was stuck during shard copying to the new nodes.
• Specifically, the .opendistro_security index — which stores security config — could not be assigned or migrated.

{
  "index": ".opendistro_security",
  "shard": 0,
  "primary": false,
  "current_state": "unassigned",
  "unassigned_info": {
    "reason": "INDEX_CREATED",
    "at": "2025-07-03T21:27:11.736Z"
  }
}

This was the index that was causing the issue (SPECULATION: but semi confirmed by aws on the chime call). The support team tried to reassign the shard, but it was still failing with the same error. This was an internal AWS index that is used to store the security configuration for the cluster. It was not able to migrate to the new nodes and was causing the cluster to be stuck in a state where it was unable to access any indices.

No Way Out

I had a few options at this point. I could either wait for the cluster to recover on its own, or abandon that cluster and create a new one. I decided to wait for a few more hours, but the cluster was still in the same state. I was unable to access any indices or perform any operations on the cluster. I confirmed that all data nodes were migrated and in active state.

I explored every workaround:

• Reassign shards manually? Blocked.
• Cancel update? Not allowed mid-migration.
• Delete the domain? Domain was locked in Processing.
• Restore from snapshot? AWS’s default 1-hour snapshots are not user-restorable.

At this point, I was stuck in limbo.

Then we found this AWS forum post from 2 years ago describing exactly the same scenario. The same .opendistro_security index was unassignable. Unfortunately, AWS support didn’t accept it as conclusive evidence and continued investigating.

Our Recovery Plan

With production deadlocked, we took matters into our own hands:

• Spun up a new domain, same configuration — excluding advanced security.
• Reingested from S3, our source of truth.
• Redirected traffic to the new cluster.

Within a few hours, ingestion was back at 100 docs/sec. The new cluster held up well.

Meanwhile, our old cluster was still at 71% migration and remained unusable but now with status yellow. AWS confirmed that shard migration was partially retriggered — but that didn’t help us regain access.

Downtime Summary

• Total outage: ~5 hours
• User impact: Minimal (due to long weekend)
• Fallback: Temporary reroute to serverless instance for select users

Lessons Learned

1. Don’t Combine Major Changes

Scaling and security updates should be rolled out in separate phases — especially in production. Always perform a dry run to test the configuration first.

2. Be Cautious with Advanced Security

Once enabled, you can’t disable advanced security. Enable it only when you’re confident it won’t block critical operations.

3. Snapshots Matter

Relying on AWS’s default hourly snapshots isn’t enough. Set up manual, restorable snapshots. You cannot restore from AWS’s default snapshots directly, so ensure you have a backup strategy that allows you to restore data when needed.

4. Monitoring is Key

Track migrations closely. If a cluster update takes more than 90 minutes, escalate immediately.

5. Have a Rollback Plan

Always have a tested fallback path (like our S3 ingestion) to recover quickly in case of failure.

Final Thoughts

We’re still awaiting AWS’s official root cause analysis. But we were lucky: Our fallback pipelines kept this from becoming a full-blown production disaster.

I hope this post helps others avoid the trap we fell into.

– Siddharth

Prerequisites

Before we begin, ensure you have the following:

• A Raspberry Pi (preferably Raspberry Pi 4 or later)
• A microSD card with Raspberry Pi OS installed
• Basic knowledge of using the terminal
• An internet connection for your Raspberry Pi

Step 1: Update Your Raspberry Pi

First, make sure your Raspberry Pi is up to date. Open a terminal and run the following commands:

sudo apt update
sudo apt upgrade -y

I installed the Ubuntu server version on my Raspberry Pi, but you can use any Raspberry Pi OS variant. I selected this version because I wanted a minimal setup without a desktop environment I can access via SSH.

Step 2: Install Jellyfin

Next, we will install Jellyfin. Run the following commands in your terminal:

sudo curl https://repo.jellyfin.org/install-debuntu.sh | sudo bash
sudo ufw allow 8096/tcp

The below links provide additional information on installing Jellyfin. They are discussion threads that can help you troubleshoot any issues you might encounter during the installation process. You can also find the official installation guide on the Jellyfin website.

https://github.com/jellyfin/jellyfin/discussions/7460

https://gist.github.com/aslafy-z/dce9fd98bbe42f21095eb231687ae4f5

That’s it! Jellyfin is now installed on your Raspberry Pi. You can access the Jellyfin web interface by navigating to http://<your-raspberry-pi-ip>:8096 in your web browser.

Add your jellyfin user in your user group to allow all access needed:

sudo usermod -aG $USER jellyfin

Commands to check status and restart Jellyfin:

sudo systemctl status jellyfin
sudo systemctl restart jellyfin

Make sure your content folder has the correct permissions:

chmod o+x /home/raspberrypi/Content

Step 3: Install Tailscale

Now, we will install Tailscale to securely access your Jellyfin server from anywhere. Run the following commands:

curl -fsSL https://tailscale.com/install.sh | sh
sudo tailscale up

To check the status of Tailscale, you can use:

tailscale status

Once you login to Tailscale, you will need to download the tailscale client on your devices (laptop, phone, etc.) to access the Jellyfin server. You can find the Tailscale client for various platforms on their official website.

Step 4: Accessing Jellyfin via Tailscale

After installing Tailscale on your devices, you can access your Jellyfin server by navigating to http://<tailscale-ip>:8096 in your web browser. The Tailscale IP can be found in the Tailscale admin console or by running tailscale status on your Raspberry Pi.

You can also use the local IP address of your Raspberry Pi if you want. It can be accessed locally and via Tailscale. The local IP address can be found by running:

hostname -I

Note

On my iPhone, I added the Tailscale client and was able to access the Jellyfin server without any issues. The streaming quality was excellent, and I could browse my media library seamlessly. I had to configure the VPN but it was straightforward. I also set up the Jellyfin app on my iPhone, which allowed me to stream content directly from the app without needing to use a web browser. I use FinAmp, a Jellyfin client for iOS, which provides a great user experience for streaming audio.

Conclusion

You have successfully set up a Jellyfin media server on your Raspberry Pi and secured it with Tailscale. Now you can enjoy your media content from anywhere, securely and privately.

– Siddharth

What You’ll Find Here

This blog is my digital notebook. I’ll be writing about things I’m working on, learning, or thinking about. Topics will include:

• Thoughts on tech and engineering – insights, opinions, or breakdowns of tools and processes I find useful
• Beginner-friendly tutorials – step-by-step guides for solving real problems (the kind I wish I found when I was learning!)
• Personal stories – my experiences in tech, career milestones, mistakes made, and lessons learned
• Ideas and experiments – side projects, coding challenges, or just “what if?” scenarios I’m tinkering with

Whether you’re new to programming or a seasoned developer, I hope there’s something here for you.

Why I Chose Jekyll

When starting a blog, I had a choice: use a platform like Medium or WordPress, or build something myself. I chose Jekyll, a static site generator, because:

• Speed – Jekyll builds a static website, meaning it’s fast and doesn’t need a database
• Simplicity – I write posts using Markdown (plain text with formatting), which is clean and distraction-free
• Customizable – I can tweak the layout, design, and features just the way I want
• Fun to tinker with – I enjoy diving into the internals and learning how things work

To make things look good and reduce setup time, I’m using the Minimal Mistakes theme – a beautiful and flexible theme designed for Jekyll blogs.

Running Jekyll Locally

Before publishing your blog online, it’s a good idea to see what it looks like on your own computer. Here’s how I run my Jekyll blog locally:

bundle exec jekyll server

This command tells Jekyll to start a local web server. You can then open a browser and go to http://localhost:4000 to see your blog as it will appear when live.

Tip: You need to have Ruby, Bundler, and Jekyll installed for this to work. Don’t worry – I’ll write a step-by-step guide soon!

My Jekyll Configuration (Explained)

Here’s a peek at my _config.yml file – the brain of the blog setup. Each line controls part of how the blog behaves:

title: tellsiddh's blog                # The blog title shown in the browser
description: A simple blog powered by Jekyll
url: "https://blog.tellsiddh.com"     # The actual URL of your site
baseurl: ""                            # Used if your blog is in a subfolder (leave blank for root)

permalink: /:categories/:title/       # Controls the URL format (e.g. /personal/hello-world/)

remote_theme: "mmistakes/minimal-mistakes@4.27.1"   # The theme used
minimal_mistakes_skin: "dark"         # Theme style (others: default, air, neon...)

excerpt_separator: "<!--more-->"      # Marks where a summary ends and the full post continues

plugins:                              # Adds extra features to the blog
  - jekyll-feed                       # Generates an RSS feed
  - jekyll-sitemap                    # Adds a sitemap for better SEO
  - jekyll-seo-tag                    # Helps improve search engine visibility
  - jekyll-include-cache              # Optimizes page includes for speed

author: siddharth                      # My author name for posts

category_archive:
  type: liquid
  path: /categories/                  # Enables viewing posts by category

tag_archive:
  type: liquid
  path: /tags/                        # Enables viewing posts by tag

twitter_username: tellsiddh           # Links to my Twitter
github_username: tellsiddh            # Links to my GitHub

Don’t worry if this looks complicated – once you get the hang of it, customizing your blog becomes second nature.

What’s Next?

Now that the blog is live, here’s what you can expect from future posts:

• In-depth guides on tools like Git, VS Code, Jekyll, and more
• Breakdowns of side projects I’m building
• Thoughts on working in tech, growing as a developer, and building a career
• Possibly some random nerdy stuff I just couldn’t resist writing about

If you’ve read this far, thank you I’m truly excited to share this space with you.

Want to connect or follow along?

GitHub
Twitter

Stay curious,
Siddharth

This post captures the most frequently used, most impactful, and uniquely powerful commands from that experience. Each one has been battle-tested and integrated into real engineering environments.

Setup & Shell Enhancements

Switch to the fish shell (Friendly Interactive Shell)

brew install fish
which fish
sudo sh -c 'echo /opt/homebrew/bin/fish >> /etc/shells'
chsh -s /opt/homebrew/bin/fish
fish

Enables better auto-suggestions, syntax highlighting, and user-friendliness.

Install Core Tools

brew install git htop jq wget tree curl tmux

These are essentials for terminal power users.

Add PATHs in fish shell

fish_add_path /opt/homebrew/bin/

Make it persistent via config.fish:

nano ~/.config/fish/config.fish

Use alias for shortcuts

alias gs="git status"
alias ll="ls -lah"
alias gc="git commit -m"
alias gco="git checkout"

Save to:

nano ~/.config/fish/config.fish

Git: Daily Driver for Projects

These are the most-used git commands that show up repeatedly across dev workflows:

git clone git@github.com:user/project.git
git checkout -b feature/my-branch
git switch branch-name
git commit -m "meaningful message"
git push --set-upstream origin branch-name
git stash / git stash pop
git reset HEAD~1
git log --oneline --graph --all

Create and jump to a new feature branch:

git checkout -b fix/critical-issue

View status or staged diff:

git status
git diff

Python Development Workflows

Create isolated environments with conda:

conda create -n myenv python=3.10
conda activate myenv
conda install numpy pandas jupyter

Run scripts:

python my_script.py
python -m http.server

Profile performance:

python -m cProfile -s cumulative script.py

Use tuna to visualize import times:

pip install tuna
python -X importtime script.py 2> import.log
tuna import.log

Networking, SSH & System Tools

SSH with custom keys

ssh -i ~/.ssh/id_rsa user@host

Copy your key:

ssh-copy-id user@host

Diagnose DNS, IP, Connectivity

ping google.com
dig example.com
nslookup example.com
ifconfig | grep broadcast

Find your local IP:

ipconfig getifaddr en0

Scan network devices

sudo nmap -sn 192.168.1.0/24

Package Managers

Brew (macOS)

brew install wget git jq
brew update
brew upgrade

Pip & Python packages

pip install package-name
pip uninstall package-name
pip freeze > requirements.txt
pip install -r requirements.txt

File and Text Utilities

Search history

history | grep "ssh"

Grep for content in files

grep -r "keyword" .
grep "error" logs.txt

Recursive file ops

find . -name "*.py"
find . -type f -exec cat {} +

Disk space

df -h
du -sh *

Clean terminal

clear
Ctrl + L

System Monitoring

htop

sudo apt install htop
htop

Better than top — shows processes, memory, swap, threads, and more.

Check ports

lsof -i :8000
sudo lsof -i -P | grep ':8080'

Kill process by PID:

kill -9 <PID>

Automation Scripts

Make a Python file executable:

chmod +x script.py
./script.py

Zip files:

zip -r archive.zip folder/

Unzip:

unzip file.zip

APIs & Web Tools

Curl with headers

curl -X POST https://api.example.com/endpoint \
  -H "Authorization: Bearer $TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"key":"value"}'

Secure file uploads (example)

curl -X PUT "$SIGNED_S3_URL" \
  -H "Content-Type: application/pdf" \
  --upload-file ./file.pdf

Audio Tools (AI & Speech)

TTS using LLaMA-TTS:

This needs llama.cpp setup, binary works.

llama-tts --tts-out-default -p "Hello world" && ffplay output.wav -nodisp -autoexit

Stream via Flask API:

curl -X POST http://localhost:5000/tts \
  -H "Content-Type: application/json" \
  -d '{"text": "Welcome to the demo"}' \
  --output output.wav

Data Engineering Tools

Terraform

terraform init
terraform plan -out=tfplan
terraform apply tfplan
terraform destroy

Target specific modules:

terraform apply -target=module.my_module

AWS CLI

aws s3 ls
aws configure
aws lambda invoke --function-name my-func out.txt

Use localstack:

LOCALSTACK_AUTH_TOKEN=dummy localstack start

High-impact One-liners

Re-run last command with sudo

!! does not work on fish, not sure why?

sudo !!

Repeat previous command

!!

View last 100 commands

history | tail -n 100

Git reset & clean

git reset --hard HEAD
git clean -fd

Miscellaneous Yet Powerful

View markdown docs in browser

python3 -m http.server 8000

Set environment variables

export AWS_PROFILE=default
export OPENAI_API_KEY=your-key

Decode JWT

echo "<JWT_PART_2>" | base64 --decode

Final Words

These commands reflect months of actual engineering activity — not just tutorials. If you found this useful:

• Follow @tellsiddh
• Star my GitHub
• Check out blog.tellsiddh.com

Stay curious and keep building!
— Siddharth