This isn’t “Normal” Work-from-home

Over the past three weeks I’ve supported a number of people who have transitioned to working from home during the Covid-19 lockdown. The one thing that keeps coming up is that this is different than standard remote work. I’ve worked remotely for 12 years of my career and this is definitely not “normal”. I think that’s a critical thing for folks working remotely for the first time to understand. When Covid-19 lockdowns are over, working from home will be easier; this is a lot harder.

Continue reading “This isn’t “Normal” Work-from-home”

Harnessing Resiliency for Security

On Thursday evening I got an alert that one of our server’s outbound network traffic exceeded normal level. By about 100-fold.

Why do we have these alerts? For security. I want to know if inbound or outbound traffic is too high. For inbound traffic, this could be the sign of a DoS attack. For outbound traffic this could indicate that one of our servers has been compromised and is now being used to send spam, distribute malware, or exfiltrate data.

I won’t get into the investigation here (the server wasn’t compromised). Instead I want to talk about how we’ve reduced the risk of a compromised server by harnessing the resiliency we designed into our operations. We can just blow everything away and start from clean images. This is a really big hammer; but, because of in-built resiliency, it’s non-disruptive.

Continue reading “Harnessing Resiliency for Security”

DIY Risers for Wooden Train Sets

A few years ago, as my kid was getting into wooden trains, I made these risers so he could build bridges and elevated tracks. While there are lots of risers available for purchase, I really like these because they were fairly simple to make, and have a lot of stability. The key difference in the design is the angled sides which adds the stability. I was inspired by Incan buildings that have withstood earthquakes for more than half a millennium.

Interlocking blocks and sloping walls make Inca buildings extremely resistant, but not immune, to earthquake damage. 500 years of earthquakes have done remarkably little damage to Inca structures left in their complete state.

Inca Architecture, Ancient History Encyclopedia.
Continue reading “DIY Risers for Wooden Train Sets”

Beginner’s Luck: When Kittens Walk Through Walls

I really enjoy working with junior developers. They are super valuable in my team. They bring rapid innovation to stagnant areas of the code base or the process. They add a breath of fresh air to stuffy corners in senior developer-dom. They walk through walls.

Photo by AndriyKo Podilnyk on Unsplash
Continue reading “Beginner’s Luck: When Kittens Walk Through Walls”

Managing Cron Jobs in Kubernetes with Ruby

We run a Rails application as Docker containers in Kubernetes. Our application and services have a fair number of scheduled tasks and before we moved to containers these ran using cron in the VM that the server ran on. When we moved to Docker we first migrated by deploying a container that just ran our cron jobs. We’ve now migrated those to native Kubernetes Cron jobs which vastly improved the resiliency of our system.

Screen Shot 2020-01-08 at 4.41.08 PM
Continue reading “Managing Cron Jobs in Kubernetes with Ruby”

The New Google Cloud Build

We recently moved several of our projects to the new Google Cloud Build for building container images and pushing them to the repository. It’s a pretty simple system (not a full CI) but it does the job well, and I liked having the “build” part separate from the “run tests” part of the toolchain. That said, I feel like this is among the many tools that leave me writing bash scripts in YAML.

Continue reading “The New Google Cloud Build”