Background

About me & how I tested Fly

The good

Container-native, buildpack-free deploy

There’s nothing wrong with buildpacks, the toolchain Heroku built and evolved.

Great first-class networking features

  • L7 load balancer

  • Private networking and addressing

A fast-moving team and fast-evolving feature set

What still needs work

Deployment reliability

I’m a vocal believer that fast, reliable release systems are one of the most important assets any production team can have. When engineers can release quickly, releases can be performed more frequently, and the size of a release stays small.

Moreover, when deploys are fast, cheap, and reliable, incident response teams are more successful: Fast deploys mean fast rollbacks. All manner of production changes can be effected quickly when code can be deployed quickly.

Unfortunately in my only modest use of Fly, there have been several situations where deploying just… didn’t work. And it seems

Instance migrations

https://community.fly.io/t/application-randomly-migrated-region/4292/3

Cloudflare issues

https://community.fly.io/t/cloudflare-525-error-randomly-occurs/1798/29

Whats different

Quirks and unusal design choices

  • Global namespace for apps. It follows heroku, but does seem odd/dangerous.

  • fly vs flyctl

  • Weird gotchas like, “set secrets before your first deploy”: https://community.fly.io/t/preview-multi-process-apps-get-your-workers-here/2316/18?u=mikey

  • No atomic release numbers. In Heroku, new releases (whether from a build or from changing env config) increment a strictly increasing version number, like v42. This value is exposed to the application at runtime, as HEROKU_RELEASE_VERSION.

    I’ve incorporated this value into build pipelines (to ensure forever-cacheable assets get recomputed hashes every release) and into debug logs (to concisely tag & understand what exact environment a customer was experiencing in a crash report).

    In Fly, deployments increment a version number that looks similar. However this value not available to env at runtime. To make things a little more mysterious, Fly engineers have noted the version number can change without the app restarting.

    Overall, if you’re looking for a single simple variable which can encapsulate all of the runtime configuration, there isn’t a good solution.

  • Secrets aren’t available during build. Secrets in Fly are only made available to the application at runtime. This means if your deploy process needs a secret during build, you will have to find some other way to get it to your container.

  • No “one-click” rollback. In Heroku, the Activity tab shows you recent deployments, and gives you an option to re-deploy a previous release. In Fly, although this would seem like a natural feature, there is no built-in support for it yet.

  • No scheduler (or other add-ons, for that matter). It’s basic, but for those requiring periodic/batch execution, Heroku’s scheduler is a simple “just a few clicks” solution.

What I’d like to see

Deployment reliability

Public/private applications


They’ve been talking about “when we replace nomad” forever: https://community.fly.io/t/request-fly-app-email-alerts/3024/2?u=mikey