Questions about JavaScript-based development
April 11th, 2019in code
« Cottage wrap-up

Updated November 2019: Adding some “helpful” links to a TV interview about the creation of JavaScript and a talk about the history of JavaScript from 1995 to 2035.

Having been a C++ guy for most of my life (and generally working on dank and musty OS stuff), I’ve rather enjoyed my recent journey into learning JavaScript.

I believe in having good tools and knowing them well, and that’s cheaper and easier if you have fewer tools. I chose JavaScript as my new frontier because it allows me to do frontend, backend, and mobile with a single language so there are fewer tools to learn. (In practice this maps to React/Gatsby, Azure Functions, and React Native, respectively.)

(Of course if I tried hard enough I could do the same with C++ but I wouldn’t be learning something new and it’s probably not the right application of C++ anyhow. And yes, I love C#, but we’re trying to learn something new here, and transpiling C# to WebAssembly is more adventure than I care for.)

After immersing myself in the world of JavaScript for about six months, a bunch of questions have come up that I haven’t yet been able to find good answers to. It’s easy to find answers on the internet for figuring out how to do things (especially in the JS universe) but it’s much harder to find correct and up-to-date answers for figuring out how to do things well.

Thus, the following questions. If you have thoughts/answers/corrections, please email me.

Front-loading correctness

I try to “front-load” correctness through my development environment. I believe for JavaScript this means:

  1. Using TypeScript (or Flow) for build-time type safety
  2. Using TypeScript in strict mode
  3. Using ESLint with all recommended defaults and close to zero exceptions
    • Use ESLint instead of TSLint since TSLint has been end-of-life’d
  4. Using Prettier for code and metadata formatting
  5. Running TypeScript checks, linting checks, and formatting checks as pre-commit hooks

Question: Is this an accurate approach to robust engineering practices?

Baseline expectation: clang -Weverything, clang-format.

Parameter validation

TypeScript gets me build-time validation of parameters that can be validated at build-time. Yay. But what about at runtime? What about incoming data like JSON snippets from web hooks, databases, etc.?

Question: What is the most elegant, DRY-compliant way to validate JSON? I’m currently using typescript-json-schema to build JSON schemas from TypeScript typescript at build time, then loading and executing them with ajv at runtime. Is this the right thing to do?

Question: What is the most elegant, DRY-compliant way to validate function parameters at runtime, having already specified parameter types in TypeScript?

Baseline expectations: C++ and templates.

Immutable source trees

Working on Windows, I was taught that the build chain should treat the source directory as immutable, placing intermediate files in a dedicated directory and projecting (copy or symlink) runnable trees into another dedicated directory.

I haven’t yet found much that does this in the world of JavaScript.

On a good day (e.g. Gatsby), output is projected into one or more subdirectories of my repo and at least all other directories are treated as immutable.

On a better day (e.g. Hugo), the path to the subdirectory is configurable so I can choose to move it out of the source tree, it’s just not a default.

On a worse day (e.g. Azure Functions), the source directory is ZIP’d up into the actual production deployment package, requiring that tsc place its outputs in a subdirectory of the source, that said outputs be excluded from source control with .gitignore, that unnecessary/unwanted inputs (e.g. local dev secrets) be excluded from the ZIP file with .funcignore, and that for order-of-magnitude size savings, the node_modules directory be yanked from dev mode to production mode with an npm prune --production.

(And of course I have to close VS Code while doing this since its resident TypeScript host holds onto stuff that interferes with the npm prune. Dope.)

Question: This is nuts, right?

Question: What build tools would be the right tools to add into my build chain to fix this, e.g. to project the right outputs including a prod-filtered node_modules tree? Gulp? Grunt? Webpack? Plain JavaScript build scripts?

I don’t mind doing the work but it feels like every small-n months there’s a new shiny way to build a build chain, write a command-line app, or whatever else. Help!

Deploying to prod from my laptop

Both major clouds (the fragmented-at-best landscape of the Azure CLI and AWS’s impressive Amplify CLI, respectively) make it really easy to stand up cloud resources from my laptop without source control. That’s really impressive. It’s what the docs put front and center.

But it’s also really fucking frightening.

I believe that any system that’s not a toy project with zero users should be deployed through CI/CD only, off configuration-as-code data only, with proven/inspect-able chain-of-custody showing how we got from The Commit to The Production Environment as it stands. Whether it’s a question of functional correctness, security, or debug-ability, it comes down to the same process.

Question: Why is the difficulty of doing this right (e.g. Terraform, CloudFormation, ARM templates) vastly greater than doing it wrong (e.g. jumbles of imperative scripts, or even scarier, deploy-from-VSCode)?

Question: Why do we (as an industry) send a siren call to engineers to use our “super-easy-to-use” platform and point them to tools and samples that we (should) know have no relation to best operational practices and just leave them with “look how shiny, now good luck with that”?

In Amplify’s partial defense, they do at least seem to generate CloudFormation goop from ad-hoc CLI invocations and you can source-control that instead, but it’s not exactly their elevator pitch.

Deploying per branch

Question: I can have a million cheap branches with git so why is it so hard to get a per-branch cloud environment stood up? (I got it working for a simple SPA with a bunch of Terraforming and some CloudFlare worker magic but that took me a week.) It feels like we’re clapping with one hand.

Question: In Amplify’s partial defense they support environments and that’s finally out of beta. Is there an Azure equivalent, not that I’m dogmatically attached to Azure?

Engineering quality of starter/example repos

There’s an amazing amount of sample code out there. Many frameworks (e.g. React, Gatsby, …) offer “seed” kits for their framework to get you off the ground.

But most of what I’ve found doesn’t follow my front-loading correctness setup beyond step 1, use TypeScript (without intending to undersell the importance of broad TypeScript adoption, which I find impressive), or any of the other stuff discussed above.

Gatsby has a TypeScript-based starter but it uses TSLint instead of ESLint and doesn’t bother to hook up ESLint as an npm package script, making it the relative winner by getting kind of halfway down my list.

Azure Functions publishes a TypeScript-based sample but it doesn’t lint anything nor does it even pass TypeScript in strict mode (it explicitly turns strict off), making it a runner-up by getting one step down the list.

Question: Why do I have such a hard time finding sample repos that actually teach robust engineering practices? If you’re advertising using TypeScript with service X (e.g. Azure Functions), shouldn’t you showcase the best practices of TypeScript for using X?

Them’s the questions for now - I’m sure there’ll be more in the future.

« Cottage wrap-up