April 11th, 2019in code
|« Cottage wrap-up||The importance of terminology »|
Updating February 2020: Adding a few proposed Answers throughout the post, based in part on conversations with a Microsoft architect for TypeScript and Azure who kindly read over my post, and adding a new “lessons learned” section.
(Of course if I tried hard enough I could do the same with C++ but I wouldn’t be learning something new and it’s probably not the right application of C++ anyhow. And yes, I love C#, but we’re trying to learn something new here, and transpiling C# to WebAssembly is more adventure than I care for.)
Thus, the following questions. If you have thoughts/answers/corrections, please email me.
- Using TypeScript (or Flow) for build-time type safety
- Using TypeScript in
- Using ESLint with all
recommendeddefaults and close to zero exceptions
- Use ESLint instead of TSLint since TSLint has been end-of-life’d
- Using Prettier for code and metadata formatting
- Running TypeScript checks, linting checks, and formatting checks as pre-commit hooks
Question: Is this an accurate approach to robust engineering practices?
Answer: Yes, pretty much. I recently tightened up the
eslintrules for WarmAndFuzzy and that finally seems like a good starter kit. I find
eslintmost useful for enforcing consistency; pretty much all the correctness stuff is caught with TypeScript proper.
TypeScript gets me build-time validation of parameters that can be validated at build-time. Yay. But what about at runtime? What about incoming data like JSON snippets from web hooks, databases, etc.?
Question: What is the most elegant, DRY-compliant way to validate JSON? I’m currently using typescript-json-schema to build JSON schemas from TypeScript typescript at build time, then loading and executing them with ajv at runtime. Is this the right thing to do?
Question: What is the most elegant, DRY-compliant way to validate function parameters at runtime, having already specified parameter types in TypeScript?
Baseline expectations: C++ and templates.
Answer: I’ve now settled on using
yupwhich allows you extract a TypeScript
yupschema. However, I also need to decorate my types for
dynamodb-data-mapper-jsso I can’t use that
yupcapability. Close enough.
Immutable source trees
Working on Windows, I was taught that the build chain should treat the source directory as immutable, placing intermediate files in a dedicated directory and projecting (copy or symlink) runnable trees into another dedicated directory.
On a good day (e.g. Gatsby), output is projected into one or more subdirectories of my repo and at least all other directories are treated as immutable.
On a better day (e.g. Hugo), the path to the subdirectory is configurable so I can choose to move it out of the source tree, it’s just not a default.
On a worse day (e.g. Azure Functions), the source directory is ZIP’d up into the actual production deployment package, requiring that
tscplace its outputs in a subdirectory of the source, that said outputs be excluded from source control with
.gitignore, that unnecessary/unwanted inputs (e.g. local dev secrets) be excluded from the ZIP file with
.funcignore, and that for order-of-magnitude size savings, the
node_modulesdirectory be yanked from
productionmode with an
npm prune --production.
(And of course I have to close VS Code while doing this since its resident TypeScript host holds onto stuff that interferes with the
npm prune. Dope.)
Question: This is nuts, right?
Answers: Yes. I now manage this pretty carefully with a
generateddirectory for codegen’d stuff needed during the build, and a
builddirectory for build outputs, and it works for the WarmAndFuzzy engineering system.
Question: What build tools would be the right tools to add into my build chain to fix this, e.g. to project the right outputs including a
I don’t mind doing the work but it feels like every small-n months there’s a new shiny way to build a build chain, write a command-line app, or whatever else. Help!
gruntare old and should no longer be used in new projects (I am told). For WarmAndFuzzy, Serverless and
Deploying to prod from my laptop
Both major clouds (the fragmented-at-best landscape of the Azure CLI and AWS’s impressive Amplify CLI, respectively) make it really easy to stand up cloud resources from my laptop without source control. That’s really impressive. It’s what the docs put front and center.
But it’s also really fucking frightening.
I believe that any system that’s not a toy project with zero users should be deployed through CI/CD only, off configuration-as-code data only, with proven/inspect-able chain-of-custody showing how we got from The Commit to The Production Environment as it stands. Whether it’s a question of functional correctness, security, or debug-ability, it comes down to the same process.
Question: Why is the difficulty of doing this right (e.g. Terraform, CloudFormation, ARM templates) vastly greater than doing it wrong (e.g. jumbles of imperative scripts, or even scarier, deploy-from-VSCode)?
Answer: Because this — doing it the shoddy way — is how we attract people to new tech, sadly. The right way would just be intimidating and off-putting. Also, it’s a bit of a choice matrix problem since there are large-N technologies to build starter projects for and medium-N ways of doing CI/CD so the matrix would just explode.
Question: Why do we (as an industry) send a siren call to engineers to use our “super-easy-to-use” platform and point them to tools and samples that we (should) know have no relation to best operational practices and just leave them with “look how shiny, now good luck with that”?
In Amplify’s partial defense, they do at least seem to generate CloudFormation goop from ad-hoc CLI invocations and you can source-control that instead, but it’s not exactly their elevator pitch.
Answers: Yep. It’s an industry-wide problem so nobody is compelled to do better.
Deploying per branch
Question: I can have a million cheap branches with
gitso why is it so hard to get a per-branch cloud environment stood up? (I got it working for a simple SPA with a bunch of Terraforming and some CloudFlare worker magic but that took me a week.) It feels like we’re clapping with one hand.
Answer: We are, and there’s a significant gap between what you see in starter projects and what big players actually do in practice. It’s too bad that this leaves smaller players with mostly anti-patterns.
Question: In Amplify’s partial defense they support environments and that’s finally out of beta. Is there an Azure equivalent, not that I’m dogmatically attached to Azure?
Engineering quality of starter/example repos
There’s an amazing amount of sample code out there. Many frameworks (e.g. React, Gatsby, …) offer “seed” kits for their framework to get you off the ground.
But most of what I’ve found doesn’t follow my front-loading correctness setup beyond step 1, use TypeScript (without intending to undersell the importance of broad TypeScript adoption, which I find impressive), or any of the other stuff discussed above.
Gatsby has a TypeScript-based starter but it uses TSLint instead of ESLint and doesn’t bother to hook up ESLint as an
npmpackage script, making it the relative winner by getting kind of halfway down my list.
Azure Functions publishes a TypeScript-based sample but it doesn’t lint anything nor does it even pass TypeScript in
strictmode (it explicitly turns
strictoff), making it a runner-up by getting one step down the list.
Question: Why do I have such a hard time finding sample repos that actually teach robust engineering practices? If you’re advertising using TypeScript with service X (e.g. Azure Functions), shouldn’t you showcase the best practices of TypeScript for using X?
Answer: See above, basically.
Them’s the questions for now - I’m sure there’ll be more in the future.
That said, I’d drop it in a heartbeat if there were a better language available that worked for both backend and frontend code. Basic stuff like not being able to use anything other than strings as dictionary keys is just goofy.
|« Cottage wrap-up||The importance of terminology »|