Just Enough Software

Local-only architecture

Thu, 27 Jul 2023 14:38:00 -0500

Continuing on from our review of the project’s end goal, let’s sketch out the architecture for the frontend, with a focus just on local-only functionality.

Keeping things as simple as possible, I want a text field that we can edit, and which will persist our changes across reloads of the page. Lucky for us, automerge has a tutorial that does just that.

Since we’ve sorted out our build system, nothing there should need to change, and whatever functionality we get locally during development should be the same after deployment (though the state will be different for both).

All that taken together:

Update the Fulcro code to have a text field we can edit, but without persistence
Follow the automerge tutorial, sorting out how to integrate automerge with Fulcro, and ultimately introducing persistence with automerge.

While each of those items has many sub-items, some of which I’m aware of and some I’m sure will surprise me, I think that’s a good enough breakdown to get moving.

Until next time!

Revisiting our end goal

Thu, 27 Jul 2023 14:24:00 -0500

In our previous post, Serving up Fulcro, we learned about what it takes to build and serve the frontend assets for our app. At the end of that post I listed out some Next Steps, but before we pursue those next steps I want to step back and revisit the end goal to make sure that the next thing we figure out is the right next step.

In the AutoFlare Overview post I talked about wanting to learn about three things:

Cloudflare’s cloud offerings
Fulcro
Automerge CRDT

So far, we’ve learned enough about Cloudflare’s offerings to identify the top-level offerings we’ll need to use, even if we haven’t learned enough about all of them to use them yet.

We’ve also learned enough about Fulcro to build frontend assets with the Pages service, though nothing else about Fulcro yet.

Finally, we’ve learned nothing about Automerge.

At the end of this project, I’d like to have a repo I can deploy with Cloudflare Pages that supports multi-user, multi-machine text editing with Automerge, using Fulcro as the library for handling the FE code.

Since Automerge was built with local-first development in mind, we can distinguish between two major architectural phases of this project:

Local only
Local and Remote

The first phase only requires frontend assets, and thus doesn’t require us to sort out Pages Functions or Durable Objects at all. The second phase requires the same things as the first phase, plus an understanding of Functions and Durable Objects. Even if we were to start with Remote Only first, we’d still have to sort out most of the same things as the Local Only phase – defining the UI, handling text input, leveraging automerge – along with all of the things the Remote phase needs, so going Remote Only as a first step doesn’t save us anything.

With that in mind, figuring out how the local-first approach should look sounds like the next, incremental step in this project.

I’ll sketch out a rough shape for the necessary components needed for this local-only approach in the next post. See you then!

Serving up Fulcro

Thu, 27 Jul 2023 10:10:00 -0500

Now that we understand the general requirements for hosting FE assets and backend functionality with Pages, let’s figure out how to get that working with Fulcro.

Pages has a long list of supported frameworks, but, perhaps unsurprisingly, none of them are a CLJS framework. Luckily, remembering from the Pages overview post, we can either build the assets ourselves and commit them to our repo, or, possibly, we can specify a custom build command that will build the assets for us as part of the Pages infrastructure.

In this post, we’ll do the following:

Get a barebones Fulcro app that we can build and view locally
Sort out how we can build this via the Pages CI/CD infrastructure.

Barebones Fulcro App

Originally, this section was going to be about creating a barebones app with the Fulcro Template, but after going down that route for a bit I realized that the frontend code relies on having the backend code running, which isn’t what I need or want at the moment.

So instead of that, we’re going to follow the Fulcro book to get the bare minimum frontend code working.

I’ve created the autoflare repo to house the code for this project.

To get this barebones Pages app to work, we need:

an index.html file in whatever our build directory is
said index file to have the necessary Fulcro things – a div to attach to, and a script tag pointing at the generated js file

Generating the frontend assets

With this commit we’ve got a barebones Fulcro app that we can see locally via npx shadow-cljs server and for which we can generate a JS file via npx shadow-cljs compile :main or, to pass it through the Closure Compiler’s optimizations, npx shadow-cljs release :main.

After experimentation with the Cloudflare Pages build environment, I discovered that it has neither a JDK nor the Clojure binary, the first of which is necessary no matter what, and the second we can skip – if we want to – by defining our dependencies in just the shadow-cljs.edn file.

I’m not exactly sure how that’ll play out with my dev environment, but the docs seem to indicate that it should be fine. The dev process was:

First, move dependency management into the shadow-cljs.edn file
Next, try installing the jdk with apt, first without sudo and then with sudo, though neither of those worked out
Finally, I discovered that the JDK can work just from an archive file, so I downloaded it and (after fixing a typo), got a successful build!

Summary and next steps

So the build environment comes with the necessary versions of NPM/Yarn, we add a JDK via an archive file, and then Shadow CLJS does the rest. If we ever need or want to use deps.edn in our build process, we can likely download a binary for Clojure in much the same way we dl’ed the JDK.

Next up is:

Learning about how Functions work
Figuring out how to incorporate that into the build process
Figuring out how that fits into the dev experience

See you next time!

Pages overview

Mon, 24 Jul 2023 16:09:00 -0500

My core goal with this post is to break down how Pages, and Pages Functions, work together.

Pages

This link has a good overview of how to set up Pages for deployment. The two that interest me are the Git provider set up, and the wrangler cli. My default is to use the git provider configuration, as I’d like to deploy this on push to the main branch, but the wrangler cli may have some useful functionality to learn about.

There is also this link, which covers how to deploy anything with Cloudflare Pages. From this page, we get the following quote

Unlike many of the framework guides, the build command and build directory for your site are going
to be completely custom. If you do not need a build step, leave the *Build command* field empty and
specify a *Build output directory*. The build output directory is where your application's content
lives.

So to serve up frontend content, the core thing we need is some kind of asset(s) in the build output directory, and potentially we can use a build command if we’d like.

This begs the question: what needs to be in the build output directory for pages to work?

Looking at the output of the hugo command – specifically the files inside it’s output directory – my first though was that just an index.html was sufficient, but I saw a 404.html file, and didn’t see a reference to that in the index.html file, so now I’m not sure.

Luckily, perusing the Platform section of the Pages docs leads me to this page, which explains how Pages chooses to route requests to your assets.

This explains how the 404.html file is served by Pages, and also how the index.html file is served up (plus all the other files).

The non-SPA behavior is described as follows:

** Route matching
   If an HTML file is found with a matching path to the current route
   requested, Pages will serve it. Pages will also redirect HTML pages to
   their extension-less counterparts: for instance, =/contact.html= will be
   redirected to =/contact=, and =/about/index.html= will be redirected to =/about/=.

** Not Found behavior
   You can define a custom page to be displayed when Pages cannot find a
   requested file by creating a =404.html= file. Pages will then attempt to
   find the closest 404 page. If one is not found in the same directory as
   the route you are currently requesting, it will continue to look up the
   directory tree for a matching =404.html= file, ending in =/404.html=.
   This means that you can define custom 404 paths for situations like =/blog/404.html= and =/404.html=, and Pages will automatically render
   the correct one depending on the situation.

Since Fulcro is a React app, we’re probably going to make more use of how it handles SPAs, which is as follows:

If your project does not include a top-level =404.html= file, Pages assumes that you are deploying a
single-page application. This includes frameworks like React, Vue, and Angular. Pages' default
single-page application behavior matches all incoming paths to the root (=/=), allowing you to
capture URLs like =/about= or =/help= and respond to them from within your SPA.

So to get Pages to serve an SPA, we should exclude a 404.html file. I’m not exactly sure if there’s a way to override this at the moment, but can simply avoid creating a 404 file for the time being.

Functions

Now that we’ve got an overview of how to serve a custom frontend – i.e. one that doesn’t use a known framework – let’s take a quick overview of Functions to get a lay of the land.

The overview covers most of what we need to know.

There’s a /functions directory which contains js files
The filenames, sans-extension, define routes
The functions we define in these files have a defined API
We can alter the routing logic in some way, though I haven’t dug into how just yet
We can also choose an Advanced Mode to define our Functions with a _workers.js file instead of the /functions directory approach

As for how you tie the frontend and backend together, I suspect looking at the Routing docs, the API Reference docs, and the Middleware docs are the proper next things to look at. Routing should help us understand how to ensure FE requests go to the right function in the BE, the API ref will tell us how to actually build the BE functionality, and the middleware docs will tell us how to add things like authentication, as needed.

Next Steps

Two possible avenues open before us:

Figure out how to build a barebones Fulcro app with Cloudflare Pages
Dig into the aforementioned Functions docs

See you next time!

Cloudflare

Mon, 24 Jul 2023 13:39:00 -0500

To host this app, I’ll need the following things:

Something to host the static assets, e.g. the html/css/js.
Something to handle the state synchronization across clients

To host a site that would allow for multi-user alteration of the content, static assets alone won’t be enough, as we’ll need a way to synchronize our changes across users. While automerge has many ways to exchange changes – essentially anything that can faithfully transmit or store binary data – if we want an always-on node that can act as a post-office of sorts, we’ll need to build the code that takes the binary data and puts it on that node.

To that end, I want to explore the solutions Cloudflare has for storing and retrieving data.

Overview

I’m going to start with the assumption that we’ll use Cloudflare Pages to host the site. From this, it looks like there’s Pages Functions and Cloudflare Workers as options. The sites aren’t super clear on the differences, or when you’d use one over the other, but a quick look at the listed tutorials for both shows a chat demo for Workers and nothing similar for Functions.

That doesn’t mean Functions would be insufficient, but I wanted to note that as a strong signal that Workers is probably the right fit for what we’re looking for.

The demo uses Durable Objects – another Cloudflare tech – among some other things, so I suspect Durable Object will be in use no matter what.

What’s the difference between Functions and Workers?

The Functions overview page has this to say about Pages Functions

Pages Functions allows you to build full-stack applications by executing code on the Cloudflare network with Cloudflare Workers. With Functions, you can introduce application aspects such as authenticating, handling form submissions, or working with middleware. Use Functions to deploy server-side code to enable dynamic functionality without running a dedicated server.

My immediate read of this is that Functions is some more sugar on top of Workers, and is meant to offer something more than just what Workers can do, or simplify something about Workers, or perhaps is a name for integrating Workers into a Pages application. I’m not exactly sure, so we’ll keep digging.

Reading through the Functions getting-started doc, it’s sounding more and more like the primary functionality is that you add Workers function definitions in a special directory, and it can automagically sort out routing issues.

Luckily, by way of a semi-random click on a search link, I’ve found the following description from a refactoring how-to:

In this guide, you will learn how to refactor a Worker made to intake form submissions to a Pages Function that can be hosted on your Cloudflare Pages application. Pages Functions is a serverless function that lives within the same project directory as your application and is deployed with Cloudflare Pages. It enables you to run server-side code that adds dynamic functionality without running a dedicated server. You may want to refactor a Worker to a Pages Function for one of these reasons:

If you manage a serverless function that your Pages application depends on and wish to ship the logic without managing a Worker as a separate service.

If you are migrating your Worker to Pages Functions and want to use the routing and middleware capabilities of Pages Functions.

So that’s the summary I believe I was looking for. Before Pages Functions, I’d have to manage either the Workers service for both frontend and backend functionality, or both a Workers and a Pages service, but now I can have just a Pages service and ship server-side code that the Pages frontend code can use.

So we’ll use Pages and Functions to serve the frontend and backend functionality, but now the question is: where do we store the automerge data?

Looking at the developer platform products, it looks like we should use either Workers KV, or Durable Objects.

From the Workers KV frontpage, we’ve got the following description:

Eventually consistent by design, KV caches data globally and is ideal for reference data or assets that don’t change frequently.

Reference data or assets that don't change frequently doesn’t sound quite like what we’re going for, since changes could, theoretically, occur frequently.

On the other hand, Durable Objects has the following to say:

Collaborative apps, like chat rooms, games, or whiteboards, require strong consistency of state to ensure everybody is working with the most up to date version. Durable Objects enables you to build and run stateful applications on Cloudflare’s global network.

Which both sounds exactly like what we need, and also matches with what the chat demo for Workers referenced above uses.

Synthesis

So from all of this, it sounds like our Cloudflare stack will be:

Pages, for the frontend functionality
Pages Functions, for the backend/durability functionality
Durable Objects, for the data synchronization between clients

Next up

Now that we know the foundational infrastructure tech we’re going to use, we can layer on top of that by exploring one of the following (or even something I haven’t foreseen yet):

How Pages Functions work, specifically, so we can understand how we tie the frontend and backend together.
How to deploy a basic CLJS app – frontend only – to Pages
How to add basic backend CLJS capabilities via Pages Functions

See you next time!

AutoFlare Overview

Thu, 20 Jul 2023 13:38:00 -0500

I’ve got a project idea, which I’ve just now decided to call AutoFlare, that I’ve been rolling around in my head, primarily as a way to learn more about three technologies:

I’ve got a lot of experience with AWS and GCP, and there are plenty of things I like about those platforms, but I’d like to branch out, and some of the blog posts I’ve seen from Cloudflare around network egress costs and things has got me curious. I’d like to learn more about their various offerings, and play around with building a mildly complex application with just what they’ve got. (This blog is hosted by Cloudflare, as a static set of assets, which I’ve really enjoyed the simplicity of.)

I’ve got some experience with Fulcro, and, of all of the web frameworks I’ve used, it seems to be the most cohesively designed, where I don’t have to spend a bunch of time choosing and tweaking routing libraries, or storage solutions, etc. I’d like to really stretch my understanding of Fulcro with an app more complex than just the Fulcro book, thus why it’s part of this experiment.

As for Automerge CRDT, I’ve wanted to explore and learn this library since I first learned about it. I’m really motivated by the idea of building local-first apps, and this library seems like it’s solved a really hard problem related to building a local-first app. Also, I’ve got a vague sense that this might be useful in ~~luring~~ teaching JS devs about the joys of CLJS development.

This series of posts will be different forms and formats of exploring these technologies and building this small project. Posting order will be largely dependent on my whims, and what strikes me as the most interesting thing to look at in the moment.

See you in the next post!

Hello World

Mon, 26 Jun 2023 13:37:00 -0500

I hope to fill this blog with various explorations and musings.

Just Enough Software

Local-only architecture

Revisiting our end goal

Serving up Fulcro

Barebones Fulcro App

Generating the frontend assets

Summary and next steps

Pages overview

Pages

Functions

Next Steps

Cloudflare

Overview

What’s the difference between Functions and Workers?

Sharing document changes between clients

Synthesis

Next up

AutoFlare Overview

Hello World