In the past year or so, every time I’ve wanted to use hot-reloading¹ for some service I’m writing, I’ve reached for a tool called reflex. This tool is not particularly original, but it is a reasonably good implementation, and it ships as a standalone-binary, which makes it very easy to move about.

It is also written in a language I have some familiarity with, and one that I’m trying to learn more about.

So of course I decided to implement my own.

The motivation was basically the same as the one that lead to HTTP::Tiny: a desire to understand how the code worked, and a search for real-world problems to attempt to solve using Raku. I might write more in a later post about the specifics of lorea, but in this post I want to focus on one aspect in particular, which I think showcases some of the strengths of Raku… and some of its weaknesses.

Some Go code to get going

One of the nice features of reflex is that it batches file-system changes to limit how often the command it needs to re-run gets executed. This is how that feature is explained in the documentation:

Part of what reflex does is apply some heuristics to batch together file changes. There are many reasons that files change on disk, and these changes frequently come in large bursts. For instance, when you save a file in your editor, it probably makes a tempfile and then copies it over the target, leading to several different changes. Reflex hides this from you by batching some changes together.

Simple enough.

Now let’s look at the implementation:

func (r *Reflex) batch(out chan<- string, in <-chan string) {
    const silenceInterval = 300 * time.Millisecond
    for name := range in {
        r.backlog.Add(name)
        timer := time.NewTimer(silenceInterval)
    outer:
        for {
            select {
            case name := <-in:
                r.backlog.Add(name)
                if !timer.Stop() {
                        <-timer.C
                }
                timer.Reset(silenceInterval)
            case <-timer.C:
                for {
                    select {
                    case name := <-in:
                        r.backlog.Add(name)
                    case out <- r.backlog.Next():
                        if r.backlog.RemoveOne() {
                            break outer
                        }
                    }
                }
            }
        }
    }
}

Yikes.

That block comes with a comment on top with these notes:

batch receives file notification events and batches them up. It’s a bit tricky, but here’s what it accomplishes:

• When we initially get a message, wait a bit and batch messages before trying to send anything. This is because the file events come in bursts.
• Once it’s time to send, don’t do it until the out channel is unblocked. In the meantime, keep batching. When we’ve sent off all the batched messages, go back to the beginning.

When I came across this piece of code, I had to re-read it several times, and cross-reference that to the comment, and read through a couple of pages of documentation before I could say I understood it. And now, to write this post, I had to spend a good couple of minutes more staring at it before I could remember how it worked.

In short, this is waiting on file-system changes to come in from a channel, and it adds them to a backlog as they come in. It continues to do this until there is a break between the incoming changes of at least 300 milliseconds, and it then processes them.

The key here is the interaction between the time.Timer to keep track of that time window, and the incoming and outgoing channels, which in Go are blocking. The nested loops are there to make it so that, if an event comes in while some other part of the system is running, it will also get processed.

Porting it to Raku

As it turns out, porting code from Go to Raku is normally quite painless. Like Go, Raku has superb support for asynchronous programming, and helper classes like channels are built-in.

However, when it came to porting this bit of code I ran into a bit of a complication: Raku has no equivalent to time.Timer.

For this particular case, the Go timer was being used to schedule an event that would trigger in the future, unless it was re-scheduled to happen further in the future via the timer.Reset call.

Now, while Raku comes with classes to represent moments in time, as well as events in the future, these are not particularly well suited for this case. For one, even if you can schedule an event in the future (eg. with Promise.new: $delay) this promise will take place. Even if you are not paying attention to it, any code you schedule will run.

While it does look like there is some intention of implementing cancellations for Promises in the future, we’re not there yet. So we’ve got to make do with what we have.

Luckily, Raku is a language that is built on the notion that you should have enough rope to shoot yourself in the foot, and that it does. Enter Timer::Breakable, which implements something alike to breakable promises, and my very own Timer::Stopwatch, which uses them to implement basically the same idea as the Go timer.

The resulting code

With that set aside, let’s look at how that Go code compares to the equivalent code in Raku:

method run ( --> Promise ) {
    use Timer::Stopwatch;
    my Timer::Stopwatch $timer .= new;

    start react {
        whenever $!supply {
            $timer.reset: 0.3;
            $!queue.add: .path;
        }
        whenever $timer { start self!process-queue }
    }
}

The beauty here is that, unlike in Go, I’m using a Supply which offers the same asynchronous guarantees as a channel with the benefit that reading and writing from it does not block, so the nested loops to make sure that we catch events that happen while we are waiting are gone.

Trade-offs, trade-offs

It’s possible that there are some subtle differences between the two versions of this code, but they are functionally identical, and I was happy to get rid of the added complexity to get code that I can understand at a glance.

It did highlight in my eyes one of the limitations in Raku, however: that the standard library, while huge in some respects, still lacks tools to do some of the things that you might argue are pretty basic.

Luckily, this kind of thing can be remedied by libraries and modules that extend the language, like the ones shown here. And as more of these real-world cases are explored, one can hope that cases like these, where we lack the tools to get the job done, become a rarity.

I’ll be happy to continue to contribute to that end.

1. “Hot reloading” is a feature made popular particularly by web frameworks wherein any change in the source code of the service will make the development version of the server reload.

   More generally, however, this can be applied to anything that will trigger some command when a file on disk has changed. And this is precisely what reflex does. ↩