From Scripts to a Resident Service: How I Built the HHY Web Runtime

In the previous two articles, I wrote about optimizing the AST interpreter and the Bytecode VM. At that point, the question I cared about was whether the same HHY program could run faster while preserving identical semantics.

With the Web Runtime, the question changed.

When a script finishes and the process exits, the operating system cleans up many lifecycle problems for you. A service, however, must stay alive: the ten-thousandth request must not read data left behind by the previous request, one handler exception must not bring down the entire process, and a slow-reading client must not cause the service to accumulate output without limit.

That is why my starting point for HHY Web was making the Runtime withstand repeated invocation. Only after that was true did routing, JSON APIs, and a framework have somewhere solid to stand.

The HHY v1.4.3 release verification includes two sets of numbers that matter to me: 100,000 repeated invocations in the same loaded context; and 1,000,000 local HTTP requests completed by 16 concurrent clients with zero failures in 131.439 seconds, or about 7,608.1 requests per second. The first validates resident invocation; the second validates a specific short-request workload. Neither is a performance promise for every web scenario. Release notes

First, Turn the Interpreter into a Runtime That Can Be Called Long-Term

I divided the v1.4 work into four capability stages and delivered them together in the final v1.4.3 release. These versions mark development boundaries; they do not mean users need to install four versions in sequence.

This order matters to me. If the initial focus is only GET /hello, it is easy to turn every request into a fresh script execution. The endpoint can return JSON, but process startup, source loading, and execution-environment initialization all end up in the request path.

The resident model moves that fixed work into the startup phase, after which requests repeatedly invoke a handler. The public embedding API uses HhyApplication and HhyContext to hide internal representations. The host passes arguments and retrieves results across a JSON boundary instead of depending directly on HHY's Value memory layout.

An application must outlive the contexts it creates, and a context cannot be called concurrently by multiple threads without coordination. These lifecycle constraints matter more than whether the API names look concise.

The web service likewise reuses prepared application state. AST and Bytecode still share the language semantics; Bytecode is the default engine, while AST continues to provide differential verification. Top-level mutable bindings are rejected in an embedding context so global variables cannot become implicit cross-request state. A handler exception returns 500, and later requests continue to be served. Runtime design

I Split the Runtime and Framework into Two Layers

HHY Language's built-in import web handles requests, routing, responses, Streams, and Workers. HHY Web 0.1.0, in a separate repository, is a layer of ordinary HHY code that requires language version v1.4.3 or later.

The framework adds application organization: a default Request ID, optional CORS and gzip, JSON errors, Bearer authentication, and more direct routing functions. The network stack and Runtime do not need to be implemented again.

For example, a Blueprint here is simply a function that accepts an application and returns an application:

fn api(application) {
    return application
        |> hhyweb.get("/api/books/:id", book)
        |> hhyweb.post("/api/books", create_book)
}

Calling that function from mount is enough. Module composition continues to use HHY's existing functions and Flow, without introducing another registration syntax. The framework's real value is reducing the repeated code every project would otherwise need while keeping execution readable directly from the source. HHY Web source

How an HTTP Request Reaches a Handler

Start by breaking down an ordinary request:

GET /api/books/42?lang=zh HTTP/1.1
Host: example.com
Accept: application/json

That first line contains three different dimensions: GET is the method, /api/books/42 is the path, and lang=zh is the query string. They ultimately affect route selection and handler input in different ways.

What the lower layer receives is not a "request object," but segments of TCP bytes. A single recv may not contain the complete request headers, or it may include part of the body as well. The parser must identify message boundaries itself.

The current C Server first accumulates data, looks for \r\n\r\n, and then separates the request line from the Headers. It checks the method token, a request target beginning with /, and the HTTP/1.1 version, then separates the path from the raw Query.

The request-body length is determined by Content-Length. The value must parse as a valid number and remain below the configured max_body; if the body is incomplete, reading continues. Exceeding the body limit returns 413, a read timeout returns 408, and malformed input returns 400. The network layer handles these boundaries before execution enters HHY business code.

One detail is easy to misunderstand: the current service supports chunked responses, but does not accept request bodies with Transfer-Encoding. Request and response capabilities are not symmetrical. An upload endpoint cannot assume it accepts chunked uploads just because the service supports streaming output.

Inside the Runtime, a request becomes a WebRequest: route parameters go into params, decoded query parameters into query_params, alongside headers, cookies, the text body, and binary bytes. After matching /api/books/:id, for example, 42 becomes request.params.id.

The Router must consider both method and path. No matching path is 404; an existing path with the wrong method is 405. Middleware can return a response early, such as rejecting a request with missing credentials, and calls the handler only after the request passes. Finally, the Runtime assembles the response, the C Server writes the Header and Body, and the connection ends. HTTP Server implementation

The Difference Between GET and POST Is More Than Where Parameters Go

When developing an API, I prefer to choose the method by asking what operation this request expresses.

"Safe" here means the method semantics do not request a resource modification; it does not mean the endpoint is authenticated or its data encrypted. "Idempotent" means repeating the same request is expected to have the same effect; it does not require an identical status code and response every time. A DELETE that succeeds the first time and reports a missing resource the second time can still satisfy idempotent semantics.

Business code must uphold these semantics. Registering a charge operation under GET does not make it safe because get was used. If POST must support reliable retries, the application needs an idempotency key or deduplication mechanism. HTTP method semantics, PATCH semantics

Query is not exclusive to GET; a POST URL can also contain a Query. JSON, forms, and binary data are request-body formats and cannot be inferred from the method name alone. A GET body has no generally defined semantics and practical APIs should avoid depending on it.

HHY Web's JSON helper is direct: it runs parse_json on request.body and turns a parse failure into a stable 400 JSON error. It does not perform business-field validation automatically, nor does this helper enforce Content-Type. An endpoint requiring strict media-type constraints should add that check at the application layer.

The create-book example in the repository demonstrates this boundary:

fn create_book(request) {
    let parsed = hhyweb.request_json(request)
    if not parsed.ok {
        return parsed.error
    }
    return hhyweb.created({ id: "new-book", book: parsed.value })
}

It demonstrates JSON parsing and a 201 response; it does not implement database persistence. In a real business integration, the next steps are field validation, storage, and returning the created resource.

HEAD and OPTIONS also deserve separate consideration. The former asks for response metadata without transferring the response body; the latter asks about communication options and is used by browser CORS preflight requests. The HHY Runtime provides CORS preflight support, but the current thin framework exposes the five routing helpers in the table above. The HTTP standard does not imply that every method has a framework function with the same name.

The Value of Streams Is Controlling Production Speed

An ordinary JSON response is generally serialized first and then sent as a Body of known length. When exporting a large amount of data or continuously pushing events, this increases both waiting time and memory use.

web.stream turns output into a chunked response. Its key constraint is that it requests the next item from the Stream only after the previous chunk has been written. Client read speed therefore influences server production speed through socket write pressure.

That is backpressure in this design. It prevents the network output side from prefetching data without limit, but if the application first creates a huge array and only then converts it to a Stream, the earlier memory cost still exists.

SSE adds an event format on top of a Stream and is suited to one-way notifications and incremental output. Static files also support streaming large files and a single byte Range; cache validation uses ETag / Last-Modified to reduce unnecessary entity transfer.

These mechanisms address different costs: Stream controls output accumulation, Range narrows the read range, and cache validation reduces repeated transfer. They cannot be collapsed into the claim that "streaming is supported, therefore performance is higher."

What One Million Requests Actually Demonstrate

The load-test results in the official release record are below. This cites existing release evidence; the test was not rerun for this article.

The load-test script provides the context needed to interpret these numbers. A Python client uses a thread pool; each request creates a new HTTP connection, sends GET /, reads the response, checks the status code and JSON content, and then closes the connection. The service comes from an acceptance fixture configured with two Workers, with access logs disabled during the test. The measurement therefore covers a short-request path including connection setup, protocol handling, the handler, JSON response generation, and client-side validation. Load-test script, service fixture

This is closer to a service than benchmarking an empty function, but it still does not cover database waits, public-network latency, TLS, complex business logic, large request bodies, or slow SSE clients. The Python load generator may also be one of the constraints.

The release record does not include the machine model, CPU utilization, RSS curve, or p95 / p99 latency, so I will not use it to rank HHY against Go, Node.js, or other frameworks. Nor can 1 / RPS be treated as average latency for a single request: this is a concurrent workload.

The approximately 2.7x gain in the previous Bytecode article came from a specific CPU Flow workload and likewise cannot be applied directly to HTTP. Total web-request time also includes socket reads and writes, parsing, allocation, routing, and response encoding. Resident loading removes fixed overhead; Bytecode shortens part of the execution path; the actual bottleneck still has to be measured layer by layer.

The Benefits and Limits of Multiple Workers Both Need to Be Explicit

The current source uses a prefork model: multiple Worker processes share the listening socket, while the parent monitors and replaces Workers that exit. After accepting a connection, each Worker synchronously completes the read, handler execution, and response write before accepting another connection.

There is a select call in the source, but it waits on the listening socket. It does not mean a non-blocking event loop has been implemented across all client connections.

The current response also explicitly uses Connection: close. In other words, this version's performance shape is multi-process, synchronous per-Worker handling, and short-lived connections. Adding Workers can increase concurrent handling capacity, but it does not automatically turn the service into a massive-connection scheduler; a connection that reads slowly or remains active for a long time still occupies one Worker.

This also explains why "supports SSE" and "can sustain a large number of online SSE connections" are two different stages. Backpressure controls output accumulation, but it does not eliminate connection occupancy. Supporting long-lived connections at scale requires further design for non-blocking I/O, connection scheduling, timeouts, and coordinated cancellation, followed by verification under a mixed slow-client workload.

The current implementation exposes another cost: for each connection, the receive buffer allocates capacity according to the configured Header and Body limits. max_body is therefore both an input boundary and an influence on allocation size; it cannot be increased without limit for convenience. Whether this should move to progressive allocation later must be decided from allocation hot spots and memory measurements.

I would first add measurements for tail latency, Worker occupancy, and RSS changes under a mixture of short requests and long-lived connections, then decide whether to optimize route lookup, allocation strategy, or the connection model. Optimization has direction only after the time-consuming layer is identified.

Putting the Service in Front of a Real Browser

The live Dashboard is a direct entry point into this path: an HHY handler returns HTML, and the browser then calls /api/status and /api/hello to display JSON.

It is useful for confirming that the page, routing, Query, and Request ID path is connected. However, the Runtime version, engine name, and Worker count in the example source are explicitly written display fields; those cards must not be treated as live process probes. Real operating status still requires health endpoints, logs, metrics, and process monitoring. Dashboard source

The production boundary remains explicit: an HHY service listens on an internal address, while Nginx, Caddy, or a cloud load balancer terminates TLS / HTTP/2; v1.4 does not include WebSocket. Enable trust_proxy only when every directly connected client comes from a trusted proxy.

During development, I care more about behavior after failure. Does invalid JSON return 400? Does an oversized Body return 413? Can the next request succeed after a handler throws? Are temporary upload files cleaned up on both success and failure paths? These facts say more about whether a resident service truly works than a home page displaying "server running."

From Executing a Program to Continuously Receiving Requests

After completing the Web Runtime, my expectations for HHY became concrete at another level.

Originally, Flow connected files, processes, HTTP clients, and Streams into a task. Now the same language capabilities can live inside a resident handler and provide a continuously running service. The framework stays thin, while the Runtime handles lifecycle, error isolation, and resource boundaries.

One million short requests with zero failures is one piece of evidence along this path. Synchronous Workers, short-lived connections, and long-stream occupancy also define the next set of questions clearly.

A Runtime can serve reliably over the long term not only because successful requests are fast enough, but because failure, waiting, and resource release all have a defined destination.

That is the capability I truly wanted to complete when moving from scripts to the web.

References

• HHY Language: Web Runtime Guide
• HHY Web Live Demo
• HHY Web Framework Repository
• HHY v1.4.3 Release and Verification Record
• HTTP Semantics - RFC 9110
• PATCH Method - RFC 5789