Test Racket Package

Top-level Files of tip
Login

Top-level Files of tip

All File Ages Tree-View Trunk

Files in the top-level directory from the latest check-in

pluto

Testing Racket package deployment on Fossil

This will be on the a branch; it probably won't ever get merged.

Background:

The theory was that I could, via settings in the Fossil repo, add these URL aliases:

…and this would satisfy the requirements for a manually deployed Racket package, in a way that requires no manual deployment steps beyond syncing checkins from the local dev environment.

The tricky part turns out to be the .CHECKSUM. Fossil’s web interface/API doesn’t provide a way to get only the checksum of a particular checkin (such as trunk), and it also uses SHA-256 by default.

The /raw?name=trunk method returns the manifest of the current checkin. It is a much longer string than a typical checksum, but it is unique to the checkin.

The Racket docs define a checksum as

checksum — a string that identifies different releases of a package. A package can be updated when its checksum changes, whether or not its version changes. The checksum normally can be computed as the SHA1 (see openssl/sha1) of the package’s content.

The wording (emphasis added) seems to indicate a package’s checksum can be any unique string — i.e. it doesn’t have to be SHA1. On Discord Matthew Flatt was unsure, but seemed to think that it didn’t have to be SHA1.

Reality

I stood up this repo to test all of the above.

It turns out you do have to use the SHA1 hash as your package’s checksum if adding to the main package server, because raco pkg install manually checks using SHA1 specifically.

See this package’s page on the package server:

pkg: mismatched checksum on package
  package source: https://joeldueck.com/code/pluto/pluto.zip
  expected: "C Initial\\scommit\nD 2025-03-20T17:26:52.192\nF LICENSE-APACHE 147adf8d49ee07cdd6c6bc489be2b766d5a645b48e9d76bce77420164b93de93\nF LICENSE-MIT 6822c02131842dc38a7cf804155d0eed5a3a2846ac4f01fdc2961cd00a342165\nF README.md 1aaa21906f2af3b7688b63ebbcb59f...
  got: "610bdaf3cc73ed4a6f31bf12a0a549bbc4e62ad1"
  context...:

If I then try to install the package locally by looking it up on the package server, it fails with an empty checksum, presumably because the package server doesn't provide one with the package in this failed state:

❯ raco pkg install pluto 
Resolving "pluto" via https://download.racket-lang.org/releases/8.16/catalog/
Resolving "pluto" via https://pkgs.racket-lang.org
Downloading https://joeldueck.com/code/pluto/pluto.zip
raco pkg install: mismatched checksum on package
  package source: https://joeldueck.com/code/pluto/pluto.zip
  expected: ""
  got: "610bdaf3cc73ed4a6f31bf12a0a549bbc4e62ad1"

Attempting to install locally using the package’s direct zip URL results in the same failure as on the package server:

❯ raco pkg install https://joeldueck.com/code/pluto/pluto.zip
Downloading checksum for pluto
Downloading https://joeldueck.com/code/pluto/pluto.zip
raco pkg install: mismatched checksum on package
  package source: https://joeldueck.com/code/pluto/pluto.zip
  expected: "C Initial\\scommit\nD 2025-03-20T17:26:52.192\nF LICENSE-APACHE 147adf8d49ee07cdd6c6bc489be2b766d5a645b48e9d76bce77420164b93de93\nF LICENSE-MIT 6822c02131842dc38a7cf804155d0eed5a3a2846ac4f01fdc2961cd00a342165\nF README.md 1aaa21906f2af3b7688b63ebbcb59f...
  got: "610bdaf3cc73ed4a6f31bf12a0a549bbc4e62ad1"

Possible routes forward

Easy: light changes to raco pkg and Fossil

It seems the real use of .CHECKSUM by Racket is simply to detect package updates. In any case, it cannot actually protect against malicious activity1. Therefore, raco pkg could simply stop trying to validate the downloaded zip file against the checksum. This would bring its behavior in line with the permissive language of the documentation, and allow any unique string to be used as a checksum. It might require other updates to raco pkg in places where the checksum is displayed (such as in raco pkg show).

It would be nice if Fossil added a page like /raw that would return only the checksum of a named artifact. It would probably be SHA-256 rather than SHA1, but it would be friendlier than a manifest which is long and contains newlines, etc.

Medium: Teach Racket to parse Fossil manifests

This would correspond closely to the “remote URL naming a directory” scenario (fourth bullet in Racket Package Sources). A fossil+http:// or fossil+https:// URL without the --clone options would tell Racket to expect a Fossil manifest (a simple but consistent human-readable text format). Racket could parse and download each of the artifacts named in the manifest over HTTP or HTTPS, reconstructing the package as a local directory and then following the rules for local directory paths (which do not use checksums).

The raco pkg command could also potentially support the use of --clone with Fossil URLs, which would shell out to fossil to clone the repo2.

Hard: Implement Fossil sync/clone in Racket

One could create a Racket library that implements just enough of the Fossil sync protocol to clone a Fossil repository and reconstruct the files/contents of the most recent checkin. The package manager could use this to support package sources given as fossil+http:// or fossil+https:// URLs.

The Racket core team is understandably disinclined to support Fossil in a way that involves simply shelling out to fossil by default, since Fossil may not be available on the system.2 Everything in the “medium” option above would presumably satisfy their preferences in this respect, so it’s not clear to me what the advantage of the “hard” approach would be; I’m just listing it for the sake of completeness.

Give up: use scripts to publish package updates

Publish using a shell script that creates the zip file, creates the checksum file, and rsync or scp the file to a web server. This takes Fossil out of the picture completely and uses Racket’s “manual deployment” scenario the way it already works.

If this is the short-term solution, Racket’s docs should in any case be updated to reflect the fact that using SHA1 for the checksum is mandatory, not optional as is currently implied. (Update: opened PR #5231 for this purpose.)

Make users install and use Fossil

Give users instructions for installing Fossil (pretty simple) and cloning the repo manually, installing as a local linked directory.

  1. ^ Anyone who would be able to substitute the zip file, whether at rest or in-transit, would also be able to substitute the checksum file.
  2. ^ a b For comparison, raco pkg can download packages from any git source without git being installed, but git does need to be installed if the --clone option is used.