3 Feed Constructs
| (require splitflap/constructs) | package: splitflap-lib |
3.1 Tag URIs
Feeds, and items contained in feeds, require some globally unique identifier. Although any kind of reasonably unique identifier can be used in a feed, Splitflap takes the unreasonably opinionated stance of allowing only tag URIs, which are easy to create and read, and which can remain stable even if the resource’s URL changes.
A tag URI is an identifier of the form tag:‹authority›,‹date›:‹specific›. The ‹authority› is a domain name (or email address) held by you as of ‹date›; together, the authority and the date form a unique tagging entity, which acts kind of like a namespace. The ‹specific› is a string uniquely identifying a particular resource (e.g., a page or file) within the tagging entity.
The tag URI scheme is formalized in RFC 4151.
procedure
(mint-tag-uri authority date specific) → tag-uri?
authority : tag-authority? date : tag-entity-date? specific : tag-specific-string?
The authority must be a domain name or email address in lowercase. (See tag-authority?.)
The date must be any date on which you had ownership or assignment of the domain or email address at 00:00 UTC (the start of the day). (See tag-entity-date?.)
The specific is a string that must be reliably and permanently unique within the set of things that your feed is serving. See tag-specific-string? for information about what characters are allowed here.
> (mint-tag-uri "rclib.example.com" "2012-04-01" "Marian'sBlog") #<tag-uri "tag:rclib.example.com,2012-04-01:Marian'sBlog">
> (mint-tag-uri "diveintomark.example.com" "2003" "3.2397") #<tag-uri "tag:diveintomark.example.com,2003:3.2397">
Changed in version 1.4 of package splitflap-lib: The authority must now satisfy tag-authority?.
procedure
(tag-uri->string tag) → non-empty-string?
tag : tag-uri?
> (define rclib-id (mint-tag-uri "rclib.example.com" "2012-04-01" "Marian'sBlog")) > (tag-uri->string rclib-id) "tag:rclib.example.com,2012-04-01:Marian'sBlog"
procedure
(append-specific tag suffix) → tag-uri?
tag : tag-uri? suffix : tag-specific-string?
> (define kottke-id (mint-tag-uri "kottke.example.com" "2005-12" "1")) > kottke-id #<tag-uri "tag:kottke.example.com,2005-12:1">
> (append-specific kottke-id "post-slug") #<tag-uri "tag:kottke.example.com,2005-12:1.post-slug">
The tag URI spec defines tags as being equal when their byte-strings are indistinguishable.
Returns #t if the tag-uri->string representation of tag1 and tag2 are equal?, #f otherwise.
procedure
(tag-authority? v) → boolean?
v : any/c
It contains no uppercase letters. RFC 4151 recommends lowercase, and the W3C Feed Validator rejects tag URIs that do not follow this recommendation.
If it is an email address, the part before the @ includes only a–z, 0–9, or characters in the set -._, as RFC 4151 requires.
> (tag-authority? "rclib.example.com") #t
> (tag-authority? "marian@rclib.example.com") #t
> (tag-authority? "RCLib.example.com") #f
> (tag-authority? "marian+blog@rclib.example.com") #f
Added in version 1.4 of package splitflap-lib.
procedure
(tag-entity-date? str) → boolean?
str : string?
; Equivalent to January 1, 2012 > (tag-entity-date? "2012") #t
; Equivalent to June 1, 2012 > (tag-entity-date? "2012-06") #t
; take a guess on this one > (tag-entity-date? "2012-10-21") #t
> (tag-entity-date? "2012-1-1") #f
procedure
(tag-specific-string? str) → boolean?
str : string?
> (tag-specific-string? "abcdABCD01923") #t
> (tag-specific-string? "-._~!$&'()*+,;=:@/?") #t
> (tag-specific-string? "") #t
> (tag-specific-string? "^") #f
procedure
str : string?
> (tag-specific-string? "my blog") #f
> (normalize-tag-specific "my blog") "my-blog"
> (tag-specific-string? (normalize-tag-specific "my blog")) #t
> (tag-specific-string? " tra^^shy\\` GarB§ºage ###") #f
> (normalize-tag-specific " tra^^shy\\` GarB§ºage ###") "tra-shy-GarB-age"
> (tag-specific-string? (normalize-tag-specific " tra^^shy\\` GarB§ºage ###")) #t
3.2 Persons
procedure
name : non-empty-string? email : email-address? url : (or/c valid-url-string? #f) = #f
The Atom 1.0 and RSS 2.0 specs both have opinions about how people should be referenced in feeds. Atom requires only a name but also allows up to one email address and up to one URI. RSS requires one email address optionally followed by anything. So person requires both a name and an email, and the url is optional.
procedure
(person->xexpr p entity dialect) → txexpr?
p : person? entity : symbol? dialect : (or/c 'rss 'atom 'itunes)
> (define frank (person "Frankincense Pontipee" "frank@example.com")) > (person->xexpr frank 'author 'atom) '(author (name "Frankincense Pontipee") (email "frank@example.com"))
> (person->xexpr frank 'contributor 'atom) '(contributor (name "Frankincense Pontipee") (email "frank@example.com"))
> (person->xexpr frank 'author 'rss) '(author "frank@example.com (Frankincense Pontipee)")
> (person->xexpr frank 'itunes:owner 'itunes)
'(itunes:owner
(itunes:name "Frankincense Pontipee")
(itunes:email "frank@example.com"))
3.3 Date and time information
Feeds and feed items must be timestamped, and these values must include timezone information.
Splitflap leans on the gregor library for this functionality —
procedure
(infer-moment [str]) → moment?
str : string? = ""
If str is "", then the result of now/moment is returned. Otherwise str must be in the form "YYYY-MM-DD [hh:mm[:ss]]" or an exception is raised. If the seconds are ommitted, 00 is assumed, and if the hours and minutes are ommitted, 00:00:00 (the very start of the date) is assumed.
> (infer-moment "2012-08-31") #<moment 2012-08-31T00:00:00-05:00[America/Chicago]>
> (infer-moment "2012-08-31 13:34") #<moment 2012-08-31T13:34:00-05:00[America/Chicago]>
> (infer-moment "2015-10-02 01:03:15") #<moment 2015-10-02T01:03:15-05:00[America/Chicago]>
> (parameterize ([current-timezone -14400]) (infer-moment "2015-10-02 01:03:15")) #<moment 2015-10-02T01:03:15-04:00>
> (infer-moment "2012-09-14 12") #<moment 2012-09-14T00:00:00-05:00[America/Chicago]>
> (infer-moment) #<moment 2026-09-15T19:46:19.500291016-05:00[America/Chicago]>
Changed in version 1.2 of package splitflap-lib: Added no-argument form for current moment
procedure
(moment->string m dialect) → non-empty-string?
m : moment? dialect : (or/c 'atom 'rss)
> (define m1 (infer-moment "2012-10-01")) > (moment->string m1 'atom) "2012-10-01T00:00:00-05:00"
> (moment->string m1 'rss) "Mon, 1 Oct 2012 00:00:00 -0500"
> (parameterize ([current-timezone 0]) (moment->string (infer-moment "2012-10-01") 'atom)) "2012-10-01T00:00:00Z"
3.4 Enclosures and MIME types
An enclosure is an arbitrary resource related to a feed item that is potentially large in size and may require special handling. The canonical example is an MP3 file containing the audio for a podcast episode.
struct
url : valid-url-string? mime-type : (or/c non-empty-string? #f) size : exact-nonnegative-integer?
The mime-type, if provided and not set to #f, must be a useable MIME type, but is not currently validated to ensure this. The size should be the resource’s size in bytes.
This struct qualifies as food, so it can be converted to XML with express-xml.
procedure
(file->enclosure file base-url) → enclosure?
file : path-string? base-url : valid-url-string?
This procedure accesses the filesystem; if file does not exist, an exception is raised.
; Make a temporary file > (define audio-file (make-temporary-file "audio-~a.m4a")) > audio-file #<path:/var/folders/zx/hsykzj7s7bg8x3ql9k5s18380000gn/T/audio-17895195791789519579508.m4a>
> (display-to-file (make-bytes 100 66) audio-file #:exists 'truncate) ; Pass the temp file to an enclosure
> (display (express-xml (file->enclosure audio-file "http://example.com") 'atom)) <link rel="enclosure" href="http://example.com/audio-17895195791789519579508.m4a" length="100" type="audio/mp4" />
; Cleanup > (delete-file audio-file)
value
This table is built directly from the list maintained in the Apache httpd repository.
A hash table mapping file extensions (in lowercase symbol form) to MIME types.
> (hash-ref mime-types-by-ext 'epub) "application/epub+zip"
Changed in version 1.3 of package splitflap-lib: Now a plain hash rather than a
promise.
Changed in version 1.4: The table is loaded the first time it is used.
procedure
(path/string->mime-type path) → (or/c string? #f)
path : path-string?
> (path/string->mime-type ".m4a") "audio/mp4"
> (path/string->mime-type "SIGIL_v1_21.wad") "application/x-doom"
> (path/string->mime-type "mp3") ; No period, so no file extension! #f
3.5 Domains, URLs and email addresses
procedure
(dns-domain? v) → boolean?
v : any/c
Must contain one or more labels separated by .
Each label must consist of only the characters A–Z, a–z, 0–9, or -.
Labels may not start or end with a hyphen.
Labels with -- in the third and fourth positions must be valid A-labels (see Internationalized domain names). If any A-label contains right-to-left text, every label must satisfy the Bidi rule of RFC 5893.
The last label may not consist entirely of digits (see RFC 3696), so IPv4 addresses never qualify.
No individual label may be longer than 63 bytes, and the entire domain may not be longer than 253 bytes. (RFC 1035 sets the limit at 255 bytes, but that count includes a length byte before each label and a zero byte for the root label.)
> (dns-domain? "a") #t
> (dns-domain? "rclib.org") #t
> (dns-domain? "a.b.c.d.e-f") #t
> (dns-domain? "a.b1000.com") #t
> (dns-domain? "1.example.com") #t
> (dns-domain? "192.0.2.16") #f
> (dns-domain? "xn--bcher-kva.example") #t
> (dns-domain? "bücher.example") #f
> > (define longest-valid-label (make-string 63 #\a))
> (define longest-valid-domain (string-append longest-valid-label "." longest-valid-label "." longest-valid-label "." (make-string 61 #\a))) > (string-length longest-valid-domain) 253
> > (dns-domain? longest-valid-label) #t
> (dns-domain? longest-valid-domain) #t
> (dns-domain? (string-append longest-valid-label "a")) #f
> (dns-domain? (string-append longest-valid-domain "a")) #f
Changed in version 1.4 of package splitflap-lib: Labels may now start with a digit. The length limits are now 63 bytes per label and 253 bytes overall (previously 62 and 254). The empty string no longer qualifies. Labels with -- in the third and fourth positions must now be valid A-labels.
procedure
(valid-url-string? v) → boolean?
v : any/c
> (valid-url-string? "http://rclib.example.com") #t
> (valid-url-string? "telnet://rclib.example.com") #t
> (valid-url-string? "gonzo://example.com") ; scheme need not be registered #t
> (valid-url-string? "https://user:p@example.com:8080") ; includes user/password/port #t
> (valid-url-string? "file://C:\\home\\user?q=me") ; Look, you do you #t
> ; Valid URIs but not URLs: > (valid-url-string? "news:comp.servers.unix") ; no host given, only path #f
> (valid-url-string? "http://subdomain-.example.com") ; invalid label #f
> > ; Valid URLs but not allowed by this library for use in feeds > (valid-url-string? "ldap://[2001:db8::7]/c=GB?objectClass?one") ; Host is not a DNS domain #f
> (valid-url-string? "telnet://192.0.2.16:80/") ; ditto #f
> (valid-url-string? "https://bücher.example/") ; not ASCII; see url-string->ascii #f
Changed in version 1.4 of package splitflap-lib: Follows the changes to dns-domain?. In particular, URLs with an empty host no longer qualify. URLs must now contain only ASCII characters.
procedure
(url-domain u) → dns-domain?
u : valid-url-string?
This function is convenient for mint-tag-uri (which needs a valid dns-domain?) when you already have a base URL for the site.
> (url-domain "http://example.com") "example.com"
> (url-domain "https://user:p@example.com:8080/path/to/file") "example.com"
procedure
(url-join base rel) → valid-url-string?
base : valid-url-string? rel : relative-path?
This is a convenient front-end to Racket’s combine-url/relative and relative-path->relative-url-string.
> (url-join "http://example.com" "path/to/my file.html") "http://example.com/path/to/my%20file.html"
> (url-join "http://example.com/" "path/to/resource") "http://example.com/path/to/resource"
> (url-join "http://example.com" "/absolute/path") url-join: contract violation
expected: relative-path?
given: "/absolute/path"
in: the 2nd argument of
(->
valid-url-string?
relative-path?
valid-url-string?)
contract from:
<pkgs>/splitflap-lib/constructs.rkt
blaming: program
(assuming the contract is correct)
at: <pkgs>/splitflap-lib/constructs.rkt:18:11
procedure
(email-address? v) → boolean?
v : any/c
Must be in the format ‹local-part›@‹domain›
The ‹local-part› must be no longer than 64 bytes and only include a–z, A–Z, 0–9, or characters in the set !#$%&'*+/=?^_`{|}~-..
The ‹local-part› may not start or end with ., and may not contain two . characters in a row.
The ‹domain› must be valid according to dns-domain?.
The entire email address must be no longer than 254 bytes.
> (email-address? "test-email.with+symbol@example.com") #t
> (email-address? "#!$%&'*+-/=?^_{}|~@example.com") #t
> (email-address? "marian..paroo@example.com") #f
> ; See also dns-domain? which applies to everything after the @ sign > (email-address? "email@123.123.123.123") #f
> (email-address? "λ@example.com") #f
Changed in version 1.4 of package splitflap-lib: Fixed a bug that accepted any local part containing at least one allowed character and rejected local parts with no lowercase letters. The allowed characters now include ` (as in RFC 5322) instead of ‘. Local parts may no longer start or end with a period or contain two periods in a row.
procedure
(validate-email-address addr) → string?
addr : string?
> (validate-email-address "marian@rclib.example.com") "marian@rclib.example.com"
> (validate-email-address "@") validate-email-address: domain is missing
domain: ""
in: "@"
> (validate-email-address "me@myself@example.com") validate-email-address: address must not contain more than
one @ sign
address: "me@myself@example.com"
in: "me@myself@example.com"
> (validate-email-address ".marian@rclib.example.com") validate-email-address: local part must not start with a
period
local part: ".marian"
in: ".marian@rclib.example.com"
> (validate-email-address "marian..paroo@rclib.example.com") validate-email-address: local part must not contain two
periods in a row
local part: "marian..paroo"
in: "marian..paroo@rclib.example.com"
> (validate-email-address "λ@example.com") validate-email-address: local part may only include a–z,
A–Z, 0–9, or !#$%&'*+/=?^_`{|}~-.
local part: "λ"
in: "λ@example.com"
> (validate-email-address "lambda@example..com") validate-email-address: domain must be a valid RFC 1035
domain name
domain: "example..com"
in: "lambda@example..com"
Changed in version 1.4 of package splitflap-lib: Now accepts and rejects exactly the same addresses as email-address?. Previously it rejected local parts containing uppercase letters.
3.5.1 Internationalized domain names
Domain names, URLs and email addresses in feeds must be ASCII. Use the functions in this section to convert internationalized domain names to their ASCII form, in which each label that contains non-ASCII characters is replaced by its A-label: the prefix xn-- followed by the Punycode encoding of the label (so "bücher" becomes "xn--bcher-kva").
Splitflap follows IDNA2008 strictly:
Each label must pass the IDNA2008 registration checks: it must be in Unicode Normalization Form C, may not start with a combining mark or contain characters that IDNA2008 disallows, and must satisfy the contextual rules of RFC 5892. If any label contains right-to-left characters, every label must satisfy the Bidi rule of RFC 5893.
Nothing is mapped: for example, uppercase letters are not converted to lowercase, and the ideographic full stop 。 is not treated as a label separator. When a domain name fails only because it needs mapping of this kind, the exception message suggests the mapped form.
Character properties come from IANA’s IDNA tables for Unicode 12.0.0, the most recent version IANA has published. Characters added to Unicode after version 12.0.0 are rejected.
Labels that are already ASCII are never changed (but they are still checked).
These rules ensure that a given domain name always converts to the same A-labels. This matters for tag URIs, which must never change once published: the older IDNA2003 standard and the “transitional” processing in UTS #46 convert "faß.de" to "fass.de", for example, while IDNA2008 produces "xn--fa-hia.de".
procedure
(domain->ascii domain) → dns-domain?
domain : string?
> (domain->ascii "bücher.example") "xn--bcher-kva.example"
> (domain->ascii "faß.de") "xn--fa-hia.de"
> (domain->ascii "例え.テスト") "xn--r8jz45g.xn--zckzah"
> (domain->ascii "rclib.example.com") "rclib.example.com"
> (domain->ascii "Bücher.example") domain->ascii: label contains a character that IDNA2008 does
not permit
label: "Bücher"
character: "U+0042 B"
in: "Bücher.example"
suggestion: "bücher.example"
Added in version 1.4 of package splitflap-lib.
procedure
(url-string->ascii url) → valid-url-string?
url : string?
> (url-string->ascii "https://bücher.example/straße?q=ü") "https://xn--bcher-kva.example/stra%C3%9Fe?q=%C3%BC"
> (url-string->ascii "https://rclib.example.com/my%20file.html") "https://rclib.example.com/my%20file.html"
Added in version 1.4 of package splitflap-lib.
procedure
(email-address->ascii addr) → email-address?
addr : string?
> (email-address->ascii "marian@bücher.example") "marian@xn--bcher-kva.example"
> (email-address->ascii "marían@example.com") email-address->ascii: local part may only contain ASCII
characters
local part: "marían"
in: "marían@example.com"
Added in version 1.4 of package splitflap-lib.
3.6 Language codes
> (force system-language) 'en
procedure
v : any/c
> (iso-639-language-code? 'fr) #t
> (iso-639-language-code? 'FR) #f
value