;;; web-static.arc: module to deal with static files (CSS/JS)
; designed to leverage a reverse proxy to serve the files when in production
; currently specific to my own needs, will certainly always be.
; main requirements are: eternal caching when possible, CSS/JS
; minification, ability to correctly handle external files we can't
; easily control (think images url in CSS), ** minimal overhead **
; (i.e: code as fucking simple as fucking possible)
; CSS/JS minification: we use the YUI compressor and put the
; files in minified-dir*. nginx is told to look first
; in this directory and fallback to static-dir* if not found
; (for the files "out of control")
;
; eternal caching: we set a query string "?", and told
; nginx to inform the client to cache this URL for 1 year
; (1 year is max allowed by RFC and anyway sufficient)
;
; (wipe testing*) to activate minification and query string.
; doesn't matter if you do this while not being actually behind nginx,
; nothing will break, httpd.arc is still serving the static files correctly
(= static-dir* "res/static/"
minified-dir* "res/minified/"
static-path* "/static/" ; URL (not filesystem) path
code-compress-prog* "yuicompress" ; sh wrapper around yuicompressor.jar
testing* t)
(def sendfile (fname (o mt (mimetype fname)))
(resphead http-ok+ (copy httpd-hds* 'Content-Type mt))
(prfile fname))
(register-path (string "/" static-path* "/*") ; never reached in production
(fn (req file)
(aif (file-exists (string static-dir* "/" file))
(sendfile it)
(resp-err))))
(def static-url (fname)
(string static-path* fname
(when no.testing*
(+ "?" (mtime (compress-ifstale (+ static-dir* "/" fname)
(+ minified-dir* "/" fname)))))))
(defs csss (fname) (css:static-url fname)
jss (fname) (js:static-url fname))
(def compress-codefile (fsrc fdest)
(ensure-dir:dirname fdest)
(system (+ code-compress-prog* " " fsrc ">" fdest)))
(def compress-ifstale (fsrc fdest)
(when (and (in (file-ext fsrc) 'js 'css)
(or (~file-exists fdest) (> (mtime fsrc) (mtime fdest))))
(compress-codefile fsrc fdest))
fdest)
(defmemo compress-code (str (o type 'js))
(w/tmpname tmpf
(w/outfile s tmpf (disp str s))
(out-from code-compress-prog* " --type " type " < " tmpf)))
(with (_ijs ijs _icss icss) ; redef web.arc ones
(defs ijs (str) (_ijs (if testing* str (compress-code str)))
icss (str) (_icss (if testing* str (compress-code str 'css))))
)
;; todo:
; * X-Accel-Redir in 'sendfile if behind nginx.
; heuristic: look if X-Real-IP present. or make the proxy pass
; a header with its name to be more correct (X-Forwarded-By)
;
; * img-compress-prog* (`optipng')?
;
; * a clean way to do the call to `yuicompress' asynchronously
;
; * gzip here to not have nginx do it on-the-fly each time? not sure
; if the gain is that valuable
;
; * 'compress-code[...] bad names?
;
; * use GG Closure compiler and not YUI, use its REST API, and therefore
; be obliged to make it asynchronous
;
; * hash instead of mtime maybe.
;
; * like for web.arc, '=once macro or init procedure so that one can use a !=
; path without having to change the file.
Nginx config sample to use this with:
root /home//res/;
rewrite "/static/(.*)" "/minified/$1" break;
location /minified/ {
internal;
if (!-f $request_filename) {
rewrite "/minified/(.*)" "/static/$1" break;
}
if ($query_string) {
expires +1y;
}
}
location /static/ {
internal;
}
Obviously, using a reverse proxy makes the need of 'setuid irrelevant (it is such a low-level syscall anyway. even plain old unix daemons should use the daemontools and don't do this by themselves). Nginx could be made to keep-alive and gzip, which are huge perf wins. Not serving the static files by the app server is so obvious, even news.ycombinator.com does this know.
The manual "wait 30 seconds, then kill the 'slow' client" handling of srv.arc is a crappy solution (but the crappy threading model asks for it): sometimes my wifi connection is so slow, I couldn't finish a POST to this forum (yes it happened for real, I should retried each POST several times). A reverse-proxy, by buffering and handling slow clients in the good manner (i.e: not killing them brutally: if they don't write for some time, it's OK it's just an idle fd in the select() poll) removes this problem.
----
Old version of web.arc (then called wf.arc) that does session and login handling: http://pastebin.com/3amqH2h8
I'll try to post an example of a login procedure as I do it with this combo next week, but like for "files.arc", I don't have access to it right now.
----
Clickable links: http.arc: http://pastebin.com/jiXSX8yV , web.arc: http://pastebin.com/9GmhRWqc
----
An nginx basic config file for proxying to an http/web.arc powered app server (add the previous sample in the server block if you use web-static.arc):
server {
listen example.com:80;
location / {
access_log /var/log/nginx/examplecom-access.log;
proxy_pass http://localhost:8080;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass_header Server;
}
}