(no subject)

[mdlbear]

Went to a Heather Alexander this evening in Palo Alto. Emmy was reluctant to go, but seemed to enjoy the evening.

Spent about half my day at work bringing our experimental server in Japan up to date, and the other half scheming about caching web servers.

Comments

Caching web servers?

[aerowolf.livejournal.com]

I'm actually curious about caching personal web proxies with search engine capabilities and cache-browsing and -tagging features.

Re: Caching web servers?

[mdlbear]

Last thing I did in that space can be found at http://www.risource.org/PIA/ -- at that time the main purpose was maintaining a common history across multiple browsers. It also had a cache, and the plan was to eventually use it for searching and for figuring out what changes had been made in a page since the last time it was visited.

One could also write other little "agents" that tapped into the request-and-response stream and could do things like filtering. The ad blocker was popular.

I've since stopped using it because the performance was pretty bad -- it was written in Java.

Re: Caching web servers?

[aerowolf.livejournal.com]

Aye... Java is a not-wonderful daemon language.

But what I'm interested in is, for example, a means of figuring out what resources have been viewed, as well as the history of those resources. (Among other things, for academic and job-related research.) To be able to see the tree of browsing habits (using the referer string), as well as cache the data on the page for X amount of time until it's deemed unimportant enough to trash...

Thanks for the pointer!

Re: Caching web servers?

[mdlbear]

Yes, you can do all that pretty easily with a caching proxy. If you want decent performance and total standards compliance, I'd recommend hacking mod_proxy and mod_cache in Apache -- those were the problems that always plagued the PIA.

There are also security issues if your cache is visible to anyone but the user. Not so bad if it's just for research.

Re: Caching web servers?

[aerowolf.livejournal.com]

Isn't that what proxy authorization is for? ;)

My understanding is that there's a sourceforge project to take the PIA and turn it into an apache module, written in C?

(I was thinking about trying to abuse squid's caching and logging mechanism, but... bleh.)