2005/02/07
IRC [00:01] <Dossy> Well, at least it's been an /interesting/ weekend on the AOLserver mailing list.
IRC [00:06] <holycow> hi dossy
IRC [00:06] <holycow> how so?
IRC [00:06] * holycow thinks he should join
IRC [00:08] <Dossy> heh. yeah, if you're not on it, then you missed it. :)
IRC [00:09] <Dossy> It's midnight, time to go to bed ...
IRC [00:09] <holycow> later
IRC [02:27] *** holy_cow parted the chat.
IRC [03:30] *** frankie joined the chat.
IRC [05:05] *** torbutte joined the chat.
IRC [06:46] *** torbutte parted the chat.
IRC [11:14] *** holycow joined the chat.
IRC [12:40] *** frankie parted the chat.
IRC [13:00] <jcdldn> Dossy: what are the chances doing gzip return in fastpath gets done?
IRC [13:01] <jcdldn> is that next after adp gzip stuff?
IRC [13:04] <Dossy> in 4.1.0, yes.
IRC [13:04] <Dossy> the gzip in 4.0 was just stop-gap
IRC [13:05] <Dossy> Jim's sitting on some changes to rework the whole conn thing for 4.1
IRC [13:05] <Dossy> What needs to happen soon is rigorous testing of 4.1 so we can launch it ...
IRC [13:06] <Dossy> I don't know if someone else wants to spend the time to implement gzip for fastpath, but frankly: if you're serving static assets, just pre-gzip them on the disk
IRC [13:06] <Dossy> I'm thinking of writing a simple hack to fastpath to look for "filepath.gz" and if it exists and the client accepts gzip, to return the gzip'
IRC [13:06] <Dossy> version
IRC [13:07] <Dossy> but I am 100% against the idea of on-the-fly compression of static assets served through fastpath. that just feels foolish to me.
IRC [13:07] <Dossy> even if we compress it once and cache it in memory ... once the cache grows and assets get purged, when they get requested again, we'll have to recompress again, etc.
IRC [13:10] <Dossy> Of course, if someone else wants to implement it, that's cool, too.
IRC [13:11] <martinh> gzip and write to disk on the fly?
IRC [13:13] <martinh> if ever the non gzipped mtime is newer than the .gz then re-gzip it.
IRC [13:16] <jcdldn> Dossy: it's more an administrative issue, if you have two versions it;s easy to forget to gzip a new plain text version.
IRC [13:17] <jcdldn> also if the cache is sized right you are not doing a lot of flushes anyway, and I expect the majority of people run browsers that accept gzip so in terms of overall throughput always gzipping is probably a win
IRC [13:18] <Dossy> It'd be interesting to see the impact on real-world apps.
IRC [13:18] <Dossy> Having a conn thread spend even 50msec more per request means a thread being unavailable to service another request for that much longer, etc.
IRC [13:21] <jcdldn> yeah but gzip is pretty fast (I doubt its more than 5ms for most normal sized files)
IRC [13:21] <jcdldn> and almost always you are serving from the fastpath cache.
IRC [13:22] <jcdldn> and I suspect the 5-1 compression ratio you get for css and js means enough less data being sent that your overall throughput is increased.
IRC [13:25] <jcdldn> certainly the halfassed benchmarks I did for compressing dynamic content made it seem like a win for overall throughput and if it's a win for dynamic content it's an absolute certainty that its a win for static content since you can amortize the compress cost over mutiple deliveries
IRC [13:59] <Dossy> See, gzip for dynamic content isn't a "clear win"
IRC [13:59] <Dossy> The reason why we added it is because AOL still supports a LARGE (think: perhaps 10 million) members who are still on narrowband dialup.
IRC [13:59] <Dossy> In those cases, the few dozen msec to compress saves a disproportionate amount of time for the narrowband user's download of those compressed bits.
IRC [14:00] <Dossy> For dynamic sites whose audience is primarily broadband, compression really is likely to add more latency.
IRC [14:00] <Dossy> And considering that browsers cache static assets, the only material gain you get out of compressing static assets is saving your outbound bandwidth.
IRC [14:01] <Dossy> i.e., the extra time the client takes to decompress the incoming data may be equal to the extra time it would take to download the raw data over broadband.
IRC [14:02] <Dossy> Think: 30K raw over 1.5mbit link = 0.16 sec raw transfer (not incl. TCP header overhead/etc.)
IRC [14:03] <Dossy> If the 30K raw becomes 8K compressed, that reduces transfer time to 0.042 sec, a savings of 0.12 sec. Does it take 0.12 sec to decompress that 8K back to its 30K original state? On slower client machines, who knows.
IRC [14:03] <holycow> the margin is so low it doesn't matter i would think
IRC [14:04] <Dossy> The only place you clearly win is cutting your hosting/bandwidth requirement by whatever compression ratio you can achieve.
IRC [14:04] <Dossy> holycow: So, in a world where two choices are equal, I'd go with the simpler one (no compression) because it's easier to get right. Or, harder to mess up. Whatever the case may be.
IRC [14:04] <holycow> same here
IRC [14:05] <Dossy> Plus, you still have many user agents out there on the net that /claim/ to "Accept-Encoding: gzip" but don't.
IRC [14:05] <Dossy> So, now you get into the business of maintaining a blacklist of user agents to not send compressed data to. That's at best an O(n) search /per request/. Ouch!
IRC [14:05] <Dossy> (Or, maybe you can do better of O(n) - but the point is, it's a per-request cost.)
IRC [14:07] *** anlater joined the chat.
IRC [15:06] <jcdldn> Dossy: compressing a 45k file with gzip writing to the filesystem for 10k iterations takes 5.5ms/iteration, uncompressing the same file takes 2.6ms/iteration.
IRC [15:08] <Dossy> So ... what about decompressing the data?
IRC [15:09] <Dossy> And, how small does that 45K file compress down to?
IRC [15:09] <jcdldn> from 45k to 11k
IRC [15:10] <jcdldn> (it's some javascript for something or other)
IRC [15:18] <jcdldn> Dossy: unzip is about 2x faster than zip. So it seems like you would need a something like a 5MB/s line before you broke even assuming you were doing it once and if you could amortize the compress cost over say 5 connections
IRC [15:19] <jcdldn> that would go to ~10MB/s
IRC [15:31] *** Skuld joined the chat.
IRC [15:31] *** Skuld parted the chat.
IRC [16:11] *** frankie joined the chat.
IRC [17:23] *** frankie parted the chat.
IRC [17:24] *** frankie joined the chat.
IRC [17:57] *** frankie parted the chat.
IRC [19:40] *** holycow parted the chat.
IRC [19:42] *** holycow joined the chat.
IRC [19:54] *** holycow parted the chat.