Quantcast

Gag Order Lifted on Internet Archive, NSL Withdrawn

EFF Press Release:

FBI Withdraws Unconstitutional National Security Letter After ACLU and EFF Challenge

Gag Order Lifted on Internet Archive, Allowing Founder to Speak Out for First Time

San Francisco - The FBI has withdrawn an unconstitutional national security letter (NSL) issued to the Internet Archive after a legal challenge from the American Civil Liberties Union (ACLU) and the Electronic Frontier Foundation (EFF). As the result of a settlement agreement, the FBI withdrew the NSL and agreed to the unsealing of the case, finally allowing the Archive’s founder to speak out for the first time about his battle against the record demand.

“The free flow of information is at the heart of every library’s work. That’s why Congress passed a law limiting the FBI’s power to issue NSLs to America’s libraries,” said Brewster Kahle, founder and Digital Librarian of the Internet Archive. “While it’s never easy standing up to the government — particularly when I was barred from discussing it with anyone — I knew I had to challenge something that was clearly wrong. I’m grateful that I am able now to talk about what happened to me, so that other libraries can learn how they can fight back from these overreaching demands.”

Best Customer Service Ever

I usually get terrible customer service. Especially when I buy products from companies that I’ve worked at for 5+ years. Stupid Apple.

But lately we’ve had a couple really great customer service experiences. First, we broke our Wii playing too much Super Smash Brothers. Apparently, the GPU overheats(?) when you play too much, or when you play too much over wifi, or something. This caused our Wii to always show flickering black vertical lines. I called up Nintendo on a Sunday, explained the problem, and after 18.5 minutes on the phone, they said I would have to send it in for repair. They emailed me a FedEx ship label, which I printed out. I sent the Wii in on a Tuesday, and a fixed one arrived today, just three days later!

Even better was our experience with Philips. The charger to our awesome Sonicare toothbrush stopped working. Jess called up Philips, spent a couple minutes on the phone, and about a week later, an entire new toothbrush and charger arrived. YAY!

How to Make Naan Using a Pizza Stone

We don’t have a tandoor, so when we have a craving for fresh-baked naan, we have to improvise. Here is some video showing how we made naan using a pizza stone in our regular oven.

(click play to start) (link to other sizes)

The next step is to build a tandoor in the backyard.

On Software Development

PaulR said this at lunch and we all lost it:

Software isn’t finished until the last user is dead.

Original source unknown…

Trip Report: A Wholesome Evening at the Silver Crest

The Silver Crest is so sketchy. It’s like Sketch Central. I’ve lived next to it for six years and had never been inside. But look at all that great neon! Their sign proclaims: “We Never Close”. We decide we have to go.

Friday night Sam was driving us back from the spaz party at the Li Po Lounge, and we pass the Silver Crest. We’re full of Mai Tais and thumpy beats, but Bob and I have the same thought. We have to go.

We walk in and transported fifty years into the past. There is a bar in the back with a crowded pool table, but it’s 2am and the bar is just closing. We grab a booth in the front. These jukebox things on every table:

It doesn’t work and and eats Bob’s quarter. That probably for the best, since the jukebox is full of with Greek waltzes.

The menu is written on the wall. The food isn’t cheap, but we hope it’s good. The coffee is a ridiculous $2.75. Even for diner coffee, it tastes like crap.

They have these old pinball machines, which mang decides to check out while we wait for the food. There is a sign saying you have to be over 21 to play. We ask the server why, and she says they don’t want to encourage the kids to gamble. Whatever. The kids are all at Li Po, anyway, playing dice for drinks.

The food comes out. I’m surprised by how bad it is, but I don’t know why. I guess I expect shitty diners to have great food. That is not the case tonight. My hotcakes arrive with an entire stick of “butter” on top, which promptly slides off onto the table as soon as the server sets them down.

I’m disgusted but Bob thinks we need to come back. The Silver Crest has donuts, and they might actually be good. Maybe we’ll bring our own coffee next time.

No Home Depot on Bayshore for Now

Home Depot executives have abandoned a seven-year quest to open a hardware store at the borders of the Bayview and Bernal Heights neighborhoods, blaming a slow approval process, falling nationwide sales and the poor economy.

Good. I don’t want those racist fuckers two blocks from my house.

linky

Internet Archive Brings Free Ultra High-Speed Internet to Public Housing

Go Brewster and Ralf!!

The Internet Archive, a San Francisco-based organization dedicated to preserving a record of the Internet and to increasing access to the Internet, today began offering free Internet service to public housing projects at speeds far greater than any other city resident can receive.

Valencia Gardens Housing, with 240 units, is the first area to be connected in a pilot project that expects to wire more than 2,500 units in the city in the next eight months, according to Internet Archive founder Brewster Kahle.

What makes the project unique is that the apartments will be connected to the Internet, and to the educational resources at the Internet Archive, at 100 megabits per second (Mbits/second). That speed contrasts sharply with the normal Internet service offered by telephone companies, which is usually less than 6 Mbits/second.

The residents can instantly view DVD-quality videos of the thousands of lectures and other educational information from the Internet Archive’s collections, as well as traditional Internet access.

The Internet Archive is able to achieve this high speed by connecting the San Francisco municipal fiber optic network, which runs through the public housing developments, to an Archive switching center, which connects to the Internet.

“We are pleased to be the first non-profit organization to bring public housing online,” Kahle said.

He added: “We are excited to see much faster access to the Internet as a way to experiment with advanced applications, and are pleased that the underserved get first access to advanced technology.”

See also: NYTimes Bits Blog, The Reg, Cnet article by Greeter Dan.

Bring Your Own Big Wheel Races 2008

Last year a small crew showed up at Vermont St. after the official races ended, and we had an absolute blast. This year, the official races were on Vermont St., and it was awesome:

(click play to start) (link to other sizes)

IMG_4029

IMG_4023

IMG_4027

Happy Holi!

We went to the Holi festival in Sunnyvale today! Lots of fun :)


(click play to start) (Link to other sizes)

Happy Holi!

modeselektor

We went to see modeselektor last night at Mighty! It was super-packed.. a huge crowd for a Wednesday:
IMG_3943

The Camo Bus was rockin outside… I love the fact that no one complains that there is an absolutely thunderous soundsystem set up on the street :)
IMG_3945

Here is a short video of modeselektor being introduced. The Mighty soundsystem absolutely crushes the little microphone in my camera, so this video is kind of pointless but still fun. What you can’t hear in the video is an insane amount of soul-destroying bass:

(click play to start video)

Gorgeous Sunset over the Pacific… With Paragliders!

Check out this sunset timelapse that Mike shot using CHDK. Paragliders streak across the sky like lasers!

(click play to start) (link to other sizes)

Using git For Large Scale Digital Archiving: An Outline

Here are some notes on how one might re-architect Internet Archive infrastructure to meet some additional goals:

  • easy to set up and replicate
  • provide versioning and transactions
  • handle more media types well
  • better ingest/locate/read apis
  • better search

The current architecture looks like this:
iaarch.png

The diagram is simplified a lot. There are currently about 1800 nodes in the cluster, most of which are storage nodes (low power 1U nodes with 4 1TB hard drives). The deriver nodes are used for crunching things like pdfs and h.264s, and there are about 300 of those. There are 5 www frontends, hidden behind a couple load balancers, and database server has at least one read-only secondary.

What I like about the current infrastructure:

  • Easy to add more storage. Some other archival solutions do not scale well, since they insist all hard drives be connected to the same machine. This starts to break down at the petabox scale.
  • Easy to add more bandwidth. Currently IA is pushing 5+Gbps of outbound bandwidth. Every storage node runs an Apache server, which lessens load on the homenode, which is a problem with other archival systems.
  • Database hits are not required to locate an item on the cluster. When an item is requested through the Locator service, a multicast is sent, and machines that have the item will respond. The lessens load to the DB server, which is important when getting thousands of web requests per second.

What I find interesting about the current infrastructure:
  • RAID is not used. Items are backed up on to a secondary machine when added to the archive.
  • This is mostly due to “RAID is hard to get right” and cost
  • This means there are two machines (and two apaches) ready to serve the same content.
  • One machine can be taken down for repair while the content is still online.
  • I would like to see use of either RAID or maybe RAID_Z

An idea on how to re-architect things using git as a storage backend to provide versioning and transactions
  • git is the version control system used for the linux kernel.
  • git is a totally new way to operate on data. Read this if you are a non-believer.
  • We could keep the infrastructure mostly the same as IA, but store items as git repositories. This would not be a large architecture change.
  • git would become a supported access protocol, in addition to http, ftp, and rsync. Backups could be simple a git pull. We could git clone the entire cluster.
  • We would get versioning!

Changes needed to repo.git to make it useful in an archive cluster:
  • Change reguser.cgi to tie into the existing user database (talk to dbserver)
  • Change regprog.cgi to work in a cluster environment. Repositories are inited in /{0-4}/items/id/id.git on a primary node (talk to catalog/homenode)
  • Use post-commit hook to queue backup and derive tasks (talk to catalog)
  • Change gitweb to show custom view of movie, audio, texts (book), and photo collections. Software collections would show standard gitweb view.

I don’t think this would take too long to implement, but I’m lacking co-conspirators these days.. Maybe when shag makes it to SF we will have to knock something out :)

Digitizing old books with large foldouts

This panorama is one of the first images from our test of digitizing books with foldouts. It is from this book. The full-size image is here.

These foldouts are hard to image.. This picture shows how it was placed under the camera.

The Mushroom Theory of Management

I decided to check out The Soul of a New Machine after reading this account of working at Atari and writing the Donkey Kong cart for the 2600 (if you are an Atari or DK fan, read this).

Anyway, the wiki page for The Soul of a New Machine talks about the Mushroom Theory of Management:

Tom West practices the “Mushroom Theory of Management” - “keeping them in the dark and feeding them shit.” That is, isolating the design team from outside influences and instead using the fear of the unknown to motivate the team.

I’ve encountered this a lot over the years, most notably at Apple, where Mushroom Management was one of my manager’s only motivational techniques.


CC by-nc-sa licensed photo by 23bit_grrrl

See also: Mikee’s awesome mushroom pictures

David Sedaris delivers a pizza


(via)

Announcing YouTubeFilter

YouTubeFilter is a simple tool that scrapes the MetaFilter RSS feed and embeds the YouTube videos inline. I wrote it to make it easier to find cool videos to watch on my Wii.

Unfortunately, the Wii runs out memory when loading YouTubeFilter! And of course, Firefox bugs on the mac prevent some of the embedded videos from showing up unless you resize the window just right. Stupid firefox.

The code is checked into SourceForge. I use Beautiful Soup for parsing the RSS. Someone please help me make it work on the wii!

Quite possibly the worst venue for a Burning Man event

This year’s Burnal Equinox event (called Whispered Dream) is going to be held at the Whisper Ultralounge, which sounds like one of the worst venues in SF for a burning man event.

Their webpage lists the “upscale dress code” as follows:

NO; Athletic/Sports Attire of any kind including expensive designer court shoes, Timberland and/or similar style Boots, Excessively Baggy Jeans, Dark Sunglasses, T-shirts, Athletic Apparel, and Most Hats (Fedora style top hats are acceptable). You’ll have to step it up if you wana step in.

Bwahahahaha… BM keeps getting less and less interesting to me.

We have five grades of chili…

Found while searching for something South Park related:

Visualization of how an ISP in Pakistan hijacked access to YouTube worldwide.

On Sunday, the government of Pakistan ordered ISPs to block access to YouTube in the country, for containing content that was offensive to Islam. (This occurred soon after videos exposing vote-rigging in the recent Pakistani election appeared on YouTube.)

In response to this order, Pakistan Telecom began advertising a hijacked route for YouTube, which caused all internet traffic to YouTube from around the world to be blocked for several hours. Here is a full writeup of how this happened.

I found this great Slashdot comment that explains how to use BGPlay to see an animation of Pakistan Telecom re-routed YouTube for the world.

Here is how the net looked during the hijack. The red dot on the left is YouTube, and the red dot on the right is Pakistan Telecom:
before.png

After YouTube announced new routes, traffic returned to normal for those outside Pakistan:
after.png

The animation is pretty neat! Also, Wikipedia has a good summary of the Border Gateway Protocol, which is the protocol that routers use to talk to each other.

Announcing TikiCards: Flashcards for the Web

sweet.pngI was inspired by peliom’s web-2.0 Japanese flashcards, so I made some Hindi flashcards this weekend. Or rather, I made an open-source framework for javascript-powered flashcards called TikiCards, and pre-populated it with vocabulary words from the awesome Door Into Hindi lessons that I’ve been working on. I’ll work on adding more words and more languages soon. The code is checked in here.

Unfortunately, Firefox on the Mac doesn’t ship with a Devanagari fonts, and it doesn’t use the OS X system font, so all the characters show up as question marks. And unlike peliom’s Japanese flashcards that work great on the iPhone, the Devanagari characters show up as square boxes on the iPhone. So if you want to use these for Hindi, use Safari on a Mac or FF on unix.

Anyway, check it out and let me know what you think.

Ruby 1.9 gains block-level scope

I was watching this Google TechTalk that Yukihiro Matsumoto gave on Ruby and learned that Ruby 1.9 had block-level scope.. cool!

Apparently, block scope is some sort of thing with me:

Spherical Video in Flash

These spherical videos are stunning! It’s like Motion-QTVR. (via)

Dr. Alexander Shulgin’s First Lab Notebook

Dr. Alexander Shulin’s first laboratory notebook has been scanned and put online.
p.jpg
If you want higher resolution images, check out the JPEG 2000 files here.

How To Filter San Francisco Tap Water

Background
Our water has gunk in it (possibly due to this construction). This gunk gets caught in our kitchen faucet’s aerator, and if we don’t clean it out, the water flow will stop completely. Here is what the gunk looks like:

a1.jpg

So we decided it was time to install a water filter. The easiest way to filter drinking water is to use a Britta and Pur pitcher, but these don’t work for me. I use them for a bit, and then get tired of changing the expensive replacement filters every month.

The water filter industry has a standard size for under-sink adsorption filters: 9 3/4″. These filters are similar to but much larger than the standard Britta pitcher-style filter, and they only need to be changed every 4-12 months (depending on filter type). You can buy 9 3/4″ filters from several different manufacturers, which helps keep the prices down.

Choosing a Filter

Deciding what kind of filter you need should be easy. In theory, you can call your water utility (or check their website), find out what contaminants are in your water, and then check which filters are certified by the NSF to filter out those contaminants.

I did a bunch of research and found that SF tap water is generally safe to drink. It is disinfected with Chloramine and contains trace amounts of parasites, copper, and lead. The SF PUC monitors Cryptosporidium and Giardia levels every two weeks, and their 2006 Water Quality Report (published 6/07 PDF) shows that all measurable contaminants are below maximum contaminant levels.

Even after finding that SF water is safe for us to drink, I decided to get an activated carbon filter in addition to a sediment filter, because I’m worried that chemicals might enter into the water main during construction, just as visible sediment enters into the water main. Also, I decided that activated carbon was really cool (one gram has a surface area of 500 m² - 1500 m²)!

Choosing a 9 3/4″ Filter Housing

My first thought was to get an OmniFilter OT32 for $140. This unit has a double housing that can fit 2 9 3/4″ filters, and it comes with a string-wound sediment filter and an NSF-certified 0.5-micron enhanced carbon block filter.

After some digging, I found I get a cheaper 2-filter housing from Pentek Filtration. I bought a BFS-201 housing (PDF) from FiltersFast for $57. It’s seems well-built and looks like this:
IMG_3837

Installing the Filter Housing

The recommended way of using under-sink filters is to install a second, low-flow-rate faucet to get the maximum effect from your filter. I didn’t want to do this, because I didn’t want to drill a new hole through our granite countertop, so I investigated splicing the filter into our cold water line.

If we only wanted to filter sediment, we could easily add the filter to cold water line, because sediment filters can handle 10 gallons per minute, and our faucet only had a 2gpm flow rate.

Unfortunately, activated carbon filters only work at reduced flow rates. If you only want to filter for what the NSF calls “Aesthetic Effects” (Standard 42), then you can buy a carbon filter that works at 2gpm. But if you want to filter for things like lead, mercury, VOCs, asbestos, MTBEs, Cryptosporidium, and Giardia (what the NSF calls “Health Effects”, or Standard 53), then you have to reduce to the flow rate to 0.5-0.6 gpm.

We decided that it was OK to reduce the flow rate of the cold water at the faucet, since we will still have the full hot water flow rate for washing, and the dishwasher cold supply is already split off. This means we don’t need to drill for a new faucet.

The Pentek filter housing has two 3/8″ female NPT water connections. Our cold water line has a 1/2″ IP straight thread on the faucet side and a 3/8″ compression fitting on the other. To splice the housing into the cold water line, we need to add two 3/8″ NPT to 3/8″ compression adapters to the filter housing, using teflon tape to ensure a good seal:

Choosing a Sediment Filter
Now comes the fun part! We get to choose some filters. Since these are a standard size, we can choose from a bunch of brands, and mix-and-match. Sediment filters are very inexpensive. They are usually made of wound string, last for 10-15,000 gallons, and cost under five bucks. I got the Pentek CW-MF from FiltersFast for $3.30. Looking at the spec sheet, I should have gotten the WP-5 for $5.60, since it has a 5 micron rating instead of a 30 micron rating. It might not matter, since the water will be flowing through a carbon filter anyway. This filter is mostly to lengthen the life of the carbon filter.

Choosing an Activated Carbon Filter
The OmniFilter CB3 seems pretty good. It costs $36 dollars and has NSF Standard 42 and Standard 53 ratings to reduce Asbestos, Atrazine, Cyst, Lead, Lindane, Mercury, and VOCs, as well as Chloramine and Chlorine.

I found Pentek CBR2-10R, which is basically the same thing as the OmniFilter CB3, but it lacks the NSF Standard 53 rating. However, it costs only $17 at FiltersFast, and I decided it was good enough for me. It also has a built-in 0.6gpm flow restrictor, which makes sure you get the full effect of the filter.

After installing everything, this is how it looks:
IMG_3870

The water definitely tastes different; it has a slightly-metallic aftertaste, similar to distilled water. I’m interested to see how much stuff the sediment filter catches after 3 months.

Costs
This is my bill from FiltersFast:

Filter Housing BFS0-201 $56.99
Carbon Filter CBR2-10R $16.95
Sediment Filter CW-MF $3.30
Filter wrench SW-1a $2.70
Shipping $1.99
retailmenot.com coupon -$5.00
Total $76.93

I also had to buy two 3/8″ compression to 3/8″ MPT adapaters for $4.79 and a couple hoses from Cole Hardware, bringing the total up to a hundred bucks.

How to tunnel VPN over SSH

Today I had to use VNC to debug a remote machine, but firewalls were blocking VNC ports.

After I failed to get my VNC client (Chicken of the VNC) to use a SOCKS proxy, I was able to use SSH port forwarding to get it working. On your local machine type:

NOCODE:
  1. ssh user@remotehost -L 5900/localhost/5900

This forwards port 5900 on localhost to port 5900 on the remotehost. Then in Chicken of the VNC, open a new connection to localhost. That’s it! EEZ!

It turned out that Xorg was eating all available memory and invoking the oom killer. Sigh.

I guess I could have figured this out without VNC, but I couldn’t reproduce the bug locally, so I watched as a remote user was working on the machine.

Older Posts »