plebd: new experimental file sharing service, coming soon

So, I've been preparing a file sharing service called plebd for release in a few weeks. Check out the feedback survey to get early access.

Here's what plebd is all about: 

plebd is an experiment in transparency + file sharing

I'm tired of the stigma around file sharing, and the assumption that it's just being a bunch of people stealing content. We all know that there's plenty of legitimate sharing going on. The problem is, if we keep posting files in the same places where illegal activity is going on, we're going to get blamed by association, regardless of whether we're doing anything wrong. So, I'm simply choosing to create a service where what we do and the files we share are fully public by default.

Think of it like the Twitter of file sharing. Everything you upload is public to everyone, and whatever you share has your name on it. Having your name on it means you get credit for being awesome and sharing it with everyone. It also means there's transparency so we can see where files come from.

Transparency is really important, and probably the most interesting and controversial part of this experiment. But there's a good reason for it. Transparency is a force that fosters a self-governing community of people that want to share awesome content, but don't want to be treated like criminals. When we start communities where the blatantly illegal stuff isn't allowed, we take control of our own community, and the accusations largely go away.

Furthermore, I fully support the voices trying to point out that file sharing can be so much more than a scapegoat for illegal activity and failing business models in the digital era. plebd is my contribution to that effort.

The internet is awesome, and sharing rocks

There's tons of awesome free and independent media out there to be discovered. I don't mean that in a, "Support local artists," kind of way, where the appeal to your conscience is supposed to make you feel a twinge of guilt, and thereby get you to participate. I mean there really is awesome stuff out there being created by independent media artists, and I want plebd to be, in part, a place where people can share that kind of stuff with each other.

I love free, but not freeloaders

Don't worry, I'm not about to accuse anyone of anything. Stick with me.

I think it's 100% okay for someone to put a price on the content they create, especially if it's an independent artist trying to make their way.

I don't, however, like freeloaders. I'm not talking about people who think the prices of digital media are too high (in many cases they're right). I'm talking about folks like that "friend" of yours who has been living on your couch for six months without paying a penny of rent, and can't fathom why that's not okay.

I sort of look at independent media the same way. Artists gotta eat. If all they ask from me is a few bucks for their work, then I should either pony up, or move on; I don't just take. 

Rather than complain about the RIAA / MPAA...

...I prefer not to host their content in the first place, or allow it to be distributed on plebd. I'm one of those people that basically stopped watching television years ago, and the music and movies I do consume, I pay for.

But that alone won't change the way digital media is distributed. There needs to be an alternative to those sources of media, a place that focuses on promoting independent digital content and files that aren't produced by the RIAA and MPAA folks of the world. That means, I don't want plebd to ever host any files that belong to the RIAA, MPAA, or other rights groups. I want to build a place for the independent artists.

While many will continue to use file sharing to take content from RIAA / MPAA for free as a form of protest, I just prefer not to consume their content to begin with. The internet is this amazing collaboration and sharing platform, and with each passing day more people are utilizing it to promote their amazing work. I want to try being part of that community - the future, really - and in my own way help build and support services that are moving in that direction.

Won't you join me?

plebd will be opening for early access in a few weeks. Check out the feedback survey and get on the short list!

The plebd feedback survey

Read about the reason we posted this survey.

Experimenting with Amon: lightweight server and application monitoring

You know what I love more than puppies and Thanksgiving stuffing? Really great tools that are both awesome, and easy to use. Amon falls into that "awesome and easy" category.

Amon is a great monitoring solution for single-server applications that allows you to see, in just a few minutes, the critical statistics on your server, and its resources. This works because, while some may be all abuzz about scaling, the truth is that most applications out there (and certainly most test/dev apps) basically run on one or two servers - an app server, and a database server. If you're an independent developer, or someone working on a side/hobby project, then why would you go through all the work of setting up a complicated monitoring structure when you could just add Amon in literally 10 minutes?

I'm experimenting with Amon on an AWS Ubuntu instance, which basically runs a small JVM-based network utility. I didn't configure exception logging for my experiment.

Setting up Amon is basically three steps:

• Run curl install.amon.cx | bash
• Edit /etc/amon.conf with your specific settings
• Run sudo /etc/init.d/amon start

The Amon config file on my test system is a simple set of JSON settings, shown below:

{
    "acl": "False", 
    "system_check_period": 60, 
    "secret_key": "[omitted]", 
    "process_checks": [
        "java",
	"mongo",
        "amon",
        "amond"
    ], 
    "web_app": {
        "host": "keyspace.karmanebula.com", 
        "port": "80"
    }, 
    "backend": {
        "mongo": {
            "host": "127.0.0.1", 
            "port": [omitted]
        }
    }
}

keyspace server properly recovers after first "real world" server crash

From the beginning, keyspace was designed to automatically recover from server and application crashes, hangs, and other events that are often outside your control, and certainly outsite the scope of the keyspace service itself. During development I tested a number of scenarios that covered events like forced application kills, to server/system crashes, and other events that would cause the service to stop due to an unknown outside event. As a result of these tests, I was fairly confident that it would survive in a real world crash, but you never really know until it actually happens.

Enter an email I received this week from Amazon's AWS service, stating that the EC2 instance upon which the public keyspace server runs had experienced hardware failure, and the need to migrate my instance to another physical host. Unfortunately, I missed this email until after the physical hardware was already down, and the service stopped working.

The instructions from AWS were to stop, then start the EC2 instance. When I entered the AWS console, I saw that my instance appeared to still be running, but I could tell, from the fact that I could neither successfully retrieve key sets nor ssh into the box that things weren't normal.

So, I did as AWS instructed, stopped, then started the EC2 instance. In just a couple minutes the keyspace service had come back to life. Not only that, I discovered that the recovery feature (which automatically skips a pre-defined number of keysets when a crash is detected) worked flawlessly. Then next key set I got from keyspace.karmanebula.com was several thousand key sets later than the last one issued before the crash, which indicated that the server had skipped key sets that it couldn't know whether or not they had been issued, and simply resumed functioning at a safe point in the key set generator's life. This automatic recovery is handled in the loadOrResetKeyGeneratorState() method.

Nothing makes you sleep better at night than knowing that automated recovery systems, when put to the test, actually work!

FedEx Day for all

FedEx Day is an idea put forth by the fine folks at Atlassian. At NUBIC, the department I work in at Northwestern University decided to try doing a FedEx day of our own. It wasn't long before the project idea discussions started, and people wondered, "Wouldn't it be nice if we had a place to post ideas and get feedback?"

A couple days later, the fedexday app was born - a very simple, no frills way to post an idea, and get feedback.

Below is a quick screen cap that gives you an idea of what the interface looks like. So, if you want to run a FedEx day at your workplace, now it's as simple as spinning up a small Rails app.

Users don't need to log in, and all submissions are anonymous by design. Simply post an idea, and others can comment and endorse the ideas to give you a rought measure of the level of interest. Everything in the app is free-form wiki style editing without restrictions.

keyspace project goes live

keyspace is a simple network service for generating unique 128-bit IDs for any purpose.  Think of it as a serial number generator for the entire planet.

It's an experiment to allow anyone to tag anything with a unique ID. Once a thing is tagged with a unique ID you can use that ID to lookup the original item.

The first public server is up and accessible right now. Details on the protocol can be found here.

keyspace: the next karmanebula project

About keyspace

keyspace is a public service which generates unique, 128-bit keys, for any purpose.

Inspired by a question asked on StackOverflow.com, keyspace is designed to fill similar needs as those that are filled by Snowflake (a similar open source project lead by Ryan King at Twitter), but rather than being a service you setup on an internal server and run for your own benefit, keyspace is intended to be an open, public experiment, available for use by anyone.

A public keyspace server is being setup soon at keyspace.karmanebula.com When it goes live, and is ready for first public use, it will be announced here.

FeelGoodTrader's mission lives on at Copious.com

FeelGoodTrader was closed down in early June, but if you're looking for an alternative without going back to CraigsList, check out Copious: social buying with Facebook integration. Using Facebook as the login system means you aren't dealing with anonymous strangers.

FeelGoodTrader code released as open source on GitHub

You can get FeelGoodTrader's app code on Github here: https://github.com/normalocity/feelgoodtrader The code is there for anyone to use or learn from, for educational, commercial, or other purposes.

Much of the test and developement data was removed. If you were to download and setup the project yourself:

• Under "app/dev-notes/server and dev build" there are instructions for building a dev environment in OS X 10.6, and a production environment under Ubuntu 9.10 server.
• Once the app is setup, you run the http://[host]/setup/new action to setup an admin account and get the app running

Thank you for everyone who supported the project!

FeelGoodTrader closing

FeelGoodTrader is closing. We will release the code to the public on GitHub, sometime soon.