plebd: new experimental file sharing service, coming soon

UPDATE Jan. 26, 2013: It's been about a year since I originally came up with and was going to launch plebd. It was actually live for a number of months but was unsuccessful as I discovered that my ethos (documented below) was misguided at best, and at worst completely missed the point that a file sharing service such as I was describing was neither necessary nor something that I really needed to build for myself.

To say it another way, plebd didn't solve a problem that anyone actually had, including me.

As such, the project didn't go anywhere in terms of user adoption. Bit of a bummer I suppose, but I still own the domain and have plans to do something else completely different with it.

Thanks to the folks to who did try out plebd, and who gave their feedback on alternatives and the overall (lack of) utility of the design that plebd was. Even though the project itself was a failure, I'd say I learned a lot along the way.

If I had to sum up my learning experience with plebd in a sentence: "The internet doesn't need a 'legitmate' file sharing service; it is already, at its core, a data exchange platform, so plebd is therefore redundant."

______________________

So, I've been preparing a file sharing service called plebd for release in a few weeks. Check out the feedback survey to get early access.

Here's what plebd is all about: 

plebd is an experiment in transparency + file sharing

I'm tired of the stigma around file sharing, and the assumption that it's just being a bunch of people stealing content. We all know that there's plenty of legitimate sharing going on. The problem is, if we keep posting files in the same places where illegal activity is going on, we're going to get blamed by association, regardless of whether we're doing anything wrong. So, I'm simply choosing to create a service where what we do and the files we share are fully public by default.

Think of it like the Twitter of file sharing. Everything you upload is public to everyone, and whatever you share has your name on it. Having your name on it means you get credit for being awesome and sharing it with everyone. It also means there's transparency so we can see where files come from.

Transparency is really important, and probably the most interesting and controversial part of this experiment. But there's a good reason for it. Transparency is a force that fosters a self-governing community of people that want to share awesome content, but don't want to be treated like criminals. When we start communities where the blatantly illegal stuff isn't allowed, we take control of our own community, and the accusations largely go away.

Furthermore, I fully support the voices trying to point out that file sharing can be so much more than a scapegoat for illegal activity and failing business models in the digital era. plebd is my contribution to that effort.

The internet is awesome, and sharing rocks

There's tons of awesome free and independent media out there to be discovered. I don't mean that in a, "Support local artists," kind of way, where the appeal to your conscience is supposed to make you feel a twinge of guilt, and thereby get you to participate. I mean there really is awesome stuff out there being created by independent media artists, and I want plebd to be, in part, a place where people can share that kind of stuff with each other.

I love free, but not freeloaders

Don't worry, I'm not about to accuse anyone of anything. Stick with me.

I think it's 100% okay for someone to put a price on the content they create, especially if it's an independent artist trying to make their way.

I don't, however, like freeloaders. I'm not talking about people who think the prices of digital media are too high (in many cases they're right). I'm talking about folks like that "friend" of yours who has been living on your couch for six months without paying a penny of rent, and can't fathom why that's not okay.

I sort of look at independent media the same way. Artists gotta eat. If all they ask from me is a few bucks for their work, then I should either pony up, or move on; I don't just take. 

Rather than complain about the RIAA / MPAA...

...I prefer not to host their content in the first place, or allow it to be distributed on plebd. I'm one of those people that basically stopped watching television years ago, and the music and movies I do consume, I pay for.

But that alone won't change the way digital media is distributed. There needs to be an alternative to those sources of media, a place that focuses on promoting independent digital content and files that aren't produced by the RIAA and MPAA folks of the world. That means, I don't want plebd to ever host any files that belong to the RIAA, MPAA, or other rights groups. I want to build a place for the independent artists.

While many will continue to use file sharing to take content from RIAA / MPAA for free as a form of protest, I just prefer not to consume their content to begin with. The internet is this amazing collaboration and sharing platform, and with each passing day more people are utilizing it to promote their amazing work. I want to try being part of that community - the future, really - and in my own way help build and support services that are moving in that direction.

Won't you join me?

plebd will be opening for early access in a few weeks. Check out the feedback survey and get on the short list!

 

Experimenting with Amon: lightweight server and application monitoring

You know what I love more than puppies and Thanksgiving stuffing? Really great tools that are both awesome, and easy to use. Amon falls into that "awesome and easy" category.

Amon is a great monitoring solution for single-server applications that allows you to see, in just a few minutes, the critical statistics on your server, and its resources. This works because, while some may be all abuzz about scaling, the truth is that most applications out there (and certainly most test/dev apps) basically run on one or two servers - an app server, and a database server. If you're an independent developer, or someone working on a side/hobby project, then why would you go through all the work of setting up a complicated monitoring structure when you could just add Amon in literally 10 minutes?

I'm experimenting with Amon on an AWS Ubuntu instance, which basically runs a small JVM-based network utility. I didn't configure exception logging for my experiment.

Setting up Amon is basically three steps:

  • Run curl install.amon.cx | bash
  • Edit /etc/amon.conf with your specific settings
  • Run sudo /etc/init.d/amon start

The Amon config file on my test system is a simple set of JSON settings, shown below:

{
    "acl": "False", 
    "system_check_period": 60, 
    "secret_key": "[omitted]", 
    "process_checks": [
        "java",
	"mongo",
        "amon",
        "amond"
    ], 
    "web_app": {
        "host": "keyspace.karmanebula.com", 
        "port": "80"
    }, 
    "backend": {
        "mongo": {
            "host": "127.0.0.1", 
            "port": [omitted]
        }
    }
}

keyspace server properly recovers after first "real world" server crash

From the beginning, keyspace was designed to automatically recover from server and application crashes, hangs, and other events that are often outside your control, and certainly outsite the scope of the keyspace service itself. During development I tested a number of scenarios that covered events like forced application kills, to server/system crashes, and other events that would cause the service to stop due to an unknown outside event. As a result of these tests, I was fairly confident that it would survive in a real world crash, but you never really know until it actually happens.

Enter an email I received this week from Amazon's AWS service, stating that the EC2 instance upon which the public keyspace server runs had experienced hardware failure, and the need to migrate my instance to another physical host. Unfortunately, I missed this email until after the physical hardware was already down, and the service stopped working.

The instructions from AWS were to stop, then start the EC2 instance. When I entered the AWS console, I saw that my instance appeared to still be running, but I could tell, from the fact that I could neither successfully retrieve key sets nor ssh into the box that things weren't normal.

So, I did as AWS instructed, stopped, then started the EC2 instance. In just a couple minutes the keyspace service had come back to life. Not only that, I discovered that the recovery feature (which automatically skips a pre-defined number of keysets when a crash is detected) worked flawlessly. Then next key set I got from keyspace.karmanebula.com was several thousand key sets later than the last one issued before the crash, which indicated that the server had skipped key sets that it couldn't know whether or not they had been issued, and simply resumed functioning at a safe point in the key set generator's life. This automatic recovery is handled in the loadOrResetKeyGeneratorState() method.

Nothing makes you sleep better at night than knowing that automated recovery systems, when put to the test, actually work!

FedEx Day for all

FedEx Day is an idea put forth by the fine folks at Atlassian. At NUBIC, the department I work in at Northwestern University decided to try doing a FedEx day of our own. It wasn't long before the project idea discussions started, and people wondered, "Wouldn't it be nice if we had a place to post ideas and get feedback?"

A couple days later, the fedexday app was born - a very simple, no frills way to post an idea, and get feedback.

Below is a quick screen cap that gives you an idea of what the interface looks like. So, if you want to run a FedEx day at your workplace, now it's as simple as spinning up a small Rails app.

Users don't need to log in, and all submissions are anonymous by design. Simply post an idea, and others can comment and endorse the ideas to give you a rought measure of the level of interest. Everything in the app is free-form wiki style editing without restrictions.

 

 

keyspace project goes live

keyspace is a simple network service for generating unique 128-bit IDs for any purpose.  Think of it as a serial number generator for the entire planet.

It's an experiment to allow anyone to tag anything with a unique ID. Once a thing is tagged with a unique ID you can use that ID to lookup the original item.

The first public server is up and accessible right now. Details on the protocol can be found here.

keyspace: the next karmanebula project

About keyspace

keyspace is a public service which generates unique, 128-bit keys, for any purpose.

Inspired by a question asked on StackOverflow.com, keyspace is designed to fill similar needs as those that are filled by Snowflake (a similar open source project lead by Ryan King at Twitter), but rather than being a service you setup on an internal server and run for your own benefit, keyspace is intended to be an open, public experiment, available for use by anyone.

A public keyspace server is being setup soon at keyspace.karmanebula.com When it goes live, and is ready for first public use, it will be announced here.

FeelGoodTrader code released as open source on GitHub

You can get FeelGoodTrader's app code on Github here: https://github.com/normalocity/feelgoodtrader The code is there for anyone to use or learn from, for educational, commercial, or other purposes.

Much of the test and developement data was removed. If you were to download and setup the project yourself:

  • Under "app/dev-notes/server and dev build" there are instructions for building a dev environment in OS X 10.6, and a production environment under Ubuntu 9.10 server.
  • Once the app is setup, you run the http://[host]/setup/new action to setup an admin account and get the app running

Thank you for everyone who supported the project!

The new Reputation system

The problem is and old one: how do you know if someone will flake out on you? The idea is simple: implement a reputation system that tracks when people flake out.

Today we launched the first version of our new reputation system. It tracks the following three things:

  1. The number of items you bought
  2. The number of items you sold
  3. The number of deals you flaked out on

Everyone starts at 0 reputation. When you buy/sell items on the site, your reputation goes up. When you make someone an offer, but then back out of it (flake out), your reputation goes down. Flaking out is also weighted more heavily than not flaking out, so in order to have a positive reputation score, you have to stick to your word more often than not.

We're going to need some better humans...

by John Bigane / FGT Community


"And yet, I, for one, am not ready to submit that easily!" - John BiganeAs many of you know, the IBM supercomputer named Watson defeated the greatest human champions in the game of Jeopardy in February. Despite confusing Chicago with another famous American city, Toronto, Watson handily beat our most brilliant triviaologists.  I sat watching, amazed and scared as he (she? it?) blew through the boards of questions, leading the brilliant Ken Jennings to comment, "I for one welcome our new computer overlords."

And yet, I, for one, am not ready to submit that easily! I'm dedicated to improving myself and encouraging other humans in the quest that is to come.  But since I'm no Luddite and I don't expect a mobile version of Watson any time soon, we're going to be stuck with "lesser" devices for the time being.  Fortunately, with the recent release of iPad 2 and other powerful tablets, we have improved devices to help us learn and play... and play a lot more.  I've been my using my various Apple and Android devices to listen to podcasts to help improve my Spanish and study guitar chords. Plus, I figure all the time I spend conquering Angry Birds must be helpful in the long run, right?

The popularity of the iPad 2 also means there are a bunch of original iPad showing up all over the internet, including FeelGoodTrader. Nothing like using a computer, in tablet form or otherwise, to fight our new supposed overlords.

There are other ways we can stay ahead of the Watsons, HALs, Matrixes, and WOPRs.  They may be able beat us in trivia, chess, and numerous other "games" but Skynet has yet to develop anything that can defeat us in a race, a bike event or other various sporting events.  Until that time, we have the physical advantage.  I'm looking forward to the swimming, running and biking that spring bring.  I may even search out a new road bike while looking for an iPad.

To find all these amazing tools to conquer the future Watsons, I have many choices out there, on and off the internet.  When I choose to buy via an on-line service I use the new and improved notifications on FeelGoodTrader, I'm emailed or sent a text message to let me know when iPads or fast bicycles are added to the site!  I don't have to constantly search and search... when the items are available, I get notified!

If only I had a device that could build a perfect NCAA bracket, I'd be really set!  Until then, I'm stuck with the old ways ... and I'm adding a new notification of "Supercomputer Killer" to FeelGoodTrader so I'm always informed and ready.

Improved notifications

From the popular to the obscure, finding things on FeelGoodTrader has never been easier; so easy that you don't even have to come to the site to find new listings!

If you sign up for an account on FeelGoodTrader (it's free after all), then you can setup notifications for anything you can imagine, and get an email or text message when those things are posted on FeelGoodTrader.

Say you want to know when someone posts a listing for an "Xbox." Simply create a notification for it, and as the listings roll in, you'll get a message in your inbox, and a little notification count at the top of your FeelGoodTrader page after logging in.

Notifications work equally well for really obscure items. Say you're looking for a 1957 Chevy - just create a notification for it and you'll know as soon as it's posted. You can create as many notifications as you like!

Notifications are collected into a single place so they're easy to see.

Create a notification

 

You get a count of all your notifications

 

Sample notification email

 

A list of your open notifications

Hot chocolate and trustworthy shopping


I'm not a shopper...  I've never been much of a shopper.  Every year around the holidays, I fight the urge to listen to Christmas music or start doing any of my Christmas/holiday shopping until after Thanksgiving.  You'll never see me at one of the Black Friday sales.  And then, after the November holiday, I immediately feel behind.  One of my aunts already has her gifts WRAPPED long before Halloween and I won't even have started brainstorming.  Previous years, I've found myself wandering the malls in/around Chicago thinking what I could buy this or that person much to my growing dismay.  Forget my daydreaming of insane consumerism or why Chicago has so many outdoor malls (when most of us are shopping in the dead of winter!), I'd give anything to be done and drinking hot chocolate in front of a warm fire somewhere.
 
As a result, this past year, I, like many others, shopped more on-line than ever before.  Outside of the gifts I made for others, most of the gifts I bought came from Amazon and other on-line services.  Why are web-based retailers' numbers up so much this past holiday?  You can depend on it.  You can trust your transaction and the prices are better than many of the brick and mortar stores.  It was quick... it was easy and I could stay at home, shopping to my heart's content drinking, the aforementioned, hot chocolate.
 
But even buying from Amazon wasn't ideal.  I would have preferred to have done more of my shopping "locally," but away from the malls.  Buying locally has tons of advantages, among them: the ability to support local business in my own neighborhood, it's better for the environment (less shipping fuel and costs), and I can get my items quickly without waiting for them to arrive on-time from across the country and world.
 
Sometimes, buying locally isn't just convenient, it's necessary.  A few months ago after running the Chicago Marathon (I didn't die!), I wanted to buy a couple of the hats volunteers received from assisting at the run.  The buyer I found on a well known web-based service didn't know me from Adam... I didn't trust him/her so we ended up meeting at a random Starbucks in Chicago to make the swap, and I was relieved when the transaction was finally complete.  This type of transaction would have been so much simpler and calmer with a service like FeelGoodTrader.  I could at least have gone into the purchase with some comfort that I was buying from someone from the community, someone that others in the community trusted and had some previous success buying and selling from...and I'll bet some wonderful, tasty, hot chocolate wouldn't have hurt either! :)

 

Happy New Year!

It's changes, changes, and more changes.

After over a month of back end cleanup, and a push to get some more traffic and listings things are going well.

However, after over a month of back end clean up and getting more listings and stuff on the site, it is time once again to focus on new features. In the coming weeks, look for more news on what's new at FeelGoodTrader.com

Happy New Year!

FGT Launches notifications

Wouldn't it be nice if FeelGoodTrader would let you know when new things were posted? As of today, it does.

We've rolled out the initial notification feature just a short while ago. What it lets you do, essentially, is type in a search term (e.g. "Google TV"), and if FeelGoodTrader doesn't have any listings for it a button shows up saying "Notify me!" when listings like this become available.

Give me some notification love!What you need to get notifications

FeelGoodTrader is now open to everyone. Simply sign up for a free account, and you can setup notifications for any search term that you like. Just go to your "profile", put any search term you want into "Create new notification", and bingo - you're ready to go.

Next time you log into FeelGoodTrader you'll see all the new buyer and seller listings for any/all items that match that filter. Pretty cool, huh?

List of notifications in your profileCan I get notifications in email and SMS?

Not yet. We're really close to having this finished, and expect to have an announcement by next month. Email notifications will come first, followed by SMS.

Development update: Off-site listings

This week we're completely focused getting more content on the site, and building the community.

The thing we're working on this week is off-site listings - a way to promote your listings on other sites through the FGT site.

So, let's say that you're trying to sell your car, but you're a little nervous about trying to sell it on FeelGoodTrader because there's not a lot of traffic or users right now. We totally get that.

Go ahead and post it wherever you want to post it so you can get your item sold. Then come on over to FGT and simply post the link to your off-site listing on FGT, and use it as a place to promote your currently listing. How does this help you? Simple - when a user comes to FGT and searches for "used car", in the search results they'll find your off-site listing, and head straight to it. That means you can sell your items anywhere on the web that works for you, and users can search for it on FGT and find it in seconds. Better still, when you close your listing on the other site you don't need to come back to FGT to close it. We will automatically detect that the listing is no longer available within 24 hours, and remove it from the FGT database, saving you time.

So you may ask, "Why does this have to be user-submitted? Why don't we just scrape the listings of other sites, and repost them on FGT?" Well, because doing so would be unethical - those sites have spent a lot of time building their community, and we're not interested in just stealing their listings. That's not cool. However, if you post a listing on another site you're perfectly free to promote it any way you want (using FGT off-site listings, Twitter, etc.), in order to drive potential buyers back to you. You can even post a friend's listing to FGT - so long as you're using it as a promotion engine, and not scraping content from another site, then you're good to go.

Look for an update on this feature later this week. As soon as we're ready to push this feature to the site we'll let you know.

Thanks for reading!

FeelGoodTrader Launches Beta

Friday morning, at 12:04am we launched the beta for FeelGoodTrader.com. Welcome emails went out to all the beta users with instructions on how to create accounts, and many have already. There are now a few items on sale in the market place.

It's been many months coming, what with all the code to be written, hosting services to be ordered, and servers to be built. We're thrilled that it's up, running, and ready to go, so we wanted to say "thank you" to everyone that believed in us before we even launched.

We would also like to thank the family and friends behind the team that have quietly supported the project from its inception.