Release 123/124

cortex will cease operation on January 1st, 2014. cortex was an interesting experiment, but I would say that it's not particularly great at any one of the things for which you might use it. I've turned to alternative tools myself, for the most part.

Thank you, eveyone, who tried it out and provided your feedback!

  • If you used cortex as a to-do list, is a great option, and allows shared to-do lists.
  • If you used cortex as a real-time collaborative tool, the closest thing I've seen that meets my personal needs for this is Google Docs, or just writing on a whiteboard in a room with other people.
  • If you used cortex as a way to keep your notes organizaed then there are lots of good alternatives, any of which are probably as good as the next, including just carrying around pen and paper.

For me, closing down cortex is about going back to simple tools that work, rather than trying to push the app further on its own. Real-time collaborative tools that are simple and lightweight, I think, is the most interesting category of tool that cortex fell into, even if it was only intended as a kind of digital scratch paper.

Release 122

Removed twitter login option.

I'd canceled all my twitter accounts a couple weeks ago, and forgot that cortex had twitter login.

Rather than sign up for twitter again to get the twitter developer account I just looked at which users in cortex were using twitter login. I only found two: a test account that I created and a user who only logged in once, and then never again.

Seemed pretty harmless to just remove twitter login altogether.

Release 115

This patch was released on 2013-02-09. It was discovered that Google OAuth2 uids had changed from the users' email address to a unique integer. This meant that previous users who logged in both before and after Google made this change would lose their star-menu list, because that list is tied to your uid, which had changed.

I added a site-wide message to the top of the page that asked users to contact me if they weren't seeing things that they expected to see in their star menu, but it either didn't affect anyone else, or no one cared, because I didn't get any contacts of this sort.

This site-wide message was later removed in release 119, which was deployed earlier today.

Release 114: Removed the "canvas" theme and applied a security patch

The "canvas" theme is no more. It has been replaced with a theme called "light". That means that cortex has only two themes, light and dark, with the dark theme being the new default.

In other news...

In release 110 (not noted in this blog) I upgraded to Rails 3.2.11 in order to patch the two critical security bugs found in ActiveRecord and elsewhere. However, this update also broke the jQuery support, which broke the ability to drag-and-drop notes to reorder them.

As a result, in release 113 I applied a patch to jQuery to fix the drag-and-drop reordering problem.

And that's all for now...

I've got some bugs and exceptions to handle in my app currently, but none of them are critical, or even visible to the user, I believe. After 36 days of work, and 3 days of documenting it via this blog, I'm ready to break away from cortex for a couple of months, let it simmer, try to get some additional user feedback, and then I'll dive back in with another round of enhancements from users.

I hope this is been insightful for many people. The goal here, of course, is to give back to the open source community that has already given me so much value. While cortex certainly isn't a groundbreaking app, I feel that sharing what we do benefits everyone in the long run by giving us multiple examples and resources from which to learn.

The net effect of all this seems to be that the level of innovation in software just keeps accelerating. As far as I'm concerned, this is almost universally a good thing.

Multiple physical layouts

Adding to the work done in creating multiple themes, issue #34 added multi-layout support.

This commit started the final set of modifications necessary to get layout changes to apply across all clients.

Unlike visual theme, layout is brainstorming page specific. The reason is because the physical layout is more of a functional thing, while color scheme is a matter of personal preference. In terms of physical layout, when you're working on a list you want a list, and you want it to look that way for everyone.

This commit fixed a problem where the size of the DOM elements on the page was too large, causing a though appearing later in the list to overlap the though just previous to it. This made drag-and-drop much more difficult, because it was too easy to grab the wrong thought box when you wanted to rearrange thing.

And we're done with multi-layout support.

Star menu with Google, Facebook, Twitter, and Githib login support

Omniauth is a great gem that makes adding 3rd party login support a breeze. It supports a ton of services, and is being improved all the time. I've used it on three apps in the past, so I was really able to get through this functionality pretty quickly.

The point of adding login functionality to what was previously a 100% anonymous app, was to allow users to build an in-app list of brainstorming pages that they wanted to save and go back to in the future. Before login was supported, it was absolutely possible to do this on your own, but it required you to bookmark the pages manually in your browser.

The in-app bookmarking (and login, for that matter) are controlled through the "star" menu in the upper-left corner of cortex's page layout.

CanCan users will notices that my authorization file allows managing of all assets for all users. This is intentional (since no one can edit user info currently, and cortex has always supported anonymous, free-flowing brainstorming for everyone).

In the future I plan to add functionality that allows users who "own" a brainstorming page to make it private if they choose, but for now realize that I'm fully aware that the app is wide open to editing, and that this is intentional. There are no actions exposed to the public that allow deletion/editing/viewing of users, or even deletion of Thoughts and ThoughtWalls, so there should be no danger yet.

Strangely, the missing end wasn't throwing any errors in development, but it was failing on heroku - as it should.

This migration is what makes the star menu work - by creating the relationships between users and their in-app bookmarked brainstorming pages.

This was the point at which multiple login services were actually enabled in production.

This was a fix to an issue where one of the OAuth providers' uid field was being interpreted as an integer, even though I store that field in my database as a string.

And that pretty much wrapped up login support for now.

Multiple themes and .CSV export

Not sure why I was committing theme related code against a .CSV export issue, but here they are nonetheless. Probably just an oversight on my part.

.JPG/.PNG export was also requested, but found not to be practical, so that functionality was canceled. The reason image export isn't practical is twofold: first, why send an image to someone when you can just send them the link, and second, can you imaging the export UI? Which resolutions should I support? Which formats? How would I capture the screen?

Instead I decided that creating a theme that was printer friendly (which is what my user really wanted - something they could print out and mark up with a pen) was a better approach to image export. Not only that, I wanted to support more themes long term anyway, so I figured this was a good time to get started on this work.

The once planned, but now scrapped "paper" theme (it was replaced with the "canvas" theme, which I'm not sure that I actually like all that much) was simply another visual theme in addition to the original dark-themed layout. Implementing a second theme forced me to refactor my theme code to support an arbitrary number of future themes. This meant breaking up my CSS so color schemes were kept separate from physical layout on the page.

This was the big commit. It put forth the overall plan that I would follow when adding future layouts and themes. As of this writing, cortex supports two themes, and three layouts, thanks largely to the work done in this commit, and following commits, to separate colors and style from physical layout on the page.

The theme is user specific, not brainstorming page specific. This means multiple people can be looking at the same brainstorm, but each individual gets to see it in their preferred visual style.

In this commit, the control/navigation bar got a revamp which made it neutral grey. This control bar never changes its look and feel, regardless of the theme selected. This is primarily about separating responsibilities, and only adding abstractions and splitting up the code when and where it's necessary.

At this point I started referring to the "navigation" bar as the "control" bar, FYI. I think it's consistently called the "control" bar hereafter.

One other thing I noticed here is that I was using confusing terminology throughout the implementation of this feature. So, cortex has two visual elements that you can currently alter, the theme, and the layout.

The theme is the color scheme, essentially. The layout is the physical layout of elements on the page, especially their size and arrangement relative to one another.

Whenever possible, all size/arrangement CSS is in a layout CSS file, while all color scheme elements are in a theme CSS file.

Drag-and-drop ordering

This is the single largest feature in all of cortex, in terms of the number of commits it took to get it working both locally and collaboratively (31 in all). The work on this feature is probably what I'm most proud of in cortex, because it's not only a cool feature, but it caused me to refactor so much of the code involving real-time collaboration. This made the overall code quality much higher, I think, and got the architecture of the app heading in a better direction.

I think this massive commit message (on the final "drag-and-drop ordering feature) covers most of what I spent time on in all these details - trying to get collaborative changes to work consistently:

So, in using cortex, something that I observed was that if you moved an item up in the order everything was fine; it would find that out of order item in the client first, and move it. However, when moving items DOWN in the order, instead of just moving that one item, it would instead move all the intermediate items to acheive consistency with the server, even though the net effect was the same. That meant, when moving items down in the list, you'd move lots more thought boxes a short dista nce in order to acheive consistency, instead of a few boxes a large distance.

This was terribly inefficient for two reasons. First, every shift requires a server request, so the number of requests to the server was significantly higher than the most efficient move possible. Second, it was inefficient for the client because it meant more total moves (and more time the user spends watching the screen, waiting for it to become consistent).

When thinking about how to solve this problem I originally thought that I would need to implement the diff algorithm to find the differences between client and server, and then perform the first move in the diff list with each server request. This seemed like a complex solution to what seemed like a pretty simple problem, so I punted on the solution and waited to see if I could come up with something better.

It turns out that, instead of calculating the diff for the whole list, that you could instead take the client list and the server list, and calculate an array of integers that represent the distance between elements. So, if the client and server lists were identical, you'd have an array of zeroes - meaning that nothing was out of order on the client, when compared to the server ordering. However, in the case that you move something down in the visual ordering, say by seven spaces, you'd get an array of distances that contained seven 1s, and one 7. Once you have that array of distances, instead of trying to move all the items that are out of order, you simply move the one that is furthest out of order. To say it another way - instead of moving the seven items that were off by one, you could achieve the same result by moving the one item that was off by seven. The net effect is that, whenever the client can shift just one item in order to make many items consistent in their ordering with the server, it will prefer to move that one item by a larger amount one time, rather than move a lot of items a smaller amount over several server requests.

So far this has proven to be both effective, and much less complicated a solution when compared to implementing a diff.

The commits:

Tracking the proper order of the thoughts on the page is actually just a matter of a single column. Getting it to work collaborative, in real time, and to be "eventually consistent" across all client browsers? That's another story.

This added some scopes to the Thought model to make retrieving the order of thoughts based on various criteria, a trivial thing.

Prerequisite for figuring out what the "next" sort order value should be for new Thoughts added to the page.

jQuery actually does most of the heavy lifting when it comes to making drag-and-drop functionality work in the UI.

At this point I'm getting pretty deep into the problem. You can see me relying on a lot of debug output to figure out what's going on client and server side. This will continue for some time.

This step was important. While a user is dragging an item on the page, you want updates from other users to be temporarily suspended so you can drop a thought in the intended position. From there, collaborative updates can resumes.

This took changes from the master brach and updated the ordering branch. This is what is supposed to happen when you're doing proper branching and merging - develop logically isolated features on their own branch, and integrate them when you're ready. It makes development of any kind a lot easier the more complex a piece of software becomes.

Sometimes, even when you know something isn't working, you have to commit what you've got before you forget where you were.

This commit included code that was entirely for debugging and testing. I clearly was confused as to why it wasn't working. I don't really add big debugging functions like this unless I'm confused about something.

Duplicates were sometimes generated during weird drag-and-drop operations. This commit corrected those problems.

When coding on the bus, this would happen to me all the time. This commit makes it so your connection can drop, you can put your computer to sleep, etc., but when you come back the periodical refresh will just come back to life, letting you pick up where you left off.

I also removed those huge JavsScript debugging functions.

At long last, I breathed a sigh of relief, knowing that I could move on to another feature, ending six days of continuous work on nothing but this feature.

Thought editing and history

Issue #33 represented an important feature: the ability to edit thoughts (to correct typos, add more information, etc.), and to retain the most recent five versions of that though, allowing you to rever to them at any time.

While developing this feature I also saw, and added, a couple of other tweaks.

This commit mostly added view and layout bits. Nothing was wired up to control anything at this point.

If I had been more careful, this migration never would have been necessary. I discovered this error the day after it had been pushed to production, and making this corrective migration, while helping a lot in terms of usability, once again wiped out all the votes. Drive me crazy when I make mistakes like this.

As mentioned in the previous commit, I had to apply a corrective migration to the production server. Because of this, I had to diable to the "edit thought" UI control before deploying because otherwise a control that did absolutely nothing would have been visible to users.

It's been a few weeks, but my guess as to why I had to do this is because I was probably developing directly on master, instead of developing on a feature-specific branch.

This is yet another mistake. If thought editing had been on an isolated branch, the corrective migration could have been applied in its own branch, and I wouldn't have needed to disable this control in the first place.

Let this be a lesson as to what happens when you build directly on the master branch.

  • d3a812f - Removed the "close" button from the overlay.

This is where I decided to basically drop support for mobile devices.

In working on thought editing, I realized that simply truncating the thought text to 100 characters, as I'd been doing up to this point, wasn't going to work long term. after a little research I found this CSS property that would accomplish the same thing so I didn't have to constantly tweak the truncation to make it fit.

This commit came from stopping and testing it on Firefox and Safari. I tend to develop on Chrome exclusively these days, but it's good to stop and try other browsers from time to time, to make sure things work the way you expect them to.

Editing and history was already working on a functional basis before this final commit, but the final commit adds some layout tweaks and rebuilt the assets.