This is the single largest feature in all of cortex, in terms of the number of commits it took to get it working both locally and collaboratively (31 in all). The work on this feature is probably what I'm most proud of in cortex, because it's not only a cool feature, but it caused me to refactor so much of the code involving real-time collaboration. This made the overall code quality much higher, I think, and got the architecture of the app heading in a better direction.
I think this massive commit message (on the final "drag-and-drop ordering feature) covers most of what I spent time on in all these details - trying to get collaborative changes to work consistently:
So, in using cortex, something that I observed was that if you moved an item up in the order everything was fine; it would find that out of order item in the client first, and move it. However, when moving items DOWN in the order, instead of just moving that one item, it would instead move all the intermediate items to acheive consistency with the server, even though the net effect was the same. That meant, when moving items down in the list, you'd move lots more thought boxes a short dista
nce in order to acheive consistency, instead of a few boxes a large distance.
This was terribly inefficient for two reasons. First, every shift requires a server request, so the number of requests to the server was significantly higher than the most efficient move possible. Second, it was inefficient for the client because it meant more total moves (and more time the user spends watching the screen, waiting for it to become consistent).
When thinking about how to solve this problem I originally thought that I would need to implement the diff algorithm to find the differences between client and server, and then perform the first move in the diff list with each server request. This seemed like a complex solution to what seemed like a pretty simple problem, so I punted on the solution and waited to see if I could come up with something better.
It turns out that, instead of calculating the diff for the whole list, that you could instead take the client list and the server list, and calculate an array of integers that represent the distance between elements. So, if the client and server lists were identical, you'd have an array of zeroes - meaning that nothing was out of order on the client, when compared to the server ordering. However, in the case that you move something down in the visual ordering, say by seven spaces, you'd get an array of distances that contained seven 1s, and one 7. Once you have that array of distances, instead of trying to move all the items that are out of order, you simply move the one that is furthest out of order. To say it another way - instead of moving the seven items that were off by one, you could achieve the same result by moving the one item that was off by seven. The net effect is that, whenever the client can shift just one item in order to make many items consistent in their ordering with the server, it will prefer to move that one item by a larger amount one time, rather than move a lot of items a smaller amount over several server requests.
So far this has proven to be both effective, and much less complicated a solution when compared to implementing a diff.
Tracking the proper order of the thoughts on the page is actually just a matter of a single column. Getting it to work collaborative, in real time, and to be "eventually consistent" across all client browsers? That's another story.
This added some scopes to the
Thought model to make retrieving the order of thoughts based on various criteria, a trivial thing.
Prerequisite for figuring out what the "next" sort order value should be for new
Thoughts added to the page.
jQuery actually does most of the heavy lifting when it comes to making drag-and-drop functionality work in the UI.
At this point I'm getting pretty deep into the problem. You can see me relying on a lot of debug output to figure out what's going on client and server side. This will continue for some time.
This step was important. While a user is dragging an item on the page, you want updates from other users to be temporarily suspended so you can drop a thought in the intended position. From there, collaborative updates can resumes.
This took changes from the master brach and updated the
ordering branch. This is what is supposed to happen when you're doing proper branching and merging - develop logically isolated features on their own branch, and integrate them when you're ready. It makes development of any kind a lot easier the more complex a piece of software becomes.
Sometimes, even when you know something isn't working, you have to commit what you've got before you forget where you were.
This commit included code that was entirely for debugging and testing. I clearly was confused as to why it wasn't working. I don't really add big debugging functions like this unless I'm confused about something.
Duplicates were sometimes generated during weird drag-and-drop operations. This commit corrected those problems.
When coding on the bus, this would happen to me all the time. This commit makes it so your connection can drop, you can put your computer to sleep, etc., but when you come back the periodical refresh will just come back to life, letting you pick up where you left off.
I also removed those huge JavsScript debugging functions.
At long last, I breathed a sigh of relief, knowing that I could move on to another feature, ending six days of continuous work on nothing but this feature.