In a previous post, I compared the trend of virtualization (coupled with remote access) to the trend of having more powerful hand held devices like the iPhone - which half of Bit Group now owns. I concluded that remote was the way to go, asked the question “how simple can we make the device?” and proposed (with no further details) the following:

The Further Details

So, the question was “how simple?” First, let me briefly reiterate why I want simple:

• The simpler, the cheaper. If I lose it or break it, I don’t lose a lot of data and it won’t cost a lot to replace
• “The less it does, the less it can’t not do.” If it doesn’t really do anything, it won’t get out of date (as fast). New software, hardware, features, products etc all go on the centralized systems. And, ideally, these changes become invisible.
• I want a lot of them. If it works well, why bother having any of my own “computers?” Just get a bunch of these, hook em up to I/O devices all around the house, and, everywhere they are, I have access to all my stuff.

Basically what I want, is a good ‘ol dumb terminal. Really dumb, and really cheap. The minimum it needs, as crudely indicated in the image, is

• Power
• USB
• A/V (just an 1/8″ jack)
• Wireless internet connection (maybe)
• Bluetooth (maybe/probably not)

So what exactly does it do? It gets an IP address and transmits data over TCP/IP. And what kind of data? 100% I/O. Essentially it’s a little IP KVM (though hopefully we’ll be using more interesting things than those ancient Ks and Ms). I’ll call it a ‘PIP’ (Personal IP, vs Personal Computer)

The key to keeping it small is the USB port. If you want to hook it up to your 30″ display, you’ll need a good USB graphics card, but if you just want just a VGA HMD, you can plug it directly in to the built in jack. If you want to plug in an ethernet cable, you’ll need the USB device for that. Otherwise use the built in wireless. (Or maybe wireless is a separate plug-in so the cost/size of the wireless component is eliminated if you’re just going to use it with wired internet)

The key to the viability of these USB plug-ins is going to be good design. (As in industrial design). If this is going to be portable, the plug-ins will need to stay plugged in, and the resulting shape needs to be something you can easily hold and put in a pocket. It can’t be a bunch of random things connected by USB cables all loosely jumbled together. Gross.

They are dumb, but they are not so dumb

So I lied. This thing is going to need to do something more than IO. First of all, it’s going to have to have some interface that lets you pick which remote system you’re connecting to. In fact it should probably save information about the different systems you have access to. It will also need to be able to pass through some USB devices to those remote systems while it uses others just for itself. It’s going to be some kind of “computer”, and it’s got to have some kind of OS. But that OS will be really small, so no “boot” time allowed, thank you.

And as far as data goes, the assumption is that there will be broadband everywhere very shortly. So if your data is stored safely, redundantly and centrally, you always have it. But in the odd case where you have to go without those sweet, sweet packets, the device should probably store important data for local access. So what’s important? Well, if you’re stranded in some horrible, awful place without the internet, you’ll probably want contact info. Names, phone numbers and addresses (physical…emails probably won’t help right now). It would be great if it automatically detected and saved contact information as you were entering text into your remote systems. In this ’stranded’ situation you probably also want directions to the nearest airport, so it would also be great if it had some location-awareness (either via some IP-based schema or USB GPS plugin) and automatically saved most-recently-found local emergency contact information.

For business purposes, it would be good if it stored copies of the most recent text-based documents you’ve been working on. I guess that means this thing should have a text editor too. Perhaps it just stores a log of all your recent text input. Perhaps the remote OSes will need something like VMware or Parallels tools installed to keep ‘recent’ docs in sync. Either way, most of this is just text data (and the software to run it all) and shouldn’t (in my mind) bloat the cost or size of the thing too much.

The Other Side

So now that we’ve got this awesome device, what do we do with it? This is actually where things get really interesting - where all the possibilities are unimaginable by one person alone. I’ll sketch out some ideas, but it’s probably better if you just imagine your own imaginations. Or some others’ imaginations (Amazon, David Young of Joyent). Warning - rambling ensues:

The basic core of it are these “remote” and/or “centralized” computing systems. These could be some central servers in your home that you maintain. These could be a hosting center running something like VMWare ESX, and you can create or destroy any number of traditional computers with traditional OSes. Pricing could be structured any number of ways. On these systems, I just want one logical networked drive. Just one. If I request (buy) more physical drive space, I shouldn’t have to change folder structures around. The available space just grows. When I create a VM, I have immediate access to all my existing stuff. These hosting centers should have automatic backup and restore systems that are more or less invisible to you. They should also upgrade old hardware and core software as it’s available. Other than the appearance of new features, this should also be invisible.

Now that we’ve gotten this far away from the PC, most of our OSes don’t make sense anymore. This new architecture - I’m over here with my input and output “things,” working with “computing things” somewhere else - should allow some new OS paradigms. Perhaps most of the software and tools you use are just collections of web services. I don’t need to be remoting into a whole Windows XP virtual computer. I can just connect to my weather “application” (I send the app inputs, it sends me back visual/audio/whatever outputs). This is NOT to say that the PIP itself should communicate with XML or XHTML based services and then have to transform that into visual information. Standards like that changes too often, you have to upgrade, etc. Lame. Better to stick with “pure” video signals (transmitted over TCP/IP I suppose…however that would work) that change every, oh, 20 years.

Perhaps instead of having a handful of connections for different VMs or apps stored on your PIP, you can connect to one ‘launcher’ ‘VM’ - an ‘OS’ that just acts as a proxy to other VMs and apps. Imagine that your start menu or apple dock is entirely independent from the rest of the tools you use. New versions, different UIs, etc can be made to replace your old one, and it has nothing to do with all the other applications you use. Every granular component of your computer is completely separate. Your dock is separate from your ‘windowing’ system (if such a construct needs to exist) which is separate from each ‘application’ or ‘tool’.

Another advantage of this is an even larger separation between the UI and the internal functionality. I can choose to use some tool, lets say the weather app again, along with any number of UI’s or ’skins’ for it. The tool itself allows requests for data from the UI app, which in turn exchanges I/O information with the PIP. (This is much like a web-services type of model, without the need for entire computers on either side). Realistically I imagine each app (or piece of an app) will be much like an entire PC, but with an OS stripped down to do exactly what it needs to for that app.

No more phones

Maybe a PIP is just a cell-phone-plus. Maybe there’s a cellular USB plugin to make it a phone. I think it’s more likely that cellular just becomes a broadband channel, and your cellular USB-plugin is just for data. Wherever you go you can communicate via whatever medium you like - text, video, audio (e.g. VOIP), touch… smell… It’s all just data transfer. The internet provides the medium, and devices on either end produce the results.

A PIP itself is nothing - just the “plug” at one end that you attach to a device that makes the little bits of information available to you in some meaningful way. A PIP is whatever you want it to be.

There’s plenty more philosophizing down this road…but I’ll save that for another post. So for now….

The List

In summary, (plus a few extra) here’s my ‘feature’ list of everything I’d want in a PIP. At least as far as I can think of for now. Now someone go make one! (I’ll even help if you want…)

PIP

• Uses simple communication standards (IP based?)
• A/V
• USB
• Wireless/bluetooth?
• Power
• ‘Boot’ is instant
• Minimal and necessary client-side data storage
• Minimal UI for configuring and selecting connections to hosted systems and changing settings for the locally connected USB devices

USB-Plugins

• Ethernet
• Wireless
• Sound card
• Vid card
• Cellular connection
• Temperature, accelerometers, GPS, and other sensors for all sorts of projects
• Plugged in devices can be applied to the PIP or passed through to the host OS.

Hosting centers

• Associate your PIP(s) with your account
• Pay service fee for access to hosting center (pay-per-use? flat rates?)
• Data backups restores
• Hardware & software upgrades

Hosted Systems

• Access to multiple VMs
• Virtual ‘network’ drives (all storage can be under one root, grows as much as needed)
• Highly scalable virtual hardware (for traditional-based systems)
• Stand-alone apps or tools
• Access to your other PIP devices via the central hosts
• Access to other people’s PIP devices via the central hosts (?)

I was going to use a cute title like “Here, There and Everywhere,” but after a quick search I realized I wasn’t being very original in applying that phrase to the issue at hand:

• NYTimes article (largely about Slingbox) - TV Here, There, Everywhere
• A historical look at the growth of computers - Software and processors: here, there, and everywhere
• Review of portable music players - Digital Music: Here, There, and Everywhere
• Distributed computing - Here, There, Everywhere.
• Ubiquitousness of computing (and thoughts on social implications) - Ubiquity: Here, There and Everywhere
• A legal review - Cyberspace - Here, There or Everywhere: A Study of Jurisdiction

Interestingly, I found almost as many relevant articles by searching for “here there everywhere computing” as I did for “local vs remote computing” “localization of computers” “remote computing” and “localized computing.” However, these drier searches did hit on better discussions on the concept rather than specific reviews of local or remote computing products.

The Concept

So what is the concept? Simply, it’s the observation of two trends:

1. More powerful portable devices (e.g. iPhone)
2. Virtualization (e.g. VMWare servers and Amazon web services) - particularly in combination with remoting technologies

(I’m considering virtualization as the virtualization of computing in general, not just running existing operating systems on top of virtualized hardware).

Generally speaking these trends are not at odds, but when you combine virtualization (v12n) with remoting technologies (software and hardware), you have a very compelling story for not caring so much about the trend towards more powerful portables.

The Story

It begins with a simple comparison of the local and remote options. “Local” means having a portable device that stores the data you need and is powerful enough to do what you want with that data. This could be anything from a phone to a laptop. “Remote” means having a device that provides the same user interactions (audio, video, text entry, pointing devices) but doesn’t have the power or storage to do all, or perhaps even most, of what you need. Instead it can just connect to the internet and allow you to interact with whatever you may have there.

As I see it, the main points of comparison are these:

Local Computing

• Everything is with you - no worries about whether you have a connection
• Fast - no network lag
• Other people don’t have your data
• You’re in charge

Remote Computing

• Nothing is with you - no worries about data loss from losing or breaking a device
• More available resources
• You can get at data without having to plan ahead
• Scales as needed

Balancing these issues is largely just personal preference. For me, this evaluation tips heavily toward “remote.” Generally I trust companies to store my data more reliably, do a better job of protecting it (while not using it themselves), and keep their hardware up to date more than I trust myself to do the same. If there are entire companies dedicated to doing pretty much just these things, why would I presume to be able to do it better myself - or, at least, why would I want to? (There must be at least one that’s good at it!)

The only real plus on the “local” side for me is not having to worry about having an internet connection to get at the information I need or the performance of whatever connection I have. However, we can see this problem slowly disappearing with the spread of wireless access (including airplanes).

Mix in some virtualization

The story began by comparing the ideas of “local” and “remote” by themselves. When we add v12n to the picture, I believe it bolsters the argument for “remote” beyond the question of personal preference. Why? To put it succinctly, its because it allows us to re-imagine human-computer interactions and lets us get there in small steps - not needing to take big chances on any particular paradigm-changing system. …This could (and will!) turn into a whole discussion by itself, so suffice it to say that the combination of remoting and v12n allows us to separate out the information processing completely from the interaction therewith.

Get Real

It’s a bit inaccurate to say that the “local” and “remote” trends are in competition. So why am I bothering to compare them? To get at the real core of the issue, lets start with something completely unrealistic and do a quick thought experiment. Imagine your ideal “local” and “remote” systems. Go ahead, do it. It’s fun to imagine.

…

…

…

…


OK, turns out this is a bit of a trick question. In imagining your ideal system for “wherever-you-are computing,” you probably don’t care about the implementation details. Those should be hidden and only the features should be important. In other words, your ideal “local” and “remote” systems would be identical (or nearly so) to any user. So lets think of some of those features you’d want.

• The systems themselves woudln’t break down.
• You’re going to want an internet connection, so infrastructure problems for ubiquitous access would be solved.
• You would trust (in terms of security) the systems storing your data.
• Your data would be safe (in terms of persistence) - you won’t lose it if something breaks
• Your software could be upgraded automatically
• Your would have access to as much power and storage as you need
• If something goes wrong (you can’t idealize that away) someone else would fix it and restore your data. Maybe everything would be redundant enough that you’d never even notice
• Your device would be “context aware” - know where you are, what’s going on around you, etc)

Two groups emerge from this list - features for the device and features for the systems that support the device. It’s obvious that the features for the device need to be implemented locally, and it seems clear that the features for the systems are best implemented on the remote side (to whatever extent that’s feasible). So now we see the real problem is not “Local vs Remote: Diverging Trends in Computing” but “Local vs Remote: Everything in its right place,” which fits well with the earlier statement that “…the combination of remoting and v12n allows us to separate out the information processing completely from the interaction therewith.”

The Proposal

I propose something like this (not to scale):

Details to come.

Recently I’ve been looking to get some plants (trees) for my apartment. At one point after googling for “italian cypress” I came across this link:

http://www.tytyga.com/VINE-BUSH-SHRUB

which soon lead me to this page:

http://www.tytyga.com/product/Italian+Cypress+Shrubs+and+Trees

Cool - the Italian Cypress trees I was looking for. Though I’d never before thought plants were so…. cool.

After looking around the site a bit, I discovered that that guy in the red shirt is a big fan of plants:
http://www.tytyga.com/category/Crape+Myrtle+Trees-Bushes-Shrubs-Plants
http://www.tytyga.com/flowering_trees

I also quickly discovered that these pictures were clearly photoshopped (and not always that well). And the “subliminal” messaging wasn’t so “subliminal.”
http://www.tytyga.com/pine_trees

I can get the idea that “sex sells” but, I’m not sure why this guys got a gun. (4th pic on the right - Is that attractive?)
http://www.tytyga.com/product/Wax+Myrtle

And…speaking of guns…
http://www.tytyga.com/category/Wildlife+Shrub+Plants

Can I plant some cheesecake?
http://www.tytyga.com/product/Tifblue+Blueberry+Bush

I suppose I could say something interesting about how this reflects on society or the current state of affairs for the “average” website, or something general about marketing…but no, I’m just amused. And for some reason really want to buy a Palm Tree.

As far as .NET generic collections go, you’ve pretty much got two basic types. There’s List<T>, which gives you an array-style type-safe collection of objects. And then there’s Dictionary<tkey, tval> (and a number of other related types), which you can use for hash-style type-safe collections. But what if you want a collection of objects that you access by key and index? Or, more subtly, take this example:

class Thing {

  private string txt;

  public Thing(string text)
  {
    txt = text;
  }

  public override string  ToString()
  {
    return txt;
  }
}

class Stuff : Thing
{
  public Stuff(string text) :  base(text) { }
}

class Program
{
  static void Main(string [] args)
  {
    Dictionary<Stuff, Thing> stuffAndThings = new Dictionary<Stuff, Thing>();
    stuffAndThings.Add(new Stuff("1: "), new Thing("One Thing"));
    stuffAndThings.Add(new Stuff("2: "), new Thing("Two Thing"));
    stuffAndThings.Add(new Stuff("3: "), new Thing("Red Thing"));
    stuffAndThings.Add(new Stuff("4: "), new Thing("Blue Thing"));

    foreach (KeyValuePair<Stuff, Thing> pair in stuffAndThings)
    {
      Console.WriteLine(pair.Key.ToString() + pair.Value.ToString());
    }
}

Which Outputs:

1: One Thing
2: Two Thing
3: Red Thing
4: Blue Thing

Right? Well, maybe not…
In all the MSDN descriptions of .NET Generic hash-type collections we see this perfectly clear little statement “The order in which the items are returned is undefined.” Fantastic. Although I’ve never seen items returned in any order other than the one they were entered, I guess we can’t rely on it. If anyone has any details of how/why/when items will be returned in any other order, I’d love to hear it.

So, now we’re looking for a class that will store key-value pairs that’s accessible by index (which also implies it will return items in the order entered). There is the System.Collections.Specialized.NameValueCollection class, which does exactly this, but only for string-string pairs. Unfortunately there is no .NET framework class to do this generically. The closest thing is the KeyedCollection<tkey, tval> class, which is abstract (and has some other interesting qualities that we’ll look at later), and the IOrderedDictionary class, which, as you can probably guess, is just an interface. So it looks like we’re up to writing our own class. Or search around and see if someone’s already done it for us (of course they have!).

After a bit of searching I found a couple implementations that do the job nicely, and discuss some of the performance plusses and minuses.

MarcClifton.com - KeyedList<tkey, tval>
CommonGenius.com - OrderedDictionary<tkey, tval>

It turns out that these two implementations are pretty similar.

The KeyedList class implements IDictionary<TKey, TVal> and IList<KeyValuePair<TKey, TVal>>.
The OrderedDictionary class implements IOrderedDictionary and IDictionary<TKey, TVal>.
Both classes have an internal Dictionary<Tkey, TVal> field and List<KeyValuePair<TKey, TVal>> field for storing all the data. This means that all the data stored by this class is doubled (or at least the number of references to each piece of data is doubled). Additionally, all of the add and remove actions need to be applied to each internal object, so the time for these operations is increased. As David Nelson (the OrderedDictionary<tkey, tval> author) points out, “OrderedDictionary should only be used when insert/delete operations will be minimal, AND there is a need to efficiently access elements by index and/or key.”

It feels like there should be an easier or better way to do this. Or at least somehow not have to store two separate datasets. One thing to note is that in these implementations, all the necessary data is stored, in order, in a List<KeyValuePair<TKey, TVal>>. So a very simple option would be to just use this list of pairs. For exmaple:

List<KeyValuePair<Stuff, Things>> stuffAndThings =
          new List<KeyValuePair<Stuff, Things>>

Or if you want a new type for it:

public class PairList<TKey, TVal> : List<KeyValuePair<TKey, TVal>

That’s it! Now you have a “leaner” implementation of ordered key-value pairs. But you can’t look things up by key. And there’s nothing to validate that the keys are all unique. And you have to use it like this:

PairList<Stuff, Things> stuffAndThings = new PairList<Stuff, Things>
stuffAndThings.Add(new KeyValuePair<Stuff, Things>("stuff","things"));

Ew. This isn’t at all what we were talking about, but now is a good time to really think about what you actually need to do. For our initial example (putting together a list of key-value pairs and getting them out in order) this does just fine. And you can solve the ugly ‘Add’ problem easily by creating a new Add method:

public void Add(TKey key, TValue value)
{
  base.Add(new KeyValuePair<TKey,TValue>(key, value));
}

Not bad.
But if we want to add the ability to look up by key, we have to do something like this:

public List<TKey> Keys
{
  get
  {
    List<TKey> keys = new List<TKey>();
    foreach(KeyValuePair<TKey, TValue> kvp in this)
    {
       keys.Add(kvp.Key);
    }
    return keys;
  }
}

public TValue this[TKey key]
{
  get
  {
    return this[Keys.IndexOf(key)].Value;
  }

  set
  {
    this[Keys.IndexOf(key)] = new KeyValuePair<TKey, TValue>(key, value);
  }
}

Now it’s getting ugly. Especially because every time you access by key you have to generate a new list of keys. Super slow. It definitely looks like this is not the road to take for a true generic “OrderedDictionary” kind of functionality.

So lets look back at this abstract KeyedCollection<tkey, tval> class.
If you derive this class you have to implement:

protected override tkey GetKeyForItem(tval  item)

Basically this means that it expects you to be using objects where the key is stored as part of the object (or can be systematically derived from it). This seems annoying at first, but might be a really good idea - you always have the object’s key value any time you need to access the object.

In particular, we can genericize things a bit to make it so we don’t have to create separate concrete types for every kind of thing we want to store:

public interface IHasKey<TKey>
{
  TKey Key
  {
    get ;
    set ;
  }
}

public class KeyedObjectCollection<TKey, TVal> :
   System.Collections.ObjectModel.KeyedCollection<TKey, TVal>
   where TVal : IHasKey <TKey>
{
  protected override TKey GetKeyForItem(TVal item)
  {
    return item.Key;
  }

  public void Add(TKey key, TVal item)
  {
    item.Key = key;
    base.Add(item);
  }
}

Lets try this out with our stuffAndThings example:

class KeyedThing: Thing, IHasKey< string>
{
  public string key;

  public KeyedThing(string keyText, string valueText)
    : base(valueText)
  {
    key = keyText;
  }

  public string Key
  {
    get { return  key; }
    set { key = value ; }
  }
}

class Program
{
  static void Main(string [] args)
  {
    KeyedObjectCollection<string, KeyedThing> stuffAndThingsTwo =
         new KeyedObjectCollection<string, KeyedThing>();
    stuffAndThingsTwo.Add( new KeyedThing ("1: ", "One Thing"));
    stuffAndThingsTwo.Add( new KeyedThing ("2: ", "Two Thing"));
    stuffAndThingsTwo.Add( new KeyedThing ("3: ", "Red Thing"));
    stuffAndThingsTwo.Add( new KeyedThing ("4: ", "Blue Thing"));

    //using the for instead of foreach to show that the items
    //_are_ available by index
    for (int i = 0; i < stuffAndThingsTwo.Count; i++)
    {
      Console.WriteLine(stuffAndThingsTwo[i].Key +
         stuffAndThingsTwo[i].ToString());
    }
  }
}

Not bad…except we have to declare the key type for the KeyedObjectCollection as string, even though the KeyedThing class inherently require’s its key type to be string. That gets a bit redundant.

It turns out that’s fixable by being even more generic:

public class KeyedCollection<tkey, tval> :
       System.Collections.ObjectModel.KeyedCollection<tkey, KeyValuePair<tkey, tval>>
{
  protected override tkey GetKeyForItem(KeyValuePair<tkey, tval> item)
  {
    return item.Key;
  }

  public void Add(TKey key, TVal item)
  {
    item.Key = key;
    base.Add(item);
  }
}

Now when you access the collection by key, it will return a KeyValuePair. This will probably be OK, but it also means you have to add everything by KeyValuePair, so there are a few extra methods we may want to add:

public ICollection<tkey> Keys
{
  get { return base.Dictionary.Keys; }
}

//This one's particularly ugly.
public ICollection<tval> Values
{
  get
  {
    ICollection <tval> values =  new List<tval>();
    foreach(KeyValuePair<tkey, tval> kvp in base.Dictionary.Values)
    {
      values.Add(kvp.Value);
    }
    return values;
  }
}

public void Add(tkey key, tval val)
{  base.Add( new KeyValuePair<tkey, tval>(key, val)); }

public void Insert(int index, tkey key, tval val)
{  base.Insert(index, new KeyValuePair<tkey, tval>(key, val)); }

public int IndexOf(tkey key)
{
  for (int i = 0; i < base.Count; i++)
  {
    if (this[i].Key.Equals(key))
        return  i;
  }
  return -1;
}

Here we see that the base abstract KeyedCollection class contains internal Dictionary (base.Dictionary) and internal List (base.Items) objects. It’s looking quite familiar to what we were doing with the OrderedDictionary and KeyedList, but the Dictionary in the KeyedCollection class is storing Keys and KeyValuePairs instead of just Keys and Values, which is not ideal.

However, there’s a lot less code and room for error. And, for better or worse, it’s using a lot more functionality provided in the .NET framework class library. Why might this be good?

Looking at the OrderedDictionary implementation, there’s this property:

private List<KeyValuePair<TKey, TValue>> List
{
  get
  {
    //_list is an internal field
    if  (null == List) //oops! Looks like we've got an infinite loop
    {
      _list = new List<KeyValuePair <TKey,TValue>>(_initialCapacity);
    }
    return  _list;
  }
}

Obviously it’s supposed to be if(null == _list) so it’s just a typo. But it’s one that doesn’t get caught until you actually run some code that calls the right methods on the object. This wouldn’t have happened if Microsoft had given you the class you needed.

After all debugging and testing is done, the OrderedDictionary and KeyedCollection classes are much cleaner implementations of a generic class for collections that is accessible by key and by index. However, there are times when the simple List<KeyValuePair<tkey, tval>> does everything we need, and others when some particular implementation of KeyedCollection may do the job best. This just point out again how a proper design process may save us a lot of refactoring down the road.