Carson Brown

I was fortunate enough to attend Google I/O 2012 back in June of 2012 (thanks Shopify!). One of the perks beyond the great sessions was receiving the Nexus Q, a device who’s actual release has been cancelled. It was, at the time, kind of neat, though spartan in terms of features: Android users can play music and YouTube videos to it over WiFi. That’s it. Under the hood, however, the device is running a full install of Android 4.0.

My first point of entry was enabling development mode. This is done from within the Nexus Q companion app, and was as simple as navigating to the advanced settings and flipping a switch. From the debug info screen, I could see what IP address the Nexus Q had on my network. From there, it was a simple remote ADB:

adb connect <IP Address>:4321

From that point, all standard ADB commands will be sent over the network—no cables needed! At that point, I opened the Settings app and tried connecting a Bluetooth mouse. How? You can always fire intents from an ADB shell; opening the Settings app goes like this:

am start -a android.settings.SETTINGS

From here, however, it’s a little tricky: there are no buttons to interact with what’s on screen, except mute and volume controls. It’s possible to send input codes as if they were typed on hardware buttons from ADB, though. This list of key codes really helped out. Entering a key code from an ADB shell:

input keyevent <Key Code Number>

Fairly simple stuff. But not all apps (such as Netflix) respond well to caret-style app navigation, so you can’t move from button to button with d-pad keys. For that, we can use MonkeyRunner. This was originally designed for controlling applications programmatically for testing, as part of your testing routine. To get this working from your terminal, run monkeyrunner (.bat, if you’re in Windows) from your Android SDK tools directory, and run each of these lines from the strange Java-meets-Python console:

from com.android.monkeyrunner import MonkeyRunner, MonkeyDevice
device = MonkeyRunner.waitForConnection()

From here, the sky’s the limit. Have a look at the full MonkeyDevice API, notably the touch() and type() methods. It’s certainly more painful that using an attached mouse, but it works. It’s worth noting here that none of these instructions are specific to the Nexus Q: you could do any of this with any development-enabled Android device.

My next step is writing a companion Android app operating as a hybrid remote launcher, keyboard and mouse. One helpful thing might be installing a launcher, I suggest Cyanogenmod’s Trebuchet (grab it out of a Galaxy Nexus CM9 build). In fact, many applications designed for a Galaxy Nexus ought to work here, as they share a lot of the same hardware. Just side-load the same way you would any APK—it even works via remote ADB like the above commands.

I’ve started watching Star Trek: The Next Generation from the beginning again. Instead of just enjoying it as a nostalgic trip through my childhood, I’ve been trying to actively watch what’s happening in the show. I’ve noticed a number of personality and behaviour patterns that totally clash with how I remember each character as a child. Bear with me as I review what the first half of STNG’s first season was like a second time around.

Futuristic humanity is still fallible. From the very first episode, the bridge crew must deal with the (still very annoying) Q and his needless pestering. Tasha Yar and Worf are quick to anger and violence, and Picard is stubborn, and not nearly as reserved as I remember him—he is also very quick to admit mistake. The crew is, however, able to readily accept the limits of their knowledge, and measure possibilities before dismissing them as impossible. The very first episode has giant space squid, and that’s all quite reasonable to the crew.

That said, knowledge in Roddenberry’s new future, appears heavily specialized. The moment the ship’s computer is unresponsive, the crew—save for basically Data, Wesley and Geordi—resorts to shouting and anger. An unresponsive computer, for a bridge member, is broken and his world is in peril until it is magically fixed. An example of this can be found in The Big Goodbye, when Riker breaks down to shouting and hitting an unresponsive holodeck control.

Mistakes and elitism still run rampant. While Roddenberry’s humanity has overcome global war, currency and the majority of illnesses, they quickly judge disagreeing foreign cultures as inferior or primitive, comparing them to how Earth used to be. To an Enterprise bridge crew member, the 20th century was a veritable dark age of unenlightened racist warriors.

To be fair, though, other cultures often have similar views of the Federation: the Ferengi cannot comprehend the uncivilized nature of humans clothing women or wearing golden communicators. STNG has not yet found its roots this far into the series, concepts such as the Prime Directive play a larger role as the series progresses, where suspension of judgement is required. The Prime Directive was argued then disregarded in favour of giving Wesley a deus ex machina exit from a planet’s death sentence.

The bridge crew is also quick to judge the less experienced. Nearly every guess and wager Wesley makes (that the audience can see) turns out to be true, and yet when he accuses Data as actually being Lore in disguise (from Datalore), Picard and his mother refuse his conjecture. This is after Data remarks—in the same episode—that Wesley’s mind is beyond any mere teenager.

STNG’s humanity isn’t smarter or necessarily better than our own. They have an advanced society, and have much technological and medical prowess, but individually they are merely the same old quick-to-judge humans in color-sorted unitards. There are serious advancements made as a society, notably the reduction of class, social rank and the acceptance of duty, but this is within the confines of the naval-like social contract of the Federation. They are not the super heroes of my childhood, and that’s a good thing. Seeing a relatable humanity is what makes the show more interesting as an adult.

We design and create interfaces with the user in mind. How does the user complete the task of composing an email? How does the user save a webpage to her bookmarks? In particularly complex systems, sometimes the solution involves examining more than just how a user can complete a single task, but how the system as a whole supports (or doesn’t support) certain actions. In this way, you’re creating a user interface that allows the user to complete their task in potentially myriad ways, not only what you intended. This is truly difficult, as predicting user behaviour is nigh impossible.

Borrowing concepts from the HCI masters and treat the system of user, device, and environment as a whole is one way of side-stepping the issue of not being capable of mind-reading. Edward Hutchins of UCSD has presented many different ways of examining these kinds of systems, but perhaps his best known example is that of an airplane cockpit. When treating the entire cockpit—it’s dials, pilots, controls and pedals—as a whole system flying through the air, how might it operate? It is difficult for an outside observer to understand the complex thought processes inside another person’s head, but it is straight-forward to understand what a cockpit is doing at any given moment. Hutchins, in How a Cockpit Remembers Its Speeds, applies the normal cognitive science analytical approach to the entire cockpit, rather than the mind of a single pilot. In particular, Hutchins examined how the system recorded and memorized the necessary speed limits on approach of a runway, as well as the wing configurations necessary for landing. It is a fantastic read.

The part I want to highlight here is the concept of speed bugs. Pilots had devised a system for when the flaps and slats on the wings needed to be adjusted and extended in order to gradually reduce speed and altitude for landing. The airspeed indicator works similarly to the car speedometer, and so the pilot-not-flying affixed mere paper clips to the frame of the indicator, denoting the points at which the flaps and slats needed to be changed. The benefit here is that the pilot-not-flying was able to project his internal working memory onto the shared memory of the cockpit. The pilot-flying can then access this information far more quickly than asking the co-pilot. The added benefit of attaching clips to the airspeed indicator is that the pilots can infer how close to the next wing configuration they may be—translating math into a visually inferred distance. These speed bugs were later adapted to become sliding clips pre-installed to the airspeed indicator, and now are part of the graphical UI now used in the latest commercial aircraft indicators.

In the end, the airspeed indicator happened to be created in a way that was extensible by users. It followed the general design of speedometers and other radial dials, so it was familiar to any who’ve seen them in other domains. The design of the speed bugs capitalizes on the social context between two professionals, and that humans are better at reading dials than lookup tables in a book. We can perceive the tendency of the speed from the direction and speed the needle is moving, and how soon the next wing configuration is required based on the proximity of the needle to the next speed bug.

A side-effect of the card-like switching between applications on WebOS and Windows Phone 7 has the added benefit of viewing more than one application’s UI simultaneously. A user can swipe instant messaging to the side to see what time-slots are free on her calendar. This extended use-case is implied by the interface, but isn’t the direct purpose.

A related issue exists when analogue interfaces are brought to digital ones in the extreme: bringing more than just the experience of the original, but all of its limitations too. A great re-implementation that we all use is Google (or Bing or Yahoo) maps: the limitations of scale, detail and annotations of paper maps are all removed, leaving only the core experience to be built upon by digital enhancements.

parislemon:

Why do I hate Android? It’s definitely one of the questions I get asked most often these days. And most of those that don’t ask probably assume it’s because I’m an iPhone guy. People see negative take after negative take about the operating system and label me as “unreasonable” or “biased” or worse.

I should probably explain.

Believe it or not, I actually don’t hate Android. That is to say, I don’t hate the concept of Android — in fact, at one point, I loved it. What I hate is what Android has become. And more specifically, what Google has done with Android.

Read More

This post, as a piece written for this personal blog, is designed to be sensationalist. As with all of my posts here, opinions expressed here are my own, not my employer (or friends, for that matter). The posts are designed to stir discussion and start a conversation; this post is no different.

If you haven’t had a chance to glance at the title of this, please take a moment to do so. Notice that the post isn’t that they do in fact want out of the race, but that they seem to provide a list of reasons as to why that conclusion may not be entirely fantasy. I’d like to list these as a pseudo-editorial, as both an argument and therapeutic exercise.

Before I delve into this flame-bait further, allow me to preface this with a secondary disclaimer: I’m an Android fan through and through. I’ve been experimenting, using and developing on the platform since it was initially released. I’ve used the emulator in all its stages (beta to 4.0), owned and loved two kinds of Android handsets—and gone through four Android phones in the process. The platform has novel concepts to develop within and experience as a user, and many of the new changes in Ice Cream Sandwich are exciting. I like Android.

With that said, the ball was dropped months ago, and no one seems to have noticed.

Unpolished Press Releases: I first formed this opinion after the ICS / Galaxy Nexus press conference in Hong Kong, on October 18th, 2011. The lackluster presentation, demurred excitement and horrible video quality led me to believe that Google was happy to take a backseat for Samsung. This is the legacy of the Nexus One, after all, the phone that was to change everything about buying and owning phones. That phone was designed from the ground up by both HTC and Google, too. I can’t feel the same way about the Galaxy Nexus. I mean, Nexus even takes a backseat to Samsung’s over-used Galaxy branding, and seems to reuse many components of the Galaxy S line.

Embarrassing Support: Combined with the atrocious number of orphans Google allowing to be left behind, is leaving me concerned about future purchases as a user, and confused as a developer. I’ve built AOSP. I’ve run my own compilations of CyanogenMod with custom modifications. I’ve done these things as a coping strategy when it was unclear if the Nexus One would be granted official support for Gingerbread. This was without a clear or adequate explanation, mind you, as Android 2.3 contains no technologically advanced changes to bar any 2.2-capable handset. It isn’t clear to me why phones aren’t being updated the moment the AOSP tree is updated. Oh wait, it’s due to carrier and manufacturer modifications that no one likes, or simply just a shirking of duties.

Google, as a customer and maintainer of its own Android OS, ought to be in control of how Google-loved devices (that is, those that receive the Android Market officially) are supported. News about a minimum support program came at Google I/O 2011, nearly three years after the first Android handset was released (October 2008). I’ve heard nothing about this program since I/O. It’s interesting to see that CyanogenMod, at the time of this writing, supports fifty devices.

Clandestine Releases: Google, in its infinite wisdom, has often released products with only questionable scoops on blogs or other news outlets, without building the serious fanfare one other company is capable of. The leaks before the final release of the Nexus One were ridiculous, and also surreptitiously stole the limelight from the Motorola Droid/Milestone, released only three months prior, totally alienating early adopters. The same thing is about to happen with the Motorola (Droid) Razr and the Galaxy Nexus. Why didn’t Motorola, soon to become part of the Google fold, not get a peek at ICS for their new flagship? Why is HTC determining their upgrade plans after the ICS press release? These giant manufacturers—pillars of the Open Handset Alliance—are kept in the dark about a product that is being built by the organization they’re a part of.

Late or Missing Source Code Releases: Honeycomb will never appear in the AOSP source tree. It was initially assumed that it was merely late, like Froyo or Gingerbread. After no official response from Google for so long, we now have answers such as from Matias Duarte’s interview with This Is My Next:

On Honeycomb we cheated, we cut the corner of all that smaller device support. That’s the sole reason we haven’t open sourced it.

I’ve heard from other comments from Google staff that the code is just too messy to be released, as APIs are unclear or whatever. How did Honeycomb get released to consumers at all then? Was it really the knee-jerk reaction to the iPad we all suspect it is? Six million in Android tablet sales says “yes” to me. I’m not holding my breath for ICS to appear in the AOSP tree.

Another interesting point here involves the downtime of kernel.org. AOSP’s most well-known location was unaccessible for weeks. Google obviously had a backup, as the development process is closed, but chose to do nothing in terms of providing a mirror until much later. Now that kernel.org is coming back up, there is talk of providing an alternative to that canonical hosting.

In summary: I’m not pleased with the Android legacy Google is creating. These problems, however large they may seem, can be mitigated if Google feels like taking point. Rather than the silent observer (whether that’s true or merely the appearance they make), they can dive into maintaining a line of truly Google-blessed handsets, ones without heavy manufacturer modifications and with clear support cycles. The Nexus line is too young to cite as an instance of this, as it should be more than just one manufacturer favourite at a time. I must say that as a consumer, all of the court cases of Apple v. Samsung decidedly cheapen their products.

The good news is that most of the “wow” factors of Android are the (closed-source) Google apps, such as Books, Docs, Gmail, Goggles , Google+, Maps, Market, Reader, Talk, Translate, Voice, and Youtube. For a platform where “all applications are created equal,” there sure are a lot of first-party apps, but these are well-designed and work on just about every device very well.

It remains to be seen if Google knows about / cares about / is already fixing the problems described above. As a developer and “expert” user, I’m awaiting their decision, but it feels like we’re in for a tough winter of growing pains.

I’m a huge fan of video games. The other day, I finally got a chance to sit down with the demo for The Gunstringer. I’ve been looking forward to this title since it was first announced, and downloaded it when it was released but never tried it. First off, I was able to sit down with the game rather than stand, which is unique to the title! That was a pleasant change, as most of the other Kinect titles I’ve played involve some form of mild gymnastics, and aren’t designed to be played for long periods of time. The uniqueness of this Wild Western game doesn’t stop here.

The game makes use of your left hand to manipulate an onscreen puppet, moving him left and right, and firing his gun with your right hand. The puppeteer aspect suits the Kinect platform particularly well, and in my opinion much better than moving an onscreen figure with your whole body—the Kinect sensor just isn’t slick enough to grasp complicated movements (such as when limbs cross). The illusion is broken the moment the avatar does not act as you do. In The Gunstringer, however, the movements are simple enough—and each hand moves differently enough—that it’s a fluid and consistent experience to what you’re doing with your body.

How can this argument apply to other platforms and experiences? In the smartphone space, we’ve all seen the trivial accelerometer-and-compass games: replicating labyrinth tilt puzzles, fake spirit levels and tilt-to-steer racing games are a dime a dozen. Can we do better? Yes: augmented reality apps like Layar are pushing the boundaries of what you can do with a sum of precise and powerful sensors like accelerometers and compasses. What else can an always-on Internet connection, Bluetooth, NFC, state-of-the-art GPS, ad-hoc Wi-Fi, accurate gyroscopes and multi-megapixel cameras do? More like, what can’t they do.

Returning to The Gunstringer, it brings a simplified puppeteering concept to the Kinect but does not limit gameplay to what’s possible with a puppet on a string. Your right hand, remember, controls the avatar’s gun. This is done by mimicking the firing of a pistol with heavy recoil. I even found myself making a gun shape with my hand while I did this, and thus the game had won me over. I’m not alone in that feeling. The Gunstringer’s puppeteer mechanic was streamlined such that it did not limit the game as it was adapted for the platform.

Now that the gimmick of the Kinect hardware has worn off, I’m looking forward to titles that make use of the platform in unique ways, as The Gunstringer has. I know they can do better than the sports games and dance games, and create an experience full of intuitive gestures and unique experiences.

Many times, the adaptation of a physical concept to digital ruins the experience. The perennial example is the calculator: why click buttons that resemble a real calculator with a mouse? Why not allow direct text entry, and skip the buttons? Why limit the number of digits visible, as the old liquid crystal displays of old did? When picking an experience to model your digital creation from, take only the experience, and leave behind the limitations of the medium. When designing, try winnowing the chaff that weighs down your experience.

With the explosion of applications and brands, I, like all humans in commercial society, have gravitated towards preferring some over others. In other news, the sky is blue and water is wet. I’ve found myself really enjoying using and discussing these applications with my peers, trading information about how to better use a product, or which one to switch to, even defending one over another.

Why do I like talking about them? Why should I bother giving them the free publicity and advertising? It’s not always a conscious decision. Websites I do this with seem to share the following in common:

Informal language: The language used in the applications reflect the informality and anticipated use. Groupon, for example, uses informal, sometimes silly lingo, filling in what I would expect to be boring legal gobbledygook with humorous jokes and quips. This works in a positive way to support the fun environment the site is trying to create. Look, they just got another bit of free advertising because of it.

Not using formal, complex or highly technical wording lends personality—and even credibility—to the site. It feels like there are humans on the other side of the application, and they seem more honest.

Encouraging commentary: If a site or application has a lot of features, users can’t be expected to read a manual. Coaching users to do the right thing, or providing simplified instructions at the tricky parts can solve the problem. Being silly or extra helpful in a playful tone is less serious, but sometimes that’s ok, depending on the intended users.

After submitting a post on Google+, the service offers an encouraging “Nice post!” message, along with some tips for editing typos or changing sharing options. They could have offered the same functionality via terse links before posting, but instead offer the information like a friend sitting next to you. It also encourages uses to amend information after submission, meaning mistakes are ok. Their target audience—at least during the open preview—are technically savvy Internet users, so this sort of post-action coaching works great.

In summary: if the target audience isn’t policy writers, lawyers or super-serious engineers, deliberately lending humanity—and readability—to the site’s copy text in the form of informality is a great way to befriend users, and also get the point across without causing users’ eyes to roll back into their skulls.

As applications move to mobile, the context and intended usage change. You can’t expect a swiss-army knife of a desktop app to provide the same functionality on an iPhone or Android device. This is already well known, and there are many blog posts and books and courses on this topic. It’s great for users, because well made mobile apps are tailored well to the mobile lifecycle they need to survive in.

When it comes to input methods on handheld devices, the same rules apply. You can’t expect a standard desktop date-picker to perform well on a touch-only screen. That’s why controls like the iOS UIDatePicker are so wonderful: they translate the desktop task to mobile. Some of the functionality is lost, however; in the date picker, showing which days are weekends doesn’t translate to the iPhone at all. These reductions streamline the process on a device dealing with a far lower attention span.

If you have the benefit of selecting the input methods for the app you’re designing, take to heart exactly what the intention of the input method is for. This includes your target users, attention span and amount of user effort. An expert user might be willing to use a hard to use but accurate input method some of the time, but if the usage scenario includes using the app while rock climbing, you’ll need to select an easier to use method, at the cost of precision.

In the Winter 2011 term of my PhD program, I took Robert Biddle’s course on Human-Computer Interaction for User Interface Design. I ran a preliminary user study comparing different touch-screen input methods for numerical input. I compared the two commonly found input methods of increment/decrement buttons and seek bar with thumb against the more whimsical method of a rotary dial—like a volume knob. The seek bar was the only absolute-scale input method, so values that users selected fell within the entire range of the seek bar used. Users “spun” the dial clockwise or counter-clockwise to increase or decrease the value for selection.

The study actually ended up revealing an interesting result: the timing to select a value with rotary dial wasn’t statically significant from the buttons or seek bar. Anecdotally, users preferred the dial as an input mechanism due to it’s perceived ease of use. While very common, especially on Android, dragging a thumb across a seek bar requires more concentration. Personally, I feel that buttons are simply tiresome and dated.

Think carefully about what emotions your input method of choice evoke. If you want a fun or highly interactive application, losing some precision might be okay. More serious or rigorous applications have different design constraints too. Simply going with the default input method for your platform of choice is not always the best idea.

The dial lives on, if you’re interested. A good friend of mine, David Underwood, has adapted my simple testing dial in his Magic: The Gathering life counter, Life Dial. Eventually, we’ll get around to releasing source so you can add it to your own Android apps.