|Back to Open Slate Home Page||Date: 2007.11.20 (2007.11.21)|
Chalk is the name of the Open Slate Project's pen-based user interface. It will be based on an existing window manager. The goal is to allow common Free and Open Source Software (FOSS) applications to work unmodified, but to reveive the full benefit of the handwriting recognition system it may be necessary to create Chalk specific versions.
The Open Slate reference design will use commonly available hardware components that function like a traditional IBM PC. The intent is to have a hardware platform capable of running a variety of FOSS operating systems such as Linux or FreeBSD with little or no modification. Some designers may want to explore alternative designs, perhaps using a chip set optimized for low power consumption. All such experimentation is welcome so long as the user land software operating environment remains compatible.
Staying close to the widely used IBM PC hardware design means that it will be easy to get X running, and once that is done there are a number of window managers to choose from. It should even be possible to run KDE or Gnome, either of which provides a consistent user interface across applications. This may sound like a good thing, but such an approach fails to take into account the Open Slate pen interface.
A mouse has buttons. Pens, pencils, and erasers do not. Neither do crayons, felt markers, and paint brushes. The Open Slate pen will not have buttons, either. Pressure sensitive input may be an option, but will play no part in the window manager user interface.
A user interface with only a button-less pen for input may sound like wings for pigs, but Apple has done it. The Newton can do everything with just a simple pen. The trick is the Newton's clever user interface design.
In order to provide a useful and satisfying user experience it will be necessary to develop a pen-based window manager and associated libraries. We call our pen-based user interface Chalk. The Chalk window manager will not be built from scratch. It will be based on an existing window manager, such as Blackbox or Metacity. We might even pursue a Chalk version of Gnome or KDE.
Chalk must take into account the conflicting goals of power efficiency and eye candy. Obviously a window manager that moves windows by drawing a ghostly wire frame will consume less power than one that redraws the entire window while filling in the exposed areas. Simple window ornaments will take less work to render than shaded gradients with transparency.
How much of a difference is there and how much difference will it make on overall power consumption? To do that we need to come up with a way to measure power consumption. Sounds like a topic for a research paper. Maybe we can get a grant to study the problem. The thing is, if we take this approach and study everything, we will never get it built. What we really need is common sense design guided by experience and a willingness to fail once in a while.
The Chalk pen driver will need to filter input to determine what the user is trying to accomplish. A mouse driver does this, but the pen is more complex. Here are a few situations inspired by the Newton that suggest how Chalk will work:
Text is selected by executing a tap-and-hold, then drawing a diagonal line through the desired text. Individual letters or parts of words cannot be selected; any part of a word that falls within the selection area causes the entire word to be added to the selection.
When the pen immediately starts to move after touching the screen the system will shift into input mode. If the object receiving input is a graphic object it will receive the raw input stream, the same as if the input device were a mouse with button #1 down. If the target is a text object the handwriting recognition system will intervene in such a way that the target only receives the result of the recognizer.
There is one more thing the Newton does that makes it such a joy to use. It retains a list of potential solutions for each of the last ten or so words recognized. The list even includes the Ink Text version of the word.
Double tapping a word brings up this list. The first entry is always the same as the current word. Sounds redundant but sometimes the actual text is drawn in a difficult to read font whereas the list is always in the very clean system font. The second entry is the same word with reversed initial capitalization. These two are followed by any other possibilities that scored high enough. The last entry is the ink text version, an image of exactly what was entered with the pen. At the bottom of the list box are two buttons, one to open a virtual keyboard, the other to open a correction tool.
About ten words later the word's recognizer data is tossed. (If I were writing this on a Newton, the recognizer data for "About" in the previous sentence would be lost around the point I entered "tossed.") After that, double tapping a word always brings up the first two entries, for easy correction of capitalization, as well as any similar words from the dictionary.
To implement this feature in Chalk will require more than splicing a pen interface into the X input device system. Either the recognizer will maintain the data and pass a pointer to the target application, or the target application will accept and manage the data. I suspect the former will work best. Perhaps all it will take is an additional X input event which consists of the text, one word at a time, along with an ID number. Standard applications will understand the traditional text input event and ignore the Chalk version, while Chalk enabled applications will be able to use the ID number to interact with the recognizer. More than likely there are other, better ways to do this.
As much work as there is to be done in other areas, it is vital that Chalk development get underway at once due to the impact it will have on how applications behave.Gary Dunn