A short history of the Hindsight Letterbug, circa 1986-1987

About

The Hindsight Letterbug was conceived in 1986. MultiMate International (the makers of the then best-selling word processor MultiMate) had entered into a sale agreement with Ashton-Tate (the makers of the best selling database, dBase). The sale of MultiMate (the product) and MultiMate (the company) for $21 million was going to leave a number of people in the company with more than a little pocket change. Wil Jones, MultiMate’s president and founder, was planning to buy himself a little boat (93 foot??) and sail around for a while.

Aboard the Pegasus, Wil wanted some sort of software to help his kids learn and practice handwriting. He had an Apple ][ gs and was hoping to find *something* that would make the kids’ time with the computer something better than babysitting. This was long before anyone dreamed that we’d soon have satellites to let people use the internet from the middle of the ocean.

Michael Wiggins was one of Wil’s partners at W. H. Jones & Associates when MultiMate was written and the first employee of the new company. He ran the software development effort for a while and eventually ended up on the front lines of new product conception and development. Through a complicated sequence of events, Howard Eglowstein, a MIT Media Lab* guy ended up working with Mike in the (very small - 2 people) advanced product development group after developing the Coleco ADAM just across the river.

When MultiMate sold and Wil cashed his check, he asked Mike if he knew of any products (hopefully for the Apple) that would help his kids. We talked about what was out there and it became clear that the problem was a dearth of hardware as much as software. Howard had worked with touch screens since 1979; the concept for the Letterbug appeared quickly on the back of a napkin or one of Mike’s legal pads (we’re not sure which).

It seems primitive and obvious now in 2014, but the Letterbug was a touch screen tablet computer. It had a (then) high resolution plasma display screen, resistive touch screen, 8086 processor, 512K of RAM and 256K of ROM. Besides looking cool it talked - it had a Votrax SC-01 (found in pinball machines at the time) and speech input that would later be used for recording speech and then using it as part of lessons.

Innovations

Ignoring the obvious - that this was the first tablet computer shown publicly (at a teaching technology show in 1987 - a fact later confirmed by IP attorneys at Wilmerhale prior to a trial in 2011), the Letterbug did a couple of things that had never been done prior or since.

  • Because it was intended to teach handwriting to dyslexic kids, the educators on the Letterbug team stressed the importance of the pen *feeling* correct to the kids. The touch screen surface was a piece of plastic laid over a conductively coated piece of glass. It’s a resistive technology that has largely been abandoned. Hindsight had the screen custom made by Elographics out of Tennessee to fit the Finlux plasma display. The trick was to make the experience as close to using a pen on paper as possible; to manage that, Mike and Howard measured the coefficient of friction of a #2 pencil on school paper. They then worked with Elographics and looked for a stylus material that would give the same feel as a pencil on paper. With a little tweak of the coating by Elographics and a custom stylus tip based on a Sanford felt tip pen (treated with a special concoction instead of the usual Sharpie ink), they got what they needed. "Electronic paper".
  • Displays in the mid 1980s were were small, high power and expensive. Nothing to do about that but there was a serious issue that could be fixed. Parallax. When you try and touch a target on a screen, you want the tip of the pen (or your finger nowadays) to line up with the target. While we take that for granted now, at that time it was a big problem. The actual lit pixel on the plasma screen was covered by several layers of glass. Add an air gap and another layer of glass for the touch screen, another gap and the plastic conductive sheet and you had at least 1/8" and up to 1/4" of space between the stylus tip and the actual pixel. To work around that, the Letterbug had a geometric model built into the screen & touch panel drivers. When it presented you with a target it knew where you were supposed to touch. When you touched slightly to the left, right, above or below the target, it watched those errors and built a 3D model of where your eye would have to be for you to be making those errors. As you moved your head and made different touch errors, the model tracked your position and could do so with amazing accuracy. The end result was a touch system that behaved very much like a modern touch screen but starting with a serious handicap.
  • With no keyboard, we needed a way of accepting text input on rare occasion (a student may need to enter their name, perhaps). To do that, the Letterbug used an onscreen keyboard that Howard developed in 1980 at the MIT Media Lab* for a fingerpainting system (also intended for kids).
  • The Letterbug used an expensive screen and needed to be as compact as possible. To help keep costs down and make it smaller, there was no magnetic storage. You didn’t load programs into the tablet with a floppy; since the device was intended for use in a Special Ed. classroom, it was built with a network interface. The units in a classroom would be daisy-chained together with a host computer on the teacher’s desk at one end. The teacher’s computer would store the software for the tablets and let the teacher interact with the kids. When you powered up a Letterbug it put itself on the network and looked for a host to boot from. From a topology standpoint the network (it never did get a clever name or acronym) was a mash of a token network, bits swiped from the implementation of Coleco’s ADAMNet and from elsewhere. The host computer was originally supposed to be an Apple ][ since that’s what classrooms had but when Apple wasn’t interested in being supportive, Tandy stepped up and the host became a Tandy 1000 running DOS. The prototypes could also talk on a network simulation to a Macintosh.
  • Multiple device inputs - since we had limited resources and often only one of a piece of hardware, the ‘touchscreen driver’ actually accepted input simultaneously from a number of sources. The prototypes could use graphics tablets, joysticks and a PC keyboard for input as well; blending all the data together and letting the application think it all came from a single device. This seemed trivial at the time but turned out to have interesting implications as prior art for several legal cases.

    • Prototypes

    The first Letterbug prototype drew from an old-fashioned school slate. We had originally thought it should resemble a workbook, but the technology simply wasn’t available to make anything truly as thin as a school workbook. We then thought about using the image of an old timey school slate. True, kids in the 1980s have never seen anything like a real school slate, but the concept made some of the more experienced schoolteachers on the team smile - so a slate it was. The frame was made of ash and the bezel was an actual blackboard surface. It seemed like fun to let kids customize their Letterbug with chalk. To accomodate the bulky hardware in the first prototype the case was taller than we liked. We raised the back end and slanted the case to help make it better ergonomically.

    By the time we built prototype 2, we had slimmed down the hardware substantially and things all fit in a case 1 inch thick. The ash frame turned into a sleek ivory-colored plastic case and the chalkboard surface turned into black plastic. The case design was done by Bob Gault of Gault Designs, a noted designer of television and radio cabinetry in the 1970s and 80s. Since it was much harder to make sample cases back then (no 3D printing!!) this second prototype is made of cabinet-grade birch plywood.

    And?

    With a functional prototype it was time to get support to finish developing the content. The team by then included several Special Ed. experts and handwriting teachers (remember when schools taught handwriting?) who all contributed content ideas. The software for the tablet was written in 8086 assembly language for the Letterbug’s custom API. This API would later find new life as part of Penmanship’s KidPad in the mid 1990s. What would eventually be named PIGS (Personal Interactive Graphics System) made it pretty easy to write some fun and educational things for the Letterbug. The demos included connect-the-dots to help with hand/eye coordination, tracing letters on ‘lined paper’ with the stylus while the system offered feedback in real time, or games which emphasized letter shapes. The first of these was a racetrack shaped like the letter ‘S’. The kids used the stylus to move a car from one end of the track to another without hitting any of the hay bales along the sides. It wasn’t about speed, but accuracy; the goal was to get from one end to the other on the ideal path without straying out of the track. The Votrax helped out by making motor noises and simulating special effects as only an early-80s phoneme synthesizer could do. A description wouldn’t do it justice, but imagine making motor noises by saying "rmm, rmm" and wiggling your finger up and down on your lips.

    A presentation to the MIT Enterprise forum in December of 1987 was promising but made it clear there was work to do. The audience and panel feedback consistently said the device was impressive but (a) the apps had to be done before we could sell any and (b) we were about 10 years ahead of our time. The Hartford Courant did an article on the evening’s presentations.

    In late 1988 or early 1989 it became clear that our desire to get these in the classroom was not going to be enough. Special Ed. administrators in CT were reluctant to let us set up a system in a school so we decided to relegate the Letterbug to the history books. The prototypes ended up in Mike’s spare room closet, the trade show materials ended up in Howard’s house in NH and the idea of a display tablet computer disappeared for a short while. Who actually came out with one first is still being contested. It is widely believed that The GRiDPad, designed by GRiD in Fremont, CA** was the first self contained, commercially *sold* tablet in 1989, two years after the Letterbug appeared at shows. The Letterbug was always photographed from all angles and by companies including IBM and Apple. It is not known whether it ever directly influenced any later designs.

    *The MIT Media Lab proper didn't exist until 1985. Prior to that it was known as the Architecture Machine Group and was the bulk of the 5th floor of Bldg 9.

    **I was pleased to have been contacted by Gavin Peacock, one of the designers of the GRiDPad who helped clarify a common misconception that the first GRiDPad was a Samsung design. It was done by GRiD's engineers as an internal project. - Howard