There has been tremendous development around input devices and methods for all kinds of computing device form factors in the last 12 years that has been accelerated with the advent of smartphone and tablet computers. But the foundation of all input - text input has only seen gradual improvements, but nothing that changes ho we think about text input.
I think this has to change and it can be changed if a big player - Apple, Microsoft - sets their minds to it. Here are some ideas.
Over the years (since the early 90s), I've been using a lot of devices like iPhones, iPads & iPad Pros, MacBooks, Android smart phones and tablets, a Chromebook, Surface Go and of course PCs & laptops. I've used a lot of input methods as well. E.g. keyboard, mouse, track point, trackpad / touchpad, touch bar, touchscreen, virtual (touchscreen) keyboards and pens.
Currently, I favour the mechanical keyboard and trackpad of Apple's MacBooks and the touchscreen of my iPad Pro. I don't like typing on virtual keyboards because I don't trust the resulting text is what I wanted to type - no matter which OS. I have no use for mice anymore because I always have a touchpad available and really like the gestures. I like Apple's Magic Keyboard Folio for my iPad Pro (12.9") but I'm not sure if I'll take it with me once we can again visit customers or if it just stays on my desk. I like the Apple Pencil for taking notes.
So, how would my ideal input device work?
First, I want to have touchscreens everywhere - i.e. on my MacBook as well. But I also acknowledge the fact that you can't just slap a touchscreen on a desktop OS and be happy. Microsoft is still demonstrating that this is not the best approach - although probably better than no touchscreen at all. So I'll ignore touchscreen input - at least for the main display - for the moment.
I'm also focusing on Apple products as I am currently comfortable in this world. But the ideas should apply to other ecosystems as well.
Here is what I want:
a mechanical keyboard
a touchpad
dynamic keyboard layout
one input device connected to multiple devices and easily switchable
My ideal keyboard would on first glance pretty much look like the bottom part of ab MacBook. The top part would be a mechanical keyboard and the bottom part would contain a touchpad. I like having keyboard and touchpad combined and the laptop layout does not bother me. I think this is the best setup for mobile use cases.
I'm envisioning a keyboard that combines the benefits of mechanical keyboards and virtual touchscreen based keyboards. Therefore the keycaps of this keyboard would be topped by ePaper displays - yes, I'd also accept color-OLED displays, but I think the benefits would only be marginal. By each key being it's own display, ...
The user could dynamically switch between different layouts, e.g. US layout for coding and your national layout for writing.
Apple could reduce the numbers of SKUs dramatically because just one - universal - keyboard would be enough.
Users would be less locked in to a keyboard layout - everyone could decide for themselves which layout to use. This would also enable new, improved layouts to gain adoption over time.
Users could see which characters they have available when they press a modifier key. This would make alternative key assignments be more discoverable.
Applications could provide their own keyboard layouts - especially those apps with lots of shortcuts and little text input could make those shortcuts more discoverable. No more cheat sheets for app specific keyboard shortcuts.
Users could design their own keyboard layouts and share them with others.
First, this keyboard would require a new HID-type (human interface device). AFAIK, keyboards are expected to be more or less hard wired. they send a key ID and the OS maps the typed character based on the configured keyboard layout. This is rather dumb. The new HID-type would have to be able to bidirectionally communicate with the device.
load layout and keycap symbols from device to keyboard
send selected symbol from keyboard to device
send control messages in both directions
This should not be a big problem, but it means changes in the device OS which could be a problem regarding backwards compatibility. This also implies the question whether to standardise first or start with something proprietary.
Second, graphics. Let's assume each of 6 rows of keys has ~15 normal keys and a normal key has 128x128 pixels (roughly 300 dpi) in b/w. The resulting display would be 15x128=1.920 pixels wide and 6x128=768 pixels high with a color depth of 1 Bit. This is a Full HD display with 1 Bit color depth (~1.5MB) display that has to be controlled by the keyboard control - not the computer. The keyboard controller would have to keep many keycap symbols in (persistent) memory to be able to efficiently switch between layouts. This is no insurmountable hurdle but it may increase the cost. The required processing power would also require a compromise regarding the keyboard battery.
Third, I have no idea whether you can put e-ink displays on keycaps and create a keyboard that is acceptable design-wise. I don't think that Apple would release a keyboard that looks like the Optimus Maximus keyboard.
This keyboard has to offer wired (USB C) and wireless (Bluetooth) connectivity. Wired connectivity is necessary for charging anyway and may also be necessary if Apple implemented Touch ID in this external keyboard. I think it could also be valuable to offer several types:
Just the input device - Keyboard and touchpad with wired and wireless connectivity and a battery that allows for sensible charging intervals.
Desktop Hub - Keyboard and touchpad separate, but the touchpad can be attached to the keyboard magnetically - left, right side or in front of the keyboard; wired and wireless connectivity and a battery that allows moving the device around the desk; works as a hub - USB C 3.0, 3.1, 4.0, TB, Audio, ...; studio microphones and speakers, card reader, ...
Mobile Hub - smaller and lighter than the desktop hub, but still providing the connectivity for mobile use cases; bigger battery that can charge mobile devices; PD from a power socket.
Some features would be nice to have but not essential.
Touch sensitive keys ...
use gestures on single keys, e.g. selecting capital letters by swiping down on a key or tap and hold a key for accented characters.
use gestures on the space bar, e.g. a value on a selected control or change formatting on a selected text without switching to the touchpad.
use gestures on a row of keys in specialized apps like Photoshop, e.g. fine grained control of multiple values.
use the whole keyboard as a touchpad.
Touchpad with display
enable SW controlled buttons
Display / UI for iOS apps that run on the MacBook.
There are some things that I don't need.
RGB lighting
Maybe I'm too old to adopt the new way of typing on a touchscreen. But if this is the case, them there are many even more reluctant to change. It is time to improve the way keyboards work without sacrificing what is best about them. That is to me adding the flexibility of the iPad virtual keyboard to the feedback rich tactile mechanical keyboard.
Combine this with the Touchpad in a layout that we are all used to by using laptops and you have a winner.
I have - sadly - payed €399 for the 12.9" Magic Keyboard Folio and would pay even more for my ideal keyboard.
For Comments, please respond to this tweet: