top | item 7358333

Keyboard Ghosting Explained

142 points| taylorbuley | 12 years ago |microsoft.com | reply

97 comments

order
[+] ChuckMcM|12 years ago|reply
I am always amazed that keyboards still do this. We have scanning matrices because back when I was a new engineer pins on a microprocessor were both expensive (larger packages) and they required more gates in the CPU (more expensive CPU) or chip doing the scanning. Today transistors are free and surface mount BGA packages can put down a lot of pads pretty simply[1]. So your typical 104 key keyboard could actually have 104 actual wires that the CPU scanned. And even if you don't want to put down a 144 ball BGA (that is only 12 balls by 12 balls, logic is fast enough that you could connect all of the keys to a bunch of serial shift registers, and scan a complete bitstream of all keys 1000 times a second. Easily matching the reaction times of humans.

So where is that keyboard?

Just to be crisp here, an STM32F429 can have 135 digital I/Os in the 208 quad flat pack. It also has a megabyte of flash and 192K of RAM. It also has a USB Phy built right in so you can just wire it up to a USB connector. Its $8 in 1K quantities in that package. So parts : chip, PC board, 120 or so switches and plastic case and key caps. Sure its more expensive than the $0.75 AVR chip they currently use but we're talking high end keyboards here.

[+] leoc|12 years ago|reply
Indeed there are now lots of high-end keyboards with no (or effectively almost no) keyboard-matrix ghosting problems. Actually, many of them still have USB-related rollover limitations: it's hard to fathom how someone can design a brand-new USB keyboard with no rollover limitations in the keyboard matrix, price it at $100+, and still not think (or bother?) to implement the obvious multiple-USB-keyboards workaround—looking at you, Razer!—but there you are. Still, you can find ones (like the Sidewinder discussed in the OP) with competent USB, or you can fall back to PS/2 (welcome to the future).

To my mind the interesting problem is how to get NKRO down into every keyboard. I'm sure it's possible with sufficiently clever engineering, but even with an almost-zero cost burden over bog-standard 2KRO this is almost certainly one of those frustrating situations where mass quality is achievable but only if you can get customers and manufacturers to know and care enough.

> Sure its more expensive than the $0.75 AVR chip they currently use but we're talking high end keyboards here.

Would extra PCB layers would be another significant source of increased cost here?

[+] ballard|12 years ago|reply
A serial bus of unique identifier switches (1wire or similar) connecting all the keys with a single loop could be cheaper at scale and solve NKRO. Possibly work on many vastly different (a few buttons to hundreds) product lines with the same ASIC with say only one or two support components per application.
[+] klipt|12 years ago|reply
> However, because this requires printing a circuit board with over a hundred diodes, it is relatively expensive.

So we can stick millions of transistors on an integrated circuit, but a hundred diodes? Too difficult!

[+] M4v3R|12 years ago|reply
This is funny because the first keyboard I had, back in the 386 days (1992 I think), didn't had this problem. I remember playing two player games on it without any "ghosting". The keyboard was very solid - hard plastic keys and metal backplate. It was nearly indestructible. Even though new keyboards appeared over time, we still used this one for about 12 years, particularly because that all new keyboards had this ghosting issue. I never understood why. After the 12 years space key began to malfunction a bit and we had to throw it out. It was a sad day for us :(.
[+] Sheepshow|12 years ago|reply
Well $8 for a chip is extremely expensive. I bet a vast majority of keyboards on the market sell in bulk for less than $8.
[+] BoppreH|12 years ago|reply
When developing Flash games this was a personal hell. Many keyboards are unable to process Up+Left+Space, usually meant for shooting at northeast. In this case one of the keys is simply ignored. Worse yet the other directions work, so you get a hard to diagnose bug.

The users end blaming you for their character deaths and the overall experience is frustrating. The best solution I found was to use a different key for shooting, such as left Ctrl, 'a' or 'z'.

If you ever wondered why you can't use the large button on your keyboard to trigger the main action in a game, blame your keyboard manufacturer.

[+] simonh|12 years ago|reply
Or a small, but significant number of other people's keyboard manufacturers. Welcome to lowestcommondenominatorland.
[+] dools|12 years ago|reply
I wonder why they scan the keyboard as a set of columns and rows!! When I was at University, I created a 49 note midi controller from a discarded electric piano keyboard.

Basically I got the hardware and wrote software for a Motorola 68hc11 to control it. The way it worked was that you would put a value on an 8 bit register that would activate one "bit". This bit would be connected to 8 different keys on the keyboard. You would get an 8 bit value out the other end that would tell you which keys were connected at the time.

So for example the value:

01000000

might correspond go keys 1, 6, 11, 16, 21, 26, 31 and 36. If you got the value:

01011001

out the other end, you knew that keys 6, 16, 21 and 36 were pressed. The 68hc11 scanned so fast that you couldn't fool it. I remember my professor standing there for ages mashing the keyboard trying to get it to break with different key combinations.

If such an old microcontroller could handle 48 keys so easily, why the hell would a modern microcontroller not be able to use the exact same method? ie. instead of doing it in "rows and columns" just treat the keys as a linear array and deal with which keys are pressed and which aren't.

[+] dfox|12 years ago|reply
Your scheme is essentially same as is used by keyboards and has the same problem. When you press two keys that are connected to same output wire, additional keys on these two inputs are ambiguous. There are some software tricks related to the fact that nobody presses multiple keys at exactly same time, but the root problem still exists.
[+] qnaal|12 years ago|reply
Fun Fact: The standard USB keyboard protocol can't handle more than... 6? non-modifier keys being pressed at the same time.

IIRC, to get past this, all you need to do is emulate a hub with fifteen or twenty keyboards attached. Then you can press every key at once!

[+] Zr40|12 years ago|reply
Not true. A USB keyboard must declare the number of simultaneously pressed keys it is able to report. It just happens that the example descriptor included in the USB spec has this value set to 6.
[+] ilyanep|12 years ago|reply
Yup. I'm using a Das Keyboard, and can only get six keys at a time on their applet. One way some people get around this is by using PS/2.
[+] reidrac|12 years ago|reply
I wrote a shoot 'em up for PyWeek and I got a report from one of the judges of the challenge saying the controls didn't work very well. I didn't know what was going on at first, because looking at the code it was "perfect", but turns out using the arrow keys for direction and space for shooting was a bad idea because a diagonal movement (up and left) when shooting won't work in some keyboards.

Since then I use z for shooting and problem solved :) Avoid three keys combinations in the same area of the keyboard and it will be fine.

[+] Perseids|12 years ago|reply
Just a note from the non US world: Please have such keys customizable. z is an especially awkward choice, because with qwertz it is between the t and u and thus far from all the other keys you might use for your gameplay.
[+] couchand|12 years ago|reply
Arrows and space are the same area of the keyboard?
[+] moron4hire|12 years ago|reply
No, no, no, no. This is all wrong. This is not why ghosting happens. This is how ghosting happens. The why is "because stakeholders are cheap bastards". Manufacturers are cheap. Consumers are cheap. Consumers don't want to pay for quality and Manufacturers won't hire quality engineers.

But there is an even bigger issue: engineers are cheap, too. Once you get a design that "works", for varying values of "work", it's like pulling teeth to get an electrical engineer to change anything.

Some of it has good reason. It's really difficult to prototype new circuits. We can prototype new code as fast as we can write it. Have you ever tried using Eagle? Or LabView? Constantly fighting with a goofy UI to build circuit designs, and then you really only have a simulation at that point. To get the real thing (analogous to compiling?), you have to fab a prototype, which fewer and fewer places are doing in-house anymore, which will take a day, or maybe a week. I compile my software 15 times an hour!

And it doesn't necessarily have to be this way. LabView sucks because EEs are shitty programmers. Every hardware shop I've walked into, I was the guy to suggest they start using such outlandishly cutting edge technology as source control. It's been even harder to convince some of the older engineers that no, nobody uses CVS anymore, and the little you remember of it from 15 years ago isn't going to help you anyway.

"Oh yeah, let's write our own atoi(). That sounds like a great idea. What's that? You didn't realize that I'd be sending you a direct copy of what was on the LCD screen? Why wouldn't you want to know that the LCD shows a space in the 10s column between the negative sign and the value when the value is more than -10?"

So yeah. Everyone is a bunch of cheap bastards. I'm not saying it's any better on the software side. Software engineers couldn't be counted on to put a florescent light tube in the right way if their life depended on it. But at least most software engineers try to abdicate hardware responsibility to other people. EEs think "it's just code, anyone can do it."

[+] chinpokomon|12 years ago|reply
In the high school computer lab, after I showed my friends how to disconnect from the Novell network so our instructor couldn't watch what we were doing, we used to play a very good Street Fighter 2 clone. I quickly learned all about keyboard ghosting and used it to my advantage to block my opponent from blocking my attacks.

When I tried the same tactic on my computer at home, I discovered it didn't have the same problem. The chords that were unrecognizable at school worked just fine on my keyboard at home. I surmised that the difference was in how the keyboards detected the key presses.

Over the next few days/weeks I wrote a program that could actually inspect the key queue in memory and mapped out the chords which would fail. I could actually map out what appeared to be groups of keys that caused this problem. I did the same thing for the keyboard I had at my house and while there were some combinations that would lock up the keys, I found that it could handle many more simultaneous keys.

Maybe it wasn't so surprising that the next year I started my EE degree?

[+] comex|12 years ago|reply
> The internal electronics on the SideWinder X4 use a variant of resistive multitouch technology. Each key has a screen printed resistor in series with its switch. This allows the internal electronics to read the state of each key switch independently for very large multiple-key combinations.

This is pretty vague. How exactly does this disambiguate which keys are being pressed?

[+] comex|12 years ago|reply
(Unrelated to my other post here...)

Aside from gaming, one application that's hurt by rollover is the open source stenotype program Plover:

http://plover.stenoknight.com

I would like to try it, but I tote around a MacBook Pro and I don't care enough about Plover to try to carry a keyboard around and awkwardly put it on top of the built-in keyboard or something. Still, this is one of the few cases where I feel constrained by the unreplaceable hardware.

[+] gggggggg|12 years ago|reply
Microsoft make some of the best keyboards there are. Just did some tests on my Sidewinder keyboard, no issue at all there.
[+] kabdib|12 years ago|reply
Microsoft used to make some really good keyboards. But about 8 years ago they started making junk. The last MS keyboard I could actually type on was the Wireless Multimedia one (now discontinued).

Keyboards in the last few years have had:

- Mushy key feel. I can't tell when I've actually done a keystroke.

- Keys that are hard to strike (need to press on them just right, or they hang up or don't register). This is particularly bad on the wider keys.

- Keytops too close together; I mash multiple keys.

- Embarrassingly bad features: Function-key lock is the classic one here. Let's not talk about the 'diamond' arrow keys that have occasionally appeared.

I'd buy half a dozen "Natural" keyboards with decent Cherry hardware running the show, ESC in the right spot, and no wacky features. I don't even need a numeric keypad. I'd pay more money for some remapping (though just about any 20 cent microcontroller will have enough EEPROM to do this).

Whaddya say, Microsoft? Care to make a good keyboard again?

[+] francispelland|12 years ago|reply
Wish I could say the same about my Logitech keyboard... Sometimes it gives up after 3 keys, other times after 5. It seems to be totally random.

I also wonder if there is a problem with the diagram. Sometimes when I was pressing 10 or more keys, the enter key would turn on.

[+] taylorbuley|12 years ago|reply
It's easy to take good hardware design for granted. Suddenly that Das Keyboard I've been eyeing doesn't look so expensive...
[+] wvenable|12 years ago|reply
My Microsoft Natural Keyboard 4000 has some really annoying ghosting that forced me to remap keys in at least one game because I couldn't move using the arrow keys and shoot at the same time.
[+] ilitirit|12 years ago|reply
I remember having to work around this when I created an arcade stick with the guts of an old PS1 keyboard. I used an Excel Spreadsheet to map the keys and to figure out which combinations of inputs would result in the least amount of blocking/ghosting in practice.

So basically, if you have a two-stick console with 6 buttons for each player, you had to wire it in a way that was impossible for Player 1 to ghost/block Player 2's keys (and vice versa). So when you found a combination that ghosted/blocked, you'd just map them to opposite directions on the same side since it's not possible to press UP+DOWN or LEFT+RIGHT at the same time on a joystick.

Here are some details: http://arcadecontrols.com/arcade_input.shtml#KeyboardGhostin...

[+] Avitas|12 years ago|reply
This explanation is poor. The keyboard controller and interface are crucial. NKRO is possible, but can it can be flaky over USB. There are many variables involved.

Want to learn more? Hit the geekhack forums (search for NKRO) or geekhack channel on freenode.

[+] malkia|12 years ago|reply
One of my favourite games - Star Control II had a DOS program to test what keys were not blocking - so two players were able to choose 2x6 = 12 keys in total that would never block.

And then for a while was the craze for finding this good old keyboard (IBM, HP, whatever) that did not block to play SC2 and other games (back in the days multiplayer was divinding the keyboard - to two - and much better fun if you ask me)

[+] beagle3|12 years ago|reply
The ZX Spectrum in 1982 (and I'm quite sure its ancestors ZX-80 and ZX-81 from their eponymous years) had a keyboard just as cheap or cheaper, with a similar matrix arrangement), but had no ghosting at all:

Instead of having an "all hot" electrical configuration, it would cycle through the rows with a "one hot" configuration, and read the columns. If I recall correctly, it would be something like 20 cycles of a 4Mhz Z-80 to read one 5-column row (you'd need 8 of them to read the entire 40-key keyboard), and it was done every 50hz/60hz interrupt by the main CPU - modern keyboards have an ASIC on par or 100 times faster than a Z-80 just for the keyboard.

[+] Bhel|12 years ago|reply
In which cases are such keyboards necessary?

I use a $3 one (which I bought to replace the one I used for over a decade) and I've never had such issues. I use this keyboard for coding and gaming, not just for common tasks which require low performance.

[+] garrettgrimsley|12 years ago|reply
I use a Model M and sometimes encounter ghosting. Some types of gaming are worse than others. For example, while playing RTS or MOBA I do not encounter ghosting while playing FPS I encounter it on a regular basis.

This is because in RTS and MOBA while you might be rapidly hitting your keys you should be hitting them in succession, not simultaneously! In the FPS on the other hand you might be trying to walk forward and right while crouching and reloading, which requires four keys to do. This would cause problems for my keyboard.

Programming is more like the RTS than the FPS in this respect. You may be hitting a lot of keys, but the combinations are more likely to involve modifiers such as CTRL, ALT, or shift rather than a combination of letters.

[+] User8712|12 years ago|reply
I like how they mention keyboard manufacturers using the term up to as a way to deceive customers, and then in the next paragraph mention the same phrase when referring to the Microsoft keyboard.

'Another marketing strategy is to state that the keyboard allows "up to" some large number of key presses.'

'Microsoft's SideWinder X4 features multitouch technology that allows it to detect, and report ANY combination of QWERTY keys, up to 17 keys.'

[+] mistercow|12 years ago|reply
If I understand correctly, that's just a slight error of wording. They actually mean "any combination containing less than 18 keys".
[+] khc|12 years ago|reply
> The next div class="asContentParagraph" discusses

Placeholder fail