New keyboard scanner (v0.96 feature candidate)

For release v0.96, we are overhauling the keyboard scanner to improve typing accuracy. The legacy keyboard scanner relies on reading the CIA lines once per IRQ, which has a familiar vintage feel but is unfriendly to modern-day fast typists. The new keyboard scanner collaborates with a new feature of the MEGA65 core to queue typing events in hardware (FPGA) registers. For the convenience of the ROM and other applications, the hardware also performs ASCII and PETSCII translation, as well as key repeat and debouncing.

As of September 19, 2023, the core and ROM code that implement this feature are here:

The latest core from the development branch, at least 20230924 build #61
ROM beta release 920388

File bugs in the mega65-rom-public repo, and contact @dddaaannn on Discord for more information.

Tip: If you notice buggy behavior that may be attributed to ROM 920388, try reproducing it, then try again in ROM 920386, which uses the original keyboard scanner. You do not need to go back to the previous version of the core for this test. Include reproduction steps and a description of the behavior in both ROMs in your bug report.

ROM 920388 will not work with older cores. Older ROMs work with the latest core, so you may want to install the development core in slot 1, use 920377 or 920386 as your default MEGA65.ROM, then put 920388 in a non-default ROM slot such as MEGA652.ROM (where 2 is the ROM slot number: hold the "2" key during boot to select it). If you use ROM 920388 with an old core, typing will not work.

Known issues

The virtual keyboard registers are glitchy with the new keyboard scanner with regards to modifier keys. The m65 tool’s typing feature has been updated to accommodate this.
MegaAssembler and Coffeebreak Compiler are not working, apparently getting 0 from GETKEY for all typing events. Not all uses of GETKEY are affected. This is still being researched.

Test plan

This change is not expected to break any ROM features or MEGA65 applications. Software that uses the ROM’s getin entry point will benefit from the new typing accuracy automatically. This includes BASIC programs that use the GET / GETKEY commands. The change is expected to be compatible with existing applications that use the documented ASCIIKEY register, or do their own keyboard scanning of the CIA lines. It is not expected to affect GO64 mode or older ROMs running on the latest core.

In theory, this change might violate pre- and post-conditions of keyboard related KERNAL entry points, described later in this document. In the context of the MEGA65, these behaviors are undocumented to an extent that MEGA65 applications should not be using them. For example, C64 coders might be used to manipulating the KERNAL’s keyboard buffer directly, and this no longer works the same way in the MEGA65. The keyboard buffer location and behavior is not documented for direct access by a program in the MEGA65 ROM.

KERNAL features touched by this change that are worth testing:

Typing in the screen editor causes characters to appear, cursor to move. Command and line entry work.
Typing keystrokes with modifier keys works as expected: Left Shift, Right Shift, Mega, Ctrl, Alt (typically no effect), Caps Lock (only letters change case).
Mega + Shift toggles the character set casing. Ctrl-K locks this feature; Ctrl-L unlocks it.
No Scroll toggles scroll pausing. Trying LIST’ing a long program (such as AUTOBOOT.C65 off of MEGA65.D81) then press No Scroll during the listing to pause and unpause it.
Ctrl-S toggles scroll pausing.
1. Note: In previous ROMs, Ctrl-S set pausing, and any other typing event unpaused it. Technically, the MEGA65 documented this behavior as Ctrl-S being equivalent to No Scroll, so the behavior now matches the documentation, even if it doesn’t match the original C65 prototype ROM.
2. Ctrl-S never emits PETSCII $13 to the input stream. (This complies with previous behavior.) PETSCII $13 is equivalent to Home, so if Ctrl-S ever homes the cursor, that’s a bug.
Function key macros work correctly. For example, pressing the Help key types the HELP command followed by a Return. F1 through F14, Help, and Shift+Run/Stop are all considered assignable function keys. KEY OFF disables macros and causes raw function key codes to be emitted to the input stream. KEY ON re-enables macros.
Run/Stop aborts a running BASIC program.
Run/Stop + Restore aborts a BASIC or ML program, and resets the screen editor.
The BASIC AUTO line numbering utility works as expected. (This is relevant because this feature injects keystrokes into a keyboard buffer to type line numbers.)
BASIC GET and GETKEY commands work as expected. Ctrl-@ does not emit a key event recognized by these commands (as expected).
Ctrl-@ in quote mode emits a PETSCII 0 (reverse @ symbol). (Using such a string results in a ?Syntax Error. That’s not part of the test, just interesting. 😊)
See the User’s Guide Appendix B for a list of interesting Ctrl and Esc codes. These all work as expected.
Holding Restore for a second then releasing enters the Freezer. Pressing F3 in the Freezer resumes the computer and does not emit an F3 code afterward.

We are interested to know about any bugs in existing MEGA65 software titles potentially caused by this change.

How the legacy keyboard scanner works

The legacy keyboard scanner (the one being replaced) uses a keyboard buffer managed by the KERNAL, stored in KERNAL variables. The screen editor’s IRQ routine interrupts whatever is happening to check the input of the mouse and keyboard, triggered 50 times per second in PAL mode and 60 times per second in NTSC mode. Part of this routine, the keyboard scanner, tests the CIA chip lines to see if any keys are pressed. If so, it translates the key locations to a PETSCII character and puts the character in the keyboard buffer. Outside of the IRQ routine, when the screen editor is waiting for keyboard input, it tries to get PETSCII characters from several places, one of which is the keyboard buffer. It processes any characters put into the buffer by the IRQ routine, removing them from the buffer as it goes to make room for more key presses.

A program detects a key press by calling KERNAL routines that know how to pull characters from the buffer. The program can consume a character from the buffer with the getin routine at jsr $ffe4. If the program has disabled the KERNAL IRQ, it must also call the KERNAL keyboard scanning routine (in its own IRQ or otherwise), the key routine, at jsr $ffdf. These addresses are guaranteed to be consistent across ROM versions by the KERNAL jump table.

The C65 version of getin actually reads characters from two sources. If a function key macro is active, then getin pulls the next character of the macro definition instead of the keyboard buffer.

Commodore 64 programmers are accustomed to being able to access all of the KERNAL’s internal variables from their program, including the keyboard buffer. This is not recommended with the MEGA65 ROM: we need to keep some details internal in order to evolve the ROM with bug fixes and enhancements. But some C64 programming documentation will suggest that programs access the keyboard buffer directly for some special effects, such as to insert characters into the input stream as if they were typed. The BASIC 65 AUTO feature uses this technique internally to generate line numbers.

The legacy scanner potentially misses keystrokes because of a limitation in how keys are wired to the CIA chip. The CIA can only detect so many keys pressed at one time, constrained to a pattern in the wiring. Fast typists tend to press the next key before releasing the previous key, and will notice that specific keys in specific words tend to get skipped more than others. (Personally, I can’t type the word “MOUNT” at my usual typing speed using the legacy keyboard scanner. It always comes out as “MONT” unless I slow down.) The MEGA65 core fully replicates this behavior of CIA chips.

I’m not sure to what extent the frequency of the keyboard scan is considered an issue for supporting fast typing, but that may also be a notable detail with this approach. It may be possible for a fast typist to press and release a key between scans at a frequency of 60 Hz.

The hardware typing event queue

The new keyboard scanner relies on the core itself to manage the keyboard buffer, or more specifically a typing event queue. Each press of a non-modifier key, with zero or more modifier keys held down at the time, is a typing event. The core scans the keyboard at a much faster rate than the KERNAL, and does not miss keys due to CIA wiring. The top of the queue can be read from hardware registers, and can be dequeued by writing a value (any value) to a register.

The typing event at the top of the queue is presented on three registers with three different interpretations of the event:

ASCIIKEY $d610 represents the event as an ASCII value, or $ff if there is no equivalent ASCII value for the event.
PETSCIIKEY $d619 represents the event as a PETSCII value or $ff.
MODKEY $d60a[0:6] represents the modifier keys that were held down during the typing event. (Note that ASCIIKEY and PETSCIIKEY take modifier keys into account, such as Shift plus a letter resulting in a capital letter.)

If the queue is empty, ASCIIKEY is $00 and PETSCIIKEY is $ff. To dequeue the topmost event, a program (or the ROM) writes any value to ASCIIKEY or PETSCIIKEY.

$d611[7] is the queue status: 1=non-empty. Writing a 0 to this bit flushes the queue in a single instruction.

A few notes about the typing event queue:

The ASCIIKEY feature originated in a previous version of the core. To maintain backwards compatibility, ASCIIKEY=$00 is the nonce that represents the empty queue. All callers, including those preferring the PETSCIIKEY interpretation, must test ASCIIKEY for $00 to know when the queue is empty.
There are possible typing events that have a translation in one of ASCII or PETSCII but not both. If an event has a PETSCII character but not an ASCII character, ASCIIKEY=$ff to distinguish between a non-ASCII event and an empty queue. $ff is a possible value in ASCII (umlaut-Y: Ÿ), so this is ambiguous. Any caller that needs to distinguish this case can test PETSCIIKEY for $59 when ASCIIKEY is $ff. This is not a backwards compatible behavior for previous callers that don’t know to disambiguate. The side effect of spurious Ÿ characters for 13 obscure typing events in previous ASCIIKEY callers was chosen as a tradeoff from designs that were more backwards compatible but would have caused other issues.
The modifier keys are presented separately to disambiguate cases that have the same ASCII or PETSCII code but are caused by different key combinations. In the screen editor, Ctrl-S has the same PETSCII code as Home, but has different behavior. We may eventually implement a raw key number view register that is completely unambiguous. For now, existing needs are well served by either ASCIIKEY or PETSCIIKEY.
There is only one typing event queue. When an event is dequeued, all views are updated simultaneously. Each typing event is only represented once for the entire system.
MODKEY $d60a[0:6] is not to be confused with $d611[0:6], which presents the current (immediate) state of the modifier keys independently of the typing event queue. The bit assignments are the same for both registers.

Example of polling for typing events indefinitely:

loop:
  lda $d610  ; ASCIIKEY
  beq loop
  lda $d619  ; PETSCIIKEY
  cmp #$ff
  beq loop   ; Ignore non-PETSCII typing event

  ; ... Do something with the PETSCII value in A

  sta $d619  ; Store any value to dequeue
  bra loop

How the new keyboard scanner works

The new KERNAL ROM does not populate a keyboard buffer in the IRQ routine. There is still an IRQ routine that handles special keyboard features, such as Mega + Shift, No Scroll, and Ctrl-S. These features do not use the typing event queue, and instead test immediate keyboard state with other registers.

The getin KERNAL routine processes the hardware typing event queue. Each time the KERNAL or a program tries to read a key via this routine, it reads from either 1) the active function key macro, 2) an internal buffer, or 3) the hardware typing event queue, in that order. When it reads from the hardware queue, it dequeues what it reads. The screen editor only calls getin when it is waiting for input, unlike the keyboard IRQ routine which is called 50/60 times per second regardless of what program is running.

This allows programs that use ASCIIKEY as it was previously defined to continue to see keystrokes even with the KERNAL keyboard IRQ active. When the program is running, the screen editor is not actively polling getin, so it leaves typing events on the queue for the program to consume via ASCIIKEY. If the program returns to the screen editor, the editor only sees keystrokes typed since the program’s last dequeue attempt.

Notes about the ROM implementation:

When the user presses Ctrl-S, it appears on the typing event queue. getin always skips Ctrl-S when it finds it in the queue, allowing the KERNAL IRQ routine to handle it via CIA lines as before. (This is functionally identical to previous ROM behavior: getin never returns Ctrl-S.)
Mega + Shift and No Scroll do not appear on the typing event queue when typed alone because they are modifier keys and not considered typing events. The keyboard IRQ can detect these key presses by testing $d611, which presents the immediate state of the modifier keys independently of typing events.
The keyboard IRQ routine is still accessible via the key routine at jsr $ffdf. It is still worth calling from a program if the program replaces the KERNAL IRQ and wants to support Mega + Shift and No Scroll. The routine no longer populates a buffer, because this is handled in the core.
The KERNAL still has an internal key buffer to support BASIC65 AUTO, and possibly future KERNAL features that wish to inject keystrokes into the input stream. (These variables are still undocumented for now.)
The KERNAL routine that scans for Run/Stop at jsr $ffe1 is unaffected by this change. It still reads it from the CIA lines.
The code that initiates a function key macro now lives in getin instead of the keyboard IRQ. This is an implementation detail that won’t be noticed by programs, because only the next call to getin is affected by activating a function key macro.
All other changes to the flow are internal to the KERNAL and screen editor. There is an internal screen editor subroutine jump table not documented for use by programs, and its API has changed.
The C65 ROM has a collection of extension vectors that are intended for use by programs but not yet documented. For example, once documented, a program can replace the vector that points to the KERNAL’s implementation of getin with another routine, typically one that calls the original getin routine along with some new extended behavior. None of these vectors are considered documented yet. The new ROM changes the extension vector APIs that Commodore intended for the keyboard IRQ. The MEGA65 project will document these vectors once we are confident that they work, are well designed, and won’t need to change again.