Story Details

Michael Steil's blog post explores the behavior of illegal or undocumented opcodes on the MOS 6502 processor. Rather than simply halting or throwing an error, these opcodes execute as combinations of shorter, legal instructions. The 6502's instruction decoding mechanism, which combines bits from different parts of the opcode byte, leads to these unintended combinations. Steil demonstrates how these combined instructions can be predicted and even utilized for creative programming tricks, offering a deep dive into the processor's architecture. He provides examples of how these illegal opcodes can manipulate registers and flags in unexpected ways, opening a window into the inner workings of this classic CPU.

Michael Steil's blog post, "How MOS 6502 Illegal Opcodes Work," delves into the fascinating, and often unpredictable, behavior of undocumented instructions within the MOS 6502 microprocessor. The 6502, famed for its role in powering early computing devices like the Apple II and the Commodore 64, possesses a relatively small instruction set. However, the chip's designers didn't explicitly define behavior for all possible combinations of bits that could represent an opcode. These undefined combinations are known as "illegal" or "undocumented" opcodes.

Steil's post meticulously explains that these illegal opcodes don't simply halt the processor or trigger an error. Instead, they often execute a combination of existing, legal instructions, stitched together in unpredictable ways based on how the bit patterns of the illegal opcode align with portions of documented instructions. He illustrates this with detailed examples, dissecting specific illegal opcodes and showing how the processor interprets them as a sequence of shorter, valid operations. This can lead to unexpected side effects, modifying registers or memory locations in ways not readily apparent from the initial illegal instruction.

The post clarifies that the behavior of these illegal opcodes is not random. It arises from the internal logic of the 6502's instruction decoding mechanism. The processor doesn't recognize the illegal opcode as a whole but instead tries to interpret its component bits as parts of legal instructions. This often results in the execution of multiple, shorter instructions sequentially, leading to the observed, often bizarre, behavior. He highlights that this behavior can vary slightly across different 6502 revisions and even between different manufacturers of the chip, adding further complexity to the study of these undocumented instructions.

Steil emphasizes the value of understanding these illegal opcodes for several reasons. First, it provides a deeper understanding of the 6502's internal architecture. Second, it allows programmers to potentially exploit these behaviors for specific tasks, like code optimization or creating unique effects in games and demos. Finally, understanding these quirks can be crucial for debugging and troubleshooting software that inadvertently stumbles upon an illegal opcode. The post concludes by showcasing a practical application: the creation of an exhaustive list of all illegal opcodes and their resulting behaviors on different 6502 variants, a valuable resource for anyone working with this iconic microprocessor.