Register, in writing or speaking, is the socially or contextually conscious choice of vocabulary and grammatical structure. Those people who are apt to adopt more than one register most commonly do so in formal and informal registers. In some cultures this is part of formal education — notably France where written language has a distinctly different register from spoken. In other cultures, changes of register are used by people who move between different social groups as part of their daily lives, such as teachers and social-workers. Register differs from diction in that it is socially or culturally aware (even if subconsciously).
Writers love register. It enables them to set the scene, the atmosphere, the social status, the background.
a) "Oh woe! Oh woe!" she cried, tremulous but stentorian, "we are undone!"
Whoever she is, her register is telling us something about her. At the very least, this is set in the past - anywhere from the middle ages backwards. If I put her in a toga, we are in ancient Rome, and she's a soothsayer.
b) "Lawksamercy if he hasn't gone and done it again," she hawked and spat, "and it's these old bones what'll have to clear up the mess if I'm any judge."
Now to demonstrate the extremes of register I'm going to put those two statements into the mouth of my son's favourite character, The English Butler.
a) "I'm sorry," he announced to the assembled guests, "but an incident has occurred."
b) "Ahem," the English Butler cleared his throat to gain the attention of the assembled company; "his Lordship has had one of his turns, and I must deal with it."
The English Butler's register is without emotion - and consciously so.
Writers can use register to great effect, therefore, to distinguish between characters, between time, place and setting. Between formal situations and informal; between characters who know eachother well and ones who are strangers. This has its pitfalls, of course. The two most common are register blindness and register seepage.
Register blindness is when the author has his own strong style, and it dominates to such an extent that all characters start to sound like the author - or whichever of the author's styles he happens to be using. I know that Derek Prior won't mind my mentioning that in an earlier draft of Cadman's Gambit, he had an aboriginal shaman who kept talking like Basil Rathbone, and a group of professional assassins who spoke like a gathering of English country vicars. It wasn't through poor characterisation, but through inattention to register. (I should add that in rewriting these parts Derek really went to town!)
Register seepage occurs when the characters' register creeps into the narration. Sometimes this is not an issue. In Mike Dennis' novels, strong first person point of view means that the register of narration is the register of the main character. In most chicklit and pretty much all romance, the register is informal vernacular with good grammar - so the narrator's register is the same as most of the characters'.
Register seepage is at its worst in historical and fantasy fiction. Heroic fantasy is set in the Forgotton Realms™ European middle ages, where everyone says "unto" instead of "to" and "wherefore" instead of "why" (or instead of "where" in the bad cases). You get an awful lot of Yoda-speak. (Word order change will you, and wise will you sound.)
This is okay as long as only the characters speak this way, but it is all to easy for the author to get swept along by the characters to the point where he uses "unto" in the narration. And "atop".
"Atop" is a bugbear of mine. It isn't a fancy way of saying "on". "Atop" means "on top of the highest point of". So in the missionary position, the man is "on top of" the woman, not "atop" her. But the Star can be atop the Christmas tree, however silly that sounds. YKWYA...
Register can be a powerful tool, and is a key part of the author's armoury, as it enables him to avoid lengthy character establishment and even to minimise nametagging in direct speech. It is something of which authors should be aware, so feel free to prod me for more if my explanation isn't clear or detailed enough.