What does surprisal mean?
Surprisal is a technical noun meaning the degree to which something is unexpected. It is especially important in information theory, where the surprisal of an event is determined from the probability that the event would occur: improbable outcomes have greater surprisal because they convey more information. The concept is also used in statistics, natural-language processing, and computational linguistics to compare how predictable words, sounds, or sequences are. A language model, for example, assigns higher surprisal to an unlikely continuation and lower surprisal to a familiar one. In general discussion, surprisal can mean unexpectedness more broadly, but it is more specialized than surprise, which usually names an event, response, or feeling. The word is useful because it distinguishes the measurable quantity of unexpectedness from the experience of being surprised.
nounThe degree or amount of unexpectedness associated with an event, observation, or piece of information. In information theory, it is the information content of an outcome, commonly expressed as the negative logarithm of its probability.
- The degree or amount of unexpectedness associated with an event, observation, or piece of information.
- In information theory and related fields, the information content of an outcome, usually defined as the negative logarithm of its probability.
"The model calculates surprisal for each possible next word and gives greater weight to less likely alternatives."
"High surprisal indicates that an observation is unlikely given the model."
"The linguist compared the surprisal of common words with that of unusual constructions."
The plural is used when referring to separate surprisal values, outcomes, or measurements; the uncountable form is common when discussing surprisal as a general property.
"The table reports the average surprisals for words in each category."
Surprisal turns the familiar feeling of surprise into a measurable quantity, showing how much information an unexpected result contains.
Reviewed by Deb Chak, Editor. AI-assisted content curated by RJS Tech Solutions LLP.
Where does surprisal come from?
Surprisal was formed in English by adding the suffix -al to surprise. Surprise ultimately comes through Anglo-Norman French from the Latin prehendere, meaning “to seize” or “to hold,” with the prefix sur- expressing concealment or an action from an unexpected direction. Cognates such as comprehend and apprehend share the same Latin root, although surprisal itself became especially associated with the technical language of information theory.
How surprisal is actually used
Surprisal is primarily a technical term in information theory, computational linguistics, and statistics. In ordinary English, surprise is more natural for both an unexpected event and the feeling caused by it. The noun can be countably or uncountably used, although uncountable use is especially common in technical descriptions.
Easily confused with surprisal
A surprise is an unexpected event or feeling, whereas surprisal is the degree or quantity of unexpectedness associated with an event or outcome.