Home > information, intelligence, linguistics > >A definition of information

>A definition of information

>Message theory has been variously defined in terms of how we recognise design, what information means, and how information can be measured.

German engineer Werner Gitt identifies 5 levels of information.

  1. Statistics
  2. Syntax
  3. Semantics
  4. Pragmatics
  5. Apobetics

I understand the choice of the term “statistics” though I think a preferable term would be something like “storage,” or perhaps “transmission quantity” (as storage media may be redundant).

The level of statistics is concerned with the possible options for each symbol and the number of symbols contained in the text block of interest. The number of symbols for binary are 2: 0,1. For English we have 27: letters and word space (punctuation and numbers excluded). For DNA we have 4: A, T, G, C.

Shannon’s theory of information does analysis at this level. This level is completely devoid of any meaning, and Shannon may give higher values to messages with zero actual meaning, such as random numbers, than to meaningful statements.

Syntax is the choice of code. A deliberate, though arbitrary, convention of what groups of symbols mean. In English “cat” has meaning, but “ith” does not. And “come” has meaning in English and Italian, but not the same meaning. DNA at the gene level has the convention of codons; groups of 3 nucleotides such as CAC which codes for valine.

Semantics are at the level of communicating ideas. The code itself does not communicate ideas. We need words but we talk in sentences. It is at this level we have meaning. Further, it is at this level meaning is invariant. We could communicate the same idea in a different language which would use a different code (syntax) and result in different storage requirements (statistics).

It is at this level (at least) that information (meaning) needs to be considered when discussing gain and loss of information.

Gitt gives a useful illustration of semantics.

A: The bird singed the song.
B: The green freedom prosecuted the cerebrating house.

Sentence B is perfectly correct syntactically, but is semantically meaningless. In contrast, the semantics of sentence A is acceptable, but its syntax is erroneous.

Pragmatics is about action based on the ideas. And apobetics is about the purpose for the action.

Is important to note that much work has been done at the level of statistics. This is probably largely a result of the computer revolution; though possibly partly because this is the easiest level to define and analyse. But statistics does not discriminate meaning from non-meaning.

  1. 2009 September 13 at 08:34

    That’s a cool post! :)

  2. 2009 September 13 at 10:59

    Thanks Grant.

  3. Andrew
    2009 September 19 at 02:10

    Hi, I’ve just read Dr John Sanford’s book, “Genetic Entropy & the Mystery of the Genome” available from CMI. Great read. Suggest it should be compulsory reading for all members of TANSA and especially Dr Graeme Finlay. Then get them to try an explain where all their millions of years fit in. At the current rate of mutations, we should be extinct by now if we are as old as they say. Dr Sanford should know what he is talking about: professor at Cornell University for 25 years, received his PhD from the University of Wisconsin in the area of plant breeding and plant genetics, conducted genetic research at the New York State Agricultural Experiment Station in Geneva, NY, and became heavily involved in the field of plant genetic engineering. His most significant scientific contributions involve three inventions, the biolistic (“gene gun”) process, pathogen-derived resistance, and genetic immunization. Regards Andrew Thomson

  4. 2009 September 19 at 10:15

    Hi Andrew, I have got that book. I have only read the beginning so far. It is on my shelf of unread and partially read books waiting for me to finish!
    I agree with his thesis that DNA is accumulating mistakes. Interestingly Blythe described natural selection before Darwin and saw natural selection as a culling exercise; a way of removing the more error laden organisms. He was probably partially correct in this claim. Though DNA seems to have built in variety generating code that natural selection can also work on.

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: