Archive for the ‘information’ Category

>A definition of information

2009 September 13 4 comments

>Message theory has been variously defined in terms of how we recognise design, what information means, and how information can be measured.

German engineer Werner Gitt identifies 5 levels of information.

  1. Statistics
  2. Syntax
  3. Semantics
  4. Pragmatics
  5. Apobetics

I understand the choice of the term “statistics” though I think a preferable term would be something like “storage,” or perhaps “transmission quantity” (as storage media may be redundant).

The level of statistics is concerned with the possible options for each symbol and the number of symbols contained in the text block of interest. The number of symbols for binary are 2: 0,1. For English we have 27: letters and word space (punctuation and numbers excluded). For DNA we have 4: A, T, G, C.

Shannon’s theory of information does analysis at this level. This level is completely devoid of any meaning, and Shannon may give higher values to messages with zero actual meaning, such as random numbers, than to meaningful statements.

Syntax is the choice of code. A deliberate, though arbitrary, convention of what groups of symbols mean. In English “cat” has meaning, but “ith” does not. And “come” has meaning in English and Italian, but not the same meaning. DNA at the gene level has the convention of codons; groups of 3 nucleotides such as CAC which codes for valine.

Semantics are at the level of communicating ideas. The code itself does not communicate ideas. We need words but we talk in sentences. It is at this level we have meaning. Further, it is at this level meaning is invariant. We could communicate the same idea in a different language which would use a different code (syntax) and result in different storage requirements (statistics).

It is at this level (at least) that information (meaning) needs to be considered when discussing gain and loss of information.

Gitt gives a useful illustration of semantics.

A: The bird singed the song.
B: The green freedom prosecuted the cerebrating house.

Sentence B is perfectly correct syntactically, but is semantically meaningless. In contrast, the semantics of sentence A is acceptable, but its syntax is erroneous.

Pragmatics is about action based on the ideas. And apobetics is about the purpose for the action.

Is important to note that much work has been done at the level of statistics. This is probably largely a result of the computer revolution; though possibly partly because this is the easiest level to define and analyse. But statistics does not discriminate meaning from non-meaning.

>An analogy between information and energy

2009 September 9 Leave a comment

>Information technology is a useful term. We store information and transfer it, all of which is helpful in understanding the information is not matter. However the association with storage capacity means that people may think in terms of size—number of bits—not the nature of what is stored.

This is somewhat analogous to talking about fuel tank capacity. Vehicles need energy to travel and all things being equal a larger fuel tank will supply more energy. But the issue here is energy. If one focuses on fuel tank size he can forget that it matters what is in the fuel tank. Different grades of gasoline contain different amounts of energy, and the same is true for other fuels such as methanol, ethanol, and coal. One can even fill a tank with substances that contain no energy such as sand or water (fusion disregarded). And other forms of energy such as electricity do not even require a petrol tank.

So while fuel tank size can be a proxy for energy, it may be inadequate, or even inappropriate. One must remember that it is energy that is being discussed.

Categories: energy, information

>Establishing the model before doing the maths

2009 September 7 1 comment

>My recent post on information led to a call for a precise definition of what is meant by information. This request is eminently reasonable. I have yet to give an exact mathematical definition, or even an exact philosophical one. The reason is that “information” has a variety of meanings to different people; and the association with computing and information technology colours people’s thinking.

Consider gravity. My understanding is that ideas about gravity around the time of Bacon, Galileo, Kepler, and Newton changed the way people thought about motion. Prior to then it was obvious that objects fell to the ground, but the Aristotelian rationale for such motion was that (some) objects tend toward the centre of the earth. And observations would support this claim. Post-Aristotelian physics described the same phenomena, but for a different reason: massive objects attract each other. The stone moves toward the earth, and the earth moves toward the stone; but the earth imperceptibly because of its immense size. Same observation (as much as perceptible) but different explanation.

The old theorem explains pendulums better. Pendulums tend to the centre of the earth so they stop after a time, not doing so immediately because the weight is moving when it gets to the lowest point. Newton’s theory needs friction to explain the same observation; one could argue Newton is less parsimonious. However Newton was shown correct over time. And his theory was well defined, gave predictions, and was generalisable to the heavens.

My diversion was to highlight that the differences between the two approaches did not require them to be rigorously defined. The mass-attraction concepts antedated the maths by a significant amount of time; Newton had to invent maths for his theory! But a philosophical discussion on the merits of objects tending to the centre of the earth versus objects attracting is still very possible.

Similarly, it is possible to discuss other topics in general before rigour is attained, including information.

Information is a non-material concept that contains instructions thru language.

I use the term instruction somewhat broadly including being informed, not just commanded. That is, meaning of some form exists.

>The importance of information in the evolution debate

2009 August 30 30 comments

> Creationists often mention the concept of information as a challenge to the grand theory of evolution.

Information as a concept has long been recognised. It is something all people agree exists even if there is debate about how it is categorised. Archaeologists discovering inscriptions know there is a meaning even if they do not know what the meaning is, and most people would not dispute this. People recognise several things that are designed. Though one could say they do so because they already know these things are designed, such as a car or computer; there are examples of things we are not previously aware of, but we would still recognise intention.

As I have mentioned previously, information is not composed of, nor derived from matter. Of course it can be stored in matter.

There are several concepts of what information is, at least in terms of how we should represent information theory mathematically.

The application to evolution centres on the connection to DNA. DNA is recognised as carrying information. It has meaning. It resembles a blueprint, and metaphorically is one.

We can study how information originates. If the source of all information can be shown to be greater information (that is intelligence), then this conclusion also applies to DNA.

There are 2 potential ways that one could show information cannot be produced by itself. It may be possible to show this mathematically, in which case we can be absolutely certain (or essentially certain if the proof is statistical).

If not mathematically, it may be possible to show this empirically: that is, in investigating all the billions of examples of information that have been directly observed; if all are shown to have come from higher information sources and zero are self producing, then we can be extremely confident of our thesis.

Therefore the impossibility of information coming from non-information, mathematically or empirically, disproves Darwinism. Rather than being a red-herring as is sometimes claimed, information theory is absolutely central.

Categories: evolution, information, logic

>Message and matter

2008 November 2 7 comments

>It is important to understand the fundamental difference between these 2 concepts. Matter is all around us. Everything physical in the universe is matter or energy and Einstein showed us that they are essentially interchangeable, at least in essence if not always in practice. The stars, the earth, the moon. All the objects on the earth, both animate and inanimate. All are material. Composed of atoms and/ or photons.

As such they obey the laws of physics. Mass attracts, objects fall, momentum and energy are conserved, and entropy increases. They all obey the laws of chemistry which at a foundational level are laws of physics for elements. Why the chemistry laws should be as they are, ie. could the elements theoretically be different, is a different question.

None of this is too complex to understand, neither is it modern. While the ancients may not have understood the scientific laws in such detail, the concept of the material was well understood. And the material was often distinguished from the spiritual.

What I think it very important to comprehend is that information is utterly distinct from matter and not reliant on it. It exists independent of matter and there is no reason to think it could not exist even if matter itself did not. Though the existence of matter without information is unlikely to be possible.

Information or intelligence is difficult to quantify, though it can be done. Information is frequently stored in matter but it is in no way dependant on the matter in which it is stored. This post as you read it is stored magnetically on your hard-drive, having been copied from a server elsewhere. However it could be printed and stored in toner on paper. Or you could memorise it and it would be stored in your neurons. But the message is not derived nor is dependant on magnetism, paper, ink or anything else composed of matter.

This concept is fundamental. And it has significant implications.

  1. It means that the 2 (message and matter) are to be distinguished from each other, something that may not be done in defending various theories.
  2. They are not derived from each other. Information cannot make material and material cannot make information.
  3. We need a source for both matter and information.
  4. The laws that govern information are not those that govern matter. Information does not obey the law of gravity, it does not contain momentum, it cannot be transformed to energy.

Expanding on item 2: One might argue that a powerful source of information can create matter, but this is a being, not just an idea.

Information that appears to arise from matter is merely the level of information that already exists in the matter, it is not created by the matter. The limited information to describe a crystal structure is intrinsic to the information already contained within the physics and chemistry of the molecules.

Note that the crystal structure of salt is low information content and no more than can be known from our (potential) understanding of sodium, chloride, solutions and temperature. However deoxyribonucleic acid (DNA) has information that is imposed on it which is not intrinsic to nucleic acids, sugar bases, or phosphate. One can transcribe the code onto a computer or paper and the code remains intact.

Categories: information, philosophy

>The design argument

2007 June 11 Leave a comment


The heavens declare the glory of God,/
and the sky above proclaims his handiwork. (Psalm 19:1)

In essence this is straightforward. In everyday life it is apparent that many things are designed by men. We recognise design. And we know that designed objects have a designer. The parallel is that we see many other things that are not designed by men but are clearly designed.

We recognise design by purpose or intent. While a splatter painting may have been drawn by an artist there is no clear intent so whether or no it was designed is not clearly apparent. However the more intent something has the more clearly it was designed.

Information content is a helpful way of assessing design. The higher the information content the more obvious it is designed. Information is based on specification and complexity. So if something is ordered it is not particularly complex. If something is complex based on the number of bits to store it, it is may or may not be specified. So storing 100 addresses is specific, 100 random letters is not. A measure of information can be made by the the storage capacity required to hold the generator of the data. For a story it is the data (ie. every single letter, though thru lossless compression it may be possible to minimise information content) as a story is specified and not predictable. However large quantity data that is predictable (⅓ in decimal), calculable (π, e) or random has a low information content.

The issue with random numbers or noise needs more explanation. It is low information because describing a random number generator requires little information. To describe any single random number will require as much data as an equivalent length story. The reason this is not high information is that it lacks any specificity. The binary data that describes a jpeg picture may appear random but it is specific, it stores data about a specific object.

People don’t perform these calculations rigorously but as information content ranges over multiple orders of magnitude design is easily recognised. A smoothed rock is easily distinguishable from a carving. We can tell a painting from spilled paint (usually). We know that a book has an author.

So it is clear that design can be seen even if the designer is not known. The existence of a designer can be inferred from design. So when one sees all the design in this world that is not from the hands of man the logical conclusion is that a designer exists and he is other than man. One may claim this designer is other than the Grand Designer; whether that be an angel, a lesser god of a pantheon, an extraterrestrial being. But whatever the immediate source, these beings are still creatures and the ultimate source must be the Creator.