IT Math Case Study Application Paper

User Generated

xvatznaqbehvm10

Mathematics

Description

mth221_r2_coding_theory_case_study.pdf

mth221_r2_food_webs_case_study.pdf

Choose one of the following Case Studies:

  • Food Webs
  • Coding Theory

Write a 750- to 1,250-word paper in which you complete one of the following options:


Option 1: Food Webs Case Study

Explain the theory in your own words based on the case study and suggested readings.

Include the following in your explanation:

  • Competition
  • Food Webs
  • Boxicity
  • Trophic Status

Give an example of how this could be applied in other real-world applications.

Format your paper according to APA guidelines. All work must be properly cited and referenced.

Include at least 1 properly formatted citation from the textbook.

Option 2: Coding Theory Case Study

Explain the theory in your own words based on the case study and suggested readings.

Include the following in your explanation:

  • Error Detecting Codes
  • Error Correcting Codes
  • Hamming Distance
  • Perfect Codes
  • Generator Matrices
  • Parity Check Matrices
  • Hamming Codes

Give an example of how this could be applied in other real-world applications.

Format your paper according to APA guidelines. All work must be properly cited and referenced.

Include at least 1 properly formatted citation from the textbook.


Unformatted Attachment Preview

PART III GRAPH THEORY 224 13 Food Webs Author: College. Robert A. McGuigan, Department of Mathematics, Westfield State Prerequisites: The prerequisites for this chapter are basic concepts of graph theory. See Sections 9.1 and 9.2 of Discrete Mathematics and Its Applications. Introduction A food web is a directed graph modeling the predator-prey relationship in an ecological community. We will use this directed graph to study the question of the minimum number of parameters needed to describe ecological competition. For this purpose we will consider how graphs can be represented as intersection graphs of families of sets. We will also investigate the axiomatic description of measures of status in food webs. Competition In an ecological system, the various species of plants and animals occupy niches defined by the availability of resources. The resources might be defined in terms of factors such as temperature, moisture, degree of acidity, amounts of nutrients, 225 226 Applications of Discrete Mathematics and so on. These factors are subject to constraints such as temperature lying in a certain range, pH lying within certain limits, etc. The combination of all these constraints for a species then defines a region in n-dimensional Euclidean space, where n is the number of factors. We can call this region the ecological niche of the species in question. For example, suppose we restrict ourselves to three factors, such as temperature, nutrients, and pH. Assume that the temperature must be between t1 and t2 degrees, the amount of nutrients between n1 and n2 and the pH between a1 and a2 . Then the ecological niche these define occupies the region of 3-dimensional Euclidean space shown in Figure 1. Figure 1. An ecological niche. Euclidean space which has as dimensions the various factors of temperature, pH, etc., is called an ecological phase space. Generally, no two distinct species will have the same ecological niche in phase space; however, two species compete if their ecological niches have non-empty intersection. A basic principle of ecology, known as the principle of competitive exclusion, dictates that species whose niches are too similar, or overlap too much, cannot coexist. If the factors defining the niche are independent, then the niche in phase space would be a box such as that in Figure 1. If the factors are not independent, i.e. the level of one depends on levels of others, then the niche would be some other type of set, e.g. convex, but not a box. For example, consider the two factors temperature (t) and per cent humidity (h). We might have constraints such as: t must be between 0 and 100, and h must be between 0 and 100t − t2 . In this case temperature and humidity are not independent; the possible values of h depend on the values of t. The region in two-dimensional space defined by these constraints is not a rectangle. Our discussion of ecological communities and related concepts such as Chapter 13 Food Webs 227 species, food webs, and competition will be somewhat oversimplified in order to make a brief presentation possible. Interested readers should consult reference [1] for an in-depth treatment of these topics. Our mathematical treatment follows that of reference [6]. Food Webs It may be difficult to know all the factors which determine an ecological niche, and some factors may be relatively unimportant. Hence it is useful to start with the concept of competition and try to find the minimum number of dimensions necessary for a phase space in which competition can be represented by niche overlap. One approach to this question is to consider the notion of the food web of an ecological community. Definition 1 A food web of an ecological community is a directed graph with a vertex for each species in the community and a directed edge from the vertex representing species A to the vertex representing species B if and only if A preys on B. Figure 2 shows a simple food web for a community of seven species: robin, fox, grasshopper, raccoon, salamander, milksnake, and toad. Figure 2. A simple food web. We can define competition using the food web. Two species compete if and only if they have a common prey. Thus, in the example of Figure 2, raccoon and fox compete (since robin is a common prey), milksnake and raccoon compete, 228 Applications of Discrete Mathematics while salamander and robin do not compete. We use this competition relation to define a graph called the competition graph. Definition 2 The competition graph of a food web is a simple graph with a vertex for each species. Two vertices are joined by an (undirected) edge if and only if the species they represent have a common prey. Example 1 Solution: Find the competition graph for the food web of Figure 2. The competition graph for this food web is shown in Figure 3. Figure 3. A competition graph. To represent the competition relation in phase space we want to assign to each vertex of the competition graph a subset of Euclidean space of some dimension in such a way that two vertices are joined by an edge in the competition graph if and only if the sets assigned to these vertices have non-empty intersection. Figure 4 shows a representation of the competition graph of Figure 3, using an interval for each vertex. We have thus represented the competition graph using only one dimension. Figure 4. Interval representation of a competition graph. We can now state a general mathematical problem, but first we need to develop some terminology. Chapter 13 Food Webs 229 Definition 3 A graph is an intersection graph for a family of sets if each vertex is assigned a set in such a way that two vertices are joined by an edge if and only if the corresponding sets have non-empty intersection. Definition 4 A graph is called an interval graph if it is the intersection graph for a family of closed intervals. Our goal is the representation of competition graphs of families of sets in Euclidean n-space. Clearly the simplest case would be that of competition graphs that are interval graphs. This would mean that only one ecological factor is necessary to describe niche overlap. Example 2 Find the interval graph for the family of closed intervals A = [1, 3], B = [2, 6], C = [5, 8], D = [4, 5]. Solution: Figure 5. We use the definition of intersection graph to obtain the graph of Figure 5. Example 3 graph. An intersection graph. Prove that the 4-cycle graph C4 of Figure 6 is not an interval Solution: The proof depends on the order properties of the real numbers. Let the interval corresponding to vertex n be [nl , nr ]. Since the intervals for vertices 1 and 2 overlap, we must have either 1l ≤ 2l ≤ 1r ≤ 2r or 2l ≤ 1l ≤ 2r ≤ 1r , Assume for specificity that 1l ≤ 2l ≤ 1r ≤ 2r . The argument for the other case is analogous. Since the interval for vertex 3 must meet that for vertex 2 and must not meet that for vertex 1, we must have 1l ≤ 2l ≤ 1r < 3l ≤ 2r . Now the interval for vertex 4 must meet those for both vertices 1 and 3, so we have to have 1l ≤ 4l ≤ 1r and 3l ≤ 4r ≤ 3r since interval 1 lies entirely to the left of interval 3. However, since 2l ≤ 1r < 3l ≤ 2r , the intervals for vertices 2 and 4 overlap, which is forbidden. 230 Applications of Discrete Mathematics Figure 6. A graph that is not an interval graph. The 4-cycle can, however, be represented as the intersection graph of a family of boxes in Euclidean 2-space, as shown in Figure 7. There are several methods known for determining whether a simple graph is an interval graph. A detailed discussion of this topic may be found in Roberts’ book [6]. We simply state the characterization due to Gilmore and Hoffman [3] without proof. Before the characterization can be stated, we need some definitions. Figure 7. A box representation. Definition 5 A graph H is a generated subgraph of a graph G if the vertices of H are a subset of the vertices of G and vertices in H are adjacent in H if and only if they are adjacent in G. Definition 6 The complement of a graph G is the graph G where the vertices of G are the vertices of G, and two vertices in G are adjacent if and only if they are not adjacent in G. Definition 7 An orientation of a graph G is an assignment of a direction to each edge in G (which makes G into a directed graph). An orientation is transitive if whenever (u, v) and (v, w) are directed edges, then (u, w) is a directed edge. Chapter 13 Food Webs 231 The characterization due to Gilmore and Hoffman is given by the following theorem. Theorem 1 A graph G is an interval graph if and only if it satisfies the following two conditions: (i) The four-cycle C4 is not a generated subgraph of G, (ii) The complement of G is transitively orientable. Our goal in our study of ecological competition is the representation of niches in Euclidean space and competition by niche overlap. It seems desirable in an ideal representation that the factors determining the dimension of the ecological phase space would be independent and the niches would be represented as “boxes”, or Cartesian products of intervals. This leads us to the next part of this discussion, namely, when can we represent a graph as the intersection graph of a family of boxes in n-space. Boxicity Definition 8 The boxicity of a graph G is the smallest n such that G is the intersection graph of a family of boxes in Euclidean n-space. Note that an interval graph is simply a graph with boxicity equal to 1. It is not entirely clear that every simple graph has a boxicity. The following theorem resolves this difficulty. Theorem 2 Every graph G with n vertices is the intersection graph of a family of boxes in Euclidean n-space. Proof: Let v1 , v2 , . . . , vn be the vertices of G. A box in Euclidean n-dimensional space is the set of all n-tuples of real numbers (x1 , x2 , . . . , xn ) such that each xi is in some closed interval Ii . Now, for each k = 1 . . . , n and each vertex vi , define closed intervals Ik (vi ) as follows. ⎧ ⎪ ⎨ [0, 1] if i = k Ik (vi ) = [1, 2] if i = k and {vi , vk } is an edge in G ⎪ ⎩ [2, 3] if i = k and {vi , vk } is not an edge in G. For each vertex vi define a box B(vi ) in Euclidean n-space by B(vi ) = {(x1 , x2 , . . . , xn ) | xj ∈ Ij (vi ) for j = 1, . . . , n}. 232 Applications of Discrete Mathematics Thus, the box B(vi ) corresponding to vi is the Cartesian product of the intervals Ij (vi ) for j = 1, . . . , n. Now we show that vi and vj are adjacent in G if and only if B(vi )∩B(vj ) = ∅. Thus the graph G is the intersection graph of the family of boxes B(vi ). First, suppose that there is an edge joining vl and vm . If k is different from both l and m, then according to the definition, Ik (vl ) ∩ Ik (vm ) is [1, 2] ∩ [1, 2], [1, 2] ∩ [2, 3], or [2, 3] ∩ [2, 3]. In any case we have Ik (vl ) ∩ Ik (vm ) = ∅. If k=l or k=m then Ik (vl ) ∩ Ik (vm ) = [1, 2] ∩ [0, 1] = ∅. So, if there is an edge joining ve and vm , then for all k, Ik (vl ) ∩ Ik (vm ) = ∅. Hence B(vl ) ∩ B(vm ) = ∅. Now suppose that B(vl ) ∩ B(vm ) = ∅. Then for each k from l to n, Ik (vl ) ∩ Ik (vm ) = ∅. Set k = l then Il (vl ) = [0, 1] and Il (vm ) must be [1, 2] for the intersection to be nonempty. By definition of Il (vm ), vl and vm are adjacent. Thus G is the intersection graph of the family of boxes B(vi ). This theorem shows that boxicity is well-defined. Unfortunately, there is no efficient algorithm known for determining the boxicity of a general graph. There is no characterization known for graphs of any specific boxicity other than 1. In fact, there are not many general classes of graphs for which the boxicity is known. It is not hard to see that the boxicity of the n-cycle Cn is 2 for n = 4 or larger, and this is left as Exercise 6. Another general class of graphs for which the boxicity is known is the complete p-partite graphs. These are the graphs Kn1 ,n2 ,...,np defined as follows: there are n1 + · · · + np vertices partitioned into p classes, where the ith class has ni vertices. Within a class no vertices are adjacent, and every vertex in any class is adjacent to all vertices in the other classes. Roberts [6] showed that the boxicity of Kn1 ,...,np is equal to the number of ni that are larger than 1. One result which helps somewhat in calculating the boxicity of a graph is due to Gabai [2]. This theorem depends on the concept of independence of a set of edges. Definition 9 in common. A set of edges in a graph is independent if they have no vertices Gabai’s theorem [2] is the following, stated without proof. Theorem 3 Let G be a simple graph. If the maximum size of an independent set of edges of G is k, then G has boxicity less than or equal to k. Also, if G has a generated subgraph consisting of k independent edges then the boxicity of G is greater than or equal to k. Chapter 13 Food Webs 233 Gabai’s theorem is useful in determining the boxicity of relatively small graphs and for certain families. In any case it limits the amount of trial and error needed. In our study of competition we search for the representation of the competition graph of a food web as the intersection graph of a family of sets in Euclidean n-space for some n. As a consequence of the theorem proved above, this representation is always possible. Furthermore, we can use the boxicity of the competition graph as an indicator of the minimum number of factors essential for describing competition in the community. Cohen [1] has studied more than 30 single-habitat food webs published in the ecological literature and has found that the competition graphs of all of them are interval graphs. That is, in all cases one dimension suffices to represent competition by niche overlap. It is not known whether this is a general law of ecology, but it does raise many interesting questions. In some single-habitat communities a single dimension for the niche space can be identified. It may be some obviously linear factor such as temperature, body length or depth in water. However, it may well be that more than one single dimension will work. And, of course, we can’t expect the single-niche dimension to be the same from community to community. Hypothetical food webs have been constructed such that their competition graphs are not interval graphs, but these combinations of species have never been observed in nature at the same time and place. The representation of graphs as intersection graphs of boxes has important applications in ecology, as we have seen. Applications to such diverse fields as archaeology and automobile traffic control have also been investigated (see reference [6]). We conclude with an additional application of food webs. Trophic Status In the study of social systems it is often useful to measure the status of an individual in an organization. Harary [4] first introduced the idea of measuring the status of a species in a food web. In ecology this status is usually called the trophic level and is helpful in assessing the complexity and diversity of a web. The idea is that a web with many species at each trophic level has a high degree of complexity. In ecology it is generally thought that more complex ecosystems are more stable. In this section we study the question of how trophic status can be defined in a food web. If the food web is simply a directed path (a food chain) then it is easy to define trophic status; just follow the order of the species in the chain. Some other structures also allow for an easy definition of trophic status. For example, we might think of species with no outgoing edges as being at the bottom of the web. Suppose that for every vertex, all directed paths to vertices at the bottom have the same length. Examples of such webs are given in Figure 8. 234 Applications of Discrete Mathematics Figure 8. Graphs of two food webs. In this kind of web, the trophic status of a vertex can be defined as the length of a directed path from the vertex to the bottom. In general it is difficult to define trophic status in complicated food webs. Because more than one approach may be possible, we will use the term trophic status in this context rather than the term trophic level which is well-known in the context of food chains. Our goal is to investigate how trophic status could be measured rather than to develop a unique possibility. To start, we need some basic assumptions about food webs. In particular, we assume that our food web is acyclic, i.e. that the directed graph has no cycles. Thus, there are no species s1 , . . . , sn such that for i = 1, . . . , n − 1, si preys on si+1 and sn preys on s1 . In particular there are no two species such that each preys on the other. Thus, the prey relationship is asymmetric. We will take an axiomatic approach to defining measures of trophic status. That is, we will state conditions which any reasonable measure should satisfy in the form of axioms. A measure will then be acceptable if and only if it satisfies the axioms. The axioms will define an ideal model for the concept of measure of trophic status. Our approach will follow that of Harary [4] and Kemeny and Snell [5], who work with status in an organization, and the treatment in Roberts [6], which is more detailed. Definition 10 In a food web a species v is a direct prey of a species u if there is a directed edge from u to v. A species v is an indirect prey of u if there is a directed path from u to v. It could well happen that there are two species u and v neither of which is an indirect prey of the other. Definition 11 If v is a direct or indirect prey of u, then the level of v relative to u is the length of the shortest directed path from u to v. Chapter 13 Food Webs 235 We can now state some reasonable axioms for measures of trophic status. Let tW (u) be the measure of status in the food web W . The axioms are: Axiom 1: If a species u has no prey then tW (u) = 0. Axiom 2: If, without otherwise changing the food web, we add a new vertex which is a direct prey of u to get a new web W  , then tW  (u) > tW (u). Axiom 3: Suppose the web W is changed by adding edges and/or vertices in such a way that the level of some direct or indirect prey of u is increased, and no direct or indirect prey of u has its level relative to u decreased. If W  is the new web, then tW  (u) > tW (u). These axioms make sense intuitively when we consider that we are saying that a species with no prey is at the bottom level (Axiom 1), that if the number of prey of a species increases its status increases (Axiom 2), and that the status of a species increases if its level relative to some indirect prey is increased (Axiom 3). There is a measure of status which satisfies the axioms. Harary [4] suggested the following definition. Definition 12 k, then If a species u has nk species at level k relative to u for each hW (u) = knk . k Theorem 4 The measure hW (u) satisfies Axioms 1-3. Proof: If u has no prey, then hW (u) = 0 because all the nk = 0. If we add a direct prey for u, then n1 increases by 1, so the sum defining hW (u) also increases. Likewise, if some direct or indirect prey of u at level k relative to u is moved to level k + n below u and no other direct or indirect prey of u has its level decreased, the sum for h increases by at least kn, verifying Axiom 3. Kemeny and Snell [5] also show that if tW is any other measure of trophic status satisfying Axioms 1–3 and having all its values nonnegative, then for all species u, tW (u) ≥ hW (u). Thus, h is in a sense a minimal measure of trophic status. While h satisfies our axioms it fails to have other desirable properties. For example, it seems reasonable that if tW is a measure of trophic status and v is a direct or indirect prey of u, then tW (u) ≥ tW (v). 236 Applications of Discrete Mathematics The measure h does not have this property. Figure 9 shows an example of an acyclic food web W with two vertices u and v for which v is a direct prey of u but hW (v) > hW (u). Figure 9. An acyclic food web. In this example, hW (u) = 6 and hW (v) = 8. The problem we have found can be avoided if we modify our definition of level of one species relative to another: If v is a direct or indirect prey of u, then the level of v relative to u is the length of the longest directed path from u to v. It is not hard to show that if h is defined by the same formula as before, but using the new definition of level, then h satisfies Axioms 1–3 as well as having the property that any species has higher status than any of its direct or indirect prey (see reference [5]). The problem we encountered here demonstrates one of the difficulties with the axiomatic approach. Our problem lay in the definition of level and this would not show up in any consideration of the reasonableness of the axioms. Ideally, all of the terms used in specifying the axioms should either be left undefined or else be checked for “reasonableness”, just as the axioms themselves are. In this light we would also have to examine the new definition of level. Without referring to the notion of relative level in a food web, perhaps the only requirement we can state for a measure of trophic status is that if there is a directed path from u to v, then tW (u) ≥ tW (v). There are other ways to investigate complexity of food webs and relative importance of species in food webs. General methods of measuring complexity in graphs can be applied to competition graphs and food webs. For example, such ideas as the number of edges divided by the number of vertices, and the average out-degree and average in-degree might be useful. The importance, or criticality, of a species in a food web could be studied by investigating what happens to the web when the species is deleted from the web. For example, if the web is disconnected when a species is removed that would indicated a high level of importance. More information on these questions can be found in [6]. Chapter 13 Food Webs 237 Suggested Readings 1. J. Cohen, Food Webs and Niche Space, Princeton University Press, Princeton, N.J., 1978. 2. H. Gabai, “N -dimensional Interval Graphs”, mimeographed, York College, C.U.N.Y., New York, 1974. 3. P. Gilmore and A. Hoffman, “A Characterization of Comparability Graphs and Interval Graphs”, Canadian J. Math., Vol. 16, 1964, pp. 539–548. 4. F. Harary, “Status and Contrastatus”, Sociometry, Vol. 22, 1959, pp. 23– 43. 5. J. Kemeny and J. Snell, Mathematical Models in the Social Sciences, MIT Press, Cambridge, MA, 1972. 6. F. Roberts, Discrete Mathematical Models with Applications to Social, Biological, and Environmental Problems, Prentice Hall, Upper Saddle River, N.J., 1976. Exercises 1. Find the ecological niche in Euclidean space of the appropriate dimensions in each case. a) Temperature between 10o F and 90o F ; nitrate concentration in soil between 1% and 5%. b) Carbon monoxide in atmosphere between 0% and 1%; relative humidity between 20% and 100%; nitrogen gas content in atmosphere between 15% and 20%. 2. Find the competition graph for the given food webs in each case: a) b) 238 Applications of Discrete Mathematics 3. Find a representation for each graph as the intersection graph of a family of rectangles in the plane. a) b) 4. Find a representation for each graph as the intersection graph of a family of intervals on the line. a) b) 5. Show that if a graph G is an interval graph then it satisfies the conditions of the theorem of Gilmore and Hoffman characterizing interval graphs. Hint: For an interval representation let I(v) be the interval assigned to the vertex v. If u and v are adjacent in G, make the orientation (u, v) if and only if I(u) lies entirely to the left of I(v). 6. Show that if Cn is the cycle of length n, then the boxicity of Cn is 1 for n = 3 and 2 for n ≥ 4. 7. According to Roberts’ result quoted in the text, the boxicity of the complete bipartite graph K(3, 3) is 2. Find a representation of K(3, 3) as the intersection graph of a family of boxes in the plane. 8. Let Q3 be the graph formed by the edges and corners of a cube in Euclidean three space. Is Q3 an interval graph? Why? Determine the boxicity of Q3 . 9. A food web for some species in the Strait of Georgia, B.C. ([1], page 165) is given by the following table. The numbers atop columns indicate predator (consuming) species and those at the left of rows indicate prey (consumed) species. An entry 1 indicates that the predator in that column consumes the prey for that row, an entry 0 that it does not. The key identifies the various species. Chapter 13 2 3 4 5 6 7 1 2 3 4 5 1 1 0 1 0 0 0 0 0 0 1 0 0 0 1 0 1 1 0 0 0 1 0 0 Food Webs 239 Key 0 0 0 0 1 1 1. 2. 3. 4. 5. 6. 7. Juvenile pink salmon P. minutus Calanus and Euphausiid furcilia Euphausiid eggs Euphausiids Chaetoceros socialis and debilis mu-flagellates a) Construct a directed graph for this food web. b) Construct the competition graph for this food web. c) Find a set of intervals on the real line such that the graph of part b) is the intersection graph of this family of intervals. 10. Repeat Exercise 9 for the following food web for a community of pine feeders [1], p.148. 1 2 3 4 5 8 9 10 2 3 4 5 6 7 8 9 10 1 0 0 0 0 0 0 0 0 1 1 0 0 0 0 1 0 1 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 1 1 1 1 1 0 Key 1. 2. 3. 4. 5. 6. 7. 8. 9. 10. Pine Caterpillars, moths Aphids, secretion Digger wasps Ichneumons Bugs Ants Syrphids Ladybugs Spiders 11. Give an example of a food web which has two species, neither of which is a direct or indirect prey of the other. 12. In the section on trophic status two different definitions of relative level were given and two corresponding versions of the measure of trophic status hW were also given. Calculate the trophic status of each vertex in each of the following food webs using both versions of h. 240 Applications of Discrete Mathematics a) b) 13. If the only requirement we make for a measure of trophic status tW is that if there is a directed path from u to v then tW (u) > tW (v), show that every acyclic food web has such a measure of trophic status. 14. (Roberts [6]) If relative level is measured using the length of the shortest directed path (our first definition), a plausible measure of trophic status is tW (u) = hW (v), v where the sum is taken over all vertices v for which there is a directed path from u to v. Show that this possible measure has the property that if there is a directed path from u to v, then tW (u) ≥ tW (v). Which of the Axioms 1–3 does this measure satisfy? 15. In our discussion of trophic status we assumed that the food web was acyclic. How restrictive is this assumption? Can you think of two species each of which could have the other as prey? Computer Projects 1. Write a program to calculate trophic status in acyclic food webs. 2. Write a program to calculate the adjacency matrix for the intersection graph of a family of intervals given as pairs (a, b) of their endpoints. 5 Coding Theory Author: Kenneth H. Rosen, AT&T Laboratories. Prerequisites: The prerequisites for this chapter are the basics of logic, set theory, number theory, matrices, and probability. See Sections 1.1, 2.1, 2.2, 3.4–3.7, and 6.1 of Discrete Mathematics and Its Applications. Introduction The usual way to represent, manipulate, and transmit information is to use bit strings, that is, sequences of zeros and ones. It is extremely difficult, and often impossible, to prevent errors when data are stored, retrieved, operated on, or transmitted. Errors may occur from noisy communication channels, electrical interference, human error, or equipment error. Similarly, errors are introduced into data stored over a long period of time on magnetic tape as the tape deteriorates. It is particularly important to ensure reliable transmission when large computer files are rapidly transmitted or when data are sent over long distances, such as data transmitted from space probes billions of miles away. Similarly, it is often important to recover data that have degraded while stored on a tape. To guarantee reliable transmission or to recover degraded data, techniques from coding theory are used. Messages, in the form of bit strings, are encoded by translating them into longer bit strings, called codewords. A set of codewords 73 74 Applications of Discrete Mathematics is called a code. As we will see, we can detect errors when we use certain codes. That is, as long as not too many errors have been made, we can determine whether one or more errors have been introduced when a bit string was transmitted. Furthermore, when codes with more redundancy are used, we can correct errors. That is, as long as not too many errors were introduced in transmission, we can recover the codeword from the bit string received when this codeword was sent. Coding theory, the study of codes, including error detecting and error correcting codes, has been studied extensively for the past forty years. It has become increasingly important with the development of new technologies for data communications and data storage. In this chapter we will study both error detecting and error correcting codes. We will introduce an important family of error correcting codes. To go beyond the coverage in this chapter and to learn about the latest applications of coding theory and the latest technical developments, the reader is referred to the references listed at the end of the chapter. Error Detecting Codes A simple way to detect errors when a bit string is transmitted is to add a parity check bit at the end of the string. If the bit string contains an even number of 1s we put a 0 at the end of the string. If the bit string contains an odd number of 1s we put a 1 at the end of the string. In other words, we encode the message x1 x2 . . . xn as x1 x2 . . . xn xn+1 , where the parity check bit xn+1 is given by xn+1 = (x1 + x2 + . . . + xn ) mod 2. Adding the parity check bit guarantees that the number of 1s in the extended string must be even. It is easy to see that the codewords in this code are bit strings with an even number of 1s. Note that when a parity check bit is added to a bit string, if a single error is made in the transmission of a codeword, the total number of 1s will be odd. Consequently, this error can be detected. However, if two errors are made, these errors cannot be detected, since the total number of 1s in the extended string with two errors will still be even. In general, any odd number of errors can be detected, while an even number of errors cannot be detected. Example 1 Suppose that a parity check bit is added to a bit string before it is transmitted. What can you conclude if you receive the bit strings 1110011 and 10111101 as messages? Solution: Since the string 1110011 contains an odd number of 1s, it cannot be a valid codeword (and must, therefore, contain an odd number of errors). Chapter 5 Coding Theory 75 On the other hand, the string 10111101 contains an even number of 1s. Hence it is either a valid codeword or contains an even number of errors. Another simple way to detect errors is to repeat each bit in a message twice, as is done in the following example. Example 2 Solution: Encode the bit string 011001 by repeating each bit twice. Repeating each bit twice produces the codeword 001111000011. What errors can be detected when we repeat each bit of a codeword twice? Since the codewords are those bit strings that contain pairs of matching bits, that is, where the first two bits agree, the third and fourth bits agree, and so on, we can detect errors that change no more than one bit of each pair of these matching bits. For example, we can detect errors in the second bit, the third bit, and the eighth bit of when codewords have eight bits (such as detecting that 01101110, received when the codeword 00001111 was sent, has errors). On the other hand, we cannot detect an error when the third and fourth bit are both changed (such as detecting that 00111111, received when the codeword 00001111 was sent, has errors). So far we have discussed codes that can be used to detect errors. When errors are detected, all we can do to obtain the correct codeword is to ask for retransmission and hope that no errors will occur when this is done. However, there are more powerful codes that can not only detect but can also correct errors. We now turn our attention to these codes, called error correcting codes. Error Correcting Codes We have seen that when redundancy is included in codewords, such as when a parity check bit is added to a bit string, we can detect transmission errors. We can do even better if we include more redundancy. We will not only be able to detect errors, but we will also be able to correct errors. More precisely, if sufficiently few errors have been made in the transmission of a codeword, we can determine which codeword was sent. This is illustrated by the following example. Example 3 To encode a message we can use the triple repetition code. We repeat a message three times. That is, if the message is x1 x2 x3 , we encode it as x1 x2 x3 x4 x5 x6 x7 x8 x9 where x1 = x4 = x7 , x2 = x5 = x8 , and x3 = x6 = x9 . The valid codewords are 000000000, 001001001, 010010010, 011011011, 100100100, 101101101, 110110110, and 111111111. 76 Applications of Discrete Mathematics We decode a bit string, which may contain errors, using the simple majority rule. For example, to determine x1 , we look at x1 , x4 , and x7 . If two or three of these bits are 1, we conclude that x1 = 1. Otherwise, if two or three of these bits are 0, we conclude that x1 = 0. In general, we look at the three bits corresponding to each bit in the original message. We decide that a bit in the message is 1 if a majority of bits in the string received in positions corresponding to this bit are 1s and we decide this bit is a 0 otherwise. Using this procedure, we correctly decode the message as long as at most one error has been made in the bits corresponding to each bit of the original message. For example, when a triple repetition code is used, if we receive 011111010, we conclude that the message sent was 011. (For instance, we decided that the first bit of the message was 0 since the first bit is 0, the fourth bit is 1, and the seventh bit is 0, leading us to conclude that the fourth bit is wrong.) To make the ideas introduced by the triple repetition code more precise we need to introduce some ideas about the distance between codewords and the probability of errors. We will develop several important concepts before returning to error correcting codes. Hamming Distance There is a simple way to measure the distance between two bit strings. We look at the number of positions in which these bit strings differ. This approach was used by Richard Hamming* in his fundamental work in coding theory. Definition 1 The Hamming distance d(x, y) between the bit strings x = x1 x2 . . . xn and y = y1 y2 . . . yn is the number of positions in which these strings differ, that is, the number of i (i = 1, 2, . . . , n) for which xi = yi . Note that the Hamming distance between two bit strings equals the number of changes in individual bits needed to change one of the strings into the other. * Richard Hamming (1915–1998) was one of the founders of modern coding theory. He was born in Chicago and received his B.S. from the University of Chicago, his M.A. from the University of Nebraska, and his Ph.D. from the University of Illinois in 1942. He was employed by the University of Illinois from 1942 to 1944 and the University of Louisville from 1944 to 1945. From 1945 until 1946 he was on the staff of the Manhattan Project in Los Alamos. He joined Bell Telephone Laboratories in 1946, where he worked until 1976. His research included work in the areas of coding theory, numerical methods, statistics, and digital filtering. Hamming joined the faculty of the Naval Postgraduate School in 1976. Among the awards he won are the Turing Prize from the ACM and the IEEE Hamming Medal (named after him). Chapter 5 Coding Theory 77 We will find this observation useful later. Example 4 Find the Hamming distance between the bit strings 01110 and 11011 and the Hamming distance between the bit strings 00000 and 11111. Solution: Since 01110 and 11011 differ in their first, third, and fifth bits, d(01110, 11011) = 3. Since 00000 and 11111 differ in all five bits, we conclude that d(00000, 11111) = 5. The Hamming distance satisfies all the properties of a distance function (or metric), as the following theorem demonstrates. Theorem 1 Let d(x, y) represent the Hamming distance between the bit strings x and y of length n. Then (i) d(x, y) ≥ 0 for all x, y (ii) d(x, y) = 0 if and only if x = y (iii) d(x, y) = d(y, x) for all x, y (iv) d(x, y) ≤ d(x, z) + d(z, y) for all x, y, z. Proof: Properties (i), (ii), and (iii) follow immediately from the definition of the Hamming distance. To prove (iv), we use the fact that d(x, y) is the number of changes of bits required to change x into y. Note that for every string z of length n the number of changes needed to change x into y does not exceed the number of changes required to change x into z and to then change z into y. How can the Hamming distance be used in decoding? In particular, suppose that when a codeword x from a code C is sent, the bit string y is received. If the transmission was error-free, then y would be the same as x. But if errors were introduced by the transmission, for instance by a noisy line, then y is not the same as x. How can we correct errors, that is, how can we recover x? One approach would be to compute the Hamming distance between y and each of the codewords in C. Then to decode y, we take the codeword of minimum Hamming distance from y, if such a codeword is unique. If the distance between the closest codewords in C is large enough and if sufficiently few errors were made in transmission, this codeword should be x, the codeword sent. This type of decoding is called nearest neighbor decoding. Example 5 Use nearest neighbor decoding to determine which code word 78 Applications of Discrete Mathematics was sent from the code C = {0000, 1110, 1011} if 0110 is received. Solution: We first find the distance between 0110 and each of the codewords. We find that d(0000, 0110) = 2, d(1110, 0110) = 1, d(1011, 0110) = 3. Since the closest codeword to 0110 is 1110, we conclude that 1110 was the codeword sent. Will nearest neighbor decoding produce the most likely codeword that was sent from a binary string that was received? It is not hard to see that it will if each bit sent has the same probability p of being received incorrectly and p < 1/2. We call a transmission channel with this property a binary symmetric channel. Such a channel is displayed in Figure 1. Figure 1. A binary symmetric channel. Example 6 Suppose that when a bit is sent over a binary symmetric channel the probability it is received incorrectly is 0.01. What is the probability that the bit string 100010 is received when the bit string 000000 is sent? Solution: Since the probability a bit is received incorrectly is 0.01, the probability that a bit is received correctly is 1 − 0.01 = 0.99. For 100010 to be received, when 000000 is sent, it is necessary for the first and fifth bits to be received incorrectly and the other four bits to be received correctly. The probability that this occurs is (0.99)4 (0.01)2 = 0.000096059601. We will now show that nearest neighbor decoding gives us the most likely codeword sent, so that it is also maximum likelihood decoding. Chapter 5 Coding Theory 79 Theorem 2 Suppose codewords of a binary code C are transmitted using a binary symmetric channel. Then, nearest neighbor decoding of a bit string received produces the most likely codeword sent. Proof: To prove this theorem we first need to find the probability that when a codeword of length n is sent, a bit string with k errors in specified positions is received. Since the probability each bit is received correctly is 1 − p and the probability each bit is received in error is p, it follows that the probability of k errors in specified positions is pk (1 − p)n−k . Since p < 1/2 and 1 − p > 1/2, it follows that pi (1 − p)n−i > pj (1 − p)n−j whenever i < j. Hence, if i < j, the probability that a bit string with i specified errors is received is greater than the probability that a bit string with j specified errors is received. Since is more likely that errors were made in fewer specified positions when a codeword was transmitted, nearest neighbor decoding produces the most likely codeword. The Hamming distance between codewords in a binary code determines its ability to detect and/or correct errors. We need to make the following definition before introducing two key theorems relating to this ability. Definition 2 The minimum distance of a binary code C is the smallest distance between two distinct codewords, that is, d(C) = min{d(x, y)|x, y ∈ C, x = y}. Example 7 Find the minimum distance of the code C = {00000, 01110, 10011, 11111}. Solution: To compute the minimum distance of this code we will find the distance between each pair of codewords and then find the smallest such distance. We have d(00000, 01110) = 3, d(00000, 10011) = 3, d(00000, 11111) = 5, d(01110, 10011) = 4, d(01110, 11111) = 2, and d(10011, 11111) = 2. We see that the minimum distance of C is 2. Example 8 Find the minimum distance of the code C = {000000, 111111}. 80 Applications of Discrete Mathematics Solution: Since there are only two codewords and d(000000, 111111) = 6, the minimum distance of this code is 6. The minimum distance of a code tells us how many errors it can detect and how many errors it can correct, as the following two theorems show. Theorem 3 A binary code C can detect up to k errors in any codeword if and only if d(C) ≥ k + 1. Proof: Suppose that C is a binary code with d(C) ≥ k + 1. Suppose that a codeword x is transmitted and is received with k or fewer errors. Since the minimum distance between codewords is at least k + 1, the bit string received cannot be another codeword. Hence, the receiver can detect these errors. Now suppose that C can detect up to k errors and that d(C) ≤ k. Then there are two codewords in C that differ in no more than k bits. It is then possible for k errors to be introduced when one of these codewords is transmitted so that the other codeword is received, contradicting the fact that C can detect up to k errors. Theorem 4 A binary code C can correct up to k errors in any codeword if and only if d(C) ≥ 2k + 1. Proof: Suppose that C is a binary code with d(C) ≥ 2k + 1. Suppose that a codeword x is transmitted and received with k or fewer errors as the bit string z, so that d(x, z) ≤ k. To see that C can correct these errors, note that if y is a codeword other than x, then d(z, y) ≥ k + 1. To see this note that if d(z, y) ≤ k, then by the triangle inequality d(x, y) ≤ d(x, z) + d(z, y) ≤ k + k = 2k, contradicting the assumption that d(C) ≥ 2k + 1. Conversely, suppose that C can correct up to k errors. If d(C) ≤ 2k, then there are two codewords that differ in 2k bits. Changing k of the bits in one of these codewords produces a bit string that differs from each of these two codewords in exactly k positions, thus making it impossible to correct these k errors. Example 9 Let C be the code {00000000, 11111000, 01010111, 10101111}. How many errors can C detect and how many can it correct? Solution: Computing the distance between codewords shows that the minimum distance of C is 5. By Theorem 3, it follows that C can detect up to 5 − 1 = 4 errors. For example, when we use C to detect errors, we can detect the four errors made in transmission when we receive 11110000 when the codeword 00000000 was sent. Chapter 5 Coding Theory 81 By Theorem 4, it follows that C can correct up to (5 − 1)/2 = 2 errors. For example, when we use C to correct errors, we can correct the two errors introduced in transmission when we receive 11100000 when the codeword 11111000 was sent. Perfect Codes To allow error correction we want to make the minimum distance between codewords large. But doing so limits how many codewords are available. Here we will develop a bound on the number of codewords in a binary code with a given minimum distance. Lemma 1 Suppose x is a bit string of length n and that k is a nonnegative integer not exceeding n. Then there are C(n, 0) + C(n, 1) + · · · + C(n, k). bit strings y of length n such that d(x, y) ≤ k (where d is the Hamming distance). Proof: Let i be a nonnegative integer. The number of bit strings y with d(x, y) = i equals the number of ways to select the i locations where x and y differ. This can be done in C(n, i) ways. It follows that there are C(n, 0) + C(n, 1) + · · · + C(n, k) bit strings such that d(x, y) ≤ k. We can describe the statement in Lemma 1 in geometric terms. By the sphere of radius k centered at x we mean the set ofall bit strings y such k that d(x, y) ≤ k. Lemma 1 says that there are exactly i=0 C(n, i) bit stings in the sphere of radius k centered at x. Lemma 2 Let C be a binary code containing codewords of length n and let d(C) = 2k + 1. Then given a bit string y of length n, there is at most one codeword x such that y is in the sphere of radius k centered at x. Proof: Suppose that y is in the sphere of radius k centered at two different codewords x1 and x2 . Then d(x1 , y) ≤ k and d(x2 , y) ≤ k. By the triangle inequality for the Hamming distance this implies that d(x1 , x2 ) ≤ d(x1 , y) + d(x2 , y) ≤ k + k = 2k, 82 Applications of Discrete Mathematics contradicting the fact that the minimum distance between codewords is 2k + 1. We can now give a useful bound on how many codewords can be in a code consisting of n-tuples that can correct a specified number of errors. Theorem 5 The Sphere Packing or (Hamming) Bound Suppose that C is a code of bit strings of length n with d(C) = 2k + 1. Then |C|, the number of codewords in C, cannot exceed 2n C(n, 0) + C(n, 1) + · · · + C(n, k). Proof: There are 2n bit strings of length n. By Lemma 1 the sphere of radius k centered at a codeword x contains C(n, 0) + C(n, 1) + · · · + C(n, k) bit strings. Since no bit string can be in two such spheres (by Lemma 2), it follows that the number of bit strings of length n is at least as large as the number of codewords times the number of bit strings in each such sphere. Hence, 2n ≥ |C|[C(n, 0) + C(n, 1) + · · · + C(n, k)]. We obtain the inequality we want by dividing by the second factor on the righthand side of this inequality (and writing the inequality with the smaller term first). Example 10 Find an upper bound for the number of codewords in a code C where codewords are bit strings of length seven and the minimum distance between codewords is three. Solution: The minimum distance between codewords is 3 = 2k + 1, so that k = 1. Hence the sphere packing bound shows that there are no more than 27 /[C(7, 0) + C(7, 1)] = 128/8 = 16 codewords in such a code. The sphere packing bound gives us an upper bound for the number of codewords in a binary code with a given minimum distance where codewords are bit strings of length n. The codes that actually achieve this upper bound, Chapter 5 Coding Theory 83 that is, that have the most codewords possible, are of special interest because they are the most efficient error correcting codes. Such codes are known as perfect codes. Example 11 Show that the code consisting of just two codewords 00000 and 11111 is a perfect binary code. Solution: The minimum distance between codewords in this code is 5. The sphere packing bound states that there are at most 25 /[C(5, 0) + C(5, 1) + C(5, 2)] = 32/16 = 2 codewords in a code consisting of 5-tuples with minimum distance 5. Since there are 2 codewords in this code, it is a perfect binary code. The code in Example 11 is called a trivial perfect code since it only consists of the two codewords, one containing only 0s and the other containing only 1s. As Exercise 8 demonstrates, when n is an odd positive integer there are trivial perfect codes consisting of the two codewords which are bit strings of length n consisting of all 0s and of all 1s. Finding perfect binary codes different from the trivial codes has been one of the most important problems in coding theory. In the next section we will introduce a class of perfect binary codes known as Hamming codes. Generator Matrices Before describing Hamming codes, we need to generalize the concept of a parity check bit. When we use a parity check bit, we encode a message x1 x2 . . . xk as x1 x2 . . . xk xk+1 where xk+1 = (x1 + x2 + · · · + xk ) mod 2. To generalize this notion, we add more than one check bit. More precisely, we encode a message x1 x2 . . . xk as x1 x2 . . . xk xk+1 . . . xn , where the last n − k bits xk+1 ,...,xn , are parity check bits, obtained from the k bits in the message. We will describe how these parity check bits are specified. Consider a k-bit message x1 x2 · · · xk as a 1 × k matrix x. Let G be a k × n matrix that begins with the k × k identity matrix Ik . That is, G = (Ik |A), where A is a k × (n − k) matrix, known as a generator matrix. We encode x as E(x) = xG, where we do arithmetic modulo 2. Coding using a parity check bit and using the triple repetition code are special cases of this technique, as illustrated in Examples 12 and 13. Example 12 We can represent encoding by adding a parity check bit to a 84 Applications of Discrete Mathematics three-bit message as E(x) = xG, where ⎞ 0 1 0 1⎠. 1 1 ⎛ 1 0 G = ⎝0 1 0 0 Note that to obtain G we add a column of 1s to I3 , the 3 × 3 identity matrix. That is, G = (I3 |A), where ⎛ ⎞ 1 A = ⎝1⎠. 1 Example 13 We can represent encoding using the triple repetition code for three-bit messages as E(x) = xG, where ⎛ 1 0 G = ⎝0 1 0 0 0 1 0 0 1 0 0 0 1 0 0 1 1 0 0 1 0 0 ⎞ 0 0⎠. 1 Note that G is formed by repeating the identity matrix of order three, I3 , three times, that is, G = (I3 |I3 |I3 ). We now consider an example which we will use to develop some important ideas. Example 14 Suppose that ⎛ 1 G = ⎝0 0 0 0 1 0 0 1 1 1 1 1 1 0 ⎞ 1 0⎠, 1 that is, G = (I3 |A), where ⎛ 1 1 A = ⎝1 1 1 0 ⎞ 1 0⎠. 1 What are the codewords in the code generated by this generator matrix? Chapter 5 Coding Theory 85 Solution: We encode each of the eight three-bit messages x = x1 x2 x3 as E(x) = xG. This produces the codewords 000000, 001101, 010110, 011011, 100111, 101010, 110001, and 111100. For example, we get the third of these by computing ⎛ 1 E(010) = (0 1 0)G = (0 1 0) ⎝ 0 0 0 0 1 0 0 1 1 1 1 1 1 0 ⎞ 1 0 ⎠ = (0 1 0 1 1 0). 1 It is easy to see that we can find the codewords in a binary code generated by a generator matrix G by taking all possible linear combinations of the rows of G (since arithmetic is modulo 2, this means all sums of subsets of the set of rows of G). The reader should verify this for codewords in the code in Example 14. It is easy to see that the binary codes formed using generator matrices have the property that the sum of any two codewords is again a codeword. That is, they are linear codes. To see this, suppose that y1 and y2 are codewords generated by the generator matrix G. Then there are bit strings x1 and x2 such that E(x1 ) = y1 and E(x2 ) = y2 , where E(x) = xG. It follows that y1 + y2 is also a codeword since E(x1 + x2 ) = y1 + y2 . (Here we add bit strings by adding their components in the same positions using arithmetic modulo 2.) We will see that there is an easy way to find the minimum distance of a linear code. Before we see this, we need to make the following definition. Definition 3 The weight of a codeword x, denoted by w(x), in a binary code is the number of 1s in this codeword. Example 15 Find the weights of the codewords 00000, 10111, and 11111. Solution: Counting the number of 1s in each of these codewords we find that w(00000) = 0, w(10111) = 4, and w(11111) = 5. Lemma 3 Suppose that x and y are codewords in a linear code C. Then d(x, y) = w(x + y). Proof: The positions with 1s in them in x + y are the positions where x and y differ. Hence d(x, y) = w(x + y). We also will need the fact that 0, the bit string with all 0s, belongs to a linear code. 86 Applications of Discrete Mathematics Lemma 4 in C. Proof: Suppose that C is a nonempty linear code. Then 0 is a codeword Let x be a codeword in C. Since C is linear x + x = 0 belongs to C. Theorem 6 The minimum distance of a linear code C equals the minimum weight of a nonzero codeword in C. Proof: Suppose that the minimum distance of C is d. Then there are codewords x and y such that d(x, y) = d. By Lemma 3 it follows that w(x + y) = d. Hence w, the minimum weight of a codeword, is no larger than d. Conversely, suppose that the codeword x is a nonzero codeword of minimum weight. Then using Lemma 4 we see that w = w(x) = w(x + 0) = d(x, 0) ≥ d. It follows that w = d, establishing the theorem. Parity Check Matrices Note that in Example 14 the bit string x1 x2 x3 is encoded as x1 x2 x3 x4 x5 x6 where x4 = x1 + x2 + x3 , x5 = x1 + x2 , and x6 = x1 + x3 (here, arithmetic is carried out modulo 2). Because we are doing arithmetic modulo 2, we see that x1 + x2 + x3 + x4 = 0 x1 + x2 + x5 = 0 x1 + x3 + x6 = 0. Furthermore, it is easy to see that x1 x2 x3 x4 x5 x6 is a codeword if and only if it satisfies this system of equations. We can express this system of equations as ⎛ ⎞ x1 ⎞ ⎜ x2 ⎟ ⎛ ⎞ ⎛ 0 1 1 1 1 0 0 ⎜ ⎟ x3 ⎟ ⎝ ⎠ ⎝1 1 0 0 1 0⎠⎜ 0 , = ⎜ ⎟ ⎜x ⎟ 0 1 0 1 0 0 1 ⎝ 4⎠ x5 x6 that is, t HE(x) = 0, t where E(x) is the transpose of E(x) by ⎛ 1 1 H = ⎝1 1 1 0 and H, the parity check matrix, is given 1 0 1 1 0 0 1 0 0 ⎞ 0 0⎠. 1 Chapter 5 Coding Theory 87 Note that H = (At |In−k ). With this notation we see that x = x1 x2 x3 x4 x5 x6 is a codeword if and only if Hxt = 0, since checking this equation is the same as checking whether the parity check equations hold. In general, suppose that G is a k × n generator matrix with G = (Ik |A), where A is a k × (n − k) matrix. To G we associate the parity check matrix H, where H = (At |In−k ). Then x is a codeword if and only if Hxt = 0. Note that from a generator matrix G we can find the associated parity check matrix H, and conversely, given a parity check matrix H, we can find the associated generator matrix G. More precisely, note that if H = (B|Ir ), then G = (In−r |Bt ). We have seen that the parity check matrix can be used to detect errors. That is, to determine whether x is a codeword we check whether Hxt = 0. Not only can the parity check matrix be used to detect errors, but when the columns of this matrix are distinct and are all nonzero, it also can be used to correct errors. Under these assumptions, suppose that the codeword x is sent and that y is received, which may or may not be the same as x. Write y = x + e, where e is an error string. (We have e = 0 if no errors arose in the transmission). In general, the error string e has 1s in the positions where y differs from x and 0s in all other positions. Now suppose that only one error has been introduced when x was transmitted. Then e is a bit string that has only one nonzero bit which is in the position where x and y differ, say position j. Since Hxt = 0, it follows that Hyt = H(xt + e) = Hxt + et = et = cj where cj is the jth column of H. Hence, if we receive y and assume that no more than one error is present, we can find the codeword x that was sent by computing Hyt . If this is zero, we know that y is the codeword sent. Otherwise, it will equal the jth column of H for some integer j. This means that the jth bit of y should be changed to produce x. Example 16 Use the parity check matrix to determine which codeword from the code in Example 14 was sent if 001111 was received. Assume that at most one error was made. 88 Applications of Discrete Mathematics Solution: We find that ⎛ 1 1 Hyt = ⎝ 1 1 1 0 1 0 1 1 0 0 1 0 0 ⎛ ⎞ 0 ⎞⎜0⎟ ⎛ ⎞ 0 0 ⎜ ⎟ ⎜1⎟ 0⎠⎜ ⎟ = ⎝1⎠. ⎜1⎟ 0 1 ⎝ ⎠ 1 1 This is the fifth column of H. If follows that the fifth bit of 001111 is incorrect. Hence the code word sent was 001101. Hamming Codes We can now define the Hamming codes. We define them using parity check matrices. Definition 4 A Hamming code of order r where r is a positive integer, is a code generated when we take as parity check matrix H an r × (2r − 1) matrix with columns that are all the 2r − 1 nonzero bit strings of length r in any order such that the last r columns form the identity matrix. Interchanging the order of the columns leads to what is known as an equivalent code. For details on equivalence of codes the reader is referred to the references at the end of this chapter. Example 17 Solution: Find the codewords in a Hamming code of order 2. The parity check matrix of this code is $ # 1 1 0 . H= 1 0 1 # $ 1 . Hence, the generator matrix G of 1 this code equals G = (I3−2 |Bt ) = (1 1 1). Since the codewords are linear combinations of the rows of G, we see that this code has two codewords, 000 and 111. This is the linear repetition code of order 3. We have H = (B|I2 ) where B = Example 18 Find the codewords in a Hamming code of order 3. Chapter 5 Coding Theory Solution: 89 For the parity check matrix of this code we use ⎛ 0 1 H = ⎝1 0 1 1 1 1 0 1 1 1 0 1 0 ⎞ 0 0 1 0⎠. 0 1 We have H = (B|I3 ) where ⎛ 0 B = ⎝1 1 ⎞ 1 1⎠. 1 1 1 0 1 1 0 Hence the generator matrix G of this code equals ⎛ 1 ⎜0 t G = (I7−3 |B ) = ⎝ 0 0 0 1 0 0 0 0 1 0 0 0 0 1 0 1 1 1 1 0 1 1 ⎞ 1 1⎟ ⎠. 0 1 The 16 codewords in this code C can be found by taking all possible sums of the rows of G. We leave this as an exercise at the end of the chapter. To show that the Hamming codes are perfect codes we first need to establish two lemmas. Lemma 5 2r − 1. A Hamming code of order r contains 2n−r codewords where n = Proof: The parity check matrix of the Hamming code is an r × n matrix. It follows that the generator matrix for this code is a (n − r) × n matrix. Recall that the codewords are the linear combinations of the rows. As the reader can show, no two linear combinations of the rows are the same. Since there are 2n−r different linear combinations of row, there are 2n−r different codewords in a Hamming code of order r. Lemma 6 The minimum distance of a Hamming code of order r is 3 whenever r is a positive integer. Proof: The parity check matrix Hr has columns which are all nonzero and no two of which are the same. Hence, from our earlier discussion, a Hamming code of order r can correct single errors. By Theorem 3 we conclude that the minimum distance of this code is at least 3. Among the columns of Hr are the 90 Applications of Discrete Mathematics following three columns: ⎛ ⎞ 1 ⎜1⎟ ⎜ ⎟ 0⎟ c1 = ⎜ ⎜ . ⎟, ⎝ .. ⎠ 0 ⎛ ⎞ 1 ⎜0⎟ ⎜ ⎟ 0⎟ c2 = ⎜ ⎜ . ⎟, ⎝ .. ⎠ 0 ⎛ ⎞ 0 ⎜1⎟ ⎜ ⎟ 0⎟ c3 = ⎜ ⎜ . ⎟. ⎝ .. ⎠ 0 Note that c1 + c2 + c3 = 0. Let x be the bit string with 1 in the positions of these columns and zero elsewhere. Then Hxt = 0, since it is c1 + c2 + c3 . It follows that x is a codeword. Since w(x) = 3, by Theorem 6 it follows that the minimum distance is no more than 3. We conclude that the minimum distance of a Hamming code of order r is 3. Theorem 7 The Hamming code of order r is a perfect code. Proof: Let n = 2r − 1. By Lemma 5 a Hamming code of order r contains 2n−r codewords, each of which is an n-tuple. By Lemma 6 the minimum distance of the Hamming code of order r is 3. We see that this code achieves the maximum number of codewords allowed by the sphere packing bound. To see this, note that 2n−r (1 + C(n, 1)) = 2n−r (1 + n) = 2n−r (1 + 2r − 1) = 2n . This is the upper bound of the sphere-packing bound. hence a Hamming code of order r is perfect. By Theorem 7 we see that the Hamming codes are examples of perfect codes. The study of perfect codes has been one of the most important areas of research in coding theory and has lead to the development of many important results. See the references at the end of the chapter to learn about what is known about perfect codes. Summary In this chapter we have studied how codes can be used for error detection and error correction. We have introduced an important class of codes known as the Hamming codes. However, we have only touched the surface of a fascinating and important subject that has become extremely important for modern computing and communications. The interested reader can consult the references listed at the end of this chapter to learn about many other classes of codes that have practical applications. Chapter 5 Coding Theory 91 For example, pictures of the planets taken by space probes have been encoded using powerful codes, such as a code known as the Reed-Muller code (see [5] for details). This code has been used to encode the bit string of length 6 representing the brightness of each pixel of an image by a bit string of length 32. This Reed-Muller code consists of 64 codewords, each a bit string of length 32, with minimum distance 16. Another interesting example is the use of a family of codes known as Reed-Solomon codes used in digital audio recording (see [5] for details). Finally, many concepts and techniques from both linear algebra and abstract algebra are used in coding theory. Studying coding theory may convince the skeptical reader about the applicability of some of the more abstract areas of mathematics. Suggested Readings 1. E. Berlekamp, editor, Key Papers in the Development of Coding Theory, IEEE Press, New York, 1974. 2. R. Hamming, Coding and Information Theory, 2nd Edition, Prentice Hall, Upper Saddle River, N.J., 1986. 3. R. Hill, A First Course in Coding Theory, Oxford University Press, Oxford, 1990. 4. V. Pless, Introduction to the Theory of Error-Correcting Codes, 3rd Edition, John Wiley & Sons, Hoboken, N.J., 1998. 5. S. Vanstone and P. van Oorschot, An Introduction to Error Correcting Codes with Applications, Springer, New York. 1989. Exercises 1. Could the following bit strings have been received correctly if the last bit is a parity bit? a) 1000011 b) 111111000 c) 10101010101 d) 110111011100 2. Find the Hamming distance between each of the following pairs of bit strings. 92 Applications of Discrete Mathematics a) 00000,11111 b) 1010101,0011100 c) 000000001,111000000 d) 1111111111,0100100011 3. Suppose the bit string 01011 is sent using a binary symmetric channel where the probability a bit is received incorrectly is 0.01. What is the probability that a) 01011, the bit string sent, is received? b) 11011 is received? c) 01101 is received? d) 10111 is received? e) no more than one error is present in the bit string received? 4. How many errors can each of the following binary codes detect and how many can it correct? a) {0000000, 1111111} b) {00000, 00111, 10101, 11011} c) {00000000, 11111000, 01100111, 100101101} 5. Suppose that the probability of a bit error in transmission over a binary symmetric channel is 0.001. What is the probability that when a codeword with eight bits is sent from a code with minimum distance five, the bit string received is decoded as the codeword sent (when nearest neighbor decoding is used)? 6. Show that if the minimum distance between codewords is four it is possible to correct an error in a single bit and to detect two bit errors without correction. 7. Use the sphere packing bound to give an upper bound on the number of codewords in a binary code where codewords are bit strings of length nine and the minimum distance between codewords is five. 8. Show that whenever n is an odd positive integer, the binary code consisting of the two bit strings of length n containing all 0s or all 1s is a perfect code. 9. Suppose that x and y are bit strings of length n and m is the number of positions where both x and y have 1s. Show that w(x+y) = w(x)+w(y)− 2m. 10. Find the parity check matrix associated with the code formed by adding a parity check bit to a bit string of length 4. 11. Find the parity check matrix associated with the triple repetition code for bit strings of length 3. Chapter 5 Coding Theory 93 12. Suppose that the generator matrix for a binary code is ⎛ 1 ⎜0 ⎝ 0 0 0 1 0 0 0 0 1 0 0 0 0 1 1 1 0 1 1 0 1 1 ⎞ 1 1⎟ ⎠. 1 0 What is the parity check matrix H for this code? 13. Suppose that the parity check matrix for a binary code is ⎛ 1 0 ⎝1 1 0 1 1 0 0 1 0 0 ⎞ 0 0⎠. 1 What is the generator matrix G for this code? 14. Find the 16 codewords in the Hamming code of order 3 described in Example 18. 15. Sometimes, instead of errors, bits are erased during the transmission of a message or from a tape or other storage medium. The position, but not the value, of an erased bit is known. We say that a code C can correct r erasures if a message received with no errors and with no more than r erasures can be corrected to a unique codeword that agrees with the message received in all the positions that were not erased. a) Show that a binary code of minimum distance d can correct d − 1 erasures. b) Show that a binary code of minimum distance d can correct t errors and r erasures if d = 2t + r + 1. Computer Projects 1. Given a binary code, determine the number of errors that it can detect and the number of errors that it can correct. 2. Given a binary code with minimum distance k, where k is a positive integer, write a program that will detect errors in codewords in as many as k − 1 positions and correct errors in as many as (k − 1)/2 positions.
Purchase answer to see full attachment
User generated content is uploaded by users for the purposes of learning and should be used following Studypool's honor code & terms of service.

Explanation & Answer


Anonymous
I was struggling with this subject, and this helped me a ton!

Studypool
4.7
Trustpilot
4.5
Sitejabber
4.4

Related Tags