Foundations of Calculus

Background

#

There is no need to emphasize the importance of math. However, for most people learning math, efforts are put into calculations rather than theory. In some circumstances, a lack of theoretical knowledge leads to the misuse of theorems, yielding wrong results.

This series includes notes based on Calculus, a two-volume textbook series by Tom M. Apostol. The series not only covers calculus (including single-variable calculus and multivariable calculus) on a relatively strong theoretical foundation, but also covers linear algebra and probability. These topics¹ are the main topics taught in university for a variety of programs.

The notes follow the same order in which I read the chapters of the books, rather than the order given by the books, because, as stated by the author in the preface of Volume 2, the chapters can be read in different orders. Chapters I deemed unimportant are also skipped.

Introduction to Calculus

#

What is Calculus?

#

Calculus originates from two fundamental geometric problems formulated from concepts in various fields. Consider a curve \(C\) lying above a horizontal base line intersected by any vertical line at most once. The two problems are:

• The Area Problem: To measure the area bounded by \(C\), the base line, and two vertical segments.
• The Tangent Problem: To measure the steepness of a line tangent to \(C\).

Calculus provides the precise formulation and solution for these problems, enabling us to define and calculate area and tangents. Integral calculus deals with the area problem, and differential calculus deals with the tangent problem.

Some History

Integral calculus traces back to ancient Greece, where the method of exhaustion was used to compute areas. For instance, Archimedes showed that a parabolic segment with base length \(b\) has area \(b^3/3\)², which was accepted as a mathematical theorem until mathematicians realized that it could not be so considered until a satisfactory definition of area was given first.³

Nevertheless, Archimedes’ work suggests a reasonable way to define area for arbitrary figures, and to define a much more general concept known as the integral.

A Set of Axioms for the Real-Number System

#

A thorough and complete treatment of calculus depends on a careful study of the real number system. There are many ways to introduce the real-number system.⁴ The approach taken in the book is to take the real numbers as the undefined objects satisfying a number of properties used as axioms.

Basic Concepts of the Theory of Sets

In discussing any branch of math, it’s helpful to use the notation and terminology of set theory⁵. Before continuing, please make sure you understand the basic notions:

• What is a set? What are elements or members of a set?
• What is the roster notation?
• What is the definition of set equality?
• What is the definition of a subset? What about a proper subset? If \(A \subseteq B\) and \(B \subseteq A\), what can we say about \(A = B\)? What about the converse?
• What is the universal set? What is the empty set or the void set? What is the relationship between the empty set and any other set?
• What are Venn diagrams?
• What are set operations? What operations are commutative or associative?
• What is a class? What does \(\bigcup_{A \in \mathcal{F}} A\) or \(\bigcap_{A \in \mathcal{F}} A\) mean?

The axioms fall into three groups: the field axioms, the order axioms, and the least-upper-bound axiom (also called the completeness axiom).

The field axioms

#

Along with the set \(\mathbb{R}\) of real numbers, we assume the existence of two operations called addition and multiplication, such that for every pair of real numbers \(x\) and \(y\) we can form other real numbers:

• the sum of \(x\) and \(y\), denoted by \(x + y\)
• the product of \(x\) and \(y\), denoted by \(x \cdot y\)

It is assumed that \(x + y\) and \(x \cdot y\) are uniquely determined by \(x\) and \(y\).

Axiom 1. Commutative Laws. \(x+y=y+x,\ xy=yx.\)
Axiom 2. Associative Laws. \(x+(y+z)=(x+y)+z,\ x(yz)=(xy)z.\)
Axiom 3. Distributive Law. \(x(y+z)=xy+xz.\)
Axiom 4. Existence of Identity Element. There exist two distinct real numbers, denoted by \(0\) and \(1\), such that for every real \(x\) we have \(x+0=x\) and \(1\cdot x=x\).
Axiom 5. Existence of Negatives. For every real number \(x\) there is a real number \(y\) such that \(x+y=0\).
Axiom 6. Existence of Reciprocals. For every real number \(x\neq0\) there is a real number \(y\) such that \(xy=1\).
Note: The numbers \(0\) and \(1\) in Axioms 5 and 6 are those of Axiom 4.

From the above axioms we can deduce all the usual laws of elementary algebra. Here are some of them⁶:

Theorem I.1. Cancellation Law for Addition. If \(a+b=a+c\), then \(b=c\). (In particular, this shows that the number \(0\) of Axiom 4 is unique.)
Theorem I.2. Possibility of Subtraction. Given \(a\) and \(b\), there is exactly one \(x\) such that \(a+x=b\). This \(x\) is denoted by \(b-a\). In particular, \(0-a\) is written simply \(-a\) and is called the negative of \(a\).

Example: Obtaining Theorem I.1. as a Consequence of the Axioms

Proof: Given \(a+b=a+c\). By Axiom 5, there is a number \(y\) such that \(y+a=0\). Since sums are uniquely determined, we have \(y+(a+b)=y+(a+c)\). Using the associative law, we obtain \((y+a)+b=(y+a)+c\) or \(0+b=0+c\). But by Axiom 4 we have \(0+b=b\) and \(0+c=c\), so that \(b=c\).

The order axioms

#

This group of axioms allow us to establish an ordering among the real numbers. We introduce the order properties as a set of axioms about an undefined concept called positiveness, and define terms like less than and greater than in terms of positiveness.

It is assumed that there exists a subset \(\mathbb{R^+}\subset\mathbb{R}\), called the set of positive numbers, which satisfies the three order axioms:

Axiom 7. If \(x\) and \(y\) are in \(\mathbb{R^+}\), so are \(x+y\) and \(xy\).
Axiom 8. For every real \(x\neq0\), either \(x\in\mathbb{R^+}\) or \(-x\in\mathbb{R^+}\), but not both.
Axiom 9. \(0\notin\mathbb{R^+}\).

Then we can define the symbols \(\lt, \gt, \leq, \geq\), called, respectively, less than, greater than, less than or equal to, greater than or equal to, as follows:

• \(x\lt y\) means that \(y-x\) is positive
• \(y\gt x\) means that \(x\lt y\)
• \(x\leq y\) means that either \(x\lt y\) or \(x=y\)
• \(y\geq x\) means that \(x\leq y\)

From the order axioms we can derive all the usual rules for calculating with inequalities. Here are some of them⁷:

Theorem I.16. Trichotomy Law. For arbitrary real numbers \(a\) and \(b\), exactly one of the three relations \(a\lt b\), \(b\lt a\), \(a=b\) holds.
Theorem I.17. Transitive Law. If \(a\lt b\) and \(b\lt c\), then \(a\lt c\).

Integers, Rational Numbers, and Irrational Numbers

The set of integers \(\mathbb{Z}\) consists of the positive integers⁸, their negatives, and 0. The set of rational numbers \(\mathbb{Q}\) consists of all quotients \(a/b\) where \(a, b \in \mathbb{Z}\) and \(b \neq 0\). \(\mathbb{Z}\) and \(\mathbb{Q}\) are special subsets of \(\mathbb{R}\) in that they satisfy all the field axioms and the order axioms. ⁹

Real numbers not in \(\mathbb{Q}\) are called irrational numbers. They arise when we solve certain equations like \(x^2=2\). From the nine axioms above, we cannot prove that such an \(x\) exists in \(\mathbb{R}\), because these nine axioms are also satisfied by \(\mathbb{Q}\), and there is no rational number \(x\) whose square is 2.

The least-upper bound axiom (completeness axiom)

#

To establish the existence of irrational numbers, we need another axiom. It allows us to introduce irrational numbers in the real-number system, giving it a property of continuity that is a keystone in the logical structure of calculus.

Before introducing Axiom 10, we define some basic terminology. Suppose \(S\) is a nonempty set of real numbers and there is a number \(B\) such that \(x\leq B\) for every \(x\) in \(S\), then \(S\) is bounded above by \(B\), and \(B\) is an upper bound for \(S\). If an upper bound \(B\) is also a member of \(S\), then \(B\) is the largest member or the maximum element of \(S\). A set with no upper bound is unbounded above.

For sets that are bounded above but have no maximum element (e.g. \(\{x\mid x\in\mathbb{R},\ 0\leq x\lt 1\}\)), the least upper bound takes the place of the maximum element.

Definition of Least Upper Bound. A number \(B\) is called a least upper bound of a nonempty set \(S\) if:
(a) \(B\) is an upper bound for S;
(b) no number less than \(B\) is an upper bound for \(S\).

Theorem I.26. Two different numbers cannot be least upper bounds for the same set.\

Hint for Proof

Suppose \(B\) and \(C\) are two least upper bounds for a set \(S\). Use property (b) and the Trichotomy Law.

This theorem tells us that if there is a least upper bound for a set \(S\), there is only one, and we may speak of the least upper bound, or the more concise term supremum, abbreviated sup.

AXIOM 10. Every nonempty set \(S\) of real numbers which is bounded above has a supremum; that is, there is a real number \(B\) such that \(B = \sup S\).

By symmetry, definitions of lower bound, bounded below, smallest member (or minimum element), greatest lower bound (or infimum) can be similarly formulated.

Theorem I.27. Every nonempty set \(S\) that is bounded below has a greatest lower bound; that is, there is a real number \(L\) such that \(L = \inf S\).

Hint for Proof

Let \(-S\) denote the set of negatives of numbers in \(S\) and apply Axiom 10.

Existence of Square Roots of Nonnegative Real Numbers

The existence of at least one square root¹⁰ can be deduced from an important theorem in calculus¹¹, but it’s instructive to see how it can be proved directly from Axiom 10.

Theorem I.35.¹² Every nonnegative real number \(a\) has a unique nonnegative square root.
Proof Sketch. Let \(S = \{x > 0 \mid x^2 < a\}\). First we show \(S\) is nonempty and bounded above. By Axiom 10, \(b = \sup S\) exists. Then, show that both \(b^2 > a\) and \(b^2 < a\) lead to contradictions: for \(b^2 > a\), construct \(c = \frac12(b + a/b)\) and show that \(c < b\) and \(c^2 > a\); for \(b^2 < a\), choose a positive number \(c\) such that \(c \lt b \) and \(c\lt(a-b^2)/(3b)\), and show that \(b + c\) is in \(S\). Uniqueness follows from \(x^2 - y^2 = (x-y)(x+y) = 0\).

The least-upper-bound axiom can also be used to show the existence of roots of higher order. For example, if \(n\) is a positive odd integer, then for each real \(x\) there is exactly one real \(y\) such that \(y^n=x\). This \(y\) is called the \(n\)th root of \(x\), denoted by \(y=x^{1/n}\).

Using these roots, we can define rational powers: for \(r = m/n\) where \(m\) and \(n\) are positive integers, we define \(x^r = (x^m)^{1/n}\) and \(x^{-r} = 1/x^r\).

Representation of Real Numbers by Decimals

Decimal expansions can be defined analytically using the least-upper-bound axiom.¹³ For a positive real number \(x\), let \(a_0\) be the largest integer \(\leq x\). Having chosen \(a_0, a_1, \dots, a_{n-1}\), let \(a_n\) be the largest integer such that

$$ \tag{I.17} a_0 + \frac{a_1}{10} + \dots + \frac{a_n}{10^n} \leq x. $$

Let \(S\) be the set of all numbers \(a_0 + a_1/10 + \dots + a_n/10^n\) for \(n = 0, 1, 2, \dots\). Then \(S\) is nonempty, bounded above, and \(x = \sup S\). The decimal expansion \(x = a_0.a_1a_2a_3\cdots\) is defined by this construction, where the \(n\)th digit \(a_n\) is the largest integer satisfying (I.17).

If we replace \(\leq\) in (I.17) by \(<\), we obtain a slightly different definition of decimal expansions. The supremum of the corresponding set is still \(x\), although the integers \(a_0, a_1, a_2, \dots\) need not be the same as those satisfying (I.17).

The fact that a real number might have two different decimal representations is merely a reflection of the fact that two different sets of real numbers can have the same supremum.

Further Properties of the Real-Number System

#

The field axioms, order axioms, and completeness axiom are the fundamental properties of the real-number system. There are several other important properties that are worth discussing.

The Archimedean Property

#

This section contains a number of important properties of the real-number system which are consequences of the least-upper-bound axiom.

Theorem I.28. The set \(\mathbb{P}\) of positive integers \(1, 2, 3, \dots\) is unbounded above.

Hint for Proof

Assume \(\mathbb{P}\) is bounded above and use Axiom 10 to find a least upper bound for \(\mathbb{P}\). Find a contradiction.

As corollaries of Theorem I.28, we obtain the following consequences:

Theorem I.29. For every real \(x\) there exists a positive integer \(n\) such that \(n \gt x\).

Hint for Proof

Assume this were not so and find a contradiction with Theorem I.28.

Theorem I.30. Archimedean Property of the Real-number System. If \(x \gt 0\) and if \(y\) is an arbitrary real number, there exists a positive integer \(n\) such that \(nx \gt y\).

Hint for Proof

Replace \(x\) by \(\frac{y}{x}\) in Theorem I.29.

Theorem I.31. If three real numbers \(a\), \(x\), and \(y\) satisfy the inequalities \(a \leq x \leq a+\frac{y}{n}\) for every integer \(n \gt 1\), then \(x = a\).

Hint for Proof

Assume \(x\gt a\) and find a contradiction using Theorem I.30.

Fundamental Properties of the Supremum and Infimum

#

This section discusses three fundamental properties of the supremum and infimum that are used in our development of calculus.

Theorem I.32. Let \(h\) be a given positive number and let \(S\) be a set of real numbers.
(a) If \(S\) has a supremum, then for some \(x\) in \(S\) we have \(x \gt \sup S - h\).
(b) If \(S\) has an infimum, then for some \(x\) in \(S\) we have \(x \lt \inf S + h\).

Hint for Proof

For (a), assume \(x < \sup S - h\) for all \(x\) in \(S\) and find a contradiction using the definition of supremum.

Theorem I.33. Additive Property. Given nonempty subsets \(A\) and \(B\) of \(\mathbb{R}\), let \(C\) denote the set \(C = \{a + b \mid a \in A, b \in B\}\).
(a) If each of \(A\) and \(B\) has a supremum, then \(C\) has a supremum, and \(\sup C = \sup A + \sup B\).
(b) If each of \(A\) and \(B\) has an infimum, then \(C\) has an infimum, and \(\inf C = \inf A + \inf B\).

Hint for Proof

For (a), show that \(\sup C \lt \sup A + \sup B\), then use Theorem I.32. (with \(h = 1/n\)) and Theorem I.31.

Theorem I.34. Given two nonempty subsets \(S\) and \(T\) of \(R\) such that \(s \leq t\) for every \(s\) in \(S\) and every \(t\) in \(T\). Then \(S\) has a supremum, and \(T\) has an infimum, and they satisfy the inequality \(\sup S \leq \inf T\).

Hint for Proof

Each \(t\) in \(T\) is an upper bound for \(S\).

Mathematical Tools for Calculus

#

Mathematical Induction

#

Method of Proof by Induction. Let \(A(n)\) be an assertion involving an integer \(n\). We conclude that \(A(n)\) is true for every \(n \geq n_1\) if we can perform the following two steps:
(a) Prove that \(A(n_1)\) is true.
(b) Let \(k\) be an arbitrary but fixed integer \(\geq n_1\). Assume that \(A(k)\) is true and prove that \(A(k + 1)\) is also true.

The logical justification for this method of proof is the following theorem about real numbers.

Theorem 1.36. Principle of Mathematical Induction. Let \(S\) be a set of positive integers which has the following two properties: (a) The number \(1\) is in the set \(S\); (b) If an integer \(k\) is in \(S\), then so is \(k + 1\). Then every positive integer is in the set \(S\).

Hint for Proof

Properties (a) and (b) imply that \(S\) is an inductive set.

The Well-ordering Principle

The well-ordering principle is another important property of the positive integers. It is equivalent to the principle of mathematical induction.

Theorem I.37. Well-ordering Principle. Every nonempty set of positive integers contains a smallest member.
Proof Sketch. Let \(T\) be a nonempty collection of positive integers. Suppose \(T\) has no smallest member. Show that this leads to a contradiction.

What’s Wrong with the Following "Proof" by Induction?

Statement. Given any collection of \(n\) blonde girls. If at least one of the girls has blue eyes, then all \(n\) of them have blue eyes.
"Proof". The statement is obviously true when \(n = 1\). The step from \(k\) to \(k + 1\) can be illustrated by going from \(n = 3\) to \(n = 4\). Assume, therefore, that the statement is true when \(n = 3\) and let \(G_1, G_2, G_3, G_4\) be four blonde girls, at least one of which, say \(G_1\) has blue eyes. Taking \(G_1\), \(G_2\), and \(G_3\) together and using the fact that the statement is true when \(n = 3\), we find that \(G_2\) and \(G_3\) also have blue eyes. Repeating the process with \(G_1\), \(G_2\), and \(G_4\) we find that \(G_4\) has blue eyes. Thus all four have blue eyes. A similar argument allows us to make the step from \(k\) to \(k + 1\) in general.

Corollary. All blonde girls have blue eyes.
Proof. Since there exists at least one blonde girl with blue eyes, we can apply the foregoing result to the collection consisting of all blonde girls.¹⁴

The Summation Notation

#

We denote the sum of several real numbers \(a_1, a_2, \dots, a_n\) by the symbol

$$\tag{I.21} a_1 + a_2 + \dots + a_n$$

or, using summation notation:

$$\tag{I.22}\sum_{k=1}^n a_k.$$

From a strictly logical standpoint, the symbols in (I.21) and (I.22) do not appear among the primitive symbols for the real-number system. In a more careful treatment, we could define these new symbols in terms of the primitive undefined symbols of our system using definition by induction:

(a) We define \(\sum_{k=1}^1 a_k = a_1\).
(b) Assuming that we have defined \(\sum_{k=1}^n a_k\) for a fixed \(n\geq1\), we further define \(\sum_{k=1}^{n+1} a_k = (\sum_{k=1}^n a_k) + a_{n+1}\).

Notice that definition by induction and proof by induction involve the same underlying idea. A definition by induction is also called a recursive definition.

Absolute Values and the Triangle Inequality

#

Calculations with inequalities arise frequently in calculus. They are of particular importance in dealing with the notion of absolute value.

If x is a real number, the absolute value of x is a nonnegative real number denoted by \(\lvert x \rvert\) and defined as \(\lvert x \rvert = \begin{cases} x & \text{if}\ x \geq 0,\\ -x & \text{if}\ x \leq 0.\end{cases}\)

Theorem I.38. If \(a \gt 0\), then \(\lvert x \rvert \leq a\) if and only if \(-a \leq x \leq a\).

Hint for Proof

Use the definition of absolute value.

The following theorem is a consequence of Theorem I.38.

Theorem I.39. Triangle Inequality.¹⁵ For arbitrary real numbers x and y, we have \(\lvert x + y \rvert \leq \lvert x \rvert + \lvert y \rvert \).

Hint for Proof

Add the inequalities \(-\lvert x\rvert \lt x \lt \lvert x \rvert\) and \(-\lvert y \rvert \lt y \lt \lvert y \rvert\), and use Theorem I.38.

Using mathematical induction, we may extend the triangle inequality as follows:

Theorem I.40. For arbitrary real numbers \(a_1, a_2, \dots, a_n\), we have \( \lvert \sum_{k=1}^n a_k \rvert \leq \sum_{k=1}^n \lvert a_k \rvert \).

Hint for Proof

Use mathematical induction.

The next theorem describes an important inequality that we shall use later in connection with our study of vector algebra.

Theorem I.41. The Cauchy-Schwarz Inequality. If \(a_1, \dots, a_n\) and \(b_1, \dots, b_n\) are arbitrary real numbers, we have \( (\sum_{k=1}^n a_kb_k)^2 \leq (\sum_{k=1}^n a_k^2)(\sum_{k=1}^n b_k^2) \). The equality sign holds if and only if there is a real number \(x\) such that \(a_kx + b_k = 0\) for each \(k = 1,2,\dots,n\).

Hint for Proof

We have \(\sum_{k=1}^n(a_kx+b_k)^2 \geq 0\) for every real \(x\). Write it in the form \(Ax^2 + 2Bx + C \geq 0\).