Scale Invariance, Power Laws, and Regular Variation (Part II)

This is part II in a series on scale invariance, power laws, and regular variation, so you should definitely click on over to part I if you haven’t read that. In part I, we talked about a formalization of the notion of scale invariance and showed that a distribution is scale-invariant if and only if it has a power-law tail. This highlights that scale invariance is a very fragile property that one should not expect to see in reality and, in the strictest sense, that is true. It is quite unusual for the distribution of an observed phenomenon to exactly match a power-law distribution, and thus be scale-invariant. Instead, what tends to be observed in practice is that the body of a distribution is not scale-invariant, and the tail of a distribution is only approximately scale-invariant. Thus, it is natural to focus on distributions that have asymptotically scale-invariant tails, rather than imposing exact scale invariance.

Asymptotic scale invariance and regular variation While there are many ways to interpret the idea of approximate or asymptotic scale invariance, we will formalize the idea as follows.

Definition 3 A distribution {F} is asymptotically scale-invariant if there exists a strictly positive, finite, and continuous function {g} such that for any {\lambda>0,}

\displaystyle \lim_{x \rightarrow \infty} \frac{\bar{F}(\lambda x)}{\bar{F}(x)} = g(\lambda).

The above notion of asymptotic scale invariance almost exactly parallels the notion of scale invariance, except that it only requires the property to hold in the limit as {x\rightarrow\infty}, i.e., it only requires the property to approximately hold for the tail. As a result, it is immediate to see that Pareto distributions are asymptotically scale-invariant:

\displaystyle \bar{F}(\lambda x)/\bar{F}(x) = \lambda^{-\alpha}.

Similarly, it is easy to see that asymptotic scale invariance is still quite a special property that is not satisfied by most distributions. For example an Exponential distribution is not asymptotically scale-invariant since, as {x\rightarrow\infty}, {\frac{\bar{F}(\lambda x)}{\bar{F}(x)} = e^{-\mu(\lambda-1)x} \rightarrow \{ \infty,} if {\lambda<1}; 1, if {\lambda=1}; and 0 if {\lambda>1 \}}. So, it is not a strictly positive, finite, or continuous function of {\lambda}. However, asymptotic scale invariance is significantly broader than scale invariance, and it is also easy to see that other distributions besides power law distributions are asymptotically scale-invariant. For example, the convolution of a Pareto and an Exponential distribution is asymptotically scale-invariant. In general, since asymptotic scale invariance only focuses on the tail of the distribution, the body of such a distribution may behave in an arbitrary manner as long as the tail is approximately scale-invariant. The fact that “scale-invariant” can be thought of equivalently to “power-law” leads to the suggestion that “asymptotically scale-invariant” should correspond to some notion of “approximately power-law,” and this turns out to be true. In particular, it turns out that asymptotically scale-invariant distributions have tails that are approximately power-law in a rigorous sense that can be formalized via the class of “regularly-varying distributions.”

Definition 4 A function {f : {\mathbb R}_+ \rightarrow {\mathbb R}_+} is said to be regularly-varying of index {\rho \in {\mathbb R}} if for all {y > 0,}

\displaystyle \lim_{x \rightarrow \infty} \frac{f(xy)}{f(x)} = y^{\rho}.

Further, a distribution {F} over the non-negative reals is regularly varying of index {\rho}, denoted as {F\in RV(\rho)}, if {\bar{F}(x) = 1-F(x)} is a regularly-varying function of index {\rho.}

The form of the definition makes it very clear that regularly-varying distributions are asymptotically scale-invariant. Further, since {\lim_{x\rightarrow\infty}\bar{F}(xy)/\bar{F}(x) = y^{\rho}}, they seem to mimic the behavior of power-law distributions, such as Pareto distribution, asymptotically. This suggests that, possibly, all asymptotically scale-invariant distributions are regularly varying distributions — which turns out to be true!

Theorem 5 A distribution {F} is asymptotically scale-invariant if and only if it is regularly-varying.

Pleasantly, the proof of this parallels the proof of the earlier result showing the connection between scale invariance and power-law tails. Proof: It is immediate that regularly-varying distributions are asymptotically scale-invariant, and so we need only prove the other direction. Fix {x,y > 0.} The asymptotic scale-free property implies that

\displaystyle \lim_{z \rightarrow \infty} \frac{\bar{F}(xyz)}{\bar{F}(z)} = g(xy).

We can also compute the same limit by writing {\frac{\bar{F}(xyz)}{\bar{F}(z)} = \frac{\bar{F}(xyz)}{\bar{F}(xz)} \frac{\bar{F}(xz)}{\bar{F}(z)}.} Note that {\frac{\bar{F}(xyz)}{\bar{F}(xz)} \rightarrow g(y)} and {\frac{\bar{F}(xz)}{\bar{F}(z)} \rightarrow g(x)} as {z \rightarrow \infty,} implying that

\displaystyle \lim_{z \rightarrow \infty} \frac{\bar{F}(xyz)}{\bar{F}(z)} = g(x)g(y).

We therefore conclude that the function {g} satisfies \begin{equation*} g(xy) = g(x) g(y) \quad \text{ for all } x,y > 0. \end{equation*} It follows then that there exists {\theta \in {\mathbb R}} such that {g(x) = x^{\theta}}. Of course, by definition, this means that {\bar{F}} is a regularly-varying function, and {F} is a regularly varying distribution. \Box The fact that regularly-varying distributions are exactly those distributions that are asymptotically scale-free suggests that they, in a sense, should be able to be analyzed (at least asymptotically) like they are simply power-law distributions. In fact, this can be formalized explicitly, and regularly-varying distributions can be analyzed nearly as if they were power-law (Pareto) distributions as far as the tail is concerned. This makes them remarkably easy to work with and highlights that the added generality from working with the class of regularly varying distributions, as opposed to working specifically with Pareto distributions, comes without too much added complexity. I’ll illustrate this in part III of this series.


2 thoughts on “Scale Invariance, Power Laws, and Regular Variation (Part II)

  1. Pingback: Rigor + Relevance | Scale Invariance, Power Laws, and Regular Variation (Part III)

  2. Pingback: Rigor + Relevance | Scale Invariance, Power Laws, and Regular Variation (Part I)

Leave a Comment

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s