## 数学代写|信息论代写information theory代考|ELEN90030

statistics-lab™ 为您的留学生涯保驾护航 在代写信息论information theory方面已经树立了自己的口碑, 保证靠谱, 高质且原创的统计Statistics代写服务。我们的专家在代写信息论information theory代写方面经验极为丰富，各种代写信息论information theory相关的作业也就用不着说。

## 数学代写|信息论代写information theory代考|A coin hidden in one of eight boxes

Bob placed a coin in one of eight boxes, Fig. 1.41. Bob tells Linda that the box, in which the coin is, was chosen at random, i.e. with equal probability of $1 / 8$. To eliminate any traces of subjectivity, a random integer between one and eight was chosen and then placed the coin in the box with that number. Linda was also told that there are exactly eight boxes, and that the coin is in one of the boxes. Linda does not know where the coin is, and she has to ask binary questions in order to find out where the coin is.
I tell you, the reader, that the SMI for this game is:
$$\text { SMI(coin in eight boxes) }=\log _2 8$$
I also tell you that this number may be interpreted as a measure of information associated with the distribution $\left(\frac{1}{8}, \frac{1}{8}, \cdots, \frac{1}{8}\right)$ in the following sense: If you know only the distribution, you can find out the missing information on where the coin is, by asking binary questions, and if you are smart enough you are guaranteed to obtain this information with just three questions.
Now, pause and answer the following questions:
(i) Is the SMI for this game a subjective quantity?
(ii) Does the SMI for this game depend on who plays the game?
(iii) Does Bob calculate a different SMI for this game than Linda?

The answer to each of these three questions is No! This seems strange to someone who does not read carefully the description and rules of the game. In this description, we used the word “information” that Bob knows, but Linda doesn’t. We also used the word “smart,” which might suggest to some that if the person who plays the game is not smart, he or she might calculate a different SMI for this game. All these “words” do not change the fact that the number: $\log _2 8=3$ is not a subjective number. In the description of the game I told you that Bob placed the coin in one of the boxes, so he must know the information on the location of the coin, while Linda doesn’t. However, when I ask you about the SMI that Bob will calculate for this game, the answer is $\log _2 8=3$, independently of what Bob knows or doesn’t. When Bob plays the game, it means that all he knows is that there are eight equally probable possibilities. With that information he still has to ask three questions.

## 数学代写|信息论代写information theory代考|A dart hit a board divided into eight regions of unequal areas

This game is a little more difficult since it involves a non-uniform distribution.
It is known that a dart was thrown on a board with a unit area. The board is divided into eight regions with areas $p_1, p_2, \cdots, p_8$. It is also known that the dart is in one of those areas and the probabilities of being in one of those regions is proportional to the ratio of the area of that region and the total area of the board (which was chosen as unity). Thus, we know that:
$$\sum p_i=1$$
And we define the SMI for this distribution as:
$$\text { SMI(dart on eight regions })=-\sum p_i \log p_i$$
The sum is over al $i=1,2, \ldots, 8$. Now, we play the same game as before. Bob threw the dart and Linda has to ask binary questions in order to find out where the dart is.

Read questions (i) to (iii) asked in connection with the previous game and answer them. Again, the answers to all those questions is No! Clearly, if the distribution is not uniform the average number of questions one needs to ask in order to obtain the missing information is smaller than $\log _2 8$. This was proven in Chap. 2 of BenNaim [1]. However, whatever the distribution is, it determines the value of the SMI as defined in Eq. (1.49), and this value is independent of who plays the game, who knows or does not know where the dart is, and whether or not the game is played at all. The value of the SMI is determined once you are given the distribution, and this number has no element of subjectivity. The game we built upon this distribution, and the identification of specific persons involved in this game are parts of the interpretation of the SMI; they do not affect the value of the SMI.

# 信息论代写

## 数学代写|信息论代写information theory代考|A coin hidden in one of eight boxes

$$\text { SMI(coin in eight boxes) }=\log _2 8$$

(i)这款游戏的SMI是否属于主观数量?
(ii)这个游戏的SMI是否取决于谁玩这个游戏?
(iii) Bob在这个游戏中计算的SMI是否与Linda不同?

## 数学代写|信息论代写information theory代考|A dart hit a board divided into eight regions of unequal areas

$$\sum p_i=1$$

$$\text { SMI(dart on eight regions })=-\sum p_i \log p_i$$

## 有限元方法代写

tatistics-lab作为专业的留学生服务机构，多年来已为美国、英国、加拿大、澳洲等留学热门地的学生提供专业的学术服务，包括但不限于Essay代写，Assignment代写，Dissertation代写，Report代写，小组作业代写，Proposal代写，Paper代写，Presentation代写，计算机作业代写，论文修改和润色，网课代做，exam代考等等。写作范围涵盖高中，本科，研究生等海外留学全阶段，辐射金融，经济学，会计学，审计学，管理学等全球99%专业科目。写作团队既有专业英语母语作者，也有海外名校硕博留学生，每位写作老师都拥有过硬的语言能力，专业的学科背景和学术写作经验。我们承诺100%原创，100%专业，100%准时，100%满意。

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。

## 数学代写|信息论代写information theory代考|ELEN90030

statistics-lab™ 为您的留学生涯保驾护航 在代写信息论information theory方面已经树立了自己的口碑, 保证靠谱, 高质且原创的统计Statistics代写服务。我们的专家在代写信息论information theory代写方面经验极为丰富，各种代写信息论information theory相关的作业也就用不着说。

## 数学代写|信息论代写information theory代考|THE HALTING PROBLEM AND THE NONCOMPUTABILITY OF KOLMOGOROV COMPLEXITY

This statement is false.
This paradox is sometimes stated in a two-statement form:

These paradoxes are versions of what is called the Epimenides liar para$d o x$, and it illustrates the pitfalls involved in self-reference. In 1931, Gödel used this idea of self-reference to show that any interesting system of mathematics is not complete; there are statements in the system that are true but that cannot be proved within the system. To accomplish this, he translated theorems and proofs into integers and constructed a statement of the above form, which can therefore not be proved true or false.

The halting problem in computer science is very closely connected with Gödel’s incompleteness theorem. In essence, it states that for any computational model, there is no general algorithm to decide whether a program will halt or not (go on forever). Note that it is not a statement about any specific program. Quite clearly, there are many programs that can easily be shown to halt or go on forever. The halting problem says that we cannot answer this question for all programs. The reason for this is again the idea of self-reference.

To a practical person, the halting problem may not be of any immediate significance, but it has great theoretical importance as the dividing line between things that can be done on a computer (given unbounded memory and time) and things that cannot be done at all (such as proving all true statements in number theory). Gödel’s incompleteness theorem is one of the most important mathematical results of the twentieth century, and its consequences are still being explored. The halting problem is an essential example of Gödel’s incompleteness theorem.

One of the consequences of the nonexistence of an algorithm for the halting problem is the noncomputability of Kolmogorov complexity. The only way to find the shortest program in general is to try all short programs and see which of them can do the job. However, at any time some of the short programs may not have halted and there is no effective (finite mechanical) way to tell whether or not they will halt and what they will print out. Hence, there is no effective way to find the shortest program to print a given string.

## 数学代写|信息论代写information theory代考|UNIVERSAL GAMBLING

Suppose that a gambler is asked to gamble sequentially on sequences $x \in{0,1}^*$. He has no idea of the origin of the sequence. He is given fair odds (2-for-1) on each bit. How should he gamble? If he knew the distribution of the elements of the string, he might use proportional betting because of its optimal growth-rate properties, as shown in Chapter 6. If he believes that the string occurred naturally, it seems intuitive that simpler strings are more likely than complex ones. Hence, if he were to extend the idea of proportional betting, he might bet according to the universal probability of the string. For reference, note that if the gambler knows the string $x$ in advance, he can increase his wealth by a factor of $2^{l(x)}$ simply by betting all his wealth each time on the next symbol of $x$. Let the wealth $S(x)$ associated with betting scheme $b(x), \sum b(x)=1$, be given by
$$S(x)=2^{l(x)} b(x) .$$
Suppose that the gambler bets $b(x)=2^{-K(x)}$ on a string $x$. This betting strategy can be called universal gambling. We note that the sum of the bets
$$\sum_x b(x)=\sum_x 2^{-K(x)} \leq \sum_{p: p \text { halts }} 2^{-l(p)}=\Omega \leq 1,$$
and he will not have used all his money. For simplicity, let us assume that he throws the rest away. For example, the amount of wealth resulting from a bet $b(0110)$ on a sequence $x=0110$ is $2^{l(x)} b(x)=2^4 b(0110)$ plus the amount won on all bets $b(0110 \ldots)$ on sequences that extend $x$.

# 信息论代写

## 数学代写|信息论代写information theory代考|UNIVERSAL GAMBLING

$$S(x)=2^{l(x)} b(x) .$$

$$\sum_x b(x)=\sum_x 2^{-K(x)} \leq \sum_{p: p \text { halts }} 2^{-l(p)}=\Omega \leq 1,$$

## 有限元方法代写

tatistics-lab作为专业的留学生服务机构，多年来已为美国、英国、加拿大、澳洲等留学热门地的学生提供专业的学术服务，包括但不限于Essay代写，Assignment代写，Dissertation代写，Report代写，小组作业代写，Proposal代写，Paper代写，Presentation代写，计算机作业代写，论文修改和润色，网课代做，exam代考等等。写作范围涵盖高中，本科，研究生等海外留学全阶段，辐射金融，经济学，会计学，审计学，管理学等全球99%专业科目。写作团队既有专业英语母语作者，也有海外名校硕博留学生，每位写作老师都拥有过硬的语言能力，专业的学科背景和学术写作经验。我们承诺100%原创，100%专业，100%准时，100%满意。

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。

## 数学代写|信息论代写information theory代考|CHERNOFF INFORMATION

statistics-lab™ 为您的留学生涯保驾护航 在代写信息论information theory方面已经树立了自己的口碑, 保证靠谱, 高质且原创的统计Statistics代写服务。我们的专家在代写信息论information theory代写方面经验极为丰富，各种代写信息论information theory相关的作业也就用不着说。

## 数学代写|信息论代写information theory代考|CHERNOFF INFORMATION

We have considered the problem of hypothesis testing in the classical setting, in which we treat the two probabilities of error separately. In the derivation of the Chernoff-Stein lemma, we set $\alpha_n \leq \epsilon$ and achieved $\beta_n \doteq 2^{-n D}$. But this approach lacks symmetry. Instead, we can follow a Bayesian approach, in which we assign prior probabilities to both hypotheses. In this case we wish to minimize the overall probability of error given by the weighted sum of the individual probabilities of error. The resulting error exponent is the Chernoff information.

The setup is as follows: $X_1, X_2, \ldots, X_n$ i.i.d. $\sim Q$. We have two hypotheses: $Q=P_1$ with prior probability $\pi_1$ and $Q=P_2$ with prior probability $\pi_2$. The overall probability of error is
$$P_e^{(n)}=\pi_1 \alpha_n+\pi_2 \beta_n .$$
Let
$$D^=\lim {n \rightarrow \infty}-\frac{1}{n} \log \min {A_n \subseteq \mathcal{X}^n} P_e^{(n)}$$
Theorem 11.9.1 (Chernoff) The best achievable exponent in the Bayesian probability of error is $D^$, where
$$D^=D\left(P_{\lambda^} | P_1\right)=D\left(P_{\lambda^} | P_2\right),$$ with $$P_\lambda=\frac{P_1^\lambda(x) P_2^{1-\lambda}(x)}{\sum_{a \in \mathcal{X}} P_1^\lambda(a) P_2^{1-\lambda}(a)},$$ and $\lambda^$ the value of $\lambda$ such that
$$D\left(P_{\lambda^} | P_1\right)=D\left(P_{\lambda^} | P_2\right) .$$

## 数学代写|信息论代写information theory代考|FISHER INFORMATION AND THE CRAMER–RAO ´INEQUALITY

A standard problem in statistical estimation is to determine the parameters of a distribution from a sample of data drawn from that distribution. For example, let $X_1, X_2, \ldots, X_n$ be drawn i.i.d. $\sim \mathcal{N}(\theta, 1)$. Suppose that we wish to estimate $\theta$ from a sample of size $n$. There are a number of functions of the data that we can use to estimate $\theta$. For example, we can use the first sample $X_1$. Although the expected value of $X_1$ is $\theta$, it is clear that we can do better by using more of the data. We guess that the best estimate of $\theta$ is the sample mean $\bar{X}_n=\frac{1}{n} \sum X_i$. Indeed, it can be shown that $\bar{X}_n$ is the minimum mean-squared-error unbiased estimator.

We begin with a few definitions. Let ${f(x ; \theta)}, \theta \in \Theta$, denote an indexed family of densities, $f(x ; \theta) \geq 0, \int f(x ; \theta) d x=1$ for all $\theta \in \Theta$. Here $\Theta$ is called the parameter set.

Definition An estimator for $\theta$ for sample size $n$ is a function $T$ : $\mathcal{X}^n \rightarrow \Theta$.

An estimator is meant to approximate the value of the parameter. It is therefore desirable to have some idea of the goodness of the approximation. We will call the difference $T-\theta$ the error of the estimator. The error is a random variable.

Definition The bias of an estimator $T\left(X_1, X_2, \ldots, X_n\right)$ for the parameter $\theta$ is the expected value of the error of the estimator [i.e., the bias is $\left.E_\theta T\left(x_1, x_2, \ldots, x_n\right)-\theta\right]$. The subscript $\theta$ means that the expectation is with respect to the density $f(\cdot ; \theta)$. The estimator is said to be unbiased if the bias is zero for all $\theta \in \Theta$ (i.e., the expected value of the estimator is equal to the parameter).

Example 11.10.1 Let $X_1, X_2, \ldots, X_n$ drawn i.i.d. $\sim f(x)=(1 / \lambda)$ $e^{-x / \lambda}, x \geq 0$ be a sequence of exponentially distributed random variables. Estimators of $\lambda$ include $X_1$ and $\bar{X}_n$. Both estimators are unbiased.

# 信息论代写

## 数学代写|信息论代写information theory代考|CHERNOFF INFORMATION

$$P_e^{(n)}=\pi_1 \alpha_n+\pi_2 \beta_n .$$

$$D^=\lim {n \rightarrow \infty}-\frac{1}{n} \log \min {A_n \subseteq \mathcal{X}^n} P_e^{(n)}$$

$$D^=D\left(P_{\lambda^} | P_1\right)=D\left(P_{\lambda^} | P_2\right),$$与$$P_\lambda=\frac{P_1^\lambda(x) P_2^{1-\lambda}(x)}{\sum_{a \in \mathcal{X}} P_1^\lambda(a) P_2^{1-\lambda}(a)},$$和$\lambda^$的值$\lambda$，这样
$$D\left(P_{\lambda^} | P_1\right)=D\left(P_{\lambda^} | P_2\right) .$$

## 有限元方法代写

tatistics-lab作为专业的留学生服务机构，多年来已为美国、英国、加拿大、澳洲等留学热门地的学生提供专业的学术服务，包括但不限于Essay代写，Assignment代写，Dissertation代写，Report代写，小组作业代写，Proposal代写，Paper代写，Presentation代写，计算机作业代写，论文修改和润色，网课代做，exam代考等等。写作范围涵盖高中，本科，研究生等海外留学全阶段，辐射金融，经济学，会计学，审计学，管理学等全球99%专业科目。写作团队既有专业英语母语作者，也有海外名校硕博留学生，每位写作老师都拥有过硬的语言能力，专业的学科背景和学术写作经验。我们承诺100%原创，100%专业，100%准时，100%满意。

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。