### 数学代写|信息论作业代写information theory代考|Shannon-Fano-Elias Coding

statistics-lab™ 为您的留学生涯保驾护航 在代写信息论information theory方面已经树立了自己的口碑, 保证靠谱, 高质且原创的统计Statistics代写服务。我们的专家在代写信息论information theory代写方面经验极为丰富，各种代写信息论information theory相关的作业也就用不着说。

• Statistical Inference 统计推断
• Statistical Computing 统计计算
• (Generalized) Linear Models 广义线性模型
• Statistical Machine Learning 统计机器学习
• Longitudinal Data Analysis 纵向数据分析
• Foundations of Data Science 数据科学基础

## 数学代写|信息论作业代写information theory代考|Shannon-Fano-Elias Coding

Codes that use the codeword lengths of $l(x)=\left[\log \frac{1}{P(x)}\right\rceil$ are called Shannon Codes. Shannon codeword lengths satisfy the Kraft inequality and can therefore be used to construct a uniquely decodable code. In this section we will study another simple method for constructing uniquely decodable codes based on ShannonFano-Elias encoding technique. It uses the Cumulative Distribution Function to allocate the codewords. The cumulative distribution function is defined as
$$F(x)=\sum_{z \leq x} P(z)$$
where $P(z)$ are the probability of occurrence of the $z$. The cumulative distribution function consists of steps of size $P(x)$, as shown in Fig. $1.15$. Let us define a modified cumulative distribution function as $$\bar{F}(x)=\sum_{z<x} P(z)+\frac{1}{2} P(x)$$ symbols less than $x$ plus half the probability of the symbols $x$. The value of the function $\bar{F}(x)$ is the midpoint of the step corresponding to $x$ of the cumulative distribution function.
Fig. 1.15 The cumulative distribution Since probabilities are positive, $F(x) \neq F(y)$ if $x \neq y$. Thus it is function. possible to determine $x$ given $\bar{F}(x)$ merely by looking at the graph of the cumulative distribution function. Therefore, the value of $\bar{F}(x)$ can be used to code $x$.

In general, $\bar{F}(x)$ is a real number. This means we require an infinite number of bits to represent $\bar{F}(x)$, which would lead to an inefficient code. Suppose we round off $\bar{F}(x)$ and use only the first $l(x)$ bits, denoted by $\lfloor\bar{F}(x)\rfloor_{l(x)}$. By the definition of rounding off we have
$$\bar{F}(x)-\lfloor\bar{F}(x)\rfloor_{l(x)} \leq \frac{1}{2^{l(x)}}$$
If $l(x)=\left[\log \frac{1}{P(x)}\right]+1$, then,
$$\frac{1}{2^{l(x)}}<\frac{P(x)}{2}=\bar{F}(x)-F(x-1)$$
This implies that $\lfloor\bar{F}(x)\rfloor_{l(x)}$ lies within the step corresponding to $x$, and $l(x)$ bits are sufficient to describe $x$. The interval corresponding to any codeword is of length $2^{-l(x)}$. From (1.55) we see that this interval is less than half the height of the step corresponding to $x$. Since we use $l(x)=\left[\log \frac{1}{P(x)}\right]+1$ bits to represent $x$, the expected length of this code is
$$\bar{R}=\sum_{x} P(x) l(x)=\sum_{x} P(x)\left(\left[\log \frac{1}{P(x)}\right]+1\right)<H(X)+2$$
Thus the Shannon-Fano-Elias coding scheme achieves codeword length within two bits of the entropy.

## 数学代写|信息论作业代写information theory代考|Arithmetic Coding

As we have seen from Example 1.14, Huffiman codes are only optimal if the probabilities of the symbols are negative powers of two. This is because all prefix codes work at the bit level. There are two ways to look at it.
(a) Prefix codes try to match the self information of the symbols using codewords whose lengths are integers. The length matching may ascribe a codeword either longer than the self information or shorter. The exact match is possible if and only if the self information is in integral number of bits.
(b) If we consider the prefix codes being generated using a binary tree, the decisions between tree branches always take one bit. This is regardless of the fact whether the probabilities for the branches are $0.5 / 0.5$ or $0.9 / 0.1$. In the latter case it would theoretically take only $0.15$ bits

$\left(-\log {2}(0.9)\right)$ to select the first branch and $3.32$ bits $\left(-\log {2}(0.1)\right)$ to select the second branch, making the average code length $0.467$ bits $(0.9 \times 0.15+0.1 \times 3.32)$. The Huffiman code still needs one bit for each decision.

Arithmetic coding does not have this restriction. It works by representing the file to be encoded by an interval of real numbers between 0 and 1. Successive symbols in the message reduce this interval in accordance with the probability of that symbol. The more likely symbols reduce the range by less, and thus add fewer bits to the message.

## 数学代写|信息论作业代写information theory代考|The Lempel-Ziv Algorithm

Huffman coding requires symbol probabilities. But most real life scenarios do not provide the symbol probabilities in advance (i.e., the statistics of the source is unknown). In principle, it is possible to observe the output of the source for a long enough time period and estimate the symbol probabilities. However, this is impractical for real-time application. Also, Huffman coding is optimal for a DMS source where the occurrence of one symbol does not alter the probabilities of the subsequent symbols. Huffman coding is not the best choice for a source with memory. For example, consider the problem of compression of written text. We know that many letters occur in pairs or groups, like ‘ $q$ – $u$ ‘, ‘ $t-h$ ‘, ‘ $i=n-g$ ‘ etc. It might be more efficient to use the statistical inter-dependence of the letters in the alphabet along with their individual probabilities of occurrence. Such a scheme was proposed by Lempel and Ziv in 1977. Their source coding algorithm does not need the source statistics. It is a variable-to-fixed length source coding algorithm and belongs to the class of universal source coding algorithms.

The logic behind Lempel-Ziv universal coding is as follows. The compression of an arbitrary sequence of bits is possible by coding a series of 0 ‘s and l’s as some previous such string (the prefix string) plus one new bit. Then, the new string formed by adding the new bit to the previously used prefix string becomes a potential prefix string for future strings. These variable length blocks are called phrases. The phrases are listed in a dictionary which stores the existing phrases and their locations. In encoding a new phrase, we specify the location of the existing phrase in the dictionary and append the new letter. We can derive a better understanding of how the Lempel-Ziv algorithm works by the following example.

## 数学代写|信息论作业代写information theory代考|Shannon-Fano-Elias Coding

F(X)=∑和≤X磷(和)

F¯(X)=∑和<X磷(和)+12磷(X)符号小于X加上符号概率的一半X. 函数的价值F¯(X)是对应于的步骤的中点X的累积分布函数。

F¯(X)−⌊F¯(X)⌋l(X)≤12l(X)

12l(X)<磷(X)2=F¯(X)−F(X−1)

R¯=∑X磷(X)l(X)=∑X磷(X)([日志⁡1磷(X)]+1)<H(X)+2

## 数学代写|信息论作业代写information theory代考|Arithmetic Coding

(a) 前缀码尝试使用长度为整数的码字匹配符号的自身信息。长度匹配可以归因于比自身信息更长或更短的码字。当且仅当自身信息是整数位时，精确匹配是可能的。
(b) 如果我们考虑使用二叉树生成的前缀码，则树分支之间的决策总是占用一位。这与分支的概率是否为0.5/0.5或者0.9/0.1. 在后一种情况下，理论上只需要0.15位

(−日志⁡2(0.9))选择第一个分支和3.32位(−日志⁡2(0.1))选择第二个分支，使平均代码长度0.467位(0.9×0.15+0.1×3.32). Huffiman 代码仍然需要一个位来进行每个决定。

## 数学代写|信息论作业代写information theory代考|The Lempel-Ziv Algorithm

Lempel-Ziv 通用编码背后的逻辑如下。通过将一系列 0 和 l 编码为一些先前的此类字符串（前缀字符串）加上一个新位，可以压缩任意位序列。然后，通过将新位添加到先前使用的前缀字符串而形成的新字符串成为未来字符串的潜在前缀字符串。这些可变长度的块称为短语。短语列在存储现有短语及其位置的字典中。在编码新短语时，我们指定字典中现有短语的位置并附加新字母。通过以下示例，我们可以更好地理解 Lempel-Ziv 算法的工作原理。

## 有限元方法代写

tatistics-lab作为专业的留学生服务机构，多年来已为美国、英国、加拿大、澳洲等留学热门地的学生提供专业的学术服务，包括但不限于Essay代写，Assignment代写，Dissertation代写，Report代写，小组作业代写，Proposal代写，Paper代写，Presentation代写，计算机作业代写，论文修改和润色，网课代做，exam代考等等。写作范围涵盖高中，本科，研究生等海外留学全阶段，辐射金融，经济学，会计学，审计学，管理学等全球99%专业科目。写作团队既有专业英语母语作者，也有海外名校硕博留学生，每位写作老师都拥有过硬的语言能力，专业的学科背景和学术写作经验。我们承诺100%原创，100%专业，100%准时，100%满意。

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。