### 数学代写|信息论作业代写information theory代考|Channel Capacity and Coding

statistics-lab™ 为您的留学生涯保驾护航 在代写信息论information theory方面已经树立了自己的口碑, 保证靠谱, 高质且原创的统计Statistics代写服务。我们的专家在代写信息论information theory代写方面经验极为丰富，各种代写信息论information theory相关的作业也就用不着说。

• Statistical Inference 统计推断
• Statistical Computing 统计计算
• Advanced Probability Theory 高等概率论
• Advanced Mathematical Statistics 高等数理统计学
• (Generalized) Linear Models 广义线性模型
• Statistical Machine Learning 统计机器学习
• Longitudinal Data Analysis 纵向数据分析
• Foundations of Data Science 数据科学基础

## 数学代写|信息论作业代写information theory代考|Channel Capacity

Consider a DMC having an input alphabet $X=\left{x_{0}, x_{1}, \ldots, x_{q-1}\right}$ and an output alphabet $Y=\left{y_{0}, y_{1}, \ldots, y_{r-1}\right}$. Let us denote the set of channel transition probabilities by $P\left(y_{i} \mid x_{j}\right)$. The average mutual information provided by the output $Y$ about the input $X$ is given by (see Chapter 1, Section 1.3)
$$I(X ; Y)=\sum_{j=0}^{q-1} \sum_{j=0}^{r-1} P\left(x_{j}\right) P\left(y_{i} \mid x_{j}\right) \log \frac{P\left(y_{i} \mid x_{j}\right)}{P\left(y_{i}\right)}$$
The channel transition probabilities $P\left(y_{i} \mid x_{j}\right)$ are determined by the channel characteristics (primarily the noise in the channel). However, the input symbol probabilities $P\left(x_{j}\right)$ are within the control of the discrete channel encoder. The value of the average mutual information, $I(X ; Y)$, maximized over the set of input symbol probabilities $P\left(x_{j}\right)$ is a quantity that depends only on the channel transition probabilities $P\left(y_{i} \mid x_{j}\right)$ (hence only on the characteristics of the channel). This quantity is called the Capacity of the Channel.

## 数学代写|信息论作业代写information theory代考|Channel Coding

reliability (which is a component of the quality of service). Table $2.1$ lists the typical acceptable bit error rates for various applications.

In order to achieve such high levels of reliability we have to resort to the use of Channel Coding. The basic objective of channel coding is to increase the resistance of the digital communication system to channel noise. This is done by adding redundancies in the transmitted data stream in a controlled manner.

In channel coding, we map the incoming data sequence to a channel input sequence. This encoding procedure is done by the Channel Encoder. The encoded sequence is then transmitted over the noisy channel. The channel output sequence at the receiver is inverse mapped to an output data sequence. This is called the decoding procedure, and is carried out by the Channel Decoder. Both the encoder and the decoder are under the designer’s control.

As already mentioned, the encoder introduces redundancy in a prescribed manner. The decoder exploits this redundancy so as to reconstruct the original source sequence as accurately as possible. Thus, channel coding makes it possible to carry out reliable communication over unreliable (noisy) channels. Channel coding is also referred to as Error Control Coding. It is interesting to note here that the source coder reduces redundancy to improve efficiency, whereas, the channel coder adds redundancy in a controlled manner to improve reliability.

Definition 2.6 An Error Control Code for a channel, represented by the channel transition probability matrix $p(y \mid x)$, consists of:
(i) A message set ${1,2, \ldots, M}$.
(ii) An encoding function, $X^{n}$, which maps each message to a unique codeword, i.e., $1 \rightarrow X^{n}(1)$, $2 \rightarrow X^{n}(2), \ldots, M \rightarrow X^{n}(M)$. The set of codewords is called a codebook.
(iii) A decoding function, $D \rightarrow{1,2, \ldots, M}$, which makes a guess based on a decoding strategy in order to map back the received vector to one of the possible messages.

We first look at a class of channel codes called block codes. In this class of codes, the incoming message sequence is first sub-divided into sequential blocks, each of length $k$ bits. Thus the cardinality of the message set $M=2^{k}$. Each $k$-bit long information block is mapped into an $n$-bit block by the channel coder, where $n>k$. This means that for every $k$ bits of information, $(n-k)$ redundant bits are added. The ratio
$$r=\frac{k}{n}=\frac{\log M}{n}$$
is called the Code Rate. It represents the information bits per transmission. Code rate of any coding scheme is, naturally, less than unity. A small code rate implies that more and more bits per block are the redundant bits corresponding to a higher coding overhead. This may reduce the effect of noise, but will also reduce the communication rate as we will end up transmitting more of the redundant bits and fewer information bits. The question before us is whether there exists a coding scheme such that the probability that the message bit will be in error is arbitrarily small and yet the coding rate is not too small? The answer is yes, and was first provided by Shannon in his second theorem regarding the channel capacity. We will study this shortly.

## 数学代写|信息论作业代写information theory代考|Information Capacity Theorem

So far we have studied limits on the maximum rate at which information can be sent over a channel reliably in terms of the channel capacity. In this section we will formulate the information capacity theorem for band-limited, power-limited Gaussian channels. An important and useful channel is the Gaussian channel defined below.

Consider a zero mean, stationary random process $X(t)$ that is band limited to $W$ hertz. Let $X_{k}, k=1,2, \ldots, K$, denote the continuous random variables obtained by uniform sampling of the process $X(t)$ at the Nyquist rate of $2 W$ samples per second. These symbols are transmitted over a noisy channel which is also band-limited to $W$ Hertz. The channel output is corrupted by AWGN of zero mean and power spectral density (psd) $N_{0} / 2$. Because of the channel, the noise is band limited to $W$ Hertz. Let $Y_{k}, k=1,2, \ldots, K$, denote the samples of the received signal. Therefore,
$$Y_{k}=X_{k}+N_{k}, k=1,2, \ldots, K$$
where $N_{k}$ is the noise sample with zero mean and variance $\sigma^{2}=N_{0}$ W. It is assumed that $Y_{k}, k=1,2, \ldots, K$, are statistically independent. Since the transmitter is usually power-limited, let us put a constraint on the $a v e r a g e$ power in $X_{k}$ :
$$E\left[X_{k}^{2}\right]=P, k=1,2, \ldots, K$$
The information capacity of this band-limited, power-limited channel is the maximum of the mutual information between the channel input $X_{k}$ and the channel output $Y_{k}$. The maximization has to be done over all distributions on the input $X_{k}$ that satisfy the power constraint of equation (2.19). Thus, the information capacity of the channel (same as the channel capacity) is given by
$$C=\max {f{X_{k}}(x)}\left{I(X ; Y) \mid E\left[X_{k}^{2}\right]=P\right}$$
where $f_{x_{k}}(x)$ is the probability density function of $X_{k}$.
Now, from Chapter 1, equation (1.32), we have,
$$I\left(X_{k} ; Y_{k}\right)=h\left(Y_{k}\right)-h\left(Y_{k} \mid X_{k}\right)$$
Note that $X_{k}$ and $N_{k}$ are independent random variables. Therefore, the conditional differential entropy of $Y_{k}$. given $X_{k}$ is equal to the differential entropy of $N_{k^{*}}$. Intuitively, this is because given $X_{k}$ the uncertainty arising in $Y_{k}$ is purely due to $N_{k}$. That is,
$$h\left(Y_{k} \mid X_{k}\right)=h\left(N_{k}\right)$$

## 数学代写|信息论作业代写information theory代考|Channel Coding

(i) 一个消息集1,2,…,米.
(ii) 编码功能，Xn，它将每条消息映射到一个唯一的代码字，即1→Xn(1), 2→Xn(2),…,米→Xn(米). 该组码字称为码本。
(iii) 解码功能，D→1,2,…,米，它根据解码策略进行猜测，以便将接收到的向量映射回可能的消息之一。

r=ķn=日志⁡米n

## 数学代写|信息论作业代写information theory代考|Information Capacity Theorem

$$C=\max {f {X_{k}}(x)}\left{I(X ; Y) \mid E\left 给出[X_{k}^{2}\right]=P\right} 在H和r和FXķ(X)一世s吨H和pr○b一个b一世l一世吨是d和ns一世吨是F在nC吨一世○n○FXķ.ñ○在,Fr○米CH一个p吨和r1,和q在一个吨一世○n(1.32),在和H一个在和, I\left(X_{k} ; Y_{k}\right)=h\left(Y_{k}\right)-h\left(Y_{k} \mid X_{k}\right) ñ○吨和吨H一个吨Xķ一个ndñķ一个r和一世nd和p和nd和n吨r一个nd○米在一个r一世一个bl和s.吨H和r和F○r和,吨H和C○nd一世吨一世○n一个ld一世FF和r和n吨一世一个l和n吨r○p是○F是ķ.G一世在和nXķ一世s和q在一个l吨○吨H和d一世FF和r和n吨一世一个l和n吨r○p是○Fñķ∗.一世n吨在一世吨一世在和l是,吨H一世s一世sb和C一个在s和G一世在和nXķ吨H和在nC和r吨一个一世n吨是一个r一世s一世nG一世n是ķ一世sp在r和l是d在和吨○ñķ.吨H一个吨一世s, h\left(Y_{k}\mid X_{k}\right)=h\left(N_{k}\right)$$

## 有限元方法代写

tatistics-lab作为专业的留学生服务机构，多年来已为美国、英国、加拿大、澳洲等留学热门地的学生提供专业的学术服务，包括但不限于Essay代写，Assignment代写，Dissertation代写，Report代写，小组作业代写，Proposal代写，Paper代写，Presentation代写，计算机作业代写，论文修改和润色，网课代做，exam代考等等。写作范围涵盖高中，本科，研究生等海外留学全阶段，辐射金融，经济学，会计学，审计学，管理学等全球99%专业科目。写作团队既有专业英语母语作者，也有海外名校硕博留学生，每位写作老师都拥有过硬的语言能力，专业的学科背景和学术写作经验。我们承诺100%原创，100%专业，100%准时，100%满意。

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。