### 数学代写|信息论代写information theory代考| Cross-Entropies, Divergences, and Hamming Distance

statistics-lab™ 为您的留学生涯保驾护航 在代写信息论information theory方面已经树立了自己的口碑, 保证靠谱, 高质且原创的统计Statistics代写服务。我们的专家在代写信息论information theory代写方面经验极为丰富，各种代写信息论information theory相关的作业也就用不着说。

• Statistical Inference 统计推断
• Statistical Computing 统计计算
• (Generalized) Linear Models 广义线性模型
• Statistical Machine Learning 统计机器学习
• Longitudinal Data Analysis 纵向数据分析
• Foundations of Data Science 数据科学基础

## 数学代写|信息论代写information theory代考|Cross-Entropies

Given two probability distributions $p=\left(p_{1}, \ldots, p_{n}\right)$ and $q=\left(q_{1}, \ldots, q_{n}\right)$ on the same sample space $U={1, \ldots, n}$, we can again consider the drawing of a pair of points but where the first drawing is according to $p$ and the second drawing according to $q$. The probability that the points are distinct would be a natural and more general notion of logical entropy that would be the:
$$h(p | q)=\sum_{i} p_{i}\left(1-q_{i}\right)=1-\sum_{i} p_{i} q_{i}$$
Logical cross entropy of $p$ and $q$
which is symmetric. Adding subscripts to indicate which probability measures are being used, the value of the product probability measure $\mu_{p q}$ on any $S \subseteq U^{2}$ is $\mu_{p q}(S)=\sum\left{p_{i} q_{i^{\prime}}:\left(i, i^{\prime}\right) \in S\right}$. Thus on the standard information set $S_{i \neq l^{\prime}}=$ $\left{\left(i, i^{\prime}\right) \in U^{2}: i \neq i^{\prime}\right}$, the value is:
$$h(p | q)=\mu_{p q}\left(S_{i \neq i^{\prime}}\right)$$
The logical cross entropy is the same as the logical entropy when the distributions are the same, i.e., if $p=q$, then $h(p | q)=h(p)=\mu_{p}\left(S_{i \neq i^{r}}\right)$.

Although the logical cross entropy formula is symmetrical in $p$ and $q$, there are two different ways to express it as an average in order to apply the dit-bit transform: $\sum_{i} p_{i}\left(1-q_{i}\right)$ and $\sum_{i} q_{i}\left(1-p_{i}\right)$. The two transforms are the two asymmetrical versions of Shannon cross entropy:
$$H(p | q)=\sum_{i} p_{i} \log \left(\frac{1}{q_{i}}\right) \text { and } H(q | p)=\sum_{i} q_{i} \log \left(\frac{1}{p_{i}}\right)$$
which is not symmetrical due to the asymmetric role of the logarithm, although if $p=q$, then $H(p | p)=H(p)$.

## 数学代写|信息论代写information theory代考|Divergences

The Kullback-Leibler (KL) divergence [3] (or relative entropy) $D(p | q)=$ $\sum_{i} p_{i} \log \left(\frac{p_{i}}{q_{i}}\right)$ is called a ‘measure’ of the distance (even though it is not symmetric and does not satisfy the triangle inequality) or divergence between the two distributions where $D(p | q)=H(p | q)-H(p)$. A basic result is the:
$D(p | q) \geq 0$ with equality if and only if $p=q$
Information inequality[1, p. 26].

The KL divergence of a distribution $p$ from the uniform distribution is: $D\left(p |\left(\frac{1}{n}, \ldots, \frac{1}{n}\right)\right)=\log (n)-H(p)$.

But starting afresh, one might ask: “What is the natural notion of distance between two probability distributions $p=\left{p_{1}, \ldots, p_{n}\right}$ and $q=\left{q_{1}, \ldots, q_{n}\right}$ that would always be non-negative, and would be zero if and only if they are equal?” The (Euclidean) distance metric between the two points in $\mathbb{R}^{n}$ would seem to be the logical answer-so we take that distance squared as the definition of the:
$$d(p | q)=\sum_{i}\left(p_{i}-q_{i}\right)^{2}$$
Logical divergence(or logical relative entropy)
which is symmetric and we trivially have:
$$d(p | q) \geq 0 \text { with equality iff } p=q$$
Logical information inequality.
We have component-wise:
$$0 \leq\left(p_{i}-q_{i}\right)^{2}=p_{i}^{2}-2 p_{i} q_{i}+q_{i}^{2}=2\left[\frac{1}{n}-p_{i} q_{i}\right]-\left[\frac{1}{n}-p_{i}^{2}\right]-\left[\frac{1}{n}-q_{i}^{2}\right]$$
so that taking the sum for $i=1, \ldots, n$ gives:
\begin{aligned} d(p | q) &=\sum_{i}\left(p_{i}-q_{i}\right)^{2} \ &=2\left[1-\sum_{i} p_{i} q_{i}\right]-\left[\left(1-\sum_{i} p_{i}^{2}\right)+\left(1-\sum_{i} q_{i}^{2}\right)\right] \ &=2 h(p | q)-[h(p)+h(q)] \end{aligned}

## 数学代写|信息论代写information theory代考|Hamming Distance

A binary relation $R \subseteq U \times U$ on $U=\left{u_{1}, \ldots, u_{n}\right}$ can be represented by an $n \times n$ incidence matrix $\operatorname{In}(R)$ where $$\operatorname{In}(R){i j}=\left{\begin{array}{l} 1 \text { if }\left(u{i}, u_{j}\right) \in R \ 0 \text { if }\left(u_{i}, u_{j}\right) \notin R . \end{array}\right.$$
Taking $R$ as the equivalence relation indit $(\pi)$ associated with a partition $\pi=$ $\left{B_{1}, \ldots, B_{m}\right}$, the density matrix $\rho(\pi)$ of the partition $\pi$ (with equiprobable points) is just the incidence matrix In (indit $(\pi)$ ) rescaled to be of trace 1 (i.e., sum of diagonal entries is 1):
$$\rho(\pi)=\frac{1}{|U|} \operatorname{In}(\text { indit }(\pi)) .$$
From coding theory [4, p. 66], we have the notion of the Hamming distance between two 0,1 vectors or matrices (of the same dimensions) which is the number of places where they differ. The powerset $\wp(U \times U)$ can be viewed as a vector space over $\mathbb{Z}{2}$ where the sum of two binary relations $R, R^{\prime} \subseteq U \times U$, is the symmetric difference (or inequivalence) symbolized $R \Delta R^{\prime}=\left(R-R^{\prime}\right) \cup\left(R^{\prime}-R\right)=$ $R \cup R^{\prime}-R \cap R^{\prime}$, which is the set of elements (i.e., ordered pairs $\left(u{i}, u_{j}\right) \in$ $U \times U)$ that are in one set or the other but not both. Thus the Hamming distance $D_{H}\left(\operatorname{In}(R), \operatorname{In}\left(R^{\prime}\right)\right)$ between the incidence matrices of two binary relations is just the cardinality of their symmetric difference: $D_{H}\left(\operatorname{In}(R), \operatorname{In}\left(R^{\prime}\right)\right)=\left|R \Delta R^{\prime}\right|$. Moreover, the size of the symmetric difference does not change if the binary relations are replaced by their complements: $\left|R \Delta R^{\prime}\right|=\left|\left(U^{2}-R\right) \Delta\left(U^{2}-R^{\prime}\right)\right|$.
Hence given two partitions $\pi=\left{B_{1}, \ldots, B_{m}\right}$ and $\sigma=\left{C_{1}, \ldots, C_{m^{\prime}}\right}$ on $U$, the unnormalized Hamming distance between the two partitions is naturally defined as:
\begin{aligned} D_{H}(\pi, \sigma) &\left.\left.=D_{H}(\operatorname{In} \text { (indit }(\pi)), \operatorname{In} \text { (indit }(\sigma)\right)\right)=\mid \text { indit }(\pi) \Delta \text { indit }(\sigma) \mid \ &=|\operatorname{dit}(\pi) \Delta \operatorname{dit}(\sigma)|, \end{aligned}
and the Hamming distance between $\pi$ and $\sigma$ is defined as the normalized $D_{H}(\pi, \sigma)$ :
\begin{aligned} \frac{D_{H}(\pi, \sigma)}{|U \times U|} &=\frac{|\operatorname{dit}(\pi) \Delta \operatorname{dit}(\sigma)|}{|U \times U|}=\frac{|\operatorname{dit}(\pi)-\operatorname{dit}(\sigma)|}{|U \times U|}+\frac{|\operatorname{dit}(\sigma)-\operatorname{dit}(\pi)|}{|U \times U|} \ &=h(\pi \mid \sigma)+h(\sigma \mid \pi)=2 h(\pi \vee \sigma)-h(\pi)-h(\sigma) \end{aligned}

## 数学代写|信息论代写information theory代考|Cross-Entropies

H(p|q)=∑一世p一世(1−q一世)=1−∑一世p一世q一世

H(p|q)=μpq(小号一世≠一世′)

H(p|q)=∑一世p一世日志⁡(1q一世) 和 H(q|p)=∑一世q一世日志⁡(1p一世)

## 数学代写|信息论代写information theory代考|Divergences

Kullback-Leibler (KL) 散度 [3]（或相对熵）D(p|q)= ∑一世p一世日志⁡(p一世q一世)被称为距离的“度量”（即使它不是对称的并且不满足三角不等式）或两个分布之间的散度，其中D(p|q)=H(p|q)−H(p). 一个基本的结果是：
D(p|q)≥0当且仅当p=q

d(p|q)=∑一世(p一世−q一世)2

，我们通常有：

d(p|q)≥0 有平等 iff p=q

0≤(p一世−q一世)2=p一世2−2p一世q一世+q一世2=2[1n−p一世q一世]−[1n−p一世2]−[1n−q一世2]

d(p|q)=∑一世(p一世−q一世)2 =2[1−∑一世p一世q一世]−[(1−∑一世p一世2)+(1−∑一世q一世2)] =2H(p|q)−[H(p)+H(q)]

## 有限元方法代写

tatistics-lab作为专业的留学生服务机构，多年来已为美国、英国、加拿大、澳洲等留学热门地的学生提供专业的学术服务，包括但不限于Essay代写，Assignment代写，Dissertation代写，Report代写，小组作业代写，Proposal代写，Paper代写，Presentation代写，计算机作业代写，论文修改和润色，网课代做，exam代考等等。写作范围涵盖高中，本科，研究生等海外留学全阶段，辐射金融，经济学，会计学，审计学，管理学等全球99%专业科目。写作团队既有专业英语母语作者，也有海外名校硕博留学生，每位写作老师都拥有过硬的语言能力，专业的学科背景和学术写作经验。我们承诺100%原创，100%专业，100%准时，100%满意。

## MATLAB代写

MATLAB 是一种用于技术计算的高性能语言。它将计算、可视化和编程集成在一个易于使用的环境中，其中问题和解决方案以熟悉的数学符号表示。典型用途包括：数学和计算算法开发建模、仿真和原型制作数据分析、探索和可视化科学和工程图形应用程序开发，包括图形用户界面构建MATLAB 是一个交互式系统，其基本数据元素是一个不需要维度的数组。这使您可以解决许多技术计算问题，尤其是那些具有矩阵和向量公式的问题，而只需用 C 或 Fortran 等标量非交互式语言编写程序所需的时间的一小部分。MATLAB 名称代表矩阵实验室。MATLAB 最初的编写目的是提供对由 LINPACK 和 EISPACK 项目开发的矩阵软件的轻松访问，这两个项目共同代表了矩阵计算软件的最新技术。MATLAB 经过多年的发展，得到了许多用户的投入。在大学环境中，它是数学、工程和科学入门和高级课程的标准教学工具。在工业领域，MATLAB 是高效研究、开发和分析的首选工具。MATLAB 具有一系列称为工具箱的特定于应用程序的解决方案。对于大多数 MATLAB 用户来说非常重要，工具箱允许您学习应用专业技术。工具箱是 MATLAB 函数（M 文件）的综合集合，可扩展 MATLAB 环境以解决特定类别的问题。可用工具箱的领域包括信号处理、控制系统、神经网络、模糊逻辑、小波、仿真等。