Wikipedia, Entziklopedia askea
Informazioaren teorian baldintzazko entropia k zera neurtzen du:
X
{\displaystyle X}
Y
{\displaystyle Y}
informazio kantitatea . Entropia kontzeptuaren hedapen bat da.
X
{\displaystyle X}
zorizko aldagai diskretuak
x
{\displaystyle x}
Y
{\displaystyle Y}
H
(
Y
|
X
=
x
)
{\displaystyle \mathrm {H} (Y|X=x)}
Y
{\displaystyle Y}
probabilitate-funtzioa
p
Y
(
y
)
{\displaystyle p_{Y}{(y)}}
entropia (ez baldintzazkoa) horrela definitzen da:
H
(
Y
)
:=
E
[
I
(
Y
)
]
{\displaystyle \mathrm {H} (Y):=\mathbb {E} [\operatorname {I} (Y)]}
itxaropen matematikoa . Kalkuluak eginez, zera lortzen da:
H
(
Y
)
=
∑
i
=
1
n
P
r
(
Y
=
y
i
)
I
(
y
i
)
=
−
∑
i
=
1
n
p
Y
(
y
i
)
log
2
p
Y
(
y
i
)
,
{\displaystyle \mathrm {H} (Y)=\sum _{i=1}^{n}{\mathrm {Pr} (Y=y_{i})\,\mathrm {I} (y_{i})}=-\sum _{i=1}^{n}{p_{Y}(y_{i})\log _{2}{p_{Y}(y_{i})}},}
I
(
y
i
)
{\displaystyle \operatorname {I} (y_{i})}
Y
{\displaystyle Y}
y
i
{\displaystyle y_{i}}
informazio kantitatea .
Antzeko moduan, baina baldintzazko itxaropen matematikoa erabiliz, defini daiteke
X
{\displaystyle X}
x
{\displaystyle x}
Y
{\displaystyle Y}
H
(
Y
|
X
=
x
)
=
E
[
I
(
Y
)
|
X
=
x
]
=
−
∑
i
=
1
n
Pr
(
Y
=
y
i
|
X
=
x
)
log
2
Pr
(
Y
=
y
i
|
X
=
x
)
.
{\displaystyle \mathrm {H} (Y|X=x)=\mathbb {E} [\operatorname {I} (Y)|X=x]=-\sum _{i=1}^{n}{\Pr(Y=y_{i}|X=x)\log _{2}{\Pr(Y=y_{i}|X=x)}}.}
X
{\displaystyle X}
x
{\displaystyle x}
H
(
Y
|
X
=
x
)
{\displaystyle \mathrm {H} (Y|X=x)}
H
(
Y
|
X
)
{\displaystyle \mathrm {H} (Y|X)}
H
(
Y
|
X
)
≡
∑
x
∈
X
p
(
x
)
H
(
Y
|
X
=
x
)
=
−
∑
x
∈
X
p
(
x
)
∑
y
∈
Y
p
(
y
|
x
)
log
p
(
y
|
x
)
=
−
∑
x
∈
X
∑
y
∈
Y
p
(
x
,
y
)
log
p
(
y
|
x
)
=
−
∑
x
∈
X
,
y
∈
Y
p
(
x
,
y
)
log
p
(
y
|
x
)
=
−
∑
x
∈
X
,
y
∈
Y
p
(
x
,
y
)
log
p
(
x
,
y
)
p
(
x
)
.
{\displaystyle {\begin{aligned}\mathrm {H} (Y|X)\ &\equiv \sum _{x\in {\mathcal {X}}}\,p(x)\,\mathrm {H} (Y|X=x)\\&=-\sum _{x\in {\mathcal {X}}}p(x)\sum _{y\in {\mathcal {Y}}}\,p(y|x)\,\log \,p(y|x)\\&=-\sum _{x\in {\mathcal {X}}}\sum _{y\in {\mathcal {Y}}}\,p(x,y)\,\log \,p(y|x)\\&=-\sum _{x\in {\mathcal {X}},y\in {\mathcal {Y}}}p(x,y)\log \,p(y|x)\\&=-\sum _{x\in {\mathcal {X}},y\in {\mathcal {Y}}}p(x,y)\log {\frac {p(x,y)}{p(x)}}.\\\end{aligned}}}
Konbenioa:
0
log
0
{\displaystyle 0\log 0}
0
log
c
/
0
{\displaystyle 0\log c/0}
c
>
0
{\displaystyle c>0}
Oro har,
H
(
Y
|
X
)
≤
H
(
Y
)
{\displaystyle \mathrm {H} (Y|X)\leq \mathrm {H} (Y)}
X
{\displaystyle X}
Y
{\displaystyle Y}
elkarrekiko independenteak badira,
H
(
Y
|
X
)
=
H
(
Y
)
{\displaystyle \mathrm {H} (Y|X)=\mathrm {H} (Y)}
X
{\displaystyle X}
Y
{\displaystyle Y}
elkarren mendekoak badira,
H
(
Y
|
X
)
=
0
{\displaystyle \mathrm {H} (Y|X)=0}
X
{\displaystyle X}
Y
{\displaystyle Y}
Aurreko propietatetik ondoriozta daiteke
H
(
X
|
X
)
=
0
{\displaystyle H(X|X)=0}
Bayesen teoremaren arabera, zera betetzen da:
H
(
Y
|
X
)
=
H
(
X
|
Y
)
−
H
(
X
)
+
H
(
Y
)
.
{\displaystyle \mathrm {H} (Y|X)\,=\,\mathrm {H} (X|Y)-\mathrm {H} (X)+\mathrm {H} (Y).}
I
(
X
;
Y
)
≤
H
(
X
)
{\displaystyle \operatorname {I} (X;Y)\leq \mathrm {H} (X)}
I
(
X
;
Y
)
{\displaystyle \operatorname {I} (X;Y)}
X
{\displaystyle X}
Y
{\displaystyle Y}
elkarrekiko informazioa .
![{\displaystyle \mathrm {H} (Y):=\mathbb {E} [\operatorname {I} (Y)]}](https://wikimedia.org/api/rest_v1/media/math/render/svg/f114631caeb95e508a74994486e35e972220b378)
![{\displaystyle \mathrm {H} (Y|X=x)=\mathbb {E} [\operatorname {I} (Y)|X=x]=-\sum _{i=1}^{n}{\Pr(Y=y_{i}|X=x)\log _{2}{\Pr(Y=y_{i}|X=x)}}.}](https://wikimedia.org/api/rest_v1/media/math/render/svg/bcac23d77dcc6a01cc367830184893bd13eda9f9)
Datuak: Q813908