Back

My first preprint is online: https://arxiv.org/abs/2402.04613 :)

We define and analyse Maximum Mean Discrepancy (MMD) regularized $f$-divergences $D_{f,\nu }$ and their #Wasserstein gradient flows.

We define the $\lambda$-regularized $f$-divergence for $\lambda >0$ as
$$D_{f,\nu }^{\lambda }(\mu ):=\underset{\sigma \in M_{+}({\mathbb{R}}^d)}{min}{D}_{f,\nu }(\sigma )+\frac{1}{2\lambda }{d}_K(\mu ,\sigma {)}^2,$$
(yes, the min is attained!) where $d_K$ is the kernel metric
$$d_K(\mu ,\nu ):=\parallel m_{\mu -\nu }{\parallel }_{{\mathcal{H}}_K},$$
where $({\mathcal{H}}_K,\parallel \cdot {\parallel }_{{\mathcal{H}}_K})$ is the Reproducing Kernel Hilbert Space for the kernel $K:{\mathbb{R}}^d\times {\mathbb{R}}^d\to \mathbb{R}$ and
$$m:M({\mathbb{R}}^d)\to {\mathcal{H}}_K,\mu \mapsto {\int }_{{\mathbb{R}}^d}K(x,\cdot )\text{d}\mu (x)$$
is the kernel mean embedding (KME) of finite signed measures in the RKHS.
One can image the KME to be the generalization of the #KernelTrick from points in ${\mathbb{R}}^d$ to measures on ${\mathbb{R}}^d$.

We then show that for any $\nu \in M_{+}({\mathbb{R}}^d)$ there exists a proper convex lower semicontinuous functional $G_{f,\nu }:{\mathcal{H}}_K\to (-\mathrm{\infty },\mathrm{\infty }]$ such that $$D_{f,\nu }^{\lambda }=G_{f,\nu }^{\lambda }\circ m,$$ where $F^{\lambda }$ denotes the normal Hilbert space #MoreauEnvelope of $F$.

We can now use standard Convex Analysis in Hilbert spaces to calculate the ($\frac{1}{\lambda }$-Lipschitz-continuous) gradient of $D_{f,\nu }^{\lambda }$ and find the limits for $\lambda \to \{0,\mathrm{\infty }\}$ (pointwise and in the sense of Mosco), showing that $D_{f,\nu }^{\lambda }$ interpolates between $D_{f,\nu }$ and $d_K(\cdot, \nu)^2\).