Information Theory (cs.IT)

Abstract: We establish that constructive continued fraction dimension originally defined using $s$-gales is robust, but surprisingly, that the effective continued fraction dimension and effective (base-$b$) Hausdorff dimension of the same real can be unequal in general. We initially provide an equivalent characterization of continued fraction dimension using Kolmogorov complexity. In the process, we construct an optimal lower semi-computable $s$-gale for continued fractions. We also prove new bounds on the Lebesgue measure of continued fraction cylinders, which may be of independent interest. We apply these bounds to reveal an unexpected behavior of continued fraction dimension. It is known that feasible dimension is invariant with respect to base conversion. We also know that Martin-L\"of randomness and computable randomness are invariant not only with respect to base conversion, but also with respect to the continued fraction representation. In contrast, for any $0 < \varepsilon < 0.5$, we prove the existence of a real whose effective Hausdorff dimension is less than $\varepsilon$, but whose effective continued fraction dimension is greater than or equal to $0.5$. This phenomenon is related to the ``non-faithfulness'' of certain families of covers, investigated by Peres and Torbin and by Albeverio, Ivanenko, Lebid and Torbin. We also establish that for any real, the constructive Hausdorff dimension is at most its effective continued fraction dimension.

Abstract: We analyze connections between two low rank modeling approaches from the last decade for treating dynamical data. The first one is the coherence problem (or coherent set approach), where groups of states are sought that evolve under the action of a stochastic matrix in a way maximally distinguishable from other groups. The second one is a low rank factorization approach for stochastic matrices, called Direct Bayesian Model Reduction (DBMR), which estimates the low rank factors directly from observed data. We show that DBMR results in a low rank model that is a projection of the full model, and exploit this insight to infer bounds on a quantitative measure of coherence within the reduced model. Both approaches can be formulated as optimization problems, and we also prove a bound between their respective objectives. On a broader scope, this work relates the two classical loss functions of nonnegative matrix factorization, namely the Frobenius norm and the generalized Kullback--Leibler divergence, and suggests new links between likelihood-based and projection-based estimation of probabilistic models.

Abstract: One of the most employed yet simple algorithm for cluster analysis is the k-means algorithm. k-means has successfully witnessed its use in artificial intelligence, market segmentation, fraud detection, data mining, psychology, etc., only to name a few. The k-means algorithm, however, does not always yield the best quality results. Its performance heavily depends upon the number of clusters supplied and the proper initialization of the cluster centroids or seeds. In this paper, we conduct an analysis of the performance of k-means on image data by employing parametric entropies in an entropy based centroid initialization method and propose the best fitting entropy measures for general image datasets. We use several entropies like Taneja entropy, Kapur entropy, Aczel Daroczy entropy, Sharma Mittal entropy. We observe that for different datasets, different entropies provide better results than the conventional methods. We have applied our proposed algorithm on these datasets: Satellite, Toys, Fruits, Cars, Brain MRI, Covid X-Ray.

Abstract: Encoding data as a set of unordered strings is receiving great attention as it captures one of the basic features of DNA storage systems. However, the challenge of constructing optimal redundancy codes for this channel remained elusive. In this paper, we address this problem and present an order-wise optimal construction of codes that are capable of correcting multiple substitution, deletion, and insertion errors for this channel model. The key ingredient in the code construction is a technique we call robust indexing: simultaneously assigning indices to unordered strings (hence, creating order) and also embedding information in these indices. The encoded indices are resilient to substitution, deletion, and insertion errors, and therefore, so is the entire code.

Abstract: We study the structure of the optimal sampling policy to minimize the average age of information when the channel state (i.e., busy or idle) is not immediately perceived by the transmitter upon the delivery of a sample due to random delays in the feedback (ACK) channel. In this setting, we show that it is not always optimal to wait for ACKs before sampling, and thus, early sampling before the arrival of an ACK may be optimal. We show that, under certain conditions on the distribution of the ACK delays, the optimal policy is a mixture of two threshold policies.