Margin-based Ranking and an Equivalence between AdaBoost and RankBoost

We study boosting algorithms for learning to rank. We give a general margin-based bound for ranking based on covering numbers for the hypothesis space. Our bound suggests that algorithms that maximize the ranking margin will generalize well. We then describe a new algorithm, smooth margin ranking, that precisely converges to a maximum ranking-margin solution. The algorithm is a modification of RankBoost, analogous to “approximate coordinate ascent boosting.” Finally, we prove that AdaBoost and RankBoost are equally good for the problems of bipartite ranking and classification in terms of their asymptotic behavior on the training set. Under natural conditions, AdaBoost achieves an area under the ROC curve that is equally as good as RankBoost’s; furthermore, RankBoost, when given a specific intercept, achieves a misclassification error that is as good as AdaBoost’s. This may help to explain the empirical observations made by Cortes andMohri, and Caruana and Niculescu-Mizil, about the excellent performance of AdaBoost as a bipartite ranking algorithm, as measured by the area under the ROC curve

The Rate of Convergence of AdaBoost

The AdaBoost algorithm was designed to combine many "weak" hypotheses that perform slightly better than random guessing into a "strong" hypothesis that has very low error. We study the rate at which AdaBoost iteratively converges to the minimum of the "exponential loss." Unlike previous work, our proofs do not require a weak-learning assumption, nor do they require that minimizers of the exponential loss are finite. Our first result shows that at iteration $t$, the exponential loss of AdaBoost's computed parameter vector will be at most $\epsilon$ more than that of any parameter vector of $\ell_1$-norm bounded by $B$ in a number of rounds that is at most a polynomial in $B$ and $1/\epsilon$. We also provide lower bounds showing that a polynomial dependence on these parameters is necessary. Our second result is that within $C/\epsilon$ iterations, AdaBoost achieves a value of the exponential loss that is at most $\epsilon$ more than the best possible value, where $C$ depends on the dataset. We show that this dependence of the rate on $\epsilon$ is optimal up to constant factors, i.e., at least $\Omega(1/\epsilon)$ rounds are necessary to achieve within $\epsilon$ of the optimal exponential loss.Comment: A preliminary version will appear in COLT 201

Adjointness Relations as a Criterion for Choosing an Inner Product

This is a contribution to the forthcoming book "Canonical Gravity: {}From Classical to Quantum" edited by J. Ehlers and H. Friedrich. Ashtekar's criterion for choosing an inner product in the quantisation of constrained systems is discussed. An erroneous claim in a previous paper is corrected and a cautionary example is presented.Comment: 6 pages, MPA-AR-94-

Lab Retriever: a software tool for calculating likelihood ratios incorporating a probability of drop-out for forensic DNA profiles.

BackgroundTechnological advances have enabled the analysis of very small amounts of DNA in forensic cases. However, the DNA profiles from such evidence are frequently incomplete and can contain contributions from multiple individuals. The complexity of such samples confounds the assessment of the statistical weight of such evidence. One approach to account for this uncertainty is to use a likelihood ratio framework to compare the probability of the evidence profile under different scenarios. While researchers favor the likelihood ratio framework, few open-source software solutions with a graphical user interface implementing these calculations are available for practicing forensic scientists.ResultsTo address this need, we developed Lab Retriever, an open-source, freely available program that forensic scientists can use to calculate likelihood ratios for complex DNA profiles. Lab Retriever adds a graphical user interface, written primarily in JavaScript, on top of a C++ implementation of the previously published R code of Balding. We redesigned parts of the original Balding algorithm to improve computational speed. In addition to incorporating a probability of allelic drop-out and other critical parameters, Lab Retriever computes likelihood ratios for hypotheses that can include up to four unknown contributors to a mixed sample. These computations are completed nearly instantaneously on a modern PC or Mac computer.ConclusionsLab Retriever provides a practical software solution to forensic scientists who wish to assess the statistical weight of evidence for complex DNA profiles. Executable versions of the program are freely available for Mac OSX and Windows operating systems

The Rate of Convergence of AdaBoost

The AdaBoost algorithm was designed to combine many “weak” hypotheses that perform slightly better than random guessing into a “strong” hypothesis that has very low error. We study the rate at which AdaBoost iteratively converges to the minimum of the “exponential loss”. Unlike previous work, our proofs do not require a weak-learning assumption, nor do they require that minimizers of the exponential loss are finite. Our first result shows that the exponential loss of AdaBoost's computed parameter vector will be at most ε more than that of any parameter vector of ℓ[subscript 1]-norm bounded by B in a number of rounds that is at most a polynomial in B and 1/ε. We also provide lower bounds showing that a polynomial dependence is necessary. Our second result is that within C/ε iterations, AdaBoost achieves a value of the exponential loss that is at most ε more than the best possible value, where C depends on the data set. We show that this dependence of the rate on ε is optimal up to constant factors, that is, at least Ω(1/ε) rounds are necessary to achieve within ε of the optimal exponential loss.National Science Foundation (U.S.) (Grant IIS-1016029)National Science Foundation (U.S.) (Grant IIS-1053407

Characterization of the Sequential Product on Quantum Effects

We present a characterization of the standard sequential product of quantum effects. The characterization is in term of algebraic, continuity and duality conditions that can be physically motivated.Comment: 11 pages. Accepted for publication in the Journal of Mathematical Physic

Power of unentangled measurements on two antiparallel spins

We consider a pair of antiparallel spins polarized in a random direction to encode quantum information. We wish to extract as much information as possible on the polarization direction attainable by an unentangled measurement, i.e., by a measurement, whose outcomes are associated with product states. We develop analytically the upper bound 0.7935 bits to the Shannon mutual information obtainable by an unentangled measurement, which is definitely less than the value 0.8664 bits attained by an entangled measurement. This proves our main result, that not every ensemble of product states can be optimally distinguished by an unentangled measurement, if the measure of distinguishability is defined in the sense of Shannon. We also present results from numerical calculations and discuss briefly the case of parallel spins.Comment: Latex file, 18 pages, 1 figure; published versio

Density of states of a two-dimensional electron gas in a non-quantizing magnetic field

We study local density of electron states of a two-dimentional conductor with a smooth disorder potential in a non-quantizing magnetic field, which does not cause the standart de Haas-van Alphen oscillations. It is found, that despite the influence of such ``classical'' magnetic field on the average electron density of states (DOS) is negligibly small, it does produce a significant effect on the DOS correlations. The corresponding correlation function exhibits oscillations with the characteristic period of cyclotron quantum $\hbar\omega_c$.Comment: 7 pages, including 3 figure