Minimum Divergence Methods in Statistical Machine Learning
From an Information Geometric Viewpoint
Write a Review
Minimum Divergence Methods in Statistical Machine Learning
Delivery estimate
Please enter a valid 4-digit Australian postcode.
to ·
Delivery estimate
Please enter a valid 4-digit Australian postcode.
to ·
Minimum Divergence Methods in Statistical Machine Learning
Description
This book explores minimum divergence methods of statistical machine learning for estimation, regression, prediction, and so forth, in which we engage in information geometry to elucidate their intrinsic properties of the corresponding loss functions, learning algorithms, and statistical models. One of the most elementary examples is Gauss's least squares estimator in a linear regression model, in which the estimator is given by minimization of the sum of squares between a response vector and a vector of the linear subspace hulled by explanatory vectors. This is extended to Fisher's maximum likelihood estimator (MLE) for an exponential model, in which the estimator is provided by minimization of the Kullback-Leibler (KL) divergence between a data distribution and a parametric distribution of the exponential model in an empirical analogue. Thus, we envisage a geometric interpretation of such minimization procedures such that a right triangle is kept with Pythagorean identity in the sense of the KL divergence. This understanding sublimates a dualistic interplay between a statistical estimation and model, which requires dual geodesic paths, called m-geodesic and e-geodesic paths, in a framework of information geometry.
We extend such a dualistic structure of the MLE and exponential model to that of the minimum divergence estimator and the maximum entropy model, which is applied to robust statistics, maximum entropy, density estimation, principal component analysis, independent component analysis, regression analysis, manifold learning, boosting algorithm, clustering, dynamic treatment regimes, and so forth. We consider a variety of information divergence measures typically including KL divergence to express departure from one probability distribution to another. An information divergence is decomposed into the cross-entropy and the (diagonal) entropy in which the entropy associates with a generative model as a family of maximum entropy distributions; the cross entropy associates with a statistical estimation method via minimization of the empirical analogue based on given data. Thus any statistical divergence includes an intrinsic object between the generative model and the estimation method. Typically, KL divergence leads to the exponential model and the maximum likelihood estimation. It is shown that any information divergence leads to a Riemannian metric and a pair of the linear connections in the framework of information geometry.
We focus on a class of information divergence generated by an increasing and convex function U, called U-divergence. It is shown that any generator function U generates the U-entropy and U-divergence, in which there is a dualistic structure between the U-divergence method and the maximum U-entropy model. We observe that a specific choice of U leads to a robust statistical procedure via the minimum U-divergence method. If U is selected as an exponential function, then the corresponding U-entropy and U-divergence are reduced to the Boltzmann-Shanon entropy and the KL divergence; the minimum U-divergence estimator is equivalent to the MLE. For robust supervised learning to predict a class label we observe that the U-boosting algorithm performs well for contamination of mislabel examples if U is appropriately selected. We present such maximal U-entropy and minimum U-divergence methods, in particular, selecting a power function as U to provide flexible performance in statistical machine learning.
Shipping & Processing
This item is in stock with our supplier and usually ships within 3–5 business days.
Processing times
- In stock: Items marked “In Stock” are on our shelves. We pick, pack and ship them within 1–2 business days.
- In stock with our supplier: Items not marked “In Stock” are held by one of our Australian or overseas suppliers. We order them in for you and they usually dispatch within 2–5 business days (3–5 for some suppliers). The estimate next to the price is for this item.
- Special orders: Some supplier items, such as specialist and imported items, titles awaiting reprint and print-on-demand titles, take longer to arrive. If your order includes one, we’ll update you as soon as we hear from our supplier.
- Supplier delays: Occasionally a supplier is delayed, or their stock information turns out to be inaccurate. We’ll keep your order open for up to 60 days. If an item hasn’t shipped or isn’t on its way to us by then, we’ll cancel that item and refund it.
How accurate is availability?
Availability is based on the stock levels our suppliers tell us they hold, and we update it regularly. If a supplier issue affects your order, we’ll let you know with an updated estimate.
Delivery
- As soon as your order ships you’ll get an email with tracking.
- Australia Post usually delivers within 2–10 business days of shipping (around 4 in metro areas and around 6 in regional areas).
- If your order has more than one item, we hold the in-stock items until everything is ready, so we can ship your whole order together in one parcel.
- Shipping cost: Free on orders over $100 within Australia, otherwise $9.95 flat rate per order however many items you buy. Free shipping applies to Australian addresses only; delivery to New Zealand is $14.95 per order.
- Check your address: If an order goes to an incorrect address you gave us, or isn’t collected and Australia Post returns it to us, you’ll need to pay the postage to send it again.
Cancellations
Need to change or cancel an order? Contact us as soon as possible at contact@bookloop.com.au or through our contact page.
- Within 30 minutes: You can change or cancel your order for a full refund within 30 minutes of placing it.
- After 30 minutes: Your order is already being processed, so it can no longer be changed or cancelled. If we can accept a late cancellation, we may offer store credit instead of a refund.
- Items in transit: Once an item is on its way, either from our supplier to us or from us to you, it can’t be cancelled. If you no longer want it once it arrives, you can return it as a change of mind, with return postage at your cost. See Returns.
- Separate orders can’t be combined once they’ve been placed, and adding or swapping items can change the delivery estimate.
- Supplier delays: Occasionally a supplier is delayed, or their stock information turns out to be inaccurate. We’ll keep your order open for up to 60 days. If an item hasn’t shipped or isn’t on its way to us by then, we’ll cancel that item and refund it.
Faulty or damaged items are always covered. See Returns.
Returns
Damaged or incorrect items (within 30 days)
If an item arrives damaged or isn’t what you ordered, contact us within 30 days of delivery. We may ask for a photo, and if we need the item back we’ll send a prepaid return label, so it costs you nothing. You’ll receive a replacement or refund, as guaranteed under the Australian Consumer Law.
Change of mind (within 30 days)
We offer a refund or store credit for change of mind within 30 days of purchase when:
- you have your proof of purchase;
- the item is unused and in its original condition and packaging;
- it isn’t a sale item, special order, custom item or gift voucher.
You pay the postage to return change-of-mind items. Change-of-mind returns aren’t available for print-on-demand titles or bulk orders with special pricing.
Returned to sender
If an order is sent to an incorrect address you gave us, or isn’t collected and Australia Post returns it to us, you’ll need to pay the postage to send it again.