ISIPTA final day 5

Bayesian robustness, by Fabrizio Ruggeri

Basics: To do Bayesian analysis of some data, you need (a) to select a parametric Model, (b) some Priors and (c) a Loss function to optimize over.

To test the model goodness after the analysis: visual fits, tests like Khi 2 or KS are done even if not casher. The real Baysian thing is to look at Bayes factors BF = f1(x) / f2(x), which can be used to select the model that maximizes the probability of the observed data. Posterior odds are also Baysian, but not as popular as BF, they are too sensitive to the priors.

Personally, I would rather use frequencies than improper priors.

Giron and Rios (1980) paper is famous:
If the preferences satisfy some axioms, then
There is a set of prior probabilities such that
b is preferred to a iff the expected value of the loss with a is >= ev loss with b FOR ANY probability in the set

Historical notes and references:
Bayesian robustness was booming in early 90's and busting by mid 90. The new new thing now for Bayesians is Markov Chain Monte Carlo. What remains is that people are aware of the need for multiple priors and sensitivity analysis to hyperparameters. What is missing is COTS software.
- Review paper in 1994 by Jim Berger in the Spanish Journal "Tests", with discussion.
- Proceedings of the two conferences in Spain (Valencia ?), one published in JSDI, the other in the Institute for Mathematical Statistics lecture note.
- Kadane's book.
- The handbook "Robust bayesian analysis" by Berger, Shyamalkumar and myself was perhaps the swan's song.

Glenn Shafer: What is risk? What is probability? Game-theoretic answers.

Objective vs. subjective goes on for 171 years (Simon-Denis Poisson 1837).

Our more concrete question: is there a repetitive structure for the question and the data ? By that, we mean online prediction (information feedback before you make the next prediction). Not necessarily iid or physical aleatory.
Yes => we can make good probability forecasts
No => we must weight evidence

GS 76: repetitive structure weak
GS 96: repetitive structure strong
S/Vovk 01: unifying with Game Theory

The foundations are not measure theory or Kolmogorov axioms. We start with games.

Cournot's citation "A physically impossible event is one whose probability is infinitely small. This remark alone gives substance -an objective and phenomenological value- to the mathematical theory of probability" Examples: Balancing a cone on its head. Being hit on the head by a rooftop tile. Another way to say "Small probabilities don't happen to me." is the way to connect mathematical theory with real world.

Our fundamental principle swiches that to the efficient market hypothesis: you won't multiply your capital by a large factor if you bet with the probabilities (no line of credit allowed). NB: IRL, traders get rich by betting someone else's money.

Fondations des probas par la théorie des jeux

Méthode générale:
La nature joue une séquence y = (y_i) , on n'impose aucune contrainte sur les (y_i) c'est sa stratégie.
Pour prouver une propriété P(y), i.e. la loi des grands nombres

1. On construit un jeu tel que la condition de gain est:
   - le capital reste toujours positif
   - le joueur gagne si P ou si il devient infiniment riche


2. On montre qu'il existe une stratégie gagnante


3. Principe fondamental: pas de martingales positives qui marchent


4. Donc P

Remarques:
L'idée est que la nature peut toujours
- empêcher le joueur de s'enrichir, ou
- violer P
mais si le joueur mise astucieusement sur l'écart à P, pas les deux en même temps.

Intérêts de l'approche:

• C'est une exploration mathématique des méthodes de spéculation. Les preuves étant constructives, elles sont encore plus intéressantes si on ne croit pas au principe fondamental !


• Pas besoin de l'axiome d'additivité dénombrable ou finie.


• Preuves parfois plus courtes que dans l'approche traditionnelle, il suffit d'exhiber la stratégie.


• On introduit directement les lower et upper expectations (probabilités imprécises)

Pub: Special issue of the JEHPS e-journal.

Comment faire de bonnes prévisions

(Hot from my desk, see August 2007 Working paper #22 on defensive forecasting.)

Remark: if skeptic has a strategy A that gets infinitely rich if Pa is violated, and B that gets infinitely rich if Pb is violated, then (A+B)/2 gets infinitely rich if either Pa or Pb is violated. By averaging repeatedly, Skeptic can build a quasi-universal strategy that gets rich if nature violates any number of laws of how probability should behave. The universal test is not computable, so it is a quasi-universal test in practice.

Now take the point of view of the forecaster. The skeptic uses the fixed strategy above and plays before forecaster. We assume, critically, that the strategy of skeptic is continuous with respect to the forecaster's play at the next step.

Justifications:
- The strategies we had in the book are indeed continuous.
- Brouwer held that all constructible functions are continuous.
- Another way around is to allow forecaster imprecision.

Then there are forecasting strategies that prevent skeptic from making money.

Pub: See our Online prediction wiki.

Idea: see what happens if we allow the forecaster to make upper and lower previsions.