## Prediction Mathematics

Before I go any further into this topic, I want all the other (and more-qualified-than-I) statisticians out there please to hold off on quibbles about minutiae, because this is a fairly simplistic overview, not an academic treatise about the topic. For the record, however, let me remind everybody that I was involved in designing predictive algorithms in my past life as a consultant in the supermarket industry, and my specialty was assessing and assigning the different weighting factors involved in predicting incremental sales created by price- and other kinds of promotions. I didn’t design the algorithms — that was the job of some seriously-brainy boffins from MIT, University of Chicago and Northwestern — but I did advise them on the above, and the results were predictive algorithms that generated forecasts which were generally between 95% and 97% accurate.

What prompted this post was this article, which I urge  you to read before continuing, because otherwise what I’m going to say may not make sense.

Here’s a quick thumbnail sketch as to how all this works — and I’m not going to use the supermarket business because even I fall asleep because of its mind-numbing boredom. Let’s make it more current, more contemporaneous.

Say we want to establish the likelihood of someone becoming a terrorist who wants to blow a bunch of innocent people up in a suicide attack. Note the terms of the discussion carefully, because they are important.

• “Terrorist” = somebody who wants to terrorize the population at large
• “Innocent people” = people who are not actively inimical to the terrorist’s philosophy, group or society
• “Suicide” = someone who knows that he will perish in the attack.

Note that this predictive algorithm is not going to identify Timothy McVeigh, for example, because while some innocent people were killed in his Oklahoma City attack, the bomb he created was specifically targeted at an IRS building as opposed to, say, a Pink Floyd concert. Likewise, McVeigh made careful plans to avoid being killed in the bomb blast, and his attack was probably designed to create fear among government employees. (Yes, of course he was a terrorist, just not the kind we’re trying to predict below.)

So how does one establish an algorithm to foresee (and, one hopes, guard against) a terrorist attack such as described in the brief? One looks at history (without which all predictions are called “guesswork”) and looks at the profiles of all other people who have perpetrated such crimes in the past, and not the distant past either, because time has a way of making predictive algorithms irrelevant as circumstances change. From that, we can deduce the following contemporary factors:

• religious fanaticism
• age
• sex
• exposure to radical philosophy
• societal alienation
• socio-economic status

That’s not a comprehensive list by any means, but it will give you an idea of what’s involved. What this algorithm is supposed to do is drill down through the total population of a defined universe (a country, an area, the entire world) to identify a potential terrorist as defined above. So here we go, and let’s build a set of simple parameters for our algorithm from some of the above factors, starting with the easiest one first.

• Socio-economic status:
We can eliminate the upper echelons of society from any inspection. Saudi or Swedish princes and billionaire oil oligarchs don’t blow themselves up in Parisian shopping malls, or at least none have so far. Almost exclusively, terrorists have come from middle-class origins and the unemployed- or low-wage scale segments. These are micro-weightings, i.e. applied within the criterion itself. Using a scale of 1-100, we can estimate that upper-class: 0.5; middle-class: 40; low-wage: 50; unemployed: 65. (Note that they don’t have to add up to 100 collectively; we’re establishing a risk factor for each group.)
The more interesting question is: how important is socio-economic status as a predictive factor compared to, say, religion? Probably not as much; but how much less important? This is a macro-weighting, which is applied across all the identified criteria. For the sake of argument, let’s assign the socio-economic factor a weighting of, say, 35 overall.
• Societal alienation:
Immigrant or native-born? Immigrants or, as we used to call them, “strangers in town” or “newcomers” may feel that they’re not part of the new society in which they find themselves — especially if that society is radically different from the one they left. Newcomers also have fewer “roots” in that society, which makes anti-social activity less problematic for their conscience. If the newcomers are also part of an ethnic group which sets themselves apart from the mainstream of their adopted society — a combination of socially, philosophically or physically — this will add to their feelings of alienation. The second determinant, native-born, is probably less important, although if they are members of a “set-apart” group, that micro-weighting needs to be adjusted upwards, and especially if they have constant contact with newcomers. Once again, we can assign micro-weightings of 60 and 45 respectively.
For the macro-weighting, we can ask how important alienation is, compared to socio-economic status? Probably a lot more, but once again, how much more? — which is the weighting decision. More than socio-economic’s 35? Definitely — more like 60, almost twice as likely.
• Age:
Most terrorists are young — under the age of forty. While an age of, say, sixty-five is not a disqualifying criterion, it certainly suggests a far smaller weighting than someone who is in their twenties (which group has supplied the far-greater proportion of terrorists than sexagenarians). We can assign weightings by specific age groups (e.g. 12-16, 17-25, 26-30 and so on), but to keep things simple, we’ll give the under-40s a cumulative micro-weighting of 90, and the over-40s a score of 5.
As a macro-weighting, age is one of the principle determinants of likely terrorists, and incidentally of most major criminal activity in general (check the distribution curve of ages among prison inmates and known terrorists to verify this statement). Let’s give this group a score of 50 — less than socio-economic status, but not much less.
• Religious fanaticism:
Almost all religions engender fanaticism in one way or another, but in recent times (remember the “recent history” issue), Islam has produced by far the greater number, and has caused by far the greatest number of terrorist-inspired incidents, which have killed by far the greatest number of innocent people. (Note that Nazi fanatics killed far more innocent people in the past two hundred-odd years, but in the past two decades have killed almost none — hence the recency determinant.) At the moment, therefore, an adherent of Islam would need to get a far greater micro-weighting than, say, a Nazi, Christian or Buddhist.
As a macro-weighting (applied against the total population), Islam is probably the single most important determinant — and if one were to apply a weighting factor along that scale of 1-100, one could easily assign a contemporary weighting of 95 or even higher.

Of course, anyone suggesting weightings such as the above is going to be accused of “profiling” by the moral relativists, SJWs, ACLU, SPLC and suchlike Useful Idiots, but I should point out that on that basis, no courts should use the COMPAS system at all.

What should be fairly obvious to anyone is that while the overall algorithm design can be a proprietary affair, the weighting factors within the algorithms need to be subject to the closest scrutiny and debate possible. I should also point out that a lack of such analysis has enabled the scam known as global warming / -cooling / climate change to be accepted by the gullible and ignorant, but we can talk about that another time.

Suffice it to say that the more daylight involved, and most certainly the daylight within the group building and implementing the forecast criteria — statisticians, intelligence services, law enforcement and the judicial system, the more accurate the algorithms will become. Most important, however, is the fact that the predictive algorithms will engender a higher degree of trust in the population.

## Malware, Change, And The Whole Damn Thing

Over at samizdata, Perry Metzger (not De Havilland) has a few trenchant observations about stupid people who don’t use condoms when they have unprotected Internet intercourse, or something. (For those who don’t know him, Perry’s writing style is often blunt and dismissive, which is one of the reasons I enjoy reading his stuff. Go figure.) Read it all, including the comments, because a lot of what I say from here may be otherwise incomprehensible.

I’m not going to argue with Perry’s main point about the need to upgrade your computer’s software regularly. From a security standpoint, it makes sense to install the patches which cover the gaping holes in the thing. I also understand that the software companies don’t care to maintain elderly platforms, for the same reasons that Ford no longer maintains Model Ts. (The fact that software changes occur at an exponentially-quicker rate than automotive ones is just the nature of the beast.)

The problem, as noted the the Comments, is that system “upgrades” are not devoted exclusively to security patches anymore. Instead, all sorts of crap is included which at best causes irritating changes in functionality, and at worst undoes a lot of the learning and experience that one has accumulated. I understand why this occurs, but that doesn’t mean I’m at all happy about it. And so far, Microsoft has accommodated us Old Farts by including a “traditional” desktop view for all new Windows operating system versions, so I don’t have to memorize all the silly new pictograms in Windows 7 – infinity. (Note to MS: remove that feature and I’m gone.)

And this is the point. One of the commenters at samizdata made the excellent point that Microsoft (and all software developers, as far as I can see) are more interested in getting new customers, who would be more comfortable with apps, pictograms, symbols and what have you than they would be with the old icons or, gawd forbid, text (all those words and stuff? dude!). That’s stupid, for all sorts of reasons, and here’s why.

I might not be worth much to Microsoft as an existing customer right now; but there was a time when people like me — the early personal-computer adopters — helped build Microsoft into what it is today. When you have a person who like myself has been through all the hardware iterations of the PC, XT, AT, 386x all the way through to the current whatever-it-is-I’m-writing-on-right now, and has likewise been through all the software iterations of DOS 2.0 through Windows 7/8/not-9 [ahem] and 10; when you have a longtime customer group like that, then surely I, and all the countless millions of people like me, deserve just a little accommodation in the Grand Microsoft Marketing Plan? (Okay, you can stop laughing now.)

I know, everything these days falls into the “But what have you done for me lately?” category, but it’s still a basic truism of marketing that it’s ten times easier to get an existing customer to stay with your product than it is to lure a new one away from a competitor. But if you persist in changing your product so that it not only becomes a purely new-customer attractant, but also an existing-customer repellent, then I would suggest that someone in Marketing needs to go back to business school and/or get a swift kick in the teeth to adjust their thinking.

I know that it’s expensive and resource/time-consuming to maintain old products. Of course it is. But I would suggest that it’s also a lot easier than new product development — we old-timers don’t ask for much, because we’re used to working with, by today’s standards, relatively unsophisticated products.

Using the automotive industry one more time: Ford, GM and Chrysler have discovered that there is a huge market for nostalgia models such as the Dodge Charger/Challenger, Ford Shelby Mustang and the like. These new iterations of the venerable hot rods of yore have been improved, of course: better brakes, suspension and so on; they’re still simple and unsophisticated by modern whizzbang standards, but their manufacturers can’t make them quickly enough. Let’s go exotic: a new 2016 LaFerrari costs about \$1.5 million; a 1966 275 GTB recently sold for \$2.1 million. (I know, that’s mostly a factor of scarcity; at the same time, however, the 2016 model is a hundred times better than its 50-year-old counterpart — and still, someone was prepared to pay good money for it.)

Somewhere is all the above rambling is the seed of an idea for Microsoft. Or maybe, for someone not in Microsoft who can see a niche in the PC market which is similar to the automotive restoration market.

Or maybe I’m just an old fart “shaking his fist at heaven”, as Perry Metzger suggests. Still, I’m pretty sure I’m not the only one who feels this way — in fact, there may be more of us than of them. Software developers — or to be more accurate, software maintainers — might want to take a look at that.