Prediction Mathematics

Before I go any further into this topic, I want all the other (and more-qualified-than-I) statisticians out there please to hold off on quibbles about minutiae, because this is a fairly simplistic overview, not an academic treatise about the topic. For the record, however, let me remind everybody that I was involved in designing predictive algorithms in my past life as a consultant in the supermarket industry, and my specialty was assessing and assigning the different weighting factors involved in predicting incremental sales created by price- and other kinds of promotions. I didn’t design the algorithms — that was the job of some seriously-brainy boffins from MIT, University of Chicago and Northwestern — but I did advise them on the above, and the results were predictive algorithms that generated forecasts which were generally between 95% and 97% accurate.

What prompted this post was this article, which I urge  you to read before continuing, because otherwise what I’m going to say may not make sense.

Here’s a quick thumbnail sketch as to how all this works — and I’m not going to use the supermarket business because even I fall asleep because of its mind-numbing boredom. Let’s make it more current, more contemporaneous.

Say we want to establish the likelihood of someone becoming a terrorist who wants to blow a bunch of innocent people up in a suicide attack. Note the terms of the discussion carefully, because they are important.

  • “Terrorist” = somebody who wants to terrorize the population at large
  • “Innocent people” = people who are not actively inimical to the terrorist’s philosophy, group or society
  • “Suicide” = someone who knows that he will perish in the attack.

Note that this predictive algorithm is not going to identify Timothy McVeigh, for example, because while some innocent people were killed in his Oklahoma City attack, the bomb he created was specifically targeted at an IRS building as opposed to, say, a Pink Floyd concert. Likewise, McVeigh made careful plans to avoid being killed in the bomb blast, and his attack was probably designed to create fear among government employees. (Yes, of course he was a terrorist, just not the kind we’re trying to predict below.)

So how does one establish an algorithm to foresee (and, one hopes, guard against) a terrorist attack such as described in the brief? One looks at history (without which all predictions are called “guesswork”) and looks at the profiles of all other people who have perpetrated such crimes in the past, and not the distant past either, because time has a way of making predictive algorithms irrelevant as circumstances change. From that, we can deduce the following contemporary factors:

  • religious fanaticism
  • age
  • sex
  • exposure to radical philosophy
  • societal alienation
  • socio-economic status

That’s not a comprehensive list by any means, but it will give you an idea of what’s involved. What this algorithm is supposed to do is drill down through the total population of a defined universe (a country, an area, the entire world) to identify a potential terrorist as defined above. So here we go, and let’s build a set of simple parameters for our algorithm from some of the above factors, starting with the easiest one first.

  • Socio-economic status:
    We can eliminate the upper echelons of society from any inspection. Saudi or Swedish princes and billionaire oil oligarchs don’t blow themselves up in Parisian shopping malls, or at least none have so far. Almost exclusively, terrorists have come from middle-class origins and the unemployed- or low-wage scale segments. These are micro-weightings, i.e. applied within the criterion itself. Using a scale of 1-100, we can estimate that upper-class: 0.5; middle-class: 40; low-wage: 50; unemployed: 65. (Note that they don’t have to add up to 100 collectively; we’re establishing a risk factor for each group.)
    The more interesting question is: how important is socio-economic status as a predictive factor compared to, say, religion? Probably not as much; but how much less important? This is a macro-weighting, which is applied across all the identified criteria. For the sake of argument, let’s assign the socio-economic factor a weighting of, say, 35 overall.
  • Societal alienation:
    Immigrant or native-born? Immigrants or, as we used to call them, “strangers in town” or “newcomers” may feel that they’re not part of the new society in which they find themselves — especially if that society is radically different from the one they left. Newcomers also have fewer “roots” in that society, which makes anti-social activity less problematic for their conscience. If the newcomers are also part of an ethnic group which sets themselves apart from the mainstream of their adopted society — a combination of socially, philosophically or physically — this will add to their feelings of alienation. The second determinant, native-born, is probably less important, although if they are members of a “set-apart” group, that micro-weighting needs to be adjusted upwards, and especially if they have constant contact with newcomers. Once again, we can assign micro-weightings of 60 and 45 respectively.
    For the macro-weighting, we can ask how important alienation is, compared to socio-economic status? Probably a lot more, but once again, how much more? — which is the weighting decision. More than socio-economic’s 35? Definitely — more like 60, almost twice as likely.
  • Age:
    Most terrorists are young — under the age of forty. While an age of, say, sixty-five is not a disqualifying criterion, it certainly suggests a far smaller weighting than someone who is in their twenties (which group has supplied the far-greater proportion of terrorists than sexagenarians). We can assign weightings by specific age groups (e.g. 12-16, 17-25, 26-30 and so on), but to keep things simple, we’ll give the under-40s a cumulative micro-weighting of 90, and the over-40s a score of 5.
    As a macro-weighting, age is one of the principle determinants of likely terrorists, and incidentally of most major criminal activity in general (check the distribution curve of ages among prison inmates and known terrorists to verify this statement). Let’s give this group a score of 50 — less than socio-economic status, but not much less.
  • Religious fanaticism:
    Almost all religions engender fanaticism in one way or another, but in recent times (remember the “recent history” issue), Islam has produced by far the greater number, and has caused by far the greatest number of terrorist-inspired incidents, which have killed by far the greatest number of innocent people. (Note that Nazi fanatics killed far more innocent people in the past two hundred-odd years, but in the past two decades have killed almost none — hence the recency determinant.) At the moment, therefore, an adherent of Islam would need to get a far greater micro-weighting than, say, a Nazi, Christian or Buddhist.
    As a macro-weighting (applied against the total population), Islam is probably the single most important determinant — and if one were to apply a weighting factor along that scale of 1-100, one could easily assign a contemporary weighting of 95 or even higher.

Of course, anyone suggesting weightings such as the above is going to be accused of “profiling” by the moral relativists, SJWs, ACLU, SPLC and suchlike Useful Idiots, but I should point out that on that basis, no courts should use the COMPAS system at all.

What should be fairly obvious to anyone is that while the overall algorithm design can be a proprietary affair, the weighting factors within the algorithms need to be subject to the closest scrutiny and debate possible. I should also point out that a lack of such analysis has enabled the scam known as global warming / -cooling / climate change to be accepted by the gullible and ignorant, but we can talk about that another time.

Suffice it to say that the more daylight involved, and most certainly the daylight within the group building and implementing the forecast criteria — statisticians, intelligence services, law enforcement and the judicial system, the more accurate the algorithms will become. Most important, however, is the fact that the predictive algorithms will engender a higher degree of trust in the population.


  1. Thoughtful as always. Your college-educated musings are a bit more highbrow than my barely-acquired high school diploma may permit me to analyze. I never supported Trump, but I’ll bet Shrillary would condemn me as Deplorable, regardless. I hope so, anyway. I’d wear it like a badge.

    My own predictive algorithm is what I’d like to believe is a reasonably sober survey of the relevant history (full disclosure: much of this is cobbled together from a letter to the ed I wrote to my local rag, in response to a simpering lib’s letter attributing Manchester to the wages of “Western terrorism.” I shit you not).

    732: Charles Martel fought off a Muslim invasion at the Battle of Tours. The invasion wasn’t precipitated by Western terrorism.

    1453: Constantinople had its name changed to Istanbul. It was not the result of a friendly chat, nor the byproduct socioeconomic privation.

    1532: The Venetians were obliged to confront an invading Muslim fleet at Lepanto. I don’t know the average age of the invaders, but suspect it was not terribly relevant to their overall motivation.

    1683: The Austrians and Germans had to hold out against a brutal siege. I can’t declare if the Ottomans were feeling ‘set apart’, but I suspect not.

    1786: Thomas Jefferson and John Adams were ordered to London determine why our maritime civilians were being murdered and enslaved in the Mediterranean. They were received by the envoy to Tripoli, Sidi Haji Abdrahaman. He explained very matter-of-factly (as reported by Jefferson to Secretary of Foreign Affairs John Jay) “…that all nations which had not acknowledged the Prophet were sinners, whom it was the right and duty of the faithful to plunder and enslave; and that every mussulman who was slain in this warfare was sure to go to paradise.” Indistinguishable from the rhetoric of present-day ISIS; and it’s remarkable how apologists for what is now called ‘Islamic extremism’ are so determined to ignore the same old, same old.

    Every president of my lifetime – Republican and Democrat – has taken pains to identify the “moderate” Muslim nations for our benefit. Saudi Arabia has always been on that list. In moderate Saudi Arabia, distributing Bibles will land you in one of their moderate prisons. Rejecting the teachings of the Prophet Mohammad (Propeller Beanie Upon Him) will earn you a public beheading in Riyadh. I hope you’re tipping your hat to their marvelous moderation. If not, you must be a bigot.

    There is no Christian analogue to Sharia law of the sort practiced by the moderate House of Saud; to say nothing of the more extreme iterations being preached in mosques all over the West. Nor a Western parallel to last year’s United Nations Resolution 16/18 – the latest in a long line of regularly submitted anti-blasphemy resolutions, calling upon the entire world to criminalize any and all criticism of Islam. Its signatories include every Muslim nation on earth, including those whom our elites assure us are “moderate.”

    Islam is not just a religion. Alone among the major faiths, it is a political movement. Its followers have been trying to impose the same theocratic nightmare for more than a millennium. That’s predictive enough for me.

  2. Good ideas, except that our ruling elites actually like the terrorists and terrorism.

    The elites never get hit by the terrorists, they never lose jobs or money from unrestricted immigration and they use the fears of the populace to bolster their powers and perks. Because the fight against terror, just like the fights against poverty and (non-existent) global warming require higher taxes.

    This is because, the elites claim, we need lots of money for the management at the terror fighting agencies and those who work for them.

    How else will some middle management government idiot get the big office and two new assistants with big boobies?

    How else will the owners of Academi Security increase their profit?

    Never forget your Mencken: The whole aim of practical politics is to keep the populace alarmed (and hence clamorous to be led to safety) by menacing it with an endless series of hobgoblins, all of them imaginary.

Comments are closed.