Prediction Mathematics

Before I go any further into this topic, I want all the other (and more-qualified-than-I) statisticians out there please to hold off on quibbles about minutiae, because this is a fairly simplistic overview, not an academic treatise about the topic. For the record, however, let me remind everybody that I was involved in designing predictive algorithms in my past life as a consultant in the supermarket industry, and my specialty was assessing and assigning the different weighting factors involved in predicting incremental sales created by price- and other kinds of promotions. I didn’t design the algorithms — that was the job of some seriously-brainy boffins from MIT, University of Chicago and Northwestern — but I did advise them on the above, and the results were predictive algorithms that generated forecasts which were generally between 95% and 97% accurate.

What prompted this post was this article, which I urge  you to read before continuing, because otherwise what I’m going to say may not make sense.

Here’s a quick thumbnail sketch as to how all this works — and I’m not going to use the supermarket business because even I fall asleep because of its mind-numbing boredom. Let’s make it more current, more contemporaneous.

Say we want to establish the likelihood of someone becoming a terrorist who wants to blow a bunch of innocent people up in a suicide attack. Note the terms of the discussion carefully, because they are important.

  • “Terrorist” = somebody who wants to terrorize the population at large
  • “Innocent people” = people who are not actively inimical to the terrorist’s philosophy, group or society
  • “Suicide” = someone who knows that he will perish in the attack.

Note that this predictive algorithm is not going to identify Timothy McVeigh, for example, because while some innocent people were killed in his Oklahoma City attack, the bomb he created was specifically targeted at an IRS building as opposed to, say, a Pink Floyd concert. Likewise, McVeigh made careful plans to avoid being killed in the bomb blast, and his attack was probably designed to create fear among government employees. (Yes, of course he was a terrorist, just not the kind we’re trying to predict below.)

So how does one establish an algorithm to foresee (and, one hopes, guard against) a terrorist attack such as described in the brief? One looks at history (without which all predictions are called “guesswork”) and looks at the profiles of all other people who have perpetrated such crimes in the past, and not the distant past either, because time has a way of making predictive algorithms irrelevant as circumstances change. From that, we can deduce the following contemporary factors:

  • religious fanaticism
  • age
  • sex
  • exposure to radical philosophy
  • societal alienation
  • socio-economic status

That’s not a comprehensive list by any means, but it will give you an idea of what’s involved. What this algorithm is supposed to do is drill down through the total population of a defined universe (a country, an area, the entire world) to identify a potential terrorist as defined above. So here we go, and let’s build a set of simple parameters for our algorithm from some of the above factors, starting with the easiest one first.

  • Socio-economic status:
    We can eliminate the upper echelons of society from any inspection. Saudi or Swedish princes and billionaire oil oligarchs don’t blow themselves up in Parisian shopping malls, or at least none have so far. Almost exclusively, terrorists have come from middle-class origins and the unemployed- or low-wage scale segments. These are micro-weightings, i.e. applied within the criterion itself. Using a scale of 1-100, we can estimate that upper-class: 0.5; middle-class: 40; low-wage: 50; unemployed: 65. (Note that they don’t have to add up to 100 collectively; we’re establishing a risk factor for each group.)
    The more interesting question is: how important is socio-economic status as a predictive factor compared to, say, religion? Probably not as much; but how much less important? This is a macro-weighting, which is applied across all the identified criteria. For the sake of argument, let’s assign the socio-economic factor a weighting of, say, 35 overall.
  • Societal alienation:
    Immigrant or native-born? Immigrants or, as we used to call them, “strangers in town” or “newcomers” may feel that they’re not part of the new society in which they find themselves — especially if that society is radically different from the one they left. Newcomers also have fewer “roots” in that society, which makes anti-social activity less problematic for their conscience. If the newcomers are also part of an ethnic group which sets themselves apart from the mainstream of their adopted society — a combination of socially, philosophically or physically — this will add to their feelings of alienation. The second determinant, native-born, is probably less important, although if they are members of a “set-apart” group, that micro-weighting needs to be adjusted upwards, and especially if they have constant contact with newcomers. Once again, we can assign micro-weightings of 60 and 45 respectively.
    For the macro-weighting, we can ask how important alienation is, compared to socio-economic status? Probably a lot more, but once again, how much more? — which is the weighting decision. More than socio-economic’s 35? Definitely — more like 60, almost twice as likely.
  • Age:
    Most terrorists are young — under the age of forty. While an age of, say, sixty-five is not a disqualifying criterion, it certainly suggests a far smaller weighting than someone who is in their twenties (which group has supplied the far-greater proportion of terrorists than sexagenarians). We can assign weightings by specific age groups (e.g. 12-16, 17-25, 26-30 and so on), but to keep things simple, we’ll give the under-40s a cumulative micro-weighting of 90, and the over-40s a score of 5.
    As a macro-weighting, age is one of the principle determinants of likely terrorists, and incidentally of most major criminal activity in general (check the distribution curve of ages among prison inmates and known terrorists to verify this statement). Let’s give this group a score of 50 — less than socio-economic status, but not much less.
  • Religious fanaticism:
    Almost all religions engender fanaticism in one way or another, but in recent times (remember the “recent history” issue), Islam has produced by far the greater number, and has caused by far the greatest number of terrorist-inspired incidents, which have killed by far the greatest number of innocent people. (Note that Nazi fanatics killed far more innocent people in the past two hundred-odd years, but in the past two decades have killed almost none — hence the recency determinant.) At the moment, therefore, an adherent of Islam would need to get a far greater micro-weighting than, say, a Nazi, Christian or Buddhist.
    As a macro-weighting (applied against the total population), Islam is probably the single most important determinant — and if one were to apply a weighting factor along that scale of 1-100, one could easily assign a contemporary weighting of 95 or even higher.

Of course, anyone suggesting weightings such as the above is going to be accused of “profiling” by the moral relativists, SJWs, ACLU, SPLC and suchlike Useful Idiots, but I should point out that on that basis, no courts should use the COMPAS system at all.

What should be fairly obvious to anyone is that while the overall algorithm design can be a proprietary affair, the weighting factors within the algorithms need to be subject to the closest scrutiny and debate possible. I should also point out that a lack of such analysis has enabled the scam known as global warming / -cooling / climate change to be accepted by the gullible and ignorant, but we can talk about that another time.

Suffice it to say that the more daylight involved, and most certainly the daylight within the group building and implementing the forecast criteria — statisticians, intelligence services, law enforcement and the judicial system, the more accurate the algorithms will become. Most important, however, is the fact that the predictive algorithms will engender a higher degree of trust in the population.

Then And Now

Just to prove that there’s nothing new under the sun…

Then:

Now:

I should point out that while the girl in the first pic has assumed a rock-steady stance for a gun rest, the same is not true for the girl in the second pic. I blame the public school system, NBC’s Olympic coverage of Women’s Gymnastics, and Democrats in general.

Also, modernity does have its drawbacks. The girl in the first picture is unlikely to experience any untoward effects, other than perhaps some black-powder smoke in her eyes (and she’s wearing a bonnet to help prevent that; very responsible). Unfortunately,and thanks to the rapid-fire ability of the AR-15 and the concomitant warming of its barrel, the girl in the second pic is likely to experience a condition known to us Old Guys as “blistered furburger”.

Just sayin’. (And thanks to Reader Old Texan for the second pic.)

Road Music

As a general rule, I don’t listen to music in the car, other than perhaps Dallas-Fort Worth’s classical music station WRR (101.1 FM) if I’m caught in a traffic jam.

On long trips, however, and especially driving through the bleak nothingness  that is northwest Texas, some sterner stuff is needed. Here’s what I brought along for this particular trip:

  • Steely Dan: Citizen Steely Dan
  • Procol Harum: Prodigal Stranger, Shine On Brightly
  • Lindisfarne: Magic In The Air
  • Kate Bush: The Kick Inside
  • Chicago: Greatest Hits Vol I and II
  • Genesis: Duke, …And Then There Were Three
  • Level 42: Running In The Family, World Machine
  • Joe Walsh: Look What I Did (greatest hits)
  • Wishbone Ash: Time Was (greatest hits)
  • Earl Klugh: Heartstring, Living Inside Your Love
  • Strawbs: Bursting At The Seams
  • Peter Skellern: Sentimentally Yours, Cheek To Cheek

…and some classical stuff that nobody’s interested in: Schumann, Rachmaninoff, Saint-Saëns, Chopin, the usual stuff.

Yeah, it’s a strange assortment. I like variety in my music. And yes, they’re all CDs. I see no reason to buy online music when I already have most of what I like to listen to.

 

On The Road (1)

Blogging will be scarce, and possibly even of lower quality than normal, as I’m driving to visit a friend in another state. (No flying until I have to, thank you very much.) I’ll try to post while on the road, but I’ll be at the mercy of the Fleabagge Motel’s wi-fi, so if nothing appears, that’s the reason.

In the meantime, enjoy a little eye candy:

It’s the Beretta Model 75 in .22LR, and it’s the gun I learned to shoot handguns with. I’ve been spoiled ever since by the experience.

Oh, you wanted that kind of eye candy? For shame, for shame. Okay, here’s Janine Turner, suitably dressed:

 

Welcome Back

Back when I was a consultant, The Mrs. and I went through several periods of “chicken and feathers” — wealthy one minute, impoverished the next. During the chicken times, we’d travel, treat the kids and save, but during the feathers times, we’d be in trouble. In many cases, the trouble was a short-term problem: waiting for a client’s check to arrive, or for the check to clear, that kind of thing. There was always a credit card to tide us over, and then life could resume once the funds were released.

Then there were the longer periods of feathers, such as when a couple of our start-up business ideas failed. The first cost us about a hundred and fifty thousand dollars, which we paid out of our savings; the second cost us well over a third of a million — we had salaries to pay even though no money was coming in, and on that occasion we lost everything: savings, retirement accounts, and almost, the house. (I have no idea how we averted that disaster, but somehow one of us or the other always managed to get a small consulting gig which was just enough to keep a roof over our heads, but with nothing left over to pay bills. I even had a car repossessed during this time.)

So I had to sell off my guns. Pretty much all of them went, except for a junky old 16ga side-by-side which wasn’t worth diddly to anyone, and Connie’s little NAA Mini revolver. Most of them I sold to good friends, on the understanding that they could sell the guns if they wished, but that if they did so, I could at least get first refusal, as it were. Gradually over the past couple years, I’ve been able to buy a few guns back here and there, and a couple of really good friends even lent me a gun or two to tide me over — “on the non-return basis”, so to speak, because they couldn’t bear to see my family defenseless. These angels know who they are, and I’m not going to embarrass them by revealing their identities.

I won’t go into details of each of the guns thus sold and recovered because it’s unnecessary to this story, but here’s a pic (taken on a trip to the range with Mr. Free Market many years ago) of some of the guns I had to sell:

But one of those guns in the picture has stuck with me, because while it pained me to dispose of various beautiful Colts, Browning High Powers, Rugers and so on, my most anguished sale was the rifle at the top of the pic, this little thing:

To give the rifle its official name, it’s the Taurus Model 62C (.22LR) Pump-Action Carbine. To anyone who has ever shot one, its actual name is “OMG I haven’t had so much fun in years!”, always followed by a firm refusal to release the gun back to its rightful owner and a demand for more ammo.

As I’d mentioned a few days ago, I was seeking a semi-auto plinker — basically to replace this little darling — but all the time in the back of my mind, I kept thinking, “I wonder if [name redacted] would consider selling the Taurus back to me?”

There was really only one big concern: that its new owner would have fallen in love with the piece, as I had done, and would refuse to do so. (I should point out that this has happened to me with several of my friends; they’ve fallen in love with the guns and refused to part with them, even back to me.)

You see, the Marketing Department at Taurus are basically a bunch of morons, because several years ago they decided to drop this line from their catalog — all the Model 62 variants — which meant that a replacement would be difficult to find.

Happily, my kind friend decided, after considerable and doubtlessly-tortured reflection, that the rifle’s proper place would be back with its original owner. So today I can announce that I no longer need that .22 plinker: I’ve got my baby back. It’s resting by my chair, as I write these words.

One final word of explanation is probably necessary. The Taurus 62 is a pump-action, not a semi-auto rifle. Why, then, would I not need a semi-auto plinker, still?

The answer is obvious if one knows me well: I prefer to operate machinery rather than just use it — stick shift over auto transmissions, bolt-action over semi-auto rifles, revolvers over semi-auto pistols, and so on. I get added pleasure from working an action — it’s a feeling of control which is difficult to explain, but to people like myself, all too easy to understand.

I cannot wait to get out to some open country, toss a few cans or oranges onto the ground, and start shooting. And if you want to know how much I want to do this: I’d rather do this for a few hours than spend the same amount of time in Nigella Lawson’s boudoir. Yes, that much.

And now, if you’ll excuse me, I’m not going to the range. I’m off to find some open country…

The Thief Of Time

I am really, really old-fashioned when it comes to being on time for anything. In the first place, I spent seven years in boarding school whose attention to time meant that second bell for morning roll-call rang at 7:02 am, and if you arrived even a few seconds after that time you were adjudged late, and got punished. (Note that it was 7:02 am, and not 7:00 or 7:05.) In the second instance, I have always worked in industries where time was measured in seconds or minutes, not hours or days, and deadlines were critical — hence the “dead” in deadline — so timeliness was not only important, but tardiness was expensive.

My motto is quite simple: “Five minutes early is on time”, and no matter where I’m coming from or how fraught the journey, I make every effort to make the 5-minute deadline. The corollary to the motto is that I get pissed when people are as little as a minute late, which makes me something of a tight-ass, and I’ve been thus called many, many times.

I don’t care.

As far as I’m concerned, by arriving late you’re telling the other person that his or her time isn’t important, or at least isn’t as important as your time, and that attitude is unbearably rude and inconsiderate. Don’t even ask me how I feel about “Mediterranean-” or “African Time”, where an appointed time is not even a “guideline” but wishful thinking. Likewise, I’ve walked away from doctors’ offices when I’m kept waiting for longer than half an hour — and when one doctor had the effrontery to bill me $70 for “breaking the appointment”, you can imagine his surprise when in return he got a bill from me for $250 (my rate for the hour-and-a-half I spent driving to his office, being ignored and driving home). The only excuse for lateness that I’ve ever accepted in a business situation is if the guy was talking to a client, or if the guy is the client.

So you can imagine my reaction to this little snippet, wherein a woman admits cheerfully that she’s always a few minutes late for social engagements, but always on time when it’s a business appointment — and is then astonished when she’s called a bunch of rude names by people who have the same standards as I have. Here’s a tip: if you know that you may be held up by traffic, or a family emergency (e.g. a full diaper belonging to a baby), then leave half an hour before you would otherwise do.

We’re always told that “time is money” by so-called efficiency experts. It isn’t. It’s worth a lot more than money. Time is the most precious resource on Earth, we each have only a finite amount of it, and when people waste my time through their careless and rude tardiness, I get so angry I have to be restrained from slipping the safety off the 1911.

And please: of course I make exceptions if someone had an actual car crash, or had to take their kid to the Emergency Room or some such situation. I’m not an unreasonable man. But outside those situations, it pisses me off that when I excoriate someone for their rudeness that somehow, I’m the bad guy for being so persnickety about time. Well, you’re fucking right I’m persnickety — and I’m going to get worse as I get older and time becomes all the more limited and precious.