The Roots vs The Flower

I think that the most important issue that we need to grapple with in American today is, what kind of racism continues to exist? In my opinion, claims on either end of the spectrum lack legitimacy, i.e. we are not experiencing a more camouflaged version of Institutional racism but we are not experiencing Incidental racism by a few bad apples either. Rather, we are experiencing a racial bias within the fabric of our society. It is of the same variety that James Baldwin describes in his great debate with William F. Buckley (below). Thus, for example, the notion that the police are, in effect, patrolling the border between black and white communities is not only an inaccurate depiction but also a counter-productive one, as it tilts the frame towards a kind of Institutional racism that does not exist. As well, the notion that policing is uniformly applied across black and white communities, and the disproportionate representation of African Americans in the criminal justice system is as a result of disproportionate criminal activity is an equally inaccurate and counter-productive depiction, as it glosses over a much more militarized and aggressive form of law enforcement that has evolved in high-crime, high-poverty communities, which are typically black communities. The question remains, how do we address a racial bias in the fabric of our society? I do not think it can be addressed by asking white America to take ownership of sins that they have not committed and ask forgiveness for views that they do not hold (as an aside, I would not be considered 'white' by the normal standard, as I am half Indian). This is under the assumption that the vast majority of white Americans are not overt or covert racists, and do not commit racist acts by commission or omission. This may or may not be the case, but, given that it is not a supposition that can be derived from hard data, I think that we would be much better off assuming it is the case. In my view, the only path forward is to re-frame this debate on Individual terms, and adopt measures that benefit all Individuals, even as they will, in fact, benefit African Americans disproportionately. This tracks to the classic answer that Barack Obama gave in a debate, whereby, when asked how he will address the socioeconomic woes of African Americans, he replied that he would address the socioeconomic woes of all Americans, and African Americans would enjoy an incidental, disproportionate benefit by virtue of their more challenging circumstances. To get there, I think that it will be important for everyone to rally around a common cause for preserving Individual life, liberty and the pursuit of happiness, without regard to color, but with full knowledge that, in doing so, they will address the disparities of color within our economy, politics and criminal justice system. This would entail enforcing policies of deescalation and removing criminal liability protections in policing, fixing upside-down voting districts, investing in education and economic enterprise zones, etc, but also enforcing a more uniform code of Individual responsibility and accountability, like eliminating the welfare state and reverse discrimination, such that all Americans that need help can get it, but all Americans are, ultimately, on a level playing field. Rather than burning calories on continually fixating the racist roots of America, let’s focus the measures that will produce a non-racist flower ;-)

https://www.youtube.com/watch?v=WPz7kTnEWKE&t=975

Divided We Fall

Societies are most vulnerable when disparate groups within them are fighting with each other. I do not mean ‘fight’ as in healthy debate and legislative maneuvering, but ‘fight’ as in confrontations in the street, hatred online and the kind of callout culture that has people not wanting to say anything, opting instead to just carry unspoken and, many times, unexamined views in private. This kind of friction may very well be necessary at this juncture, but you can only imagine that the Russians, et al, are delighted at their prospects of fomenting more divisions in advance of the November election. We should give our friends and neighbors the benefit of the doubt when they express controversial views, and we should fact check every news story before we get outraged by the media.

News Blues

A traverse a lot of news stations when I actually watch the news, which is, itself, becoming a rare event these days. My first observation is that there is no truly objective news on TV, and probably elsewhere, which, I am sure, will surprise both CNN and Fox viewers. Both of these particular networks are guilty of sins of omission and commission, as it pertains to the left-right narrative that they are constantly spinning. I cannot stand to watch either one of them, so I watch them both, and like a spectator sitting mid-court at a tennis match, I feel like I am seeing the game much more clearly than if were on either end of the court or either player, for that matter. It is an ugly game. It cannot be won by either ‘side’. It can only be lost by everyone.

The American Theme

I think that it is better to refer to this shared experience of ours not as the American Dream but, rather, as the American Theme. A dream might not be realized, even if it is unique to each individual, but a theme can be realized in success or failure, and across a range of circumstances. So what is it? Well, again, it could be unique to each individual, but I contend that unless we have common theme to govern our shared experience, we won't have a shared experience for very long.

I define it this way: Individual liberty and individual accountability; Local government and local accountability; State government as the default, Federal government only as the Constitution allows; Malice towards none [...] Charity for all; and... well... all of those other great, American ideals that characterize American Values, notwithstanding everything that we, as a nation, have done to erode our own values.

At the very base of everything I've just described is the individual's rights. If a society cannot protect the rights of the individual, then there is, in effect, no social contract (Rousseau, 1762). And if a particular group experiences this lack of protection by its individual members, it can be argued that there is no social contract at that level either, but we should all be offended, because a crack in our foundation is not a local concern. It is a threat to every layer that rests upon it. Therefore, we should all be mobilized to protect this most basic right, and be clear-eyed when the mobilization is coopted.

The more polarizing forces in our society would have you confuse rioters with protesters, and, worse, confuse the right to protest with the right to riot. In reality, rioters do more damage to protesters than they do to the owners whose property they destroy, because they pervert the effort to defend the very foundation of our society. It does not mean that protests will not need to be modified to diminish the risk of riots, or that police response should not be pervasive and swift when it comes to rioters, because, after all, if we are marching in protest against the abject disregard for an individual's rights, to the point of murder, then we cannot justify someone else's murder as a side-effect of that march, even if perpetrated by outside parties taking advantage of the situation.

I think that this is the most complicated collaboration between individuals, groups and governments that we can undertake as a society, which can only be resolved by intelligent cooperation. I cannot even begin to offer solutions, outside of recognizing that we need solutions. Sometimes recognition is a good start, and that is my main objective here.

Division Divided by Division

The debate that rages on about the COVID-19 quarantine suffers from the classic complexity of trade-offs that are non-resolvable. Shutting down saved lives but it also destroyed [socioeconomic] lives. Opening up too soon or too quickly will cost lives, but remaining closed will destroy more lives. Saving lives means delaying herd immunity, but achieving herd immunity means losing lives. The list of trade-offs goes on, and the real socioeconomic costs of quarantine measures can only be justified by estimated benefits of undertaking them, making such measures difficult to defend in concrete terms. In these kinds of situations, no amount of data or science can build adequate consensus, and this is especially true when the data comes into question (e.g. the variance in death rate based on the true infection rate, which we do not know) and the science comes into question (e.g. the general population being unable to grasp that the very basis of the scientific method). The only thing that can build consensus in these types of situations is trust in government, as in: "I know that they do not have all the answers, I know that they might get it wrong from time to time, but I also know that they are taking advantage of the best data and science that the country has to offer and are making the most informed decision they can on my behalf." I need not say why we do not have this level of trust in government today, but I do need to say that the breakdown along party lines is a sad state of affairs. If we had stronger leadership, we would be more unified, especially in troubled times. I think we can all agree on that, but maybe not.

Salesforce Einstein Analytics Dataflow to Deduplicate a Dataset

Let's say you have the following data:

Key,Value,Timestamp
K1,V1,2020-05-01
K1,V2,2020-05-02
K1,V3,2020-05-03
K1,V4,2020-05-04
K2,V1,2020-05-04
K2,V2,2020-05-04
K2,V3,2020-05-02
K2,V4,2020-05-01
K3,V4,2020-05-01
K3,V3,2020-05-02
K3,V2,2020-05-03
K3,V1,2020-05-04
K4,V4,2020-05-04
K4,V3,2020-05-03
K4,V2,2020-05-02
K4,V1,2020-05-01
K5,V1,2020-05-01
K5,V2,2020-05-01
K5,V3,2020-05-01
K5,V4,2020-05-01

Further, let's say that you want to remove duplicate rows (based on the Key) and keep the latest row (based on the Timestamp), with the additional complexity of there being duplicate Timestamps.

Your target result would consist of the following, reduced data:

Key,Value,Timestamp
K1,V4,2020-05-04
K2,V2,2020-05-04
K3,V1,2020-05-04
K4,V4,2020-05-04
K5,V4,2020-05-01

To achieve this result, you can generalize the following approach, in reference to the sample dataflow included in its entirety at the bottom of this post.

1. Get the dataset ;-)
2. computeRelative (computePartitionCounter)
• Partition by the Key
• Order by the Timestamp (ascending)
• Compute a partitionCounter column (number, default = 1)
• saqlExpression: previous(partitionCounter) + 1
3. computeRelative (computeKeepRow)
• Partition by the Key
• Order by the partitionCounter (descending)
• Note that weather you sort by descending timestamp and then ascending partitionCounter, or vice versa (as I have done here) is immaterial
• Compute a keepRow column (text)
• saqlExpresion: case when current(partitionCounter) == first(partitionCounter) then "true" else "false" end
• Note that in the case of K2 and K5, since some or all of the rows have the same Timestamp, keepRow will be "true" for one of them, arbitrarily, more than likely based on any internal ordering with Einstein Analytics. 
4. Filter by keepRow == "true"
5. Slice off parititionCounter and keepRow
6. Register the de-duplicated dataset

In my example, I went from this...

 

...to this...

Cheers! 

Dataflow:

{
  "getDataset": {
    "action": "edgemart",
    "parameters": {
      "alias": "KeyValueTimestamp"
    }
  },
  "computePartitionCounter": {
    "action": "computeRelative",
    "parameters": {
      "source": "getDataset",
      "computedFields": [
        {
          "name": "partitionCounter",
          "label": "partitionCounter",
          "expression": {
            "saqlExpression": "previous(partitionCounter) + 1",
            "type": "Numeric",
            "scale": 2,
            "default": "1"
          }
        }
      ],
      "orderBy": [
        {
          "name": "Timestamp",
          "direction": "asc"
        }
      ],
      "partitionBy": [
        "Key"
      ]
    }
  },
  "computeKeepRow": {
    "action": "computeRelative",
    "parameters": {
      "source": "computePartitionCounter",
      "computedFields": [
        {
          "name": "keepRow",
          "label": "keepRow",
          "expression": {
            "saqlExpression": "case when current(partitionCounter) == first(partitionCounter) then \"true\" else \"false\" end",
            "type": "Text"
          }
        }
      ],
      "orderBy": [
        {
          "name": "partitionCounter",
          "direction": "desc"
        }
      ],
      "partitionBy": [
        "Key"
      ]
    }
  },
  "filterDataset": {
    "action": "filter",
    "parameters": {
      "source": "computeKeepRow",
      "saqlFilter": "keepRow == \"true\""
    }
  },
  "sliceDataset": {
    "action": "sliceDataset",
    "parameters": {
      "mode": "drop",
      "fields": [
        {
          "name": "keepRow"
        },
        {
          "name": "partitionCounter"
        }
      ],
      "source": "filterDataset"
    }
  },
  "registerDataset": {
    "action": "sfdcRegister",
    "parameters": {
      "alias": "KeyValueTimestampDeduplicated",
      "name": "KeyValueTimestampDeduplicated",
      "source": "sliceDataset"
    }
  }
}

Conscientious Courtesy

I am seeing a lot of posts about ‘government control’, ‘mask Nazis’, ‘media fear mongers’, etc, as it pertains to the practice of wearing masks in public, on flights, etc. To me, it isn’t about any of those mechanisms or motivators. It is about courtesy. Not just the courtesy of protecting the health of others, but courtesy in kind. For me, this morning was a perfect example. I finally got an appointment at the groomer for Maggie (my dog), who has been shedding balls of wool lately. I had no notion of wearing my mask to drop her off, but when I saw the person receiving dogs wearing a mask, I put mine on as well. This is the new social grace, and, as surely as there is no mind control at play when I respond “good morning” in kind, there is no mind control at play here either. Of course, I would hate to be fined for not saying “good morning”, but the lack of a greeting in kind cannot possibly make someone sick, whereas not wearing a mask presents that risk, however small, so a different level of conscientious courtesy is required. Remember that.