Analysts under time and deadline pressure need to know that the information distilled by an AI solution is relevant to their context and is not simply the result of key word searches.  Understanding context is foundational to the collaborative AI solution offered by our partner BEA.

In another guest article, Tom Marsh, CTO at Boulder Equity Analytics (BEA), talks about context and why, without it, computer algorithms will never be able to truly know “you”.

Since Robert Scoble and Shel Israel just released their new book “The Fourth Transformation”, I decided to revisit “Age of Context“, a global survey of the contributions to the forces influencing technology.  The five forces were mobile, social media, data, sensors and location.  Scoble called these the “five forces of context”, the future of computing.  The five forces are still there but hardly tamed and in the rear view mirror.

Revisiting this topic three years later (see my ai-one post), there is still a lot of work to do in mainstream applications of cognitive computing.  For our BEA clients, it remains a critical challenge to building analytics for analysts under time and deadline pressure, faced with exploding amounts of information.

Why is context so important

First, context is fundamental to our ability to understand the text we’re reading and the world we live in.  When reading a sentence, you draw on the semantics of the words, the sentence, the paragraph, the context of the page, chapter, book and prior works or conversations.  Added to your ability to understand the sentence is your education and experience, and your reasons and objectives for reading it.  This diagram from Chris Campion’s blog is instructive in this regard.

Second, if you broaden the challenge it overlaps with personal intelligent agents (Siri, Alexa, Cortana, Google Now), the bigger problem of complexity.  The inability to provide context has always made it difficult for computers and people to understand each other.  Three years ago Scoble felt Google Glass could be that enabling breakthrough and now maybe VR will be the answer.

People and the language used to describe the world is a complex system.  No matter how much data is crunched or how sophisticated the technology (e.g., new generation convolutional neural nets), you can’t be reduced to an algorithm.  Claiming personalization, these tools create approximations of you based on people like you, not a true you.

There’s a difference, especially when applied to automating the decisions of experts in investing, reinsurance contracts or road-maps for new technologies. The intelligent agents that make those decisions need to have a complete model of a complex world.

The five forces of context as the foundation

Pete Mortensen also addressed the problem of context at the same time in his article “The Future of Technology Isn’t Mobile, It’s Contextual.”  Mortensen argues that the five forces are finally giving computers the foundational information needed to understand “your context” and that context is expressed in four graphs.  These data graphs are
 
•    Social (friends, family and colleagues)
•    Interest (likes & purchases)
•    Behavior (what you do & where)
•    Personal (beliefs & values)
 
At BEA, we build the foundations for context by extracting Mortensen’s graphs from the complex data generated by your digital activity, your domain and a complete view of the material you’re researching.  We’ve been using our technology to process and store unstructured and structured text in diverse domains to deliver a representation of that knowledge through powerful visualizations of the models that before now existed only on the page and in your head. 

The five forces of context – and beyond

So first, we get the context right.  Then we add metadata and attributes using NLP, entity extraction, sentiment and other proprietary linguistic algorithms.  This is processed, indexed and stored via python middle-ware that allows us to extract all possible attributes for each paragraph. 

Running in the cloud and integrated with big data sources and ecosystems of existing APIs and applications, we can quickly create and test your investment models or add intelligence to old ones.  Our interactive Tableau visualizations respect the analyst’s expertise, neutralize human bias while acknowledging the limitations of the technology, never delivering “black box” results that aren’t transparent and verifiable.

You work with concepts and use context constantly in your offline life.  We work to bring the same intuitive and human experience to your work as an analyst, without compromising your privacy or creating an approximation of you from the data of others.
 

Tom

@tom_semantic

KDD Analytics and Boulder Equity Analytics are partnering to deliver collaborative artificial intelligence to the financial and competitive analysis industries.

The post Context, Then Concepts, Words Last appeared first on KDD Analytics.

Language and the human brain

How many geese in this picture?  You didn’t need to count. Your answer probably contained the words “approximately”, “average”, “almost”, “sort of”, “guesstimate” or “about”.  One of the most powerful features of your brain is that it does not treat language as math, a series of binary yes or no formal constructs.  Humans are masters of writing the same idea in many ways, understanding what you meant even if you didn’t say it perfectly.  You also know when someone is being so careful with their words, they’re lying (we’re all tested on this one daily). This critical skill is used by analysts all the time.

Many technical approaches to textual analysis try to convert sentences into math or use the presence of specific words to return a yes or no result.  Our approach at BEA overcomes this issue with software agents that compare arrays (models) of patterns and return “similarity scores”.  These normalized scores allow us to determine whether a paragraph is about the same as another and by how much. 

In my last post, I discussed the nature and definition of concepts and how our solution is built to find them. Whether it’s a concept you’ve created or an example you’ve found in a document, if you can’t describe it you can’t find it.

Describing concepts

You can spot a concept when you read one, but learning to describe one isn’t as easy as it sounds and its rarely exact. Our AI platform enables us to create and teach “intelligence agents” to “read” documents to score similarity to concepts.

The first step in the process is to provide examples in a simple text box. This is surprisingly difficult the first time because our brain is never reading text without injecting our own education, bias and assumptions into the process.  Unfortunately the software only sees exactly what you provide it, no more, no less. When one user asked us how to teach the concept of “fraud” to our agents, we had to take a step back.

Risk and opportunity in legal issues

In the financial filings we focus on, concepts are often expressed indirectly, through a description of the act or its consequences.  The objective of the analyst is to find indications of wrongdoing or their cover up in company filings, earnings calls, news and social media. The critical evidence is never a clear statement like “I just gave my friend inside information on our earnings announcement so they could trade ahead of our disclosure.”

People don’t recite the definition of the crime when they talk about it. The language used is always more subtle and disguised. It also varies dramatically from one context to another (email vs. interview transcript for example). Tweets and text messages are full of acronyms, slang, phrases and partial sentences.

The guilty party is usually aware of the act and tries to avoid being discovered. He is more likely to say, “Hi Dave, here are some stats you might find interesting.” Is he talking about the company earnings report or his fantasy football team? If he says, “we’ll have a big surprise for you tomorrow”, is it a surprise birthday party or a merger announcement?

Kant’s tree concept (again)

To underscore this point, let’s revisit the example of the tree concept from my last post. The concept of a tree abstracted from descriptions of many trees is clear enough. The challenge in language analytics and many (most) language processing problems is that we are looking for the indirect effect or consequences resulting from the existence or actions of the tree.

This is easier to understand with examples. “The fall colors in New England are beautiful this time of year”.  Or “We need to get some shade for the yard at the nursery.” The concept of the tree is there but if I was searching for trunks, roots, branches and leaves, the “criminal” tree would escape detection!

Context is key

The context of the concept is critical.  Defining the context for the software can be complex.  But in most cases the patterns are there once you clear away the noise.

Capturing context can be as simple as setting filters.  For example, documents between specific individuals during specific time periods during which they had access to the information, resources and counterparts needed to commit wrongdoing.  Metadata from documents and entity attributes like organization, location and titles are commonly used for this purpose.

Where possible, we extract context from the source documents (and the source) to ensure that the more complex contextual factors are incorporated automatically into the intelligence agent. For example, the same person in the same period does not use the same language on twitter as they do in email.  Our powerful “almost engine” is what makes our system resilient and with our user in charge of how tight to set the “almost” meter, it adapts to a range of problems.

Is “big data” the answer?

We are all guilty of injecting our own bias and filters into understanding language. Good technical solutions capture the richness and subtlety in the context of the communications and ensure consistency of review.  This is not a problem solved by more and bigger data. 

We believe analysts have more than enough information.  They just don’t have enough time to find what matters.  BEA can help with that.

Want to know how we would use “almost” to solve your problems?  Drop us a line.

Tom @tom_semantic

KDD Analytics and Boulder Equity Analytics are partnering to deliver collaborative artificial intelligence to the financial and competitive analysis industries.

The post Good Concept Detection Requires an “Almost Engine” appeared first on KDD Analytics.

Our partner at BEA takes textual analysis a step further and teaches artificial intelligence (AI) software to find concepts and themes, not just words.  It’s one thing to find all occurrences of the word “decline” in an earnings call transcript.  It is another thing altogether to understand the concept of “decline” within the context of a paragraph.  Is it a decline in sales?  or a decline in bad accounts?

In another guest article, Tom Marsh, CTO at Boulder Equity Analytics (BEA), talks about keyword vs. theme detection and why “concepts are key, not words”.

A critical skill for the analyst during earnings season is detecting changes in the key indicators or themes for a company and its peers. Keyword detection is often passed off as theme detection but it’s not and the difference is critical.  Here at BEA, teaching software (AI) to find themes buried in SEC filings, earnings calls and press coverage from investor relations is a critical technology advantage.

First, understand the terms. Analysts tell us that with all the buzzwords and claims by vendors, it’s hard to understand the difference between real and apparent performance.  For us, the goal is to replicate an expert analyst’s ability to read and understand a document, whether its a filing, earnings call or interview.

While there’s more, in this post I want to make sure we understand each other when we use the term “theme”, “topic” or “concept”.

What is a “concept”?

Since we claim to teach software agents to find “concepts”, let’s check the definition of the term “concept” to make sure we are using it correctly. While I didn’t expect this to lead me all the way back to Philosophy class with references to Kant, Locke, Mill etc., our approach and use of this term are fundamentally consistent with the excerpts below from Wikipedia.

Concept– definition from Wikipedia

A concept is a general idea, or something conceived in the mind.

Notable definitions:

John Locke‘s description of a general idea corresponds to a description of a concept. According to Locke, a general idea is created by abstracting, drawing away, or removing the uncommon characteristic or characteristics from several particular ideas. The remaining common characteristic is that which is similar to all of the different individuals.

John Stuart Mill argued that general conceptions are formed through abstraction. A general conception is the common element among the many images of members of a class. “…When we form a set of phenomena into a class, that is, when we compare them with one another to ascertain in what they agree, some general conception is implied in this mental operation” (A System of Logic, Book IV, Ch. II).

Philosopher Arthur Schopenhauer argued that concepts are “mere abstractions from what is known through intuitive perception, and they have arisen from our arbitrarily thinking away or dropping of some qualities and our retention of others.” (Parerga and Paralipomena, Vol. I, “Sketch of a History of the Ideal and the Real”).

By contrast to the above philosophers, Immanuel Kant held that the account of the concept as an abstraction of experience is only partly correct. He called those concepts that result from abstraction “a posteriori concepts”.

A concept is a common feature or characteristic. Kant investigated the way that empirical a posteriori concepts are created.

“The logical acts of the understanding by which concepts are generated as to their form are:

1. comparison, i.e., the likening of mental images to one another in relation to the unity of consciousness;

2. reflection, i.e., the going back over different mental images, how they can be comprehended in one consciousness; and finally

3. abstraction or the segregation of everything else by which the mental images differ …

In order to make our mental images into concepts, one must thus be able to compare, reflect, and abstract, for these three logical operations of the understanding are essential and general conditions of generating any concept whatever. For example, I see a fir, a willow, and a linden. In firstly comparing these objects, I notice that they are different from one another in respect of trunk, branches, leaves, and the like; further, however, I reflect only on what they have in common, the trunk, the branches, the leaves themselves, and abstract from their size, shape, and so forth; thus I gain a concept of a tree.”

— Logic, §6

Optimization of AI software

We worked with our ai-one partner to optimize their AI for this task. 

At the core, our application processes each line of text much the way our brains do it, learning the patterns of language, the “key” words, their importance and the words most closely associated with them. The AI provides commands to extract as an array those key words and associations, their direction and values (strengths).

While the ability to score the similarity of concepts is important, my observation from years of applying it to problems from NASA (Topic Mapping pg.198) to SwissRe is that it’s even more proficient at filtering out the noise, giving lowest values to the unimportant words and associations.

Filtering is fundamental to our brain’s ability to find the topic that’s important.

Locke describes a concept as an idea “created by abstracting, drawing away, or removing the uncommon characteristic or characteristics”. This is very close to the way our solution builds a model of a concept after learning from the examples provided to teach it. The fingerprint we extract from that text is an array that represents the concept in the same way. The similarity score for that comparison is a powerful attribute we use in a number of ways to deliver a great user experience.

Building powerful qualitative analytics for financial analysts and investors starts with the right core technology. Finding concepts buried inside documents is the first part and foundation of extracting actionable insight.

Now that I think about it, maybe Philosophy 101 wasn’t a liberal arts waste of money after all.

Tom

@tom_semantic

KDD Analytics and Boulder Equity Analytics are partnering to deliver collaborative artificial intelligence to the financial and competitive analysis industries.

The post Concepts are Key, Not Words appeared first on KDD Analytics.

At Boulder Equity Analytics (BEA), “Our mission is to build an intelligent, enriched and fully interactive database from all the publicly available reports to improve the productivity and insight of the analyst covering an industry sector.”  We call it O.A.K.L.E.Y, collaborative artificial intelligence, a 2nd order of intelligence.  And we’re now creating the services around it.

Convenient, comparative, contextual and interactive services like ours will provide a foundation and no doubt be successful.  But that’s not our end goal.  

Competitive

We build tools for competitive analysis.  But more importantly we want to build tools that help you compete, in your job and in your markets. We can’t stop with a better version of what you can get elsewhere.  While there might be a market for a less expensive Bloomberg, what’s the competitive differentiator for you?  You have the same service as the rest of the market.  Which brings us to an interesting dilemma.  If we build a great service, after the early adopters exploit the advantage, everyone is equal again.

This is where our AI needs to augment your intelligence, not supply its own.

1st order intelligence

With our first products we’re using AI to “read” the market and company information so you can be more efficient in your research. This is the 1st order of intelligence.  It is like going to school to learn finance and then working the back room reading and reporting on company results.  It is the foundation for the next step in your career, using that knowledge to make investment decisions.

As the impact of MIFID II hits, our products will help you build a bigger and faster knowledge base with no bias and 100% recall at web speed and scale. Our products don’t get tired. That alone will help you beat the market.

But here’s where most take the easy way out: focus on speed and trades, not investing.  High-frequency trading (HFT) uses AI  It is in fact one of the most pervasive and advanced AIs in the world, second only to computer viruses and cyber bots.

This is a common execution of an AI strategy.  Go straight from the research department to the 3rd order of intelligence, an AI making autonomous decisions.  Think IBM Watson.  No humans required.

We think that’s the wrong approach. Black boxes powered by AI are as risky as they are powerful.

2nd order intelligence

Our goal is to deliver collaborative AI, a 2nd order of machine intelligence.  Our soon to be announced Boulder Earnings Call Tracker (BECT) for Investor Relations, Analysts, Portfolio Managers and Independent Research is our first offering at the next level.  

Simply, the 2nd order is collaborative and pulls out multiple dimensions of sentiment and tone built with new analytics methods that provide the nuance missing from today’s offerings.  It enables you to create completely new strategies once you’ve been able to look at your markets and competitors at a deeper level and at machine speed and scale.

In the past, long term was long term and even when you were right, it may have been years before the market rewarded you for your positions. Information was created, reported and analyzed from the paradigm of quarterly reports and press releases.  Your technology, economic and geopolitical models were relatively fixed.  Sectors followed predictable patterns.

We think that has changed.  

Markets have moved to the “second half of the chessboard” where exponential changes will create and destroy large enterprises (wealth) in quarters not decades.  What are considered fundamental long term investment decisions today, will look like day trades in the future.

Our goal is to bring the 2nd order of intelligence to you, providing a foundation for contextual decisions.  This will empower you with individualized tools to compete and win, whether against a machine or another trader. 

You won’t be able to read fast enough.  You need to retool.  And the first ones to do it will win.  In the immortal words of Walter Gretzky, as passed on through his son Wayne: “Skate to where the puck is going, not where it has been.” 

Tom 

@tom_semantic

KDD Analytics and Boulder Equity Analytics are partnering to deliver collaborative artificial intelligence to the financial and competitive analysis industries.

The post Read Faster? Researchers Need to Retool to Compete appeared first on KDD Analytics.