Counsel of the Wise

One of the consistent recommendations to innovators is to seek the counsel of some very smart people who understand the field of the innovation. One must be open to objective critical feedback in order to determine if the innovation should be developed. Initially, I started with family and friends, but recently I sought the feedback and input of academics and businesspeople with whom I was not formerly acquainted. So far the response has been very positive. In fact, four of those smart people have volunteered to become members of the project's board of advisers.

In addition to all of the great feedback from those meetings and follow-up discussions, the board has another important function - as references. Anyone who would so easily dismiss my claims of a novel disruptive internet paradigm, or the "preference engine" technology, must first reconcile the existence of this volunteer advisory board. My hope is that the added legitimacy of the board will further my contacts in the field, and especially among expert developers and angel investors. Here is the current board that can be contacted for references:

• Avi Noy, Ph.D. [Research Page, LinkedIn]
  Adjunct Lecturer, Graduate School of Business, University of Haifa
  PhD, Information Systems, The University of Haifa
  MBA, Technion Institute, Haifa
  LLB, The University of Haifa
  BSc, Computer Engineering, Cum Laude, Technion Institute
  
• Doron Erblich [LinkedIn]
  CEO, Noya Software, Haifa
  Center of Computing and Information Systems (Mamram), Israel
  BA, Computer Science, Open University, Israel
  
• Yosi Dagan, MBA [LinkedIn]
  MBA, Heriot-Watt University, Edinburgh, Scotland
  BSc, Electrical Engineering, Ben Gurion University
  
• Yuval Dan-Gur, Ph.D. [Work Page, LinkedIn]
  VP Engineering, RADA Electronic Industries
  Winner of the Israel National Security Prize
  PhD, Information Systems, The University of Haifa
  MsC, Industrial and Management Engineering, Technion Institute
  BSc, Electrical Engineering, Technion Institute
  

The Serendipity Revolution

Traditionally, the success of recommender systems is evaluated by predicting accuracy of recommendations off-line using existing datasets. For example, see the million dollar Netflix prize for a meager 10% improvement of their collaborative filtering algorithm. Netflix provided access to 100 million of its customers’ movie ratings to train new algorithms and test them. In other words, the algorithm is judged more accurate the more it recommends movies the user has already seen. Recommendations based upon this traditional accuracy metric are not the most useful to users.

Researchers know that success of recommendations is better measured by recording user satisfaction - the positive emotional response at having discovered something new that one likes. But that is more difficult to measure - as it requires a community of users and a useful mechanism to compel (or at least strongly encourage) the reporting of satisfaction, it's strength and perhaps type. Satisfaction of recommendations seems to follow in ascending order of the following recommendation types:

1. Low quality, low accuracy recommendation. Users obviously don't appreciate having their time wasted in evaluating something that the system should have known the user would not be likely to appreciate. These are "trust-busters"; the user will lose trust in the system.
2. An accurate, but known recommendation. An item the user is already aware of. The user likes the item, but it is not novel. Trust is maintained because at least the system recommended something that the user already likes. Too many of these recommendations imply an excess number of false-negatives or "missed opportunities".
3. A novel, but obvious recommendation. A novel recommendation is something new and appreciated, but something the user would have discovered on his/her own. For example, a new song from a favorite musician, or a new movie from a favorite director. The user will have a positive, though muted, reaction. Many users will suspect that there were "missed opportunities", given the huge number of unfamiliar items in any domain.
4. A serendipitous recommendation. A serendipitous recommendation is something new, non-obvious and appreciated that the user would likely not have discovered on his/her own. For example, an unfamiliar song from an unfamiliar musician, or a unfamiliar movie from an unfamiliar director. The user will likely have a very positive reaction, though it has been argued that, in some users, such recommendations may be seen as obscure and not immediately appreciated.

The serendipitous recommendation is obviously the ideal for most users, the problem is that collaborative filters tend to focus on what is commonly known and popular - items that the user has heard about or items that the user would have experienced eventually because of their "blockbuster nature". Many of the most interesting items for the user may be buried in the "long tail", so some collaborative filtering systems have attempted to tweak their algorithms to try to maximize this type of recommendation by reducing the more popular recommendations. Even so, recommendation diversity tends to be reduced in collaborative filtering systems, leading to a large number of false-negatives or "missed opportunities".

Recommendations based on a user's core identity will not focus on the popular, or items from artists or directors the user likes, or that the user's friends like. Instead, the user will be recommended items from the entire item landscape that by definition the user is most likely to appreciate based on that core identity (their "preference engine"). Thus the recommendation diversity (coverage of item space) within a domain (such as music) is as large as the diversity of items within that domain, leading to a large number of serendipitous recommendations - possibly the vast majority. Keep in mind that the number of domains in our community is also unlimited, and the same core identity can be used to recommend anything and everything in life.

Forget Everything You Know

I'm getting a relatively large amount of email feedback, now that I'm putting a little effort into PR - and much of the feedback reflects existing biases. Recommender and social technologies have largely plateaued, with new innovations being relatively minor and more fine-tuning in nature. The possibility that something is being developed that is completely new, disruptive, and significantly better than existing services is not something many in the field are willing to consider. Admittedly, the secrecy of this project has not helped to convince people of that possibility, but this emotional rejection often takes the form of certain mistaken assumptions.

So I'll make another attempt to preempt the preconceptions and misconceptions with a discussion of some of the mistaken assumptions, both technical and paradigm-related, from the feedback. We can call this the Frequently Made Assumptions (FMA) list:

There are so many social networks, nobody wants another one.

Our system is not a social net, but one effect is that it creates a completely new community paradigm. Immediately after registration, we can quantify a person’s relationship to every other person, item, idea, endeavor, etc. in the system. Those relationships emerge and are presented to the new user as options and suggestions that then leads to further exploration, enjoyment, socializing, purchases and perhaps community involvement. So you don’t NEED to have (or get) your friends or contacts on the system to derive value.

There are so many recommender sites, and the results are lukewarm.

Instead of looking at your past choices and making certain assumptions, we are looking at the CAUSE of your preferences. We call this your "preference engine" which yields results of far higher accuracy and quality than existing recommender systems. Much of the recommendations will be serendipitous and cover the "long tail" as well. We don't care what your friends prefer, or what your past choices were. We know what you will like - from the inside.

"Web 2.0" stuff like this has no business model beyond advertising. I can't see any concrete business or how you want to generate revenue from your invention.

Imagine that every user has immediate access to their future favorites of: people, music, movies, books, recreation, groups, products, services, ads, travel destinations, vocations, jobs, teams, politics, religion, ideas, websites, articles, news items, games, etc. Highly successful affiliations of all types, direct sales and downloads, and highly targeted advertising would be the obvious business model components. Note that even advertisements are objects in our system, and can be targeted with person-level granularity.

No idea is new, no matter how good it is, at least 50 other people have thought of it.

Show me.

The future of the internet is the Semantic Web, haven't you heard?

Yeah, I've been hearing that for a long long time. The semantic web is too primitive and highly problematic. Certain aspects will be adopted by some, but there will be no utopia there.

Are you trying to promote just an idea?

We have the paradigm, a test site, and three patent applications. The test site is the first part of the application, which I am developing in Ruby on Rails. It is operating currently and in the process of being tested by select individuals. I am trying to form a team to develop the second part, which will require some sophisticated social software development and efficient algorithms programming.

LinkedIn (or some other site) is doing the exact same thing!

No, LinkedIn is not doing anything even remotely similar. If it’s not clear what our system does, read on.

Why would visitors want to come to your site or spread the word?

Again, imagine that every user has immediate access to their future favorites of: people, music, movies, books, recreation, groups, products, services, ads, travel destinations, vocations, jobs, teams, politics, religion, ideas, websites, articles, news items, games, etc. The system itself acts as a good friend that knows you best, and understands all. A personal and trusted relationship will develop with each user. Users need not reveal any personal information, so privacy is intact. Add these to the benefits of existing recommender systems and social and professional networks, and I think you will agree that popularity will not be an issue.

Nobody likes unlimited connectivity, you are creating a flood of people/info that will overwhelm users.

This is about reducing information overload and the tyranny of choice – it’s not about creating unlimited connectivity. It’s about giving the user access to only the things he/she is most likely to appreciate: people and every other thing in life. This is the holy grail of internet futurists and marketers, etc. There will be no unwelcome flood – only welcome and surprising options.

People only want to interact with their primary network. They don't want to be bothered by strangers.

First, we are not trying to be another Social Network. Social Networks separate people as nodes or degrees of separation. Separation is not good if it keeps away ideal relationships. But removing those separations, while preventing a flood, requires new kinds of filtration. A user’s potential ultimate friends and romantic partners, ideal business partners, etc. most likely lie far outside of one’s immediate network. Our novel discovery techniques provide the filter, thus there is no flood – only the most ideal are presented as options for the user.

Are you just creating another matchmaking site?

Objects in our system also include people: we expect matchmaking to be a significant part of the interest in our system. But remember that people can also be matched with every other kind of object, so the system is much, much more. Also, user-user matching can also be used to discover friendships, business partnerships, roommates, travel buddies, etc.

Are you just creating a technology to sell to the Amazon.coms of this world so they can make better business?

The discovery engine is not separable from the community. The community must be created (possibly by aggregating or integration of existing communities), but it’s not a social network, nor just a recommendation engine. It’s a holistic and completely new paradigm, and it is destined to change the world.

Are you just creating yet another standalone dotcom, or will this be a feature of other services? Folks are fatigued with so many separate services.

The discovery engine is not separable from the community, however, the community need not be a standalone data island - the format is ideal for integration of multiple services. But even as a standalone entity, the novel paradigm will make it quite popular. Others will want to integrate and take advantage of our universal recommendation system.

You must be constructing a commonality map. This is not new.

There is no “commonality map”, no use of shared things to relate people. This is not a people-centric or shared object-centric social model, as everything within our system is an object. It is beyond the "object-centered" or "people-only" debate. You’ll have to put aside your knowledge of existing methods and community/recommender concepts in order to understand the new disruptive paradigm.

You must be using data mining edge activity to produce (more) accurate node profiles. This is not new.

Our system does not involve data mining. Data mining generates very poor quality information compared to the kind of information we get. As I said, ours is something completely new.

You must be using existing algorithms and filtering methods. Collaborative filtering? Artificial intelligence? Content Based? Horting?

Ours is a completely new paradigm, not horting or any of the other existing algorithms or systems. It is not a collaborative filter of any sort. It does not rely on personal or demographic data and it is not content based, requiring feature analysis. It is something completely novel.

Registration involves Psychological testing and interpretation.

Psychologists will be very interested in what we are doing, but we are not using Psychological testing and interpretation. There are no Rorschach inkblots or squiggles and shapes, no questionnaires, no puzzles, etc.

Registration involves some sort of metaphysical or astrological process.

No pseudoscience, metaphysics or mysticism, I promise. It is a completely logical and quantitative process. Hard to imagine, I know, but I assure you it is quite legitimate. In the future, PhD theses will be based on it.

Registration involves personal and demographic data used to enable audience segmentation.

No personal or demographic information needs to be entered at any time. Audience segmentation is a severe limitation to behavioral targeting and recommendations. Our system provides object-level granularity.

You fill in a sparse matrix to infer missing relationships and preferences.

The matrix is dense from the very start. No missing relationships are inferred and filled in.

Core identity is impossible to determine, and certainly not in a quick registration.

Don't judge what is impossible if you don't understand it. I would be happy to arrange a meeting with you to demonstrate how it works. If you are not willing to listen, then you deserve the box that imprisons you.

How can you derive someone's essential identity from a name and password?

Now THAT would be magic! Of course the answer is that we don't. There is a quick one-page registration and we get their essential identity from this - more than you can possibly imagine. But this is not the essential innovation, only a very cool result.

Why universal? Wouldn't it be easier to stick with a single vertical domain?

Our system prefers diversity, the more object types the better. It can operate on only one domain, but this would be a huge waste. Don’t forget, everything in life can be discovered, separately or simultaneously. The difference in complexity, between one domain and universality, is not great.

How are you going to compile a database of all the "objects" in the world? This is no trivial task!

Our staff will not have to compile anything (as with Pandora), nor are we relying on users to enter feature data. Whatever you're thinking - that ain't it. We are using a completely novel approach.

No matter how you do it, reaching critical mass is gonna be a bitch.

Ours is a slave to the network effect like all the others. However, as long as the system is sufficiently populated with objects of any variety (people OR things), the system will begin to work. In other words, with the right conditions the first user can see benefit. The obvious advantages over social nets, recommender applications, and the multitude of vertical applications should spread the word quickly. Most agree that popularity would not be an issue.

You don't reveal the "magic" on your blog, so the project must be hype and at least appears not to be serious.

First let me apologize for the lack of technical info in this blog. Certainly this will be off-putting to some, but I assure you that it is very serious. This is not an academic project - it is business, and the lack of technical information is simply a necessity for protecting confidential information. In the world of internet start-ups, innovations are rarely disclosed publicly before the first public beta site. Patents offer some protection, but there is no replacement for a healthy maintenance of secrecy at this early stage. I am happy to disclose the technical solutions to appropriate individuals under confidentiality assurances. Contact me at the email address in my profile to arrange a meeting in Israel or the States.

Why be so stealthy - this is not the Manhattan Project!

If this really IS the next internet revolution as we suspect, then we shouldn't treat is as just another cool web app. We’re planning to change the world.

I can't see the purpose of this blog.

Consider the articles in this blog to be teasers for the purpose of attracting interest from those with vision and curiosity. We are looking for investors and hackers (highly competent programmers and developers). Do you know any?

If the system performs as you say, then it will clearly be disruptive. But these claims are a bit extraordinary... why should I take time to arrange a meeting to learn more?

Everyone's gotta gamble with their time - weighing the potential upside, probabilities, and time spent. I know the frustration of wasting my time hearing a stupid idea. But natural curiosity keeps me listening, just in case. Contact me at the email address in my profile to arrange a meeting in Israel or the States and let me change your world. In fact, invite a group of the smartest web-focused people (including angel investors and hackers) you know.