Introduction

Universal Recommendations is a web technology that can predict a person's future favorites in any domain of life: social, career, products, services, media, etc. We do it cold, rapidly, without tracking a user's behavior, without data mining, and without a lengthy registration process. This completely novel technology is a product of cross-pollination between bioinformatics and web 3.0.


Jan 30, 2008

Skepticism vs. Cynicism

I just received a response email from a professor at a large American university:

"I'm sorry, but your presentation of your idea makes it sound like the equivalent of a perpetual motion machine, as you promise a secret ingredient that will figure out the core of who an individual is without essentially no data about them.

I wish you good luck, but I'm afraid I'm not willing to invest any further time in exploring your ideas. "

Now, I appreciate feedback and certainly don't want to discourage it - but I thought this was a good opportunity to review some of the objections to this project and the limited technical information disclosed. What can we draw from the above email?

  1. A perpetual motion machine would be revolutionary if it were true, and so would this project.
  2. A perpetual motion machine cannot exist because it violates the laws of physics, and so would this project violate our expectations about what is possible.
  3. Anyone proposing such a revolution/violation is not worthy of further consideration, and the ideas are not worthy of further exploration, even a meeting would be a waste of time.

I am a skeptic myself, and so I certainly appreciate skepticism, but our project does not violate any laws of physics. It sounds fantastic, but I assure you the algorithm is sound. The professor claims we have "essentially no data about them [users]" - though obviously we know a huge amount about a registering user. The difference is that we get information that can be used to quantify relationships, and the information can be gathered during a quick one-page registration. Teaching the machine about the "core of who an individual is", i.e. a person's 'preference engine', has never been done - and if this professor can't do it, certainly nobody can. Meetings with three local professors in the last two weeks produced the expected jaw-dropping effect, and because of them I may now have a linear algebra option for the core algorithm, though I'm still not convinced.

Anyway, this emailing professor has chosen not only not to meet, but seems annoyed at having been exposed to such ridiculous suggestions. I also understand a certain amount of frustration that the technical solutions, enabling full evaluation, are not disclosed (though I doubt the emailing professor would have read that far). Readers must also understand that technical solutions cannot be made public, despite patent-pending status. There are too many horror stories about disclosed innovations being stolen by large, well-lawyered corporations. I am willing to meet, traveling considerable distance, in order to satisfy that skepticism. Vision, curiosity and open-mindedness should be sufficient for a meeting, and perhaps this is a useful filter: why would I want to associate with those without these traits?

6 comments:

Jason M. Adams said...

Well I, for one, am dying to know how you're coming at this problem. Knowing the core of who an individual is obviously would be useful, but what I'm really curious to see is how to make that assessment and what help it provides. However, I'm probably of no value to you, being a lowly grad student. :)

Skepticism is necessary for science, so I think he must first doubt your claims. But letting skepticism turn into cynicism and closing himself off to new ideas is bad business. One of my professors listened to an idea from what you might call a crazy person, but that led to one of the best machine translation systems out there.

If you know your algorithm is sound, then his loss. In any case, I hope you can publish the key ideas at some point.

Andreas said...

Why not be transparent who the guy is who wrote to you? I think it is a bit broad to just discredit large american univesity profs in general.
Andreas Weigend
Stanford University
weigend.com

Steven Ruttenberg said...

Jason: you probably know others who might be interested - I need hackers and money. I'll be in Philly in May - if you can make the short trek, we can do coffee.

Andreas: it's a good policy not to "out" those who write to you by email. "discredit large american univesity profs"??? You're joking, right?

Jason M. Adams said...

May will probably be a good time for me. You have my info (I assume? if not you can find it on my blog), so drop me a note when the time gets closer and hopefully we can work something out.

Andy Hilal said...

I have a feeling that my "core identity" would come out to be something like "42" if looked at this way but it'd be interesting to try.

Like many things, this seems like it would have the best chance of being interesting as a feature built into one of the big social networking platforms, not as a new standalone site. There are a lot of great feature ideas out there, and not every one of them needs to be a standalone dotcom. Indeed, folks' fatigue with multiple services is worse than ever.

Not in your FAQ - how are you going to compile a database of all the "objects" in the world? If you want to recommend "music, movies, books, recreation, groups, products, services, ads, travel destinations, vocations, jobs, teams, politics, religion, ideas, websites, articles, news items, games," etc etc then you need to know about all of them. This is no trivial task, even for a straightforward vertical like Movies. Trust me, relying on users to enter the data won't work well. You'd need to dedupe and normalize all their disparate data in order to do any meaningful relationship mapping.

But hey, these are petty concerns compared with creating a "new community paradigm." Rock on, pilgrim. As you say: "Show me."

And one final note, if you're going out to seek money, you might prepare some more diplomatic responses for legitimate skepticism. Good luck telling a potential investor, "you lose, stay in the box" when he asks how you can derive someone's essential identity from a name and password. That's actually a completely valid question, and your dismissive response discredits you and makes your idea seem like a delusion of grandeur.

Steven Ruttenberg said...

Andy, this post was made for you!!! I don't blame you for your cynicism and anger as it is likely a result of deep and unfulfilled curiosity. I would love to meet with you and a group of the smartest web-focused people (angel investors and hackers) you know and satisfy that curiosity. It will be a meeting that will change your world. Now for the technical and conceptual misunderstandings:

1.Core identity is not something like "42", it is very sophisticated, with person-level granularity. You assume this is only a standalone data island, which is not necessarily true. The format is ideal for integration with multiple services, including possibly Epinions.com, where you work. But even as a standalone entity, the novel paradigm will make it quite popular. Technical skepticism aside for the moment, the paradigm is clearly the next internet revolution. Others will have to integrate or fade away.

2. Our staff will not have to compile anything (like Pandora), nor are we relying on users to enter feature data. Whatever you're thinking - that ain't it.

3. I would be happy to "Show you". Make the meeting.

4. Regarding diplomatic responses, we are not "deriving someone's essential identity from a name and password". Where did you get this notion? You should read before you comment. There is a quick one-page registration. Yes, we get their essential identity from this, more than you can possibly imagine. But this is not the essential innovation, only a very cool result. Anyone whose cynicism is so great that they will criticize as "impossible" what they don't understand, and then refuse to even make a meeting to give a chance to the possibility that they are wrong... this is a person that deserves to stay in their box. I don't want anything to do with someone like this. I hope you are not stuck like the professor.

Everyone's gotta gamble with their time - weighing the potential upside, probabilities, and time spent. I know the frustration of wasting my time hearing a stupid idea. But natural curiosity keeps me listening, just in case.