We’re both inside companies and outside them. The boundaries that separate our conversations look like the Berlin Wall today, but they’re really just an annoyance. We know they’re coming down. We’re going to work from both sides to take them down.
I agree. I experience these annoyances on a daily basis. Sometimes I have to really constrain myself not to let go of my temper, because I feel that our insights in many ways far preceed our ability to apply these to practical use. For instance, I cannot understand that while I do 95% of my banking via the internet, most banks do not put 95% of their ressources to work to give me as a customer the best possible online banking experience. Even less would probably do. If they just put 80% behind it, that would probably suffice. But they don’t. I am also annoyed when I have to communicate with my daughter’s doctor via an online form which permits only a limited amount of characters, largely because they do not trust email as a means of communication. In fact, I am always annoyed when people who presumably wants to communicate with me, don’t let me communicate back on equal terms. I find that arrogant. As far as possible I resist their attempts to control the way in which I should communicate with them.
The rooms in which we speak
Architectures are important. They are the ways we construct the rooms in which we speak. The “conversations” of The Cluetrain Manifesto take place within the framework of such architectures. They have names such as Facebook, Twitter and Google. Other such architectures are called things like WordPress, Joomla, MediaWiki, Firefox, RSS and GNU/Linux. They have a tremendous impact on the ways we communicate online, on the ways in which we filter our incoming information streams, and on the ways we enable new connections and enable new ideas to reach others, and enable their ideas to reach us. As important as architecture is, so more important is ownership : that we claim ownership to the tools we use. That we claim ownership to the channels and the walls that decide who will learn to know us, who will receive our message, and who will be filtered out, who will not. We decide what walls are torn down and what are built. With the web and simple tools we can, and we do.
Many of the software architectures that we employ, from the webserver and webscripting functionality of Apache and PHP to the popular self-publishing power of tools such as WordPress, are free software. I.e. built and easily adaptable by anyone who wants to adapt them for particular purposes.
Walled gardens
Other architectures are walled gardens, maintained by organizations and companies, who are not concerned about the choice of their customers. While companies such as Facebook, Twitter and Google offer greatly useful applications to their customers, their services impose limits on their use. In short, they choose to remain in control. They choose not to release the source code. Not to let their users adapt the tools they’re offering to their particular purpose.
If a company such as Facebook or Twitter goes bankrupt, users will lose their data – no compensation, no anything. There’s no obvious way to retrieve data from these services, and since the code is not free, one can’t write tools to retrieve those data by oneself. While most of these services offer useful and advanced interfaces so that programmers can access their data from the outside, the service stays in control. You can’t obtain access to data they don’t want you to obtain access to. Facebook ultimately decides who they like to write applications for “their” platform. Twitter abolishes user accounts at their whim, because ultimately Twitter decides. Ultimately, Google decides to pull the plug on a GMail or YouTube account, on grounds they choose, not their customers.
These walls of proprietary ownership are the Berlin walls of today. We meet them everywhere, when we are annoyed we can’t do certain things with the tools we use. When we communicate within the confines of architectures that we do not own and do not feel comfortable with, because they disallow us to be ourselves. In the worst case, we hit them head on when we find that our account on some service has been abolished unfairly, with nothing we can say to get our data back. When a service ceases to be in business, a product ceases to be supported, or a new company policy is enforced in spite of what we feel about it.
So how are these walls going to come crumbling down?
Free software
As I do here and have often argued, the only way we can operate freely in our online environments is if and when we ourselves are able to create, adapt, control and empower the architectures that we employ. We need our software and online services to be as easily adaptable as any article on Wikipedia. Wikipedia is enabled by the clever use of a particular architecture in combination with a copyright in reverse known as “copyleft”. The GNU Free Documentation License (GFDL) license ensures that every contribution to Wikipedia’s articles can be freely adapted and re-distributed by others.
Until now, free software have also relied on copyright. Similar to Wikipedia’s license, the General Public License (GPL) which is commonly used for most free software projects ensures that the code stays open and can be manipulated by anyone, no matter who distributes or sells it.
But free software need not depend on copyright. The greatest barrier to the spread of free software is that so many do not understand why it is important. Too many business executives cannot see, that it is as beneficial to them as to their customers, that they facilitate their customers’ ability to change and adapt the code. Too many organizations do not understand that releasing their source code opens up new, decentralized, flexible and less costly ways to organize their activities. And too many internet users (myself included) are too convenient with their habitual uses of proprietary online tools to question deeply and realize what’s at stake. We also find time to be a scarce good, since we also have to work to pay our bills – often inside companies led by execs who don’t “get it”.
How things look from the inside
The free software movement is “working on the outside” to bring down these walls. But on the inside, every Facebook, Google and Twitter employee is also an internet user and customer. They are people who talk using these same tools, they have other lives, they quit and start their own businesses, in short they engage in conversations where they go (or are allowed to go, by their companies). What limitations in ownership are put in place by their companies also limit their ability to deliver the best possible product, the best possible service and the best way to help solve their customer’s problems. They are equally annoyed by the corporate walls put in place beyond their control.
There are two great problems which faces the walled corporations, now and in the future :
1. They will increasingly encounter free architectures and services, which may yet perform poorly, but have much greater potential to outgrow and outperform their proprietary competitors.
2. As clever candidates everywhere discover their own ability to build and employ free architectures of their own choosing and flavour and adapt them to suit their own particular purposes, companies will find it increasingly harder to find qualified candidates to fill positions. What’s attractive in working under the command of a boss, if you can work for yourself? What’s the attraction in working for a company, whose business model is not adaptable to the open environments spreading on the web?
What if Facebook went GPL?
In closing, let me speculate aloud to show an example of the business landscape I believe will replace the walled gardens of today’s corporate environment. Among many online applications, Facebook is probably the service with which I have the most problematic relationship. There’s no doubt in my mind that Facebook does something very well : it helps facilitate connections and conversations. It helps me get in touch and stay in touch with family, friends and business contacts who wouldn’t otherwise read my blog or relate to me via other online tools. It works really great for friends you don’t see a lot on a daily basis, but still want to stay in touch with. But for all it’s merits, I hate the fact that I can’t easily search and access data in Facebook. I dislike that I can’t extract the information I need with RSS from my Facebook archives, and that I cannot play even further with the category layers, to adapt the service even further to suit my needs.
Let’s imagine that Facebook decides to go open source. Facebook releases it’s source code and invites developers to join in and contribute to the code. They still are leading the development of the core Facebook application, but also offer anyone a downloadable package, which can be freely modified and redistributed. Anyone is free to fork Facebook and set up a rival site.
What would happen then?
First, we’d fix everything that is wrong with it. We’d add RSS feeds to all the places where we’d like RSS feeds. We’d work to make what’s going on transparent, so that we could learn from it. And we’d make those changes publically available to anyone, who’d like to take a look and use them for their own purposes.
But what I think makes Facebook really brilliant as a free software package is the way it can adopt external applications within itself. Facebook as a general purpose communications platform is great and extremely adaptable. This makes it well suited as a platform for almost any corporate website. Most companies need to enable conversations across the organization, with suppliers, customers, investors, and everyone else slightly related to the company. The fact that most companies’ employees already spend a good deal of time using Facebook during work hours shouldn’t lead to abolishing and blocking Facebook from office computers, but should rather be seen as an encouragement to take this brilliant tool and give it a form of their own choosing. If Facebook was released under GPL, that would indeed be a viable option.
Adopting Facebook as a corporate platform would not only allow employees and customers to communicate on equal footing, it would also allow them to create applications for the platform, which would help adapt the package in very particular, employee- and customer-centric ways to suit the company’s purposes. That the package already has proven so scalable on a global level is a testament to it’s robustness in even the most trying of corporate environments.
But even if Facebook is not released under GPL, we’re already well on our way to build, use and sustain software like this, and many businesses do build their own social networking architectures. In fact, many CMS packages which are free software already implement features which mirror the successful features of Facebook and other social networking services. For WordPress, we already have BuddyPress, a prebundled collection of plugins which convert a WordPress installation to a fullblown social networking site.
But businesses and developers will continue to get it wrong, if they do not offer their employees, members and customers the same freedoms by releasing their source code, as they had when they chose to base their solution on free software.
In the long term, the question is, if Facebook and other proprietary businesses will still have a business model, if they do not give up control and release their code? If they do not enable the free adaptability of their software, chances are, with time, we’ll just build our own.
Torrent index sites like The Pirate Bay are often compared to search engines such as Google in that both offer vast indexes of information, and both give easy access to unauthorized copies of copyrighted material.
One thing which surfaced during the Pirate Bay trial in late February was IFPI’s cooperation with Google and other search services in their battles against copyright infringement. When IFPI’s representative John Kennedy was asked why they sued The Pirate Bay and not Google (as in “or any other major information filtering service using the internet”), the answer was that Google cooperated, and The Pirate Bay didn’t :
When asked about the differences between TPB and Google, Kennedy said there is no comparison. “We talk to Google all the time about preventing piracy. If you go to Google and type in Coldplay you get 40 million results – press stories, legal Coldplay music, review, appraisals of concerts/records. If you go to Pirate Bay you will get less than 1000 results, all of which give you access to illegal music or videos. Unfortunately The Pirate Bay does what it says in its description and its main aim is to make available unauthorized material. It filters fake material, it authorizes, it induces.”
(…) Kennedy was asked why they haven’t sued Google the same way as TPB. He said that Google said they would partner IFPI in fighting piracy and he has a team of 10 people working with Google every day, and if Google hadn’t announced they were a partner, IFPI would have sued them too.
I think the truth of the matter is, that Google’s business is based on copyright infringement from the start. When Brin and Page started Google, they started by downloading the entire internet and offering their index of it online. In the words of Larry Page himself, in David Vise’s The Google Story :
Google was started when Sergey and I were Ph.D. students at Stanford University in computer science, and we didn’t know exactly what we wanted to do. I got this crazy idea that I was going to download the entire Web onto my computer. I told my advisor that it would only take a week. After about a year or so, I had some portion of it.
In order to offer Google’s search of their index to the world, they had to keep all the internet’s content on their own servers, otherwise their results wouldn’t be very fast. Did they ask every single website owner or administrator for permission to use said material? No. Did they need to? No, in fact they couldn’t. That would have been prohibitive for what they were doing. The cost alone of asking would have been prohibitive for what Google was doing, if they even knew themselves, what they were doing.
However, was what they did beneficial to the world? Yes, one may very well say so, to a degree that Google is now a hugely successful business whose operations span the globe and benefit millions, if not billions of people on a daily business. What Google did was transformative, defining of the internet. It defined the web.
What Google added was their filtering index of the web. On their servers, the content of sites were analyzed and ranked according to PageRank, an algorithm which rewards sites which are greatly linked to with a better placement in search results than sites which have generated fewer links.
But for this to work they needed the data to work with. Google has done a lot to give users the impression, than when one is using their core product (search), it appears like one has instant access to all of the World Wide Web. This is a brilliant illusion, but no matter how good it is, one is still only surfing around on Google’s own servers, which store Terabyte after Terabyte of unauthorized copies of copyrighted material. The fact remains, that Google took this data, without asking anyone for permission. Perhaps they didn’t need to, perhaps they didn’t deem it necessary. What Google did was one of the greatest things that could have happened to the web at the time, and what everyone else involved in the search industry was doing. Throwing around data without paying any kind of homage to copyright owners. To the great benefit of everyone of us today, most will say.
What The Pirate Bay and other sites are doing today – is no less transformative. But they’re not cooperating.
What happened since Google introduced their filters to the world was that the “war on piracy” became greatly intensified. Napster and peer-to-peer networks threatened the monopolies of first the record industry, since the Hollywood-based entertainment industry. Google and other services which offer online metadata – i.e. access to “other people’s” information via the internet, got trapped in that battle. Some felt they had to choose sides. And most chose to cooperate with the entertainment industries – over what was right or true or just. Whether this line of business was born out of the pragmatism of doing “business” and avoid expensive law suits or out of a mission to “do no evil” doesn’t matter. Google and likeminded companies will do a lot to cover up the fact that what they are doing is based on massive copyright infringement – including cooperating with IFPI to filter online information – every day. Which in my humble opinion is very creepy.
I say this as a big fan of Google, as a daily user of countless Google products, which I would hate to live without.
It’s a pretty good fraud. Cooperate with IFPI and other copyright holders to only slightly cover up the fact, that the whole thing is based on copying other people’s material. Blur the distinctions to the extent that it even confuses the courts as to what they should believe. What is really the difference between Google and similar search filters and a service such as The Pirate Bay? Both store and provide access to metadata. But while the first stores everything on their own servers, from where they provide access to local sites and material – The Pirate Bay and others employ a superior technology, which offers nothing but hyperlinks directly to material stored on their users’ own machines. So why should The Pirate Bay lose the case which is going on right now in Sweden? Because they do not cooperate. They do not care about anyone’s material. What they’re interested in is developing a new technology to the benefit of all of us. They do what Google did in 1998, except they do not commit any copyright infringement at all.
On a curious note, Google also ranks web sites according to how “unique” their contents are. This means, that if you run an aggregation site, i.e. a site which harvests and provides access to the content of other web sites – just like Google did, and still do – Google assigns you penalty points, and your site will be harder to find using Google’s search. Your site will rank lower, if you do what Google does : copy the content of other websites.
What’s really scary however is the degree to which we rely on proprietary filtering services such as Google’s search, which are influenced by interests we don’t know about. Google presents itself as an almost universally neutral service, which can give us an instant answer to almost every problem we face. The truth is, Google is in fact a highly weighted information filtering service, which is influenced by the special interests of organizations such as IFPI, on no legal grounds except what pleases and what not pleases Google and is completely dependant on their choice to cooperate. We don’t know what other special interests Google chooses to cooperate with, and we have absolutely nothing to say as to whether they do and how they let their search results be influenced by them. I can only conclude, that while a few young people in Sweden are willing to stand up for our freedom of speech (for this is what I consider the “freedom to link” to be) – it is shameful to realize again and again, that the world’s information filtering superpower is not.
In my view there is no other way out of this misery than to create and help build new sets of truly de-centralized information filtering tools and services, which are based on free software, which cannot be influenced, manipulated or dominated by any particular third party. Tools which enable better, faster and more precise connections between someone who wants a message or query out – and those who wish to receive and answer it. We’re still throwing around rocks in our information stone age when playing with proprietary services and tools such as Twitter, YouTube and the many many others we use on a daily basis.
Kaplak is changing it’s course again. Since the inception of the first kaplak idea, we’ve come a long humbling way to only realize over and over again, how much we still have to learn. But slowly, we also realize what kind of knowhow we have and are building, and how Kaplak can help crack the problems and meet the challenges, which we set out to originally. Hence we also begin to understand what kind of value we add – and just as importantly, what we don’t add. Among many other things, this is key to learn what kind of business model we want to build – and, just as importantly, what kind of business we don’t want.
Let’s take a look at what happened with our traffic since the somewhat bumpy rolling-out of Kaplak Stream in 2008, from November 1st last year to February 1st this year :
The above is a screenshot from the Google Analytics Dashboard for Kaplak.com including subdomains. Following the launch of Kaplak Stream, sometime in November our traffic started to take off. Kaplak Stream basically consists of the present WordPress MU installation of which the Kaplak Blog is also part, along with a handful of customized plugins, of which the most important one is FeedWordPress. The idea (as sketched out in this previous blog post) is that items in the stream can be “fed out” from the stream again, which will reveal new contexts, which didn’t exist before. When two separate items which are both tagged “Barack Obama” are fed from the stream, they create a new “Barack Obama” context, even though the original items may have been produced and published in wildly different contexts.
The first installment of Kaplak Stream came with just about fifteen feeds, of which a handful were submitted by owners of niche websites. Others were feeds from sites such as YouTube, Amazon.com, Twitter (tracking particular subjects or keywords) and Boing Boing. Enough to provide the stream with some variety and “head” which would also test the autotagging performed by Open Calais via a modified version of Dan Grossman’s WordPress plugin.
Kaplak Stream managed to aggregate well over about 15.000 items, i.e. about 1000 items from each feed on average. Grossly more tweets than regular blog posts were aggregated, but posts attracted the greater amount of traffic, given that they worked much better with the autotagging functionality in place. Since they had more text, the tagging tended to be more precise – although some times tags were wildly misleading and out of place. Room for lots of improvement. Most, about 90-95% of all traffic came from search, notably Google. Visitors tended to not stay long, but quickly be on their way again. This could seem to suggest that only few found what they were looking for. However, reports also came in from feed owners, that our traffic managed to produce a meaningful sample of visits on the actual sites aggregated. This was really good news, as it suggests that a sample of our visitors actually found what they were looking for, or was curious enough to click through.
So what pulled the rise in traffic? No subject in particular, but the variety of subjects covered. What attracted users were more often than not pretty obscure pages and topics. For example, top result were the “tag page” for the tag “university-of-illinois-arctic-climate-research-center” with 641 views, and there was absolutely no recoginzable pattern in the rest of the more popular pages reached by visitors. I have not given our sample here substantial analysis, but my guess would be that there would be a neat power law graph, if one dotted in the number of visits to each page in Kaplak Stream and ranked them besides each other. But there is no discernable pattern as to what determined what aggregated items were more popular than others.
While some things seem to work, albeit still just barely, there are also problems. One of these is that apparently something happened on January 26th, which made our traffic drop drastically to before Kaplak Stream levels. Presumably this drop was caused by a Google penalty from duplicate content, which Google have been known to give websites which carry identical content across different domains. While Kaplak’s goals are somewhat aligned with Google’s, although not completely, I’m not unsure the penalty (if there was one) was not “right” in the sense that there were clearly limits to how informative and appropriate the search results which led visitors to our site, were. At least to justify the dramatically beneficial position we gained by aggregating just 15 feeds.
Another problem is the “noise” level, in our tagging, and in the combinations of feed items tagged with similar tags. Tags can be and mostly are very local. A post only remotely connected with a person and a piece which is solely about that person are usually tagged identically. My instinct tells me we need to use automated tools for what they are good for, and let filtering be more in the hands of expert users, in the contexts where it matters.
Clearly, more experiments are needed, and we need much more sustained analysis and methods to analyze our data. All this takes time and costs money. Right now Kaplak has no business model except what we can put into it of our own pockets (meaning mine) – and these are rapidly emptied. This means, for the time being, i.e. for several months now – and several months (and perhaps even years) ahead, I will not be able to work and develop Kaplak on full time. Thanks to the benevolence of our host, we can keep and continue to work on all Kaplak’s sites and projects, but we’ll make some changes which prepares us best to run Kaplak as a part-time operation.
We’ll convert the Kaplak setup to a setup more similar to that of the UMW Edublogs set up by Jim Groom at the University of Mary Washington. Among other things, this means we’ll focus more on building each smaller site in the network, and keep each site focused on it’s subject or theme. We’ll focus more on aggregating what happens within the Kaplak network of sites than what is going on outside the Kaplak WPMU install. We’ll still use aggregation tools to track very particular subjects, keywords and tags, but each different subject will be treated in a site of it’s own, to make things more manageable (it’s a mess cleaning up a large site based on aggregated items). In other words, we’ll run a network of small, very low-maintenance sites, and delay bigger experiments and improvements for a while. Meanwhile, Kaplak Stream will still be able to track tags across all sites and offer feeds from particular tags used in the network.
Reducing the amount of my time which goes into actual development of Kaplak also means I can focus better at building a new constellation of ressourceful people and (real) investors, which we will need to come back stronger with a revived Kaplak at a later time. This is what I hope to achieve, while I work simultanously on other things, making a living.
However, there is also a risk, that we don’t. That our ways may go in other directions. This is not necessarily all bad. See this video with Tim O’Reilly in a previous post to see why. I will try very hard to keep an open mind and attitude and not get stuck in ideas I ought better to leave behind. That said, I can’t see any companies or services which presently really cracks the problems we set out to – and this means we still need to fill that space, one way or the other. And more than anything, I can’t stay away.
Lessig has some very interesting remarks re: his meeting with Barack Obama years ago, where Lessig encouraged Obama to seek public office. According to Lessig, Obama responded, “you know, Larry, guys like me don’t make it in public service like that”. Obama presumably felt politics would demand too great compromise, than he felt he was able to give. Seems like he changed his mind. Here’s fingers crossed he’ll have some success in changing that game.
On another interesting note, Rose asks Lessig if he has any concerns in regard to filesharing and what’s happening in ‘free culture’, if he’s to take the perspective of ‘the other side’, i.e. the entertainment industry and MPAA intellectual property lobbyists. His answer is thoughtful and enlightening. Lessig fears, he says, the extent to which the reactionary and draconian IP legislation has created such resistance to copyright law, that the new generation – or a substantial segment of the new generation, will simply abolish copyright law altogether – just get rid of it :
Lessig : I think there is a real risk, because of the growing – I think of it as a kind of abolitionist movement with copyright. People who think that copyright was a great system for the 20th century, but we just need to get rid of it now. It’s not doing any good now, it’s not necessary, let’s just abolish it. Well, I am not an abolitionist. I believe copyright is essential in the digital age. I think we have to find a way to make it sensible in the digital age, but we have a richer, more diverse culture with it than we would without it. But my real fear is that the last ten years has unleashed a kind of revolutionary attitude among the generation that will take over in ten years and it’ll be hard for them to distinguish between sensible copyright legislation and the kind that we’ve got right now. So my real fear is we’re gonna lose control of this animal. Not in the sense, that we’re trying to guide it, but in the sense that we’re creating an environment where we can really have rich, diverse culture. So in this sense I feel like I’m Gorbachyov, not Yeltsin, I’m like an old communist who’s just trying to preserve..
Rose : (laughs) – who’s not gonna let go of everything…
Lessig : Yes exactly right. They just wanted to reform it, to make it make sense.
Rose : But can you do that, I mean?
Lessig : Gorbachyov couldn’t. So I don’t know. But that’s what I’m afraid of. I’m afraid we’re gonna destroy something important. Because the thing copyright does, when it works well, is it’s very democratic. It gives the artist an independent ability to create. He doesn’t have to worry about his patron, supporting his kind of creativity. He can create on his own. And he creates on his own, and he owns what he creates.
Lessig emphasizes the importance of businesses to understand, harness and protect creative communities, like a Yahoo does in ’securing’ the community of Flickr and the built-in ability of that community to use Creative Commons licensing of their images, or a Google does in similar ways with Picasa.
However, on this occasion as on others Lessig fails to enlighten us on what good copyright does us, when businesses vigorously seek to uphold IP rights in software. Google may harness the creative community of Picasa and enable free licensing within their software and as long as it provides value to their business, but what about the rights of Googlers, whose entire creative work by contract ends up being owned by Google, not by themselves? What good does it do us as a society, that companies benignly builds in free licensing, if, when and where they choose to do so, but seek to uphold IP barriers for users to change the actual software they use daily and operate on their own machines? Does that make us more free as a society, or less free? Does it give us a more diverse or less diverse culture?
Wouldn’t it be better, for transparency, for competition, for our culture of understanding and sharing; for our die hard focus on what’s really at stake; the big problems and big challenges we face as a global community : poverty, disease, pollution and international, armed conflict – to abolish a system, which systematically gets in the way of solving problems we face and which we need to solve? Which systematically gets in the way of enabling us to work together to help share information to crack the hard problems facing all of us? In what way is it democratic for a western author to deny the unauthorized distribution of his audiobook in a third-world country? In what ways do the distribution of Lord of the Rings (itself based on another work of fiction) via p2p networks harm anyone in this culture or another?
Lessig have always been careful not to associate himself with the pro-piracy movement. In 2006 a very nervous Morten Blaabjerg met briefly with Lessig to conduct an interview for a film project. Lessig was then visiting Denmark on the occasion of the official launch of the set of Danish-context adapted Creative Commons licenses.
Among the things I asked Lessig about in this interview was his attitude to what was then happening in Sweden, the police raid on the Pirate Bay. Lessig responded :
The Pirate Party and the people behind it are extraordinarily sophisticated, and this most recent post, a speech at the Reboot conference, called The Grey Commons, is an extremely sophisticated analysis of the problems.
In America, in my view, it’s counterproductive to encourage something called quote ‘piracy’. And the reason it’s counterproductive, is that if that’s what you push, then people stop listening to your argument, because they think that it’s all about, you just wanna be able to get something for free. And, if they stop thinking about the argument, then we’re not gonna make any real progress. So in America, I think this would be a bad strategy, and in fact, I’ve come to regret my role in certain lawsuits, that have gone to the supreme court, defending the right of peer-to-peer filesharing. Not because I don’t believe in the right of peer-to-peer filesharing, but because, as a strategic or even tactical move, focusing on that activity causes more confusion, than it causes understanding.
Now it could be, certainly could be different in Sweden and in Denmark.
There’s a long way from Lessig’s warning to ‘talk of piracy’ as a ‘bad strategic move’ to his talking with Charlie Rose about an ‘abolitionist movement’. This goes to say a lot about what has happened during the last 2-4 years. Use of bittorrent has been and is rapidly expanding, some of this no doubt due to the publicity surrounding the Pirate Bay. What’s more important, IMHO, is that social networking have become near mainstream, as a recent local television story about Facebook, in which I participated, made totally clear. Apparently, the popularity of Facebook among the +45 years old is a lot greater than people usually think. The sharing practices of these social networks have made copyright concerns a lot less practical. If I want to share photos with my friends, why bother thinking about copyright? Why bother about what they do with those photos? Why protect us against who they’re going to show them to, if they will make money on it or not, or whatever protectionist concern there may be. If you put it out there, it’s beyond you and your control. With or without copyright. It hardly makes any difference, as the ability of law enforcement to actually crack down on these sharing practices, is inefficient and good for nothing.
The only problem remains that services, such as Facebook or Google, seek to retain all rights to their users’ activities and information. This creates great problems for users, if they wish to ‘take out’ information and use it elsewhere. The loss of freedom lies not in the architecture, but the inability to help change those architectures, so that users may take their data where they’d like to go, in what ways they’d like to do so. To create a free and culturally diverse online environment, we need not protect ourselves from the use of ‘our data’, but from the entrapment of ‘our data’ in systems beyond our control. We can wait until doomsday for such companies to embrace the GPL. It’s not likely to happen. Our focus should not be the data, on ‘works of art’ – it should be on the systems which enable us to transport data, enable us to work together, share information and solve problems. Right now IP is used to prohibit or make this harder, as it is used to protect software and software companies and their incumbent business models – not the creativity of individual ‘artists’. This is why it is enlightening to read about the open source business strategies of companies such as Sun Microsystems and others. There are other ways to go. Abolition or not-abolition is not really the question. It doesn’t really matter, in so far as just discussing it doesn’t improve our architectures of communication or our problem solving capabilities. Embracing free software now does. Embracing Wikipedia now does. Embracing copyleft licensing does. Embracing tools of sharing, aggregation and open publishing does. Showing the effects of what you do does. Theory doesn’t. Fighting over legal matters doesn’t. Arguing back and forth about abolition of copyright with someone somewhere who doesn’t understand what you’re talking about (and doesn’t care to either) doesn’t.
P2p filesharing is the hope that we can create and maintain architectures of data transportation beyond centralized control. That we can reach out on our own, to reach others and understand each other. That other someone chose to share that particular movie, book or piece of software with us, which might or might not otherwise have reached us via different channels. That particular movie, book or software today – tomorrow something else of great importance. The channels of distribution are not really that interesting, except if your business model depends on measuring numbers of eyeballs, so that you may cash in on the commercials broadcast to these numbers of eyeballs. That’s what seems to be the concern of IP holders. Not to harness creativity, not to nurture a rich, diverse cultural landscape, but to protect incumbent business models, which stands in the way of creating and improving our decentralized methods of data and information sharing.
What’s interesting and what’s at stake is far more important than creating a culturally diverse environment : it’s about saving lives, about enabling us to live together peacefully and take a deep look at the world we’re in and imagine, what kind of place this could be, if we treated it with the same kind of generosity, as it treats us with.
We believe that real change can only come from the bottom up. And technology empowers people to come together to make that change. —Barack Obama, speech at Google, Nov. 14th 2007.
What’s at stake in this election
I believe this US presidential election is a matter of grave importance, not just to the US but perhaps even more so to the rest of the world. First, the American leadership sets an example for the world. The damages caused by the one set by the administration headed by George W. Bush is best scrutinized in Michael Moore’s film Fahrenheit 9/11. We need new leadership, a new vision and new examples to be set. We need a US president who can deliver this.
Second, the American economy influences the world’s economy. When a global economic power sinks into the bottomless financial pit of a brutal war of aggression, it not only sets a bad example for other nations, including Russia and all kinds of dictatorships, it also cannot help but bode ill for the world’s economy. We need a strong US economy, or at least a US economy which is capable of dealing with the challenges facing the United States internally, in order to lift the more serious challenges we face, such as world poverty and hunger and threats to our environment posed by our lifestyle and consumption patterns.
Third, and this is where we need real leadership, and real change, is to straighten out global priorities in the way the internet is, can and will be used. In particular, the US leadership is important in what is sometimes called ‘intellectual property law’. We need to stilt the draconian IP laws enacted in the US (I’m talking about the DMCA and similar legislation, if anyone should be in doubt). We need to stop these laws from becoming even more draconian, and we need to ultimately push them back. What’s at stake is hinted at in Lawrence Lessig’s marvelous book Free Culture, but isn’t limited to the remixing of music videos and animations – it’s our problem-solving capabilities which are at stake. Our ability to make good decisions based on trustworthy information, processed and exchanged freely using the internet, on a global scale, is at stake. If the US technology sector cannot lead the way because of the draconian American IP legislation, it will be a poorer world, and the struggle to get there will be harder and fraught with greater difficulties.
Barack Obama is now following you on Twitter!
As I am not American, unfortunately I can’t vote in the US presidential election taking place tomorrow. If I could I’d have no doubts. I’d vote for Obama. Not because he necessarily stand for and will enact all the things that I hope for the world, but because I think he will enact the kind of leadership, which sets an example and will facilitate the changes we need.
I have not been following the course of Barack Obama very closely, but even from a long distance he’s been able to make an impression. All thanks to the wonderful powers of the internet, which he’s utilized in his campaign with such intelligence and vitality. It made an impression when he followed me back on Twitter. Him or his campaign staff, either way, it’s an attention to detail which impresses. Coming from someone I respected and consider of high integrity, it also made a great impression on me reading Marc Andreessen’s personal account of a meeting with Obama. What impressed me in Andreessens account is the way Obama listened rather than talked, a characteristic which I felt showed his genuine interest in the problems presented to him. And lastly, he gained great respect in my book for appearing so genuinely as himself on the Daily Show and being able to stand up to the jokes of Jon Stewart in such as relaxed manner.
Lately I’ve been reading up on Obama’s tech policies, and the one thing I note with the greatest clarity is his emphasis on what can be done in America, in order to lift not only America, but also the challenges we face globally.
Seizing the moment : Obama’s speech at Google
Obama spoke last year (November 14th 2007) at Google’s HQ in Mountain View about his technology and innovation program. Watching this video of his speech I found I learned a great deal about Obama – and not just about his views on technology. If you haven’t had a chance to see this before, please join me here :
I took some time out to transcribe Obama’s entire speech below. It’s one of those speeches which apparently is not included in any of the official lists. Most blogs quote his technology policy press release, but I find it illuminating to read the words he actually used speaking at Google, and I learned a great deal about him in the process of transcribing his words. It’s a great speech, and one that resonates greatly with me.
Full transcript of Obama’s speech
When you start to think about it, there is something improbable about this gathering. Afterall it wasn’t much more than a decade ago that Larry and Sergei got together in a dormroom as graduate students with a big idea to organize all of the world’s information into an accessible form. And at the time I was a [novice? Illinois?] state senator, doing my best to help people get a better shot at their dreams.
What we shared is a belief in changing the world from the bottom up, not the top down. That a bunch of.. that ordinary people can do extraordinary things. We shared that, and we also shared a bunch of student loans that still needed to be paid off. (laughter) And you would have found it hard to predict that Larry and Sergei would now be the co-founders of one of the most successful companies in recent history. And that I would be standing on this stage today as a candidate for president of the United States.
But this is where improbable journeys have led. This is where the moment finds us. And I’d like to say a few words about what I believe we have to do together, to seize this moment with a sense of purpose and a sense of urgency.
We know how the first chapters of the Google story have turned out. Afterall, all of you have good jobs. But we also know that the Google story is more than just being about the bottom line. It’s about seeing what we can accomplish when we believe in things that are unseen. When we take the measure of our changing times and we take action to shape them. And that’s why we’re here today. That’s why many of you decided to work here instead of somewhere else.
Technology and innovation have reshaped our economy and our lives at breathtaking speed. America’s been fighting to figure out how to tap this awesome new ressource we have. And Google’s helped to show us the way. But the story is far from over. Google’s story is far from over. The story about how we shape our changing times is far from over. What comes next depends on the choices that we make right now, at this moment, in this election.
We could see the spirit of innovation that started this company be stifled. We could see the internet divided up to the highest bidders. We could see a government that uses technology to shut people out, instead of letting them in. Tax breaks shuffled to special interests, while the next startup, the next Google can’t get a fair shot. Challenges like health care and energy that hold our country back while competition from other nations picks up. That’s one alternative.
Another alternative is for us to unlock a new future of opportunity. Together we could open up the government and invite all citizens in, while connecting all of America to 21st century broadband. We could use technology to help achieve universal health care. To reach for a clean energy future. And to ensure that young Americans can compete and win in the global economy.
If America recommits itself to science and innovation then we can lead the world to a new future of productivity and prosperity.
That’s what we can do if we seize this moment. That’s the choice we face.
As president, I intend to work with you to write the next chapter in the story of American innovation. That’s part of the reason why I’m running for president of the United States.
To seize this moment we have to ensure free and full exchange of information. And that starts with an open internet. (applause) I will take a backseat to noone in my commitment to network neutrality. Because once providers start to privilege some applications or websites over others then the smaller voices get squeezed out and we all lose. The internet is perhaps the most open network in history and we have to keep it that way.
To seize this moment, we have to connect all of America to 21st century infrastructure. As president, I will set a goal of ensuring that every American has broadband access, no matter where you live, no matter how much money you have or don’t have. We will raise the standards for broadband speed. We will connect schools and libraries and hospitals. We will take on the special interests so that we can finally unleash the power of wireless spectrum for our safety, our security and our connectivity.
To seize this moment, we have to use technology to open up our democracy. It’s no coincidence that one of the most secretive administrations in our history has favored special interests and pursued policies that could not stand up to the sunlight. As president, I’m gonna change that. We will put government data online in universally accessible formats. (applause) I’ll let citizens track federal grants, contracts, earmarks and lobbying contracts. I’ll let you participate in government forums, ask questions, in realtime, offer suggestions that will be reviewed before decisions are made. And let you comment on legislation before it is signed. And to ensure that every government agency is meeting 21st century standards, I will appoint the nation’s first Chief Technology Officer to coordinate and make certain that we are always at the forefront of technology and that we are incorporating it into every decision that we make. (applause)
And if you wanna know how I’ll govern, just look at our campaign. We received over 370.000 donations online, half of which have been under 25 dollars. Nearly 300.000 Americans have their own accounts on Barackobama.com. They’ve created thousands of grassroots groups. They’ve offered up over 15.000 policy ideas. Because we believe that real change can only come from the bottom up. And technology empowers people to come together to make that change.
Because at this moment I think we have to do more than get our house in order. The opportunity in front of us is bigger than that. Seizing this opportunity is gonna depend on more than what the government does or even what the technology sector does. It’s gonna depend on how together we harness technology to confront the biggest challenges that America faces.
Just imagine what we could do. If we commit ourselves to electronic medical records, then we can lift up the quality of health care and reduce error at dramatically lower costs. (applause) If we take on special interests and make aggressive investments in clean and renewable energy, like Google’s done with solar here in Mountain View, then we can end our addiction to oil, create millions of jobs, and save the planet in the bargain.
If we make technological literacy a fundamental part of education, then we can give our children the skills they need to compete and ensure the next generation of scientists and engineers as being educated right here in America.
We can do this, but we can’t wait. Because Silicon Valley is not the only corner of innovation in the world. If America doesn’t seize this moment, then we will face only more competition from Dubai and Dublin, from Shanghai and Mumbai. So instead of George Bush’s policy of undermining science, I intend to double federal funding for basic research and make the RND tax credit permanent. (applause) To keep the door open for the next generation of startups, I will enforce tough antitrust laws. And to ensure that America continues to attract the worlds’ best and brightest, we need comprehensive immigration reform, that strengthens permanent residence VISA’s like the H1B program.
We need to make sure that the next success story, the next Google, happens here, in America. The Google story is about what can be achieved when we cultivate new ideas and keep the playing field level for new businesses. But it’s also about not settling for what we’ve already achieved. It’s about constantly raising the barr, so that we’re more competitive, and so that we use technology to reach ever expanding horizons.
You know, the first time I was back here, in 2004, Larry showed me the image that tracks all the internet searches taking place in the world. I wrote about this in my book. I saw the Earth rotating on a flat panel monitor with the different lights for different languages marking all of the traffic on this wondrous network, a network, that didn’t even exist when almost all of us here were born. (laughter) Almost.
But what struck me wasn’t the light on that globe. It was the darkness. Most of Africa, chunks of Asia, even parts of the United States. The disconnected corners of our interconnected world. Where the promise of the 21st century is being eclipsed by peril. You and I must not settle for anything less than an America that replaces that darkness with a new light.
Because the promise and prosperity of a new economy must not be the property of the few.
It must be a force that lifts up our entire country and ultimately lifts up the entire world. (applause) We have the privilege to live in a transformational moment. A moment when an idea can change the world. A moment when technology empowers us to come together as never before, while letting each of us reach for our own individual dreams. A moment when we can finally progress and move beyond the huge challenges that have stood in the way of progress for far too long.
We can not and we must not look back and regret that we settled for anything less. And that’s why I’m asking you to join me in seizing this moment. I’m asking you to join me in changing the world. Thank you very much everybody. Thank you. (drowning in standing ovations)
Interview : Break the fever of fear
Obama’s speech is followed in the video with an insightful, relaxed and entertaining on stage interview hosted by Google CEO Eric Schmidt. In this interview Obama talks very open-heartedly about why he’s running for president in the first place, and why he’s running now. He’s also asked how he will actually end the war in Iraq, if he presumably takes office. As most will know, Obama opposed the war in Iraq. This speech from 2002 against the war in Iraq is well worth another read.
Obama sums up his foreign policy with the French president Sarkozy’s words, to “be more liked”. Meaning, that if the US is more respected and recognized for it’s diplomatic efforts, it will be easier to build up trust which enables problem-solving and diplomatic solutions and harder to create mistrust against Americans. Obama will “break the fever of fear” which has been exploited by the Bush administration to instill fear and distrust :
We’re told to be afraid of terrorists, of immigrants.. and each other … Our values are distorted .. not being certain if simulated drownings are really torture… That’s not who we are, as Americans. Sometimes I’m accused of being this progressive, far-out… – I’m conservative, in the sense that I want us to get back to those values that were essential to building America.
As I mentioned in an earlier post, syndicating stuff is also one huge grey area of legal hazzle. Stumbled over this discussion from a couple of years back (as well as this one), which airs not at all uncommon concerns. You risk being called a scraper, a spammer and a splogger, if you pursue the path of syndication.
Pariah S. Burke wrote :
RSS feeds are published for individual, private consumption; they are not a blanket license to, or waiver of, reprint rights. Taking and republshing content—no matter how much or how little—without the original author’s permission is a violation of U.S. and international Copyright laws. There are exceptions, of course, detailed in the Fair Use doctrine, but such exceptions are very specific and do not apply to the vast majority of sites using FeedWordPress, Autoblog, and the like. In fact, Charles Johnson, the creator of FeedWordPress is in constant and frequent violation of copyright law because the apparent majority of his blog’s content is stolen without the original authors’ permission.
In that case, Google, which enables users to very easily tag and share (i.e. republish) feeds they find interesting via their popular service Google Reader, is guilty of same said constant and frequent violation of copyright law, or at least, in willful and assisting infringement. The same of course goes for YouTube and any web service, which allows anyone to embed their videos, images and games on your own local site.
Who says a tool has to be used in one way only? Let’s get creative! That’s how problems are solved and new business models are developed!
To be honest, I’m not a big fan of people scraping content that people have sweated over. However, one thing I don’t mind doing is thieving from thieves.
You’re on the hunt for “disposable” content – generally not text based. Think along the lines of Flash games, funny videos, funny pictures, hypnomagical-optical-illusions – that kind of thing. The Internet is awash with blogs that showcase this stuff. Check out Google blogsearch and try a search like funny pictures blog. There’s hundreds of the leeching bastards showcasing other peoples pictures, videos, games and hypnomagical-optical-illusions for their website. They can hardly call it “their” content. With this ethical pebble tossed aside, we can go and grab some content.
There’s loads of ways you can hunt down potential content. You’re on the lookout for RSS feeds with this rich media. So you could try; Google Blogsearch, Technorati, MyBlogLog – basically any site that lets you search the blogosphere.
My personal point of view (this is also Kaplak’s stand) is that the problem of visibility for sites and products is larger than the largely fictional problem of “theft”. If you make syndicated feeds publicly available, you implicitly want and ask for syndication, because you want your message out. Syndication will help your site or product become visible in places and contexts it would not otherwise be seen in, and that’s why you use it and why you should use it. If you do not want your message out in other contexts and do not want to see your articles appear on other websites in a syndicated format, you can simply choose not to make articles available for syndication. The benefits however, in the Google Juice and traffic which syndication brings back to your sites and products, are in most cases much greater than the disadvantages.
Accusing syndication sites and services for theft and copyright infringement is IMHO ridiculous at best, as these services actually help your site become seen and achieve better rankings in search engines. It helps your interested readers and users find you in the first place. And if you don’t want to be read – why publish to the web?
At worst, these allegations are harmful, as they instill an atmosphere of fear and create distrust of using RSS, feeds and aggregation tools. Instead, we need to urge and encourage syndication and use of syndicated feeds, as it enables rich web contexts, which would otherwise not be possible, and makes it easier to direct interest and relevant traffic to sites and subjects of interest. It is above all a tool, which can be used for our mutual benefits – or for spamming and creating yet more “get rich quick” mentality kind of sites filled with stuff the world could care less about (but apparently doesn’t). I am of the opinion that these types of sites may provide their owners with short-term rewards, but ultimately will fade to authentic sites of much stronger lasting value. How to build lasting value, and help these sites and products build lasting value, is what we’re interested in here.
This early sketch illustrates how a product/widget from a niche producer is made visible in a niche context somewhere else on the web :
A web user and niche producer (A) encounters a Kaplak widget on a website, he knows and trusts (B). The producer finds Kaplak can be used to distribute a product of his own. He decides to sign up, and subsequently uploads a product and submits basic product information.
The Kaplak interface (C) spits out a widget a.k.a. a “kaplaklink” for the product. The widget is also published to the Kaplak market network, from where it may be fed via RSS or other means, to subscribers within particular channels or categories.
A website-owner (D) run what we may term a “filtersite” (E). D feeds or filters widgets from the Kaplak network from a range of categories or tags, in order to capitalize on sales, i.e. earn a share of kaplak from each sale made on E. His motive is primarily of commercial character. Among the widgets filtered is the widget for A’s new product.
In order to avoid what we term the mainstream problem, i.e. that just a handful of “hits” are prominently displayed and amplified, Kaplak depends on filtering sites of all kinds, i.e. index websites which seek to filter Kaplak’s feeds according to particular specialized interests or criteria. We have a lot of this kind of websites in the online landscape today, many of which are financed by advertising. Kaplak will offer one more type of income for index type sites, and one which may allow a sharper edge in filtering, because the size of income streams may not always be proportional with the amount of traffic generated by a site. A large site may suffer from greater problems in making the “slim end of the long tail” presentable, than a smaller and more well-defined niche-friendly site will. Both may be filtering sites, though, basically performing the same task of feeding and filtering.
The widget from A on D’s site is now discovered by (F), who puts the link into her blog, because she finds that the product is interesting and relevant to the article she’s about to publish. F’s blog is visited by a much more select crowd than D’s site, who rely mainly on search as a source of traffic. F gains a lot of attention through a social networking site popular within her field of expertise (G). Motives here weighs more heavily towards the professional, contextual, idealist side than the money side. F earns a fair share from her Kaplak widgets though, as her choice in widgets is much more finetuned to her readers, than the bulk filtering of D, which earns from a few sales of a lot of products (the “pure” Chris Anderson model).
Finally, a friend from G alerts another friend, who happens to be the owner of a nichesite (H), which deals particularly with A’s subject and finds the new product intensely interesting. The regulars of H knows the deal and can instantly see the value of A’s product. A’s product finds a potential market here, he otherwise wouldn’t have found.
None of H’s users would have discovered A’s product without Kaplak, even if it was accessible via Google or filesharing networks. First, none of them would know about the project. Had one of them actively searched for the product, she would have had to pick very delicate keywords, endure the timeconsuming process of browsing search results to page 7 or 8, only to discover a dead link to a torrent, which may have been alive and kicking, but of which there are no seeders.
The owner of website H publishes A’s widget from both professional and financial motives. The professional, interested motives weighs in the heaviest, but since the site engages A’s target group, the collective sales pays off decently in kaplak, which contribute to financing the site. H’s traffic may be slight – if the group of “regulars” is sufficiently interested and the price right, then H need not care greatly about the amount of traffic.
The producer A expands his market with H’s users and anyone who made a transaction along the widget’s “route”, who wouldn’t otherwise know about the product. The process repeats itself, this time with one of H’s users in the role as producer A, who discovers she may use Kaplak to distribute one of her own products. This process happens across Kaplak’s entire global network, with the intensity dependant on the demand for the products offered by users, and on the ease or difficulty by which a product/widget can gain an entrance into the niche environments and markets “in the other end”.
The sketch illustrates what Kaplak’s primary product is. As we’re on the web, all sites and actors in the above diagram are accessible to everyone all the time, from anywhere they may be situated in the world. The problem is knowing the product exists and next, to find where it is. Search engines such as Google and others offer one model, filesharing index sites such as The Pirate Bay and others offer another. Both however, are primarily based on active search for information, from the buyer’s end.
Kaplak offers a third model, which brings the product to the target group, through the web services and communities the target group uses every day. When Kaplak works, web users will find interesting links/widgets on sites and services they regularly visit and trust, before they even know they want the particular product – and long before anyone even thought of using Google or something else to go look for it. Finally, the Kaplak model can be fully financed by the market, which is opened up, rather than rely on upfront payments from our niche producer, before he or she knows if there is a market.
When Steve Spalding wrote about Kaplak on How To Split An Atom, he chose a small picture of some pictoresque 19th century houses to accompany the article. The picture bears the title “Denmark”, which is shown when hovering over the image with the mouse cursor. While nice, the image is a bit on the tiny side, and doesn’t quite justify what it looks like living here. I’ve thought about for a while that I would like to put in a little work to expand the online imagery of Odense, so that the next person to write something about us (or about Odense or Denmark in general), will have a greater variety of images to pick from. What better day to do this, than on a sunny saturday, which is also a market’s day in Odense?
While the pictoresque quality of Odense is not completely true – and not completely untrue either, these are the places which do attract the camera the most. I had my daughter and my dog with me, and we were even lucky enough to see a whole swan family as they were feeding.
I’ve previously written about the merits of attributing value to the context of finding information, rather than on any particular piece of information. This makes sense in an environment which literally explodes with new information, and shows no signs it’s gonna stop in any foreseeable future.
Google seems to think so too. After all, this is what Google do, and do really well. But it’s true no less of a somewhat overlooked product of Google’s. I’m talking about Google’s Custom Search. This service allows anyone to composit their own search engine, and place it on their own website. More accurately, your custom search engine filters Google’s index of webpages. Say you want a search engine on your site about your niche subject only to return results which relates to your site. It’s simple : type in your site name, and allow Google to show results from your site as well as all the sites your site links to. Or you can be even more specific, or list a range of sites you want results to be taken from. Or you’d like Google to still show results from the web, but emphasize results from your own site – this is also easily doable.
The only problem so far with Google’s Custom Search has been on the one hand that Google’s crawlers don’t seem to index every website too tightly and too frequently, and on the other, that results are still based on PageRank. Say you want your users to find a great piece on your blog about a particular subject, when they search for that subject, but that piece isn’t greatly linked to by other sites or articles. Chances are, that Custom Search will show a largely irrelevant, but greatly linked to article from another site, or simply not show that post at all, if it hasn’t been properly indexed. Your built-in blog search, such as WordPress’ search, will find that article very fast, because it searches your database directly. For smaller sites, local search as we know it, is still much more effective.
However, as sites grow and we as internet users and bloggers spread our activities over many sites and platforms, platform-specific search is too limited. We begin to look for more tailormade solutions. Google’s Custom Search is one, but there are others who want a piece of the action.
New kid on the block
Lijit is an internet startup based in Boulder, Colorado, which offers a promising version of “local” or “contextualized search”, which searches one’s blog, “content” (on sites such as YouTube, Flickr and many others) and the network of sites and “friends” your online activities connect you to. We’ve already created a Kaplak search engine powered by Lijit, and the Lijit widget is featured in the outer right column on this blog. I think Lijit could potentially be a very useful addition to the Kaplak toolbox. I plan to expand this search engine with further feeds and sites as our network and activities grow.
When I first tried Lijit, I wasn’t satisfied with the search results. I searched for a direct title in one of our blog posts, and it didn’t come up. As the impatient web customer I am, not hesitant to make a fuss about my problems with a free online service – on another free online service, I postedmyquibbles on Twitter. It turns out, Lijit is on Twitter too, and so is Micah Baldwin, who works for Lijit and took time out to answer my quibbles.
It turned out Lijit based their first version on Google’s Custom Search, while developing their own web crawler. Switching Kaplak’s search to Lijit’s own crawler was a huge improvement from Google’s occasional crawl, and made me look much more enthusiastically at what this small team of extremely talented people are doing. I take my hat off for a company which acts so swiftly in response to “customer” sentiments, and make it a priority to help their users along with such friendliness. There are a lot of companies who could learn so much from Lijit. Micah and Lijit gives the expression “listening to the groundswell” a whole new meaning.
I like the freshness of Lijit and I like the results after being switched to their own crawler. I have only a few quibbles with it now. It’s got what I’d call some weaknesses in the versatility department, because I can’t control and finetune texts, messages and included sites/webpages as much as I’d like to and was quickly getting accustomed to in my short period of experience using Google’s Custom Search. For instance, I found all of my del.icio.us network automatically included in the search engine, where I’d like the opportunity to handpick whose links got to be included. Lijit’s search engine also wants to categorize results very neatly into “my blog” (even though the Kaplak Blog is not precisely “mine” – it’s the company blog and maintained by me, but not “mine”), “my content” and “my network”. What if we (which we’re probably going to) put the widget on our wiki? – that’s not exactly “mine” either. Our Kaplak universe is not so neatly organized, and while I do like the “Lijit picks” category, I prefer being able to scrap all categorization schemes altogether, get our own adsense stuff on the search results and just get on with finetuning and putting in more sites and feeds to give our visitors the best possible experience.
Lijit can potentially be a great key to tying together the many different platforms we operate on in Kaplak – and one we’d even pay for, if they included premium options we needed. As a company, we still do need search, and if Lijit could potentially even crawl user and product profile pages on our later-upcoming Kaplak Marketplace, we’d have something here, which we’d probably like to pay good human money for.
Imagine a service which have taken upon itself the daunting task of searching all things on Twitter instantly and is capable of threading and translating posts to and from numerous languages – globally. Then you have Summize.
Using Twitter a lot these last few months, I’ve found Summize indispensible to keep track of tweets, users and subjects. I’ve also used it for market research, i.e. “listening” to what other users are twittering. I find this stuff utterly incredible. There’s a lot of things happening in the search business these days.
I’m sure this is only the beginning.
[EDIT : Twitter's acquisition of Summize has broken the above link to the Summize search with my conversation with Micah. Here's a similar search on the new http://search.twitter.com which supposedly replaces Summize...]
Presently, Kaplak has a need for an internal, simple yet powerful online file sharing utility. We’re talking about simple intranet stuff, but without the complexity of setting up an intranet. We don’t need an intranet. We need to read and save documents and spreadsheets.
We can get it now with Gmail and the Gmail Drive Shell extension by Bjarke Viksøe. As a hack, this extension has some limitations, but it is a great temporary solution. I wondered if we’d be able to search through attachments, but alas, this doesn’t seem to be the case.
They should do that had an interesting article from two years back discussing the extension of Gmail into an attachments browsing utility, which I roughly translate into a storage & sharing facility for attachments – also known as files. This is essentially a simple filesharing movement building out of Gmail, building on the simple needs of people with free storage on their hands.
It should be obvious the need is there. This is simply the most needed functionality for lots of people and businesses. Forget the Google Apps, Google Groups or Google Sites. We need a GDrive, a simple place to place stuff, where people we work with can get it and upload their stuff too. No profiles, no more social networks or group functionality, we have lots of that – simply storage available for simple filesharing.
I can think of one reason Google won’t move down that alley, and that’s fear of intellectual property problems and the threats of litigation costs etc. The same reason why Google Books never took off, and why most books are not fully readable or available to non-US residents. Why “snippets”?
Well, that ought to leave this window of opportunity open to others, but noone has the power of the Gmail apparatus already in place.
Another reason might be the risk of undermining Google’s own business in intranet and local search for businesses, as well as their Groups and Apps product ranges. But as Google earns most of the company’s revenue from advertising (including ads in Gmail), this should be a small camel to swallow.
Thanks for the tip about GDrive to Anders! (Anders Nicolaisen and Jesper Lund have recently joined the Kaplak Team – a formal introduction will follow shortly :-) )
