Bye Bye Craigslist(blog.padmapper.com)
blog.padmapper.com
Bye Bye Craigslist
http://blog.padmapper.com/2012/06/22/bye-bye-craigslist/
88 comments
Disagree. Craig Newmark has said that he's scared to make drastic changes to the UX specifically because he doesn't know what magic recipe keeps people coming back. Do I like the CL layout? Nope. Does that mean other people should steal the content and profit from it? Probably not.
The magic recipe that keeps people coming back is that Craigslist has a large network.
I would be willing to bet a large amount of money that adding a map to search results and letting me put an upper limit on the number of bedrooms I'm looking for would not make people less likely to use Craigslist.
I would be willing to bet a large amount of money that adding a map to search results and letting me put an upper limit on the number of bedrooms I'm looking for would not make people less likely to use Craigslist.
To be fair to Craigslist, since everyone and their grandma uses it, I'd imagine a lot of users have underpowered computers, outdated browsers, and low bandwidth connections.
I've seen padmapper slow wayyyy down on my i7 macbook with 8GB ram, I can't imagine what it'd be like on grandma's 5 year old budget PC.
I've seen padmapper slow wayyyy down on my i7 macbook with 8GB ram, I can't imagine what it'd be like on grandma's 5 year old budget PC.
Features can be added without alienating users with outdated tech. Just conditionally show the improved site only to users who can run it. It's a poor excuse, especially since Craigslist hasn't even tried to improve the layout of the site, which has no effect on load speed and compatibility.
The same is true of Facebook. That hasn't stopped them from evolving their design.
I don't see how the existence of padmapper could hurt the grandmas that use craigslist.org
Don't discount the speed. Every Craigslist page loads and renders very quickly. The simple, text-only UI helps keep it fast, and that matters.
While the pageload speed may be faster the total speed to find what I'm looking for is drastically slower due to the fact that the interface sucks.
Which paradoxically, seems to be an advantage, because it puts people in a feedback loop where they stay on the site, and every new search provides a tiny bit of reinforcement that builds the habit.
Failing to find what you're looking for because of a frustrating UI is not positive reinforcement. People stay on the site because it is the largest marketplace and they have no alternatives.
That's weird, I've never had any trouble finding what I was looking for on Craigslist. There definitely are ways that they could improve the search interface, but the current implementation works fine for me, and given their user base it works well enough for a lot of people.
It's actually much more variable reinforcement - of course, the most addictive kind.
Each individual search has a low probability of success, but occasionally you'll succeed.
Each individual search has a low probability of success, but occasionally you'll succeed.
Their magic recipe is that an amazing amount of non-technical people have figured out how to work with the site.
The Dutch version of Craigslist (Marktplaats, owned by eBay) is in a similar predicament.
They have a very sucky UI and have experimented with new and improved versions, but every improvement was quickly reverted after large drops in engagement.
It's almost surreal to look at their site (http://marktplaats.nl) and realize that there's tons of highly talented people behind it.
The Dutch version of Craigslist (Marktplaats, owned by eBay) is in a similar predicament.
They have a very sucky UI and have experimented with new and improved versions, but every improvement was quickly reverted after large drops in engagement.
It's almost surreal to look at their site (http://marktplaats.nl) and realize that there's tons of highly talented people behind it.
Yup, there's a catch-22 here: when your site has shitty usability but is the only source of information, people hit more pages and spend more time there. When you fix your UI mistakes, they're in and out quicker because they actually get what they need efficiently. Because you're optimizing for engagement by measuring time on site or something like that, you end up making sure never to make your site easy to use.
This is not a recipe for success. It's a symptom of measuring the wrong metric, especially when you have a near monopoly on a market due to network effects. If this is really why Craigslist is not improving their UI, they need to hire some analysts that actually know WTF they're doing.
This is not a recipe for success. It's a symptom of measuring the wrong metric, especially when you have a near monopoly on a market due to network effects. If this is really why Craigslist is not improving their UI, they need to hire some analysts that actually know WTF they're doing.
Haha, I see what you did there. You confused UI with Usability. Craigslist is incredibly usable. Have you taken a look at Ebay lately? Does it look familiar? That's cos it hasn't significantly changed in 10 years. Is that because they don't care? Nope, it's because gradients and drop shadows make your product look better, but don't make it more usable.
Craigslist is a utility, not a website. They care about keeping it useful not only for users today, but for users in 10 years. That's not something startups care about at all. Startups are looking to amass users and then sell. Craigslist cares not about such goals.
Go ahead, get mad at their inflexibility. But then take a step back and think about what craigslist represents. And then wonder if you yourself would have the cajones to stick to your guns in todays startup market. I for one commend their ability to keep their product strong in the face of all the bells and whistles of web 2.0 or whatever it's called these days.
Craigslist is a utility, not a website. They care about keeping it useful not only for users today, but for users in 10 years. That's not something startups care about at all. Startups are looking to amass users and then sell. Craigslist cares not about such goals.
Go ahead, get mad at their inflexibility. But then take a step back and think about what craigslist represents. And then wonder if you yourself would have the cajones to stick to your guns in todays startup market. I for one commend their ability to keep their product strong in the face of all the bells and whistles of web 2.0 or whatever it's called these days.
You can't view apartment search results on a fucking map.
Are you really going to argue that the site is more usable without that very basic feature?
Are you really going to argue that the site is more usable without that very basic feature?
Just because the site is simple in design, doesn't mean that it is built to be used efficiently. Craigslist, as is, is a usability nightmare. When I use the site, I am using it with various Greasemonkey scripts, Firefox extensions and 3rd party web apps to make their content easier to consume/see/understand/filter. These tools are the only reason that I can bare to use Craigslist.
If I am searching for a bike, a simple script to show image thumbnails by every listing will let me see which listings interest me. This saves me time, because now I only have to open a few interesting links. I don't have to scour through many of text links, just to see if they interest me.
The rentals section is constantly bombarded by spam and has very few filtering options. There is no way to exclude keywords. Boolean operators would greatly improve the usability of the site.
If I am searching for a bike, a simple script to show image thumbnails by every listing will let me see which listings interest me. This saves me time, because now I only have to open a few interesting links. I don't have to scour through many of text links, just to see if they interest me.
The rentals section is constantly bombarded by spam and has very few filtering options. There is no way to exclude keywords. Boolean operators would greatly improve the usability of the site.
You can exclude keywords and there is a boolean OR operator: |
The problem is that you have to enter some search query before you can use the negation operator.
Also, protip to get past the spam in rentals section (Austin is holy crap bad about this): enter 'google' as your search term. This will only return ads that have an address specified and hence have the 'google maps' links. Plus, since you've entered a search term, you can filter out the locator who keeps spamming and screwing up your search by putting his phone number in the address section.
Here's a query I might have used during my last apartment search.
The problem is that you have to enter some search query before you can use the negation operator.
Also, protip to get past the spam in rentals section (Austin is holy crap bad about this): enter 'google' as your search term. This will only return ads that have an address specified and hence have the 'google maps' links. Plus, since you've entered a search term, you can filter out the locator who keeps spamming and screwing up your search by putting his phone number in the address section.
Here's a query I might have used during my last apartment search.
google downtown | central -"512-555-5555" -"CALL NOW" -"austin apartments NOW"
I disagree, craigslist sucks. It's filled with spam and it's searching and sorting options could be way better.
The only reason they don't have many competitors is because they have an information/platform monopoly. Just like many other tech companies, ebay and facebook come to mind as obvious examples.
The only reason they don't have many competitors is because they have an information/platform monopoly. Just like many other tech companies, ebay and facebook come to mind as obvious examples.
>You confused UI with Usability.
I certainly don't confuse the two.
I couldn't care less what Craigslist LOOKS like, but its usability is profoundly broken in some areas (Denver Metro, for example).
The problem is that there's a Denver, Boulder, Colorado Springs, and Fort Collins Craigslist. For those of you in the Bay Area, that would be roughly like having a distinct San Francisco, San Rafael, San Jose, and Santa Cruz Craigslist. If you're in between those places, which one do you look at? You have to look at more than one. Even if you're IN one of those places, which one do you look at? Sometimes three or four, if it's something you're willing to drive an hour to buy.
And if you're POSTING, where do you post? If you post a popular item to Boulder (because, say, you live in Boulder), then people in the next town over who only know about Denver won't see it, so you also have to post to Denver if you want everyone in a reasonable radius to see your item. Which means you have to violate Craigslist policies AND mess with your listing enough to get past its spam filter.
The sad thing is they've fixed it to the 80% level in the Bay Area, where the whole (extended!) Bay Area (even Santa Cruz, which isn't really Bay Area) is on a single Craigslist site with sub-area filters, but they just leave it broken here.
THIS is a usability problem.
And honestly so is the lack of "map these results," for times when you want to see which ones are closest, or which ones you could stop by on your way to work, or whatever.
It's not because it's not shiny or doesn't have gradients or drop shadows. It's because key features that I want from a classified site are sorely lacking.
I certainly don't confuse the two.
I couldn't care less what Craigslist LOOKS like, but its usability is profoundly broken in some areas (Denver Metro, for example).
The problem is that there's a Denver, Boulder, Colorado Springs, and Fort Collins Craigslist. For those of you in the Bay Area, that would be roughly like having a distinct San Francisco, San Rafael, San Jose, and Santa Cruz Craigslist. If you're in between those places, which one do you look at? You have to look at more than one. Even if you're IN one of those places, which one do you look at? Sometimes three or four, if it's something you're willing to drive an hour to buy.
And if you're POSTING, where do you post? If you post a popular item to Boulder (because, say, you live in Boulder), then people in the next town over who only know about Denver won't see it, so you also have to post to Denver if you want everyone in a reasonable radius to see your item. Which means you have to violate Craigslist policies AND mess with your listing enough to get past its spam filter.
The sad thing is they've fixed it to the 80% level in the Bay Area, where the whole (extended!) Bay Area (even Santa Cruz, which isn't really Bay Area) is on a single Craigslist site with sub-area filters, but they just leave it broken here.
THIS is a usability problem.
And honestly so is the lack of "map these results," for times when you want to see which ones are closest, or which ones you could stop by on your way to work, or whatever.
It's not because it's not shiny or doesn't have gradients or drop shadows. It's because key features that I want from a classified site are sorely lacking.
Because you're optimizing for engagement by measuring time on site or something like that
Oh good, nobody actually told you how they measured but you already know! Bravo.
Oh good, nobody actually told you how they measured but you already know! Bravo.
I can tell you from first-hand experience that they know what they're doing. They measure almost everything, nicely put together in dashboards with all the KPI's for the appropriate managers.
I think it's unlikely they used a suboptimal performance indicator to judge the site's new design, and would say the same thing about craigslist.
I think it's unlikely they used a suboptimal performance indicator to judge the site's new design, and would say the same thing about craigslist.
If only they would measure amount of spam in their for rent listing that currently nearly useless.
Remember, if the numbers don't support your opinion, it's because your measuring the wrong thing!
[deleted]
Keep in mind 95% of their site is not the product they are trying to sell. It is in craigslist's best interest to keep users glued to their site. If someone takes 30 minutes to find a used bike instead of five then great. Maybe there is a chance the user get's bored of cruising bike listings or on one of their many trips to see the latest postings will click on the job section. Craigslist uses their massive user base on the free portion of their site as a significant competitive advantage over all other job posting sites. It is actually in their best interest to make it harder for users to find what they want. Craigslist is not in the business of people selling stuff to each other, they are in the business of attracting enough eyeballs for enough time to their site to click on enough job listings to keep their revenues flowing. They have absolutely no incentive to change or improve. I am pretty sure when they look at their revenues (the ultimate metric), it is actually doing pretty well.
Despite the very basic UI, the Dutch version of Craigslist seems to be doing well based on the hundreds of apartment listings in, for example, Amsterdam:
http://amsterdam.craigslist.org/apa/
But if I might inject an opposing data point, nobody seems to use the Brazilian version of Craigslist. Check out Sao Paulo:
http://saopaulo.craigslist.org/apa/
Can you believe that there are just 5 apartment listings for a metropolitan area of 19.9 million!
It's incomprehensible to me why Craigslist doesn't work there, especially since all the alternatives involve paying some middleman for listing. The same goes for job sites and "things for sale" sites as well. Brazilians prefer to use some scammy-looking sites loaded with graphics and eye candy and often charging a fee, rather than the vanilla Craigslist.
I'm going to venture a guess that this has much more to do with trust (Brazilians want to feel that someone has vetted the data) than with good/bad UI or web design.
http://amsterdam.craigslist.org/apa/
But if I might inject an opposing data point, nobody seems to use the Brazilian version of Craigslist. Check out Sao Paulo:
http://saopaulo.craigslist.org/apa/
Can you believe that there are just 5 apartment listings for a metropolitan area of 19.9 million!
It's incomprehensible to me why Craigslist doesn't work there, especially since all the alternatives involve paying some middleman for listing. The same goes for job sites and "things for sale" sites as well. Brazilians prefer to use some scammy-looking sites loaded with graphics and eye candy and often charging a fee, rather than the vanilla Craigslist.
I'm going to venture a guess that this has much more to do with trust (Brazilians want to feel that someone has vetted the data) than with good/bad UI or web design.
There are various examples of Craiglist just simply not working in various countries. In fact I'm going to go out on a limb and posit that Craigslist really only had a monopoly in the U.S. Brazil uses olx (I think?), Argentina and Mexico among other Latin American countries use mercadolibre where Craigslist is more for baiting foreigners with higher rent prices and international shared accomodation. Australia has gumtree, New Zealand has trademe (and they both use Seek for jobs) and I would use marktplaats in the Netherlands as mentioned above. I forget the one for Spain. The list goes on..
'alister' made some good points up top. Let me get further into the rental side of the subject.
As someone who, on many occasions, has rented in Brazil using classifieds sites, it's a mess. 'We' really, really need a CL-type site that everyone knows and uses. As a Californian, and former user of CL, it boggles my mind why CL-Brazil isn't used. It's there (here), it's usable, simple, and in Portuguese.
Instead, people use OLX (free) a little, Vivastreet (free) a little, EasyQuarto (paid) a little, and a few other sites that are either less popular or only work sometimes. Couchsurfing city-specific groups usually each have a sub-group for renting but a strange occurrence takes place there, almost no one puts the price-point. It's called doing business, paying money for a product or service, yet there's a real aversion to giving up the rental price in one's ad. Luckily, the more popular sites I mentioned make it so one has to list their rental price.
And my final two gripes are that very few people respond to inquiries into any particular ad (I think this is universal though) and a good 60% of ads here in Brazil only want women renters.
It's tough, I tell you...
As someone who, on many occasions, has rented in Brazil using classifieds sites, it's a mess. 'We' really, really need a CL-type site that everyone knows and uses. As a Californian, and former user of CL, it boggles my mind why CL-Brazil isn't used. It's there (here), it's usable, simple, and in Portuguese.
Instead, people use OLX (free) a little, Vivastreet (free) a little, EasyQuarto (paid) a little, and a few other sites that are either less popular or only work sometimes. Couchsurfing city-specific groups usually each have a sub-group for renting but a strange occurrence takes place there, almost no one puts the price-point. It's called doing business, paying money for a product or service, yet there's a real aversion to giving up the rental price in one's ad. Luckily, the more popular sites I mentioned make it so one has to list their rental price.
And my final two gripes are that very few people respond to inquiries into any particular ad (I think this is universal though) and a good 60% of ads here in Brazil only want women renters.
It's tough, I tell you...
I suspect CL doesn't fly in some countries mainly because other companies got the jump on them there.
Same here in France. It looks like the only ads on fr.craigslist.com are either posted by americans living in France (they're allin english) or spam.
I have never heard of anyone buying/selling stuff on craigslist among my friends/relatives.
The (pretty recent) equivalent would be http://leboncoin.fr, I guess.
The (pretty recent) equivalent would be http://leboncoin.fr, I guess.
That makes sense. I think a lot of folks advocate maps and filter sliders because they over-estimate an average user's comfort level with such advanced features. I think a non-technical person is much more content with spending several times more time just sifting through listings manually instead of having to learn some whiz-bang (in their mind) gadget.
[deleted]
TBH that website isn't much of an improvement.
> I would be willing to bet a large amount of money that adding a map to search results and letting me put an upper limit on the number of bedrooms I'm looking for would not make people less likely to use Craigslist.
So when can we expect to see your show HN: Craiglist done right?
So when can we expect to see your show HN: Craiglist done right?
For the housing vertical. It looked like this. http://www.padmapper.com/
Building the right interface is simple. The real problem of building any marketplace is solving the chicken/egg problem of having a lot of buyers and sellers.
Craigslist does not avoid disruption because it has the best product, it avoids disruption because it has a huge marketplace with a lot of buyers and sellers.
Craigslist does not avoid disruption because it has the best product, it avoids disruption because it has a huge marketplace with a lot of buyers and sellers.
Other good examples of these content-over-form sites are 2ch.net in japan, and 4chan for the western world. (4chan recently updated its HTML for the first time in something like eight years.)
I suppose network effects (and the posting model) make up a large part of their popularity, but there's something to be said for a reliable and functional interface that works well and doesn't change on a whim.
I suppose network effects (and the posting model) make up a large part of their popularity, but there's something to be said for a reliable and functional interface that works well and doesn't change on a whim.
There's a lot to be said for it.
It's constant in hn discourse to put great UI up there and to give advice on it.
Yet if anyone here has sat on a Bloomberg terminal, used Craigslist, or even games like net hack or dwarf fort, it's pretty clear that interface can be damned.
Not in all cases perhaps, agreed. But in these cases, after a while of using it people get faster and develop the muscle memory to become efficient on the site, so the UI fades into the background anyway.
It's constant in hn discourse to put great UI up there and to give advice on it.
Yet if anyone here has sat on a Bloomberg terminal, used Craigslist, or even games like net hack or dwarf fort, it's pretty clear that interface can be damned.
Not in all cases perhaps, agreed. But in these cases, after a while of using it people get faster and develop the muscle memory to become efficient on the site, so the UI fades into the background anyway.
> after a while of using it people get faster and develop the muscle memory to become efficient on the site, so the UI fades into the background anyway.
Jared Spool has said that when you change the user interface - fairly dramatically - you frustrate your regular users because they are now unfamiliar with it and have to restart their learning/experience curve.
"At eBay, they learned the hard way that their users don't like dramatic change. One day, the folks at eBay decided they no longer liked the bright yellow background on many of their pages, so they just changed it to a white background.
Instantly, they started receiving emails from customers, bemoaning the change. So many people complained, that they felt forced to change it back."
http://www.uie.com/articles/death_of_relaunch/
Jared Spool has said that when you change the user interface - fairly dramatically - you frustrate your regular users because they are now unfamiliar with it and have to restart their learning/experience curve.
"At eBay, they learned the hard way that their users don't like dramatic change. One day, the folks at eBay decided they no longer liked the bright yellow background on many of their pages, so they just changed it to a white background.
Instantly, they started receiving emails from customers, bemoaning the change. So many people complained, that they felt forced to change it back."
http://www.uie.com/articles/death_of_relaunch/
"If it works, its good enough?"
Honestly thinking about other sites, digg died because of content, reddit survives because of content; even if someone could improve the interface, its not really going to matter.
Honestly thinking about other sites, digg died because of content, reddit survives because of content; even if someone could improve the interface, its not really going to matter.
It's interesting to consider digg... the canonical reason for its death (CEO replaced within a week, 40% of staff laid off two months later) was Digg-v4, a graphical overhaul that also removed features.
Google is definitely content over interface (most of the time). Facebook... not so much - millions of people join protest petitions against every minor interface change - but the value of having all your friends together is still so much greater than the alternative of changing service, whereas reddit was a short step away for digg users.
4chan definitely remains popular because of content - canv.as, a similar project by the same owner, has nowhere near the traction despite being more demographically targeted for mass-market appeal.
I think the lesson is, if you have a close-enough competitor and a fickle userbase, one screwup is all it takes to push critical mass over the edge. If i ran a large website i would study the Digg-v4 case very carefully. Going back to Craigslist, i don't think they have to worry too much about losing their top spot just yet.
Google is definitely content over interface (most of the time). Facebook... not so much - millions of people join protest petitions against every minor interface change - but the value of having all your friends together is still so much greater than the alternative of changing service, whereas reddit was a short step away for digg users.
4chan definitely remains popular because of content - canv.as, a similar project by the same owner, has nowhere near the traction despite being more demographically targeted for mass-market appeal.
I think the lesson is, if you have a close-enough competitor and a fickle userbase, one screwup is all it takes to push critical mass over the edge. If i ran a large website i would study the Digg-v4 case very carefully. Going back to Craigslist, i don't think they have to worry too much about losing their top spot just yet.
What utter bs - padmapper is not "stealing" anything - padmapper is the best method of viewing cl housing listings there is.
I berated Craig over on Quora about this a while back - to say that he doesn't want to break it is bullshit - he is scared and lazy to better the site through even the smallest of updates.
The fact that "this posting has been deleted by its author" and "this post has been flagged for removal" still show up in results is down right infuriating.
Providing simply no way to browse items by picture is also laughable.
The content on CL is what is valuable, and Craig has been rumored to make 25 million per year off the site, and he doesn't want to muck with it because he doesn't want to hurt that income - but while doing so, he is keeping the content locked up in an abhorrent UX.
I berated Craig over on Quora about this a while back - to say that he doesn't want to break it is bullshit - he is scared and lazy to better the site through even the smallest of updates.
The fact that "this posting has been deleted by its author" and "this post has been flagged for removal" still show up in results is down right infuriating.
Providing simply no way to browse items by picture is also laughable.
The content on CL is what is valuable, and Craig has been rumored to make 25 million per year off the site, and he doesn't want to muck with it because he doesn't want to hurt that income - but while doing so, he is keeping the content locked up in an abhorrent UX.
You just answered your own question. If your project made 25 million per year, would you change it? I know I wouldn't... Additionally, he is keeping the data locked up because that is his prerogative. Why should he go through the time and expense of developing some kind of API if he does not want to do that? If he is happy with how the website looks and works right now, why in the blue blazes should he change it?
> [Craig] is scared and lazy to better the site through even the smallest of updates.
I'd be the same. Why mess with a goose that's already laying more gold eggs than you can use? For public service reasons maybe, but I'm confident CL is a (highly profitable) business foremost.
I'd be the same. Why mess with a goose that's already laying more gold eggs than you can use? For public service reasons maybe, but I'm confident CL is a (highly profitable) business foremost.
Bah. Just because improve a web site is hard doesn't mean that you don't have a responsibility to your users to TRY. Craigslist has not. There are ways to experiment in a low-impact way. Hell, they could offer a "plugin" system, so that users could add the padmapper plugin.
People keep building things using Craigslist data that users LOVE and Craigslist keeps shutting them down.
Craigslist is one of the great squandered opportunities for awesomeness on the internet.
People keep building things using Craigslist data that users LOVE and Craigslist keeps shutting them down.
Craigslist is one of the great squandered opportunities for awesomeness on the internet.
People come back because craigslist has monopoly power. Craigslist knows this. They are afraid to release an API that would allow people to experiment with UI because they know that would train people to go to sites other than craigslist (I save hours using padmapper over Craigslist).
If they don't want to change their layout, why not let sites like PadMapper build on it the way that it has been doing? That way, old-fashioned users can keep using just CL and people in need of something slicker can use PM+CL.
I also don't understand why classifieds content is proprietary? Aren't they just facts? Please can someone explain?
I also don't understand why classifieds content is proprietary? Aren't they just facts? Please can someone explain?
Because that would endanger their monopoly.
Does that mean other people should steal the content and profit from it?
Craigslist is unwilling to let Padmapper buy a license to use their content. This isn't as black and white as "stealing content".
Craigslist is unwilling to let Padmapper buy a license to use their content. This isn't as black and white as "stealing content".
Content not available for licensing is a justification for infringement? That doesn't ring true at all for me - exclusive content is often much more valuable to a business than content that's licensed. I doubt anyone would think that goldman sachs coverage being distributed by morgan stanley would fit under some "no harm, no foul" ethos even if Morgan significantly improved on the layout of the pages.
I don't agree with the infringement, I just think that boiling it down to "stealing" is too reductionist. The reality here is a bit more complicated.
Seriously, I think I am the only person who doesn't think it is OK to torrent Game of Thrones. :)
Seriously, I think I am the only person who doesn't think it is OK to torrent Game of Thrones. :)
Is there a rock bottom hackernews will reach for leaning on shockingly bad analogies to make a point?
Its their content, just because they won't let you use it doesn't mean you can break in and grab it.
They also don't have an API, and they also don't allow you to pay for a license or API for websites.
Do you have anything you're unwilling to sell? And if so is it cool if I just take it?
This is unfortunately an increasingly common viewpoint. See for example the recent "Take My Money, HBO" site where people justify pirating the content because HBO won't sell episodes/seasons directly to them.
While I agree with you that taking something somebody isn't selling is wrong, it is still a force any content creator must come to terms with if they want to be realistic about 21st century content creation.
The difference here is that CL still has the content. We always end up in the same fruitless discussion with different subjects: software patents, content, digital media. Should you be able to own something if copying it is practically free? The only right answer is "no", but nobody likes it.
What about if instead of taking it I take a picture with my "3D camera" and then have my "3D printer" create one.
Am I still stealing it?
Am I still stealing it?
[deleted]
If this is true, then he (Craig Newmark) really needs to read the innovator's dilemma (http://www.amazon.com/The-Innovators-Dilemma-Revolutionary-B...). TLDR: If you allow yourself to become a prisoner to your success, you're actually making your continued success unlikely. You can solve the dilemma by spinning out a smaller, irreverant version of yourself & giving them what they need to destroy the old company.
In craigslist's case, he could include a link on the old site that allows users to upgrade to a more modern version, and have the modern site basically consume an api/scrape the old CL till it gained momentum.
In craigslist's case, he could include a link on the old site that allows users to upgrade to a more modern version, and have the modern site basically consume an api/scrape the old CL till it gained momentum.
Padmapper doesn't make money from Craigslist listings.
It also sends traffic back to CL. It shows the summary of the listing, and then when the user is interested, they can click through the pin to the actual Craigslist listing.
It also sends traffic back to CL. It shows the summary of the listing, and then when the user is interested, they can click through the pin to the actual Craigslist listing.
Most landlords probably never even realize the traffic first got to CL through Padmapper at all. Whenever I'm asked, I automatically just say CL. Bringing up Padmapper is just likely to confuse.
CL is deliberately making its service less useful both to its paying customers and the public at large. And the sole justification for this is fear that Padmapper could grow to be a bigger, better competitor someday in the distant future.
Leveraging a monopoly to destroy technological progress for fear of future competition is irksome.
It's almost certainly not illegal, and maybe not even morally wrong. But it still sucks.
Yet for all that, I'm moving on July 15th, and I think there's about a 95% chance I'll find the place through CL. That's where the market is.
CL is deliberately making its service less useful both to its paying customers and the public at large. And the sole justification for this is fear that Padmapper could grow to be a bigger, better competitor someday in the distant future.
Leveraging a monopoly to destroy technological progress for fear of future competition is irksome.
It's almost certainly not illegal, and maybe not even morally wrong. But it still sucks.
Yet for all that, I'm moving on July 15th, and I think there's about a 95% chance I'll find the place through CL. That's where the market is.
How does "Leveraging a monopoly to destroy technological progress" not fall under some antitrust laws? Is it just because Craigslist is free? I definitely feel the consumer is potentially losing out due to Craigslist's monopolistic practices.
CL isn't US Steel. I doubt Craigslist even has a straight 40% of rental market share. But for urban 20-somethings looking for cheap living situations and roommates? It feels like it has 95% market share. At least in Portland.
This is all theorycraft though. If anyone had hard numbers on CL's market share for rentals I'd love to see them!
This is all theorycraft though. If anyone had hard numbers on CL's market share for rentals I'd love to see them!
You would think that they'd be able to measure these things. How hard can your flavor of a/b testing (or bandit testing for the bros) be to implement for a company like Craig's list.
It's just pure laziness and I'm sure the right talented team could make a product that beats Craig's list
It's just pure laziness and I'm sure the right talented team could make a product that beats Craig's list
I would still conjecture that Craigslist succeeded in spite of their design, not because of it. That's the problem with these things, it's really hard to reverse engineer a success like Craigslist and figure out which of the moving pieces were real contributors.
>he's scared to make drastic changes to the UX specifically
a/b test
a/b test
Then open an API and let OTHERS do it at their expense...
So your argument is "Since they are not doing everything they can with their content, I can steal it"?
It's hard for me to see this as stealing content since:
1) Padmapper shows the original craigslist page when you click through to the details page. 2) Padmapper doesn't seem to be monetizing itself in any obvious way, at least not directly through ads on content. 3) Craigslist doesn't monetize its content through ads, anyway.
Essentially all it offers is a wrapper that lowers the friction to discovery. You know, like those kids over at Google. Or Bing. Or DuckDuckGo. Are they stealing content?
1) Padmapper shows the original craigslist page when you click through to the details page. 2) Padmapper doesn't seem to be monetizing itself in any obvious way, at least not directly through ads on content. 3) Craigslist doesn't monetize its content through ads, anyway.
Essentially all it offers is a wrapper that lowers the friction to discovery. You know, like those kids over at Google. Or Bing. Or DuckDuckGo. Are they stealing content?
You expect Google, Bing and DDG to honor requests not to be indexed, don't you? How is this different?
And what Craigslist does or doesn't do to make money shouldn't change what you're allowed to do with their content.
And what Craigslist does or doesn't do to make money shouldn't change what you're allowed to do with their content.
PadMapper obeys robots.txt's, for the record. Also, it doesn't repost their content, it reposts facts about the content, which is a pretty key difference.
They've had an informal amnesty for services using their stuff for a long time, Craig has stated that they're OK with services that interface with them as long as they don't use many server resources, but they recently updated their TOU and started sending out huge waves of C&D's a few weeks ago, based on my talking with people.
They've had an informal amnesty for services using their stuff for a long time, Craig has stated that they're OK with services that interface with them as long as they don't use many server resources, but they recently updated their TOU and started sending out huge waves of C&D's a few weeks ago, based on my talking with people.
The difference is that the search engines aggregate from the whole web, PadMapper was just appropriating content from CL, which is against their TOS. It's very clearly spelled out.
All the other companies that have tried have also been told to C&D. If you don't like it, start your own network...
From http://www.craigslist.org/about/terms.of.use
"Any copying, aggregation, display, distribution, performance or derivative use of craigslist or any content posted on craigslist whether done directly or through intermediaries (including but not limited to by means of spiders, robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited."
All the other companies that have tried have also been told to C&D. If you don't like it, start your own network...
From http://www.craigslist.org/about/terms.of.use
"Any copying, aggregation, display, distribution, performance or derivative use of craigslist or any content posted on craigslist whether done directly or through intermediaries (including but not limited to by means of spiders, robots, crawlers, scrapers, framing, iframes or RSS feeds) is prohibited."
PadMapper gets content from more than just Craigslist. They have their own postings (padlister.com), sublet.com, apartments.com, apartmentfinder.com and a lot more. There's really not that much difference between them and a search engine.
I've heard this argument many times before, it was the same with Oodle, but PadMapper's business model is to aggregate apartment listings (Google's is not) and this is in contradiction to the TOS so inevitably Craig Newmark sends them a C&D letter.
Several innovative solutions have been invented over the years, the most obvious one is to do the CL scraping on the client, this way there is no indexing server to block, it's just the client reading all the search listings, parsing them and offering faceted filters. It breaks the spirit of the TOS but it would be impossible for CL to block. I am not advocating this, just pointing out one of the many approaches that have been attempted over the years.
Several innovative solutions have been invented over the years, the most obvious one is to do the CL scraping on the client, this way there is no indexing server to block, it's just the client reading all the search listings, parsing them and offering faceted filters. It breaks the spirit of the TOS but it would be impossible for CL to block. I am not advocating this, just pointing out one of the many approaches that have been attempted over the years.
Not true. Padmapper has multiple DBs.
Google can scan apartment listings and show them as results in their format. PadMapper cant. What's the difference?
See http://www.craigslist.org/robots.txt .
Google can scan apartment listings and show them as results in their format. PadMapper cant. What's the difference?
See http://www.craigslist.org/robots.txt .
I'm not sure why those TOUs would prohibit Padmapper but not Google.
The immediately following text in the CL ToS reads:
As a limited exception, general purpose Internet search engines and noncommercial public archives will be entitled to access craigslist without individual written agreements executed with CL that specifically authorize an exception to this prohibition if, in all cases and individual instances: (a) they provide a direct hyperlink to the relevant craigslist website, service, forum or content; (b) they access craigslist from a stable IP address using an easily identifiable agent; and (c) they comply with CL's robots.txt file ...
As a limited exception, general purpose Internet search engines and noncommercial public archives will be entitled to access craigslist without individual written agreements executed with CL that specifically authorize an exception to this prohibition if, in all cases and individual instances: (a) they provide a direct hyperlink to the relevant craigslist website, service, forum or content; (b) they access craigslist from a stable IP address using an easily identifiable agent; and (c) they comply with CL's robots.txt file ...
What a silly analogy. If I have a car and you steal it from me, you now have a car and I don't. What PadMapper does is it takes a picture of my car and shows/sells it to people. Analogies require handling with care :-)
Please note that there was no analogy in my original comment. I was merely summing up the OP's argument. Additionally, as someone who has plans to write a craigslist-centric app, and took the time to read the TOS, I find it idiotic for a developer to charge ahead with creating an app/site without either understanding the terms, or obeying them.
What's more is that you are arguing against the notion that whoever creates the content (or own's the content) has the right to decide what gets done with that content. And no matter how much you dislike what they are doing with their content (ie. not creating a shiny interface to access said content), that does NOT give you the right to use that content without permission. It is my understanding that craigslist often gives out permissions to use it's content, and it would appear as though this company didn't bother to secure that permission.
What's more is that you are arguing against the notion that whoever creates the content (or own's the content) has the right to decide what gets done with that content. And no matter how much you dislike what they are doing with their content (ie. not creating a shiny interface to access said content), that does NOT give you the right to use that content without permission. It is my understanding that craigslist often gives out permissions to use it's content, and it would appear as though this company didn't bother to secure that permission.
Kika's analogy is also incorrect. What PadMapper does is takes a picture of information like a book, recipe, source code, etc. and then posts it on the Internet so that people can consume the information without going to Craigslist's website.
This is without the permission of Craigslist or of the original submitter or the content.
This is without the permission of Craigslist or of the original submitter or the content.
Without going to the craigslist this information is vastly incomplete. You just see where the property is. Then you click through to the craigslist website. Well, I haven't used PM for a while, but when I was renting it was that way.
Whether its 'stealing' is debatable, whether lack of innovation leads to whatever-you-want-to-call-it is not. Just ask the music and movie industries.
The content isn't theirs. They just have a license to display it.
Agreed. Housing, where location is easily the most important factor, is where Craigslist is least usable as a product. Padmapper's success is a testament to Craigslist's users wanting something better. This is a big middle finger to all those users.
Doubly agreed. With Padmapper you could tell whether or not the apartment you found was cheap because it was truly a good deal or just in the ghetto. Very useful in the bay area :)
I agree with you, but if PadMapper is really that much better, what's stopping them from getting users to post directly on their site?
Network effect. Sellers go to CL because that's where the buyers are, and buyers go to CL because that's where the sellers are. Other features come in play after that. Once the network effect sets in it's almost impossible to unseat the incumbent: eBay, Linked-in, Facebook, Craigslist, etc. The dominant company has 90%+ share, and can stay that way forever doing very little.
eBay, for example, hasn't changed its (rather mediocre) interface in years, and managed to alienate at least some sellers with their policies. And yet, no one managed to make a dent in eBay's position.
Network effect is the reason there is funding for a million social network start-ups. Everyone wants to capture it. But the odds of success, on the other hand, are also one in a million.
I believe there was a post recently on HN where someone pointed out how these things work - one day it just takes off. Some combination of factors (three out of 100, perhaps) gets right and people flock to the site. And then the fate is sealed. Even in retrospect it's often impossible to say what factors were the key. So, I wouldn't be surprised if founders of Craigslist have little clue as to why it worked.
I'd be really really interested to see what it takes for someone to unseat Craigslist (or eBay, or any other major network).
eBay, for example, hasn't changed its (rather mediocre) interface in years, and managed to alienate at least some sellers with their policies. And yet, no one managed to make a dent in eBay's position.
Network effect is the reason there is funding for a million social network start-ups. Everyone wants to capture it. But the odds of success, on the other hand, are also one in a million.
I believe there was a post recently on HN where someone pointed out how these things work - one day it just takes off. Some combination of factors (three out of 100, perhaps) gets right and people flock to the site. And then the fate is sealed. Even in retrospect it's often impossible to say what factors were the key. So, I wouldn't be surprised if founders of Craigslist have little clue as to why it worked.
I'd be really really interested to see what it takes for someone to unseat Craigslist (or eBay, or any other major network).
I think there's more to it than that. I know people for whom craigslist is one of the very few websites they interact with. It's a very different demographic from the one that hangs on HN. I think this is a big factor in the CL design and the reluctance to change it.
I also think the network effect is being overstated in this instance. eBay has much stronger lock-in because I can't list my item in more than one auction at a time. But I can certainly list my apartment in more than one service.
I already check the local paper in addition to CL when looking for an apartment. If I'm looking for a bargain, I'm fine with shopping around.
I also think the network effect is being overstated in this instance. eBay has much stronger lock-in because I can't list my item in more than one auction at a time. But I can certainly list my apartment in more than one service.
I already check the local paper in addition to CL when looking for an apartment. If I'm looking for a bargain, I'm fine with shopping around.
If you don't think Craigslist has an insanely strong network effect, you haven't been thinking about this long/hard enough. Why do you think it's one of the very few websites those people interact with? It's not just because it's simple (though that helps a lot). It's that a) everyone uses it and b) it has everything you could want. It means you don't have to remember 50 random websites, one for each vertical.
I may be in the minority in Silicon Valley, but I'm honestly a really big fan of Craigslist, even despite their C&D'ing me, and I don't think it could or should be "disrupted". I think it's perfect for most of the things on there: lightning fast, few restrictions, community driven. I think it's massively better than that other alternative of having a ton of small websites that each do one thing well, but differently. I shop for new stuff almost exclusively via Amazon for the same reason that I shop exclusively on CL for used stuff.
I may be in the minority in Silicon Valley, but I'm honestly a really big fan of Craigslist, even despite their C&D'ing me, and I don't think it could or should be "disrupted". I think it's perfect for most of the things on there: lightning fast, few restrictions, community driven. I think it's massively better than that other alternative of having a ton of small websites that each do one thing well, but differently. I shop for new stuff almost exclusively via Amazon for the same reason that I shop exclusively on CL for used stuff.
I think CL works because to most end users, it's really no different than newspaper classifieds. Classified users expect to have to wade through a bunch of useless stuff to find the diamonds in the rough.
Time for someone to make a hidden service on tor. Pad mapper is just way too useful.
What does the usability of their product have to do with it? It's their product. I know, "everything is free on the Internet, man".
[deleted]
couldn't agree more.
i've tried to write my own scraper in the past to make the search more usable (at least show photos from each post next to the link so that i don't have to click through to every single link to see what pieces are for sale) and i was quickly blocked. keep in mind this was not a public site viewed by 100's of people. it was me making one search worth of individual link requests a couple times. blocked. its craigslist way or nothing.
Craigslist, Plenty Of Fish plus some others are examples of sites that took a really minimalist approach to design but managed to pull in a massive market share through fulfilling a customer need. Revenue from advertising paid the bills.
We wrote about ugly design before, how in some cases it actually increases conversion rates. It's odd but sometimes ugly is beautiful...
http://sparknlaunch.wordpress.com/2012/02/02/keep-it-simple-...
We wrote about ugly design before, how in some cases it actually increases conversion rates. It's odd but sometimes ugly is beautiful...
http://sparknlaunch.wordpress.com/2012/02/02/keep-it-simple-...
They created a product that was against their TOS. It's pretty simple really.
If they are doing a great job and are so much better than CL then no doubt everyone will come back and use their site now they have the realtors on board.
I dislike people hating on another company for having a crappy product, just make a better one and compete against them, if they really are that bad then whatever network lock-in they have wont last. You can't argue they are crap, then try and use them to monetise your own product.
If they are doing a great job and are so much better than CL then no doubt everyone will come back and use their site now they have the realtors on board.
I dislike people hating on another company for having a crappy product, just make a better one and compete against them, if they really are that bad then whatever network lock-in they have wont last. You can't argue they are crap, then try and use them to monetise your own product.
I am totally upset. I wanted to use padmapper to search for an apartment for my next move. I had been looking with it for months and wow it was super helpful.
Please, make a client-side solution so they can't shut you off! A browser based plugin, for example that allows us to plot the listings on our end.
I wrote this email to craigslist:
Hi Craig and Jim,
I am very upset that you cut off padmapper. Using padmapper cuts out a lot of the spam that ends up in the apartment listings due to the fact that it doesn't have a lat,long pair most of the time. It also makes it much easier to search for apartments based on your preference. With padmapper, I could see if the apartment I'm looking at is in a safe area, how far it is from a potential employer, and a myriad of other great things.
Finding an apartment with craigslist, in spite of the fact that you have the data, is much harder than searching on padmapper.
I really wish there was a way you could work with them!
You guys are a solid blue chip, no doubt. But if you take this oligarchical perspective to your business, someone is going to beat you eventually.
Thank you for your continued service
Please, make a client-side solution so they can't shut you off! A browser based plugin, for example that allows us to plot the listings on our end.
I wrote this email to craigslist:
Hi Craig and Jim,
I am very upset that you cut off padmapper. Using padmapper cuts out a lot of the spam that ends up in the apartment listings due to the fact that it doesn't have a lat,long pair most of the time. It also makes it much easier to search for apartments based on your preference. With padmapper, I could see if the apartment I'm looking at is in a safe area, how far it is from a potential employer, and a myriad of other great things.
Finding an apartment with craigslist, in spite of the fact that you have the data, is much harder than searching on padmapper.
I really wish there was a way you could work with them!
You guys are a solid blue chip, no doubt. But if you take this oligarchical perspective to your business, someone is going to beat you eventually.
Thank you for your continued service
Great email... I will follow suit. They cut off the listings right in the middle of a search and it took me a little while to figure out what happened.
Hi everyone, Eric from PadMapper here, thanks for all the comments!
To everyone saying that this is lifting their content, I disagree - it makes a summary of the content and then points back to the original. If I wanted to lift their stuff, I would have made separate pages that laid things out better than the original. Literally none of the text except the title and location string make it into the summary.
It's not a copyright issue, their main legal beef is that it's against their TOU for anything but general search engines to index their content - no vertical ones allowed. Who knows what their actual motivation is.
To everyone saying that this is lifting their content, I disagree - it makes a summary of the content and then points back to the original. If I wanted to lift their stuff, I would have made separate pages that laid things out better than the original. Literally none of the text except the title and location string make it into the summary.
It's not a copyright issue, their main legal beef is that it's against their TOU for anything but general search engines to index their content - no vertical ones allowed. Who knows what their actual motivation is.
Do you think they're shutting you down in preparation of creating their own version of pad paper an releasing that without your competition?
It's possible, they've been looking for a dev with maps experience for a while.
Serious question: Do you think CL would actually follow through with a lawsuit? It seems like they fire off C&D's all the time, but I have yet to see serious follow through.
No idea, but I've done a little bit of looking, and they have a history of attacking pretty aggressively, at least against things that auto-post. Those are mostly tools for spammers, though, so who knows if they'd come after me.
I don't think you are bound by their TOS simply by indexing their site. Whether it makes business sense to test this legal theory is a wholly different question.
Dunno, I guess my servers are accessing theirs? There are other ways to get the data, but yeah, I don't really want to piss them off. Was much nicer when I thought they were OK with this kind of thing from Craig's quora comments.
Good ol CLAD!
For those wishing to take action:
[email protected] – Jim Buckmaster, CEO
[email protected] – Craig Newmark, Founder
@jimbuckmaster on twitter - http://twitter.com/jimbuckmaster
@craignewmark on twitter - http://twitter.com/craignewmark
Simple email to copy and paste:
Jim and Craig,
I'm a longtime user of Craigslist, but I'm really disappointed in your decision regarding PadMapper. It would be great if you could weigh in on the debate happening on Hacker News or elsewhere.
Thanks,
[email protected] – Jim Buckmaster, CEO
[email protected] – Craig Newmark, Founder
@jimbuckmaster on twitter - http://twitter.com/jimbuckmaster
@craignewmark on twitter - http://twitter.com/craignewmark
Simple email to copy and paste:
Jim and Craig,
I'm a longtime user of Craigslist, but I'm really disappointed in your decision regarding PadMapper. It would be great if you could weigh in on the debate happening on Hacker News or elsewhere.
Thanks,
I for one, commend Craigslist for their commitment to maintaining a strict, consistent, well-defined UI and moreover, for choosing to maintain a relatively modest operation to the benefit of their core users instead of succumbing to the promises of riches that so often end up ruining great services created by the Tech Community. Many of you are saying this is detrimental to consumers, and it may be in the short term, because services like PadMapper really do provide a better experience. But in the long-term, things seem to be much different. Too many recent startups launch with promises of "changing the world" only to be bought and dismantled a few short years later, setting their community adrift. Craigslist's chief benefit to their users, and to the web community as a whole, is that they have a proven track record of stability. Thus, who can blame them for trying to protect their marketplace from getting eroded by outside interests? If they bowed to the current pressure and allowed outside access to their data, the web would risk losing a long-term standard, and, these days, consistent, trust-worthy standards are in short supply.
Great thoughts. Is > $100 million annual revenue really "relatively modest" though?
Well considering they are the 9th most visited site in the US and the companies ahead of it are all making revenues in the Billions (except Wikipedia and Twitter), then, ridiculously enough, $100 million sounds pretty modest.
Good luck, but I doubt it will work.
My first experience with craigslist was trying to figure out why they had the capability to block my wife's ad for a free dog that contained a mispelled word while the site's creator was busy trying to tell the government that there was nothing he could do about the many child prostitution ads that appeared on craigslist.
My first experience with craigslist was trying to figure out why they had the capability to block my wife's ad for a free dog that contained a mispelled word while the site's creator was busy trying to tell the government that there was nothing he could do about the many child prostitution ads that appeared on craigslist.
> a mispelled word
Oh, irony.
Oh, irony.
It sounds like you ran afoul of the fickle community moderation. The last time I checked, CL doesn't block ads, they just get flagged off by regular users.
One depends on a spellchecker?
If you can parse text for spelling, surely you can parse text for the sexual exploitation of minors.
I would think that these ads aren't exactly to-the-point considering what is offered is extremely illegal. I sincerely doubt you could reliably identify them based on keywords alone.
If it were so easy to spot child exploitation ads, why doesn't the police or the FBI monitor sites like Craigslist? I'd much rather see the people involved in that filthy business prosecuted than simply having their ads blocked (which doesn't do anything for the victims).
If it were so easy to spot child exploitation ads, why doesn't the police or the FBI monitor sites like Craigslist? I'd much rather see the people involved in that filthy business prosecuted than simply having their ads blocked (which doesn't do anything for the victims).
I hope this doesn't get down-voted for the confusing title (I know it's the title of the Blog post itself; but out of context it's a bit misleading).
When I moved from Atlanta to Boston, PadMapper was invaluable in finding a place to live. Given that 1) we didn't know the area, and 2) most rentals in Boston and Cambridge are rented by individuals rather than part of larger complexes, it was great to be able to have the listings from Craigslist on a map.
Ditto when I moved from Boston to San Jose. My wife and I were only interested in a house for rent and the vast majority of those could be found on Craigslist. Again, we didn't know the area so having a map was beyond valuable.
I sincerely hope Craigslist does a 180 on this one; PadMapper is a great tool that deserves access to that data.
When I moved from Atlanta to Boston, PadMapper was invaluable in finding a place to live. Given that 1) we didn't know the area, and 2) most rentals in Boston and Cambridge are rented by individuals rather than part of larger complexes, it was great to be able to have the listings from Craigslist on a map.
Ditto when I moved from Boston to San Jose. My wife and I were only interested in a house for rent and the vast majority of those could be found on Craigslist. Again, we didn't know the area so having a map was beyond valuable.
I sincerely hope Craigslist does a 180 on this one; PadMapper is a great tool that deserves access to that data.
>I hope this doesn't get down-voted for the confusing title
Specially when the site is down. Pretty annoying to find this kind of reddit like title here.
From comments my guess is that it's some blog post from someone who is pissed off at CL for forbidding them to use some web app they built (PadMapper?) that scrap content from the CL site or something like that.
Specially when the site is down. Pretty annoying to find this kind of reddit like title here.
From comments my guess is that it's some blog post from someone who is pissed off at CL for forbidding them to use some web app they built (PadMapper?) that scrap content from the CL site or something like that.
PadMapper is a great tool that deserves access to that data.
Why do they deserve access?
Why do they deserve access?
"deserve" is probably the wrong word (forgive me for hastily typing out my comment before running off to a meeting).
My point was really about PadMapper acting in good faith with Craigslist; they do send traffic to Craigslist for the full data on the properties and such. They're a good company doing good things--and it's sad that Craigslist's refusal to innovate has now bled over into "no one else can innovate for us"
My point was really about PadMapper acting in good faith with Craigslist; they do send traffic to Craigslist for the full data on the properties and such. They're a good company doing good things--and it's sad that Craigslist's refusal to innovate has now bled over into "no one else can innovate for us"
Ok, no problem. I've just made a habit of calling people out when they use "deserve" a little too frivolously.
Because it's highly beneficial to people that use craigslist, including the people that list properties. Craigslist is only hurting itself!
First it was Carsabi and now Padmapper. While I'm not surprised, as the Craigslist ToS explicitly prohibits crawling their site, it is incredibly frustrating that the website with all the data is unwilling to innovate and unwilling to help others innovate.
Craigslist will continue to be a mediocre, "good enough" solution. Since everyone associates online classifieds with craigslist, none of the other classifieds sites seem to have a chance. It's not even as if services like carsabi or padmapper are competing in any way with them.
Craigslist will continue to be a mediocre, "good enough" solution. Since everyone associates online classifieds with craigslist, none of the other classifieds sites seem to have a chance. It's not even as if services like carsabi or padmapper are competing in any way with them.
Craig Newmark has often said that they innovate in response to their community. That would seem to have three effects:
- Incrementalism: faster horses instead of cars
- Suburbanism: those in the community want it to stay familiar
- Majoritarianism: outsiders, potential users, occasional users, mavericks and dreamers have little impact
- Incrementalism: faster horses instead of cars
- Suburbanism: those in the community want it to stay familiar
- Majoritarianism: outsiders, potential users, occasional users, mavericks and dreamers have little impact
Wow this is really surprising, especially after reading Craig Newmark's answer on Quora just the other day to the question "Why hasn't anyone built any products on top of Craigslist data?". His response:
"Actually, we take issue with only services which consume a lot of bandwidth, it's that simple."
http://www.quora.com/Why-hasnt-anyone-built-any-products-on-...
"Actually, we take issue with only services which consume a lot of bandwidth, it's that simple."
http://www.quora.com/Why-hasnt-anyone-built-any-products-on-...
I've used padmapper twice to get an apartment so far, and It's great! I've loved the fact that it's constantly improving and has exactly the features I need: Walkscore, bookmarking, map, proper filters. It's strictly better. To the author: I would have paid for this.
It sucks that this waas mainly an interface to craigslist, and now they're gone. I hope this gets enough critical mass that eventually so that the unusable but unexplicably popular horror that is craigslist apartments just dies an ugly death. Maybe you can even sell it to them, who knows?
All I know is while CL's response is perfectly rational, I can't help but be pissed off that I'm locked into a shitty product that is only surviving because of head start, critical mass, network effects, etc.
Edit: An alternate approach (that would admittedly not help Eric) is to open source padmapper and let people do local installs for themselves. Would be pretty hard to ban that. Although I'm not sure on the legality of scraping.
It sucks that this waas mainly an interface to craigslist, and now they're gone. I hope this gets enough critical mass that eventually so that the unusable but unexplicably popular horror that is craigslist apartments just dies an ugly death. Maybe you can even sell it to them, who knows?
All I know is while CL's response is perfectly rational, I can't help but be pissed off that I'm locked into a shitty product that is only surviving because of head start, critical mass, network effects, etc.
Edit: An alternate approach (that would admittedly not help Eric) is to open source padmapper and let people do local installs for themselves. Would be pretty hard to ban that. Although I'm not sure on the legality of scraping.
Craigslist is in major need of disruption.
Actually apartment finding in general sucks I can't think of a single goto place to look for apartments other then Craigslist and that just proves point #1!
Actually apartment finding in general sucks I can't think of a single goto place to look for apartments other then Craigslist and that just proves point #1!
I guess a lot depends on where you live - Craigslist in the Bay area is like magic for finding apartments, or a room to rent.
The Bay Area has a very advanced craigslist option, where you can sort not only by geographic area (Peninsula), you can even zoom down to a particular city. Add filtering by price and find a place to live in the Bay area has always (Since 2003) been painless for me - particularly as I don't drive, and so getting a place that is _precisely_ in the right place is important. Housingmaps provides a free map interface for those who find such a thing useful.
I don't actually know how anyone would _improve_ craigslist (for me) - I am not sure what the disruption would be - as the free service provides me 100% of what I would want from such a system.
The Bay Area has a very advanced craigslist option, where you can sort not only by geographic area (Peninsula), you can even zoom down to a particular city. Add filtering by price and find a place to live in the Bay area has always (Since 2003) been painless for me - particularly as I don't drive, and so getting a place that is _precisely_ in the right place is important. Housingmaps provides a free map interface for those who find such a thing useful.
I don't actually know how anyone would _improve_ craigslist (for me) - I am not sure what the disruption would be - as the free service provides me 100% of what I would want from such a system.
Yeah but that still sucks, because people rarely know which neighborhood they are in. With padmapper, I could see exactly what was where and not have to worry about somebody thinking excelsior is somehow the mission, and the price per room/cats and dogs/walk score/etc... extras are amazing and integrated, especially if you don't know an area.
I guess you haven't really used padmapper if you don't understand how anyone could improve craigslist for you.
I guess you haven't really used padmapper if you don't understand how anyone could improve craigslist for you.
>> Yeah but that still sucks, because people rarely know which neighborhood they are in.
"rarely" is a tough word. I have found the city to be pretty well demarcated unless you are talking about people confused about minor issues such as what counts as Lower Haight and what counts as Upper Haight.
>> I guess you haven't really used padmapper if you don't understand how anyone could improve craigslist for you.
I used Padmapper as a newbie to the city but quickly found that the latency between postings on CL and the corresponding crawled updates on Padmapper were so high that the postings would have already been hit by a few dozen responses by the time I got it and in a hyper-competitive housing market like SF, this made all the difference.
"rarely" is a tough word. I have found the city to be pretty well demarcated unless you are talking about people confused about minor issues such as what counts as Lower Haight and what counts as Upper Haight.
>> I guess you haven't really used padmapper if you don't understand how anyone could improve craigslist for you.
I used Padmapper as a newbie to the city but quickly found that the latency between postings on CL and the corresponding crawled updates on Padmapper were so high that the postings would have already been hit by a few dozen responses by the time I got it and in a hyper-competitive housing market like SF, this made all the difference.
If you're looking on the Peninsula, where school districts and crime levels vary by street within the same time, Padmapper is vastly better than craigslist.
I mean, look at all the misleading fucking tools who post "Palo Alto apartment for $1k/mo" which is actually in EAST PALO ALTO. With padmapper, it is clear. (In Menlo Park, where some areas are unincorporated, some are Menlo Park, some are Menlo Park but a warzone (Belle Haven, aka EPA North), etc., it's even more essential).
I mean, look at all the misleading fucking tools who post "Palo Alto apartment for $1k/mo" which is actually in EAST PALO ALTO. With padmapper, it is clear. (In Menlo Park, where some areas are unincorporated, some are Menlo Park, some are Menlo Park but a warzone (Belle Haven, aka EPA North), etc., it's even more essential).
In Boston, every listing is contaminated with mentions of every single nearby neighborhood to take advantage of Craigslist's comically broken search functionality. Search "back bay", and you end up with listings in Somerville, since it's "NEAR BACK BAY SOUTH END CAMBRIDGE BROOKLINE" etc etc etc.
Apartment hunting on Craigslist is a total nightmare without Padmapper, and I'm resenting Craigslist more and more for relegating us all to using their horrible UI.
Apartment hunting on Craigslist is a total nightmare without Padmapper, and I'm resenting Craigslist more and more for relegating us all to using their horrible UI.
It's hell here in Austin. The listings are so polluted with realtors, you can't find anything that's not a giant complex.
> Craigslist is in major need of disruption.
This is about the 3000th time I've read that phrase, or some variation of it, in the past ten years. If that were true then it would have happened already, no? Markets can fail in the short term, but in space as crowded with upstarts as the web, I just don't buy it over such a long period of time.
The thing I like most about Craigslist is that some guy made it just to help other people out. The thing I dislike most about all the people trying to "disrupt" Craigslist is that they're clearly in it for the money. I feel like this non-trivial selling point is lost on those who swear that CL is broken and needs replacing.
This is about the 3000th time I've read that phrase, or some variation of it, in the past ten years. If that were true then it would have happened already, no? Markets can fail in the short term, but in space as crowded with upstarts as the web, I just don't buy it over such a long period of time.
The thing I like most about Craigslist is that some guy made it just to help other people out. The thing I dislike most about all the people trying to "disrupt" Craigslist is that they're clearly in it for the money. I feel like this non-trivial selling point is lost on those who swear that CL is broken and needs replacing.
They're monopolists with a shitty product. Craigslist is the IE6 of classified ads.
Yeah, I really don't think Craigslist needs disruption, it's mostly perfect for what it does. It's just for really complex/rare/non-commodity things like housing and cars where I think it needs some help.
I've always recommended padmapper to friends moving to NYC, it has never let me down. We'll see how it goes from here, hopefully people will just start submitting directly on their site.
> Craigslist is in major need of disruption.
I rarely hear anyone mention Craigslist in Canada. We have Kijiji [1] but its UI isn't significantly different than that of Craigslist. The home page is essentially just another long list of categories, etc. Maybe there's something about that design that works exceptionally well for this type of site.
[1] http://www.kijiji.ca
I rarely hear anyone mention Craigslist in Canada. We have Kijiji [1] but its UI isn't significantly different than that of Craigslist. The home page is essentially just another long list of categories, etc. Maybe there's something about that design that works exceptionally well for this type of site.
[1] http://www.kijiji.ca
apartment hunting doesn't suck because you have to use craigslist. apartment hunting sucks because you have to deal with real estate agents.
I expected padmapper to be shut down a long time ago. Mainly because they are an incredible threat to the websites they are scraping all of their data from.
That's probably why padmapper has no way of making money at the moment.
They were hoping they could slink by, stealing other's content, until they were eventually big enough for people to warrant visiting them to list their apartments for rent. And then sell ads.
Don't get me wrong, I love padmapper, but scraping content is scraping content. And it's generally not allowed.
That's probably why padmapper has no way of making money at the moment.
They were hoping they could slink by, stealing other's content, until they were eventually big enough for people to warrant visiting them to list their apartments for rent. And then sell ads.
Don't get me wrong, I love padmapper, but scraping content is scraping content. And it's generally not allowed.
You're right, I was hoping I could go on, but I wasn't stealing anyone's content, I was posting links pasted geographically located with stat summaries. Look at a listing on PadMapper - there's no listing text, and when you click through, it goes to the original listing.
My apologies. I may have used some improper wording there.
But if I may... you must admit that you have taken "something" from Craigslist.
Whatever you want to call it... be it geographical location points, links-to-listings, content, data, images, etc... it is inevitably some form of information that was originally curated by Craigslist through their 10+ years of existence online and is thus available to be used (and not used) at their discretion.
Canada Post for example charges $5,500/year (http://www.canadapost.ca/cpo/mc/business/productsservices/ma...) just to have access to a postal code verification system. It's all publicly available information... if you walked to every city, suburb, nook and cranny in Canada and asked the local people what their postal code is, they would tell you without hesitating.
But if Canada Post gets wind that you may have copied any of that publicly available information from their databases instead of doing the hard work yourself, you can expect a lawsuit at your door fairly quickly: http://www.cbc.ca/news/business/story/2012/04/13/technology-...
I don't think Canada Post had a case in the above example because the company in question didn't actually copy anything from their database, but you did when scraping Craigslist (for however insignificant you found the information to be), see what I'm saying?
Just because you are sending them traffic, doesn't give you the right to take (insert whatever might have been taken here) from them without their permission.
That being said, I love padmapper and have used it numerous times already for myself and friends. I wish you all the best going into the future!
But if I may... you must admit that you have taken "something" from Craigslist.
Whatever you want to call it... be it geographical location points, links-to-listings, content, data, images, etc... it is inevitably some form of information that was originally curated by Craigslist through their 10+ years of existence online and is thus available to be used (and not used) at their discretion.
Canada Post for example charges $5,500/year (http://www.canadapost.ca/cpo/mc/business/productsservices/ma...) just to have access to a postal code verification system. It's all publicly available information... if you walked to every city, suburb, nook and cranny in Canada and asked the local people what their postal code is, they would tell you without hesitating.
But if Canada Post gets wind that you may have copied any of that publicly available information from their databases instead of doing the hard work yourself, you can expect a lawsuit at your door fairly quickly: http://www.cbc.ca/news/business/story/2012/04/13/technology-...
I don't think Canada Post had a case in the above example because the company in question didn't actually copy anything from their database, but you did when scraping Craigslist (for however insignificant you found the information to be), see what I'm saying?
Just because you are sending them traffic, doesn't give you the right to take (insert whatever might have been taken here) from them without their permission.
That being said, I love padmapper and have used it numerous times already for myself and friends. I wish you all the best going into the future!
Thanks for the explanation! I've talked with lawyers about this in the past week, and facts, totally rewritten summaries of fact, and collections of facts aren't copyrightable in the US, so it's not really an issue of their content, as far as I understand. They do have a right not to be crawled by me, though, especially if it causes a burden on them. I wrote that crawler specifically to be very light on their servers including making tradeoffs to usability (it never recrawls listings), but that's still their decision to make. If there are ways to get the data that don't involve touching their servers, then the TOU no longer applies, if I understand the law, but it would obviously make them angry, and so I'm most likely not going to do that.
I agree that this isn't a copyright issue. If you're operating in California, the 9th Circuit is probably your best bet for holding that your scraping of their data doesn't damage or impair their computer system. But given your other comments, it is unlikely you want to raise the funds or spend the time to battle it out in court.
But there is an another way - let us (all of us) get the Craigslist data for you. You could create a browser plugin that would save craigslist posts a user views, and then upload only those posts to a central database that feeds padmapper. With just 1,000 users you would get most but not all the data you currently are scraping.
This is the same approach used by Recap (https://www.recapthelaw.org/) as a response to the federal government's paywall to access public court filings.
But there is an another way - let us (all of us) get the Craigslist data for you. You could create a browser plugin that would save craigslist posts a user views, and then upload only those posts to a central database that feeds padmapper. With just 1,000 users you would get most but not all the data you currently are scraping.
This is the same approach used by Recap (https://www.recapthelaw.org/) as a response to the federal government's paywall to access public court filings.
Thanks for the advice! But yeah, court sounds like a pretty big clusterfuck with a potentially bigger clusterfuck at the end of the rainbow. I have a feeling that the plugin would get me sued as well, though I suppose that would be a more winnable battle.
What about copyright traps?
Say Craigslist created a fake listing or two for say a 2 bedroom apartment in the middle of an industrial complex? (ie, an apartment that doesn't exist, but was placed there by Craiglist to serve as evidence against your copying).
If the Craigslist lawyers find that fake listing on PadMapper, could they do anything to charge you with regards to a copyright violation?
I'm not trying to drag this on. I'm only trying to help.
Here is a smoking gun article on Rand McNally's copyright traps (or lack thereof): http://www.straightdope.com/columns/read/1058/do-maps-have-c...
Say Craigslist created a fake listing or two for say a 2 bedroom apartment in the middle of an industrial complex? (ie, an apartment that doesn't exist, but was placed there by Craiglist to serve as evidence against your copying).
If the Craigslist lawyers find that fake listing on PadMapper, could they do anything to charge you with regards to a copyright violation?
I'm not trying to drag this on. I'm only trying to help.
Here is a smoking gun article on Rand McNally's copyright traps (or lack thereof): http://www.straightdope.com/columns/read/1058/do-maps-have-c...
Right, but there they were wholly copying what is essentially a drawn representation of geography, which is an original work. If you made a rough sketch of that map, it wouldn't be copyright infringement.
"stealing other's content" that was completely user generated...
So is MLS data in a sense, but good luck copying that data and not getting a letter from their lawyers the very next day.
Real estate information is extremely valuable. The massive amount of trust that keeps generating a fresh database of apartments to rent on craigslist is quite valuable to them.
Why should they just allow anybody else to copy it for free?
Real estate information is extremely valuable. The massive amount of trust that keeps generating a fresh database of apartments to rent on craigslist is quite valuable to them.
Why should they just allow anybody else to copy it for free?
Craigslist should buy you, not block you.
I'm basically a Luddite and yours is the one service I always recommend.
I'm basically a Luddite and yours is the one service I always recommend.
Maybe they should block you then buy you. Probably would help them in the negotiation...
But in all seriousness, PadMapper was a great service and seemed to do it the "right" way (providing deep links back into Craigslist's site/content), if there is a "right way" to unofficially gather data without use of a public API.
But in all seriousness, PadMapper was a great service and seemed to do it the "right" way (providing deep links back into Craigslist's site/content), if there is a "right way" to unofficially gather data without use of a public API.
PadMapper is like the super helpful feature that craigslist did not figure out how to build correctly, or never bothered to build. I'm disappointed that Craigslist chose to respond by making a great social utility disappear from the market and anger the users. What about charging a license fee? What about acquiring PadMapper? What exactly does Craigslist gain by pulling its index off from PadMapper other than damaging its own public image?
Craigslist is an evil monopolist that literally wastes millions of hours of its users time every year through the enforcement of shitty UI (if the amount of money that I save using padmapper is typical). They get a good reputation for some bizarre reason because their CEO is an anti-capitalist, but they are still evil for holding back innovation.
A human lifetime is about 700,000 hours. Craigslist is the moral equivalent of a serial killer.
A human lifetime is about 700,000 hours. Craigslist is the moral equivalent of a serial killer.
For those who use Chrome web browser, there's some fixes by way of extension that really help save time on CL... My favorite: http://www.craigslistextension.com/
This is the whole reason I started foreverlist.com. Besides better/more photos, geolocation, comments, etc..., I always wanted to open up the api and let anyone develop on top of it. Alas, competing with craigslist is folly. I regret cloning the design of CL, but the logic was that it would be simple for people to use.
I absolutely LOVE pad mapper and hate craigslist even more now. Many good sites have been killed off by craigslist.
[edit spelling]
I absolutely LOVE pad mapper and hate craigslist even more now. Many good sites have been killed off by craigslist.
[edit spelling]
So should they ban Google? Because I use google to search for things on craigslist with much more success than craigslist's system, in the same way I used Padmapper to find a room/house (3 times in the past year... I move a lot)
This is stupid.
This is stupid.
Carsabi (YC W12) was also affected by this recently. Especially annoying midway through my car search. It is really disappointing that Craiglist won't license their data.
I'm kind of surprised that a company based around scraping CL would get into YC. I know a lot of people do it, but it wasn't exactly unknown that it was against craigslist's TOS.
I would say this pretty much killed Carsabi.
Actually not. Enough of their content is direct from dealers that it looks like they will be able to bounce back.
Yeah, they still have quite a lot of content, and to be honest craigslist listings were some of the worst on there. Still, it was handy to just have one go to place for car shopping.
Kudos to CL for standing up for user privacy. When I use CL it's with consideration of their TOS which bans these practices.
When padmapper copies the data they violate my privacy, my copyright, and the agreement between myself and craigslist as outlined in the TOS. What padmapper is doing is a violation of many data privacy laws like PIPEDA, EUDPD, and various copyright acts.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
Would the padmapper team be ok with me deciding to stick their logo where ever I deem it necessary? I bet they'd probably sue me for copyright and trademark violation.
When padmapper copies the data they violate my privacy, my copyright, and the agreement between myself and craigslist as outlined in the TOS. What padmapper is doing is a violation of many data privacy laws like PIPEDA, EUDPD, and various copyright acts.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
Would the padmapper team be ok with me deciding to stick their logo where ever I deem it necessary? I bet they'd probably sue me for copyright and trademark violation.
This isn't making a permanent record of casual encounters.
It's putting apartment listings on a map. People listing apartments tend to want it to be widely known.
It's putting apartment listings on a map. People listing apartments tend to want it to be widely known.
So you're upset that a listing that you created for the sole reason of being seen and generating a response is being made more public?
wat??
wat??
When you post on Craigslist, you're posting on a public forum. What expectation of privacy could you possibly have?
The expectation outlined in the TOS, as well as the privacy guidelines set out in PIPEDA.
I fail to see how PIPEDA is involved. The information being published on PadMapper is information which you've chosen to publicly disclose. The fact that this publicly disclosed information is showing up on a site you didn't intend might make a case for tort law or copyright law, but there is no disclosure of information that you expected to remain private.
[deleted](1)
Kudos to CL for standing up for user privacy.
While I appreciate the sentiment, it is very unlikely that a landlord would object to an apartment listing being aggregated so as to expose it to more users.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
How do you propose padmapper do that without violating the Craigslist TOS in some other way like bulk-emailing everyone who lists an apartment?
I'm not saying their conduct wasn't a bit of a grey area, against the TOS and maybe a copyright violation, but I don't think many Craigslist users would object personally to their posts being reposted in this manner.
While I appreciate the sentiment, it is very unlikely that a landlord would object to an apartment listing being aggregated so as to expose it to more users.
If padmapper wants the data why don't they just obtain consent from the CL posters, instead of copying it without consent.
How do you propose padmapper do that without violating the Craigslist TOS in some other way like bulk-emailing everyone who lists an apartment?
I'm not saying their conduct wasn't a bit of a grey area, against the TOS and maybe a copyright violation, but I don't think many Craigslist users would object personally to their posts being reposted in this manner.
[deleted]
Enjoy your 1984.
Here's the email I sent to Craigslist. I tried to use the following template: intro (what are we talking about), how padmapper is good for craigslist, why I care about padmapper, course of action.
Feel free to reuse it.
----
Hi Jim and Craig,
I learned today that Craigslist has sent a Cease and Desist to padmapper.com. Padmapper provides a service to visualize on a map rental listings from sources like Craigslist.
Real estate is all about location, and let's be frank, Craigslist's location-based search sucks. Padmapper helps the searcher easily see where real estate is located. It turns a grueling search into a breeze. It adds great value to Craigslist's listings without competing with Craigslist's business.
Padmapper, along with Craigslist, were essential in helping me find my current home in the San Francisco Bay Area. Without Padmapper, I wouldn't have used Craigslist in my search.
I'm sure you can find a mutually beneficial deal with Padmapper, instead of shutting them out entirely.
Feel free to reuse it.
----
Hi Jim and Craig,
I learned today that Craigslist has sent a Cease and Desist to padmapper.com. Padmapper provides a service to visualize on a map rental listings from sources like Craigslist.
Real estate is all about location, and let's be frank, Craigslist's location-based search sucks. Padmapper helps the searcher easily see where real estate is located. It turns a grueling search into a breeze. It adds great value to Craigslist's listings without competing with Craigslist's business.
Padmapper, along with Craigslist, were essential in helping me find my current home in the San Francisco Bay Area. Without Padmapper, I wouldn't have used Craigslist in my search.
I'm sure you can find a mutually beneficial deal with Padmapper, instead of shutting them out entirely.
Just out of curiosity, why would they not want you to use their classifieds? Unless I am missing something, they don't collect revenue for listings. I thought Craigslist was all about spreading good will. If Pad Mapper provides a more efficient way of spreading that good will, then why not?
As a user of PadMapper, I am very disappointed to see this happen. Craigslist is almost unusable if you don't know the area you are looking to move to.
As a user of PadMapper, I am very disappointed to see this happen. Craigslist is almost unusable if you don't know the area you are looking to move to.
As best I can tell, it's a matter of corporate philosophy. Craigslist is predicated on local, person-to-person interactions. It has long taken pains to curtail uses of the system which allow aggregation of multiple CL local sites (there is, for example, no convenient way to search all of California for listings), or to include CL listings in third-party services.
As a long-time user of Craigslist, and a recent fan of padmapper, I'm disappointed, to say the least.
I do suspect it's time for some disruption in CL's space.
As a long-time user of Craigslist, and a recent fan of padmapper, I'm disappointed, to say the least.
I do suspect it's time for some disruption in CL's space.
I'm surprised how many people here are defending Craigslist. They've got a history of both providing a terrible experience for apartment seekers and shutting down any attempts to use their data to provide a better experience. That would be okay in a competitive market, but they've got a strong monopoly in a market with a high barrier to entry. They're playing the textbook role of an evil monopolist.
Craigslist is a company whose terms of service were violated by PadMapper. Whether or not PadMapper provides a better experience is irrelevant: Craigslist has a right to defend its content and lists by whatever means it sees fit. Any businessperson on this site would expect to have the same rights.
Craigslist is not in the business of helping anyone but Craigslist be successful. It's not monopoly; it's business.
Craigslist is not in the business of helping anyone but Craigslist be successful. It's not monopoly; it's business.
Companies are not free to do whatever they want. The government compels monopolists to avoid anticompetitive behavior, and even breaks companies up, for examples. The market in this case is a little hard to define, since apartment seekers aren't purchasing the product, so I'm sure it's not actually illegal, but it sure seems to violate the spirit of these laws, which are designed to protect consumers from exactly this kind of behavior.
This is horrible. I used Padmapper extensively when looking for an apartment here in New York two months ago. I guess I'm never moving again...
Padmapper has been a wonderful service. It is confusing why Craigslist is blocking it unless they (Craigslist) are thinking about coming up with similar user interface soon. Craigslist has been hiring UI designers.
Why not buy a license and go hard on mobile? Then use that to get more direct listings, ideally with ACTUAL STRUCTURED DATA unlike the shit that is craigslist (i.e. closer to MLS, or even better, closer to the proprietary systems used inside large real estate investment companies), and then re-release a web version using only those listings.
Maybe focus on SFBA only, since the real estate situation is acutely broken here AND smart+rich people still rent here (unlike Texas or the midwest, with NYC being its own weird market).
Maybe focus on SFBA only, since the real estate situation is acutely broken here AND smart+rich people still rent here (unlike Texas or the midwest, with NYC being its own weird market).
Maybe, if you want to chat more about this, email me at [email protected]
At the end of the day it is Craigslist's data. I know padmapper was useful for a lot of people so the next smart move is to build your own data. A "craiglist" focus on rental only with the same features. The sad reality is you need your own platform to scale any type of services like that.Is it going to be easy and fast to build. The answer is no. However the reward and opportunity are huge....
Of course it would be great if craigslist could have an API or licence his data but it won't happen in a bear future ...
Of course it would be great if craigslist could have an API or licence his data but it won't happen in a bear future ...
Well, it's Craigslist's data, but it's also the advertisers' data. If I were to advertise an apartment I owned, I would rather put it on Padmapper than on Craigslist, if only Padmapper had the critical mass.
But maybe that's what Craigslist is worried about. I'll tell you one thing, though, if Padmapper wasn't competing with Craigslist before today, it certainly is now.
But maybe that's what Craigslist is worried about. I'll tell you one thing, though, if Padmapper wasn't competing with Craigslist before today, it certainly is now.
At the end of the day it is Craigslist's data.
Nope. It's the users' data. Craigslist only has a license to use it.
Nope. It's the users' data. Craigslist only has a license to use it.
In that case Craigslist is doing right by its users, the licencor of the data, since the licensor has not granted a license to padmapper to use said data.
I think its pretty well documented that craigslist is known to shut down services that use craigslist data. I really don't think should come as a surprise to the creator.
Unfortunately, scraping another site's information is generally not a long-term viable business plan (speaking from personal experience). I suppose CraigsList, much like insurance companies, doesn't like its valuable information being displayed on other websites without endorsement or consent.
That said, I love PadMapper, having used it a couple of times for finding apartments in the Bay Area. So it's a bummer that they won't be able to show CL listings anymore.
That said, I love PadMapper, having used it a couple of times for finding apartments in the Bay Area. So it's a bummer that they won't be able to show CL listings anymore.
Good luck, I think it's a sign of your success that this has happened. Providing more value with their data than they do is a threat to their business.
I can't see Craigslist ever stepping up their game and providing meaningful visualization. I can't really see why CL cares about where people are reading the listings - their business model is based on selling the listings, not monetizing the viewing of them. I suspect it's out-of-control lawyers at work.
Craigslist is closer to a non-profit than a business. You can't model them as a self-interested agent.
And like many other non-profits, the quality of the service shows - the service is merely "good enough" (in many categories, far less than good enough), with no competitive pressure to improve.
We have already seen the meteoric rise of many niche, category-specific Craigslist competitors eclipse CL in scale and influence, particularly in real estate. I suspect due to Craiglist's general crappy experience this fragmentation will continue.
And I for one think it's a good thing.
We have already seen the meteoric rise of many niche, category-specific Craigslist competitors eclipse CL in scale and influence, particularly in real estate. I suspect due to Craiglist's general crappy experience this fragmentation will continue.
And I for one think it's a good thing.
Well, the lack of competitive pressure stems more from network effects than being non-profit per se.
Because of this, I don't see the fragmentation as a good thing. One of two things will happen in each craigslist market (rentals, used goods, ridesharing, etc.): either a single for-profit company will ride networks effects and establish a monopoly, or there will be severe fragmentation.
If the former, then you can expect some features to improve (in order to steal the market from craigslist in the first place) but many other things to languish due to a lack of competitive pressure. (Also, if it's a for-profit company, you'll pay money.) This is ebay.
If the latter, then it will no longer be possible to go so one site to search in that market, greatly decreasing consumer utility. This is how classified ads worked before craigslist.
Much better is a well-run, benevolent non-profit. Wikipedia, though not flawless, is pretty much the epitome of this idea. It's scary to think what it would be like if Wikipedia's data was spread over dozens of websites of varying quality and motivation. (Then again, in the scenario no one would bothered to write most of the articles anyway.)
Craigslist doesn't fit this exactly because they impose a philosophy ("localism"?) which most people don't share and which decreases usability.
Because of this, I don't see the fragmentation as a good thing. One of two things will happen in each craigslist market (rentals, used goods, ridesharing, etc.): either a single for-profit company will ride networks effects and establish a monopoly, or there will be severe fragmentation.
If the former, then you can expect some features to improve (in order to steal the market from craigslist in the first place) but many other things to languish due to a lack of competitive pressure. (Also, if it's a for-profit company, you'll pay money.) This is ebay.
If the latter, then it will no longer be possible to go so one site to search in that market, greatly decreasing consumer utility. This is how classified ads worked before craigslist.
Much better is a well-run, benevolent non-profit. Wikipedia, though not flawless, is pretty much the epitome of this idea. It's scary to think what it would be like if Wikipedia's data was spread over dozens of websites of varying quality and motivation. (Then again, in the scenario no one would bothered to write most of the articles anyway.)
Craigslist doesn't fit this exactly because they impose a philosophy ("localism"?) which most people don't share and which decreases usability.
There are, for example, real estate websites that are eating the CL for-rent category's lunch right now. Craigslist isn't doing much in response to these competitors, despite the fact that in some markets (particularly SF and NYC where I have experience) Craigslist is now very firmly in second place (or worse).
I've been apartment hunting in NYC lately, and it's mind-boggling what proportion of Craigslist posts are scams compared to other sites like StreetEasy or NYBits.
IMO if Craigslist was run more like a scrappy startup looking for their big exit, they'd have more than responded to these competitors by now. The network effect exists, but IMO is an insufficient explanation of their lack of action.
I agree that the fragmentation is, all things being equal, a bad thing for users - but considering how utterly useless Craigslist's moderation systems are (many categories are full of duplicate posts, spammers, and scammers), and how poor the UX is otherwise (search? hah!), I welcome it. Fragmented sites like PadMapper, StreetEasy, etc, have done way more to ease the life of renters than Craigslist in the last decade.
I've been apartment hunting in NYC lately, and it's mind-boggling what proportion of Craigslist posts are scams compared to other sites like StreetEasy or NYBits.
IMO if Craigslist was run more like a scrappy startup looking for their big exit, they'd have more than responded to these competitors by now. The network effect exists, but IMO is an insufficient explanation of their lack of action.
I agree that the fragmentation is, all things being equal, a bad thing for users - but considering how utterly useless Craigslist's moderation systems are (many categories are full of duplicate posts, spammers, and scammers), and how poor the UX is otherwise (search? hah!), I welcome it. Fragmented sites like PadMapper, StreetEasy, etc, have done way more to ease the life of renters than Craigslist in the last decade.
If anyone gets too much traction they can cut out the middleman.
This is a damn shame. I used padmapper to find my last two places. CL should've been doing this feature on their own, navigating rental listings on their site is a nightmare. The map feature was a godsend.
Curious if Live Lovely (http://livelovely.com/) will get a similar cease and desist. They do essentially the exact same thing, but with a nicer design.
Except Padmapper works everywhere craigslist does. I went to livelovely just now and they don't have my city. It's not exactly a small town. Also their filtering is nothing compared to padmapper.
On a more related note, I would imagine any sites like this that come up on Craigslist's radar will get a similar notice.
On a more related note, I would imagine any sites like this that come up on Craigslist's radar will get a similar notice.
Disappointing. I'm currently living in an apartment I found via Padmapper and Craigslist. Padmapper was absolutely crucial, without it I wouldn't have found this place.
I understand why they're complying with the C+D, but I wish they'd fight it. Craigslist is better than printed classifieds, but that's about the best thing you can say about it.
I understand why they're complying with the C+D, but I wish they'd fight it. Craigslist is better than printed classifieds, but that's about the best thing you can say about it.
I think it's harder for small guys like Padmapper to fight them, because 1) There's no money for lawyering 2) Craigslist says that they listen more to users for UI.
So if you'd like to see CL give Padmapper license to use its data, send them an email. Your voice as a CL user weight more than Padmapper's would.
So if you'd like to see CL give Padmapper license to use its data, send them an email. Your voice as a CL user weight more than Padmapper's would.
Craigslist does not allow scraping. Beg for help and for support for breaking their rules elsewhere. This is Hacker News, not "help me retain a business model after I broke rules" central.
You could alternatively hire people to spam them with nice requests to reconsider.
Just sent a mail to {craig,jim}@craigslist.org. Not sure if it will help but if they hear enough noise about it, maybe they'll change their stance.
It's most likely an indication that they'll be developing their own UI.
Could Padmapper reform itself as an extremely intrusive browser extension?
Is there any evidence CL was pressured by any privacy groups or regulators?
Is there any evidence CL was pressured by any privacy groups or regulators?
The pro CL comments here are ridiculous and so anti-innovation. "Don't crawl my data bro!". PM wasn't modifying the data in any way, just redis playing it on a damn map. Now CL is useless, yet again, for housing.
I wonder why CL doesn't create an API and charge for it's usage. The data is clearly valuable and they own it, but that doesn't mean that it can't use multiple views.
This is only useful if the teams scraping CL have budget to pay a market rate for the data. Considering that CL seems to have a lock on this data for many markets, I'm guessing that's worth a lot of money (which startups will not be able to afford).
If they cannot afford, then they should not be there. Scraping illegally is not a long term solution. Craigslist should conduct some market research from their end to charge a "reasonable" fee which fills in their coffers as well as allow development an ecosystem around their data. Even DuckDuckGo pays for Yahoo! BOSS Search API to show their results. Not sure why CL hasn't thought about it.
RSS feeds are almost an API.
I'm pretty sure that Craigslist disruption is coming, and it's going to be AirBnB.
I'm not too worried. What craigslist is doing (ie, sucking) isn't sustainable.
I'm not too worried. What craigslist is doing (ie, sucking) isn't sustainable.
Hah, people have been saying that for a decade. What people fail to realize is that Craigslist is ultra successful because of what it does, not despite.
Why the hate on Craigslist? It's their information, and are free to do what they want with it - most likely, develop their own platform.
Quite a misleading title..I think many clicked thinking to read about an unexpected shutdown of Craigslist actually
Sorry, I didn't mean it that way - if you know what PadMapper is and see the URL, I think it makes sense.
With the OP here, I saw the thread title and silently went "thank god" to myself immediately knowing what this meant, having used PadMapper very extensively in the last year after moving to Austin. I knew it meant a good portion of ads being gone, but I was going to get much more qualitative results in finding a condo.
I really hope PadMapper can make up the volume but continue to have good results for people searching. It's a wonderful resource.
I really hope PadMapper can make up the volume but continue to have good results for people searching. It's a wonderful resource.
Haha you could always filter out Craigslist with the sources filter (though it's a bit hidden)
Some people don't know what PadMapper is. Like me for example.
[deleted]
I propose that Craigslist has been ultra successful because of its actions, not despite. I think it's success is due both directly to how it has built and operated the site as well as due to the overall mentality that fends off all the lame ideas from the armchair product designers.
"They allow mobile apps to display their listings if you buy a license from them, but not websites."
Really? They used to allow any website to use their data for free as long as they didn't run ads against it or monetize it in any way. Kexter.com has been doing exactly what you're doing for almost ten years now.
Really? They used to allow any website to use their data for free as long as they didn't run ads against it or monetize it in any way. Kexter.com has been doing exactly what you're doing for almost ten years now.
Padmapper's a brilliant solution. I'd like to think that Craigslist is building their own great solution, but knowing how slowly CL moves on everything, I sincerely doubt they'd build a worthwhile replacement. Bummer!
Padmapper's helped me and a few friends find our apartments in SF.
Padmapper's helped me and a few friends find our apartments in SF.
Could it not mean that they are coming up with some padmapper like functionality? Padmapper of course is the only usable rental search site after craigslist. The rest of them like rent.com and apartments.com are absolute crap. I hope they work something out.
How is this different from when Bing was using Google search results? If people want to use Padmapper great. if people want to use Craigslist then great.. Am I missing something here? I don't see the problem. What if it were the other way around?
This sucks. Craigslist is not setup to handle apartment searches at all. I literally just finished finding an apartment and I solely used padmapper. I can't imagine having to go through the process just using craiglist search. Bummer.
I personally would have tried to find a way to work with them. But it's not my site.
Craig isn't doing anything wrong or unethical here. Quite the opposite; Craigslist continues to deliver exactly what its users signed up for.
Craig isn't doing anything wrong or unethical here. Quite the opposite; Craigslist continues to deliver exactly what its users signed up for.
I'm sad to hear this. I used padmapper to find my place in NYC among an incredibly scam-filled and otherwise difficult to navigate sea of lies and frauds. PM helped me utilize CL in a way their filters could not.
I used PadMapper with great success numerous times, I hope it can survive this.
Once read about how its a bad idea to base the core of your business around someone else's platform. Once they(and their lawyers) decide to re-evaluate your relationship with them it can nearly destroy what you've built. Testament to this are the number of apps based on twitter that died when twitter decided to up their offering and cover what they were doing. How to do it well is Zynga's relationship with Facebook
The people at PadMapper are doing a good thing looking for other sources but like the author said Craiglist was/is an important source of pad listings.
All the best to them
The people at PadMapper are doing a good thing looking for other sources but like the author said Craiglist was/is an important source of pad listings.
All the best to them
I decided to consider moving to the US on a whim about 6 months back (in the end I didn't) and decided to try and find an apartment and guage the cost of living. People recommended craigslist as apparently the US doesn't have a rightmove (http://www.rightmove.co.uk/) type site. Craigslist was probably the worst experience I've had trying to find an apartment... I can't believe that people actually use it, why wouldn't they build a padmapper esque system (or acquire padmapper)?
Craigslist is, unfortunately, the "best" example of the network problem. Due to its popularity, it is entrenched.
That said, there are methods of disruption. AirBnB bootstrapped off Craigslist to get their initial dataset, and have since moved to their own content. That's probably the strongest approach if you're lucky enough to be able to move quickly when you need to.
That said, there are methods of disruption. AirBnB bootstrapped off Craigslist to get their initial dataset, and have since moved to their own content. That's probably the strongest approach if you're lucky enough to be able to move quickly when you need to.
The thing is, I think Craigslist's network effect is exaggerated. Its network is very clustered because nearly all of it (particularly housing) is focused on people in a particular area. Compare to Facebook, which has a large degree of inter-connectedness between all clusters.
If there were a site that had, say, all the rentals available in the SF Bay Area, it would be highly useful to you, as it would have gobbled up most of the benefits of the network effect available (of course, outsiders moving in wouldn't automatically know its name like they do for Craigslist, but that's relatively minor).
Once you've reduced your problem to "develop a really strong network in a very localized geographic area," it seems to me like the solution would become in reach, with a lot of hard work. You could even scrape pretty heavily early on, or enter it by hand if somehow that got around the ToS.
Maybe start at Berkeley or similar college town, maybe reducing yourself to using physical paper and highly targeted online advertising, and rely on students heavily to get the word out. Make it ridiculously easy for someone to post an ad on your site and have it automatically propagate to CL. Gradually expand to different parts of the Bay Area, and then pick out other localities to target. NYC, maybe, especially if you can somehow make its real estate market less dysfunctional.
If there were a site that had, say, all the rentals available in the SF Bay Area, it would be highly useful to you, as it would have gobbled up most of the benefits of the network effect available (of course, outsiders moving in wouldn't automatically know its name like they do for Craigslist, but that's relatively minor).
Once you've reduced your problem to "develop a really strong network in a very localized geographic area," it seems to me like the solution would become in reach, with a lot of hard work. You could even scrape pretty heavily early on, or enter it by hand if somehow that got around the ToS.
Maybe start at Berkeley or similar college town, maybe reducing yourself to using physical paper and highly targeted online advertising, and rely on students heavily to get the word out. Make it ridiculously easy for someone to post an ad on your site and have it automatically propagate to CL. Gradually expand to different parts of the Bay Area, and then pick out other localities to target. NYC, maybe, especially if you can somehow make its real estate market less dysfunctional.
Because it works just fine as it is, as far as Craigslist is concerned. It's not a mindset the HN crowd is used to, but Craigslist is an absolute behemoth, and they're doing just great. They also do a hell of a lot more than just accommodation, so might not be that incentivised to buy Padmapper.
And it's that sort of logic that sees a business/market disrupted.
One might think that Craigslist, of all companies, would understand that.
Innovation beats litigation in most long-run scenarios, IMHO.
One might think that Craigslist, of all companies, would understand that.
Innovation beats litigation in most long-run scenarios, IMHO.
Because Craigslist is a monopolist with no real competitive pressures. Their CEO and founder are also crunchy anti-capitalists who take pride in being luddites. This endears them to much of the left-wing, San Francisco tech crowd, but their shitty UI combined with monopoly power wastes dozens of human lives worth of user hours every year.
Craigslist needs to be stopped. This is the third incredibly useful service I've seen shutdown in as many weeks. I was heartbroken when Carsabi had to stop indexing Craigslist a couple weeks ago.
Could the C&D be PadMapper violating the use of the trade mark?
http://www.padlister.com/listings/new
"Better looking, effortless Craigslist Ads."
http://www.padlister.com/listings/new
"Better looking, effortless Craigslist Ads."
Was there some attempt made at monetizing Padmapper? IIRC Craigslist isn't super hard core about not-for-profit projects using their data, but takes a hard line on commercial use.
Let me guess your young and your pathetic education taught you very little, and your sheltered life made you think everything would be handed you on a silver platter.
Instead of whining, you either need to offer large sums of cash to CL, or some deal. CL does not owe you a living. Your crappy little app is just skimming the cream off the top of CL's infrastructure and you expect CL to give you a free ride? Not going to happen. You are simply lucky you got away with it as long as you did.
Instead of whining, you either need to offer large sums of cash to CL, or some deal. CL does not owe you a living. Your crappy little app is just skimming the cream off the top of CL's infrastructure and you expect CL to give you a free ride? Not going to happen. You are simply lucky you got away with it as long as you did.
FWIW Craigslist is a ghost town in Australia and virtually useless for buying & selling the type of stuff I used it for frequently when I lived in California. For example here's the furniture for sale section for Melbourne: http://melbourne.craigslist.com.au/fua/. Seems that in Australia people use Gumtree.com.au or even ebay.com.au for online classifieds.
gumtree is where it's at for apartment rentals and other big hard-to-ship odds and ends. then there's carsales.com.au for automobiles, and ebay for everything else. seems to work out okay.
Really sorry to hear this. I've successfully found 3 different apartments in perfect locations through PadMapper (and one that I'm moving into in 2 weeks actually). And I've had several friends use it too.
Great job with all the time you've put into the site, Eric - hope the service pulls through this somehow, but also looking fwd to seeing what you work on next if you decide to move on.
Great job with all the time you've put into the site, Eric - hope the service pulls through this somehow, but also looking fwd to seeing what you work on next if you decide to move on.
Solution/Question:
Could the padmapper guy just make an application, which he sells for $10-$15, with the same functionality, except all the "scraping" happens from the user's computer or browser?
And if this violates the TOS, assuming I can sell a browser which only visits Craigslist -- then where is the line where a browser does too much "formatting" and becomes a TOS violation?
Could the padmapper guy just make an application, which he sells for $10-$15, with the same functionality, except all the "scraping" happens from the user's computer or browser?
And if this violates the TOS, assuming I can sell a browser which only visits Craigslist -- then where is the line where a browser does too much "formatting" and becomes a TOS violation?
Sometimes people seem to forget that the most frequent users of Craigslist, and the ones that keep it popular, are not complaining about it having bad UX. We are.
I want PadMapper to succeed (it helped me find a place super-quick for grad school in the fall), but Craigslist has every right to legislate their content. We'll see what they do/don't do about it.
I want PadMapper to succeed (it helped me find a place super-quick for grad school in the fall), but Craigslist has every right to legislate their content. We'll see what they do/don't do about it.
If I were at Craigslist, I would strongly look at commercializing their listings. By that I mean charging a fixed amount (say $5) for every 1000 results sent to the requester (such as Padmapper). That ways Craigslist gets to keep the revenue while other companies can innovate on top of the massive data which Craigslist has.
This post could use a new title, unless you want to scare HN'ers into thinking Craigslist is shutting down.
CL is obviously in their full rights to do so, but I really wish they could have waited a couple weeks before they did it. I'm moving to another state this Tuesday and PM is especially helpful when you don't know the neighborhoods.
as someone typing this in the apartment they found via a Craigslist ad on PadMapper (and as someone who has recommended it to several people because of the feature). time for a mobile PadMapper with a proper license for the data? :/
Craigslist is hiring Senior UI / Usability / Front End Engineers - maybe they plan on releasing a similar feature?
http://sfbay.craigslist.org/sfc/sof/2841395914.html
http://sfbay.craigslist.org/sfc/sof/2841395914.html
I would make the same argument that many are making with Cragislist about IRC, the use of multi-digit telephone numbers and a huge variety of other entrenched technologies that work reliably for enormous numbers of people.
The entrenched technologies are clearly excellent in the context of what they do. Can additions be made which allow one to comprehend data faster, summarize and/or visualize groups of data, filter out garbage and/or present things in a simpler way? Of course.
The agreement-violating leeching activity of some can be easily be forgiven because is not happening to your data, your servers, your love and your life's mission.
In my mind, remoras like this are exposing themselves to additional risk by doing something that is in violation of TOU and potentially a violation of various laws.
There are so many fitting cliches here that it's making my head spin.
The entrenched technologies are clearly excellent in the context of what they do. Can additions be made which allow one to comprehend data faster, summarize and/or visualize groups of data, filter out garbage and/or present things in a simpler way? Of course.
The agreement-violating leeching activity of some can be easily be forgiven because is not happening to your data, your servers, your love and your life's mission.
In my mind, remoras like this are exposing themselves to additional risk by doing something that is in violation of TOU and potentially a violation of various laws.
There are so many fitting cliches here that it's making my head spin.
Or maybe those that are "forgiving" find that this "leeching" is creating a positive benefit for society as a whole, and that ultimately, IP laws are there to benefit the public.
Craigslist is one of the greatest blown opportunities in the history of business.
Could Craiglist shut down every single IP who scrapes? Could you build Padmapper to be less 'realtime' and more 'tell me the city and I'll set up a client-side scraper, come back in 15 minutes?'
I found this : http://m.techcrunch.com/2012/06/22/padmapper-craigslist-ceas...
So no it was not bye bye.
So no it was not bye bye.
[deleted]
>> They allow mobile apps to display their listings if you buy a license from them, but not websites.
PadMapper already has an app so couldn't they continue to use the listings in the app?
PadMapper already has an app so couldn't they continue to use the listings in the app?
Bummer. I'm completely mystified that CraigsList doesn't have an API. Fortunately, others do: http://3taps.com/ [no affiliation]
Can you open source padmapper so we can run it ourselves? That way, no matter what Craigslist does, people would still be able to use the great service.
Hm, that's kind of an interesting idea. I'd be pretty embarrassed at all the hacks I've added over the years, but I'll definitely keep that in mind.
Yes. Please open source it...
hah! I have the same feeling with my stuff.
I don't think I would have ever found a place to stay in SF after moving here 4 months ago if it wasn't for CL's data and PM's ease of use.
This is an existential threat to Padmapper. Which is very sad to me, since it's easily one of my favorite sites to search for anything.
This really sucks. Finding an apartment in Boston on Craigslist is nearly impossible without PadMapper.
Here's an idea-- why doesn't Craigslist just buy padmapper? They certainly have enough cash to do so.
Nooooo!
I was going to start looking at places in Padmapper next month, for my upcoming move!
Is there any other similar service?
I was going to start looking at places in Padmapper next month, for my upcoming move!
Is there any other similar service?
How about all the ipad apps that access craigslist data? how do they get away with that???
Counting down for a new wave of PadMapper clones / CL scrapers to start popping up.
oh sad to hear this. padmapper service with craigslist really helped me!
Can you please open source all the code specific to craigslist data extraction so that others may incorporate your work into open-source projects?
Take them to court. I will donate to your legal fund.
On what grounds, exactly? They have every right to deny you access to theirs servers.
They have every right to implement technical barriers to scraping. Whether they can invoke the law to establish a legal barrier to re-publishing their proprietary-but-public facts -- not even the apartment summary text, just extracted metadata -- is not at all clear. When an auction aggregator was sued by eBay for doing something similar, the judgment in eBay's favor rested on the fact that the aggregator was posing a real undue technical burden, not the inherent fact that they were scraping.[1] Since I assume Padmapper does intelligent scraping, they might have a fairly good case.
[1]: http://en.wikipedia.org/wiki/EBay_v._Bidder%27s_Edge
[1]: http://en.wikipedia.org/wiki/EBay_v._Bidder%27s_Edge
I think you may be confusing two issues.
It's almost certainly true that they could republish the data without violating any law. That has nothing to do with whether they have the right to scrape the data from the website.
IE do not confuse their right to republish the data with the right to get it from a particular source in a particular way.
The line of cases you are citing is also about trespass to chattels, which is a pretty bad claim to make about website scraping (for the reasons the california supreme court set out in Intel vs Hamidi). It was mainly used back in the days when courts barely knew what websites were. Trespass to chattels may still get used when you can show actual harm caused by the scraping, but most claims nowadays would be around terms of use violations, which are a much more grey area (some courts have gone for it, some not).
It's almost certainly true that they could republish the data without violating any law. That has nothing to do with whether they have the right to scrape the data from the website.
IE do not confuse their right to republish the data with the right to get it from a particular source in a particular way.
The line of cases you are citing is also about trespass to chattels, which is a pretty bad claim to make about website scraping (for the reasons the california supreme court set out in Intel vs Hamidi). It was mainly used back in the days when courts barely knew what websites were. Trespass to chattels may still get used when you can show actual harm caused by the scraping, but most claims nowadays would be around terms of use violations, which are a much more grey area (some courts have gone for it, some not).
[deleted](1)
Hi Craig and Jim,
I was disappointed to learn you dont want PadMapper to use your listings. Their site and Carsabi are two sites I recently used to find my apartment and car, respectively, only after I had given up on dealing with your firehose and sparse search options. In using both sites I knew full well I was simply using them as a better, more elegant, more feature rich version of CL. Both ultimately referred me right back to your site for specific listings, which was exactly what I wanted them to do.
I dig and respect that you guys are staying true to your mission, running a lean company and keeping it simple. I've read all I can find about both of you and how you've built and how you run CL. You are two of my heroes. But you are really betraying your mission and your credibility with this move. You should let your data be completely open. Let people do whatever they want with it. CL is an institution. You are a backbone of the internet. Just like Facebook and how they share their graph with anybody who wants to use it, as it spreads the FB brand everywhere - you should embrace people wanting to use your data. It will only serve to spread the CL brand further and even more important, it will make the lives of people all over the World better. I thought that was what you stood for. I hope you don't prove me wrong.
Regards,
-Bill
I was disappointed to learn you dont want PadMapper to use your listings. Their site and Carsabi are two sites I recently used to find my apartment and car, respectively, only after I had given up on dealing with your firehose and sparse search options. In using both sites I knew full well I was simply using them as a better, more elegant, more feature rich version of CL. Both ultimately referred me right back to your site for specific listings, which was exactly what I wanted them to do.
I dig and respect that you guys are staying true to your mission, running a lean company and keeping it simple. I've read all I can find about both of you and how you've built and how you run CL. You are two of my heroes. But you are really betraying your mission and your credibility with this move. You should let your data be completely open. Let people do whatever they want with it. CL is an institution. You are a backbone of the internet. Just like Facebook and how they share their graph with anybody who wants to use it, as it spreads the FB brand everywhere - you should embrace people wanting to use your data. It will only serve to spread the CL brand further and even more important, it will make the lives of people all over the World better. I thought that was what you stood for. I hope you don't prove me wrong.
Regards,
-Bill
I get that CL has a fast UI that is very easy to understand for non-technical users. But why can't they somehow work with others who want to improve the experience?
There have been several sites that I have personally used to aggregate searches, eliminate duplicates, filter on different attributes, etc. that all were better for me than CL. (Craiglook, padmapper, etc.)
Every, time, CL flips them off and threatens suit.
FFS, guys, just strike a deal with some of these devs and make a 'tech-friendly' portal for people who want actual search capability from your site.
There have been several sites that I have personally used to aggregate searches, eliminate duplicates, filter on different attributes, etc. that all were better for me than CL. (Craiglook, padmapper, etc.)
Every, time, CL flips them off and threatens suit.
FFS, guys, just strike a deal with some of these devs and make a 'tech-friendly' portal for people who want actual search capability from your site.
Total BS Craig... You should be REWARDING innovation, not joining the cadre of douchebag trolls that try to kill it.
You already got the traction. Offer something complete now. It's already better.
It would be one thing if Craigslist was doing their best to satisfy their customers and they wanted to shut down others who were scraping their site and leeching off their business. However, Craigslist puts absolutely no effort into making their product usable. The continued existence of Craigslist is just a testimony to the enormous strength of network effect lock in.