Adventures Revisited – Is Realtor.com brain dead or just dead?
As you may have heard, the NWMLS has decided to no longer license the display of its member’s listings on REALTOR.com® or to continue sending those listings on a broker’s behalf, effective June 1, 2007. You can read details about this here. Perhaps this is the beginning of the end for the old standby? The lumbering beast that is Move.com reminds of another lumbering beast that was once in a state of corporate old age. However, I don’t see Louis V. Gerstner, Jr. (IBM’s CEO alumnus) coming to Move’s rescue any time soon.
Anyway, because of this decision, if a broker wants to get their listings on Realtor.com it is now the IDX vendors responsibility to export or upload their client’s listings to Realtor.com. Of course you could use to just Move as your IDX vendor, but given the general lack of innovation they’ve had on their web properties, I wouldn’t want to go that route.
This is significant for many reasons. The first of which is the increased responsibility of IDX vendors now face. Historically, we just had to download MLS data and maintain web sites. Now, we have to do that and create & maintain feeds or mechanisms to other 3rd party consumers of real estate content (Trulia, Propsmart, GoogleBase, etc). Sure there are vendors like vFlyer or Postlets that help with this dilemma, but I suspect many brokers with any significant inventory would rather have their IDX vendor just automate everything.
The second of which, is now Realtor.com has to compete for the mindshare (to paraphrase Microsoft CEO Steve Ballmer) of “IDX Vendors, IDX Vendors, IDX Vendors” and “Brokers, Brokers, Brokers”. If the happenings with NWMLS, become a nationwide movement and Realtor.com no longer has MLS support, then it will have to compete against Trulia, Zillow, and a cast of dozens for your listings.
I’m sure brokers and agents have many interesting things to say about Realtor.com, however as an IDX vendor I really don’t care about the industry politics. I just want to serve my clients in a cost effective manner, make them happy, and make a fair profit.
Unfortunately, whoever designed this specification really has no clue as to how things should be done in the 21st century and didn’t read my Adventures in Digital Listing Land blog post. Sure, if this had happened in say 1992, the approach they took for listings data uploading would’ve been a reasonable approach. And although I liked 1992, I don’t use 486 66’s anymore!
For comparison’s sake, here’s what I had to do to support Trulia (BTW – Oodle and Propsmart were equally easy to support, I’m just using them as an example because they read my blog posts).
- Create an XML document that contains listing data, not unlike an RSS feed (OK, change some XML tags and tweak a few things, whatever)
- Make sure listing photos are accessible via http urls (I already have them on a web site, so this is easy enough to do)
- Tell Trulia what the url is for the feed, watch the listings appear on Trulia with no additional effort on my part
- Blog about the pleasant experience
And here’s what I’ll have in order to provide a feed for realtor.com
- Set up an FTP account with Realtor.com (great, one more username and password to lose)
- Realtor.com requires pipe delimited text for listing data (everybody else uses XML, grrrr)
- Realtor.com requires I upload the listing data to them via ftp (everybody but GoogleBase downloads it via http, yuck)
- Realtor.com requires I upload the images to them via ftp (everybody else downloads them via http, lame)
- Realtor.com requires I package said images up in a zip file (Nobody else has this requirement, Lame)
- Realtor.com requires I name/number the images a certain way (Nobody else has this requirement, LAME)
- Realtor.com prefers I resize the images to certain size (Nobody else has this requirement, LAME!!!)
- Realtor.com prefers I only do incremental photo uploads (Nobody else has this requirement, are you kidding me!?!?)
- Complain to Dustin (WTF dude!)
- Explain to customers why I have to bill them for 8 hours of engineering time, instead of 2 hours (It sucks, I’m sorry, it’s not my fault)
- Blog about the whole rotten experience while I write my image zip / ftp uploading code.
When Zillow, MS Live Expo, Realtor.com 2.0 or the next great startup wants to get into the digital listing importing game, I have 3 words of advice, Embrace and extend. Or to put it in terms more people would understand, Copy Trulia.
About the Author: Robbie Paplin
- Author's website: http://www.caffeinatedsoftware.com/
- More posts by Robbie
- email the author
I’m glad someone is blogging about this. This industry is so painful on both ends. Client interaction and back-end interaction. Both are nightmares.
MLS’s could care less about complying to standards and making your life easier, and Realtors (love ‘em/hate ‘em) want everything done for them and don’t understand why they can’t display listings like any other content and all from one easy to use source. Not to mention change the look/feel.
Web design/Technology/IDX firms get the smelly end of the stick when it comes to this wonderful industry. No pats on the back, just complaints. Yay…
Jesse,
Well, MLS’s (or at least those whose members have to deal with multiple MLS) should care because not caring either raises the costs for their vendors or prevents their IT staff from doing other things. I’m optimistic that because of the efforts of MLS vendors and the NAR CTO, that RETS might someday become a widely deployed standard. However in this case, the MLS and the lack of RETS doesn’t really have anything to do with this particular problem. (it’s just another example of the real estate data problem – just from another angle)
I think the problem is that realtor.com has taken for granted the listings it’s historically gotten from the MLSs. And now the landscape is potentially changing from getting listings from 500+ MLSs (at no cost to member brokers), to getting listings from 50,000+ brokers in a competive environment (in which brokers now have to pay IDX vendors or IT staff to support them). Given this sea change, I would’ve expected a standard that was similar to what it’s competitors are doing.
I don’t mind much that Trulia, Oodle and Propsmart are slightly different from each other since the transfer mechanism is identical (XML data over http, image data linked to in the XML over http). Even GoogleBase has more in common with them than realtor.com does (XML data over ftp, image data linked to in the XML over http).
Just because realtor.com gets 4 times the traffic of Trulia, doesn’t mean it’s standard should be 4 times as difficult to implement! Even GoogleBase wasn’t this bad!
Robbie,
It is my understanding that when NWMLS provided the feed to Realtor.com, all listings went unless the seller did’t want it there for some reason. But now that it is a Broker by Broker proposition, only Brokers who are Members of the National Association of Realtors can have their listings posted on Realtor.com
Makes sense. Not complaining. Just an FYI.
I guess I don’t see the MLS’s need to comply to RET’s as there isn’t any competition available to force them to fight for their customers. Not all of us are lucky enough to have an awesome MLS provider, we’re instead stuck with ones that still deliver listings via an FTP server and a raw comma delimited .txt file without headers.
Our area doesn’t really have a great IDX either. We’re forced to frame in most of our IDX feeds which in turn makes our websites that much less integrated. Dealing with a templated website system and IDX feeds combined with MLS law is a mixture for pain. But I’m open for suggestions if anyone’s got any.
I’ve been managing RE sites for a couple of years now and, even for this industry, the extent to which Realtor.com sucks is stunning.
I built and managed retail ecommerce sites before I got into real estate web sites. It has been an uphill battle to implement even the most basic eMerchandising practices: Like having the thing you sell on your homepage for starters. How many templated RE Web sites (from the likes of Move inc) STILL have a Homepage with a Golden Retriever waving an American flag and not a listing to be seen ?
I suspect that the motives for ditching the NWMLS feed to Realtor.com were less than pure, but you are right, if this is the beginning of a trend, then Realtor.com is going to have its lunch eaten by Trulia and Google.
Ardell,
I agree it’s the broker’s decision to make, so it should be in the hands of the IDX vendor (or the broker’s IT staff), so they have control over the process. Although, I wonder if there are any brokers who aren’t NAR members that feel cheated by this turn of events. I don’t think they should, since they were able to get free promotion of their listings via an outlet they shouldn’t have had access to in the first place.
Jesse,
As I commented over on the Flex MLS blog, the problem is that a standard that isn’t widely deployed isn’t much better than not having standards at all (or having multiple standards). I think the most important thing the RETS community can do is “reduce barriers to deployment”. As an IDX vendor, I’m at loss at how I can effect this change (other than blog and raise awareness).
As for realtor.com, they now do have competition (or at least more than they used to), and it’ll be very interesting if the NWMLS action is the beginning of a trend or just an anomaly. If it’s a trend, realtor.com is in for a big fight since their key competitive advantage (nearly all listings) will be history.
As for the customizable IDX solutions, it’s an interesting problem. I currently deal with brokers exclusively, since that eliminates the political problems of MLS law and leaves only technical ones (thus rendering the whole framed IDX issue moot).
For Zearch, I currently use ASP.net master pages which made per broker customization staightforward. For Nimitiz, I plan on studying the ASP.net skin/theme architecture and study the skinning implementation of Subtext. This will hopefully allow me greater flexability, so I can eventually serve a bigger market in a more cost effective manner. But yeah, I don’t have any great ideas that will accomplish that goal without either requiring I expose my IP or require my clients become software engineers or both.
JohnnyRock,
Yeah realtor.com needs to change or become food for the upstarts. Either that or start selling a lot more TopProducer and grow their other business interests while the cash cow goes out to pasture. Either way, they need to MOOOO-ve.
Yeah our problem with the framed IDX is that since we deal in templates, we can never utilize the function of the IDX to upload a header/footer section of code to match the clients current site. As the client can change their site on a whim, so does the need to change the code that would wrap around the listings.
Thus we’re stuck in frames for listings. We also aren’t able to grab listings and put them on the front page. I’ve often thought about the legality of using server side script to scrape the listings pages HTML and use that to inject into our served page, but currently I’m sure that doesn’t comply with MLS rules among probably many other problems.
What’s worse is it’s the Agents and their Clients who end up suffering the most because we can’t deliver them a dynamite product. What’s holding us back? The Agents and their Brokers and the laws they’re lobbying NAR to fight for against the DOJ.
An interesting conundrum. Hopefully Agents will start to see that these tech companies are really only here for their best interest and that in order for us to serve those interests and (gasp) make money we need more liberal laws for listings.
Jesse,
I’ve always thought it strange that in the NWMLS, only brokers can get feeds from the MLS. In other parts of the country, agents & brokers can get feeds. I don’t know what agents can do to change things, but if they had the same rights as brokers to the listing data, it’d make all the problems technical (which software engineers are better at solving) instead of political/legal (which we usually can’t do much about).
It’s unfortunate, we’re still stuck using frames. It sucks on many levels. Perhaps by telling agents that they need to change the MLS download rules in their favor in order for us to be able to legally create the dynamite products they desire and deserve. The bigger question is why are things this way? The only reason I can think of is it’s a way for brokers to keep their agents in check. If so, shouldn’t that be a broker by broker decision, instead of requiring everybody to handcuff their agents?
My understanding was that it was tied to the MLS laws. Such that you can’t display an MLS’s listings on a non authorized domain. So in the case of frames, it’s on the IDX’s authorized domain. In order to display it on http://www.joe-smith-the-agent.com that agent would need to have authorization to display them.
Is that what you’re referring too when you’re talking about the broker? However, even when we do broker websites, we’re not allowed to put the listings on their domain. We still have to display them on the IDX’s domain and ultimately from their server. If they get a custom website (as apposed to a dynamic template) we can always use the IDX’s system to wrap our design html around the listings on their website. But ultimately the listings still have to be at http://www.the-idxs-url.com/listings rather than http://www.joe-smith-the-broker.com/listings.
Once there’s a break in the ability to put listings on a registered agents/brokers website, we can do some killer websites, and ultimately you’ll see the software engineers blossom. (I come from a CS background and so I’ve been itching to do some killer products with listing data and brokerage office rosters). But our hands are tied.
If every agent had a different feed, each might also have a different timeframe for sweeping the changes made to the MLS. If we have one download agreement that provides updates every 15 minutes, then we know every agent’s individual site is also going to sweep the changes every 15 minutes.
If an agent hires someone else to do their website with a different download agreement, that site might have a once a day overnight sweep of changes. Is that correct?
Also, MLS comes out with a new rule, like not allowed to say ‘Search the MLS’. Broker is assured that all sites are brought into compliance by the one person who has the download agreement for all agent sites via the broker. If each agent had a different service provider, then the broker would have less control over making sure all agent’s sites are in compliance with the “new rule” in a timely fashion.
Since the broker is ultimately responsible for agents being in compliance with the myriad of laws and rules as they change from time to time, they rely on the single or few service providers to assist them in keeping all sites in compliance.
Ardell,
It is currently up to the IDX vendor, how often they check for changes and I believe the MLS only sets guidelines (You must check at least once daily – although I don’t recall where I read them). Obviously checking more often is better in terms of accuracy, but that comes at cost of server responsiveness. But the download agreement has no mention of how often you should sync up changes.
I currently check for data changes hourly, check for new photos on the fly and check for updated photos nightly. I could check more often but that will bog down my server and bogs down the MLS servers and if I check less often, the site becomes less timely and less accurate. I know the MLS IT staff tends to get very upset if you check every 5 minutes, since that kills performance of their systems for everybody. So there is a delicate tradeoff between the MLS, the IDX vendors, and the brokers/agents they serve. Bandwith and servers aren’t free but information timliness is important. In my experience, local IDX vendors tend to be more timely with updates and national IDX vendors tend to update less frequently. At least in my limited experience…
As for the rules, you are correct in that if you had more people with access, there would be greater potential for rule violations. Although, I think a lot of the rules are unneeded. What’s so bad about saying “Search the MLS”? Is that a trademark violation?
Hey Robbie,
We currently sweep data every 20 min and grab all photo updates every 20 min as well. I agree that every 5 min is excessive but there is a lot less strain on their system and their bandwidth if the data is checked more frequently. I believe anything less then 20 minutes you would be missing data due to their replication process. I know in the morning when the once a day guys sweep really slows down the system.
As far as Search the MLS goes I think it would be misleading to the public to think they are seeing everything in the MLS. Maybe something like search all the publicly available properties in the MLS would be more appropriate. Only members and VOW sites have access to all the data.
Allen,
Does that mean if I have a VOW vs. an IDX I can show Active, STI, Pending and Sold as well as Cancelled and Expired?
Ardell,
Right now you can show all the solds without a VOW. You can show STI because those are still active. The whole concept of VOW is if you can print it out and hand it to your client you should be able to do it electronically. The key is you have to have a relationship with that client. You don’t have an agency relationship with someone stopping by and searching for listings on your website. Once the relationship is established you can provide them more information then they can recieve on a public IDX site.
Hi Allen,
My image download process is a simple single threaded process (I probably should improve it one of these days). I found that if I try to download changed images more frequently, I run into the “Painting the Golden Gate Bridge” problem in which images need updating more frequently than I can download them (which I why I have a nightly image sweep).
I wish more people realized that keeping about 400,000 images in sync with the NWMLS is a non trivial challenge.
Allen,
I understand there is a petition I’m being asked to sign to reverse the decision regarding Realtor.com.
What’s your feeling on the reasons for NWMLS withdrawing in the first place? Also should we join the fight to reverse that decision?
My concern is that as we speak, brokers are paying to develop individual methods of putting the properties on Realtor.com. If it turns out the decision is reversed, all of these brokers are spending money for nothing. So we better figure it out pretty fast.
I think the decision will be reversed. When is the question.
Ardell,
I will give you 2 different perspectives on the R.com situation.
1) My personal belief.
I think agents are doing a disservice to themselves by providing listings to R.com. R.com has been extorting money from agents since the first day the site went live. Even the founding president whom is facing 15 years in prison talked openly about how much revenue they could make off the captive audience. They said they would never sell leads Now they are selling leads. I could go on for hours. I have seen the statistics from around the country and the NW is the least searched area on R.com. The main reason for this is because not all the listings are on R.com and the public wants to see all the properties.
2) NWMLS Decision
I feel the NWMLS made the correct decision. I believe this should have been the original decision the NWMLS should have made. With the number of complaints they receive about incorrect or incomplete data on R.com and having to deal with Move it was probably a very easy decision. I understand that this will put additional work on us but that’s part of the idx data provider game. We send data all over the place so not a big deal.
Ardell,
My understanding is you are not a Realtor. Is that correct? If so your listings shouldn’t be on R.com. So not sure why you would sign a petition.
1) My belief – Given that R.com changed it’s model to one that makes it’s revenue via ads to agents, it thinks its fair game that agents treat R.com like just another site that sells ads to agents (just like they treat Trulia, Zillow, etc). Why should R.com get preferential treatment from Realtors, when they don’t do anything that Trulia or Zillow wouldn’t do better if they had the listings?
2) NWMLS decision – I think it was poorly handled. Originally, brokers and their vendors only had 30 days to react to the decision (which isn’t much time). I don’t have problems with the decision itself, mind you. As an IDX vendor, I have no problem providing the service, provided I’m compensated for the time it takes me to create it. After the code is written and paid for, it’s no big deal. The problem is what used to be a free service for brokers, is now going to be a service that requires some up front investment.
I agree, they better make up their mind before the software engineers start pouring concrete.
Allen,
Whether or not you were a Realtor was irrelevant prior to “the change”. All NWMLS listings went to R.com. So this “change” can be seen as a way NWMLS larger brokers are trying to boycott Brokers who choose not to be Members of NAR, WAR, etc…
As to me, makes no difference. I can be a Realtor tomorrow. No magic in “being a Realtor”, just the price of admission.
Thanks Ardell,
You just made my point. Which is there is nothing special about R.com. If it really was a realtor product they would never allow non members listings on the site. But because it is managed by a for profit company they just want warm paying bodies.
NWMLS doesnt send data anywhere else so why R.com.
You have to love the brutal truth that my hair stylist at Supercuts has more hours sunk into their education to cut my hair then my Realtor does to make the largest purchase of my life.
Can’t you say the same about the Realtor Organization as a whole? Warm paying bodies, most forced to be members in order to access the MLS in most of the Country?
What does NWMLS save by NOT having the homes in R.com? What’s the reason for pulling out?
There is a cost to sending the data.
There is a cost to dealing with the complaints.
There is a cost to working with Move.
It only costs us $30 a month to be members of NWMLS. I’m sure agents would be willing to pay the cost of shipping data to Realtor.com through NWMLS. It certainly is a lot cheaper than every Broker having to hire a Robbie. and I doubt the cost was the issue that created the change.
You can sign an agreement with the enterprise group and its FREE. Not sure what the problem is?
http://feeds.theenterprise.com/
Well, one problem is that a broker only has the rights to 2 IDX feeds. And by signing an agreement with the enterprise group (a/k/a Move), you have to give one of those IDX feeds up.
So, if your company already has arrangements with 2 IDX vendors (lets say one for the broker web site and one for agent web sites), you can’t sign up with them unless your willing to sacrifice your relationship and/or investment in the other 2 vendors.
Robbie,
I was told yesterday by a fairly reliable source that you may sign with theenterprise and it wouldn’t count against your 2 feeds. I have not personally confirmed this. If this is not true then the IDX providers like us will have an additional offering for our customers. We will not be charging for this additional service
.
Why would Jesse Bilsten, in comment #23 above, post an off-topic and insulting comment about Realtors when apparently his company, http://www.vreo.com, is trying to sell software to this very same scorned market?
Jesse, do you think Realtors are so stupid, that they can’t read?
Do I think Realtors are too stupid to read? No… why would I think that? I was providing an example in response to the post regarding who can sign the petition. It’s pretty silly to assume that you have to be a Realtor to weight in considering how easy it is to become one. If we let Realtor’s help us make the decision of buying a home (our biggest investment) why wouldn’t we allow clients to sign the petition?
I’m sorry if I rubbed anyone the wrong way, but this industry desperately needs to have some more verification for the hard working and deserving Realtors. One of the main problems with our industry is that anyone can become a Realtor in about a weekend. It creates more competition and brings down the likelihood someone will find an experienced Realtor rather than a inexperienced one thus giving good hard working Realtors a bad name.
I’m surprised more Realtors don’t want to change this. Rather than taking it as an insult why not petition for a higher education requirement? Make the field more valuable. Instead of constantly defending the broken process, agree that it’s messed up and work to fix it. Seems fairly logical to me… which is why I try and make as many people aware of it as possible.
[...] Where should the MLS end and the IDX begin? May 10, 2007 The whole ruckus over the NWMLS no longer sending its member’s listings to realtor.com inspired many unlit pixels of commentary and many more wasted bytes of hard drive space. As I pondered a while ago, the industry appears to have a healthy appetite for technology. However, one of the comments was really insightful…. I still feel that this decision made by the board was wrong. As was the decision last year to disable the client email updates from Locator. We have the technology but are unwilling to use it. I have no love of REALTOR.com but I see no problem with sharing a limited set of data with them and offering our sellers maximum exposure of their listing. In fact, perhaps one of the reasons they discontinued the feed was because as Galen said, “Realtor.com was given the exclusive non-broker feed…” and they were getting pressure from Google and others to get a similar feed. I say give it to them. NWMLS has the ability to provide its members, all of them, with the technology usually reserved only for those with very deep pockets. [...]
[...] (If I’m refraining from too many opinions on this article, it is because my biases are so numerous I should probably not hit publish… (1) In the past I was “asked” by the NWMLS to take down RCG’s home search because we were not an “agent” site, but rather a marketing vehicle for multiple real estate professionals (It now sites at “http://www.annaluther.com” in order to comply). (2) My employer is having their own set of issues with the NWMLS. (3) Yesterday’s article and comments were probably not well received by the staff at Redfin…) [...]
Basically, what you don’t understand is that realtor.com has not created a national agregation system here. This was set up just for NWMLS brokers and the target audience is not just IDX vendors, but much less sophisticated user in brokers’ office’s who have no idea what XML is, but can certainly export a file from excel and zip up and FTP some photos. It is very much a purposely dumbed-down system.
Again showing your ignorance, you should also know that the only reason NWMLS no longer sends data to realtor.com is due to one larger broker who basically had a lot of sway. There is no national movement by MLSs to leave realtor.com. In a market like this would member’s want to reduce their listings’ exposure — especially to the number one site by far for real estate? Do forget, realtor.com is NAR (the industry)’s official site.
Have you even looked at the NWMLS data feed spec? It says “We will only accept feeds from an automated solution, and reserve the right to suspend feeds that are not being properly maintained.” The target audience is clearly IDX vendors that service NWMLS brokers, the IT staff of NWMLS brokers (or somebody else of similar technical sophistication). Given the intended target audience, why create a standard that’s more costly/difficult to implement?
Even if they did accept a manual solution, I can’t imagine a broker would have a member his or her staff update a listing spreadsheet in Excel, export it as TXT, get copies of all updated listing photos, create a zip file of listing photos, and upload the files via FTP every day! Unless NWMLS Locator has a feature to create a Realtor.com feed files that I don’t know about, it would probably be labor intensive for a broker to keep the thing updated (get quickly expensive).
I suspect what happened is that Realtor.com just documented and re-purposed the feed process that they had in place with the NWMLS since the late-1990s, instead of spending the time to create a superior XML-HTTP based solution. It’s less work for them, but more work for NWMLS members.
Regardless of the reasons why the NWMLS is no longer sending data to Realtor.com, killing the program wasn’t something that was appreciated by the majority of members (not voting board members mind you). The NWMLS stopping the feed, means if you want exposure on Realtor.com, it’s now your responsibility to figure out how to get your listings there (instead of the NWMLS’s).
Although, there is no national movement to leave Realtor.com, there is a growing movement to support Realtor.com competitors (Trulia, Zillow, etc). If Realtor.com starts losing it’s competitive edge (site traffic), it’ll be very interesting to see if a new pecking order establishes itself.
As a the tech rep for a number of Real estate brokers, their mls’s, and of course the Realtors… From my angle the entire Real estate system is somewhat neurotic, schizophrenic, and overall amusing. One mls provides pipe delimited CSV, another appears to have invented its CSV style entirely from scratch, and then there is just weird stuff… like mls’s not telling me what a given field means or doing absolutely no validation on what the Realtors submit ( Straet|ST.|str. or Cities called “Todo”). Further more adding to the fun is how unbelievably hostile some MLS vendors are to their clients (0100 – 0600 ftp time slots, no excuses) or vice versa.
[...] What’s this mean for Realtor.com? I dunno, but it’s increasingly looking like they are going to get HouseValued (yes, I just made that verb up), if they don’t show some brain activity. [...]
IT SEEMS THAT THE NWMLS IS BACK ON REALTOR.COM, NO? MAYBE CONSUMER OPINION COUNTS, MORE?
I’ve been boycotting realtor.com for about 5 years now. To think our own trade group website, given the importance of the internet to real estate, would operate the way it does i.e., blackmail agents to pay extra to have our listings contain OUR contact info, when any other IDX, zillow, yahoo, trulia, etc provides that info without charge, while selling and displaying banner advertising along with our listing content, is despicable! Realtor.com sucks.