Outspoken Charted Accountant and Toronto real estate agent Scott Ingram has been taking note of the performance of online home price estimators for four years.
On Wednesday, he released a report with the heading: “Canadian online home price estimators kind of suck.”
And it may make you think twice about the accuracy of online home price estimates.
Ingram highlights the growing prevalence of iBuying in real estate, a process that enables consumers to sell or buy a new home in as soon as three days with all cash and choose their closing date. He then questions the effectiveness of online estimators on sites like Zillow, HouseSigma, and Opendoor.
Inspired by Zillow’s recent announcement that it would move away from the iBuying space -- the company claimed it “can’t sufficiently trust its pricing model” -- Ingram felt compelled to take a closer look at the good, bad, and ugly of home price indicators.
“Curiosity inspired me. I'm a curious guy,” Ingram tells STOREYS of his desire to dig deep into the topic. “I actually started tracking HouseSigma estimates back in 2018. A lot of people make predictions (in many fields) but I find there's a lack of accountability. So I wanted to see just how well they did. Also, several of my clients have been HouseSigma users, so I wanted to be able to give them some insights, in case they were taking those estimates seriously.”
READ: 2022: Housing Sales to Ease, But Prices Expected to Rise Even Higher
Ingram says the estimates have been off for “years,” long before the pandemic inspired the current dramatic and unpredictable market (though he points out there was a 5-month stable price period this year).
Ingram examines the error rate for popular US site Zillow when it comes to assessing home values. He highlights how back in 2006 Zillow's error rate was 14%, but says the company had improved their model over the years to get the error rate down to just 2%.
But Ingram directs most of his attention to local companies HouseSigma, Zolo, and Zoocasa. This year, starting in January and going until the end of November, he tracked estimates on 95 sold properties. HouseSigma had estimates on 93 of them, Zolo on all 95, and Zoocasa on 89.
Ingram found that all three come up too low in their estimates more often than not -- particularly, HouseSigma and Zolo (about five and a half times as often, and four times as often, respectively).
“From this you'd guess that Zoocasa looks best,” writes Ingram in the report. “I'd think that the best-constructed estimator be about half the time too high and half the time too low, but always right around zero error.”
However, through his analysis, Ingram says HouseSigma is actually the most reliable.
“Although Zoocasa looked best in terms of being under on average by 1.4% (with HS at 9.1% and Zolo at 13.5%), if I look at the absolute values of how far the estimate was off, then take the median amount (akin to Zillow's "error margin"), the winner is HS at 11.7%, Zoocasa jumps to 13.4% away, and Zolo is 20.2% away,” he writes in the report.
Ingram found that HouseSigma was best when examined from an absolute value perspective, “or a sort of ‘closest to the pin,’ if you will.’”
He points to the limitations of HouseSigma, however. “I've already pointed out that it's low 85% of the time,” writes Ingram. “I should point out that it's only for Ontario properties (Zolo and Zoocasa both claim to be national in scope).”
Ingram tells STOREYS that he actually likes HouseSigma and has it on his phone. “It's way easier to look stuff up remotely there than trying to log into TRREB's clunky and mobile-unfriendly MLS,” says Ingram. “And it had the listing's history way before TRREB got around to adding an ‘address history’ button on listings, so I used to use it for that too. I just don't put too much stock in their estimates.”
As for Zillow, Ingram says he was a little surprised by their announcement.
“I mean, their estimates were seen as really good,” says Ingram. “Plus they were buying these properties in a rising market, in which flippers can make money (and think they're smarter than they are) even if they don't really add much value. So, you figure that should have made it harder to mess up.”
Ingram says Zillow’s failure is a good example of the value a real person or realtor can have, compared to a machine that can’t detect subtleties.
“For instance, I notice these estimates are worse at the extremes, like a total gut job, or a brand new gorgeous renovation,” says Ingram. “Take the same house before and after a flip and say it's got the same number of bedrooms and bathrooms and is in the same location (just the time has shifted). The computer is going to think of that place the same. Meanwhile the realtor human (and buyer humans) are going to immediately see those stark differences and value them different.”
