If you bought a home after 2009, your appraisal fee almost certainly included hundreds of dollars for a middleman you've likely never heard of.